Conversation
This should be possible, but the access checks are tricky.
…rom nested queries.
I'm not certain it's the best way of going about it but couldn't we implement this by storing the entity of the parent instead of the data and then resolving the data from the entity when it is needed? |
I'm not sure what you mean. Like, instead of yielding a let mut items = query.iter_mut().collect::<Vec<_>>();
let mapped = items.iter_mut().map(|item| item.get_mut()).collect::<Vec<_>>(); |
|
I think these changes are a great idea; I guess I would like to know how this contrasts to how flecs queries for relation data. Maybe @james-j-obrien knows? |
That's a quite involved question to answer (although a very interesting one). The big difference between the bevy and flecs queries are tied to one being defined in the type system and the other being defined dynamically. Due to this bevy queries are fundamentally based on nesting, you have tuples of query terms that each store their own state and generate their own code for managing that state. In flecs all the query terms are just stored in a flat array. For example in this PR we express querying our parent as creating a query term that traverses the relationship and then nested in that is the set of components we want to access on the target, whereas in flecs you would have a set of instructions that said: "get me any entity A with relationship of the form (ChildOf, B) and store B as a variable", "get me component Y on entity B", "get me component Z on entity B". This structure allows flecs to optimize/batch/reorder terms since they can be evaluated in the full context of the rest of the query, but for simple queries it's mostly a different path to the same goal. Since flecs also has fragmenting relations they can do stuff like cache the tables for B since you know that entities in A's table will always have parent B. All that being said, with bevy's queries as they exist today this PR seems like the shortest path to querying on multiple entities so seems like a positive step. |
…` and fix missing documentation.
…:init_nested_access`.
Thanks for the answer! I guess we could do something similar: for tuple queries, build such a node graph and use it to match entities. I guess we do already create a data structure that helps us find matching entities; that data structure is the QueryState.
And the main difference is that the flecs "QueryState" is more elaborate since it can contain sources, relationships, etc. |
|
@eugineerd, I liked your work over in #21601; can I get your review here in turn? |
…e-access # Conflicts: # crates/bevy_ecs/src/world/unsafe_world_cell.rs
This reverts commit 0660ceb. Resources are entities now!
Trashtalk217
left a comment
There was a problem hiding this comment.
The query system is starting to feel like trait soup and SingleEntityQueryData is not a great name (one word to many and Single puts me on the wrong foot). Still, I approve this because it enables some really usefull possibilities, including fixing AssetChanged (#23057). I also can't think of a better name than SingleEntityQueryData...
Really well done @chescock.
alice-i-cecile
left a comment
There was a problem hiding this comment.
I've done a careful review of this now, and think it's both well-architected and carefully constructed.
I was afraid this would be dramatically more complex, but the new traits in query/fetch.rs really are the heart of it.
Merging; I'm excited to see the follow-up work!
# Objective Follow-up to #21557 Support streaming iteration (a.k.a. lending iteration) for `QueryIter` and `QuerySortedIter` so that it's possible to iterate `Query<Parent<&mut T>>`. Double-check that all the `IterQueryData` bounds are correct and that the safety comments are correct. A few were not! ## Solution Offer a `fetch_next` method on `QueryIter` and `QuerySortedIter` that borrows the entire iterator to prevent aliasing, similar to `QueryManyIter::QueryManyIter`. Offer a `Query::iter_inner` method to produce `QueryIter` even when it is not an `Iterator`. We cannot rely on `IntoIterator` in that case, and Clippy objects to an inherent method called `into_iter()` that is not the standard trait method. This supersedes the existing `iter_inner` method that only worked on `ReadOnlyQueryData`. Add a missing `IterQueryData` bound to `iter_combinations_mut`. It yields multiple entities concurrently, so is never sound to use with non-`IterQueryData`. I think the reason I missed this originally is that it already has a `fetch_next` method and conditional `Iterator` impl. But even `fetch_next` yields multiple entities at once for `QueryCombinationIter`! Add a `ContiguousQueryData: IterQueryData` supertrait bound. `QueryContiguousIter` also yields multiple entities concurrently, so it does not make sense on non-iterable data. This was not actually unsound, though, as `QueryContiguousIter` does not call `fetch`. Finally, update some missing or incorrect safety comments. ### Verify bounds on the other iterator types To verify I didn't miss any bounds on iterator types, here is a list of every `Query*Iter` type, and whether they support non-`IterQueryData`: | Type | Supports non-`Iter`? | | --- | --- | |`CombinationIter`|No, added missing bound| |`ContiguousIter`|No, added missing bound| |`Iter`|Yes, added support| |`ManyIter`|Yes, existing support| |`ManyUniqueIter`|No, and existing bound*| |`ParIter`|No, and existing bound| |`ParManyIter`|No, and existing bound| |`ParManyUniqueIter`|No, and existing bound| |`SortedIter`|Yes, added support| |`SortedManyIter`|Yes, existing support| `Iter` and `SortedIter` were changed in this PR to support streaming iteration for all `QueryData`, while only implementing `Iterator` for `IterQueryData`. `ManyIter` and `SortedManyIter` already had streaming iteration, and only implement `Iterator` with a stricter `ReadOnlyQueryData` bound, since they may yield the same entity multiple times. `ManyUniqueIter` could theoretically be used with non-`IterQueryData`... but if you need to do streaming iteration anyway then you can call `iter_many_mut()` instead of `iter_many_unique_mut()`. `CombinationIter`, `ContiguousIter`, and the `Par*Iter` types are all fundamentally about accessing multiple entities concurrently, and should always require `IterQueryData`. `CombinationIter` and `ContiguousIter` did not have `IterQueryData` bounds, so fix that! ## Showcase This is cheating a bit because we don't yet have `Parent<&mut T>`, but once we do it would look like: ```rust fn system(query: Query<Parent<&mut Component>>) { let mut iter = query.iter_mut(); while let Some(mut parent_component) = iter.fetch_next() { // Use `parent_component` here } } ```
* 0.19.0-dev * with patches for running local examples * replace became discard * reflection reorg changes * glam 32 * nested queries have landed: bevyengine/bevy#21557
…a` impl (#23686) # Objective Demonstrate how to use `NestedQuery` to implement relational queries. There is more design work required before making first-party relational queries, but third-party implementations are already possible using `NestedQuery`. The documentation does not make it clear how to actually implement them, though, and the only real example is buried in the PR description of #21557. ## Solution Expand the examples for `NestedQuery` to include the implementation of a `Parent<D>` relational query. Keep one example that uses `#[derive(QueryData)]` with a method on the generated `Item` type, but also add an example with a manual `QueryData` impl that delegates everything. --------- Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com> Co-authored-by: Kevin Chen <chen.kevin.f@gmail.com>
…a` impl (bevyengine#23686) # Objective Demonstrate how to use `NestedQuery` to implement relational queries. There is more design work required before making first-party relational queries, but third-party implementations are already possible using `NestedQuery`. The documentation does not make it clear how to actually implement them, though, and the only real example is buried in the PR description of bevyengine#21557. ## Solution Expand the examples for `NestedQuery` to include the implementation of a `Parent<D>` relational query. Keep one example that uses `#[derive(QueryData)]` with a method on the generated `Item` type, but also add an example with a manual `QueryData` impl that delegates everything. --------- Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com> Co-authored-by: Kevin Chen <chen.kevin.f@gmail.com>
Objective
Support queries that soundly access multiple entities.
This can be used to create queries that follow relations, as in #17647.
This can also be used to create queries that perform resource access. This has been supported since #16843, although that approach may become unsound if we do resources-as-components #19731, such as #21346.
Fixes #20315
Solution
Allow a
QueryDatathat wants to access other entities to store aQueryState<D, F>in itsWorldQuery::State, so that it can create a nestedQuery<D, F>during the outerfetch.NestedQuerytypeIntroduce a
NestedQuerytype that implementsQueryDataby yielding aQuery. It is intended to be used inside other implementations ofQueryData, either for manual implementations or#[derive(QueryData)]. It is not normally useful to query directly, since it's equivalent to adding anotherQueryparameter to a system.In theory, we could directly
impl QueryData for Query, but this would be too easy to do accidentally. Having to explicitly import and writeNestedQuerywill make it clear that it's something unusual, and also allows us to remove the need for passing'staticfor the'wand'slifetimes.New
WorldQuerymethodsFor it to be sound to create the
Queryduringfetch, we need to register theFilteredAccessof the nested query and check for conflicts with other parameters. Create aWorldQuery::update_external_component_accessmethod for that purpose. ForQuery as SystemParam, call this duringinit_accessso the access can be combined with the rest of the system access. For looseQueryStates, call it duringQueryState::new.In order to keep the query cache up-to-date, create a
WorldQuery::update_archetypesmethod where it can callQueryState::update_archetypes_unsafe_world_cell, and call it from there.New
QueryDatasubtraitsSome operations would not be sound with nested queries! In particular, we want a
Parent<D>query that reads data from the parent entity by following theChildOfrelation. But many entities may share a parent, so it's not sound to iterate aQuery<Parent<&mut C>>.It is sound to
get_mut, though, so we want the query type to exist, just not be iterable. And following the relation in the other direction for aQuery<Children<&mut C>>is sound to iterate, since children are unique to a given parent.So, introduce two new
QueryDatasubtraits:IterQueryData- For anything it's sound to iterate. This is used to bounditer_mutand related methods.SingleEntityQueryData- For anything that only accesses data from one entity. This is used to boundEntityRef::get_components(see Unsound to callEntityRef::get_componentswith aQueryDatathat performs resource access #20315). It's also used to boundtransmuteat the moment, although we may be able to relax that later (see below, under Future Work).Note that
SingleEntityQueryData: IterQueryData, since single-entity queries never alias data across entities, andReadOnlyQueryData: IterQueryData, since it's always sound to alias read-only data.Here is a summary of the traits implemented by some representative
QueryData:&T&mut TParent<&T>Parent<&mut T>(&mut T, Parent<&U>)Children<&mut T>Alternatives
We could avoid the need for the
IterQueryDatatrait by making it a requirement for allQueryData. That would reduce the number of traits required, at the cost of making it impossible to supportQuery<Parent<&mut C>>.Showcase
Here is an implementation of a
Related<R, D, F>query using this PR:Details
I'd like to leave that to a follow-up PR to allow bikeshedding the API, and to take advantage of #21581 to remove the
Option, but I think it works!Future Work
There is more to do here, but this PR is already pretty big. Future work includes:
WorldQuerytypes for working with relationships #17647AssetChangedto use nested queries for resource access, and stop tracking resource access separately inAccessget_stateforNestedQuery. This is difficult because constructing aQueryStaterequires reading theDefaultQueryFiltersresource, butget_statecan be called fromtransmutewith no access.SingleEntityQueryDatabound on transmutes and joins. This will require checking that the nested query access is also a subset of the original access. Although unless we also solve the problem of implementingget_state, transmuting to a query with nested queries won't work anyway.QueryIterby offering afn fetch_next(&self) -> D::Item<'_>method and relaxing theIterQueryDatabound onQuery::into_iterandQuery::iter_mut. This would work similar toiter_many_mutanditer_many_inner.IterQueryDatabound onQuery::single_inner,Query::single_mut, andSingle<D, F>. This seems like it should be straightforward, because the method only returns a single item. But the way it checks that there is only one item is by fetching the second one!