Query: link() and backlink() over to-one relations, any depth

Author: greenrobot

AGENTS.md

@@ -32,7 +32,7 @@ Python binding for the ObjectBox C API (objectbox-c) via `ctypes`; objects are s
   - `codegen.py` — experimental, the alternative to `fast_builder.py` (only one is active: `experimental._flatbuffers_mode`): `CodegenBuilder` generates and `exec`s a marshal function per entity that does what `FastBuilder.marshal` does, unrolled; it uses a `FastBuilder`'s layouts. Changes to `FastBuilder.marshal` must be mirrored in the generated code; the same tests check both. It also generates an unmarshal function, used by `_Entity._unmarshal` in codegen mode, that must return the same values as `_unmarshal_table` (flatbuffers.Table, the reference), so changes to reading go into both. Each function is compiled at its first use; with `cache_dir` (`enable_flatbuffers_codegen`), via a .py file in that directory (named by the source hash) and Python's bytecode cache.
   - `idsync.py` — assigns IDs/UIDs by syncing the model against an `objectbox-model.json` file (read, matched by UID then name, rewritten from scratch). This is how renames/removals are tracked across schema changes. Matching by name uses `_db_name`, falling back to the other form of the name (e.g. the Python name stored before 5.0), which renames the property and keeps its data.
 - **`Store`** (`store.py`) — opening a store: resolve the model (a `Model` or a model name string → collect registered entities), locate `objectbox-model.json` (explicit `model_json_file`, else the calling module's directory found via the call stack, else `__main__`'s directory, else CWD), run `sync_model` (only once per `Model` and JSON file in a process: `Model._synced_json_file`; models for names are cached in `_models_by_name`), then build `StoreOptions` and call `obx_store_open`. `skip_model_sync=True` opens an existing store without passing the model (schema from the database). Directory prefix `memory:` opens an in-memory DB.
-- **`Box`** (`box.py`) — CRUD per entity. `box.query(condition)` returns a `QueryBuilder`; conditions (`condition.py`, combinable with `&`/`|`) are applied onto the C query builder, then `.build()` yields a `Query`. `Query.set_parameter(s)()` choose the C function by the property type (`_parameter_functions` in `query.py`); for `set_parameter(s)_alias()`, `QueryBuilder` records the property of each condition alias (`_alias_for_property`).
+- **`Box`** (`box.py`) — CRUD per entity. `box.query(condition)` returns a `QueryBuilder`; conditions (`condition.py`, combinable with `&`/`|`) are applied onto the C query builder, then `.build()` yields a `Query`. `Query.set_parameter(s)()` choose the C function by the property type (`_parameter_functions` in `query.py`); for `set_parameter(s)_alias()`, `QueryBuilder` records the entity and property of each condition alias (`_alias_for_property`). `QueryBuilder.link()`/`backlink()` return a builder of the linked entity (`_linked()`: shares the C query builder's lifetime and the aliases; `build()`, `order()` and `close()` raise on it); parameters of linked conditions use the property's entity (`prop._entity`, set by `@Entity()`).
 - **Sync** (`sync.py`) — `SyncClient` (created only via `Sync.client()`; its constructor raises) wraps the C sync client (`obx_sync_*`; ctypes callbacks passed to C, also replaced ones, are kept referenced until the client is closed). The store keeps its open client alive (`Store.sync_client()`, like Java and Dart) and closes it (close listener) before the store is closed. `@SyncEntity()` sets the entity's Sync flags.
 - **Deprecated API**: `ObjectBox` and `Builder` are thin wrappers around `Store` kept for backward compatibility (`tests/test_deprecated.py`).
 - numpy and flatbuffers (which imports numpy) are imported lazily, only where needed (vector/flex properties, the reference FlatBuffers paths); no module-level imports of either (`tests/test_lazy_imports.py`). Use `is_ndarray()` (`c.py`) for type checks and `prop._fb_format` (struct format) instead of `prop._fb_type` outside the reference paths.

CHANGELOG.md

@@ -65,6 +65,12 @@ As this is a beta version, add `--pre` to install it, e.g. `pip install --pre ob
 * Order results via `QueryBuilder.order(property, flags)`, e.g. `box.query().order(Person.name).build()`;
   `OrderFlags` (exported by the top-level module) adjust the order, e.g. `DESCENDING` or `CASE_SENSITIVE`;
   `order_desc(property)` orders descending
+* Link queries over to-one relations (`Relation` properties, see below), like in Java and Dart:
+  `QueryBuilder.link(relation, condition)` matches objects whose related object matches the condition, e.g. orders of
+  customers named "Ann" via `qb = box.query(); qb.link(Order.customer_id, Customer.name.equals("Ann"))`;
+  `backlink(relation, condition)` uses a relation in reverse, e.g. customers with a matching order. Both return the
+  builder of the linked entity to link further (any number of relations); `set_parameter()` also changes conditions
+  on linked entities.
 * New "one of" conditions `String.one_of(values)` and, for integer properties, `one_of(values)` and
   `not_one_of(values)`; their values can be changed on a built query via `set_parameter()` (see below)
 * New conditions `is_null()` and `is_not_null()` (e.g. `Task.text.is_null()`), also as `QueryBuilder.is_null()`

objectbox/c.py

@@ -956,6 +956,14 @@ def c_array_pointer(py_list: Union[List[Any], np.ndarray], c_type):
 # OBX_C_API obx_qb_cond obx_qb_any(OBX_query_builder* builder, const obx_qb_cond conditions[], size_t count);
 obx_qb_any = c_fn('obx_qb_any', obx_qb_cond, [OBX_query_builder_p, obx_qb_cond_p, ctypes.c_size_t])
 
+# OBX_C_API OBX_query_builder* obx_qb_link_property(OBX_query_builder* builder, obx_schema_id property_id);
+obx_qb_link_property = c_fn('obx_qb_link_property', OBX_query_builder_p, [OBX_query_builder_p, obx_schema_id])
+
+# OBX_C_API OBX_query_builder* obx_qb_backlink_property(OBX_query_builder* builder, obx_schema_id source_entity_id,
+#                                                       obx_schema_id source_property_id);
+obx_qb_backlink_property = c_fn('obx_qb_backlink_property', OBX_query_builder_p,
+                                [OBX_query_builder_p, obx_schema_id, obx_schema_id])
+
 # OBX_C_API obx_err obx_qb_param_alias(OBX_query_builder* builder, const char* alias);
 obx_qb_param_alias = c_fn_rc('obx_qb_param_alias', [OBX_query_builder_p, ctypes.c_char_p])
 

objectbox/model/entity.py

@@ -103,6 +103,7 @@ def _fill_properties(self):
         props_by_db_name = {}
         for prop_name, prop in variables.items():
             prop.name = prop_name
+            prop._entity = self
             prop._db_name = prop_name if self._python_names else camel_case(prop_name)
             other = props_by_db_name.get(prop._db_name)
             if other is not None:

objectbox/model/properties.py

@@ -250,6 +250,7 @@ def __init__(self, pytype: Type, uid: int = 0, **kwargs):
         self.iduid = IdUid(0, uid)
         self.name = ""  # Python attribute name; set in Entity.fill_properties()
         self._db_name = ""  # Name in the database and model JSON (camelCase by default); set in Entity.fill_properties()
+        self._entity = None  # The _Entity this property belongs to; set in Entity.fill_properties()
         self.index = kwargs.get('index', None)
 
         self._py_type = pytype

objectbox/query.py

@@ -18,7 +18,7 @@
 
 from objectbox.c import *
 from objectbox.exceptions import IllegalArgumentError
-from objectbox.model.properties import PropertyType
+from objectbox.model.properties import Property, PropertyType
 from objectbox.utils import date_value_to_int
 
 if TYPE_CHECKING:
@@ -31,13 +31,13 @@ class Query:
     It is created by calling ``Box.query()`` with conditions inside and finalizing with ``build()``.
     Note: Query objects are not thread-safe and should not be shared between threads.
     """
-    def __init__(self, c_query, box: 'Box', alias_property_ids: Optional[Dict[str, int]] = None):
+    def __init__(self, c_query, box: 'Box', alias_properties: Optional[Dict[str, Tuple['_Entity', int]]] = None):
         """ This is an internal constructor. Use ``Box.query()`` instead. """
         self._c_query = c_query
         self._box = box
         self._entity = self._box._entity
         self._store = box._store
-        self._alias_property_ids = alias_property_ids or {}
+        self._alias_properties = alias_properties or {}  # Alias -> (entity, property ID) of the condition
 
     def find(self) -> list:
         """ Finds a list of objects matching query. """
@@ -198,9 +198,9 @@ def set_parameter(self, prop, value) -> 'Query':
         :raises TypeError:
             If the value does not fit the property type.
         """
-        prop_id = self._entity._get_property_id(prop)
-        kind, value = _parameter_value(self._property_type(prop_id), value)
-        return self._set_parameter(prop_id, None, kind, value)
+        entity, prop_id = self._entity_and_property_id(prop)
+        kind, value = _parameter_value(self._property_type(entity, prop_id), value)
+        return self._set_parameter(entity, prop_id, None, kind, value)
 
     @overload
     def set_parameter_alias(self, alias: str, value: Union[str, int, float, bytes, datetime]) -> 'Query': ...
@@ -226,7 +226,7 @@ def set_parameter_alias(self, alias: str, value) -> 'Query':
             # 0 for non-scalar types (strings, vectors), so it tells integer lists from vectors
             type_size = obx_query_param_alias_get_type_size(self._c_query, c_str(alias))
         kind, value = _parameter_value(prop_type, value, type_size)
-        return self._set_parameter(None, alias, kind, value)
+        return self._set_parameter(None, None, alias, kind, value)
 
     def set_parameters(self, prop: Union[int, str, 'Property'], value_a: Union[int, float, datetime],
                        value_b: Union[int, float, datetime]) -> 'Query':
@@ -240,46 +240,54 @@ def set_parameters(self, prop: Union[int, str, 'Property'], value_a: Union[int,
         :raises TypeError:
             If the values do not fit the property type.
         """
-        prop_id = self._entity._get_property_id(prop)
-        kind, value_a, value_b = _parameter_values(self._property_type(prop_id), value_a, value_b)
-        return self._set_parameter(prop_id, None, kind, value_a, value_b)
+        entity, prop_id = self._entity_and_property_id(prop)
+        kind, value_a, value_b = _parameter_values(self._property_type(entity, prop_id), value_a, value_b)
+        return self._set_parameter(entity, prop_id, None, kind, value_a, value_b)
 
     def set_parameters_alias(self, alias: str, value_a: Union[int, float, datetime],
                              value_b: Union[int, float, datetime]) -> 'Query':
         """ Like :meth:`set_parameters`, for the condition having the given alias
         (see ``PropertyQueryCondition.alias()``). """
         kind, value_a, value_b = _parameter_values(self._alias_property_type(alias), value_a, value_b)
-        return self._set_parameter(None, alias, kind, value_a, value_b)
+        return self._set_parameter(None, None, alias, kind, value_a, value_b)
 
     def describe_parameters(self) -> str:
         """ Describes the query conditions with their current parameter values, e.g. for logging or debugging. """
         return obx_query_describe_params(self._c_query).decode('utf-8')
 
-    def _set_parameter(self, prop_id: Optional[int], alias: Optional[str], kind: str, *values) -> 'Query':
+    def _set_parameter(self, entity, prop_id: Optional[int], alias: Optional[str], kind: str, *values) -> 'Query':
         """ Calls the C function for the kind of parameter (see _parameter_functions) for the property or alias. """
         if kind == "ints":  # The C API has separate functions for 32-bit and 64-bit integer properties
             if alias is None:
-                type_size = obx_query_param_get_type_size(self._c_query, self._entity._id, prop_id)
+                type_size = obx_query_param_get_type_size(self._c_query, entity._id, prop_id)
             else:
                 type_size = obx_query_param_alias_get_type_size(self._c_query, c_str(alias))
             kind = "int32s" if type_size == 4 else "int64s"
         by_property, by_alias, to_c_args = _parameter_functions[kind]
         if alias is None:
-            by_property(self._c_query, self._entity._id, prop_id, *to_c_args(*values))
+            by_property(self._c_query, entity._id, prop_id, *to_c_args(*values))
         else:
             by_alias(self._c_query, c_str(alias), *to_c_args(*values))
         return self
 
-    def _property_type(self, prop_id: int) -> int:
-        for prop in self._entity._properties:
+    def _entity_and_property_id(self, prop) -> Tuple['_Entity', int]:
+        """ The entity is the query's, or another one's for its Property, e.g. of a linked entity (see
+        QueryBuilder.link()). """
+        if isinstance(prop, Property) and prop._entity is not None:
+            return prop._entity, prop.id
+        return self._entity, self._entity._get_property_id(prop)
+
+    @staticmethod
+    def _property_type(entity, prop_id: int) -> int:
+        for prop in entity._properties:
             if prop.id == prop_id:
                 return prop._ob_type
-        raise IllegalArgumentError(f"Property ID {prop_id} not found in entity {self._entity._name}")
+        raise IllegalArgumentError(f"Property ID {prop_id} not found in entity {entity._name}")
 
     def _alias_property_type(self, alias: str) -> Optional[int]:
         """ None if the property is unknown, i.e. the alias was set via QueryBuilder.alias(). """
-        prop_id = self._alias_property_ids.get(alias)
-        return self._property_type(prop_id) if prop_id is not None else None
+        entity_and_prop_id = self._alias_properties.get(alias)
+        return self._property_type(*entity_and_prop_id) if entity_and_prop_id is not None else None
 
 
 # Kind of parameter: (C function by property, C function by alias, converts the value(s) to the C arguments)