> built 2026-09-22 16:30 UTC from 21cec5b (master) · mongodol 0.1.7. Details: build_info.json # index.html.md # mongodol Access MongoDB through a `Mapping` (dict-like) interface. `mongodol` wraps `pymongo` collections as `Mapping`/`MutableMapping` objects (readers and persisters), so you can read and write mongo data with normal `dict`-like syntax, and compose your own key/value transforms with [`dol`](https://github.com/i2mint/dol) wrappers instead of writing backend-specific boilerplate. To install: ```default pip install mongodol ``` And of course, you need a running MongoDB – see the [installation instructions](https://www.mongodb.com/docs/manual/installation/). ## For AI agents `mongodol` publishes its documentation in forms made for coding agents. If you are one, start here. **The documentation, machine-readable**: [`llms.txt`](https://i2mint.github.io/mongodol/llms.txt) indexes every page; [`mongodol.md`](https://i2mint.github.io/mongodol/mongodol.md) is the whole documentation in one file; every page has a `.md` twin; [`objects.inv`](https://i2mint.github.io/mongodol/objects.inv) maps symbols to URLs. If you are a control freak, the rest of this README is written for you, starting at [Quick start](). ## Quick start ```python from mongodol import MongoCollectionPersister, mk_dflt_mgc # mk_dflt_mgc() gives you a pymongo collection to play with (mongodol/mongodol_test by default) mgc = mk_dflt_mgc() mgc.delete_many( {} ) # start from an empty collection (skip this to keep what's already there) s = MongoCollectionPersister(mgc, getitem_projection={"_id": False}) len(s) # 0 k = {"_id": "my_id"} s[k] = {"mongo": "uses", "json": "data"} list(s) # [{'_id': 'my_id'}] ``` Since the base reader is a thin, low-level wrapper, `s[k]` returns a `pymongo.cursor.Cursor` (a key may match zero, one, or many docs), so you fetch the value(s) explicitly: ```python next(s[k]) # {'mongo': 'uses', 'json': 'data'} del s[k] len(s) # 0 ``` ## Beyond the base classes The base `MongoCollectionReader`/`MongoCollectionPersister` classes always return cursors and never validate uniqueness. For the common case of “one key maps to one doc”, use one of the `*UniqueDoc*`/`*FirstDoc*` reader and persister classes instead: ```python from mongodol import MongoCollectionUniqueDocReader ``` `MongoCollectionUniqueDocReader` gives you `s[k]` as a plain `dict` (not a cursor), and raises `KeyNotUniqueError` if more than one doc matches `k`. See its docstring for a runnable example. For custom key/value shapes, business logic, or connecting `mongodol` stores to the rest of the [`dol`](https://github.com/i2mint/dol) ecosystem (caching, serialization, key transforms, etc.), wrap a `mongodol` store with `dol.wrap_kvs` like you would any other `dol` store. ## More See the [package documentation](https://i2mint.github.io/mongodol/) and the flat [`mongodol.md`](https://i2mint.github.io/mongodol/mongodol.md) aggregate for the full API.

This documentation as a single file: mongodol.md (Markdown, for agents).

# _autosummary/mongodol.add_ons.html.md # mongodol.add_ons Add-ons [https://github.com/i2mint/mongodol/issues/3](https://github.com/i2mint/mongodol/issues/3) ### Functions | [`add_clear_method`](_autosummary/mongodol.add_ons.html.md#mongodol.add_ons.add_clear_method)(store, \*[, clear_method, ...]) | Add a clear method to a store that doesn't have one | |-----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------| | [`disallow_if_name_exists_already`](_autosummary/mongodol.add_ons.html.md#mongodol.add_ons.disallow_if_name_exists_already)(store, ...) | Raise `MethodNameAlreadyExists` if `store` already has an attribute named `method_name`. | | [`has_exactly_one_non_defaulted_input`](_autosummary/mongodol.add_ons.html.md#mongodol.add_ons.has_exactly_one_non_defaulted_input)(func) | Return True iff function has exactly one argument without defaults | | [`number_of_non_defaulted_arguments`](_autosummary/mongodol.add_ons.html.md#mongodol.add_ons.number_of_non_defaulted_arguments)(func) | Return the number of arguments that don't have defaults in it's signature | ### Classes | [`Addons`](_autosummary/mongodol.add_ons.html.md#mongodol.add_ons.Addons)() | A collection of add-on methods. | |-------------------------------------------------------------|-----------------------------------| ### *class* mongodol.add_ons.Addons Bases: [`ABC`](https://docs.python.org/3/library/abc.html#abc.ABC) A collection of add-on methods. Addons can’t (and is not meant to) be instantiated. It’s just to group add-on functions (meant to be injected in stores) in one place #### clear() Delete every doc matching this store’s filter, without confirmation. #### clear_after_checking_with_user() Delete every doc matching this store’s filter, after the user confirms the count on stdin. #### dflt_clear_method() Delete every doc matching this store’s filter, without confirmation. ### mongodol.add_ons.add_clear_method(store, \*, clear_method=, validator=) Add a clear method to a store that doesn’t have one * **Parameters:** * **store** * **clear_method** * **Returns:** ```pycon >>> from dol.util import has_enabled_clear_method >>> from mongodol.base import MongoCollectionPersister >>> from mongodol.tests import data, populated_pymongo_collection >>> >>> whole_store = MongoCollectionPersister(populated_pymongo_collection(data.feature_cube)) >>> whole_length_before_clear = len(whole_store) >>> assert whole_length_before_clear == 7 >>> reds = MongoCollectionPersister(whole_store.mgc, filter={'color': 'red'}) >>> n_reds = len(reds) >>> assert n_reds == 4 ``` `reds` doesn’t have a clear method ```pycon >>> assert not has_enabled_clear_method(reds) ``` So let’s give it one ```pycon >>> reds_with_clear = add_clear_method(reds) >>> assert has_enabled_clear_method(reds_with_clear) ``` And it’s one that works too! ```pycon >>> r = reds_with_clear.clear() >>> assert len(reds_with_clear) == 0 ``` It’s the data that was deleted, not just the view. See what reds and whole_store say: ```pycon >>> assert len(reds) == 0 >>> assert len(whole_store) == whole_length_before_clear - n_reds == 3 ``` ### mongodol.add_ons.disallow_if_name_exists_already(store, method_name) Raise `MethodNameAlreadyExists` if `store` already has an attribute named `method_name`. ### mongodol.add_ons.has_exactly_one_non_defaulted_input(func) Return True iff function has exactly one argument without defaults ### mongodol.add_ons.number_of_non_defaulted_arguments(func) Return the number of arguments that don’t have defaults in it’s signature # _autosummary/mongodol.base.html.md # mongodol.base Base mongoDB data object layers ### Functions | [`operator_field_names`](_autosummary/mongodol.base.html.md#mongodol.base.operator_field_names)(obj) | The parts of `obj` that make it act as a query rather than an exact match. | |------------------------------------------------------------------------------|------------------------------------------------------------------------------| ### Classes | [`MongoBaseStore`](_autosummary/mongodol.base.html.md#mongodol.base.MongoBaseStore)([store]) | A `Store` that forwards the mongo bulk-read protocol through its transforms. | |--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------| | [`MongoClientReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoClientReader)([host, port, ...]) | A `Mapping` view of a mongo client. | | [`MongoCollectionCollection`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionCollection)([mgc, filter, ...]) | Base class wrapping a mongo collection with a fixed `filter` and `iter_projection`. | | [`MongoCollectionFieldsReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionFieldsReader)([mgc, filter, ...]) | A base class to read from a mongo collection, or subset thereof, with the Mapping (i.e. dict-like) interface. | | [`MongoCollectionPersister`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionPersister)([mgc, filter, ...]) | base class to read from and write to a mongo collection, or subset thereof, with the MutableMapping interface. | | [`MongoCollectionReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionReader)([mgc, filter, ...]) | A base class to read from a mongo collection, or subset thereof, with the Mapping (i.e. dict-like) interface. | | [`MongoDbReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoDbReader)([db_name, ...]) | Base Mongo Db Reader. | ### *class* mongodol.base.MongoBaseStore(store=) Bases: `Store` A `Store` that forwards the mongo bulk-read protocol through its transforms. Historically this was the *only* way to get `values()`/`items()` to honour a wrapper’s transforms – hence `mongodol.trans.wrap_kvs`, which uses it as the wrapper class. It is no longer needed for that: [`mongodol.views`](_autosummary/mongodol.views.html.md#module-mongodol.views) resolves the bulk path through any wrapper chain, so plain `dol.wrap_kvs` now works too. It is kept because it also forwards the write-side bulk methods (`append`/`extend`), and because code may call `iter_values()`/`contains_value()` directly. #### append(v) Forward `append` to the wrapped store, transforming `v` first. #### contains_item(item) Forward `contains_item` to the wrapped store, transforming key and value first. #### contains_value(v) Forward `contains_value` to the wrapped store, transforming `v` first. #### extend(values) Forward `extend` to the wrapped store, transforming each value first. #### iter_items() Bulk-read all `(key, value)` pairs, transforming each with `_key_of_id`/`_obj_of_data`. #### iter_values() Bulk-read all values, transforming each with `_obj_of_data`. #### persist_data(data, key=None) Write `data` under `key`, through this wrapper’s own `__setitem__`. Unlike the leaf’s `persist_data` (a thin `{ID: data[ID]} -> data` shortcut), this routes through `self[key] = data`, so it applies `_id_of_key`/ `_data_of_obj` instead of bypassing them (see i2mint/mongodol#11). `key` defaults to being inferred from `data[ID]`, for backward compatibility with the previous leaf-bound behaviour – but that inference itself bypasses the key codec, so pass `key` explicitly wherever the caller already knows it. ### *class* mongodol.base.MongoClientReader(host=None, port=None, document_class=, tz_aware=None, connect=None, type_registry=None, \*\*kwargs) Bases: `KvReader` A `Mapping` view of a mongo client. Keys are database names, values are `MongoDbReader` instances for the corresponding database. Takes the same arguments as `pymongo.MongoClient`. ```pycon >>> from mongodol.base import MongoClientReader, MongoDbReader >>> from mongodol.util import mk_dflt_mgc >>> _ = mk_dflt_mgc().insert_one({'x': 1}) # ensure the default db/collection exist >>> client_reader = MongoClientReader() >>> 'mongodol' in client_reader True >>> db_reader = client_reader['mongodol'] >>> isinstance(db_reader, MongoDbReader) True ``` ### *class* mongodol.base.MongoCollectionCollection(mgc=None, filter=None, iter_projection=None, \*\*mgc_find_kwargs) Bases: `Collection` Base class wrapping a mongo collection with a fixed `filter` and `iter_projection`. #### *property* mgc_repr A short `` string identifying the wrapped mongo collection. ### *class* mongodol.base.MongoCollectionFieldsReader(mgc=None, filter=None, key_fields=('_id',), val_fields=None) Bases: [`MongoCollectionReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionReader) A base class to read from a mongo collection, or subset thereof, with the Mapping (i.e. dict-like) interface. An “easier” interface for the common case where we just want to specify fixed fields for keys and vals. ### *class* mongodol.base.MongoCollectionPersister(mgc=None, filter=None, on_write_filter=None, iter_projection=('_id',), getitem_projection=None, , allow_operators_in_write_keys=None, \*\*mgc_find_kwargs) Bases: [`MongoCollectionReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionReader) base class to read from and write to a mongo collection, or subset thereof, with the MutableMapping interface. ```pycon >>> from mongodol.util import mk_dflt_mgc >>> mongo_collection_obj = mk_dflt_mgc() >>> s = MongoCollectionPersister(mongo_collection_obj, getitem_projection={'_id': False}) >>> for k in s: # deleting all docs in default collection ... del s[k] >>> k = {'_id': 'foo'} >>> v = {'val': 'bar'} >>> k in s # see that key is not in store (and testing __contains__) False >>> len(s) 0 >>> s[k] = v >>> len(s) 1 >>> list(s) [{'_id': 'foo'}] ``` Since this is a base mongo store, the values are cursors, so to get an actual value, you need to fetch the first doc ```pycon >>> next(s[k]) {'val': 'bar'} >>> next(s.get(k)) {'val': 'bar'} ``` Remember (see `MongoCollectionReader` docs) that `s.get` will never reach its default since the reader will always return a cursor (possibly empty). So in the following case, we should get an empty cursor (not a default value) ```pycon >>> list(s.get({'not': 'a key'}, {'default': 'val'})) # testing s.get with default [] ``` ```pycon >>> list(s.values()) [{'val': 'bar'}] >>> k in s # testing __contains__ again True >>> k in s.keys() # test the contains capability of s.keys() (a MongoKeysView instance) True >>> del s[k] >>> len(s) 0 ``` ```pycon >>> # Making a persister whose keys are 2-dimensional and values are 3-dimensional >>> from mongodol.util import normalize_projection >>> s = MongoCollectionPersister(mongo_collection_obj, ... iter_projection={'first': True, 'last': True, '_id': False}, ... getitem_projection=normalize_projection(('yob', 'proj', 'bdfl'))) >>> for _id in s: # deleting all docs in tmp ... del s[_id] >>> # writing two items >>> s[{'first': 'Guido', 'last': 'van Rossum'}] = {'yob': 1956, 'proj': 'python', 'bdfl': False} >>> s[{'first': 'Vitalik', 'last': 'Buterin'}] = {'yob': 1994, 'proj': 'ethereum', 'bdfl': True} >>> # Seeing that those two items are there >>> for key, val in s.items(): ... print(f"{key} --> {val}") {'first': 'Guido', 'last': 'van Rossum'} --> {'yob': 1956, 'proj': 'python', 'bdfl': False} {'first': 'Vitalik', 'last': 'Buterin'} --> {'yob': 1994, 'proj': 'ethereum', 'bdfl': True} ``` Writes stay inside the store’s scope: a key or value that contradicts a field of the write filter (`on_write_filter`, else `filter`) raises `ValueError`. Fields scoped with operators other than `$eq`/`$in` (such as `$ne`, `$gt`) are NOT checked and such writes are let through: give those stores an `on_write_filter` with plain values. Keys used to replace or delete docs may not contain `$`-operators or regexes (pass `allow_operators_in_write_keys=True`, or set it as a class attribute, to allow them), and those queries are confined by `filter` and `on_write_filter`. Reads (`s[k]`, `k in s`) still accept query keys, always within `filter`. #### allow_operators_in_write_keys *= False* Whether keys given to write/delete operations may contain `$`-operators. #### append(v) Insert a single doc `v`, merged with `on_write_filter` if set, else this store’s filter. #### extend(values) Insert several docs `values`, each merged with `on_write_filter` if set, else this store’s filter. #### persist_data(data) Write `data` (a doc with an `_id`) under the key `{ID: data[ID]}`. ### *class* mongodol.base.MongoCollectionReader(mgc=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Bases: [`MongoCollectionCollection`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionCollection), `KvReader` A base class to read from a mongo collection, or subset thereof, with the Mapping (i.e. dict-like) interface. Some examples below. For examples using actual data (with setup and tear down) see the tests/ folder. ```pycon >>> from pymongo import MongoClient >>> s = MongoCollectionReader(MongoClient()['mongodol']['mongodol_test']) >>> list_of_keys = list(s) >>> fake_key = {'_id': 'this key does not exist'} >>> fake_key in s False ``` It’s important to note that `s[k]` (for any base MongoCollectionReader instance `s`) returns a Cursor, and will always return a Cursor, no matter what key `k` you ask for – as long as the key is a valid mapping (dict usually). This cursor is a (pymongo) object that is used to iterate over the results of the `k` lookup. It may yield no results what-so-ever, or one, or many. ```pycon >>> v = s[fake_key] >>> type(v).__name__ 'Cursor' >>> len(list(v)) # but the cursor yields no results 0 ``` Indeed, `MongoCollectionReader` is really meant to provide a low level key-value interface to a mongo collection that is really meant to be wrapped in order to produce the actual key-value interfaces one needs. You shouldn’t think of it’s instances as a normal dict where any request for the value under a key, for a key that doesn’t exist, will result in a `KeyError`. Note that this means that `s.get(k, default)` will never result in the default being returned, since there are no missing keys here; only empty results (cursors that don’t yield anything). ```pycon >>> v = s.get(fake_key, {'the': 'default'}) >>> assert v != {'the': 'default'} ``` `s.keys()`, `s.values()`, and `s.items()` are `collections.abc.MappingViews` instances (specialized for mongo – see [`mongodol.views`](_autosummary/mongodol.views.html.md#module-mongodol.views): they fetch the whole collection in a single query, and keep doing so, correctly, when the store is wrapped by `dol`). ```pycon >>> assert type(s.keys()) == s.KeysView >>> assert type(s.values()) == s.ValuesView >>> assert type(s.items()) == s.ItemsView ``` Recall that `collections.abc.MappingViews` have many set-like functionalities: ```pycon >>> fake_key in s.keys() False >>> a_list_of_fake_keys = [{'_id': 'fake_key'}, {'_id': 'yet_another'}] >>> s.keys().isdisjoint(a_list_of_fake_keys) True >>> s.keys() & a_list_of_fake_keys set() >>> fake_value = {'data': "this does not exist"} >>> fake_value in s.values() False >>> fake_item = (fake_key, fake_value) >>> fake_item in s.items() False ``` Note though that since keys and values are both dictionaries in mongo, some of these set-like functionalities might not work (complaints such as `TypeError: unhashable type: 'dict'`), such as: ```pycon >>> s.keys() | a_list_of_fake_keys Traceback (most recent call last): ... TypeError: unhashable type: 'dict' ``` But you can take care of that in higher level wrappers that have hashable keys and/or values. #### ItemsView alias of [`MongoItemsView`](_autosummary/mongodol.views.html.md#mongodol.views.MongoItemsView) #### ValuesView Views that resolve the bulk-read fast path through any `dol` wrapper chain, rather than through blind attribute delegation. See [`mongodol.views`](_autosummary/mongodol.views.html.md#module-mongodol.views). alias of [`MongoValuesView`](_autosummary/mongodol.views.html.md#mongodol.views.MongoValuesView) #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### *classmethod* from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### *property* key_fields The field names (from `iter_projection`) that make up a key. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### *property* val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.base.MongoDbReader(db_name='mongodol', mk_collection_store=, mongo_client=None, \*\*mongo_client_kwargs) Bases: `KvReader` Base Mongo Db Reader. Keys are collection names and values are collection store instances. * **Parameters:** * **db_name** – Name of db * **mk_collection_store** – Function that is called on a key (collection name) to make the collection store instance. Use mk_collection_store to define what kind of collection stores you want to make. Will be called with only one unnamed argument; the collection name. Use custom classes here, and/or partials (curried functions) thereof, to fix any parameters you want to fix. * **mongo_client** – MongoClient instance, kwargs to make it (`MongoClient(**kwargs)`), or callable to make it * **mongo_client_kwargs** – `**kwargs` to make a MongoClient, that is used if mongo_client is callable ```pycon >>> from mongodol.base import MongoDbReader >>> from mongodol.util import mk_dflt_mgc >>> _ = mk_dflt_mgc().insert_one({'x': 1}) # ensure the default db/collection exist >>> db_reader = MongoDbReader() >>> 'mongodol_test' in db_reader True ``` ### mongodol.base.operator_field_names(obj) The parts of `obj` that make it act as a query rather than an exact match. That is: `$`-prefixed field names and regular-expression values, found at any depth (in mappings and lists). Regexes are reported by their `repr`. * **Return type:** [`list`](https://docs.python.org/3/builtins/stdtypes.html#list) ```pycon >>> operator_field_names({'a': 1, 'b': {'c': [{'$gt': 2}]}}) ['$gt'] >>> operator_field_names({'a': re.compile('x')}) ["re.compile('x')"] >>> operator_field_names({'a': 1}) [] ``` # _autosummary/mongodol.constants.html.md # mongodol.constants Module to centralize constants used throughout project. This includes enums, aliases, defaults, types, etc. # _autosummary/mongodol.errors.html.md # mongodol.errors Where mongodol error objects are # _autosummary/mongodol.html.md # mongodol Access mongo through a Mapping interface ### Modules | [`add_ons`](_autosummary/mongodol.add_ons.html.md#module-mongodol.add_ons) | Add-ons [https://github.com/i2mint/mongodol/issues/3](https://github.com/i2mint/mongodol/issues/3) | |----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| | [`base`](_autosummary/mongodol.base.html.md#module-mongodol.base) | Base mongoDB data object layers | | [`constants`](_autosummary/mongodol.constants.html.md#module-mongodol.constants) | Module to centralize constants used throughout project. | | [`errors`](_autosummary/mongodol.errors.html.md#module-mongodol.errors) | Where mongodol error objects are | | [`recipes`](_autosummary/mongodol.recipes.html.md#module-mongodol.recipes) | Mongodol Recipes | | [`stores`](_autosummary/mongodol.stores.html.md#module-mongodol.stores) | Some useful stores for mongoDB | | [`tracking_methods`](_autosummary/mongodol.tracking_methods.html.md#module-mongodol.tracking_methods) | Tracking functionality | | [`trans`](_autosummary/mongodol.trans.html.md#module-mongodol.trans) | Transformative functionality | | [`util`](_autosummary/mongodol.util.html.md#module-mongodol.util) | Util functions | | [`utils`](_autosummary/mongodol.utils.html.md#module-mongodol.utils) | Util modules | | [`views`](_autosummary/mongodol.views.html.md#module-mongodol.views) | Mapping views that keep working when a mongo store is wrapped by `dol`. | # _autosummary/mongodol.recipes.html.md # mongodol.recipes Mongodol Recipes ### Functions | [`disallow_sourced_interval_overlaps`](_autosummary/mongodol.recipes.html.md#mongodol.recipes.disallow_sourced_interval_overlaps)(store) | Disallow writing to a key that shares the same "source" field value and overlapping ("bt", "tt") interval. | |----------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| ### Exceptions | [`WriteNotAllowedToThatKey`](_autosummary/mongodol.recipes.html.md#mongodol.recipes.WriteNotAllowedToThatKey) | To indicate that once cannot write to some specific key one is trying to write to | |-----------------------------------------------------------------------------|-------------------------------------------------------------------------------------| ### *exception* mongodol.recipes.WriteNotAllowedToThatKey Bases: [`KeyError`](https://docs.python.org/3/builtins/exceptions.html#KeyError) To indicate that once cannot write to some specific key one is trying to write to ### mongodol.recipes.disallow_sourced_interval_overlaps(store) Disallow writing to a key that shares the same “source” field value and overlapping (“bt”, “tt”) interval. * **Parameters:** **store** – `KvPersister` (instance or class) `s` * **Returns:** The same store, but where `s[dict(source=source, bt=bt, tt=tt)] = v` writes are not permitted if there is another doc, with the same source, and an overlapping (bt, tt) interval. ```pycon >>> from mongodol.tests import get_test_collection_persister, clear_all_and_populate >>> >>> # We're going to take (make really) a store s with the two follwing documents: >>> >>> data = [ ... {'source': 'audio', 'bt': 6, 'tt': 9, 'annot': 'dog'}, ... {'source': 'visual', 'bt': 6, 'tt': 15, 'annot': 'dog'} ... ] >>> >>> # Then try to do s[k] = v with the following (k, v) pairs. >>> k1 = {'source': 'audio', 'bt': 12, 'tt': 16} >>> v1 = {'annot': 'cat'} >>> # Note here that (7, 10) overlaps with (6, 9) (in source=audio) >>> k2 = {'source': 'audio', 'bt': 7, 'tt': 10} >>> v2 = {'annot': 'cat'} >>> >>> >>> >>> s = get_test_collection_persister() # Make a persister >>> clear_all_and_populate(data,s) # empty it and populate it with the two data docs >>> assert len(s) == 2 # yep, two docs >>> assert s.distinct('annot') == ['dog'] # and only has a dog >>> >>> >>> s[k1] = v1 >>> assert len(s) == 3 # Now has three docs >>> assert 'cat' in s.distinct('annot') # has a cat now >>> >>> s[k2] = v2 >>> assert len(s) == 4 # v2 was written, indeed >>> >>> >>> # Let's start over, usingour disallow_sourced_interval_overlaps decorator this time >>> >>> protected_s = disallow_sourced_interval_overlaps(get_test_collection_persister()) >>> clear_all_and_populate(data,protected_s) # empty it and populate it with the two data docs >>> >>> s[k1] = v1 >>> # No problem! And see that you have a cat annot now! >>> assert 'cat' in s.distinct('annot') >>> # But this next one will not work since the (7, 10) overlaps with (6, 9) (in source=audio) >>> try: ... protected_s[k2] = v2 ... except WriteNotAllowedToThatKey: ... print("WriteNotAllowedToThatKey expected!") WriteNotAllowedToThatKey expected! ``` # _autosummary/mongodol.stores.html.md # mongodol.stores Some useful stores for mongoDB ### Classes | [`MongoCollectionFirstDocPersister`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionFirstDocPersister)([mgc, ...]) | A mongo collection (kv-)reader where s[key] is the first key-matching value found. | |----------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------| | [`MongoCollectionFirstDocReader`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionFirstDocReader)([mgc, filter, ...]) | A mongo collection (kv-)reader where s[key] is the first key-matching value found. | | [`MongoCollectionMultipleDocsPersister`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionMultipleDocsPersister)([mgc, ...]) | A mongo collection (kv-)reader where s[key] will return the list of all key-matching docs. | | [`MongoCollectionMultipleDocsReader`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionMultipleDocsReader)([mgc, ...]) | A mongo collection (kv-)reader where s[key] will return the list of all key-matching docs. | | [`MongoCollectionPersisterWithResultMapping`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionPersisterWithResultMapping)([...]) | MongoCollectionPersister with result mapping | | [`MongoCollectionUniqueDocPersister`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionUniqueDocPersister)([mgc, ...]) | A mongo collection (kv-)reader where s[key] is the dict (a mongo doc matching the key). | | [`MongoCollectionUniqueDocReader`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoCollectionUniqueDocReader)([mgc, ...]) | A mongo collection (kv-)reader where s[key] is the dict (a mongo doc matching the key). | | [`MongoStore`](_autosummary/mongodol.stores.html.md#mongodol.stores.MongoStore)([store]) | A `Store` wrapping a `MongoCollectionUniqueDocPersister`, built from host/db/collection names. | ### *class* mongodol.stores.MongoCollectionFirstDocPersister(mgc=None, filter=None, on_write_filter=None, iter_projection=('_id',), getitem_projection=None, , allow_operators_in_write_keys=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] is the first key-matching value found. Unlike MongoCollectionUniqueDocReader, MongoCollectionFirstDocReader doesn’t check for uniqueness. Typically, this should be used when you don’t want the overhead of checking for uniqueness, because it doesn’t matter, you like risk, or you told the mongo collection indexing system itself to ensure uniqueness for you. ```pycon >>> from mongodol.stores import MongoCollectionFirstDocPersister >>> from mongodol.tests import util >>> test_mgc = util.populated_pymongo_collection([]) >>> s = MongoCollectionFirstDocPersister(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection={'n': True, '_id': False}) >>> s[{'s': 'a'}] = {'n': 1} >>> s[{'s': 'a'}] {'n': 1} ``` A second doc matching the same key does not raise; `s[key]` keeps returning the first match found: ```pycon >>> _ = s.mgc.insert_one({'s': 'a', 'n': 999}) >>> s[{'s': 'a'}] {'n': 1} ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### allow_operators_in_write_keys bool(x) -> bool Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. #### append(v) Insert a single doc `v`, merged with `on_write_filter` if set, else this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### extend(values) Insert several docs `values`, each merged with `on_write_filter` if set, else this store’s filter. #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### persist_data(data) Write `data` (a doc with an `_id`) under the key `{ID: data[ID]}`. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoCollectionFirstDocReader(mgc=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] is the first key-matching value found. Unlike MongoCollectionUniqueDocReader, MongoCollectionFirstDocReader doesn’t check for uniqueness. Typically, this should be used when you don’t want the overhead of checking for uniqueness, because it doesn’t matter, you like risk, or you told the mongo collection indexing system itself to ensure uniqueness for you. ```pycon >>> from mongodol.stores import MongoCollectionFirstDocReader >>> from mongodol.tests import data, util >>> test_mgc = util.populated_pymongo_collection(data.three_simple_docs) >>> s = MongoCollectionFirstDocReader(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection=['n']) >>> assert list(s) == [{'s': 'a'}, {'s': 'b'}, {'s': 'b'}] ``` Unlike `MongoCollectionUniqueDocReader`, a key matching more than one doc doesn’t raise; it just returns the first match found: ```pycon >>> s[{'s': 'a'}] {'_id': 0, 'n': 1} >>> s[{'s': 'b'}] {'_id': 1, 'n': 2} ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoCollectionMultipleDocsPersister(mgc=None, filter=None, on_write_filter=None, iter_projection=('_id',), getitem_projection=None, , allow_operators_in_write_keys=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] will return the list of all key-matching docs. If no docs match, will return an empty list. `s[key] = v` first deletes every doc matching `key`, then inserts `v` (a doc, or a collection of docs) merged with `key`: ```pycon >>> from mongodol.stores import MongoCollectionMultipleDocsPersister >>> from mongodol.tests import util >>> test_mgc = util.populated_pymongo_collection([]) >>> s = MongoCollectionMultipleDocsPersister(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection={'n': True, '_id': False}) >>> s[{'s': 'a'}] = [{'n': 1}, {'n': 2}] >>> s[{'s': 'a'}] [{'n': 1}, {'n': 2}] ``` ```pycon >>> s[{'s': 'a'}] = {'n': 3} # replaces the two docs above with just this one >>> s[{'s': 'a'}] [{'n': 3}] ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### allow_operators_in_write_keys bool(x) -> bool Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. #### append(v) Insert a single doc `v`, merged with `on_write_filter` if set, else this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### extend(values) Insert several docs `values`, each merged with `on_write_filter` if set, else this store’s filter. #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### persist_data(data) Write `data` (a doc with an `_id`) under the key `{ID: data[ID]}`. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoCollectionMultipleDocsReader(mgc=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] will return the list of all key-matching docs. If no docs match, will return an empty list. ```pycon >>> from mongodol.stores import MongoCollectionMultipleDocsReader >>> from mongodol.tests import data, util >>> test_mgc = util.populated_pymongo_collection(data.three_simple_docs) >>> s = MongoCollectionMultipleDocsReader(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection=['n']) >>> s[{'s': 'a'}] [{'_id': 0, 'n': 1}] >>> s[{'s': 'b'}] [{'_id': 1, 'n': 2}, {'_id': 2, 'n': 3}] >>> s[{'s': 'nonexistent'}] [] ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoCollectionPersisterWithResultMapping(mgc=None, filter=None, on_write_filter=None, iter_projection=('_id',), getitem_projection=None, , allow_operators_in_write_keys=None, \*\*mgc_find_kwargs) Bases: [`MongoCollectionPersister`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionPersister) MongoCollectionPersister with result mapping #### append(v) Insert a single doc `v`, merged with `on_write_filter` if set, else this store’s filter. #### extend(values) Insert several docs `values`, each merged with `on_write_filter` if set, else this store’s filter. ### *class* mongodol.stores.MongoCollectionUniqueDocPersister(mgc=None, filter=None, on_write_filter=None, iter_projection=('_id',), getitem_projection=None, , allow_operators_in_write_keys=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] is the dict (a mongo doc matching the key). * **Raises:** [**KeyNotUniqueError**](_autosummary/mongodol.util.html.md#mongodol.util.KeyNotUniqueError) – if the k matches more than a single unique doc. ```pycon >>> from mongodol.stores import MongoCollectionUniqueDocPersister >>> from mongodol.tests import util >>> test_mgc = util.populated_pymongo_collection([]) >>> s = MongoCollectionUniqueDocPersister(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection={'n': True, '_id': False}) >>> s[{'s': 'a'}] = {'n': 1} >>> list(s) [{'s': 'a'}] >>> s[{'s': 'a'}] {'n': 1} ``` ```pycon >>> s[{'s': 'b'}] = {'n': 2} >>> _ = s.mgc.insert_one({'s': 'b', 'n': 99}) >>> s[{'s': 'b'}] Traceback (most recent call last): ... mongodol.util.KeyNotUniqueError: Key was not unique (i.e. cursor has more than one match): {'s': 'b'} ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### allow_operators_in_write_keys bool(x) -> bool Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. #### append(v) Insert a single doc `v`, merged with `on_write_filter` if set, else this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### extend(values) Insert several docs `values`, each merged with `on_write_filter` if set, else this store’s filter. #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### persist_data(data) Write `data` (a doc with an `_id`) under the key `{ID: data[ID]}`. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoCollectionUniqueDocReader(mgc=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Bases: `Store` A mongo collection (kv-)reader where s[key] is the dict (a mongo doc matching the key). * **Raises:** [**KeyNotUniqueError**](_autosummary/mongodol.util.html.md#mongodol.util.KeyNotUniqueError) – if the k matches more than a single unique doc. ```pycon >>> from mongodol.stores import MongoCollectionUniqueDocReader >>> from mongodol.tests import data, util >>> test_mgc = util.populated_pymongo_collection(data.three_simple_docs) >>> s = MongoCollectionUniqueDocReader(test_mgc, ... iter_projection={'s': True, '_id': False}, getitem_projection=['n']) >>> assert list(s) == [{'s': 'a'}, {'s': 'b'}, {'s': 'b'}] ``` And you see where the problem will be: There’s two {‘s’: ‘b’} in that listing, so though getting the value for {‘s’: ‘a’} won’t be a problem: ```pycon >>> assert s[{'s': 'a'}] == {'_id': 0, 'n': 1} # there's only one doc matching {'s': 'a'} ``` … but there’s more than one doc matching {‘s’: ‘b’} ```pycon >>> s[{'s': 'b'}] Traceback (most recent call last): ... mongodol.util.KeyNotUniqueError: Key was not unique (i.e. cursor has more than one match): {'s': 'b'} ``` #### aggregate(pipeline, \*\*kwargs) Run a mongo aggregation `pipeline`, prefixed with a `$match` on this store’s filter. #### contains_item(item) Bulk-read counterpart of `__contains__` for `(key, value)` pairs. #### contains_value(v) Bulk-read counterpart of `__contains__` for values: is there a doc matching `v`? #### distinct(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### from_params(db_name='mongodol', collection_name='test', mongo_client=None, filter=None, iter_projection=('_id',), getitem_projection=None, \*\*mgc_find_kwargs) Make an instance from db/collection names and connection params, instead of a live mongo collection object. #### iter_items() Bulk-read all `(key, value)` pairs in a single `find` query, splitting each doc into its key fields and the rest. #### iter_values() Bulk-read all values in a single `find` query (see the module’s bulk-read protocol). #### key_fields The field names (from `iter_projection`) that make up a key. #### mgc_repr A short `` string identifying the wrapped mongo collection. #### unique(key, filter=None, \*\*kwargs) The distinct values of `key` across docs matching `filter` (merged with this store’s own filter). #### val_fields The field names (from `getitem_projection`) that make up a value, or None if unset. ### *class* mongodol.stores.MongoStore(store=) Bases: `Store` A `Store` wrapping a `MongoCollectionUniqueDocPersister`, built from host/db/collection names. # _autosummary/mongodol.tracking_methods.html.md # mongodol.tracking_methods Tracking functionality ### Functions | [`add_tracked_methods`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.add_tracked_methods)([tracked_methods, ...]) | Factory of decorators to add method call tracking to a class | |--------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------| | [`consume`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.consume)(gen) | Exhaust an iterable/generator `gen` for its side effects, discarding all values. | | [`forward_method_calls`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.forward_method_calls)(method) | Wrap `method` so calls on `self` are forwarded to `self._instance` instead. | | [`track_calls_of_method`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.track_calls_of_method)(method[, ...]) | Wrap `method` so every call is appended to `self._tracks`, and (if `execute_call`) also run. | | [`track_calls_without_executing`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.track_calls_without_executing)(method) | Wrap `method` so every call is appended to `self._tracks`, but never actually run. | | [`track_method_calls`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.track_method_calls)([obj, tracked_methods, ...]) | Wrapping objects (classes or instances) so that specific method calls are tracked (i.e. a list of (method_func, args, kwargs) is maintained). | ### Classes | [`MongoBulkWritesMixin`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.MongoBulkWritesMixin)() | Used to accumulate write operations and execute them in bulk, efficiently | |---------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| | [`TrackableMixin`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.TrackableMixin)() | Mixin that provides a container for method call tracking, execution, and a context manager that will execute tracks and empty them. | ### *class* mongodol.tracking_methods.MongoBulkWritesMixin Bases: [`TrackableMixin`](_autosummary/mongodol.tracking_methods.html.md#mongodol.tracking_methods.TrackableMixin) Used to accumulate write operations and execute them in bulk, efficiently #### commit() Execute all pending tracked calls, clear the tracks, and return the call results. ### *class* mongodol.tracking_methods.TrackableMixin Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object) Mixin that provides a container for method call tracking, execution, and a context manager that will execute tracks and empty them. TrackableMixin is used as the default tracking_mixin in track_method_calls. It uses list as the collection for tracks, and implements a basic execute_tracks (which loops through tracks, executes them, and accumulates results in a list which it returns). TrackableMixin is meant to be subclassed and execute_tracks overwritten by a custom handler. #### clear_tracks() Discard all pending tracked calls without executing them. #### flush() Execute all pending tracked calls, clear the tracks, and return the call results. #### tracks_factory alias of [`list`](https://docs.python.org/3/builtins/stdtypes.html#list) ### mongodol.tracking_methods.add_tracked_methods(tracked_methods=frozenset({}), calls_tracker=) Factory of decorators to add method call tracking to a class * **Parameters:** * **tracked_methods** ([`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Method name or iterable of method names to track * **tracking_mixin** – The mixin class to use to inject the \_tracks attribute, and other tracking utils (flush…) * **calls_tracker** ([`Callable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable)) – The method decorator that implements the actual tracking ### mongodol.tracking_methods.consume(gen) Exhaust an iterable/generator `gen` for its side effects, discarding all values. ### mongodol.tracking_methods.forward_method_calls(method) Wrap `method` so calls on `self` are forwarded to `self._instance` instead. ### mongodol.tracking_methods.track_calls_of_method(method, execute_call=True, tracks_factory=) Wrap `method` so every call is appended to `self._tracks`, and (if `execute_call`) also run. ### mongodol.tracking_methods.track_calls_without_executing(method) Wrap `method` so every call is appended to `self._tracks`, but never actually run. ### mongodol.tracking_methods.track_method_calls(obj=None, \*, tracked_methods=frozenset({}), tracking_mixin=, calls_tracker=) Wrapping objects (classes or instances) so that specific method calls are tracked (i.e. a list of (method_func, args, kwargs) is maintained) * **Parameters:** * **obj** * **tracked_methods** ([`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Method name or iterable of method names to track * **tracking_mixin** ([`type`](https://docs.python.org/3/builtins/functions.html#type)) – The mixin class to use to inject the \_tracks attribute, and other tracking utils (flush…) * **calls_tracker** ([`Callable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable)) – The method decorator that implements the actual tracking * **Returns:** A decorated class (of obj is a type) or instance (if obj is an instance) that implements method tracking ```pycon >>> @track_method_calls(tracked_methods='__setitem__') ... class D(dict): ... pass >>> d = D(a=1, b=[1, 2], c={'hello': 'world'}) >>> assert repr(d) == "{'a': 1, 'b': [1, 2], 'c': {'hello': 'world'}}" >>> assert d._tracks == [] >>> d['a'] 1 >>> d._tracks # accessing 'a' didn't make any tracks [] >>> d['a'] = 42 >>> d['a'] # verifying that dd['a'] is now 42 42 >>> len(d._tracks) # see that dd._tracks is now non-empty 1 >>> d._tracks [(, ('a', 42), {})] ``` A common use of `track_method_calls` is to accumulate method calls without executing them, so as to be able to change the way they’re called. For example, making the calls differently (e.g. in a parallel process) or aggregating several operations and running them in bulk (e.g. data base writes). If you want to reuse your tracker decorator, it’s a good idea of use partial to make a decorator with the settings you want, like this: ```pycon >>> from functools import partial >>> my_write_tracker = partial( ... track_method_calls, ... tracked_methods='__setitem__', ... calls_tracker=track_calls_without_executing ... ) ``` Now let’s decorate a dict type with it. ```pycon >>> @my_write_tracker ... class D(dict): ... pass >>> d = D(a=1, b=[1, 2], c={'hello': 'world'}) ``` The suggested use is to do write operations in a with block. This will have the effect of automatically executing the calls accumulated in tracks and clearing the tracks when you exit the with block. ```pycon >>> with d: ... d['a'] = 21 ... assert d['a'] == 1 # still in the with block, so the operation hasn't executed yet >>> d['a'] # but now that we exited the block, we have d['a'] == 21 21 ``` But if you really need/want to, you can perform these operations manually. ```pycon >>> assert d._tracks == [] # see that we have no _tracks (these are deleted when we exit the with block >>> d['a'] = 42 >>> assert d['a'] == 21 # verifying that dd['a'] is STILL 21 >>> assert len(d._tracks) > 0 # but dd._tracks is now non-empty >>> assert str(d._tracks) == "[(, ('a', 42), {})]" ``` To execute the command in \_tracks, you can use the `.flush()` method ```pycon >>> _ = d.flush() >>> # See that the setitem call was indeed made >>> assert d['a'] == 42 >>> assert len(d._tracks) == 0 ``` Here’s what’s happening behind the scenes: ```pycon >>> d['b'] = [3, 4] # write to 'b' >>> assert d['b'] != [3, 4] # but it's not actually written >>> func, args, kwargs = d._tracks[0] # the tracks now has a (func, args, kwargs) triple >>> func(d, *args, **kwargs) # if we cann that function on the instance (and *args, **kwargs) >>> assert d['b'] == [3, 4] # Not the write is actually performed and d['b'] becomes [3, 4] ``` Above, we were wrapping a class, but you can also wrap an instance! ```pycon >>> d = dict(a=1, b=[1,2], c={'hello': 'world'}) >>> dd = track_method_calls(d, tracked_methods='__getitem__') >>> v = dd['a'] # TypeError: __getitem__() takes exactly one argument (2 given) >>> assert v == 1 # you got the value alright! >>> dd._tracks [(proxy __getitem__, ('a',), {})] >>> # It's a weird name for the function, but the function still works: >>> func, args, kwargs = dd._tracks[0] >>> func(dd, *args, **kwargs) 1 ``` ### mongodol.tracking_methods.with_bulk_writes(obj=None, \*, tracked_methods=frozenset({}), tracking_mixin=, calls_tracker=) Wrapping objects (classes or instances) so that specific method calls are tracked (i.e. a list of (method_func, args, kwargs) is maintained) * **Parameters:** * **obj** * **tracked_methods** – Method name or iterable of method names to track * **tracking_mixin** – The mixin class to use to inject the \_tracks attribute, and other tracking utils (flush…) * **calls_tracker** – The method decorator that implements the actual tracking * **Returns:** A decorated class (of obj is a type) or instance (if obj is an instance) that implements method tracking ```pycon >>> @track_method_calls(tracked_methods='__setitem__') ... class D(dict): ... pass >>> d = D(a=1, b=[1, 2], c={'hello': 'world'}) >>> assert repr(d) == "{'a': 1, 'b': [1, 2], 'c': {'hello': 'world'}}" >>> assert d._tracks == [] >>> d['a'] 1 >>> d._tracks # accessing 'a' didn't make any tracks [] >>> d['a'] = 42 >>> d['a'] # verifying that dd['a'] is now 42 42 >>> len(d._tracks) # see that dd._tracks is now non-empty 1 >>> d._tracks [(, ('a', 42), {})] ``` A common use of `track_method_calls` is to accumulate method calls without executing them, so as to be able to change the way they’re called. For example, making the calls differently (e.g. in a parallel process) or aggregating several operations and running them in bulk (e.g. data base writes). If you want to reuse your tracker decorator, it’s a good idea of use partial to make a decorator with the settings you want, like this: ```pycon >>> from functools import partial >>> my_write_tracker = partial( ... track_method_calls, ... tracked_methods='__setitem__', ... calls_tracker=track_calls_without_executing ... ) ``` Now let’s decorate a dict type with it. ```pycon >>> @my_write_tracker ... class D(dict): ... pass >>> d = D(a=1, b=[1, 2], c={'hello': 'world'}) ``` The suggested use is to do write operations in a with block. This will have the effect of automatically executing the calls accumulated in tracks and clearing the tracks when you exit the with block. ```pycon >>> with d: ... d['a'] = 21 ... assert d['a'] == 1 # still in the with block, so the operation hasn't executed yet >>> d['a'] # but now that we exited the block, we have d['a'] == 21 21 ``` But if you really need/want to, you can perform these operations manually. ```pycon >>> assert d._tracks == [] # see that we have no _tracks (these are deleted when we exit the with block >>> d['a'] = 42 >>> assert d['a'] == 21 # verifying that dd['a'] is STILL 21 >>> assert len(d._tracks) > 0 # but dd._tracks is now non-empty >>> assert str(d._tracks) == "[(, ('a', 42), {})]" ``` To execute the command in \_tracks, you can use the `.flush()` method ```pycon >>> _ = d.flush() >>> # See that the setitem call was indeed made >>> assert d['a'] == 42 >>> assert len(d._tracks) == 0 ``` Here’s what’s happening behind the scenes: ```pycon >>> d['b'] = [3, 4] # write to 'b' >>> assert d['b'] != [3, 4] # but it's not actually written >>> func, args, kwargs = d._tracks[0] # the tracks now has a (func, args, kwargs) triple >>> func(d, *args, **kwargs) # if we cann that function on the instance (and *args, **kwargs) >>> assert d['b'] == [3, 4] # Not the write is actually performed and d['b'] becomes [3, 4] ``` Above, we were wrapping a class, but you can also wrap an instance! ```pycon >>> d = dict(a=1, b=[1,2], c={'hello': 'world'}) >>> dd = track_method_calls(d, tracked_methods='__getitem__') >>> v = dd['a'] # TypeError: __getitem__() takes exactly one argument (2 given) >>> assert v == 1 # you got the value alright! >>> dd._tracks [(proxy __getitem__, ('a',), {})] >>> # It's a weird name for the function, but the function still works: >>> func, args, kwargs = dd._tracks[0] >>> func(dd, *args, **kwargs) 1 ``` # _autosummary/mongodol.trans.html.md # mongodol.trans Transformative functionality ### Functions | [`get_persistent_obj`](_autosummary/mongodol.trans.html.md#mongodol.trans.get_persistent_obj)(container, v) | Wrap `v` in a `PersistentDict`/`PersistentList` if it's a mapping/iterable, else return it as is. | |----------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | [`normalize_result`](_autosummary/mongodol.trans.html.md#mongodol.trans.normalize_result)(obj, \*[, ...]) | Decorator to transform a pymongo result object to a WriteOpResult object. | | [`set_key_and_data_fields`](_autosummary/mongodol.trans.html.md#mongodol.trans.set_key_and_data_fields)([store, key_fields, ...]) | Decorator to set key_fields and data_fields on a store. | ### Classes | [`ObjOfData`](_autosummary/mongodol.trans.html.md#mongodol.trans.ObjOfData)() | `obj_of_data` (value-only) transform functions for `wrap_kvs`. | |------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------| | [`PersistentDict`](_autosummary/mongodol.trans.html.md#mongodol.trans.PersistentDict)(container, wrapped_dict) | Extension of a dict wich triggers an event to notify the object that contains the dict that a modification has been made. | | [`PersistentList`](_autosummary/mongodol.trans.html.md#mongodol.trans.PersistentList)(container, iterable) | Extension of a list wich triggers an event to notify the object that contains the list that a modification has been made. | | [`PersistentObjectBase`](_autosummary/mongodol.trans.html.md#mongodol.trans.PersistentObjectBase)(container) | Base class to propagate a modification event through a parent-child chain structure. | | [`PostGet`](_autosummary/mongodol.trans.html.md#mongodol.trans.PostGet)() | `postget` (key-aware) transform functions for `wrap_kvs`, turning a cursor into a value. | | [`WriteOpResult`](_autosummary/mongodol.trans.html.md#mongodol.trans.WriteOpResult) | The shape of a normalized mongo write-operation result (see `normalize_result`). | ### *class* mongodol.trans.ObjOfData Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object) `obj_of_data` (value-only) transform functions for `wrap_kvs`. #### *static* all_docs_fetch(cursor, doc_collector=) Collect every doc in `cursor` into `doc_collector` (default: a list). The value-only (`obj_of_data`) counterpart of [`PostGet.all_docs_fetch()`](_autosummary/mongodol.trans.html.md#mongodol.trans.PostGet.all_docs_fetch). ### *class* mongodol.trans.PersistentDict(container, wrapped_dict) Bases: [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict), [`PersistentObjectBase`](_autosummary/mongodol.trans.html.md#mongodol.trans.PersistentObjectBase) Extension of a dict wich triggers an event to notify the object that contains the dict that a modification has been made. Requirement: The container object needs to implement the method “persist_data(self, data: Mapping)”. ```pycon >>> d = { ... 'a': 1, ... 'b': {'ba': 2, 'bb': 3}, ... 'c': [ ... {'c1a': 4, 'c1b': 5}, ... {'c2a': 6, 'c2b': '7'} ... ] ... } >>> class Container: ... def persist_data(self, data): ... """Here, you'd normally put code to ACTUALLY persist the data""" ... print(f"persisting {data}") >>> pd = PersistentDict(Container(), d) >>> pd['a'] = 8 persisting {'a': 8, 'b': {'ba': 2, 'bb': 3}, 'c': [{'c1a': 4, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['a'] == 8 # and indeed pd['a'] is 8 now! >>> pd['b']['ba'] = 8 persisting {'a': 8, 'b': {'ba': 8, 'bb': 3}, 'c': [{'c1a': 4, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['b']['ba'] == 8 # and indeed pd['b']['ba'] is 8 now! >>> pd['c'][0]['c1a'] = 8 persisting {'a': 8, 'b': {'ba': 8, 'bb': 3}, 'c': [{'c1a': 8, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['c'][0]['c1a'] == 8 # and indeed pd['c'][0]['c1a'] is 8 now! >>> pd.update({'a': 9}) persisting {'a': 9, 'b': {'ba': 8, 'bb': 3}, 'c': [{'c1a': 8, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['a'] == 9 >>> pd['b'].update({'ba': 9}) persisting {'a': 9, 'b': {'ba': 9, 'bb': 3}, 'c': [{'c1a': 8, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['b']['ba'] == 9 >>> pd['c'][0].update({'c1a': 9}) persisting {'a': 9, 'b': {'ba': 9, 'bb': 3}, 'c': [{'c1a': 9, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['c'][0]['c1a'] == 9 >>> pd.update([('a', 10)]) persisting {'a': 10, 'b': {'ba': 9, 'bb': 3}, 'c': [{'c1a': 9, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['a'] == 10 >>> pd['b'].update([('ba', 10)]) persisting {'a': 10, 'b': {'ba': 10, 'bb': 3}, 'c': [{'c1a': 9, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['b']['ba'] == 10 >>> pd['c'][0].update([('c1a', 10)]) persisting {'a': 10, 'b': {'ba': 10, 'bb': 3}, 'c': [{'c1a': 10, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert pd['c'][0]['c1a'] == 10 >>> del pd['a'] persisting {'b': {'ba': 10, 'bb': 3}, 'c': [{'c1a': 10, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert 'a' not in pd # indeed, 'a' no longer in pd >>> del pd['b']['ba'] persisting {'b': {'bb': 3}, 'c': [{'c1a': 10, 'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert 'ba' not in pd['b'] >>> del pd['c'][0]['c1a'] persisting {'b': {'bb': 3}, 'c': [{'c1b': 5}, {'c2a': 6, 'c2b': '7'}]} >>> assert 'c1a' not in pd['c'][0] ``` #### update(\*args, \*\*kwargs) Update like a normal dict, then persist the updated dict. ### *class* mongodol.trans.PersistentList(container, iterable) Bases: [`list`](https://docs.python.org/3/builtins/stdtypes.html#list), [`PersistentObjectBase`](_autosummary/mongodol.trans.html.md#mongodol.trans.PersistentObjectBase) Extension of a list wich triggers an event to notify the object that contains the list that a modification has been made. Requirement: The container object needs to implement the method “persist_data(self, data: Mapping)”. ```pycon >>> l = [1, 2, 3] >>> class Container: ... def persist_data(self, data): ... """Here, you'd normally put code to ACTUALLY persist the data""" ... print(f"persisting {data}") >>> pl = PersistentList(Container(), l) >>> assert pl == [1, 2, 3] # pl is equal to [1, 2, 3] >>> pl.append(4) persisting [1, 2, 3, 4] >>> assert pl == [1, 2, 3, 4] # indeed pl is now [1, 2, 3, 4] >>> pl.extend([5, 6]) persisting [1, 2, 3, 4, 5, 6] >>> assert pl == [1, 2, 3, 4, 5, 6] >>> pl += [7, 8] persisting [1, 2, 3, 4, 5, 6, 7, 8] >>> assert pl == [1, 2, 3, 4, 5, 6, 7, 8] >>> pl[0] = 9 persisting [9, 2, 3, 4, 5, 6, 7, 8] >>> assert pl == [9, 2, 3, 4, 5, 6, 7, 8] >>> n = pl.pop(0) persisting [2, 3, 4, 5, 6, 7, 8] >>> assert pl == [2, 3, 4, 5, 6, 7, 8] >>> pl.remove(2) persisting [3, 4, 5, 6, 7, 8] >>> assert pl == [3, 4, 5, 6, 7, 8] >>> del pl[0] persisting [4, 5, 6, 7, 8] >>> assert pl == [4, 5, 6, 7, 8] ``` #### append(\_PersistentList_\_object) Append `__object`, then persist the updated list. #### deepcopy() A deep copy of the original iterable this list was built from (not the persistent list itself). #### extend(\_PersistentList_\_iterable) Extend with `__iterable`, then persist the updated list. #### pop(\_PersistentList_\_index) Pop the item at `__index`, then persist the updated list. #### remove(\_PersistentList_\_value) Remove the first occurrence of `__value`, then persist the updated list. ### *class* mongodol.trans.PersistentObjectBase(container) Bases: [`ABC`](https://docs.python.org/3/library/abc.html#abc.ABC) Base class to propagate a modification event through a parent-child chain structure. #### persist_data(\*args) Notify the container that this object’s data has changed, by forwarding to its `persist_data`. ### *class* mongodol.trans.PostGet Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object) `postget` (key-aware) transform functions for `wrap_kvs`, turning a cursor into a value. #### *static* all_docs_fetch(k, cursor, doc_collector=) Collect every doc matching `k`, so `s[k]` is a collection of docs. The key-aware (`postget`) counterpart of [`ObjOfData.all_docs_fetch()`](_autosummary/mongodol.trans.html.md#mongodol.trans.ObjOfData.all_docs_fetch). `wrap_kvs` calls `obj_of_data` with the value alone and `postget` with `(key, value)`, so a store wired through `postget` needs this signature. #### *static* single_value_fetch_with_unicity_validation(store, k, cursor) Return the single doc in `cursor`; raise if there’s none or more than one. #### *static* single_value_fetch_without_unicity_validation(store, k, cursor) Return the first doc in `cursor`; raise only if there’s none (no uniqueness check). ### *class* mongodol.trans.WriteOpResult Bases: [`TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict) The shape of a normalized mongo write-operation result (see `normalize_result`). ### mongodol.trans.get_persistent_obj(container, v) Wrap `v` in a `PersistentDict`/`PersistentList` if it’s a mapping/iterable, else return it as is. ### mongodol.trans.normalize_result(obj, , method_names_to_normalize=('_\_setitem_\_', '_\_delitem_\_', 'append', 'extend', 'flush', 'commit')) Decorator to transform a pymongo result object to a WriteOpResult object. * **Parameters:** **func** ( *[*[*type*](https://docs.python.org/3/builtins/functions.html#type) *]*) – [description] ### mongodol.trans.set_key_and_data_fields(store=None, , key_fields=None, data_fields=None, \_\_module_\_=None, \_\_name_\_=None, \_\_qualname_\_=None, \_\_doc_\_=None, \_\_annotations_\_=None, \_\_defaults_\_=None, \_\_kwdefaults_\_=None) Decorator to set key_fields and data_fields on a store. This is to make it easier to get from an interface like this `store[{'folder': 'path', 'file': 'name'}] = {'field1': 'value1', 'field2': 'value2'}` to an interface like this: `store['path', 'name'] = ('value1', 'value2')` # _autosummary/mongodol.util.html.md # mongodol.util Util functions ### Functions | [`flatten_dict_items`](_autosummary/mongodol.util.html.md#mongodol.util.flatten_dict_items)(d[, prefix]) | Computes a "flat" dict from a nested one. | |-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [`get_key_value_specs`](_autosummary/mongodol.util.html.md#mongodol.util.get_key_value_specs)(key_fields, data_fields) | Derive `key_projection` (and, when `data_fields` is None or a non-dict iterable, `items_projection`) from `key_fields`/`data_fields`. | | [`get_mongo_collection_pymongo_obj`](_autosummary/mongodol.util.html.md#mongodol.util.get_mongo_collection_pymongo_obj)([obj, ...]) | Get a pymongo.collection.Collection object for a mongo collection, flexibly. | | [`mk_dflt_client`](_autosummary/mongodol.util.html.md#mongodol.util.mk_dflt_client)() | Make a `pymongo.MongoClient` with the default client args. | | [`mk_dflt_mgc`](_autosummary/mongodol.util.html.md#mongodol.util.mk_dflt_mgc)() | Make a default `pymongo.collection.Collection`, connecting with default client args to the default test database and collection. | | [`normalize_projection`](_autosummary/mongodol.util.html.md#mongodol.util.normalize_projection)(projection) | Normalize projection specification to be an explicit list of flattened dict of {path.to.key: True/False,. | | [`projection_union`](_autosummary/mongodol.util.html.md#mongodol.util.projection_union)(projection_1, projection_2) | Flatten and merge two mongo projection dicts, OR-ing every field against a forced default of `True` -- so a field appearing in only one of the two dicts (or with a `False` value) still comes out `True` unless both dicts agree it's `False`. | ### Exceptions | [`KeyNotUniqueError`](_autosummary/mongodol.util.html.md#mongodol.util.KeyNotUniqueError) | Raised when a key was expected to be unique, but wasn't (i.e. cursor has more than one match). | |----------------------------------------------------------------------|--------------------------------------------------------------------------------------------------| ### *exception* mongodol.util.KeyNotUniqueError Bases: [`RuntimeError`](https://docs.python.org/3/builtins/exceptions.html#RuntimeError) Raised when a key was expected to be unique, but wasn’t (i.e. cursor has more than one match) #### *static* raise_error(k) Raise `KeyNotUniqueError` for the non-unique key `k`. ### mongodol.util.flatten_dict_items(d, prefix='') Computes a “flat” dict from a nested one. A flat dict’s keys are the dot-paths of the input dict. * **Parameters:** * **d** ([`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping)) – a nested dict * **prefix** – A string to prepend on all the paths * **Returns:** A flat dict ```pycon >>> d = {'a': { ... 'a': '2a', ... 'c': {'a': 'aca', 'u': 4} ... }, ... 'c': 3 ... } >>> dict(flatten_dict_items(d)) {'a.a': '2a', 'a.c.a': 'aca', 'a.c.u': 4, 'c': 3} ``` ### mongodol.util.get_key_value_specs(key_fields, data_fields) Derive `key_projection` (and, when `data_fields` is None or a non-dict iterable, `items_projection`) from `key_fields`/`data_fields`. #### NOTE when `data_fields` is already a dict, `items_projection` is never assigned, so this branch raises `UnboundLocalError` on the `return` below. ### mongodol.util.get_mongo_collection_pymongo_obj(obj=None, client_factory=) Get a pymongo.collection.Collection object for a mongo collection, flexibly. ```text get_mongo_collection_pymongo_obj() # gives you a default mongo collection (mongodol/mongodol_test) get_mongo_collection_pymongo_obj('database_name/collection_name') # does the obvious (with default host) get_mongo_collection_pymongo_obj(... an object that has an _mgc attribute...) # return the _mgc attribute get_mongo_collection_pymongo_obj(obj) # else, asserts pymongo.collection.Collection and returns it ``` ```pycon >>> from mongodol.util import get_mongo_collection_pymongo_obj >>> c = get_mongo_collection_pymongo_obj() >>> c.name, c.database.name ('mongodol_test', 'mongodol') ``` An object with an `_mgc` attribute (such as a mongodol store) has that attribute returned directly: ```pycon >>> from mongodol.tests import util >>> mgc = util.populated_pymongo_collection([]) >>> store = type('Store', (), {'_mgc': mgc})() >>> get_mongo_collection_pymongo_obj(store) is mgc True ``` ### mongodol.util.mk_dflt_client() Make a `pymongo.MongoClient` with the default client args. ### mongodol.util.mk_dflt_mgc() Make a default `pymongo.collection.Collection`, connecting with default client args to the default test database and collection. ```pycon >>> from mongodol.util import mk_dflt_mgc >>> c = mk_dflt_mgc() >>> c.name, c.database.name ('mongodol_test', 'mongodol') ``` ### mongodol.util.normalize_projection(projection) Normalize projection specification to be an explicit list of flattened dict of {path.to.key: True/False,… (or None if projection is None to start with). This is used to be able to have a consistent specification of mongo projections. If projection is None, the output will None as well: ```pycon >>> assert normalize_projection(None) is None ``` If projection is a dict, the dict will be “flattened” to use “dot-paths” instead of nested dicts: ```pycon >>> normalize_projection({'name': {'first': True, 'last': False}, 'age': True}) {'name.first': True, 'name.last': False, 'age': True} ``` If projection is not a dict, it will make the “equivalent” dict version of the projection. One difference with mongodb’s projection language: Here, if you don’t specify that you want “_id”, it will explicitly specify that you DO NOT want that field (because mongodb will otherwise assume that you do!) ```pycon >>> normalize_projection(['name.first', 'age']) {'name.first': True, 'age': True, '_id': False} ``` But if you actually want that “_id”, just say so: ```pycon >>> normalize_projection(['name.first', 'age', '_id']) {'name.first': True, 'age': True, '_id': True} ``` Also, if you specify a string, it will think of this as a tuple containing just that string: ```pycon >>> normalize_projection('name.last') {'name.last': True, '_id': False} ``` ### mongodol.util.projection_union(projection_1, projection_2, already_flattened=False) Flatten and merge two mongo projection dicts, OR-ing every field against a forced default of `True` – so a field appearing in only one of the two dicts (or with a `False` value) still comes out `True` unless both dicts agree it’s `False`. ```pycon >>> d = {'a': { ... 'a': True, ... 'c': {'a': True, 'u': True} ... }, ... 'b': True, ... 'c': False ... } >>> dd = {'b': True, 'c': True, 'x': True, 'y': False} >>> assert projection_union(d, dd) == ( ... {'a.a': True, 'a.c.a': True, 'a.c.u': True, 'b': True, 'c': True, 'x': True, 'y': True} ... ) ``` # _autosummary/mongodol.utils.html.md # mongodol.utils Util modules ### Modules | [`werk_local`](_autosummary/mongodol.utils.werk_local.html.md#module-mongodol.utils.werk_local) | Vendored from werkzeug's local.py module, edited to our needs. | |------------------------------------------------------------------------------------------------|------------------------------------------------------------------| # _autosummary/mongodol.utils.werk_local.html.md # mongodol.utils.werk_local Vendored from werkzeug’s local.py module, edited to our needs. That single need is have a LocalProxy to subclass in making TrackedObj (see tracking_methods.py). ### Classes | [`LocalProxy`](_autosummary/mongodol.utils.werk_local.html.md#mongodol.utils.werk_local.LocalProxy)(local[, name]) | A proxy to the object bound to a `Local`. | |------------------------------------------------------------------------------|---------------------------------------------| ### *class* mongodol.utils.werk_local.LocalProxy(local, name=None) Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object) A proxy to the object bound to a `Local`. All operations on the proxy are forwarded to the bound object. If no object is bound, a [`RuntimeError`](https://docs.python.org/3/builtins/exceptions.html#RuntimeError) is raised. ```python from werkzeug.local import Local l = Local() # a proxy to whatever l.user is set to user = l("user") from werkzeug.local import LocalStack _request_stack = LocalStack() # a proxy to _request_stack.top request = _request_stack() # a proxy to the session attribute of the request proxy session = LocalProxy(lambda: request.session) ``` `__repr__` and `__class__` are forwarded, so `repr(x)` and `isinstance(x, cls)` will look like the proxied object. Use `issubclass(type(x), LocalProxy)` to check if an object is a proxy. ```python repr(user) # isinstance(user, User) # True issubclass(type(user), LocalProxy) # True ``` * **Parameters:** * **local** (`Union`[Local, [`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable)[[], [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)]]) – The `Local` or callable that provides the proxied object. * **name** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – The attribute name to look up on a `Local`. Not used if a callable is given. #### Versionchanged Changed in version 2.0: Updated proxied attributes and methods to reflect the current data model. #### Versionchanged Changed in version 0.6.1: The class can be instantiated with a callable. # _autosummary/mongodol.views.html.md # mongodol.views Mapping views that keep working when a mongo store is wrapped by `dol`. A mongo collection can serve a store’s whole `(key, value)` stream in a single `find` round trip, so [`MongoCollectionReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionReader) implements a **bulk-read protocol** – `iter_values`, `iter_items`, `contains_value` and `contains_item` – and exposes it through the `values()`/`items()` views defined here. One query instead of N is the whole point of these views. The catch is *composition*. A `dol` `Store` wrapper (what `wrap_kvs` builds) forwards every attribute it doesn’t define to the store it wraps. A view that simply calls `self._mapping.iter_values()` therefore punches straight through the wrappers and yields raw backend documents, silently skipping the value transforms the user asked for – breaking the `Mapping` contract: ```default list(store.values()) == [store[k] for k in store] ``` (see [i2mint/mongodol#7](https://github.com/i2mint/mongodol/issues/7)). This module resolves the bulk stream **explicitly** instead of relying on attribute delegation. Given the store a view was built on, [`bulk_values()`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_values) and [`bulk_items()`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_items) walk the wrapper chain inward, remembering each layer they cross, until they reach a store that actually implements the bulk-read protocol. The bulk stream is then re-transformed by the crossed layers, innermost first, so that it lands in exactly the same space as `store[k]`. A layer may only be crossed if its read path is *plain transform composition* – “read from the inner store, then apply `_key_of_id`/`_obj_of_data`”, which is what `Store` does. A layer that redefines `__getitem__` or `__iter__` (`wrap_kvs(postget=...)`, `filt_iter`, `cached_keys`, …) changes values or key sets in ways that cannot be pushed onto a bulk stream, so the resolver refuses to guess: it raises [`NoBulkReadPath`](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) and the views fall back to the generic per-key behaviour. That fallback is correct, just one round trip per key – correctness first, efficiency when it is provable. Simple use is invisible: build a mongo store, wrap it however you like, and `values()`/`items()` agree with `__getitem__`. The knobs, for store authors: - Implement the bulk-read methods to *provide* the fast path. - Set the [`BULK_READ_IS_FAITHFUL_ATTR`](_autosummary/mongodol.views.html.md#mongodol.views.BULK_READ_IS_FAITHFUL_ATTR) class attribute to `False` (see [`disable_bulk_read()`](_autosummary/mongodol.views.html.md#mongodol.views.disable_bulk_read)) when a class inherits bulk-read methods that no longer agree with its own `__getitem__`. Known limitation. [`MongoCollectionReader`](_autosummary/mongodol.base.html.md#mongodol.base.MongoCollectionReader) is deliberately a *cursor*-level store: `s[k]` is a pymongo `Cursor`, while its bulk stream already yields *documents* – one per key. The two only line up once a single-doc layer (`MongoCollectionFirstDocReader` and friends) has turned cursors into docs, which is why those are the stores you are meant to wrap. Hanging an `obj_of_data` that expects a cursor directly off the raw reader is outside the protocol: such a transform cannot be pushed onto a doc-level bulk stream, and is not detectable from here. Nothing here is mongo-specific; it is a general answer to “how does a store with a bulk-read fast path compose with `dol` wrappers?”, and would be a reasonable thing for `dol` itself to own one day. ### Module Attributes | [`ITER_VALUES_METHOD`](_autosummary/mongodol.views.html.md#mongodol.views.ITER_VALUES_METHOD) | Bulk-read method yielding a store's values in one backend round trip. | |-----------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------| | [`ITER_ITEMS_METHOD`](_autosummary/mongodol.views.html.md#mongodol.views.ITER_ITEMS_METHOD) | Bulk-read method yielding a store's `(key, value)` pairs in one backend round trip. | | [`CONTAINS_VALUE_METHOD`](_autosummary/mongodol.views.html.md#mongodol.views.CONTAINS_VALUE_METHOD) | Bulk-read method answering "is this value in the store?" in one backend round trip. | | [`CONTAINS_ITEM_METHOD`](_autosummary/mongodol.views.html.md#mongodol.views.CONTAINS_ITEM_METHOD) | Bulk-read method answering "is this item in the store?" in one backend round trip. | | [`BULK_READ_IS_FAITHFUL_ATTR`](_autosummary/mongodol.views.html.md#mongodol.views.BULK_READ_IS_FAITHFUL_ATTR) | Class attribute through which a store declares whether its bulk-read methods are value-equivalent to its own `__getitem__`. | | [`INNER_STORE_ATTR`](_autosummary/mongodol.views.html.md#mongodol.views.INNER_STORE_ATTR) | The `dol` `Store` attribute holding the store a wrapper wraps. | ### Functions | [`bulk_contains_item`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_contains_item)(store, item) | Ask the backend whether `item` is one of `store`'s items, in one round trip. | |------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| | [`bulk_contains_value`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_contains_value)(store, v) | Ask the backend whether `v` is one of `store`'s values, in one round trip. | | [`bulk_items`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_items)(store) | Iterate `store`'s `(key, value)` pairs via the backend's bulk-read path. | | [`bulk_values`](_autosummary/mongodol.views.html.md#mongodol.views.bulk_values)(store) | Iterate `store`'s values via the backend's bulk-read path, transforms honoured. | | [`disable_bulk_read`](_autosummary/mongodol.views.html.md#mongodol.views.disable_bulk_read)(store_cls) | Class decorator declaring that inherited bulk-read methods are not to be trusted. | | [`is_crossable`](_autosummary/mongodol.views.html.md#mongodol.views.is_crossable)(store) | Whether `store` is a `Store` layer whose read path is plain transform composition. | | [`provides_bulk_read`](_autosummary/mongodol.views.html.md#mongodol.views.provides_bulk_read)(store, method_name) | Whether `store`'s own class implements bulk-read `method_name`, faithfully. | | [`resolve_bulk_source`](_autosummary/mongodol.views.html.md#mongodol.views.resolve_bulk_source)(store, method_name) | Find the store providing bulk-read `method_name`, and the layers crossed to reach it. | | [`store_layers`](_autosummary/mongodol.views.html.md#mongodol.views.store_layers)(store) | Yield `store` then each store it wraps, outermost first, innermost last. | ### Classes | [`MongoItemsView`](_autosummary/mongodol.views.html.md#mongodol.views.MongoItemsView)(mapping) | An `items()` view that uses the backend's bulk read when -- and only when -- that stream provably equals `((k, store[k]) for k in store)`. | |----------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------| | [`MongoValuesView`](_autosummary/mongodol.views.html.md#mongodol.views.MongoValuesView)(mapping) | A `values()` view that uses the backend's bulk read when -- and only when -- that stream provably equals `(store[k] for k in store)`. | ### Exceptions | [`NoBulkReadPath`](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) | No bulk-read stream can be *proven* equivalent to the store's per-key reads. | |-------------------------------------------------------------------|--------------------------------------------------------------------------------| ### mongodol.views.BULK_READ_IS_FAITHFUL_ATTR *= '_bulk_read_is_faithful'* Class attribute through which a store declares whether its bulk-read methods are value-equivalent to its own `__getitem__`. It defaults to `True` (a class that implements the protocol is trusted to implement it faithfully). It exists because `dol`’s class-decorator wrapping *copies* the wrapped class’s extra methods onto the wrapper, so a wrapper that redefines value semantics – `wrap_kvs(postget=...)` – silently inherits bulk-read methods that no longer match it. Such a class sets this to `False`; see [`disable_bulk_read()`](_autosummary/mongodol.views.html.md#mongodol.views.disable_bulk_read). ### mongodol.views.CONTAINS_ITEM_METHOD *= 'contains_item'* Bulk-read method answering “is this item in the store?” in one backend round trip. ### mongodol.views.CONTAINS_VALUE_METHOD *= 'contains_value'* Bulk-read method answering “is this value in the store?” in one backend round trip. ### mongodol.views.INNER_STORE_ATTR *= 'store'* The `dol` `Store` attribute holding the store a wrapper wraps. ### mongodol.views.ITER_ITEMS_METHOD *= 'iter_items'* Bulk-read method yielding a store’s `(key, value)` pairs in one backend round trip. ### mongodol.views.ITER_VALUES_METHOD *= 'iter_values'* Bulk-read method yielding a store’s values in one backend round trip. ### *class* mongodol.views.MongoItemsView(mapping) Bases: [`ItemsView`](https://docs.python.org/3/library/collections.abc.html#collections.abc.ItemsView) An `items()` view that uses the backend’s bulk read when – and only when – that stream provably equals `((k, store[k]) for k in store)`. ### *class* mongodol.views.MongoValuesView(mapping) Bases: [`ValuesView`](https://docs.python.org/3/library/collections.abc.html#collections.abc.ValuesView) A `values()` view that uses the backend’s bulk read when – and only when – that stream provably equals `(store[k] for k in store)`. ### *exception* mongodol.views.NoBulkReadPath Bases: [`Exception`](https://docs.python.org/3/builtins/exceptions.html#Exception) No bulk-read stream can be *proven* equivalent to the store’s per-key reads. Raised by the resolvers of this module, and caught by the views, which then fall back to the generic (correct, one-round-trip-per-key) `Mapping` behaviour. It is a control-flow signal, not a user-facing error. ### mongodol.views.bulk_contains_item(store, item) Ask the backend whether `item` is one of `store`’s items, in one round trip. * **Raises:** [**NoBulkReadPath**](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) – when `item` cannot be pushed down to backend space. * **Return type:** [`bool`](https://docs.python.org/3/builtins/functions.html#bool) ### mongodol.views.bulk_contains_value(store, v) Ask the backend whether `v` is one of `store`’s values, in one round trip. * **Raises:** [**NoBulkReadPath**](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) – when `v` cannot be pushed down to backend space. * **Return type:** [`bool`](https://docs.python.org/3/builtins/functions.html#bool) ### mongodol.views.bulk_items(store) Iterate `store`’s `(key, value)` pairs via the backend’s bulk-read path. * **Raises:** [**NoBulkReadPath**](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) – when the bulk stream cannot be proven equivalent to `((k, store[k]) for k in store)`. * **Return type:** [`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator)[[`Tuple`](https://docs.python.org/3/library/typing.html#typing.Tuple)[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)]] ### mongodol.views.bulk_values(store) Iterate `store`’s values via the backend’s bulk-read path, transforms honoured. * **Raises:** [**NoBulkReadPath**](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) – when the bulk stream cannot be proven equivalent to `(store[k] for k in store)`. * **Return type:** [`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator) ### mongodol.views.disable_bulk_read(store_cls) Class decorator declaring that inherited bulk-read methods are not to be trusted. Use it on a class that changes what `__getitem__` returns (typically via `wrap_kvs(postget=...)`) while inheriting – or being handed, by `dol`’s class-decorator wrapping – bulk-read methods written for the *un*-changed semantics. Views then take the correct per-key path instead. * **Return type:** [`type`](https://docs.python.org/3/builtins/functions.html#type) ### mongodol.views.is_crossable(store) Whether `store` is a `Store` layer whose read path is plain transform composition. Such a layer reads from the store it wraps and applies `_key_of_id` to keys and `_obj_of_data` to values – and nothing else. Those two transforms can be mapped over a bulk stream, so the layer can be “crossed” on the way to the backend’s fast path. A layer that redefines `__getitem__` (`postget`) or `__iter__` (key filtering/caching) cannot. * **Return type:** [`bool`](https://docs.python.org/3/builtins/functions.html#bool) ### mongodol.views.provides_bulk_read(store, method_name) Whether `store`’s own class implements bulk-read `method_name`, faithfully. “Faithfully” means the store has not declared, via [`BULK_READ_IS_FAITHFUL_ATTR`](_autosummary/mongodol.views.html.md#mongodol.views.BULK_READ_IS_FAITHFUL_ATTR), that its bulk-read methods disagree with its `__getitem__`. * **Return type:** [`bool`](https://docs.python.org/3/builtins/functions.html#bool) ### mongodol.views.resolve_bulk_source(store, method_name) Find the store providing bulk-read `method_name`, and the layers crossed to reach it. * **Returns:** `(source, layers)` where `layers` are the crossed `Store` wrappers, outermost first. * **Raises:** [**NoBulkReadPath**](_autosummary/mongodol.views.html.md#mongodol.views.NoBulkReadPath) – if a layer that cannot be crossed is met before a provider is found. ### mongodol.views.store_layers(store) Yield `store` then each store it wraps, outermost first, innermost last. The chain ends at the first non-`Store` – the actual backend. Note that `dol` is free to insert pass-through `Store` layers of its own, so never assume one `wrap_kvs` call means exactly one layer. * **Return type:** [`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator) ```pycon >>> from dol import wrap_kvs >>> layers = list(store_layers(wrap_kvs({'a': 1}, obj_of_data=str))) >>> type(layers[0]).__name__, type(layers[-1]).__name__ ('Store', 'dict') >>> all(isinstance(x, Store) for x in layers[:-1]) True ``` # about-this-build.html.md # About this build This documentation was built on **2026-09-22 16:30 UTC** from commit 21cec5b on branch master, for **mongodol 0.1.7** (from pyproject.toml). #### WARNING The documentation and the package may be misaligned: - The documented version (0.1.7) is behind the latest release on PyPI (0.1.8): `pip install mongodol` gives newer code than these docs describe. ## Source | | | |---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------| | Commit | 21cec5ba681f37d29a47e7679554bac1a4db8168 | | Branch | master | | Tags at this commit | none | | Working tree | clean | | Remote | https://github.com/i2mint/mongodol | ## Continuous integration | | | |--------------|--------------------------------------------------------------------------------------------| | Repository | i2mint/mongodol | | Run | 35754287660 | | Ref | refs/heads/master | | Event commit | 21cec5ba681f37d29a47e7679554bac1a4db8168 (in the history of the built commit) | ## Tools | | | |----------|---------| | epythet | 0.2.12 | | Sphinx | 9.1.0 | | docutils | 0.22.4 | | Python | 3.12.14 | ## Configuration as resolved | | | |---------------|-------------------------------------------------------------------| | theme | auto (Sphinx theme pydata_sphinx_theme) | | accent | #8f3254 | | api_generator | autosummary | | ignore | tests/, scrap/, examples/ | | agent_outputs | true | | aggregates | md | | ai_artifacts | true | ## Package on PyPI Latest release: 0.1.8, newer than the documented version (0.1.7). ## Reproduce ```bash git clone https://github.com/i2mint/mongodol && cd mongodol git checkout 21cec5ba681f37d29a47e7679554bac1a4db8168 pip install "epythet==0.2.12" epythet quickstart . --ignore tests/ scrap/ examples/ ``` The same data, for machines: build_info.json (schema version 1). # api.html.md # API reference | [`mongodol`](_autosummary/mongodol.html.md#module-mongodol) | Access mongo through a Mapping interface | |-----------------------------------------------------------------------------|--------------------------------------------|