i2.key_path

Flattening maps and manipulating key paths

Functions

flatten_dict(d[, sep, prefix])	Computes a "flat" dict from a nested one.
obj_to_str_path(obj, *[, sep, name_of_obj, ...])	Get the dotpath reference for an object
rollout_dict(d[, sep, prefix])	Get the nested path of a flat (key path) dict.
str_path_to_obj(str_path, *[, sep])	Loads and returns the object referenced by the string DOTPATH_TO_MODULE.OBJ_NAME
trans_generator_output(trans)	Make a decorator that applies trans to every item a generator function yields.

Classes

KeyPathMap([store, key_type, node_type, ...])	Provides a key-path view to a nested mapping (by default, a dict).
KeyPathTrans([sep, node_type, mk_new_node])	Doing what StrKeyPath but where the store that is being operated on is not included in the object, but given to the method as input.
NoDefault()	Type of the NO_DFLT sentinel (no default value given).
StrKeyPath([store, key_type, node_type, ...])	A KeyPathMap, but where the key paths are expressed as string with a separator.

class i2.key_path.KeyPathMap(store=<class 'dict'>, key_type=None, node_type=None, auto_node_writes=False)

Bases: MutableMapping

Provides a key-path view to a nested mapping (by default, a dict). A nested mapping can be see as a tree, where if a value is itself a mapping, it is a non-terminal node, leaves (or terminal) holding the “actual values”.

When wrapping a mapping in KeyPathMap, you can pretend that you have a flat mapping from (root to leaf) paths instead of a nested structure, and do your mapping CRUD with that view.

>>> d = {'a': {
...         'a': '2a',
...         'b': {'a': 'aba',
...               'b': 3}
...         },
...      'c': 3.14
...     }
>>> kp = KeyPathMap(d)
>>> list(kp.items())
[(('a', 'a'), '2a'), (('a', 'b', 'a'), 'aba'), (('a', 'b', 'b'), 3), (('c',), 3.14)]
>>> list(kp)
[('a', 'a'), ('a', 'b', 'a'), ('a', 'b', 'b'), ('c',)]
>>> len(kp)
4
>>> assert list(kp) == list(kp.keys())
>>> list(kp.values())
['2a', 'aba', 3, 3.14]
>>> kp['a']
{'a': '2a', 'b': {'a': 'aba', 'b': 3}}
>>> kp[('a',)]
{'a': '2a', 'b': {'a': 'aba', 'b': 3}}
>>> kp['a', 'a']
'2a'
>>> kp['a', 'b', 'b']
3
>>> ('a', 'new_key') in kp
False
>>> kp['a', 'new_key'] = 'new val'
>>> ('a', 'new_key') in kp
True
>>> kp['a', 'new_key']
'new val'
>>> len(kp)
5
>>> del kp['a', 'b', 'a']
>>> len(kp)
4
>>> list(kp.items())
[(('a', 'a'), '2a'), (('a', 'b', 'b'), 3), (('a', 'new_key'), 'new val'), (('c',), 3.14)]
>>>
>>> # By default, you can only write on already created nodes. But if auto_node_writes=True, you can do this:
>>> kp = KeyPathMap(auto_node_writes=True)
>>> kp
{}
>>> kp['a', 'b', 'c'] = 'hi world!'
>>> kp
{'a': {'b': {'c': 'hi world!'}}}

items() → a set-like object providing a view on D's items

class i2.key_path.KeyPathTrans(sep='.', node_type=<class 'dict'>, mk_new_node=None)

Bases: object

Doing what StrKeyPath but where the store that is being operated on is not included in the object, but given to the method as input.

extract_key_paths(d, key_paths, field_naming='full', use_default=False, default_val=None)

getting with a key list or “.”-separated string

Parameters:

• d – dict-like

• key_path – list or “.”-separated string of keys

• field_naming – ‘full’ (default) will use key_path strings as is, leaf will only use the last dot item (i.e. this.is.a.key.path will result in “path” being used)

Returns:

>>> kp = KeyPathTrans()
>>> d = {
...     'a': {
...         'a': 'a.a',
...         'b': 'a.b',
...         'c': {
...             'a': 'a.c.a'
...         }
...     },
...     'b': 'b',
...     'c': 3
... }
>>> kp.extract_key_paths(d, 'a')
{'a': {'a': 'a.a', 'b': 'a.b', 'c': {'a': 'a.c.a'}}}
>>> kp.extract_key_paths(d, 'a.a')
{'a.a': 'a.a'}
>>> kp.extract_key_paths(d, 'a.c')
{'a.c': {'a': 'a.c.a'}}
>>> kp.extract_key_paths(d, ['a.a', 'a.c'])
{'a.a': 'a.a', 'a.c': {'a': 'a.c.a'}}
>>> kp.extract_key_paths(d, ['a.a', 'something.thats.not.there'])  # missing key just won't be included
{'a.a': 'a.a'}
>>> kp.extract_key_paths(d, ['a.a', 'something.thats.not.there'], use_default=True, default_val=42)
{'a.a': 'a.a', 'something.thats.not.there': 42}

getitem(d, key_path, default_val=None)

getting with a key list or “.”-separated string

Parameters:

• d – dict-like

• key_path – list or “.”-separated string of keys

Returns:

items(d, key_path_prefix=None)

iterate through items of store recursively, yielding (key_path, val) pairs for all nested values that are not store types. That is, if a value is a store_type, it won’t generate a yield, but rather, will be iterated through recursively.

Parameters:

• d – input store

• key_path_so_far – string to be prepended to all key paths (for use in recursion, not meant for direct use)

Returns:

a (key_path, val) iterator

>>> kp = KeyPathTrans()
>>> input_dict = {
...     'a': {
...         'a': 'a.a',
...         'b': 'a.b',
...         'c': {
...             'a': 'a.c.a'
...         }
...     },
...     'b': 'b',
...     'c': 3
... }
>>> list(kp.items(input_dict))
[('a.a', 'a.a'), ('a.b', 'a.b'), ('a.c.a', 'a.c.a'), ('b', 'b'), ('c', 3)]

setitem(d, key_path, val)

setting with a key list or “.”-separated string

Parameters:

• d – dict

• key_path – list or “.”-separated string of keys

• val – value to assign

Returns:

setitem_recursive(d, key_path, val)

Parameters:

• d

• key_path

• val

Returns:

>>> kp = KeyPathTrans()
>>> input_dict = {
...   "a": {
...     "c": "val of a.c",
...     "b": 1,
...   },
...   "10": 10,
...   "b": {
...     "B": {
...       "AA": 3
...     }
...   }
... }
>>>
>>> kp.setitem_recursive(input_dict, 'new.key.path', 7)
>>> input_dict
{'a': {'c': 'val of a.c', 'b': 1}, '10': 10, 'b': {'B': {'AA': 3}}, 'new': {'key': {'path': 7}}}
>>> kp.setitem_recursive(input_dict, 'new.key.old.path', 8)
>>> input_dict
{'a': {'c': 'val of a.c', 'b': 1}, '10': 10, 'b': {'B': {'AA': 3}}, 'new': {'key': {'path': 7, 'old': {'path': 8}}}}
>>> kp.setitem_recursive(input_dict, 'new.key', 'new val')
>>> input_dict
{'a': {'c': 'val of a.c', 'b': 1}, '10': 10, 'b': {'B': {'AA': 3}}, 'new': {'key': 'new val'}}

class i2.key_path.NoDefault

Bases: object

Type of the NO_DFLT sentinel (no default value given).

class i2.key_path.StrKeyPath(store=<class 'dict'>, key_type=None, node_type=None, auto_node_writes=False, sep='.', prefix='')

Bases: KeyPathMap

A KeyPathMap, but where the key paths are expressed as string with a separator. If sep = ‘.’, then instead of using (‘a’, ‘b’, ‘c’) as a key, you can use ‘a.b.c’.

>>> d = {'a': {
...         'a': '2a',
...         'b': {'a': 'aba',
...               'b': 3}
...         },
...      'c': 3.14
...     }
>>> # Example with sep='/'
>>> kp = StrKeyPath(d, sep='/')
>>> list(kp.items())
[('a/a', '2a'), ('a/b/a', 'aba'), ('a/b/b', 3), ('c', 3.14)]
>>> # You can also add a prefix to the keys
>>> kp = StrKeyPath(d, sep='/', prefix="http://")
>>> list(kp.items())
[('http://a/a', '2a'), ('http://a/b/a', 'aba'), ('http://a/b/b', 3), ('http://c', 3.14)]
>>>
>>> # Default sep is '.', so we'll work with that:
>>> kp = StrKeyPath(d)
>>> kp
{'a': {'a': '2a', 'b': {'a': 'aba', 'b': 3}}, 'c': 3.14}
>>> list(kp.items())
[('a.a', '2a'), ('a.b.a', 'aba'), ('a.b.b', 3), ('c', 3.14)]
>>> list(kp)
['a.a', 'a.b.a', 'a.b.b', 'c']
>>> len(kp)
4
>>> assert list(kp) == list(kp.keys())
>>> list(kp.values())
['2a', 'aba', 3, 3.14]
>>> kp['a']
{'a': '2a', 'b': {'a': 'aba', 'b': 3}}
>>> kp['a.a']
'2a'
>>> kp['a.b.b']
3
>>> ('a.new_key') in kp
False
>>> kp['a.new_key'] = 'new val'
>>> 'a.new_key' in kp
True
>>> kp['a.new_key']
'new val'
>>> len(kp)
5
>>> del kp['a.b.a']
>>> len(kp)
4
>>> list(kp.items())
[('a.a', '2a'), ('a.b.b', 3), ('a.new_key', 'new val'), ('c', 3.14)]
>>>
>>> # By default, you can only write on already created nodes. But if auto_node_writes=True, you can do this:
>>> kp = StrKeyPath(auto_node_writes=True)
>>> kp
{}
>>> kp['a.b.c'] = 'hi world!'
>>> kp
{'a': {'b': {'c': 'hi world!'}}}
>>>

items() → a set-like object providing a view on D's items

i2.key_path.flatten_dict(d, sep=None, prefix='')

Computes a “flat” dict from a nested one. A flat dict’s keys are the paths of the input dict. These paths will be expressed as tuples of the original keys by defaults. If these keys are strings though, you can use sep and prefix to get string representations of the paths.

Parameters:

• d – a nested dict

• sep – The separator character (or string) in a string representation of the paths.

• prefix – A string to prepend on all the paths

Returns:

A flat dict

>>> d = {'a': {
...         'a': '2a',
...         'c': {'a': 'aca', 'u': 4}
...         },
...      'c': 3
...     }
>>> flatten_dict(d)
{('a', 'a'): '2a', ('a', 'c', 'a'): 'aca', ('a', 'c', 'u'): 4, ('c',): 3}
>>> flatten_dict(d, sep='.')
{'a.a': '2a', 'a.c.a': 'aca', 'a.c.u': 4, 'c': 3}
>>> flatten_dict(d, sep='/', prefix='/ROOT/')
{'/ROOT/a/a': '2a', '/ROOT/a/c/a': 'aca', '/ROOT/a/c/u': 4, '/ROOT/c': 3}

i2.key_path.obj_to_str_path(obj, *, sep='.', name_of_obj=operator.attrgetter('__qualname__'), path_of_module=operator.attrgetter('__module__'))

Get the dotpath reference for an object

>>> from inspect import Signature
>>> obj_to_str_path(Signature.replace)
'inspect.Signature.replace'

obj_to_str_path is the inverse of str_path_to_obj

>>> assert str_path_to_obj(obj_to_str_path(Signature.replace)) == Signature.replace

Let’s try with a different separator.

>>> path = obj_to_str_path(Signature.replace, sep='/')
>>> path
'inspect/Signature.replace'

Remember to specify the same sep when you do the inverse!

>>> assert str_path_to_obj(path, sep='/') == Signature.replace

You can also pass in your own name_of_obj and path_of_module functions. For example you want a more permissive version of name_of_obj you may consider i2.name_of_obj. Note, thought, that str_path_to_obj might not work as an inverse for custom name_of_obj and path_of_module functions. You may have to write your own inverse function in this case.

i2.key_path.rollout_dict(d, sep=None, prefix='')

Get the nested path of a flat (key path) dict. This is the inverse of flatten_dict.

Parameters:

• d – a flat dict (i.e. one whose keys are paths of a nested dict)

• sep – If None (default), the paths should be key tuples. If a string, it it assumed to be the separator of string representations of the path

• prefix – A string that has be prepended to all each key (path) of the input dict (and therefore should be removed)

Returns:

The corresponding nested path

>>> flat_d = {('a', 'a'): '2a', ('a', 'c', 'a'): 'aca', ('a', 'c', 'u'): 4, ('c',): 3}
>>> rollout_dict(flat_d)
{'a': {'a': '2a', 'c': {'a': 'aca', 'u': 4}}, 'c': 3}
>>> flat_d = {'a.a': '2a', 'a.c.a': 'aca', 'a.c.u': 4, 'c': 3}
>>> rollout_dict(flat_d, sep='.')
{'a': {'a': '2a', 'c': {'a': 'aca', 'u': 4}}, 'c': 3}
>>> flat_d = {'/ROOT/a/a': '2a', '/ROOT/a/c/a': 'aca', '/ROOT/a/c/u': 4, '/ROOT/c': 3}
>>> rollout_dict(flat_d, sep='/', prefix='/ROOT/')
{'a': {'a': '2a', 'c': {'a': 'aca', 'u': 4}}, 'c': 3}

i2.key_path.str_path_to_obj(str_path, *, sep='.')

Loads and returns the object referenced by the string DOTPATH_TO_MODULE.OBJ_NAME

i2.key_path.trans_generator_output(trans)

Make a decorator that applies trans to every item a generator function yields.