i2.footprints

Analyzing what attributes of an input object a function actually uses

Functions

accessed_attributes(func[, object_name])

Extracts the attributes accessed by a function or method.

assigned_attributes(func[, object_name])

The attributes of an object that func assigns to (or mutates through).

assignment_dependencies(func[, object_name])

The attribute-to-attribute dependencies created by assignments inside func.

attr_list(root, func_name)

Extracts attributes from ast tree processing only func_name function or method

attribute_assignment_dependencies(cls[, ...])

Extracts (method_name, {assigned_attribute: attributes_read}) pairs for a class.

attribute_dependencies(cls[, filt, ...])

Extracts (method_name, accessed_attributes) pairs for a class or instance thereof.

attrs_used_by_method(method, *[, src_code])

Extracts a list of cls attributes which are used by a method or method_name function

attrs_used_by_method_computation(cls_method)

Tracks the access to attributes within an execution.

cls_and_method_name_of_method(method)

The (class, name) pair of a method, bound method or property.

dict_to_graph(graph[, from_key_to_values, ...])

Convert a {node: neighbours} dictionary to a graphviz (or mermaid) graph string.

dunders_diff(x, y)

The set of dunder names x has and y does not (module names are imported).

ensure_ast(o[, src_code])

Casts input object o to a AST node.

get_class_that_defined_method(method)

Get class for unbound/bound method.

get_imports_from_obj(o[, recursive])

Getting imports for an object (usually, module)

get_source(obj)

Get source string of a python object

init_argument_names(cls, *[, no_error_action])

Get the list of argument names

list_func_calls(fn)

Extracts functions and methods called from fn

module_if_string(x)

Import x if it is a module name string; otherwise return it as is.

object_dependencies(obj, *[, get_source])

Map each method of a class (or of an instance's class) to the attributes it reads.

start_tracking(tracker_instance)

Ctx manager to gracefully start/stop tracking.

trace_class_decorator(cls[, names_and_sigs])

Add tracing methods to cls, each appending (name, *args) to the instance's .trace.

Classes

AssignedAttributeVisitor(object_name)

Collect, in .assigned_attributes, (assigned, read) pairs of attribute names.

AttributeVisitor(object_name)

Collect, in .attributes, the attribute names accessed on object_name in an AST.

Import(module, name, alias)

MethodTrace()

Record the operator dunders applied to an instance, as (name, *args) tuples in .trace.

Tracker(*args, **kwargs)

Tracks the attribute access right after start_track is set to True.

class i2.footprints.AssignedAttributeVisitor(object_name)[source]

Bases: NodeVisitor

Collect, in .assigned_attributes, (assigned, read) pairs of attribute names.

For each assignment to an attribute of object_name (self.y = ..., also self.y += ..., self.y: int = ..., tuple targets, and mutation through the attribute: self.y.z = ... or self.y[k] = ..., which count as assigning y), assigned is the list of attributes assigned and read the list of attributes of the same object the assignment reads (the value, plus the target’s own subscripts; for an augmented assignment, the target too). Both lists keep first-seen order.

>>> src_code = '''
... class MyClass:
...     x = 1
...     z = 3
...     def my_method(self):
...         self.y = self.x + 1
...         return self.z + self.y
... '''
>>> visitor = AssignedAttributeVisitor('self')
>>> visitor.visit(ast.parse(dedent(src_code)))
>>> visitor.assigned_attributes
[(['y'], ['x'])]
class i2.footprints.AttributeVisitor(object_name)[source]

Bases: NodeVisitor

Collect, in .attributes, the attribute names accessed on object_name in an AST.

class i2.footprints.Import(module, name, alias)

Bases: tuple

alias

Alias for field number 2

module

Alias for field number 0

name

Alias for field number 1

class i2.footprints.MethodTrace[source]

Bases: object

Record the operator dunders applied to an instance, as (name, *args) tuples in .trace.

See: https://github.com/i2mint/i2/issues/56 for more details.

>>> t = MethodTrace()
>>> ((t + 3) - 2) * 5 / 10
<MethodTrace with .trace = ('__add__', 3), ... ('__truediv__', 10)>
>>> assert t.trace == [
...     ('__add__', 3), ('__sub__', 2), ('__mul__', 5), ('__truediv__', 10)
... ]
>>>
>>>
>>> w = t[42]
>>> t[42] = 'mol'  # an operation with two arguments
>>> # ... and now an operation with no arguments:
>>> ~t
<MethodTrace with .trace = ... ('__setitem__', 42, 'mol'), ('__invert__',)>
>>>
>>> assert t.trace == [
... ('__add__', 3), ('__sub__', 2), ('__mul__', 5), ('__truediv__', 10),
... ('__getitem__', 42), ('__setitem__', 42, 'mol'), ('__invert__',)
... ]
>>>
class i2.footprints.Tracker(*args, **kwargs)[source]

Bases: object

Tracks the attribute access right after start_track is set to True.

Add this to __metaclass__ for any class that you need to track attributes for given a target method.

on_access(key)[source]

on attribute access, record attribute if and only if its not from core attribute or attrs_to_ignore set to class.

i2.footprints.accessed_attributes(func, object_name=None)[source]

Extracts the attributes accessed by a function or method.

(This is a simpler, but narrow, version of attrs_used_by_method).

>>> def func(a, b, c):
...     return a + b.bar + c
...
>>> # Commenting out the testing code, as execution is not performed in the PCI
>>> def foo(self):
...     a = 2
...     self.method(x)
...     y = self.prop
...     return a + func(x, self, y)
...
>>> assert accessed_attributes(foo, 'self') == {'method', 'prop'}
i2.footprints.assigned_attributes(func, object_name=None)[source]

The attributes of an object that func assigns to (or mutates through).

>>> src = '''
... def foo(self):
...     self.y = self.x + 1
...     self.items[self.k] = 2
...     return self.z
... '''
>>> sorted(assigned_attributes(src, 'self'))
['items', 'y']
i2.footprints.assignment_dependencies(func, object_name=None)[source]

The attribute-to-attribute dependencies created by assignments inside func.

Returns {assigned_attribute: attributes_it_was_computed_from}, merging assignments to the same attribute. Mutating code is what makes these edges worth seeing in a dependency graph: a method that sets self.y from self.x makes y depend on x.

func is a function or method, or its source code:

>>> src = '''
... def foo(self):
...     self.y = self.x + 1
...     self.total += self.y
...     return self.z
... '''
>>> assignment_dependencies(src, 'self')
{'y': ['x'], 'total': ['total', 'y']}
i2.footprints.attr_list(root, func_name)[source]

Extracts attributes from ast tree processing only func_name function or method

Parameters:
  • root – root node of ast tree

  • func_name – name of the function

Returns:

a list of attributes names

i2.footprints.attribute_assignment_dependencies(cls, filt=<function _is_method_like>, *, name_of_obj=<function name_of_obj>, exclude_names=<function init_argument_names>)[source]

Extracts (method_name, {assigned_attribute: attributes_read}) pairs for a class.

The companion of attribute_dependencies(): where that one says which attributes a method uses, this one says which attributes it assigns and what each is computed from. Methods that assign nothing are left out.

(See i2/tests/footprints_test.py for an example; a doctest cannot define a class whose source inspect can read.)

i2.footprints.attribute_dependencies(cls, filt=<function _is_method_like>, *, name_of_obj=<function name_of_obj>, exclude_names=<function init_argument_names>)[source]

Extracts (method_name, accessed_attributes) pairs for a class or instance thereof.

Parameters:
  • cls (type) – The class or instance to analyze

  • filt – A function that filters the attributes to consider

  • name_of_obj – A function that returns the name of an object

  • exclude_names (Union[Container, Callable]) – A list of names to exclude from the analysis or a function that returns such a list given the class

Returns:

A generator of (method_name, accessed_attributes) pairs

i2.footprints.attrs_used_by_method(method, *, src_code=None)[source]

Extracts a list of cls attributes which are used by a method or method_name function

Note

The function tries to analyzed the source code deeply, gathering indirect references to the instance attributes. As a result, it is not very robust. You may want to check out the simpler (but narrow) function: accessed_attributes.

Parameters:
  • method – The method (object) to analyze

  • src_code – The source code in which the method’s class is defined, when inspect cannot retrieve it (for example in a notebook).

Returns:

A list of attribute names (of the class or instance thereof) that are accessed in the code of the said method.

Consider the method A.target_method coming from the following code in i2.tests.footprints_test:

def func(obj):
    """Accesses attributes 'a' and 'b' of obj"""
    return obj.a + obj.b

class A:
    e = 2

    def __init__(self, a=1, b=0, c=1, d=10):
        self.a = a
        self.b = b
        self.c = c
        self.d = d

    def target_method(self, x):
        """Accesses ['a', 'b', 'c', 'e']"""
        t = func(self)  # and this function will access some attributes!
        tt = self.other_method(t)
        return x * tt / self.e

    def other_method(self, x=1):
        """Accesses ['c', 'e']"""
        w = self.c * 2  # c is accessed first
        return self.e + self.c * x - w  # and c is accessed again

    @classmethod
    def a_class_method(cls, y):
        """Accesses ['e']"""
        return cls.e + y
i2.footprints.attrs_used_by_method_computation(cls_method, init_kwargs=None, method_kwargs=None, remove_duplicates=True)[source]

Tracks the access to attributes within an execution.

i2.footprints.cls_and_method_name_of_method(method)[source]

The (class, name) pair of a method, bound method or property.

>>> from i2.tests.footprints_test import A
>>> cls_and_method_name_of_method(A().target_method) == (A, "target_method")
True
i2.footprints.dict_to_graph(graph, from_key_to_values=True, *, kind='graphviz', indent='    ', prefix='', suffix='', display=False)[source]

Convert a {node: neighbours} dictionary to a graphviz (or mermaid) graph string.

You provide a graph in the form of a {from_node: to_nodes, ...} or {to_node: from_nodes, ...} dictionary, and will get a graphviz string. You can use this to visualize a graph (e.g. a dependency graph) in a jupyter notebook.

Parameters:
  • graph (dict) – The graph, in the form of a to convert to graphviz.

  • from_key_to_values (bool) – Whether the keys of the graph are from nodes or to nodes.

  • kind (Literal['graphviz', 'mermaid']) – The kind of graphviz string to return. Either “graphviz” or “mermaid”.

  • indent (str) – The indent to use for the graphviz string.

  • graphviz_template – The template to use for the graphviz string.

  • display (bool | Callable) – Whether to display the graphviz string as a graph in a jupyter notebook. Requires graphviz.

Return type:

str

Returns:

The graphviz string.

Example (but bear in mind the order of the nodes in graphviz_str may be different):

>>> graph_dict = {
...     "A": ["B", "C"],
...     "B": ["D"],  # note that "D" is not mentioned as a key
...     "C": ["D", "E", "F"],
...     "E": [],
... }
>>> # Keys are from nodes
>>> graphviz_str = dict_to_graph(graph_dict)
>>> print(graphviz_str)
digraph G {
    "A" -> "B";
    "A" -> "C";
    "B" -> "D";
    "C" -> "D";
    "C" -> "E";
    "C" -> "F";
}
>>> # Keys are to nodes
>>> graphviz_str = dict_to_graph(graph_dict, from_key_to_values=False)
>>> print(graphviz_str)
digraph G {
    "B" -> "A";
    "C" -> "A";
    "D" -> "B";
    "D" -> "C";
    "E" -> "C";
}

The default kind is graphviz, but you can also use mermaid:

>>> graphviz_str = dict_to_graph(graph_dict, kind="mermaid")
>>> print(graphviz_str)
graph TD
    A --> B;
    A --> C;
    B --> D;
    C --> D;
    C --> E;
    C --> F;
i2.footprints.dunders_diff(x, y)[source]

The set of dunder names x has and y does not (module names are imported).

i2.footprints.ensure_ast(o, src_code=None)[source]

Casts input object o to a AST node.

If the input is an AST node, it is returned as is. If the input is a filepath, the file is read and parsed as source code. If the input is a string, it is parsed as source code. If the input is an object, the source code is extracted and parsed.

Let’s get an AST for the ensure_ast function itself:

>>> assert isinstance(ensure_ast(ensure_ast), ast.AST)

Note that sometimes the source code of an object cannot be accessed via normal means (for example, in REPLs like jupyter notebooks) so we need to pass it in.

>>> src_code = '''
... class MyClass:
...     x = 1
...     def my_method(self):
...         return self.x + 1
... a = 10
... '''
>>> assert isinstance(ensure_ast('MyClass', src_code), ast.AST)
Return type:

AST

i2.footprints.get_class_that_defined_method(method)[source]

Get class for unbound/bound method.

i2.footprints.get_imports_from_obj(o, recursive=False)[source]

Getting imports for an object (usually, module)

i2.footprints.get_source(obj)[source]

Get source string of a python object

Return type:

str

i2.footprints.init_argument_names(cls, *, no_error_action=None)[source]

Get the list of argument names

>>> from dataclasses import dataclass
>>> @dataclass
... class DataClass:
...     x: str
...     y: float = 2
...     z = 3
...
>>> init_argument_names(DataClass)
['x', 'y']

Note

Some builtin types don’t have signatures, so we get:

ValueError: no signature found for builtin type ...

By default, we handle this by returning an empty list, but a callable no_error_action will call that function and return its result. Anything else will result in raising the error.

Return type:

list[str]

i2.footprints.list_func_calls(fn)[source]

Extracts functions and methods called from fn

Parameters:

fn – reference to function or method

Returns:

a list of functions or methods names

i2.footprints.module_if_string(x)[source]

Import x if it is a module name string; otherwise return it as is.

i2.footprints.object_dependencies(obj, *, get_source=<function get_source>)[source]

Map each method of a class (or of an instance’s class) to the attributes it reads.

Attributes accessed through the method’s first argument (usually self) count; attributes that are only assigned to do not. Members for which get_source raises TypeError (builtins, descriptors) are skipped.

>>> class C:
...     def __init__(self):
...         self.a = 1
...     def m(self):
...         return self.a + self.helper()
...     def helper(self):
...         return 2
>>> object_dependencies(C)["m"] == {"a", "helper"}
True
i2.footprints.start_tracking(tracker_instance)[source]

Ctx manager to gracefully start/stop tracking.

i2.footprints.trace_class_decorator(cls, names_and_sigs=(('__concat__', <Sig (self, b, /)>), ('__not__', <Sig (self, /)>), ('__iadd__', <Sig (self, b, /)>), ('__ifloordiv__', <Sig (self, b, /)>), ('__iand__', <Sig (self, b, /)>), ('__delitem__', <Sig (self, key: KT, /) -> Any>), ('__imatmul__', <Sig (self, b, /)>), ('__iconcat__', <Sig (self, b, /)>), ('__sub__', <Sig (self, b, /)>), ('__eq__', <Sig (self, b, /)>), ('__lshift__', <Sig (self, b, /)>), ('__and__', <Sig (self, b, /)>), ('__neg__', <Sig (self, /)>), ('__ipow__', <Sig (self, b, /)>), ('__ilshift__', <Sig (self, b, /)>), ('__le__', <Sig (self, b, /)>), ('__setitem__', <Sig (self, key: KT, value: VT, /) -> Any>), ('__imul__', <Sig (self, b, /)>), ('__isub__', <Sig (self, b, /)>), ('__floordiv__', <Sig (self, b, /)>), ('__inv__', <Sig (self, /)>), ('__matmul__', <Sig (self, b, /)>), ('__ior__', <Sig (self, value, /)>), ('__gt__', <Sig (self, b, /)>), ('__mul__', <Sig (self, b, /)>), ('__or__', <Sig (self, value, /)>), ('__pos__', <Sig (self, /)>), ('__rshift__', <Sig (self, b, /)>), ('__getitem__', <Sig (self, key: KT, /) -> ~VT>), ('__ixor__', <Sig (self, b, /)>), ('__imod__', <Sig (self, b, /)>), ('__contains__', <Sig (self, key: KT, /) -> bool>), ('__itruediv__', <Sig (self, b, /)>), ('__lt__', <Sig (self, b, /)>), ('__mod__', <Sig (self, b, /)>), ('__truediv__', <Sig (self, b, /)>), ('__add__', <Sig (self, b, /)>), ('__ge__', <Sig (self, b, /)>), ('__ne__', <Sig (self, b, /)>), ('__abs__', <Sig (self, /)>), ('__index__', <Sig (self, /)>), ('__call__', <Sig (self, /, *args, **kwargs)>), ('__invert__', <Sig (self, /)>), ('__irshift__', <Sig (self, b, /)>), ('__xor__', <Sig (self, b, /)>), ('__pow__', <Sig (self, b, /)>), ('__reversed__', <Sig (self, /)>), ('__iter__', <Sig (self, /) -> collections.abc.Iterator[~KT]>), ('__len__', <Sig (self, /) -> int>), ('__ror__', <Sig (self, value, /)>), ('__rrshift__, ', <Sig (self, other)>), ('__rfloordiv__, ', <Sig (self, other)>), ('__rpow__, ', <Sig (self, other)>), ('__rtruediv__, ', <Sig (self, other)>), ('__rdiv__, ', <Sig (self, other)>), ('__rand__, ', <Sig (self, other)>), ('__radd__, ', <Sig (self, other)>), ('__rsub__, ', <Sig (self, other)>), ('__rmul__, ', <Sig (self, other)>), ('__rlshift__, ', <Sig (self, other)>), ('__rxor__, ', <Sig (self, other)>), ('__rdivmod__, ', <Sig (self, other)>), ('__rmod__, ', <Sig (self, other)>)), method_factory=<function _dflt_method_factory>)[source]

Add tracing methods to cls, each appending (name, *args) to the instance’s .trace.

By default the methods are the operator, dict and reflected-operator dunders, made by method_factory(name, sig); each returns the instance so calls can be chained.