i2.footprints

Analyzing what attributes of an input object a function actually uses

Functions

accessed_attributes(func[, object_name])	Extracts the attributes accessed by a function or method.
attr_list(root, func_name)	Extracts attributes from ast tree processing only func_name function or method
attribute_dependencies(cls[, filt, ...])	Extracts (method_name, accessed_attributes) pairs for a class or instance thereof.
attrs_used_by_method(method, *[, src_code])	Extracts a list of cls attributes which are used by a method or method_name function
attrs_used_by_method_computation(cls_method)	Tracks the access to attributes within an execution.
cls_and_method_name_of_method(method)	The (class, name) pair of a method, bound method or property.
dict_to_graph(graph[, from_key_to_values, ...])	Convert a {node: neighbours} dictionary to a graphviz (or mermaid) graph string.
dunders_diff(x, y)	The set of dunder names x has and y does not (module names are imported).
ensure_ast(o[, src_code])	Casts input object o to a AST node.
get_class_that_defined_method(method)	Get class for unbound/bound method.
get_imports_from_obj(o[, recursive])	Getting imports for an object (usually, module)
get_source(obj)	Get source string of a python object
init_argument_names(cls, *[, no_error_action])	Get the list of argument names
list_func_calls(fn)	Extracts functions and methods called from fn
module_if_string(x)	Import x if it is a module name string; otherwise return it as is.
object_dependencies(obj, *[, get_source])	Map each method of a class (or of an instance's class) to the attributes it reads.
start_tracking(tracker_instance)	Ctx manager to gracefully start/stop tracking.
trace_class_decorator(cls[, names_and_sigs])	Add tracing methods to cls, each appending (name, *args) to the instance's .trace.

Classes

AttributeVisitor(object_name)	Collect, in .attributes, the attribute names accessed on object_name in an AST.
Import(module, name, alias)
MethodTrace()	Record the operator dunders applied to an instance, as (name, *args) tuples in .trace.
Tracker(*args, **kwargs)	Tracks the attribute access right after start_track is set to True.

class i2.footprints.AttributeVisitor(object_name)

Bases: NodeVisitor

Collect, in .attributes, the attribute names accessed on object_name in an AST.

class i2.footprints.Import(module, name, alias)

Bases: tuple

alias

Alias for field number 2

module

Alias for field number 0

name

Alias for field number 1

class i2.footprints.MethodTrace

Bases: object

Record the operator dunders applied to an instance, as (name, *args) tuples in .trace.

See: https://github.com/i2mint/i2/issues/56 for more details.

>>> t = MethodTrace()
>>> ((t + 3) - 2) * 5 / 10
<MethodTrace with .trace = ('__add__', 3), ... ('__truediv__', 10)>
>>> assert t.trace == [
...     ('__add__', 3), ('__sub__', 2), ('__mul__', 5), ('__truediv__', 10)
... ]
>>>
>>>
>>> w = t[42]
>>> t[42] = 'mol'  # an operation with two arguments
>>> # ... and now an operation with no arguments:
>>> ~t
<MethodTrace with .trace = ... ('__setitem__', 42, 'mol'), ('__invert__',)>
>>>
>>> assert t.trace == [
... ('__add__', 3), ('__sub__', 2), ('__mul__', 5), ('__truediv__', 10),
... ('__getitem__', 42), ('__setitem__', 42, 'mol'), ('__invert__',)
... ]
>>>

class i2.footprints.Tracker(*args, **kwargs)

Bases: object

Tracks the attribute access right after start_track is set to True.

Add this to __metaclass__ for any class that you need to track attributes for given a target method.

on_access(key)

on attribute access, record attribute if and only if its not from core attribute or attrs_to_ignore set to class.

i2.footprints.accessed_attributes(func, object_name=None)

Extracts the attributes accessed by a function or method.

(This is a simpler, but narrow, version of attrs_used_by_method).

>>> def func(a, b, c):
...     return a + b.bar + c
...
>>> # Commenting out the testing code, as execution is not performed in the PCI
>>> def foo(self):
...     a = 2
...     self.method(x)
...     y = self.prop
...     return a + func(x, self, y)
...
>>> assert accessed_attributes(foo, 'self') == {'method', 'prop'}

i2.footprints.attr_list(root, func_name)

Extracts attributes from ast tree processing only func_name function or method

Parameters:

• root – root node of ast tree

• func_name – name of the function

Returns:

a list of attributes names

i2.footprints.attribute_dependencies(cls, filt=<function _is_method_like>, *, name_of_obj=<function name_of_obj>, exclude_names=<function init_argument_names>)

Extracts (method_name, accessed_attributes) pairs for a class or instance thereof.

Parameters:

• cls (type) – The class or instance to analyze

• filt – A function that filters the attributes to consider

• name_of_obj – A function that returns the name of an object

• exclude_names (Union[Container, Callable]) – A list of names to exclude from the analysis or a function that returns such a list given the class

Returns:

A generator of (method_name, accessed_attributes) pairs

i2.footprints.attrs_used_by_method(method, *, src_code=None)

Extracts a list of cls attributes which are used by a method or method_name function

Note

The function tries to analyzed the source code deeply, gathering indirect references to the instance attributes. As a result, it is not very robust. You may want to check out the simpler (but narrow) function: accessed_attributes.

Parameters:

• method – The method (object) to analyze

• src_code – The source code in which the method’s class is defined, when inspect cannot retrieve it (for example in a notebook).

Returns:

A list of attribute names (of the class or instance thereof) that are accessed in the code of the said method.

Consider the method A.target_method coming from the following code in i2.tests.footprints_test:

def func(obj):
    """Accesses attributes 'a' and 'b' of obj"""
    return obj.a + obj.b

class A:
    e = 2

    def __init__(self, a=1, b=0, c=1, d=10):
        self.a = a
        self.b = b
        self.c = c
        self.d = d

    def target_method(self, x):
        """Accesses ['a', 'b', 'c', 'e']"""
        t = func(self)  # and this function will access some attributes!
        tt = self.other_method(t)
        return x * tt / self.e

    def other_method(self, x=1):
        """Accesses ['c', 'e']"""
        w = self.c * 2  # c is accessed first
        return self.e + self.c * x - w  # and c is accessed again

    @classmethod
    def a_class_method(cls, y):
        """Accesses ['e']"""
        return cls.e + y

i2.footprints.attrs_used_by_method_computation(cls_method, init_kwargs=None, method_kwargs=None, remove_duplicates=True)

Tracks the access to attributes within an execution.

i2.footprints.cls_and_method_name_of_method(method)

The (class, name) pair of a method, bound method or property.

>>> from i2.tests.footprints_test import A
>>> cls_and_method_name_of_method(A().target_method) == (A, "target_method")
True

i2.footprints.dict_to_graph(graph, from_key_to_values=True, *, kind='graphviz', indent='    ', prefix='', suffix='', display=False)

Convert a {node: neighbours} dictionary to a graphviz (or mermaid) graph string.

You provide a graph in the form of a {from_node: to_nodes, ...} or {to_node: from_nodes, ...} dictionary, and will get a graphviz string. You can use this to visualize a graph (e.g. a dependency graph) in a jupyter notebook.

Parameters:

• graph (dict) – The graph, in the form of a to convert to graphviz.

• from_key_to_values (bool) – Whether the keys of the graph are from nodes or to nodes.

• kind (Literal['graphviz', 'mermaid']) – The kind of graphviz string to return. Either “graphviz” or “mermaid”.

• indent (str) – The indent to use for the graphviz string.

• graphviz_template – The template to use for the graphviz string.

• display (bool | Callable) – Whether to display the graphviz string as a graph in a jupyter notebook. Requires graphviz.

Return type:

str

Returns:

The graphviz string.

Example (but bear in mind the order of the nodes in graphviz_str may be different):

>>> graph_dict = {
...     "A": ["B", "C"],
...     "B": ["D"],  # note that "D" is not mentioned as a key
...     "C": ["D", "E", "F"],
...     "E": [],
... }
>>> # Keys are from nodes
>>> graphviz_str = dict_to_graph(graph_dict)
>>> print(graphviz_str)
digraph G {
    "A" -> "B";
    "A" -> "C";
    "B" -> "D";
    "C" -> "D";
    "C" -> "E";
    "C" -> "F";
}

>>> # Keys are to nodes
>>> graphviz_str = dict_to_graph(graph_dict, from_key_to_values=False)
>>> print(graphviz_str)
digraph G {
    "B" -> "A";
    "C" -> "A";
    "D" -> "B";
    "D" -> "C";
    "E" -> "C";
}

The default kind is graphviz, but you can also use mermaid:

>>> graphviz_str = dict_to_graph(graph_dict, kind="mermaid")
>>> print(graphviz_str)
graph TD
    A --> B;
    A --> C;
    B --> D;
    C --> D;
    C --> E;
    C --> F;

i2.footprints.dunders_diff(x, y)

The set of dunder names x has and y does not (module names are imported).

i2.footprints.ensure_ast(o, src_code=None)

Casts input object o to a AST node.

If the input is an AST node, it is returned as is. If the input is a filepath, the file is read and parsed as source code. If the input is a string, it is parsed as source code. If the input is an object, the source code is extracted and parsed.

Let’s get an AST for the ensure_ast function itself:

Return type:

AST

>>> assert isinstance(ensure_ast(ensure_ast), ast.AST)

Note that sometimes the source code of an object cannot be accessed via normal means (for example, in REPLs like jupyter notebooks) so we need to pass it in.

>>> src_code = '''
... class MyClass:
...     x = 1
...     def my_method(self):
...         return self.x + 1
... a = 10
... '''
>>> assert isinstance(ensure_ast('MyClass', src_code), ast.AST)

i2.footprints.get_class_that_defined_method(method)

Get class for unbound/bound method.

i2.footprints.get_imports_from_obj(o, recursive=False)

Getting imports for an object (usually, module)

i2.footprints.get_source(obj)

Get source string of a python object

Return type:

str

i2.footprints.init_argument_names(cls, *, no_error_action=None)

Get the list of argument names

Return type:

list[str]

>>> from dataclasses import dataclass
>>> @dataclass
... class DataClass:
...     x: str
...     y: float = 2
...     z = 3
...
>>> init_argument_names(DataClass)
['x', 'y']

Note

Some builtin types don’t have signatures, so we get:

ValueError: no signature found for builtin type ...

By default, we handle this by returning an empty list, but a callable no_error_action will call that function and return its result. Anything else will result in raising the error.

i2.footprints.list_func_calls(fn)

Extracts functions and methods called from fn

Parameters:

fn – reference to function or method

Returns:

a list of functions or methods names

i2.footprints.module_if_string(x)

Import x if it is a module name string; otherwise return it as is.

i2.footprints.object_dependencies(obj, *, get_source=<function get_source>)

Map each method of a class (or of an instance’s class) to the attributes it reads.

Attributes accessed through the method’s first argument (usually self) count; attributes that are only assigned to do not. Members for which get_source raises TypeError (builtins, descriptors) are skipped.

>>> class C:
...     def __init__(self):
...         self.a = 1
...     def m(self):
...         return self.a + self.helper()
...     def helper(self):
...         return 2
>>> object_dependencies(C)["m"] == {"a", "helper"}
True

i2.footprints.start_tracking(tracker_instance)

Ctx manager to gracefully start/stop tracking.

i2.footprints.trace_class_decorator(cls, names_and_sigs=(('__floordiv__', <Sig (self, b, /)>), ('__ior__', <Sig (self, value, /)>), ('__imod__', <Sig (self, b, /)>), ('__inv__', <Sig (self, /)>), ('__ilshift__', <Sig (self, b, /)>), ('__ne__', <Sig (self, b, /)>), ('__eq__', <Sig (self, b, /)>), ('__invert__', <Sig (self, /)>), ('__contains__', <Sig (self, key: KT, /) -> bool>), ('__iconcat__', <Sig (self, b, /)>), ('__le__', <Sig (self, b, /)>), ('__ge__', <Sig (self, b, /)>), ('__matmul__', <Sig (self, b, /)>), ('__abs__', <Sig (self, /)>), ('__delitem__', <Sig (self, key: KT, /) -> Any>), ('__not__', <Sig (self, /)>), ('__add__', <Sig (self, b, /)>), ('__and__', <Sig (self, b, /)>), ('__lshift__', <Sig (self, b, /)>), ('__pow__', <Sig (self, b, /)>), ('__lt__', <Sig (self, b, /)>), ('__ixor__', <Sig (self, b, /)>), ('__index__', <Sig (self, /)>), ('__mod__', <Sig (self, b, /)>), ('__imul__', <Sig (self, b, /)>), ('__call__', <Sig (self, /, *args, **kwargs)>), ('__sub__', <Sig (self, b, /)>), ('__ipow__', <Sig (self, b, /)>), ('__iand__', <Sig (self, b, /)>), ('__isub__', <Sig (self, b, /)>), ('__xor__', <Sig (self, b, /)>), ('__iadd__', <Sig (self, b, /)>), ('__pos__', <Sig (self, /)>), ('__imatmul__', <Sig (self, b, /)>), ('__or__', <Sig (self, value, /)>), ('__truediv__', <Sig (self, b, /)>), ('__gt__', <Sig (self, b, /)>), ('__setitem__', <Sig (self, key: KT, value: VT, /) -> Any>), ('__getitem__', <Sig (self, key: KT, /) -> ~VT>), ('__itruediv__', <Sig (self, b, /)>), ('__neg__', <Sig (self, /)>), ('__concat__', <Sig (self, b, /)>), ('__rshift__', <Sig (self, b, /)>), ('__irshift__', <Sig (self, b, /)>), ('__ifloordiv__', <Sig (self, b, /)>), ('__mul__', <Sig (self, b, /)>), ('__ror__', <Sig (self, value, /)>), ('__reversed__', <Sig (self, /)>), ('__len__', <Sig (self, /) -> int>), ('__iter__', <Sig (self, /) -> collections.abc.Iterator[~KT]>), ('__rsub__, ', <Sig (self, other)>), ('__rmul__, ', <Sig (self, other)>), ('__radd__, ', <Sig (self, other)>), ('__rmod__, ', <Sig (self, other)>), ('__rdivmod__, ', <Sig (self, other)>), ('__rtruediv__, ', <Sig (self, other)>), ('__rdiv__, ', <Sig (self, other)>), ('__rpow__, ', <Sig (self, other)>), ('__rrshift__, ', <Sig (self, other)>), ('__rand__, ', <Sig (self, other)>), ('__rfloordiv__, ', <Sig (self, other)>), ('__rxor__, ', <Sig (self, other)>), ('__rlshift__, ', <Sig (self, other)>)), method_factory=<function _dflt_method_factory>)

Add tracing methods to cls, each appending (name, *args) to the instance’s .trace.

By default the methods are the operator, dict and reflected-operator dunders, made by method_factory(name, sig); each returns the instance so calls can be chained.