front.dag

Tools to dispatch dags

See below one of the dags that will often be used in this module’s doctests:

>>> from meshed.makers import code_to_dag
>>> @code_to_dag
... def dag():
...     x = foo(a, b)
...     y = bar(x, greeting)
...     z = confuser(a, w=x)  # note the w=x to test non-trivial binding
>>> print(dag.dot_digraph_ascii())
class front.dag.VarNodeRole(value)[source]

(Var)Node roles.

When a VarNode is used to source the arguments of a FuncNode, it’s playing a VarNodeRole.argument role.

When a VarNode is used to store the return value of a FuncNode, it’s playing a VarNodeRole.return_value role.

Most VarNode``s play both roles during a ``DAG computation.

front.dag.crudify_func_nodes(var_nodes: Union[str, Iterable[str]], dag: meshed.dag.DAG, var_node_name_to_store_name=functools.partial(<function simple_namer>, suffix='_store'), *, mall: Optional[Mapping[str, Mapping[str, Any]]] = None, include_stores_attribute: bool = False, save_name_param: str = 'save_name')[source]

Crudifies the given var_nodes in the dag.

Crudifying a var node means crudifying it’s FuncNode neighbors, i.e. telling the function that outputs to the VarNode (if any) to save it’s output in a store and (additionally) return the key it saved it too instead of the value itself, and telling any consumers of the var node to use that key as it’s argument instead, retrieving the value from said store.

>>> from meshed import DAG, FuncNode
>>> from inspect import signature
>>> def foo(a, b):  return a + b
>>> def bar(x, y):  return  x * y
>>> dag = DAG([
...     FuncNode(foo, name='foo', out='foo_output'),
...     FuncNode(bar, bind={'y': 'foo_output'})
... ])

Let’s crudify 'foo_output'. We don’t need to specify a mall, since crudify_func_nodes will make one for us. But in order to get access to it, to see what the function is doing, let’s define a mall with a single store (a dictionary), named 'foo_output_store' (note that the map between var_node string name and store name is controlled by the var_node_name_to_store_name argument)

>>> store = dict()
>>> mall = {'foo_output_store': store}
>>> new_dag = crudify_func_nodes(['foo_output'], dag, mall=mall)

The new_dag will have the same global behavior:

>>> assert dag(2, 3, 4) == new_dag(2, 3, 4) == 20

Notice though, that the foo node will have an extra argument, save_name, which is the name of the store to save the output to:

>>> print(new_dag.synopsis_string())
a,b,save_name -> foo -> foo_output
foo_output,x -> bar_ -> bar

This difference will be reflected in the signature of the new_dag:

>>> print(str(signature(dag)))
(a, b, x)
>>> print(str(signature(new_dag)))
(a, b, x, save_name: str = '')

Let’s have a closer look at the functions that dag and new_dag are using. The functions of the dag are the original functions we specified, behaving normally:

>>> dag.func_nodes[0].func(2, 3)
5
>>> dag.func_nodes[1].func(4, 5)
20

But the first function of new_dag outputs 'bar_last_output' instead of 5.

>>> new_dag.func_nodes[0].func(2, 3)
'bar_last_output'

Where did the 5 go? In the mall!

>>> mall
{'foo_output_store': {'bar_last_output': 5}}

So that 5 has been stored under the 'bar_last_output' key. Further, the second function’s second argument will no longer work with numbers, but with string keys, and use that same store to retrieve the value it needs for the underlying function:

>>> new_dag.func_nodes[1].func(4, 'bar_last_output')
20

This 'bar_last_output' was only the default value that is used if save_name is not given. If we give it a different name, the value will be stored under that name instead:

>>> new_dag.func_nodes[0].func(20, 22, save_name='my_save_name')
'my_save_name'
>>> mall
{'foo_output_store': {'bar_last_output': 5, 'my_save_name': 42}}
Parameters

  • var_nodes – The VarNodes we want to crudify

  • dag – The dag that contains these var_nodes

  • var_node_name_to_store_name – The function to use to make a store for a given var_node name. If you have an explicit mapping m for this, just use m.get

  • mall – A mall (store of stores, i.e. mapping of mappings) whose keys are store names, and values are the actual stores.

  • include_stores_attribute – Whether the crudified functions should have an attribute containing a pointer to the stores involved

  • save_name_param – The name that the “save as” parameter should appear as.

Returns