# meshed.slabs

Tools to generate slabs.

A slab is a dict that holds data generated by a stream for a given interval of time.

The main object of this module is `Slabs`, and object that defines how to
generate multiple streams, in the form of slabs.
More precisely, it defines how to source streams, operate on and combine these to create
further streams, and even push these streams to further processes, all through a single
simple interface: An (ordered) list of components that are called in sequence to either
pull data from some sources, compute a new stream based on previous ones, or push some
of the streams to further processes (such as visualization, or storage systems).

Main entry points:

- `Slabs`: the stream-of-slabs object; iterate it, or `run()` it for side effects.
- `IteratorExit`: raise it from a component to stop the iteration cleanly.
- `conditional_sentinel`: decorator returning a sentinel instead of calling the function
  when a condition on its arguments holds (`output_none_if_none_arguments` is one).

A slab is a collection of items of a same interval of time.
We represent a slab using a `dict` or mapping.
Typically, a slab will be the aggregation of multiple information streams that
happened around the same time.

`Slabs` is a tool that allows you to source multiple streams into a stream of
slabs that can contain the original data, or other datas computed from it, or both.

Note to developers, though the code below is a reduced form of the actual code,
it should be enough to understand the general idea.
For a discussion about the design of Slabs, see
[https://github.com/i2mint/meshed/discussions/49](https://github.com/i2mint/meshed/discussions/49).

```pycon
>>> class Slabs:
...     def _call_on_scope(self, scope):
...         '''
...         Calls the components 1 by 1, sourcing inputs and writing outputs in scope
...         '''
...
...     def __next__(self):
...         '''Get the next slab by calling _call_on_scope on an new empty scope.
...         At least one of the components will have to be argument-less and provide
...         some data for other components to get their inputs from, if any are needed.
...         '''
...         return self._call_on_scope(scope={})
...
...     def __iter__(self):
...         '''Iterates over slabs until a handle exception is raised.'''
...         # Simplified code:
...         with self:  # enter all the contexts that need to be entered
...             while True:  # loop until you encounter a handled exception
...                 try:
...                     yield next(self)
...                 except self.handle_exceptions as exc_val:
...                     # use specific exceptions to signal that iteration should stop
...                     break
```

### Functions

| [`all_arguments_are_none`](#meshed.slabs.all_arguments_are_none)(args, kwargs)             | Return True if all arguments are None.                               |
|---------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
| [`conditional_sentinel`](#meshed.slabs.conditional_sentinel)(condition_func[, sentinel]) | Decorator that returns sentinel based on a user-defined condition.   |
| [`do_nothing`](#meshed.slabs.do_nothing)()                                     | Argument-less no-op, the default handler of handled exceptions.      |
| [`log_and_return`](#meshed.slabs.log_and_return)(msg[, logger])                    | Pass `msg` to `logger` (`print` by default) and return it unchanged. |
| [`output_none_if_none_arguments`](#meshed.slabs.output_none_if_none_arguments)(func)              | Decorator that returns None if all arguments are None.               |

### Classes

| `DoNotBreak`()                                                                             |                                                                                                                    |
|--------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
| [`ExceptionHandler`](#meshed.slabs.ExceptionHandler)(\*args, \*\*kwargs)      | An exception handler is an argument-less callable that is called when a handled exception occurs during iteration. |
| [`Slabs`](#meshed.slabs.Slabs)([handle_exceptions, scope_factory]) | Object to source and manipulate multiple streams.                                                                  |
| [`SlabsIter`](#meshed.slabs.SlabsIter)                                 |                                                                                                                    |

### Exceptions

| [`ExceptionalException`](#meshed.slabs.ExceptionalException)   | Raised when an exception was supposed to be handled, but no matching handler was found.                                 |
|-------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| [`IteratorExit`](#meshed.slabs.IteratorExit)           | Raised when an iterator should quit being iterated on, signaling this event any process that cares to catch the signal. |

### *class* meshed.slabs.ExceptionHandler(\*args, \*\*kwargs)

Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)

An exception handler is an argument-less callable that is called when a handled
exception occurs during iteration. Most often, the handler does nothing,
but could be used whose output will be ignored, unless it is do_not_break,
which will signal that the iteration should continue.

### *exception* meshed.slabs.ExceptionalException

Bases: [`Exception`](https://docs.python.org/3/builtins/exceptions.html#Exception)

Raised when an exception was supposed to be handled, but no matching handler
was found.

See the `_handle_exception` function, where it is raised.

### *exception* meshed.slabs.IteratorExit

Bases: [`BaseException`](https://docs.python.org/3/builtins/exceptions.html#BaseException)

Raised when an iterator should quit being iterated on, signaling this event
any process that cares to catch the signal.
We chose to inherit directly from `BaseException` instead of `Exception`
for the same reason that `GeneratorExit` does: Because it’s not technically
an error.

See: [https://docs.python.org/3/library/exceptions.html#GeneratorExit](https://docs.python.org/3/library/exceptions.html#GeneratorExit)

### *class* meshed.slabs.Slabs(handle_exceptions=(<class 'StopIteration'>, <class 'meshed.slabs.IteratorExit'>, <class 'KeyboardInterrupt'>), scope_factory=<class 'dict'>, \*\*components)

Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)

Object to source and manipulate multiple streams.

A slab is a collection of items of a same interval of time.
We represent a slab using a `dict` or mapping.
Typically, a slab will be the aggregation of multiple information streams that
happened around the same time.

For example, say and edge device had a microphone, light, and movement sensor.
An aggregate reading of these sensors could give you something like:

```pycon
>>> slab = {'audio': [1, 2, 4], 'light': 126, 'movement': None}
```

`movement` is `None` because the sensor is off. If it were on, we’d have True or
False as values.

From this information, you’d like to compute a `turn_mov_on` value based on the
formula.

```pycon
>>> from statistics import stdev
>>> vol = stdev
>>> should_turn_movement_sensor_on = lambda audio, light: vol(audio) * light > 50000
```

The produce of the volume and the lumens gives you 192, so you now have…

```pycon
>>> slab = {
...     'audio': [1, 2, 4],
...     'light': 126,
...     'should_turn_movement_sensor_on': False,
...     'movement': None
... }
```

The next slab that comes in is

```pycon
>>> slab = {'audio': [-96, 89, -92], 'light': 501, 'movement': None}
```

which puts us over the threshold so

```pycon
>>> slab = {
...     'audio': [-96, 89, -92],
...     'light': 501,
...     'should_turn_movement_sensor_on': True,
...     'movement': None
... }
```

and the movement sensor is turned on, the movement is detected, a `human_presence`
signal is computed, and a notification sent if that metric is above a given theshold.

The point here is that we incrementally compute various fields, enhancing our slab
of information, and we do so iteratively over over slab that is streaming to us
from our smart home device.

`SlabsIter` is there to help you create such slabs, from source to enhanced.

The situation above would look something along like this:

```pycon
>>> from statistics import stdev
>>>
>>> vol = stdev
>>>
>>> # Making a slabs iter object
>>> def make_a_slabs_iter():
...
...     # Mocking the sensor readers
...     audio_sensor_read = iter([[1, 2, 3], [-96, 87, -92], [320, -96, 99]]).__next__
...     light_sensor_read = iter([126, 501, 523]).__next__
...     movement_sensor_read = iter([None, None, True]).__next__
...
...     return Slabs(
...         # The first three components get data from the sensors.
...         # The *_read objects are all callable, returning the next
...         # chunk of data for that sensor, if any.
...         audio=audio_sensor_read,
...         light=light_sensor_read,
...         movement=movement_sensor_read,
...         # The next
...         should_turn_movement_sensor_on = lambda audio, light: vol(audio) * light > 50000,
...         human_presence_score = lambda audio, light, movement: movement and sum([vol(audio), light]),
...         should_notify = lambda human_presence_score: human_presence_score and human_presence_score > 700,
...         notify = lambda should_notify: print('someone is there') if should_notify else None
...     )
...
>>>
>>> si = make_a_slabs_iter()
>>> next(si)
{'audio': [1, 2, 3],
 'light': 126,
 'movement': None,
 'should_turn_movement_sensor_on': False,
 'human_presence_score': None,
 'should_notify': None,
 'notify': None}
>>> next(si)
{'audio': [-96, 87, -92],
 'light': 501,
 'movement': None,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': None,
 'should_notify': None,
 'notify': None}
>>> next(si)
someone is there
{'audio': [320, -96, 99],
 'light': 523,
 'movement': True,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': 731.1353726143957,
 'should_notify': True,
 'notify': None}
```

If you ask for the next slab, you’ll get a `StopIteration` (raised by the mocked
sources since they reached the end of their iterators).

```pycon
>>> next(si)
Traceback (most recent call last):
  ...
StopIteration
```

That said, if you iterate through a `SlabsIter` that handles the `StopIteration`
exception (it does by default), you’ll reach the end of you iteration gracefully.

```pycon
>>> si = make_a_slabs_iter()
>>> for slab in si:
...     pass
someone is there
>>> si = make_a_slabs_iter()
>>> slabs = list(si)  # gather all the slabs
someone is there
>>> len(slabs)
3
>>> slabs[-1]
{'audio': [320, -96, 99],
 'light': 523,
 'movement': True,
 'should_turn_movement_sensor_on': True,
 'human_presence_score': 731.1353726143957,
 'should_notify': True,
 'notify': None}
```

Note that `Slabs` uses a “scope” to store the intermediate results of the
computation. This scope is a `dict` by default, but you can pass any
`MutableMapping` to the `scope_factory` argument. This means that you can use
other means to store intermediate results simply by wrapping them in a
MutableMapping. For example, you could use message broker such as Redis to
store the intermediate results, and have the components read and write to it.

To help you with this, check out the [dol](https://pypi.org/project/dol/)
and [py2store](https://pypi.org/project/py2store/) libraries.

#### close(exc_type=None, exc_val=None, exc_tb=None)

Exit the component contexts entered by `open`.

* **Return type:**
  [`None`](https://docs.python.org/3/builtins/constants.html#None)

#### dot_digraph(\*args, \*\*kwargs)

Returns a dot_digraph of the DAG of the SlabsIter (see `DAG.dot_digraph`)

#### *classmethod* from_dag(func_nodes, \*, handle_exceptions=(<class 'StopIteration'>, <class 'meshed.slabs.IteratorExit'>, <class 'KeyboardInterrupt'>), scope_factory=<class 'dict'>)

Make a Slabs object from a list of functions and/or FuncNodes, DAG, …

#### *classmethod* from_func_nodes(func_nodes, \*, handle_exceptions=(<class 'StopIteration'>, <class 'meshed.slabs.IteratorExit'>, <class 'KeyboardInterrupt'>), scope_factory=<class 'dict'>)

Make a Slabs object from a list of functions and/or FuncNodes, DAG, …

#### open()

Enter the context of every component that has one, and return `self`.

#### run()

Iterate through all the slabs, discarding them (for the side effects only).

#### to_dag()

Build a `DAG` from the `FuncNode` objects of `to_func_nodes`.

* **Return type:**
  [`DAG`](meshed.dag.html.md#meshed.dag.DAG)

#### to_func_nodes()

Yield a `FuncNode` per component.

Each node’s name and output var node are the component’s name.

* **Return type:**
  [`Iterable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable)[[`FuncNode`](meshed.base.html.md#meshed.base.FuncNode)]

### meshed.slabs.SlabsIter

alias of [`Slabs`](#meshed.slabs.Slabs)

### meshed.slabs.all_arguments_are_none(args, kwargs)

Return True if all arguments are None.

### meshed.slabs.conditional_sentinel(condition_func, sentinel=None)

Decorator that returns sentinel based on a user-defined condition.

The condition function should take the arguments and keyword arguments of the
decorated function as input and return a boolean value. If the condition is met,
the sentinel value is returned instead of the decorated function’s return value.

* **Parameters:**
  * **condition_func** ([`Callable`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable)[[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple), [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)], [`bool`](https://docs.python.org/3/builtins/functions.html#bool)]) – A function that takes the arguments and keyword
    arguments of the decorated function as input and returns a boolean value.
  * **sentinel** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any)) – The value to return if the condition is met.

```pycon
>>> division_by_zero = lambda args, kwargs: (
...     (len(args) >= 2 and args[1] == 0) or
...     kwargs.get('y') == 0
... )
>>>
>>> @conditional_sentinel(division_by_zero, sentinel=0)
... def safe_division(x, y):
...     return x / y
...
>>> safe_division(10, 2)
5.0
>>> safe_division(10, 0)
0
>>>
```

See also `output_none_if_none_arguments`, made from `conditional_sentinel`:

```pycon
>>> @output_none_if_none_arguments
... def foo(x, y):
...     return x + y
>>>
>>> foo(1, 2)
3
>>> assert foo(None, None) is None
```

### meshed.slabs.do_nothing()

Argument-less no-op, the default handler of handled exceptions.

Its `None` output lets the `Slabs` iteration stop.

### meshed.slabs.log_and_return(msg, logger=<built-in function print>)

Pass `msg` to `logger` (`print` by default) and return it unchanged.

### meshed.slabs.output_none_if_none_arguments(func)

Decorator that returns None if all arguments are None.

```pycon
>>> @output_none_if_none_arguments
... def foo(x, y):
...     return x + y
>>>
>>> foo(1, 2)
3
>>> assert foo(None, None) is None
```