creek#

A simple facade for streams: wrap an iterable, layer transformations on it, and read unbounded streams as if they were lists.

To install: pip install creek

The smallest complete example: a Creek subclass that parses each line of a file-like stream.

>>> from io import StringIO
>>> from creek import Creek
>>> class CsvCreek(Creek):
...     def data_to_obj(self, line):
...         return [x.strip() for x in line.strip().split(',')]
>>> stream = CsvCreek(StringIO('a, b, c\n1,2, 3\n'))
>>> list(stream)
[['a', 'b', 'c'], ['1', '2', '3']]

For AI agents#

creek ships tooling for coding agents. If you are one, start here.

The documentation, machine-readable: llms.txt indexes every page; creek.md is the whole documentation in one file; every page has a .md twin; objects.inv maps symbols to URLs.

If you are a control freak, the rest of this README is written for you, starting at The Creek layers.

The Creek layers#

The Creek base class offers a layer-able wrap of the stream interface. There are three layering methods, pre_iter, data_to_obj and post_iter, each the identity by default. Iterating over a creek is exactly:

yield from self.post_iter(map(self.data_to_obj, self.pre_iter(self.stream)))

  • pre_iter(stream): prepare and/or filter the raw stream (skip a header, enumerate lines, …)

  • data_to_obj(item): transform each item the stream yields

  • post_iter(objs): further process or filter the transformed objects

Everything else (seek, close, the context manager protocol, …) is delegated to the wrapped stream:

>>> src = StringIO(
... '''a, b, c
... 1,2, 3
... 4, 5,6
... '''
... )
>>> class MyCreek(Creek):
...     def data_to_obj(self, line):
...         return [x.strip() for x in line.strip().split(',')]
...
>>> stream = MyCreek(src)
>>> list(stream)
[['a', 'b', 'c'], ['1', '2', '3'], ['4', '5', '6']]
>>> stream.seek(0)  # oh!... but we consumed the stream already, so let's go back to the beginning
0
>>> list(stream)
[['a', 'b', 'c'], ['1', '2', '3'], ['4', '5', '6']]
>>> stream.seek(0)  # rewind again
0
>>> next(stream)
['a', 'b', 'c']
>>> next(stream)
['1', '2', '3']

Let’s add a filter! There are two kinds you can use: one applied to the line before the data is transformed by data_to_obj (in pre_iter), and one applied after, to the objects (in post_iter).

>>> src = StringIO(
...     '''a, b, c
... 1,2, 3
... 4, 5,6
... ''')
>>> class MyFilteredCreek(MyCreek):
...     def post_iter(self, objs):
...         yield from filter(lambda obj: str.isnumeric(obj[0]), objs)
>>>
>>> s = MyFilteredCreek(src)
>>> list(s)
[['1', '2', '3'], ['4', '5', '6']]
>>> s.seek(0)
0
>>> list(s)
[['1', '2', '3'], ['4', '5', '6']]
>>> s.seek(0)
0
>>> next(s)
['1', '2', '3']

Recipes:

  • pre_iter: itertools.islice to skip header lines

  • pre_iter: enumerate to get line indices in the stream iterator

  • pre_iter = functools.partial(map, line_pre_proc_func) to preprocess all lines with line_pre_proc_func

  • pre_iter: functools.partial(filter, condition) to filter before data_to_obj

  • post_iter: itertools.chain.from_iterable to flatten a chunked/segmented stream

  • post_iter: functools.partial(filter, condition) to filter the yielded objects

Beyond Creek#

  • InfiniteSeq and IndexedBuffer (creek.infinite_sequence): list-like s[i:j] access to an unbounded stream, through a bounded buffer.

  • BufferStats and Segmenter (creek.tools): rolling-window statistics and segmentation of a stream.

  • dynamically_index and filter_and_index_stream (creek.tools): enumerate with a custom index rule, and filtered indexing.

Documentation#

  • Rendered docs: https://i2mint.github.io/creek/

  • The whole API in one Markdown file: https://i2mint.github.io/creek/creek.md

This documentation as a single file: creek.md (Markdown, for agents).

built 2026-09-15 10:07 UTC from 7a19d20 (master) · creek 0.1.35 · about this build