Pipes

Pipes can be used to compose functionality, such that data streams through a pipe and is modified on the go. Pipes are sequence of objects, which inspect and/or modify the data that is passed through it.

A pipe consists of pipe segments, which are combined into a single pipe object. Pipe segments cannot be accessed or split in run-time, they are absorbed into some complex template-based recursive type, implementing the Pipe interface. Pipes, however, can be connected dynamically.

Pipes always have the following form, for a given type T:

1auto pipe =
2   Entry<T>{} >>
3   // pipe segments...
4   Exit{};

pipe is a pipe instance, taking data of type T, and producing it, such as 1 >> pipe;, or pipe.extract();. It can be connected to other pipe instances dynamically.

1auto pipe =
2   Entry<T>{} >>
3   // pipe segments...
4   Cap{};

In this case, pipe is an instance, with a cap at the end; it does not allow dynamic binding to other pipes.

1auto& pipe =
2   Entry<T>{} >>
3   // pipe segments...
4   Ref{};

Now, pipe is a reference, like the capped pipe above. It is automatically destroyed at shutdown (by the stored::pipes::gc Group). When a group reference is passed to Ref{...}, the pipe is still allocated using the default allocator, but added to the provided group, instead of gc.

Pipe segments can be connected, such as:

 1auto& pipe =
 2   Entry<int>{} >>
 3   Tee{
 4         Entry<int>{} >>
 5         Log<int>{"tee 1"} >>
 6         Ref{},
 7
 8         Entry<int>{} >>
 9         Call{[](int x) { printf("tee 2 = %d\n", x); }} >>
10         Ref{}
11   } >>
12   Buffer<int>{} >>
13   Cap{};

A pipe segment has to adhere to the following rules:

  • It must be a struct or class. No specific base classes have to be used. No specific (virtual) interface is defined.

  • It must implement a B inject(A) member function, where the types A and B can be freely chosen. In fact, having inject() makes a class a pipe segment. The inject() function is used to extract the pipe segment’s input type A and output type B. You are free to choose if the type is a value or reference, or cv-qualified.

  • It may implement B_ extract(), where B_ must compatible with B. However, B could be const&, while B_ is just a value, for example.

  • It may implement A_ entry_cast(B_) const and/or B_ exit_cast(A_) const, where A_ and B_ for both functions must be compatible with A and B.

  • It may implement B_ trigger(bool*), where B_ must be compatible with B.

  • It must be move-constructable.

  • Preferably, keep pipe segments simple and small. Create complexity by composition.

For examples, check include/libstored/pipes.h, and especially the Identity class implementation for the simplest form.

The following functions are available for pipes, and can be available for pipe segments:

B inject(A)

Inject a value into the pipe (segment). The function returns the output of the pipe (segment). This is the normal direction of the data flow.

B extract()

This function tries to find data from the exit of the pipe back through the segments. Usually, it returns the value of the last Buffer in the pipe. If there is no such segment, a default constructed B is returned. If a pipe segment does not support extraction, it can omit the function.

A entry_cast(B) const

Type-cast the given pipe (segment) output type to the input type. It should not modify the state of the pipe (segment). When omitted, it is assumed that the input and output types are assignable.

B exit_cast(A) const

Type-cast the given pipe (segment) input type to the output type. It should not modify the state of the pipe (segment). When omitted, it is assumed that the input and output types are assignable.

B trigger(bool*)

Some pipe segments have a special side-effect, such as reading external data. It may implement the trigger function to perform this side-effect. The parameter can be used to indicate if the pipe (segment) has returned any data by writing true to the provided pointer. The first pipe segment that provides data by a trigger, injects this data into the remainder of the pipe.

abstract PipeEntry abstract PipeExit abstract Pipe abstract PipeBase PipeBase <|-- Pipe PipeEntry <|-- Pipe PipeExit <|-- Pipe class PipeInstance class Segments Pipe <|-- PipeInstance Segments <|-- PipeInstance : private

stored::pipes::PipeBase

class PipeBase

Common base class of all pipes.

Subclassed by stored::pipes::Pipe< std::decay_t< segment_traits< S >::type_in >, std::decay_t< segment_traits< S >::type_out > >, stored::pipes::Pipe< In, Out >

Public Functions

virtual ~PipeBase() = default
virtual void trigger(bool *triggered = nullptr) = 0

stored::pipes::Pipe

template<typename In, typename Out>
class Pipe : public stored::pipes::PipeBase, public stored::pipes::PipeEntry<In>, public stored::pipes::PipeExit<Out>

Virtual base class for any segment/pipe with specific in/out types.

Public Functions

virtual ~Pipe() override = default
virtual type_out_wrapper inject(type_in const &x) = 0
inline type_out_wrapper operator()(type_in const &x)

Friends

inline friend type_out_wrapper operator<<(Pipe &p, type_in const &x)
inline friend type_out_wrapper operator>>(type_in const &x, Pipe &p)

stored::pipes::PipeEntry

template<typename In>
class PipeEntry

Virtual base class for any segment/pipe with specific in type.

Subclassed by stored::pipes::Pipe< std::decay_t< segment_traits< S >::type_in >, std::decay_t< segment_traits< S >::type_out > >, stored::pipes::Pipe< In, Out >

Public Types

using type_in = In

Public Functions

virtual ~PipeEntry() = default
inline void inject(type_in const &x)
inline void operator()(type_in const &x)
virtual void trigger(bool *triggered = nullptr) = 0

Friends

inline friend void operator<<(PipeEntry &p, type_in const &x)
inline friend void operator>>(type_in const &x, PipeEntry &p)

stored::pipes::PipeExit

template<typename Out>
class PipeExit

Virtual base class for any segment/pipe with specific out type.

Subclassed by stored::pipes::Pipe< std::decay_t< segment_traits< S >::type_in >, std::decay_t< segment_traits< S >::type_out > >, stored::pipes::Pipe< In, Out >

Public Types

using type_out = Out
using type_out_wrapper = ExitValue<Out>

Public Functions

virtual ~PipeExit() = default
inline virtual void connect(PipeEntry<Out> &p)
inline bool connected() const noexcept
inline virtual PipeEntry<Out> *connection() const noexcept
inline virtual void disconnect() noexcept
inline void extend(Pipe<Out, Out> &p)
virtual type_out_wrapper extract() = 0
template<typename Out_>
inline Pipe<Out, Out_> &operator>>(Pipe<Out, Out_> &p) &
virtual void trigger(bool *triggered, std::decay_t<type_out> &out) = 0
virtual void trigger(bool *triggered = nullptr) = 0

Friends

inline friend auto &operator<<(std::decay_t<type_out> &x, PipeExit &p)
inline friend auto &operator>>(PipeExit &p, std::decay_t<type_out> &x)

stored::pipes::Group

class Group

A set of pipes.

Public Types

using set_type = stored::Set<PipeBase*>::type

Public Functions

void add(PipeBase &p)
void add(std::initializer_list<std::reference_wrapper<PipeBase>> il)
set_type::const_iterator begin() const noexcept
void clear()
void destroy()
void destroy(PipeBase &p)
set_type::const_iterator end() const noexcept
set_type const &pipes() const noexcept
void remove(PipeBase &p)
void remove(std::initializer_list<std::reference_wrapper<PipeBase>> il)
size_t size() const noexcept
void trigger(bool *triggered = nullptr) const

There is one special group: stored::pipes::gc, which destroys pipes created with default Ref.

stored::pipes::Buffer

template<typename T>
class Buffer

Memory element for injected values.

The constructor accepts an optional argument as initial value stored in the buffer. Extracted data is returned from this memory.

Subclassed by stored::pipes::Triggered< T, Compare, N >

Public Types

using type = std::decay_t<type_in>
using type_in = T
using type_out = type const&

Public Functions

constexpr Buffer() = default
template<typename type_, std::enable_if_t<std::is_constructible<type, type_>::value, int> = 0>
inline explicit constexpr Buffer(type_ &&x)
inline type_out extract()
inline type_out inject(type_in const &x)

stored::pipes::Call

template<typename T, typename F = void(T)>
class Call

Invoke a function for every inject.

The function prototype can be:

  • void(T)

  • void(T const&)

  • void(T&)

  • T(T)

The first two cases allow a function to observe the injected value. The last two cases allow modifying it; the modified/returned value is passed downstream.

T and F are auto-deducted in C++17.

Public Types

using function_type = F

Public Functions

template<typename F_, std::enable_if_t<std::is_constructible<std::function<function_type>, F_>::value, int> = 0>
inline explicit Call(F_ &&f)
inline T const &inject(T const &x)

stored::pipes::Cast

template<typename In, typename Out>
using stored::pipes::Cast = impl::Cast<In, Out>

A pipe segment that casts between types.

Numeric types are cast using saturated_cast. For other types, static_cast is used.

stored::pipes::Changes

template<typename T, typename Compare = std::equal_to<T>>
class Changes

Forward injected values, when they are different.

By default the comparator uses == for comparison. Override with a custom type, such as stored::pipes::similar_to.

Public Types

using type = T

Public Functions

inline explicit Changes(PipeEntry<type> &p)
template<typename T_, std::enable_if_t<std::is_constructible<T, T_&&>::value, int> = 0>
inline Changes(PipeEntry<type> &p, T_ &&init)
inline type const &inject(type const &x)
template<typename T, unsigned int order = 3>
using stored::pipes::similar_to = impl::similar_to<T, order>

Like std::equal_to, as long as the difference is within 10^order.

stored::pipes::Constrained

template<typename Constraints>
class Constrained

Applies constraints to the value in the pipe.

Injected data is passed to the operator() of the Constraints instance, and its result is passed further on through the pipe. The Constraints::operator() is assumed to be stateless.

See also

stored::pipes::Bounded

Public Types

using type = typename impl::constraints_type<decltype(&Constraints::operator())>::type

Public Functions

template<typename Constraints_ = Constraints, std::enable_if_t<std::is_default_constructible<Constraints_>::value, int> = 0>
inline Constrained()
template<typename Constraints_, std::enable_if_t<std::is_constructible<Constraints, Constraints_&&>::value, int> = 0>
inline explicit Constrained(Constraints_ &&constraints)
inline type exit_cast(type x) const
inline type inject(type x)
template<typename T>
class Bounded

Constraints that bounds the value of the pipe between a min and max bound.

See also

stored::pipes::Constrained

Public Types

using type = T

Public Functions

inline explicit Bounded(type low = std::numeric_limits<type>::lowest(), type high = std::numeric_limits<type>::max())
inline type operator()(type x) const

stored::pipes::Convert

template<typename Converter>
class Convert

Converts a value in the pipe, given a converter class.

Injected data is passed to the exit_cast() of the Converter instance, and its result is passed further on through the pipe. For extract, the entry_cast() of the Converter instance is used. Both methods of the Converter are assumed to be stateless.

See also

stored::pipes::Scale

Public Types

using type = typename impl::constraints_type<decltype(&Converter::exit_cast)>::type

Public Functions

template<typename Converter_ = Converter, std::enable_if_t<std::is_default_constructible<Converter_>::value, int> = 0>
inline Convert()
template<typename Converter_, std::enable_if_t<std::is_constructible<Converter, Converter_&&>::value, int> = 0>
inline explicit Convert(Converter_ &&converter)
inline decltype(auto) entry_cast(type const &x) const
inline decltype(auto) exit_cast(type const &x) const
inline decltype(auto) inject(type const &x)
template<typename T, typename ratio = std::ratio<1, 1>>
class Scale

Converts a values by means of a scale factor.

See also

stored::pipes::Convert

Template Parameters:

  • T – the type of the data, which must be a floating point type

  • ratio – std::ratio or equivalent

Public Types

using type = T

Public Functions

inline type entry_cast(type value) const noexcept
inline type exit_cast(type value) const noexcept

stored::pipes::Get

template<typename T, typename V>
class Get

Extract a store object’s value upon every extract/inject.

The value passed through the pipe entry is ignored. Upon trigger(), the value from the object is passed through the pipe.

V can be a stored::Variant, and any reference to a fixed-type store object. Pass it to the constructor.

Usually, use C++17 auto-deduction to determine both T and V.

stored::pipes::Identity

template<typename T>
class Identity

Pipe segment without side effects.

Subclassed by stored::pipes::impl::Cast< T, T, is_number >

Public Functions

inline T const &inject(T const &x)

stored::pipes::Log

template<typename T>
class Log

Invokes a logger function for injected values.

By default, the value is printed to stdout. If a custom logger function is provided, it must accept a name (std::string const&) and value (T).

Public Types

using logger_type = void(std::string const&, type const&)
using type = T

Public Functions

inline explicit Log(std::string name)
template<typename F, std::enable_if_t<std::is_assignable<std::function<logger_type>, F>::value, int> = 0>
inline Log(std::string name, F &&logger)
inline type const &inject(type const &x)

stored::pipes::Map

template<typename T, size_t N, typename CompareValue = std::equal_to<T>>
class IndexMap

An index to T map.

It addresses an array to find the corresponding value. All indices [0,N[ should be present.

find() complexity is O(1), rfind() is O(N).

Public Types

using key_type = size_t
using value_type = T

Public Functions

inline explicit constexpr IndexMap(std::array<value_type, N> &&map)
inline explicit constexpr IndexMap(std::array<value_type, N> const &map)
template<typename T0_, typename ...T_, std::enable_if_t<sizeof...(T_) + 1 == N && std::is_constructible<value_type, T0_>::value, int> = 0>
inline explicit constexpr IndexMap(T0_ &&t0, T_&&... t)
template<typename Key>
inline constexpr value_type const &find(Key &&index) const
inline constexpr size_t rfind(value_type const &value) const
template<typename Key, typename Value, size_t N, typename CompareKey = std::less<Key>, typename CompareValue = std::equal_to<Value>>
class OrderedMap

An Key to Value map with ordered keys.

Internally, all key-value pairs are saved in a array. To perform a binary search through this array, the keys must have a natural order (usually by implementing operator< ), and the values must be sorted for initialization.

find() complexity is O(log(N)), rfind() is O(N).

Public Types

using element_type = std::pair<key_type, value_type>
using key_type = Key
using value_type = Value

Public Functions

inline explicit constexpr OrderedMap(std::array<element_type, N> &&map)
inline explicit constexpr OrderedMap(std::array<element_type, N> const &map)
template<typename T0_, typename ...T_, std::enable_if_t<sizeof...(T_) + 1 == N && std::is_constructible<element_type, T0_>::value, int> = 0>
inline explicit constexpr OrderedMap(T0_ &&t0, T_&&... t)
inline value_type const &find(key_type const &key) const
inline constexpr key_type const &rfind(value_type const &value) const
template<typename Key, typename Value, size_t N, typename CompareKey = std::equal_to<Key>, typename CompareValue = std::equal_to<Value>>
class RandomMap

A Key to value Map for arbitrary types.

In contrast to the OrderedMap, the keys do not have to be ordered, or even have some natural order. Keys are compared using == (or equivalent). This makes searching linear, but allows different types.

find() and rfind() complexity is both O(N).

Public Types

using element_type = std::pair<Key, Value>
using key_type = Key
using value_type = Value

Public Functions

inline explicit constexpr RandomMap(std::array<element_type, N> &&map)
inline explicit constexpr RandomMap(std::array<element_type, N> const &map)
template<typename T0_, typename ...T_, std::enable_if_t<sizeof...(T_) + 1 == N && std::is_constructible<element_type, T0_>::value, int> = 0>
inline explicit constexpr RandomMap(T0_ &&t0, T_&&... t)
inline constexpr value_type const &find(key_type const &key) const
inline constexpr key_type const &rfind(value_type const &value) const
template<typename Key, typename Value, size_t N, typename CompareKey = std::equal_to<Key>, typename CompareValue = std::equal_to<Value>>
auto stored::pipes::make_random_map(std::pair<Key, Value> const (&kv)[N], CompareKey compareKey = CompareKey{}, CompareValue compareValue = CompareValue{})

A helper function to create the appropriate RandomMap instance.

The resulting map may be passed to stored::pipes::Mapped.

template<typename From, typename To, typename MapType>
class Mapped

An associative map.

The inject value is translated using the constructor-provided map. If the injected value is not found in the map, the first element of the map is used.

The MapType must have:

  • a type key_type (equivalent to From) for C++17 template deduction

  • a type value_type (equivalent to To) for C++17 template deduction

  • From find(To) (where From /To may be const& )

  • To rfind(From) (where From /To may be const& )

Public Types

using type_in = From
using type_out = To

Public Functions

template<typename E0_, typename ...E_, std::enable_if_t<!std::is_same<MapType, std::decay_t<E0_>>::value && std::is_constructible<MapType, E0_&&, E_&&...>::value, int> = 0>
inline explicit constexpr Mapped(E0_ &&e0, E_&&... e)
inline explicit constexpr Mapped(MapType &&map)
inline explicit constexpr Mapped(MapType const &map)
inline decltype(auto) entry_cast(type_out const &value) const
inline decltype(auto) exit_cast(type_in const &key) const
inline decltype(auto) inject(type_in const &key)
template<typename Key, typename Value, size_t N, typename CompareKey = std::less<Key>, typename CompareValue = std::equal_to<Value>>
auto stored::pipes::Map(std::pair<Key, Value> const (&kv)[N], CompareKey compareKey = CompareKey{}, CompareValue compareValue = CompareValue{})

An associative map.

Actually, this is a helper function to create a stored::pipes::Mapped with the appropriate stored::pipes::OrderedMap.

Example: Map({{1, 10}, {2, 20}, {3, 30}})

template<typename T, size_t N, typename CompareValue = std::equal_to<T>>
auto stored::pipes::Map(T const (&values)[N], CompareValue compareValue = CompareValue{})

An associative map.

Actually, this is a helper function to create a stored::pipes::Mapped with the appropriate stored::pipes::IndexMap.

Example: Map({1, 2, 3})

template<typename T0, typename T1, typename ...T>
constexpr auto stored::pipes::Map(T0 &&v0, T1 &&v1, T&&... v)

An associative map.

Actually, this is a helper function to create a stored::pipes::Mapped with the appropriate stored::pipes::IndexMap.

Example: Map(1, 2, 3)

stored::pipes::Mux

template<typename T, size_t N>
class Mux

Multiplex pipes, given the injected index value.

Pipes passed to the constructor are saved as references. Given the injected value, the corresponding pipe is extracted when required.

Public Functions

template<typename ...P, std::enable_if_t<sizeof...(P) + 1 == N, int> = 0>
inline explicit constexpr Mux(PipeExit<T> &p0, P&... p)
inline size_t entry_cast(T x) const
inline T exit_cast(size_t x) const
inline T extract()
inline T inject(size_t i)
inline T trigger(bool *triggered = nullptr)

stored::pipes::RateLimit

template<typename T, typename Compare = std::equal_to<T>, typename Duration = std::chrono::milliseconds, typename Clock = std::chrono::steady_clock>
class RateLimit

Forward injected values, when they are different, with a minimum interval.

By default the comparator uses == for comparison. Override with a custom type, such as stored::pipes::similar_to.

Public Types

using clock_type = Clock
using duration_type = Duration
using type = T

Public Functions

inline explicit RateLimit(PipeEntry<type> &p, duration_type interval)
template<typename T_>
inline RateLimit(PipeEntry<type> &p, duration_type interval, T_ &&init)
inline type const &inject(type const &x)
inline type const &trigger(bool *triggered = nullptr, typename clock_type::time_point now = clock_type::now())

stored::pipes::Set

template<typename T, typename V>
class Set

Write a value that is injected in the pipe to a store object.

Pass the store object to the constructor.

See also

Get

stored::pipes::Signal

template<typename T, typename Key = void*, typename Token = void*>
class Signal

Public Types

using Signal_type = stored::Signal<Key, Token, type>
using type = T

Public Functions

inline explicit Signal(Signal_type &signal, Key key = Signal_type::NoKey)
inline type exit_cast(type x) const
inline type inject(type x)
inline void signal(type x) const

stored::pipes::Tee

template<typename T, size_t N>
class Tee

Forwards injected data into a fixed list of other pipes.

Pass any positive number of references to PipeEntry<T> to the constructor.

When using C++17, the template parameters are auto-deduced from the provided pipes.

Public Functions

template<typename ...P, std::enable_if_t<sizeof...(P) + 1 == N, int> = 0>
inline explicit constexpr Tee(PipeEntry<T> &p0, P&... p)
inline T const &inject(T const &x)

stored::pipes::Transistor

template<typename T, bool invert = false, typename Gate = bool>
class Transistor

Pass either the value or a default-constructed value through the pipe, depending on a gate pipe.

The gate pipe value is extracted upon every inject.

Public Functions

inline explicit Transistor(PipeExit<Gate> &gate)
inline T inject(T const &x)

stored::pipes::Triggered

template<typename T, typename Compare = std::equal_to<std::decay_t<T>>, size_t N = 1>
class Triggered : private stored::pipes::Buffer<T>, private stored::pipes::Tee<T, 1>

A buffer, which forwards the last injected data to other pipes when triggered.

Think of it as a write-back cache, where the write-back action is triggered using trigger(). To determine if the data has changed, the Compare template parameter is used.

Public Types

using type = typename buffer::type
using type_in = typename buffer::type_in
using type_out = typename buffer::type_out

Public Functions

template<typename ...P, std::enable_if_t<sizeof...(P) + 1 == N, int> = 0>
inline explicit constexpr Triggered(PipeEntry<T> &p0, P&... p)
template<typename type_, std::enable_if_t<std::is_constructible<type, type_>::value, int> = 0>
inline explicit constexpr Triggered(type_ &&x)
template<typename type_, std::enable_if_t<std::is_constructible<type, type_>::value, int> = 0, typename ...P, std::enable_if_t<sizeof...(P) + 1 == N, int> = 0>
inline explicit constexpr Triggered(type_ &&x, PipeEntry<T> &p0, P&... p)
inline type_out extract()
inline type_out inject(type_in const &x)
inline type_out trigger(bool *triggered = nullptr)