pewpew package

Submodules

pewpew.base module

The base class for PEWPEW processing. See Usage.

class pewpew.base.StreamElement(exit_flag=None, inqueue=None, outqueue=None, **kwargs)[source]

Bases: multiprocessing.context.Process

Subclass this abstract class for concrete implementation of pewpew processing

check_input_flags()[source]

Checks to see if the inputs have exited or not. Useful as the exiting condition is the Queue is empty and the inputs have all finished.

bool: True if at least one input is OK.

event_loop(**kwargs)[source]
get_data(**kwargs)[source]
on_completion()[source]

Override this method to perform an action at the process’ end of execution.

on_input_completed(**kwargs)[source]
on_start()[source]

Override this method to perform an action at the process’ beginning of execution.

abstract process(data)[source]

Abstract method. Implement this for the primary action this process will take on data

data: list or dict or None

Input data to be acted on. Primary data generators can accept None as an input, and produce data.

dict or list:

Data to be processed downstream.

put_data(**kwargs)[source]
run()[source]

Called by multiprocessing.Process. Executes main event loop for process.

set_input(other)[source]

Add an input StreamElement to this one. Creates a Queue between StreamElements in the event there is not an existing one.

other: StreamElement

An other Stream Element which will stream queued data into this one.

set_output(other)[source]

Sets self as an input StreamElement to other. Creates a Queue between StreamElements in the event there is not an existing one.

other: StreamElement

An other Stream Element which will stream queued data from this one.

signal_exit_on_failure()[source]

Helper decorator which sets appropriate flags when exceptions occur in daughter processes.

valid_data(data)[source]

Validates whether data is valid for the data stream.

datalist or dict

Input data which must be validated

bool : True if valid data

pewpew.base.exit_flag()[source]

Convenience function for creating the exit flag data type instance.

pewpew.cli module

Module contents