• Wiring up components in a Pipeline
  • Example Usage
  • How does it work?
• Test documentation

Wiring up components in a Pipeline

The Pipeline component wires up a set of components in a linear chain (a Pipeline) and encapsulates them as a single component.

Example Usage

A simple pipeline of 4 components:

Pipeline(MyDataSource(...),
         MyFirstStageOfProcessing(...),
         MySecondStageOfProcessing(...),
         MyDestination(...),
        ).run()

How does it work?

A Pipeline component gives you a way of wiring up a system of components in a chain and then encapsulating the whole as a single component. The inboxes of this component pass through to the inboxes of the first component in the Pipeline, and the outboxes of the last component pass through to the outboxes of the Pipeline component.

The components you specify are registered as children of the Pipeline component. When Pipeline is activate, all children are wired up and activated.

For the components in the Pipeline, "outbox" outboxes are wired to "inbox" inboxes, and "signal" outboxes are wired to "control" inboxes. They are wired up in the order in which you specify them - data will flow through the chain from first component to last.

The "inbox" and "control" inboxes of the Pipeline component are wired to pass-through to the "inbox" and "control" inboxes (respectively) of the first component in the Pipeline chain.

The "outbox" and "signal" outboxes of the last component in the Pipeline chain are wired to pass-through to the "outbox" and "signal" outboxes (respectively) of the Pipeline component.

During runtime, the Pipeline component monitors the child components. It will terminate if, and only if, all the child components have also terminated.

NOTE that if your child components create additional components themselves, the Pipeline component will not know about them. It only monitors the components it was originally told about.

Pipeline does not intercept any of its inboxes or outboxes. It ignores whatever traffic flows through them.

Test documentation

Tests passed:

• Children are activated as soon as the Pipeline itself is activated, but no sooner.
• Pipeline wires up children so one child's "outbox" outbox feeds to the next's "inbox" inbox.
• Pipeline wires up children so one child's "signal" outbox feeds to the next's "control" inbox.
• Pipeline wires up the first child's "inbox" and "control" inboxes to receive from the pipeline's "inbox" and "control" inboxes.
• Pipeline wires up the last child's "outbox" and "signal" outboxes to send out of the pipeline's "outbox" and "signal" outboxes.
• test_PipelineTerminatesOnlyWhenAllChildrenHaveTerminated (__main__.Test_Pipeline)