CODEWITHSUNDEEP
pythonapache-beam
Yu Watanabe
Yu Watanabe

Reputation: 641

What does the redirection mean in apache beam (python)

In apache beam python sdk , I often see '>>' operator in pipeline procedure.

https://beam.apache.org/documentation/programming-guide/#pipeline-io

lines = p | 'ReadFromText' >> beam.io.ReadFromText('path/to/input-*.csv')

What does this mean?

Upvotes: 7

Views: 582

Answers (1)

Andrew Nguonly
Andrew Nguonly

Reputation: 2621

>> is the right bitwise shift operator in Python. The equivalent dunder (double underscore) method is __rrshift__().

The implementation of Apache Beam in Python simply redefines __rrshift__() for the PTransform class so that names can be added to the transform. It's just special syntax. In your example, "ReadFromText" is the name of the transform.

Reference: https://github.com/apache/beam/blob/4844af152fface83961dd3f4e89022d1e5bef6d6/sdks/python/apache_beam/transforms/ptransform.py#L569-L570

Upvotes: 7

Related Questions