Arjunlal M.A
Reputation: 151
Apache Spark What is the difference between requiredChildDistribution and outputPartitioning?
In Apache Spark, each physical operator in the physical plan has 4 properties:
- outputPartitioning
- outputOrdering
- requiredChildDistribution
- requiredChildOrdering
But aren't outputPartioning and requiredChildDistribution the same? How are they different and what do they fundamentally represent? Same for outputOrdering and requiredChildOrdering?
Upvotes: 1
Views: 64
Answers (0)
Related Questions
- In spark, what does the parameter "minPartitions" works in SparkContext.textFile(path, minPartitions)?
- What is partition in Spark?
- What's the difference between element and partition in Spark?
- Spark: how does partitionBy (DataFrameWriter) actually work?
- Difference between writing to partition path directly and using partitionBy
- How does HashPartitioner work?
- how is "Exchange hashpartitioning" working in spark
- Apache Spark : When not to use mapPartition and foreachPartition?
- What is the Difference between mapPartitions and foreachPartition in Apache Spark
- How does range partitioner work in Spark?