pgrandjean
Reputation: 766
How to override txnVersion in Spark?
Following the instructions for idempotency in Idempotent table writes in foreachBatch, I set the txnVersion and txnApp options. It works as intended. However, those options keep their values even after foreachBatch ends.
How to override txnVersion and txnApp after foreachBatch ends (meaning no new data to write)?
Upvotes: 0
Views: 21
Answers (0)
Related Questions
- How to find the index for a given item in a list?
- How do I make a flat list out of a list of lists?
- How to use a global variable in a function?
- How slicing in Python works
- How do I check whether a file exists without exceptions?
- How do I merge two dictionaries in a single expression in Python?
- How to access the index value in a 'for' loop?
- How do I execute a program or call a system command?
- How to catch multiple exceptions in one line? (in the "except" block)
- How do I create a directory, and any missing parent directories?