CODEWITHSUNDEEP
amazon-web-servicespysparketlaws-glueaws-glue-data-catalog
Diba
Diba

Reputation: 179

How to rewind Job Bookmarks on Glue Spark ETL job?

I have read here that now Glue provides the ability to rewind job bookmarks for Spark ETL job.

Still, I haven't been able to find any information on how to do that. The sub-options in the "paused" job bookmark option seem to be useful in rewinding a job bookmark, but I can't find how to implement them (I am using Glue console.)

Upvotes: 2

Views: 3265

Answers (1)

sumitya
sumitya

Reputation: 2691

What you need to pass following parameters in "Job parameters" section. With job bookmarks paused.

job-bookmark-from <from-value> is the run ID which represents all the input that was processed until the last successful run before and including the specified run ID.

job-bookmark-to <to-value> is the run ID which represents all the input that was processed until the last successful run before and including the specified run ID. The corresponding input excluding the input identified by the is processed by the job.

Upvotes: 3

Related Questions