CODEWITHSUNDEEP
amazon-web-servicesaws-glue
niraj kumar
niraj kumar

Reputation: 209

Can we see or edit 'job bookmark' info in AWS glue or where it get stored?

I created a glue job and enabled the job bookmark, want to see the metadata stored by glue to keep track of processed files.

Upvotes: 11

Views: 3532

Answers (2)

victorx
victorx

Reputation: 3569

You can now use aws glue get-job-bookmark --job-name <job_name> to get the content of the bookmark. See https://docs.aws.amazon.com/cli/latest/reference/glue/get-job-bookmark.html for detail.

Upvotes: 5

Yuriy Bondaruk
Yuriy Bondaruk

Reputation: 4750

Unfortunately AWS Glue job bookmarking details are not available for customers. As far as I know it is based on the job name, source file name and transformationContext strings passed to DynamicFrame's methods like getSinkWithFormat(), applyMapping(), getCatalogSource().

Besides that Job.init() and Job.commit() must be called at the beginning and end of a script respectively to make bookmarking working.

Upvotes: 2

Related Questions