CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
user_1357
user_1357

Reputation: 7940

Apache Beam: Share resources among all threads within instance

In FoDn we have @Setup which is perfect to initialize cross bundle resources. Is this shared among CPUs as well? In other words, do we need to perform management of resources statically for this to be shared among all CPUs?

Upvotes: 1

Views: 312

Answers (1)

robertwb
robertwb

Reputation: 5104

@Setup is called per DoFn instance, which is always local to a single thread. If you want to share a resource across the process you can manage this manually with static/global variables. (To do so across processes would probably require some kind of file locking.)

Upvotes: 1

Related Questions