CODEWITHSUNDEEP
bazelbuildfarm
Zhuo Zhang
Zhuo Zhang

Reputation: 11

Best way to scale up Bazel cache server?

I have set up a Bazel cache server in our company and have been using it quite smoothly. I simply followed the instruction here to set up a Nginx server. The underlying storage is on a solid state disk.

However, as our number of users grows, sometimes the cache server got stuck. I don't see any document talking about how to scale up to multiple cache servers. Any idea how to do that?

On top of my head, I could do the followings:

  1. Run cache server on a dedicated machine, basically give it more resource,

  2. Run multiple cache servers behind a Nginx load-balancer. But how does Nginx load-balancer interacts with bazel cache hash strategy? Will the same cache request be sent to the same cache server?

  3. Switch to use Bazel buildfarm. But the problem remains, will the same build request be sent to the same remote worker?

Upvotes: 1

Views: 1077

Answers (1)

Ittai
Ittai

Reputation: 5915

I think that buildfarm has solved the routing request problem (though not sure how).

I'd suggest asking this question on buildfarm's mailing list.

Additionally you might be able to utilize GCS or S3 (pending PR) as your caching backends since they are much more scalable (though I'm not sure about their latencies).

Upvotes: 0

Related Questions