CODEWITHSUNDEEP
authenticationcachingarchitecturemicroservicesload-balancing
brainoverflow98
brainoverflow98

Reputation: 1278

How to better utilize local cache with load balancing strategies?

I have an Authentication service where I need to cache some user information for better performance. I chose to use local cache because Authentication service probably will be called on each request so I want it to be super fast. Compared to remote cache options local cache is a lot faster (local cache access is below 1ms while remote cache access is around 25ms).

The problem is I can not cache as much information as a distributed cache without running out of memory (talking about millions of users). I can either leave it as it is and when local cache reaches the memory limit it would evict some other data but that would be bad optimization of the cache. Or I can use some kind of load balancer strategy where users will be redirected to same Authentication service instances based on their IP address or other criteria thus the cache hits will be a lot higher.

It kind of defeats the purpose of having stateless services however I think I can slightly compromise from this principle in network layer if I want both consistency and availability. And as for Authentication both are crucial for full security (user info always has to be up-to-date and available).

What kind of load balancing techniques out there for solving this kind of problem? Can there be other solutions?

Note: Even though this question is specific to Authentication I think many other services that are frequently accesses and requires speed can benefit a lot from using local caches.

Upvotes: 0

Views: 1712

Answers (1)

Mavi Domates
Mavi Domates

Reputation: 4521

So - to answer the question here - load balancers can handle this with their hashing algorithms.

I'm using Azure a lot so I'm giving Azure Load Balancer as an example:

From the docs:

Hash-based distribution mode

The default distribution mode for Azure Load Balancer is a five-tuple hash.

The tuple is composed of the:

  • Source IP
  • Source port
  • Destination IP
  • Destination port
  • Protocol type

The hash is used to map traffic to the available servers. The algorithm provides stickiness only within a transport session. Packets that are in the same session are directed to the same datacenter IP behind the load-balanced endpoint. When the client starts a new session from the same source IP, the source port changes and causes the traffic to go to a different datacenter endpoint.

Upvotes: 1

Related Questions