CODEWITHSUNDEEP
ruby-on-railscsrf
Daniel
Daniel

Reputation: 905

Rails deployments resulting in spikes of "Can't verify CSRF token authenticity"

This is a Rails 5.1 web app migrated from a Rails 4 app (in turn migrated from a Rails 3 app) deployed to Datica (which is Docker based).

After each deployment, our logs record spikes in the occurrence of "Can't verify CSRF token authenticity" warnings and corresponding 401 statuses. The front-end is a React app, with most client-server communication happening over XHR, so some people receive 401 multiple times. Refreshing the browser has solved the 401 status for me twice, so I think the fall-off in the spikes indicates people refreshing or ceasing to use the app because it isn't working for them (see screenshot of the log graph showing the spikes).

The app uses cookies for session storage. The CSRF token is in the body of the page when it was first loaded. What I have observed would be explained by the cookies being given a new CSRF token, so that there is a mismatch between the CSRF token in the body of the page and that in the session, but I do not understand what would cause the server to issue a new CSRF token after a deployment.

Thinking this might be due to a delay in the newly launched instances receiving the Origin header, or the old instances ceasing to receive it, I set config.action_controller.forgery_protection_origin_check = false in application.rb. However, there were spikes after deploying last night and again this morning (and it wouldn't really make sense anyway since refreshing the browser appears to solve the problem).

Any ideas as to how a deployment would cause the server to issue new CSRF tokens to sessions? Or any other ideas or other hypotheses?

Thank you!

graph of 401 spikes

Upvotes: 1

Views: 452

Answers (1)

Daniel
Daniel

Reputation: 905

The problem was that MyApp::Application.config.secret_key_base was never set. Back when the app was migrated from Rails 3 to 4, the recommendation was to wait until the user base was completely on 4 before uncommenting that line. But that never happened.

I presume that if that secret_key_base isn't set, it automatically generates an in-memory key base for generating CSRF tokens, resulting in a new key base every deployment.

Upvotes: 4

Related Questions