"cap production deploy" can erroneously deploy in development

After running "cap production deploy" I had a mysterious case of "Action Controller Exception: Mysql2::Error Can't connect to MySQL server on '127.0.0.1' (111)". The settings in database.yml were correct. I could verify that the database was, indeed, accessible by using these settings in MySQL Workbench to ssh tunnel to the database on the VPS from my laptop. So what was going on?

It turned out, that the application was deployed in development, not production despite the capistrano command and the settings in deploy/production.rb. And in database.yml I had incorrect settings in the development block - they were supposed to work only on my laptop but not at the VPS.

It was a great relief to find the explanation of that mysterious error. But how could be that "cap production deploy" produced a deployment in a development environment, rather than in production? I don't have 100% certainty as I changed a number of things while trying to find the culprit but the most likely reason was the following line in /etc/nginx/nginx.conf:

server {
    rack_env           development;

Once I changed that setting and re-started nginx the problem disappeared. I am relieved but I still find the whole thing a bit troubling. Somehow I cannot escape the feeling that the rails stack is really messy and fragile - you can specify the same setting (in this case the environment) in a number of different places (in fact in different layers of the stack) so that you get inconsistency and unexpected behaviour. God knows what other settings are now over-ridden but haven't yet manifested themselves in an error. This is a bit frightening.

Anyway - you have to live with it if you want to use rails. For those who are suspecting a similar problem in their deployment:

The best way to tell which environment your application is running in is to look at the logs (/home/deploy_user/my_app/shared/log). In my case the file development.log had a current timestamp while production.log was stale (a few weeks old). That sort of nailed it. However, the situation could be a bit more complicated if you are running different instances of the application at different ports - they could run in different environments. Than you need to look up the logs and find which one has the error message that you are getting via the browser.