What are environments, anyway?

Most software companies have their product deployed to a few different environments, which have names like local, dev, test, stage, and production. On my first day as a developer I was confused about why there were so many steps between a developer’s computer and the app that customers used. It seemed inefficient and bureaucratic. I’ve come to appreciate those environments, though, and these days I break out in an ice-cold sweat if someone talks about deploying straight to production. If these terms are unfamiliar to you, or if you also wonder why applications need so many environments, stick around: I’ll be giving you the low-down on each of them.

But first of all, what is an environment? It’s a computer (real or virtual) which runs your software and which you control. If you use a Continous Integration tool (like VSTS, Travis, GitLab, or my personal favorite, TeamCity), you probably have access to a tidy list of environments alongside buttons that build the latest version of the project and deploy it to them. You may also have tasks you can run on each environment, like unit or end-to-end tests. Your own computer is an environment, referred to as local, and the server that provides customer access to your product is the production environment.

Let’s talk about the purpose of these environments and best practices for each. My assumptions are that you’re using Git and developing an app that runs on the web, but the same principles apply to other version control systems and platforms as well.

A tour of standard environments

Local

This is where all development work happens. Nobody but you has access to this environment, and thank goodness, because if you’re anything like me it’s usually broken. You use the Local environment to iterate on your own tasks and see the results of your code changes.

Best practices:

• Commit frequently.
• Merge from your team’s master branch regularly.
• Run all of the project’s automated tests before opening a pull request. You have tests, right?
• Avoid merging something back to master if you know it’s broken. Sometimes it might make sense to do so, but be sure to notify your team ahead of time.

Dev

This is the cutting edge of development work. All the latest code lives here. It’s a good place for developers to test new features against mocked data and verify that their code works well with everyone else’s.

Best practices:

• Nobody outside of the product development and engineering department should have access to Dev. Not even QA.
• Dev should never be used for demoing the product. It’s half-baked. It’s totally acceptable for it to be broken at any moment; developers try not to break it, but that’s only for the sake of their own productivity.
• Anyone who complains about Dev being broken should expect to be pelted with celery sticks and Cheez-Its.
• Dev should auto-deploy from the master branch every time there are new commits.

Test

This is the company’s first chance to check for any problems that their developers and automated tests didn’t catch.

Best practices:

• Nothing should be deployed to Test unless it has passed all of the project’s automated tests and a manual sanity check.
• The company should focus its QA resources on the Test environment. A few days before every planned release, people who are intimately familiar with the product should begin regression testing there. From the moment testing begins until the code goes to Stage, the only new code deployed to Test should be bugfixes.
• When bugs are found on Test, they should be promptly reported to a project manager. Anyone and everyone can complain when Test is broken, but Test should not be used for demoing the product.

Stage

Stage is meant to simulate Production in as many ways as possible. It’s one last line of defense against releasing bugs. Stage can also be used to demo new features and to train high-touch clients on upcoming changes to the application.

Best practices:

• Deploying an app to Stage should indicate that it has been approved by the company’s QA team and it is believed to be bug-free.
• If your app works a certain way on Stage, you should be 99.9% certain it will work the same way on Production. One possible way to enforce this is to run Stage and Production side-by-side on the same infrastructure and switch from one to the other when it’s time to release. For example, Microsoft Azure lets you deploy apps to “app slots” and swap any two slots via the command line. Similarly, Heroku lets you move apps from one environment to another using the pipeline interface.
• If possible, the Stage database should be overwritten with data from Production on a regular basis.
• Bugs on Stage should be prioritized similarly to Production bugs.
• Most of the time, Stage and Production should have exactly the same code. If a release is scheduled for the next few days, Stage may be ahead of Production.

Production (live)

Production is the only customer-facing environment. It’s the reason why every other environment exists. As a company grows, Production tends to become more and more sacred. If Production is broken, customers will leave and the company’s reputation for reliability will suffer. Anything that happens in Production is liable to end up in the news, for better or for worse.

Best practices:

• Don’t break Production.
• You’re gonna break Production. When you do, proactively notify your customers, be honest about the situation, fix it as soon as you can, and be prepared to make restitution to anyone who was affected.
• Don’t skip QA on Production bugfixes. Sure, if the situation demands it, you can expedite or trim down your QA process. But do your best to make sure the fix doesn’t break something elsewhere.

Final thoughts

These are standard environments for a large company. If your product hasn’t gone public yet and development speed is more urgent than application stability, you probably only need a couple of these. As your user base grows and becomes less fault-tolerant, you can add environments to match the structure of your organization and the resources at your disposal. Each environment is another place where the right pair of eyes could prevent a bug from reaching Production and affecting users.