From automation to immutable infrastructure

When you are starting your company, especially IT company, and its focus is on developing some software product, you probably do not put much thoughts and time into planning and building your network and server infrastructure. 

And why should you? It is not easy, it takes time, knowledge and experience and it is not important for your development. In a small team, where you can trust everybody and everybody knows everything about infrastructure, it is faster, cheaper and easier to just roll with it.

Planning and building your infrastructure will take time and resources that can never pay off if your company will not be successful, so it is basically not a wise investment.

It is all fine until you hit certain point when your company kicks off and you need to employ more people. More people mean more security risks and harder information flow between them. So you decide to hire a system administrator.

And he will start to set up your infrastructure. But your infrastructure is in poor condition, there are lot of problems and every old and new employee has problems, that need to be solved. So there is not much time to do things right and system administrator is just patching one problem after another as quickly as he can.

To prevent stuff from getting worse, you and your system administrator or system administrators team needs to set up some standards to make the maintenance as easy as possible.

First approach is to create standards and just distribute them to everybody and hope that they will hold them up. Good luck.

Better approach is to use some automation. That means to make scripts that set up your infrastructure automatically. This eliminates human errors, makes everything standardized automatically and is easier to maintain and to change later.

But why should you write your own scripts? It is hard, it takes time and it is harder to maintain. Just use some automation tool. There is a lot of them. I use Ansible.

There is not a lot of dependencies, it just uses SSH, so you do not need to install client software. All you need is Python 2 installed.

Now, instead of writing a lot of bash and other scripts, you write (mostly) Ansible playbooks. They are simpler, easier to read and easier to maintain.

After you have automatized your infrastructure configuration, you can go next level and make it immutable. What does that mean? It means that you only use your automation tool to setup your infrastructure and any changes made directly on your servers or other devices are reversed automatically to desired state defined in your automation tool.

This gives you total control over your infrastructure. At any given time you know how is any device on your network set up, who has access to what, what are the latest versions of software and configurations installed etc.

Why should you want that? You need fewer people to take care of your infrastructure, you do not need to create (almost) any documentation about the state of the infrastructure, you can easily let your system administrator leave without loosing any capability to maintain your infrastructure or (almost) without fear of it being compromised and that is just begining.

So, please, automate.