Systems Management Automation

Automation

We've recently been exalting the virtues of automation in systems management, particularly standardising on a small number of automation tools and using them for all aspects of systems management.

Automation is pretty much a necessity in any modern IT estate. Applications are more numerous, more complex, more interconnected than ever before. We also run fleets of servers where maybe we'd run one or two a decade ago. With the advent of virtualisation and cloud computing, we can create, use and destroy virtual servers in minutes, as demand grows or falls. There's simply no way to keep up with all that without extensively making use of automation.

Everything As Code

Moving to a world where your infrastructure and/or systems configuration is expressed in code (rather than some mouse clicks or manual commands run on a terminal, even if they're carefully documented) offers a series of benefits:

The code you write should be checked into a Version Control System (VCS) such as Git. This provides history and accountability (you can see how the code looked in the past, and who made changes to it). It also provides a means to recover your systems if they are completely lost - the code provides Disaster Recovery options because it can be replayed in a new environment to recreate what was made previously.

As a general rule, writing code requires a little more skill over and above the equivalent mouse-clicking (or manual commands) method. It usually take a little longer to write, but encourages "doing things right", rather than giving in to time pressures and doing them quickly, but in a way that presents issues later on.

Most "as code" tools have corresponding Static Code Analysis tools available. These tools read the code and try to determine if it will create a secure, maintainable and complete resource when it is applied to the infrastructure. This can really help improve the quality of the estate, increase security and provide a good deal of confidence in any audit or compliance activities that may take place in the future. If nothing else, they also provide the systems administrator with a great "double-check" of their work, and possibly some confirmation/improvement of their abilities!

Infrastructure Automation

Creating, destroying and generally managing your infrastructure (particularly in the cloud) is deceptively simple with a mouse, clicking around a UI. However, there's no disaster recovery, no peer reviews, no audit trail and no history if you work that way (see above!)

Tools such as Terraform mean you can setup your infrastructure (servers, networks, containers, serverless, databases etc) from some code.

Using tools such as Terraform encourages reuse. That is, if you create a server for one application, it encourages reusing that same code to create another server for another application. As such, the time spent on the first one to get all the configuration, security and compliance just right then directly benefits the second and any subsequent instances of it.

We generally find that in an Infrastructure As Code environment, the levels of security and quality tend to increase over time. That is, over time additional work on the infrastructure obviates the need to go back to older elements and "bring them into line" with newer elements (again, code re-use helps here). This gradual improvement tends not to happen so obviously in manually created environments.

Terraform code can be assessed by a variety of Static Code Analysis tools, which can look for security problems, style issues and general quality issues. Using such tools can really accelerate the improvement of infrastructure and security quality over time. Such tools are aware of many of the subtle, but common mis-steps that administrators make, so can avoid making some of the more common mistakes that befall the unwary.

Operating Systems and Applications Configuration Automation

Once you've got your servers and networks up and running, you're going to need to install an operating system, configure it and then install some applications. Again, automation helps out here, as it allows you to build systems from 'bare OS' right up to a fully running system quite easily. What's more, if you need to do multiple similar servers, you can do so knowing that they'll all be exactly identical (even if they need config specific to each machine).

Automation tools to do this sort of thing include Ansible, Puppet, Chef and Salt. We tend to prefer Ansible because it's one of the simplest and easiest to get started with (it has very few pre-requisites to use it), plus you can use it anywhere you can get an SSH login (which presumably is most, if not all of your estate).

Whichever tool you use, they all provide the ability to write code to install the software you need, template out the configuration files you need and do restarts/reloads or whatever else are required to make it all work. They all have ways to get information off one system and into another too, so if you need to share a password or some other values they can do that for you.

Aside from the general benefits of using code to do this sort of work, Configuration Management tools also offer these advantages:

The tools themselves give you options to check code without actually applying it, or to re-apply the code, only making changes where they're needed. That gives you confidence that your systems are still in the state you want them to be, or that changes are made in a timely manner. Along side standardisation and code review tools, these are the main reasons to use formal CM tools rather than crafting your own Bash scripts or whatever - those script will never achieve the same levels of consistency over time, and general compliance as a proper CM tool.

Automation tools also allow you to control your estate in an 'ad-hoc' manner. That is, if you need to make a change, or want to restart an application on a handful of servers, the automation tool can do all this for you - there's no need to log on to any servers in turn to do things. Most tools will even parallelise those sorts of operations, so they'll all happen at once too.

Even if you're only looking after a small handful of servers, an automation tool makes looking after them far easier than trying to do it all manually. Using some Static Code Analysis tools on your CM code can also really help to ensure you're doing things securely, and maintainably.

Conclusions

Moving away from manual installations (or even Bash script installations) to using standard tools to create infrastructure and/or manage the configuration of the operating system and applications within it yields a number of significant advantages. These advantages are, for the most part, no longer optional in any modern IT estate as modern estates are now too large or complex to operate successfully without automation tools.

The opportunities of running things "as code" for peer review, code analysis and continuous improvement are many. Whilst writing the code does require a little additional skill and time, it obviates the need for better quality and consideration of security and maintainability.

Automation tools also offer ways to control the estate in 'ad-hoc' ways. This can simplify or reduce repetitive tasks and can ensure more consistency across devices.

If you'd like to automate just about anything, Pre-Emptive can help. Contact us to see how we can help you.

Image credit: https://flic.kr/p/HAxUeQ