Ansible Self-Apply

What is Ansible Self-Apply?

Ansible is a Configuration Management tool, normally used to setup one or more remote servers from a laptop or similar. Self-Apply is using the same Ansible tool and code, but to configure the machine we're running on. With self-apply, Ansible runs on the machine to be managed, and "connects" to it via a local connection rather than making SSH connections over the network.

For self-apply to be possible, you have to put the Ansible code onto the server you want to manage, and you'll need to install Ansible itself too.

Why do you want Self-Apply?

As a general rule, system management and configuration should be centralised, so having a series of machines self-applying might seem like a step backwards. However, self-apply doesn't preclude remote (centralised) management, and it has some great advantages, especially in cloud environments or where servers are deliberately created and destroyed relatively frequently. If nothing else, self-apply can be used to setup all the necessary basics on a server so it is ready for centralised management (perhaps installing agents or SSH keys or other). It may also have a place in particularly big estates because it scales endlessly, but that's perhaps a matter of choice.

If you have servers being created and destroyed, then getting them to initially boot up to a fully working state is likely to be either absolutely necessary or else at a minimum, desirable. Not all software stacks allow for all machines in a cluster to be absolutely identical, so simply booting the same system OS image may not be sufficient. Ansible self-apply gives you a clean, standardised way to apply the differences you need between servers quickly and easily.

One other thing to note is that self-installing servers offer an easy Disaster Recovery solution because re-installation onto a new server is completely "hands off". They also offer patching options; that is, when patching is required it may be possible to simply destroy the server and recreate it - the new server gets all the latest OS images and packages and simply installs the required software on top of it.

Getting Started

You can use Ansible self-apply a variety of ways, and what we describe here isn't the only way, and may not always be the best way. You can get your Ansible code onto your servers via any sort of deployment means you have, although we like this method because it works in most circumstances (including going further than we talk about here).

Our chosen method of deployment is to wrap our Ansible code into an OS package (eg. Apt .deb or an RPM). These packages need to go into some sort or "repo" so we can install them - for the purposes of this article, we'll just assume that these sorts of facilities are already available.

Making packages is something of a "black art". It's simplified a great deal by using FPM. It's essentially "zip" for packages, but if you want to go crazy you can do all sorts of package-specific things too.

We'd very much recommend getting a Continuous Integration (CI) task together to "zip" up your Ansible code into a package (and to push the package to your repo). That way, every time someone merges their code, you get a shiny new package which you can deploy onto your self-apply servers.

Running Ansible

To recap, we've now got a server with Ansible code on it, which in our case arrived by OS package. We're now ready to run Ansible... well, nearly.

If your Ansible code will only ever be used to install one type of server, then you're about done - you can just run Ansible and it'll do just fine. On the other hand, if you want to use your Ansible code for more than one type of server, then you probably need to do some hacking about.

Ansible is designed to work on remote servers. Working locally works, but you end up doing some of the things yourself that it would normally do for you. The main things to do are setting groups and variables.

Setting groups is normally done via the inventory, but there's no inventory of any use in self-apply. You can't set the ansible_groups variable, but you can set something else via -e or --extra-vars on the command line when you call Ansible. If that all sounds like a faff, well, it is - but see further on for some ways out of it.

Setting variables isn't quite as easy as it is remotely either, but the include_vars task can help. You can add in a peppering of stat to only load variable files that exist, so end up with something sort of like the remote counterpart. For example:

- name: Look for variables...
  ansible.builtin.stat:
    path: "{{ item }}"
  with_items:
  - "{{ playbook_dir }}/group_vars/all.yml"
  - "{{ playbook_dir }}/group_vars/{{ ansible_distribution }}.yml"
  - "{{ playbook_dir }}/group_vars/{{ ansible_distribution_version }}.yml"
  register: vars_files_found

- name: "Load variables..."
  ansible.builtin.include_vars: "{{ item.item }}"
  with_items: "{{ vars_files_found.results }}"
  when: item.stat.exists

The above will then try to load group_vars/all.yml, then something like group_vars/Ubuntu.yml and finally group_vars/22.04.yml. This means we can now install a range of different operating systems with one Ansible code repository.

This is all fine, but we want to use the same Ansible code repository to install multiple different machine types, not just different operating system types. Normally we can just put them into different Ansible Groups, but with self-apply that's not possible. Instead, we can specify the 'groups' we want the current machine to be in on the command line when we run Ansible.

Additionally, we're going to want to have different playbooks for different groups, so we need to augment group_names which doesn't get set automatically in self-apply.

For example:

ansible-playbook site.yml -e selfapply_groups=webservers,dev

Next, we have to use selfapply_groups to load variables for specific groups. We can develop the above idea into this:

- name: Look for variables...
  ansible.builtin.stat:
    path: "{{ playbook_dir }}/group_vars/{{ item }}.yml"
  with_items: "{{ ['all', ansible_distribution, ansible_distribution_version] + selfapply_groups | default('') | split(',') ] }}"
  register: vars_files_found

- name: "Load variables..."
  ansible.builtin.include_vars:
    file: "{{ playbook_dir }}/group_vars/{{ item.item }}.yml"
  with_items: "{{ vars_files_found.results }}"
  when: item.stat.exists

- name: "Put host into Ansible groups"
  ansible.builtin.add_host:
    name: localhost
    groups: "{{ ['all'] + selfapply_groups | default('') | split(',') }}"

The complicated with_items that looks for variables steps through something like the following:

• all
• Ubuntu
• 22.04
• webservers
• dev

If we forget to set selfapply_groups then it defaults to having no groups except the operating system ones (like the first example). We use the same sort of code to build the list of Ansible groups too.

Going further, we can avoid the chances of us forgetting to set the extra variables when we run Ansible by giving ourselves a little script. This script needs to have the extra variables in it and then we run that instead of running Ansible itself. The good news is that we can use Ansible itself to save out that little script, using the selfapply_groups variable as a template value. A script template something like this:

#!/bin/bash

cd /opt/ansible-code
ansible-playbook site.yml -e "selfapply_groups={{ selfapply_groups }}"

Then template out that file to (say) /usr/local/bin/run_ansible and you have your script. You could take it a stage further and allow for --tags type additions to the command line.

Wrapping Up

Now we have some self-apply working, we can use it to auto-build servers by using their user_data to run our Ansible on boot. You can even take it a step further and start building OS images (eg. AMIs) with some basic Ansible code already in place in them (possibly using that to perform the basic hardening and installing organisation-wide tools and policies in the image itself). Then user_data needs to update to the latest Ansible code and run that (with the right groups specified).

Conclusions

We can make self-building servers with Ansible self-apply. This does involve 'bending' Ansible a little to our will, but it has all the features we need to be able to do it. Once mastered though, self-building servers offer hands-free operations, disaster recovery solutions and can help in patching and management work too.

Pre-Emptive can help you get to operations nirvana, can clean up your disaster recovery capabilities, patching strategies and much more. Please contact us - we can help you figure out what you need and make it work for you.