Dmitry Mazin
"cyberdemon.org is a cool domain"
home / email me / mastodon / RSS feed / Telegram channel
Lab Notes: Installing Docker Engine using Ansible
Hello! It is once again 2 PM. This morning, I took Eamon to the Natural History Museum. It’s an OK museum. This is the museum we visit least because there isn’t much there that captures Eamon’s fancy. Being two, he’s mostly into tactile stuff and being able to run around. That, and machines. Oh, does he love machines. I don’t mean trucks and stuff, though he loves those too. He loves the mechanical: clocks, robots, computers, and, especially, steam engines.
Anyway, back to computers. Yesterday, I ran a sysbench MySQL benchmark against a MySQL instance with capped resources (1 CPU and 1 GB memory). I also started to set up Ubuntu on an old MacBook I had laying around. Today, I want to get Docker running on that machine and run another benchmark. Today’s goal for the benchmark is to play with throughput: how many queries per second can I process? How will that affect the p95 request duration?
Meta-note
A meta-note: I understand that these notes are nearly unreadable (and also probably not interesting to anyone). I can do something about readability: today I’m going to try breaking the notes up into clear sections. With regards to them being uninteresting, I do wonder why I’m publishing them at all! The reason is that, when I write them, I am imagining an audience (even though that audience probably doesn’t exist), and that helps motivate me and also forces me to make my thinking a bit more clear.
Installing Docker Engine on Ubuntu using Ansible
Docker provides instructions for Ubuntu installation, which boil down to trusting their GPG key, adding their repo, and installing. This is a good excuse to learn a bit about apt’s sources.list and why/how you use GPG keys to indicate trust.
Indicating trust of a repo
When you add a repo to sources.list, you can add a signed-by field. This field is described in the man page for sources.list: “require a repository to pass apt-secure verification with a certain set of keys”. So, then, what is apt-secure? According to the man page for apt-secure, it’s a way to check that a repo is controlled by who you think it’s controlled by. In other words, it’s a way to verify that the repo is trustworthy.
Let’s look at Docker’s instructions for adding its repo to explain the above.
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
Docker advertises a public key at https://download.docker.com/linux/ubuntu/gpg. We download it, and store it in /etc/apt/keyrings (indeed, that’s where the sources.list manpage recommends sysadmins put keyrings). What gpg --dearmor -o does I’m not sure, but clearly it’s storing the keyring.
Next, in the instructions we create a file /etc/apt/sources.list.d/docker.list that will configure the repo. The contents are:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
See the signed-by? So, we’re saying that stuff we download from this repo needs to match that key.
(Side note: note how, like yesterday, we pipe the output into sudo tee? I now understand why. This is so that we can do the echo command without sudo, and then take the output and then use sudo only for writing the file. This is neat, and smart. It restricts escalation to the only part that actually needs it, that is, modifying /etc/apt/sources.list.d).
What happens if we don’t specify signed-by?
Actually, I’m interested in a little experiment. What will happen if I add the deb entry without signed-by? Is this repo trusted blindly? I’m going to guess no. If signed-by isn’t provided, the repo will be checked against some other set of keys, fail to match, and there will be an error or something.
Let’s try it.
echo \
"deb [arch=$(dpkg --print-architecture)] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
And then sudo apt update.
Aha! Yes, it refused to do so.
W: GPG error: https://download.docker.com/linux/ubuntu jammy InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
E: The repository 'https://download.docker.com/linux/ubuntu jammy InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
Note that this failure happened during apt update, not during the install step. This is because apt-secure verifies the repo, not packages.
If I install the keyring and reference it in docker.list, then apt update works fine:
Get:4 https://download.docker.com/linux/ubuntu jammy InRelease [48.9 kB]
Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease
Get:6 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages [13.6 kB]
Converting Docker’s instructions to an Ansible playbook
DigitalOcean has a tutorial for installing Docker Engine using Ansible, and it’s straightforward: to store the gpg key, we use the apt_key module, and to add the repo, we use the apt_repository module.
Is Ansible’s apt_key module safe to use?
I’m curious, though, what apt_key actually does. This is important because there are some completely insecure ways to install a key. For example, if you install a key into /etc/apt/trusted.gpg or /etc/apt/trusted.gpg.d, then it will be trusted for all repos (which would mean that the key holder could trick you into installing malware).
Aha! In the source, it’s clear that apt_key uses the deprecated apt-key utility. It’s been deprecated for the reason I gave above.
Avoiding apt_key
I’ll have to replace those steps with other commands. To download the key, I can use get_url, but what about the gpg --dearmor part? Looks like I’ll have to figure out what it does after all.
This is what https://download.docker.com/linux/ubuntu/gpg looks like:
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBFit2ioBEADhWpZ8/
[bunch of crap]
-----END PGP PUBLIC KEY BLOCK-----
Apparently, I need to “dearmor” this? Let’s try that and see what the output looks like.
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor
X��*�Z�|�
���s�@u�V����-��u�o�a��2�a����M�����_T�R`��
�弲)5���䷎ڐ�X��o^��U���.-��Ӿ��,P��An��L�
Binary!
OK, so, after a bit of reading, it seems that “armored” ASCII PGP keys are the old format, and the new format is binary. Anyway, I’ll need an extra step to dearmor the key. (There is much discussion about this for the apt_key module, with unhappiness that we need two commands to properly install a key).
Actually, sounds like I can just reference the ASCII version in the docker.list, so I’ll just do that.
- name: Ensure that the /etc/apt/keyrings dir exists
file:
path: /etc/apt/keyrings
state: directory
group: root
owner: root
mode: 0755
- name: Add Docker GPG key
get_url:
url: https://download.docker.com/linux/ubuntu/gpg
dest: /etc/apt/keyrings/docker.asc
- name: Add Docker repo
vars:
architecture_map:
aarch64: "arm64"
x86_64: "amd64"
apt_repository:
filename: docker
repo: 'deb [arch= signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu stable'
- name: Install Docker stuff
apt:
pkg:
- docker-ce
- docker-ce-cli
- containerd.io
- docker-buildx-plugin
- docker-compose-plugin
(Kudos to Eric Zarowny for the nifty architecture_map conversion. Otherwise, I’d have been lazy and just hardcoded amd64).
Hooray! Another day where doing the simplest thing was difficult, but at least I learned something.
Oh yeah, also I want to install GH CLI so I can clone the repo easily.
The task looks a lot like the Docker one, except following GH CLI’s official instructions.
Can I actually run the benchmark now, please?
No! I ran out of time. Well, really what happened is that my kid cut his hand while peeling a carrot.
Tomorrow, I will actually run the benchmark.