From: Jeff Layton <jlayton@kernel.org>
To: kdevops <kdevops@lists.linux.dev>
Cc: Scott Mayhew <smayhew@redhat.com>
Subject: replacing vagrant in kdevops
Date: Thu, 30 Nov 2023 08:34:46 -0500 [thread overview]
Message-ID: <3c40ea8e931f67800304402849dc83b67a02f4a8.camel@kernel.org> (raw)
I spent the last couple of days looking at replacing Vagrant in kdevops:
tl;dr: I think we can replace Vagrant in kdevops with virt-builder from
the libguestfs tools to build guest images, and then just use virsh or
ansible to bring them up and shut them down.
Why Replace Vagrant?
====================
A few months ago, Hashicorp decided to change the licensing on all of
their products to something no longer GPL compatible. Terraform has been
forked to the OpenTF project (which I think we'll want to switch to
eventually), but no one wants to step up to fork the giant Ruby hairball
that is Vagrant. Also, a lot of what makes Vagrant usable is the "box"
ecosystem that surrounds it. That's all likely to wither on the vine
now. Given that, we need to look at alternatives.
The Basic Idea
==============
Scott Mayhew mentioned that we could just use virt-builder (from the
libguestfs project) to build images for kdevops, and then just use virsh
or the libvirt API directly to launch and shut down guests. This means
that we'd no longer support Virtualbox, but I don't think that's a huge
problem for most people.
This involves a few pieces:
Building images
===============
This turns out to be fairly simple with virt-builder
(https://www.libguestfs.org/virt-builder.1.html). I was able to build an
image that is (mostly) suitable with a command like this (I even
preserved the "vagrant" user account but we could rename -- maybe
"kdevops").
virt-builder fedora-38 -o test1.qcow2 --format qcow2 \
--root-password password:'foo' --hostname
test1 \
--install qemu-guest-agent \
--run-command "useradd -m -p vagrant vagrant" \
--ssh-inject vagrant:string:"ssh-ed25519 ...."
There is a bit more work to be done here (still need to allow the user
to run sudo w/o a password), but this is pretty close. virt-builder has
profiles for all of the major distros (at least on x86_64) and we can
add other variants as needed (and potentially contribute them back
upstream).
The _really_ nice thing about virt-builder is that it can install
packages and run things inside the guest image before you ever bring up
the guest. This means that on the first boot, the guest can be more or
less ready to go with little configuration needed (hostname set, user
added, it has an ssh key injected, etc.). You can also make run one-shot
tasks on first boot, which might allow us to cut out some ansible plays,
etc.
Guest Bringup
=============
"virsh create" can take an xml file that describes the guest VM. This is
somewhat similar to a Vagrantfile, but is more readable and doesn't
require Ruby. We can just create a template, and use ansible to fill it
out and spin up the guests.
Updating ssh config
===================
This is where things get a bit more difficult. Currently we use "vagrant
ssh-config" to figure out the IP addresses of the spawned guests, but
there is no canonical method for doing this with straight libvirt/qemu.
You can try to query libvirt's "net" DHCP table, but in a
qemu:///session configuration, the user might not have access to do
that.
My proposal here is to just ensure that virt-builder installs and runs
the qemu-guest-agent in all of the guests prior to the first boot. This
allows libvirt to query guest directly over a private channel for info
including the IP addresses.
Most major distros package this program, so I'm assuming this is not a
major problem.
Implementing it
===============
The tricky bit is putting all of this into place. I started adding a new
guestfs directory and trying to build that out, but it duplicated too
much code.
Instead, what I think we should do is just add a new CONFIG_GUESTFS
boolean, and use that to make kdevops do things differently in the
existing Vagrant codepaths when that's set. It may also add some new
options for the distro handling).
Then eventually, we can just kill the Vagrant-specific parts once
everything is in place.
RFC
===
Sound reasonable? Any other thoughts? If anyone has objections to this
approach (or wants to suggest a better one), then please air them before
I start ripping things up!
Cheers!
--
Jeff Layton <jlayton@kernel.org>
[1]: virt-builder is part of the long-established libguestfs project:
https://www.libguestfs.org/virt-builder.1.html
reply other threads:[~2023-11-30 13:34 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3c40ea8e931f67800304402849dc83b67a02f4a8.camel@kernel.org \
--to=jlayton@kernel.org \
--cc=kdevops@lists.linux.dev \
--cc=smayhew@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).