Fedora CoreOS, the new operation system developed by the Fedora community and Red Hat, finally left the preview phase. As a user of the now deprecated Atomic Hosts (available for Fedora, CentOS, Red Hat Enterprise Linux) a good opportunity to try it out and give some hints how to configure your first working system.
What is it?
If you never heard about the Fedora CoreOS project, its predecessor Atomic Host or CoreOS Container Linux before, you may wonder what this new operating system is about:
Fedora CoreOS (abbreviated as FCOS) embraces the idea of an immutable operating system, specifically designed to enhance configuration consistency of multiple hosts in a distributed system and with containers as the primary building blocks to run applications and services. Therefore, container tooling like Kubernetes, Podman and Docker plays a large role in this approach.
When running a FCOS instance, the most important concept to understand is that the file system is now immutable, it is read only. To be precise: From the well known Linux file system only two folders are writable -
The idea of an immutable file system allows the introduction of a new concept: the so called OSTree. An OSTree represents a complete bootable file system tree that can be updated in an atomic or transactional manner. Instead of updating individual packages on the system by writing to the existing file system, a complete new version of the file system tree is checked out. OSTrees and the corresponding tooling therefore act like a version control for the whole operating system. This concept also enables the possibility to completely rollback the system to a previous working version in case something should go wrong.
Since, apart from a few exceptions, the file system is immutable, how to deal with applications and services I want to use? The answer is in most cases: utilizing containers. FCOS is shipped with support for Podman, Docker and the CRI-O container runtime out of the box. How this works in practice and further possibilities are topics of an upcoming article.
While FCOS is designed as a server operating system for single hosts or distributed systems, the concepts mentioned above can also be found in another project of the Fedora community: Fedora Silverblue. This is an immutable operating system with a container based workflow as well, but with the focus for desktop and workstation environments. It might be worth to check this out as well.
After this rough introduction of the basic concepts, in the following sections of this article I will explain in more detail how to setup a first working virtual machine with FCOS.
FCOS, in contrast to classic operating systems, has no install-time configuration. Instead, a generic disk image is provided, which can be configured during the initial boot process. FCOS supports deployments on bare-metal machines, in a cloud environment (e.g. using services like Amazon EC2) and on local virtual machines, with corresponding possibilities to provide the initial configuration settings.
While the former Atomic Host project used cloud-init as their configuration system, Fedora CoreOS is now using Ignition, which originated from the CoreOS Container Linux project.
Ignition itself is a low-level provisioning tool that allows for early-boot configuration. The essential part is a configuration file which is read during the initial boot process. In the case of Fedora CoreOS this file is not written directly; a transpiler from the Fedora CoreOS Configuration Format (FCC) to the Ignition file format is used instead. If you were a user of CoreOS before, you maybe already know the corresponding transpiler ct. The FCC file itself is written in yaml syntax.
Writing the FCC configuration
In this section I will go through some basic options available. For a full reference please take a look at the official documentation.
FCC allows basically three types of configuration options:
- Users and groups
- File system
- Systemd units
The minimal ignition configuration, along with some required meta data, needs to have information about the initial system users enabled to log into the system. Fedora CoreOS provides an initial user
core which can be used to administrate the system. If you want to use this user the minimal configuration a least needs to assign a corresponding login password or an ssh-key.
An example could look like this:
variant: fcos version: 1.0.0 passwd: users: - name: core ssh_authorized_keys: - "ssh-rsa AAAAB3Nza...." - name: matthias ssh_authorized_keys: - "ssh-rsa AAAAB4adsa...." password_hash: "$6$rounds=4096$yspKBgKn..." groups: - wheel
Here the system
core user has a provided public ssh-key in the
ssh_authorized_keys section and is therefore enabled to log into the system via ssh. The additionally created user
matthias has a
password_hash configured, so regular password based login is possible as well.
Obviously, allowing a regular password based login requires for a safe hash method (do not choose MD-5!). This can be done with the regular Linux tooling like:
mkpasswd --method=SHA-512 --rounds=4096
The hashed password can then be used for the FCC configuration. The example configuration also assigns the user
matthias to the
wheel group to allow privileged operations.
Apart from these standard settings, of course more sophisticated options like setting up system accounts without home directories or configuring the used login shell are available as well.
File System Settings
storage section allows for various configuration of disk devices, creation of raids, handling of directories, files and symbolic links.
Ignition allows setting up storage during the configuration phase. This includes partitioning, formatting and mounting of additional disk devices and corresponding file systems. To do this, first a list of disk devices has to be created. The absolute paths to the devices should usually be provided using the /dev/disk/by-… symbolic links, which are consistent and not prone to change. A list of partitions for each device can then be defined, with further options to wipe partition tables and pre-existing partitions.
Creating file systems on the configured disk devices is then easily done via listing them in the corresponding
filesystems section. Parameters here include the device path, the desired file system and further settings like labels.
It is worth mentioning that, although Ignition can handle various file systems like ext4 and xfs, more sophisticated options like setting up LVM is not possible.
A minimal example of setting up storage could then look like this:
variant: fcos version: 1.0.0 storage: disks: - device: <path to disk device> wipe_table: true partitions: - label: <unique partition label> number: 0 wipe_partition_entry: true filesystems: - path: <mount path of device during configuration> device: <path to disk device> wipe_filesystem: true format: ext4 label: <unique file system label>
Extended Configuration - Hostname
Ignition provides no possibility to run arbitrary shell commands during the configuration process. Therefore, the usual way to set the hostname using
hostnamectl on systemd controlled systems is not working out of the box. There are two solutions to circumvent this constraint:
- Setting up a oneshot systemd service to execute a command
- Writing to the corresponding configuration file directly
In this example I will go with the second solution, which looks like this using the storage section of the configuration file:
variant: fcos version: 1.0.0 storage: files: - path: /etc/hostname overwrite: true contents: inline: <your-hostname>
The systemd section of the FCC configuration allows the configuration of systemd units. It is possible to create new units as well as modifying existing ones with e.g. drop-in configuration files.
To refer back to the previous example, creating a new oneshot service to set up the hostname during the first boot can look like this:
variant: fcos version: 1.0.0 systemd: units: - name: set-hostname.service enabled: true contents: | [Unit] Description=Set the hostname [Service] Type=oneshot ExecStart=/usr/bin/hostnamectl set-hostname your-hostname [Install] WantedBy=multi-user.target
In this section I will go through a complete example to create a running virtual machine with Fedora CoreOS.
Example FCC / Ignition Configuration
The complete example configuration can look like this:
variant: fcos version: 1.0.0 passwd: users: - name: core ssh_authorized_keys: - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDu4cjMWNFUz4YV9ooeoaBSwPhdoM9x8cqiWUrVhlbH24ixNyCgcat7cN1kbraIuTQTqymrfUG3rqQdjyCbiIRd0gafS7hEQ9NSh1gcVQVQAzNuvJvcTrrWstXdktM41NfX/K7GcGy4u4gN8mEP6WHytEcpazh3N5TgjeJkEpzgUXi0isyfBXLKxyYGQrFzD+QTjoeM8DvKHnJFvU+WF4SPWWDyhB23mXPDBwl9XpQaQmuL1/1nkhkEComq2igr30rr2Uhyw4tiworVel0mWOByKBIPATjXdXYbS/nUzspiQ0F1kOdYv/EOeeb2X9hZCHWhefdbQCbI1NfTkjbZFmFMTBfGX7e5wDNJpY9OdTacPGTN0PKV1XtaY68S5nYOi2jqj7l2iPaL+esMSdCllBGAdBZV6/PJ511dRzpxk/CunVUi/3oQORlnP02ip0WY9HmPdqfdoq5+jfIE/iB6jbSEW93tkMeeNaXbjeO2DSq9ihG7RAyPMPQNDXi/DHMnw6fzk5MNvBi14iXHJiykI0vG4yfYvgRA0ies8/3CH/a6SRwo+DKKm10zsDN5HM76kamlWx6Nd+Eq2Hx6q6InMkIsbjp5Aya2XSrLGXAI1RKPQEk0GKx0OP5+d4Npgbg/EbM0zUiFmpszFYlxfpCVqdc9lVMmKlaMe9HsRUXx7slq3w==" - name: matthias ssh_authorized_keys: - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDu4cjMWNFUz4YV9ooeoaBSwPhdoM9x8cqiWUrVhlbH24ixNyCgcat7cN1kbraIuTQTqymrfUG3rqQdjyCbiIRd0gafS7hEQ9NSh1gcVQVQAzNuvJvcTrrWstXdktM41NfX/K7GcGy4u4gN8mEP6WHytEcpazh3N5TgjeJkEpzgUXi0isyfBXLKxyYGQrFzD+QTjoeM8DvKHnJFvU+WF4SPWWDyhB23mXPDBwl9XpQaQmuL1/1nkhkEComq2igr30rr2Uhyw4tiworVel0mWOByKBIPATjXdXYbS/nUzspiQ0F1kOdYv/EOeeb2X9hZCHWhefdbQCbI1NfTkjbZFmFMTBfGX7e5wDNJpY9OdTacPGTN0PKV1XtaY68S5nYOi2jqj7l2iPaL+esMSdCllBGAdBZV6/PJ511dRzpxk/CunVUi/3oQORlnP02ip0WY9HmPdqfdoq5+jfIE/iB6jbSEW93tkMeeNaXbjeO2DSq9ihG7RAyPMPQNDXi/DHMnw6fzk5MNvBi14iXHJiykI0vG4yfYvgRA0ies8/3CH/a6SRwo+DKKm10zsDN5HM76kamlWx6Nd+Eq2Hx6q6InMkIsbjp5Aya2XSrLGXAI1RKPQEk0GKx0OP5+d4Npgbg/EbM0zUiFmpszFYlxfpCVqdc9lVMmKlaMe9HsRUXx7slq3w==" password_hash: "$6$rounds=4096$6MT9hwjAovop2bAq$bw/29axXXBX.l0NP7igGK67BbCU4t89UOmCHIkUS5nV9M7fgU4tlFJjRZrRYRF14hRgwF8jpCh1hlpqL5CLkJ/" groups: - wheel storage: disks: - device: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_hd1 wipe_table: true partitions: - label: "disk.storage" number: 0 wipe_partition_entry: true filesystems: - path: /var/storage device: /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_hd1 wipe_filesystem: true format: ext4 label: STORAGE files: - path: /etc/hostname overwrite: true mode: 0644 contents: inline: fcos-server-test systemd: units: - name: var-storage.mount enabled: true contents: | [Mount] What=/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_hd1 Where=/var/storage Type=ext4 [Install] WantedBy=local-fs.target
So what is happening here:
matthias, are configured to be able to log in the system with ssh-keys, while the latter is also able to log in with a regular password (which here is
test for demonstration purposes). The user
matthias is also added to the
The ssh-keys in this section were created with:
ssh-keygen -t rsa -b 4096
The generated public key is then used in the configuration file.
In the first part of the
storage section an additional hard disk is configured. The corresponding device is used by its id and all partition tables will be wiped during the setup. Then the hard disk is partitioned with one partition named disk.storage and formatted with an
ext4 file system. Furthermore, this section configures the hostname of the Fedora CoreOS instance with the method explained above.
Finally a systemd mount unit, which mounts the previous created file system on the additional hard disk at the location
/var/storage, is configured.
Generating the Ignition File
In the next step it is necessary to run the FCC transpiler to create a valid Ignition file. Instead of downloading the transpiler I will use a provided container image, which makes the process somewhat easier. Running the transpiler can then be done via
docker) with the following command:
podman run -i --rm quay.io/coreos/fcct:v0.2.0 -pretty -strict < fcos.fcc > fcos.ign
Launching a VM with FCOS
To get the configuration example running I use the provided Fedora CoreOS qcow2 images and
QEMU to create a virtual machine.
As the FCC configuration is using an additional hard disk, first I have to create the corresponding image to be used with
# Creating a one gigabyte hard disk qemu-img create -f qcow2 storage.qcow2 1G
Then I can start the virtual machine with the Fedora CoreOS image, the additional hard disk and the Ignition configuration file:
qemu-system-x86_64 -machine accel=kvm \ -m 2048 -cpu host -nographic \ -device virtio-rng-pci \ -device virtio-scsi-pci,id=scsi \ -drive file=fcos.qcow2,if=none,id=hd0 -device scsi-hd,drive=hd0 \ -drive file=storage.qcow2,if=none,id=hd1 -device scsi-hd,drive=hd1 \ -fw_cfg name=opt/com.coreos/config,file=./fcos.ign
Note that the hard disks are configured to use the virtio-scsi storage backend along with the ids
hd1. Using this parameters, QEMU automatically makes the additional storage hard disk available via the symlink
/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_hd1, which is part of the FCC configuration file above.
After a short wait time you should the be able to log into the system with the user
matthias and the password
test. Running the shell command
hostname should reflect the correct value and you should be able see the mounted additional storage file system at
/var/storage as well.
A full example bash script for all tasks mentioned is provided here.
- CRI-O container runtime
- Cloud-init documentation
- CoreOS config transpiler
- CoreOS project website
- FCC documentation
- FCC transpiler container image
- Fedora CoreOS Configuration (FCC) format
- Fedora CoreOS Configuration (FCC) transpiler
- Fedora CoreOS downloads
- Fedora CoreOS
- Fedora Silverblue
- Github gist - Example script to setup FCOS VM
- Ignition configuration file format
- Ignition documentation
- Persistent block device naming
- Project Atomic
- RPM-OSTree documentation
- Systemd service types