In July the Atomic Working Group put out the first and second releases of Fedora 26 Atomic Host. This article shows you how to prepare an existing Fedora 25 Atomic Host system for Fedora 26 and do the upgrade.
If you really don’t want to upgrade to Fedora 26 see the later section: Fedora 25 Atomic Host Life Support.
Preparing for Upgrade
Before you perform an update to Fedora 26 Atomic Host, check the filesystem to verify that at least a few GiB of free space exists in the root filesystem. The update to Fedora 26 may retrieve more than 1GiB of new content (not shared with Fedora 25) and thus needs plenty of free space.
Luckily Upstream OSTree has implemented some filesystem checks to ensure an upgrade stops before it fills up the filesystem.
The example here is a Vagrant box. First, check the free space available:
[vagrant@host ~]$ sudo df -kh / Filesystem Size Used Avail Use% Mounted on /dev/mapper/atomicos-root 3.0G 1.4G 1.6G 47% /
1.6G free means the root filesystem probably needs to be expanded to make sure there is plenty of space. Check the free space by running the following commands:
[vagrant@host ~]$ sudo vgs VG #PV #LV #SN Attr VSize VFree atomicos 1 2 0 wz--n- 40.70g 22.60g [vagrant@host ~]$ sudo lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool atomicos twi-a-t--- 15.09g 0.13 0.10 root atomicos -wi-ao---- 2.93g
The volume group on the system in question has
22.60g free and the
atomicos/root logical volume is
2.93g in size. Increase the size of the root volume group by 3 GiB:
[vagrant@host ~]$ sudo lvresize --size=+3g --resizefs atomicos/root Size of logical volume atomicos/root changed from 2.93 GiB (750 extents) to 5.93 GiB (1518 extents). Logical volume atomicos/root successfully resized. meta-data=/dev/mapper/atomicos-root isize=512 agcount=4, agsize=192000 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1 spinodes=0 rmapbt=0 = reflink=0 data = bsize=4096 blocks=768000, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 768000 to 1554432 [vagrant@host ~]$ sudo lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool atomicos twi-a-t--- 15.09g 0.13 0.10 root atomicos -wi-ao---- 5.93g
lvresize command above also resized the filesystem all in one shot. To confirm, check the filesystem usage:
[vagrant@host ~]$ sudo df -kh / Filesystem Size Used Avail Use% Mounted on /dev/mapper/atomicos-root 6.0G 1.4G 4.6G 24% /
Now the system should be ready for upgrade. If you do this on a production system, you may need to prepare services for downtime.
If you use an orchestration platform, there are a few things to note. If you use Kubernetes, refer to the later section on Kubernetes: Upgrading Systems with Kubernetes. If you use OpenShift Origin (i.e. via being set up by the openshift-ansible installer), the upgrade should not need any preparation.
Currently the system is on Fedora 25 Atomic Host using the
[vagrant@host ~]$ rpm-ostree status State: idle Deployments: ● fedora-atomic:fedora-atomic/25/x86_64/docker-host Version: 25.154 (2017-07-04 01:38:10) Commit: ce555fa89da934e6eef23764fb40e8333234b8b60b6f688222247c958e5ebd5b
In order to do the upgrade the location of the Fedora 26 repository needs to be added as a new remote (like a git remote) for
ostree to know about:
[vagrant@host ~]$ sudo ostree remote add --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-primary fedora-atomic-26 https://kojipkgs.fedoraproject.org/atomic/26
It can be seen from the command that a new remote known as
fedora-atomic-26 was added with a remote url of
gpgkeypath variable was also set in the configuration for the remote. This tells OSTree that it should verify commit signatures when downloading from a remote.This is something new that was enabled for Fedora 26 Atomic Host.
Now that the system has the
fedora-atomic-26 remote the upgrade can be performed:
[vagrant@host ~]$ sudo rpm-ostree rebase fedora-atomic-26:fedora/26/x86_64/atomic-host Receiving metadata objects: 0/(estimating) -/s 0 bytes Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <firstname.lastname@example.org>" Receiving delta parts: 0/27 5.3 MB/s 26.7 MB/355.4 MB Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <email@example.com>" 27 delta parts, 9 loose fetched; 347079 KiB transferred in 105 seconds Copying /etc changes: 22 modified, 0 removed, 58 added Transaction complete; bootconfig swap: yes deployment count change: 1 Upgraded: GeoIP 1.6.11-1.fc25 -> 1.6.11-1.fc26 GeoIP-GeoLite-data 2017.04-1.fc25 -> 2017.06-1.fc26 NetworkManager 1:1.4.4-5.fc25 -> 1:1.8.2-1.fc26 ... ... setools-python-4.1.0-3.fc26.x86_64 setools-python3-4.1.0-3.fc26.x86_64 Run "systemctl reboot" to start a reboot [vagrant@host ~]$ sudo reboot Connection to 192.168.121.217 closed by remote host. Connection to 192.168.121.217 closed.
After reboot the status looks like:
$ vagrant ssh [vagrant@host ~]$ rpm-ostree status State: idle Deployments: ● fedora-atomic-26:fedora/26/x86_64/atomic-host Version: 26.91 (2017-07-23 03:12:08) Commit: 0715ce81064c30d34ed52ef811a3ad5e5d6a34da980bf35b19312489b32d9b83 GPGSignature: 1 signature Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <firstname.lastname@example.org>" fedora-atomic:fedora-atomic/25/x86_64/docker-host Version: 25.154 (2017-07-04 01:38:10) Commit: ce555fa89da934e6eef23764fb40e8333234b8b60b6f688222247c958e5ebd5b [vagrant@host ~]$ cat /etc/fedora-release Fedora release 26 (Twenty Six)
The system is now on Fedora 26 Atomic Host. If this were a production system now would be a good time to check services, most likely running in containers, to see if they still work. If a service didn’t come up as expected, you can use the rollback command:
sudo rpm-ostree rollback.
To track updated commands for upgrading Atomic Host between releases, visit this wiki page.
Upgrading Systems with Kubernetes
Fedora 25 Atomic Host ships with Kubernetes v1.5.3, and Fedora 26 Atomic Host ships with Kubernetes v1.6.7. Before you upgrade systems participating in an existing Kubernetes cluster from 25 to 26, you must make a few configuration changes.
In Kubernetes 1.6, the
--config argument is no longer valid. If systems exist that have the
KUBELET_ARGS variable in
/etc/kubernetes/kubelet that point to the manifests directory using the
--config argument, you must change the argument name to
--pod-manifest-path. Also in
KUBELET_ARGS, add an additional argument:
For example, if the
/etc/kubernetes/kubelet file started with the following:
KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --cluster-dns=10.254.0.10 --cluster-domain=cluster.local"
Then change it to:
KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --pod-manifest-path=/etc/kubernetes/manifests --cluster-dns=10.254.0.10 --cluster-domain=cluster.local --cgroup-driver=systemd"
Staying With etcd2
From Kubernetes 1.5 to 1.6 upstream shifted from using version 2 of the etcd API to version 3. The Kubernetes documentation instructs users to add two arguments to the
KUBE_API_ARGS variable in the
This ensures that Kubernetes continues to find any pods, services or other objects stored in etcd once the upgrade has been completed.
Moving To etcd3
You can migrate etcd data to the v3 API later. First, stop the etcd and kube-apiserver services. Then, assuming the data is stored in
/var/lib/etcd, run the following command to migrate to etcd3:
# ETCDCTL_API=3 etcdctl --endpoints https://YOUR-ETCD-IP:2379 migrate --data-dir=/var/lib/etcd
After the data migration, remove the
--storage-media-type=application/json arguments from the
/etc/kubernetes/apiserver file and then restart etcd and kube-apiserver services.
Fedora 25 Atomic Host Life Support
The Atomic WG decided to keep updating the
fedora-atomic/25/x86_64/docker-host ref every day when Bodhi runs within Fedora. A new update is created every day. However, it is recommended you upgrade systems to Fedora 26 because future testing and development focus on Fedora 26 Atomic Host. Fedora 25 OSTrees won’t be explicitly tested.
The transition to Fedora 26 Atomic Host should be a smooth process. If you have issues or want to be involved in the future direction of Atomic Host, please join us in IRC (#atomic on freenode) or on the atomic-devel mailing list.