The Linux startup process goes through several stages before reaching the final graphical or multi-user target. The initramfs stage occurs just before the root file system is mounted. Dracut is a tool that is used to manage the initramfs. The dracut emergency shell is an interactive mode that can be initiated while the initramfs is loaded.
This article will show how to use the dracut command to modify the initramfs. Some basic troubleshooting commands that can be run from the dracut emergency shell will also be demonstrated.
The InitRAMFS
Initramfs stands for Initial Random-Access Memory File System. On modern Linux systems, it is typically stored in a file under the /boot directory. The kernel version for which it was built will be included in the file name. A new initramfs is generated every time a new kernel is installed.
By default, Fedora keeps the previous two versions of the kernel and its associated initramfs. This default can be changed by modifying the value of the installonly_limit setting the /etc/dnf/dnf.conf file.
You can use the lsinitrd command to list the contents of your initramfs archive:
The above screenshot shows that my initramfs archive contains the nouveau GPU driver. The modinfo command tells me that the nouveau driver supports several models of NVIDIA video cards. The lspci command shows that there is an NVIDIA GeForce video card in my computer’s PCI slot. There are also several basic Unix commands included in the archive such as cat and cp.
By default, the initramfs archive only includes the drivers that are needed for your specific computer. This allows the archive to be smaller and decreases the time that it takes for your computer to boot.
The Dracut Command
The dracut command can be used to modify the contents of your initramfs. For example, if you are going to move your hard drive to a new computer, you might want to temporarily include all drivers in the initramfs to be sure that the operating system can load on the new computer. To do so, you would run the following command:
The force parameter tells dracut that it is OK to overwrite the existing initramfs archive. The no-hostonly parameter overrides the default behavior of including only drivers that are germane to the currently-running computer and causes dracut to instead include all drivers in the initramfs.
By default dracut operates on the initramfs for the currently-running kernel. You can use the uname command to display which version of the Linux kernel you are currently running:
5.0.5-200.fc29.x86_64
Once you have your hard drive installed and running in your new computer, you can re-run the dracut command to regenerate the initramfs with only the drivers that are needed for the new computer:
There are also parameters to add arbitrary drivers, dracut modules, and files to the initramfs archive. You can also create configuration files for dracut and save them under the /etc/dracut.conf.d directory so that your customizations will be automatically applied to all new initramfs archives that are generated when new kernels are installed. As always, check the man page for the details that are specific to the version of dracut you have installed on your computer:
The Dracut Emergency Shell
Sometimes something goes wrong during the initramfs stage of your computer’s boot process. When this happens, you will see “Entering emergency mode” printed to the screen followed by a shell prompt. This gives you a chance to try and fix things up manually and continue the boot process.
As a somewhat contrived example, let’s suppose that I accidentally deleted an important kernel parameter in my boot loader configuration:
The next time I reboot my computer, it will seem to hang for several minutes while it is trying to find the root partition and eventually give up and drop to an emergency shell.
From the emergency shell, I can enter journalctl and then use the Space key to page down though the startup logs. Near the end of the log I see a warning that reads “/dev/mapper/fedora-root does not exist”. I can then use the ls command to find out what does exist:
control fedora-swap
Hmm, the fedora-root LVM volume appears to be missing. Let’s see what I can find with the lvm command:
ACTIVE '/dev/fedora/swap' [3.85 GiB] inherit
inactive '/dev/fedora/home' [22.85 GiB] inherit
inactive '/dev/fedora/root' [46.80 GiB] inherit
Ah ha! There’s my root partition. It’s just inactive. All I need to do is activate it and exit the emergency shell to continue the boot process:
# exit
The above example only demonstrates the basic concept. You can check the troubleshooting section of the dracut guide for a few more examples.
It is possible to access the dracut emergency shell manually by adding the rd.break parameter to your kernel command line. This can be useful if you need to access your files before any system services have been started.
Check the dracut.kernel man page for details about what kernel options your version of dracut supports:
Cesar
Great post! Thanks.
Frederik
The bit about the initramfs not containing any drivers, for hardware you don’t have on a particular computer, was a surprise to me. Nice to know!
Stuart Gathman
Cool – lsinitrd is something I’ve needed many times. I’ve been using gunzip and cpio …
tjakobsen
Informative, good job. 🙂
Ivan
Great and useful!!! Something to study! Thanks.
svsv sarma
Thanks for the dracut discussion. I always use dracut regenerate to avoid “Failed to start load kernel modules…..” alert at boot time on every kernel update. I understand that the alert is specific to HP systems.
Perhaps a deep study can save me from further alerts permanently.
Gregory Bartholomew
Hmm, if you are just running
without any extra parameters, then you shouldn’t end up with anything different from what gets auto-generated with the kernel installation. You might try making a backup of the broken initramfs before you run the regenerate command and then compare the regenerated one with the backup by running something like:
If they are built against the same kernel version on the same machine, then there shouldn’t be any differences. If you find a difference, maybe you could specify that the the missing module should always be installed (or that the invalid module should always be removed) by making a config file under /etc/dracut.conf.d.
svsv sarma
Thank you for the hint, I will work on that.
Joao Rodrigues
I think sysv sarma was referring to the same issue I have with nvidia and virtualbox kernel modules.
Whenever there’s a kernel update, dracut generates a new kernel, and after that, akmods start building the kernel modules but it doesn’t regenerate the initrd.
If you don’t regenerate the initrd manually (after the akmods build) you’ll have a message saying that it couldn’t load the modules (because they aren’t on the initrd). It still works, because it will eventually load the modules after chrooting.
Workaround: after a kernel update wait for akmods to finish building and run dracut again.
Gregory Bartholomew
Right. In that case, the kernel module is failing to load because the one that is in the initramfs was compiled for the previous kernel version.
If the module isn’t needed in the initramfs stage (which presumably it isn’t since he can see the error message) and the correct module is loaded anyway at a later point in the boot process (which can be forced if necessary by adding a config file in /etc/modprobe.d), another workaround to prevent the error message from displaying would be to run the following to add a config file in /etc/dracut.conf.d that will prevent the incorrect module from being added to the initramfs in the future (assuming the module name is “nvidia”):
# echo ‘omit_drivers+=” nvidia “‘ > /etc/dracut.conf.d/omit-nvidia.conf
svsv sarma
It is not nvidia. It could be the VirtualBox. I uninstalled the VB. But with everything said and done the alert renews with every kernel update. The system boots and works fine in spite of the alert. So further experimenting only after i am ready for a reinstall!
duck life 4
great post. i like it. feeling great when reading your post .
Sébastien Wilmet
Thanks a lot for this article! Some time ago I had the problem when moving a hard disk to another computer, it was not able to boot and I didn’t understand why. Now I finally understand!
Maybe when dracut is entering its emergency shell, it could display a help message with some common problems and solutions, or a link to the dracut troubleshooting guide.
Sébastien Wilmet
Or even better, make it “just works” when moving a disk to another computer: when it fails to boot because of the initramfs, have an automatic fallback mode that generates/tries an initramfs containing all the hardware drivers.
Also, how does it work currently when installing Fedora on an external disk? For an external disk, it’s better to always include all hardware drivers in the initramfs, so that you can boot on different computers. Is the Fedora installation with Anaconda smart enough to do this?
The boot time improvement is nice when not including all the drivers, but it gets in the way, it’s no longer a stateless system.
Gregory Bartholomew
There is a “rescue” initramfs image that includes all drivers for that reason. I’ve seen it disabled on some systems though and the computer has no way of knowing beforehand that it will need to use it, so you’ll have to reboot and select it manually if you need it.
I doubt that Anaconda is that clever. If you have an installation on an external drive that you use on multiple machines, then you should add the following to a config file in /etc/dracut.conf.d:
Sébastien Wilmet
Thanks for the information. I think I’ll configure all my systems with hostonly=no, gaining a few seconds at boot time is not important to me, and the convenience to change a disk to another computer is more important.
I prefer to avoid to launch the rescue mode, because I think it does other things that I don’t want (in addition to use the rescue initramfs, it moves some config files in the home directory, to have a clean state when launching apps, at least I think that is the case on Fedora Workstation). It’s maybe possible to change the kernel command line in grub to just switch to the rescue initramfs, without altering the rest, but it’s not what I call “convenient” when for example a computer has crashed and I just want to boot the disk on another computer, as a quick solution 😉
Jens Bollmann
Great stuff, thanks. Very helpful!
Konstantin
Great article, can’t wait for more to come in the future!