If you’re looking for a way to contribute to Fedora, what about release validation testing? Completing test cases is a quick way to get started. Every release of Fedora that you download and use is tested by the Fedora QA team according to the Release Validation Test Plan. Each test compose of the Alpha, Beta, and Final Fedora releases is tested to see if it meets specific release criteria for that release. This post will walk you through the steps of getting set up, performing the release validation tests, and reporting them to the test matrix.
Get a FAS account
The first step required to help the release validation testing is to get a Fedora Account (FAS). Having a Fedora account allows you to post your results to the Fedora wiki automatically using the
tool. If you don’t already have a FAS account, you can easily register for one here.
Setting up a virtual machine for testing
For each compose of a Fedora pre-release that should be tested, a page on the Fedora Wiki is created to help you find the ISOs to download, and the validation test cases to run. In this example, we are going to test the Fedora 24 Beta 1.1 compose. If you navigate to the Validation Testing wikipage for this compose, we are going to choose the
(from the Fedora 24 Beta).
Once downloaded, it is recommended that you create a virtual machine (VM) to do your testing on. Easy ways to create virtual machines on Fedora include Boxes and
. The default configuration of a VM with 20GB hard drive space and 1024MB of RAM should be enough to execute the test cases.
Once your testing VM is created, you will need to install the
tool to allow you to run the validation test cases, and report your results. To install
, run the following command inside your test VM.
$ sudo dnf install python-wikitcms relval
Finding a test case and post the results
Now you are all set to begin testing. Let’s find a test case and explain the process. The test cases that should be run for a specific compose can be found on the same wiki page you found the link from to the ISO. In this example, we are going to pick the Mediakit ISO Size test case from the Image Sanity section. Each test case has a wiki page that explains in more detail what the test does, and how to run it:
The wiki page for the Mediakit ISO size test case lists the
command to use for running the test and submitting the results.
$ relval size-check --username <your_FAS_Username>
After the test finishes, your screen should look similar to the below screenshot:
And the results will of your test will be automatically be posted in the result matrix.
Congratulations, you’ve now run your first test case for Fedora release validation testing! That’s how we perform test cases and submit results for release validation tests.
Getting more help
If you’re having issues or problems running test cases or need guidance using the tools, please visit #fedora-qa on the freenode IRC network or stop by the mailing list.
My computer cpu does not support virtualization, but I have three terrabyte disks available to do testing on real hardware (8gigs ram, x64 bit dual core cpu).
Given you describe using virtualization, am I out of luck to use my system for testing ISO’s, etc.?
Giulio E. (juliuxpigface)
Yes, you can test on bare metal (real hardware) .
Sumantro described virtualization, because tools like qemu, kvm and virt-manager are very handy, easy to use and don’t require additional hardware.
But it’s really up to the tester; people are certainly allowed to install and give some tries on bare metal. Every test is useful.
Rodrigo de Araujo
I tried to adventure myself doing validation tests, but the process is taking longer to download. It asks me 8 , 10, even 18 hours to finish. There would be another source ? Nothing is wrong with my connection, my ISP is delivering fine.
What about getting it from https://getfedora.org/workstation/prerelease/? That would be the same?
Anyway, on the URL above there are no spins. Thanks!
Thanks for the article! Couple of notes for readers:
The page linked to as an example in the article is quite old, and we no longer need validation testing for that compose. You can always find the ‘current’ validation pages using these addresses:
they will redirect to the correct page for whatever compose we’re currently testing.
Also, the size-check test is kind of an exception in that relval actually runs the test and sends the results for it. For most validation tests, you will do the testing manually, then you can report the results by running ‘relval report-results’ , which gives you a simple text interface to send the results. You can also submit results by editing the wiki page directly, but it’s easier to mess up that way. 🙂