Oxidizing Fedora: Try Rust and its applications today

In recent years, it has become increasingly important to develop software that minimizes security vulnerabilities. Memory management bugs are a common cause of these vulnerabilities. To that end, the Mozilla community has spent the last several years building the Rust language and ecosystem which focuses primarily on eliminating those bugs. And Rust is available in Fedora today, along with a few applications in Fedora 27 and higher, as seen below.

Introducing Rust

The Rust programming language bills itself as “a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.” But what does that mean?

Unlike most popular programming languages like Python, Ruby, and Java, Rust is compiled to machine code instead of being executed in a runtime environment. This means all code written in Rust must be processed long before execution. The Rust compiler checks the code as it compiles to ensure that problems are eliminated in the compiled code. Examples of such problems are:

  • Data races: contention between threads accessing the same memory location while one is trying to write there
  • Memory access violations: attempts to access/manipulate data in memory that aren’t permitted or no longer exist

Of course, checks alone aren’t enough to make this work. Rust features a long list of language enhancements specifically tailored to develop “safe” code in the first place.

If you want to learn more about the language, the Rust website offers excellent documentation. You can also read about Rust in this previous Magazine article.

Rust crates

Rust modules are also known as crates. The main index of crates is crates.io. Developers manage crates and their build and runtime dependencies using the cargo tool.

Today, Fedora includes over 300 Rust crates in Rawhide, which at the time of this writing will become Fedora 28. Over 120 Rust crates are included in Fedora 27. This selection includes nearly all of the popular crates that work with the Rust stable compiler on Linux. And even more crates will become available in Fedora over time!

Rust applications

Today, Fedora includes a few interesting Rust-based applications. All of these are available in Fedora 27 and newer and can be installed via DNF. Here are some examples:

ripgrep

ripgrep is a command line tool in the tradition of grep. It is optimized for searching large directories of files using parallelism out of the box. It recursively searches your current directory by default. This utility automatically skips files matching a pattern in your .gitignore files (this option can be disabled). ripgrep supports file type filtering and the full gamut of Unicode, while remaining fast. It can even automatically search compressed files. In most cases, it is faster than grep, ag, git grep, ucg, pt and shift.

For example, a simple comparison of searching through an 8GB file with ripgrep and then GNU grep:

$ /tmp env LC_ALL=C time rg -n -w 'Sherlock [A-Z]\w+' OpenSubtitles2018.raw.en | wc -l
2.16user 0.33system 0:02.50elapsed 99%CPU (0avgtext+0avgdata 8151736maxresident)k
0inputs+0outputs (0major+127911minor)pagefaults 0swaps
4148
$ /tmp env LC_ALL=C time egrep -n -w 'Sherlock [A-Z]\w+' OpenSubtitles2018.raw.en | wc -l
7.98user 1.21system 0:09.21elapsed 99%CPU (0avgtext+0avgdata 2624maxresident)k
752inputs+0outputs (3major+223minor)pagefaults 0swaps
4148

GNU grep takes 9.2 seconds whereas ripgrep takes only 2.5 seconds. Impressive, isn’t it? If you are a user of any of the mentioned tools, you might want to look at the feature comparison table of ack, ag, git-grep, GNU grep and ripgrep. To install ripgrep, run this command using sudo:

sudo dnf install ripgrep

exa

exa is a modern replacement for ls. It uses colors for information by default, which helps you distinguish properties such as file type and ownership. It also has extra features not present in the original ls, such as viewing git status for a directory, or recursively listing files through sub-directories with a tree view.

To install the exa utility, use this command:

sudo dnf install exa

tokei

Tokei is a tool that analyzes code in a project and offers project statistics. It shows you exactly how many code, comments, or blank lines you have. It was originally inspired by the Perl script cloc. However, cloc is slower and more error-prone as it encounters cases that cause it to miscount code as comments and vice versa. One of Tokei’s biggest selling points is speed. The following example shows how Tokei performs on Fedora’s source of Firefox 58. This codebase has over 175,000 files containing over 26 million lines of code.

$ time tokei firefox-58.0.1/
-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 Ada                    10         2840         1681          560          599
 Assembly              497       281559       235805        15038        30716
 Autoconf              417        59137        43862         7533         7742
 BASH                    3          342          257           43           42
 Batch                  48         4005         3380          101          524
 C                    3761      2567794      1864710       402801       300283
 C Header            14258      3034649      1830164       782437       422048
 CMake                  72        10811         7263         2009         1539
 C#                      9         1615          879          506          230
 C Shell                 2           72           34           14           24
 CoffeeScript            4           64           34           12           18
 C++                 11055      5812950      4449843       607276       755831
 C++ Header             92        41014        32622         4627         3765
 CSS                  1401       123014        95702         8047        19265
 D                       1           34            8           22            4
 Dockerfile             76         1983         1290          320          373
 Emacs Lisp              2          338          258           38           42
 Elm                     2          542          399           29          114
 Fish                    2          152           94           26           32
 GLSL                 2952       144792        57711        68029        19052
 Go                      2          485          314          101           70
 Handlebars             17          212          211            0            1
 Happy                   3         2008         2008            0            0
 HTML                62132      3479735      2955995       140901       382839
 Java                 2872       511312       324521       120016        66775
 JavaScript          55028      5576166      3572186      1199464       804516
 JSON                 1078       803571       803571            0            0
 JSX                     6          886          706           62          118
 Makefile              723        46698        25789        12197         8712
 Markdown              572        62395        62395            0            0
 Module-Definition      52         5118         3865         1173           80
 MSBuild                 3          223          165           48           10
 Objective C            60         4055         2889          527          639
 Objective C++         238        73816        54479         8071        11266
 Org                     2           54           42            0           12
 Pascal                  5         1569         1122          210          237
 Perl                   96        21520        15188         2987         3345
 PHP                     2          864          440          284          140
 Prolog                  1           17           15            0            2
 Protocol Buffers       24         5184         1988         2466          730
 Python               4165       787017       592691        66138       128188
 R                       1           38           12           18            8
 Rakefile                1           11            9            0            2
 ReStructuredText      388        51423        51423            0            0
 Ruby                    4          181          153            5           23
 Rust                 3250      1095452       833476       163020        98956
 Sass                    6          215          157           16           42
 Scala                   1          195          164            2           29
 Scons                   1           25           18            1            6
 Shell                 652        91256        64023        15520        11713
 SVG                  3885       152642       126545        18540         7557
 Swift                   1            9            7            0            2
 TeX                     2        11081         6860         3236          985
 Plain Text           2992      1444524      1444524            0            0
 TOML                  445        10738         8291         1102         1345
 TypeScript             21        32983        28256         4544          183
 Vim Script              5            5            5            0            0
 XML                  2259       225666       204439         6691        14536
 YAML                  154        34415        31155          560         2700
-------------------------------------------------------------------------------
 Total              175813     26621471     19846093      3667368      3108010
-------------------------------------------------------------------------------
22.36user 4.79system 0:07.61elapsed 356%CPU (0avgtext+0avgdata 136184maxresident)k
1864920inputs+0outputs (0major+48004minor)pagefaults 0swaps

Here’s the same exercise using the original cloc utility:

$ time cloc firefox-58.0.1/
  220532 text files.
  209900 unique files.                                          
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/^(.*?){ <-- HERE [^$]/ at /usr/bin/cloc line 4850.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/^(.*?){ <-- HERE [^$]/ at /usr/bin/cloc line 4850.
   49681 files ignored.

github.com/AlDanial/cloc v 1.72  T=627.95 s (276.9 files/s, 40329.3 lines/s)
---------------------------------------------------------------------------------------
Language                             files          blank        comment           code
---------------------------------------------------------------------------------------
C++                                  10964         755278         602036        4450883
JavaScript                           52752         795983        1200658        3557155
HTML                                 58973         374714         133514        2819513
C                                     3684         299543         406289        1857360
C/C++ Header                         14119         415207         776222        1805495
Rust                                  3145          97425         160172         822947
JSON                                  1228            516              0         802139
Python                                3886         124724         142265         495765
Java                                  2866          66713         120321         323965
Assembly                               469          30686          32627         217460
XML                                   2190          14459           6621         202914
INI                                   8252          53608            159         202694
Bourne Shell                           672          24242          26559         151447
IDL                                   1295          15749              0         119280
XHTML                                 2399          10646           4857         100383
CSS                                   1053          18805           7891          92729
Objective C++                          222          11245           8032          54356
NAnt script                           1378           8371              0          48827
Markdown                               569          17225              4          44998
MSBuild script                          28              1              0          44320
GLSL                                  1827          12943          42011          42461
m4                                      77           4151            827          35890
YAML                                   296           3082            703          34810
make                                   634           7630          10330          27522
Perl                                   103           3632           3690          16208
DTD                                    176           3698           4696          14297
CMake                                   72           1539           2009           7263
TeX                                      2            985           3236           6860
Windows Module Definition               48             54           1161           3617
DOS Batch                               44            492             87           3200
SKILL                                    4             68              2           2419
HLSL                                    33            409            285           2045
Protocol Buffers                        24            730           2472           1982
Windows Resource File                   48            442            575           1864
Objective C                             37            459            514           1823
yacc                                     3            173             85           1750
Ada                                     10            599            560           1681
XSLT                                    26            168            142           1437
Pascal                                   8            260            504           1405
Cython                                   1             59            158           1310
Dockerfile                              74            367            315           1266
Groovy                                  14            249            316           1194
lex                                      4            237             82           1074
diff                                    14            530           2038            963
C#                                       9            230            506            879
MATLAB                                  11            162            147            874
JSX                                      6            118             62            706
Bourne Again Shell                      22            126            196            676
Jam                                     27            170            379            586
Korn Shell                               5             83            165            526
Expect                                   6            105            164            506
PHP                                      2            140            288            436
Elm                                      2            114             29            399
Ant                                      2             27            107            389
Go                                       2             70            101            314
TypeScript                              13             73             72            268
Lisp                                     2             42             38            258
Handlebars                              12              1              0            199
Mako                                     3             14              0            168
Scala                                    1             29              2            164
Ruby                                     4             25              4            163
awk                                      2             41              8            154
Sass                                     4             36             15            144
Haxe                                     2             25              5            137
Vuejs Component                          2              6              0            122
Visual Basic                             1             17             15             84
sed                                      7             16             27             73
PowerShell                               2             17            110             46
SAS                                      1             14             22             32
C Shell                                  2             13              7             28
CoffeeScript                             3             13              8             25
Prolog                                   1              2              0             15
R                                        1              8             18             12
MUMPS                                    3              2              0              9
D                                        1              4             22              8
Mathematica                              2              1              0              7
Swift                                    1              2              0              7
Freemarker Template                      3              0              0              6
Stylus                                   1              2              0              5
vim script                               1              0              0              1
---------------------------------------------------------------------------------------
SUM:                                173892        3179844        3707542       18437397
---------------------------------------------------------------------------------------
266.51user 306.34system 10:28.37elapsed 91%CPU (0avgtext+0avgdata 446552maxresident)k
6704096inputs+0outputs (12major+44888834minor)pagefaults 0swaps

On average, tokei takes 8 seconds whereas cloc takes 12 minutes. To install tokei, use this command:

sudo dnf install tokei

ternimal

Ternimal is a program that draws a glowing, animated lifeform in the terminal using Unicode block symbols. It’s not a typical Linux utility but something like an experiment in digital art. It does have niche applications like benchmarking terminal emulators or making cool or scary SSH greeting messages by combining with timeout.

You can customize almost every aspect of the rendered “animal” with command line arguments. Currently, the only documentation available for those is the (well commented) source code and some examples. Ternimal is written in pure Rust and carefully profiled and optimized. It’s quite resource efficient and can render fluid, complex animations with minimal CPU usage.

To install Ternimal, run this command:

sudo dnf install ternimal

Fedora Rust SIG

Over the last year, a lot of work has gone into smoothly integrating the Rust ecosystem into Fedora. Fedora now has a special interest group (SIG) for supporting the Rust ecosystem. The Fedora Rust SIG is the general steward of Rust in Fedora, including crates and application packaging.

The Rust SIG’s approach towards packaging and supporting the Rust ecosystem is different from some SIGs in Fedora. As Rust was a brand new ecosystem, the Rust SIG was treading on new ground. The members decided early on to make this a truly cross-distribution effort, and harmonize packaging of Rust code across Linux distributions using the RPM package manager. The SIG worked closely with members of Mageia and openSUSE to ensure  the tooling worked for everyone. As a consequence, today you can make Fedora, Mageia, and openSUSE compliant packaging of Rust crates and applications using rust2rpm in any of those distros.

Acknowledgements

Of course, none of this would have been possible without the efforts of several people:

  • Without the assistance of Michael Schröder from openSUSE, many of the fundamental pieces we needed in the dependency resolver stack would not be possible. Florian Festi from the RPM team deserves special thanks for reviewing all of this work and making sure it is all sane in RPM.
  • A lot of the refinement of Rust compiler packaging and finding/fixing weird bugs related to that can be credited to Josh Stone from Red Hat and Rémi Verschelde from Mageia. Both of them maintain the core Rust stack in their respective distributions (Fedora/EPEL and Mageia, respectively) and regularly share work with each other to ensure Rust and Cargo are available in the best form possible.
  • Adam Miller (Fedora Release Engineering), Dusty Mabe (Project Atomic), Kushal Das (Fedora Cloud), Patrick Uiterwijk (Fedora Release Engineering), and Randy Barlow (Fedora Release Engineering) deserve a ton of credit for revamping the infrastructure to help support the new features that were needed to bring a first-class experience with the Rust ecosystem to Fedora.
  • Last but not least, the Rust community has been fantastic. Engaging with them is a great experience and Rust and its ecosystem exist thanks to them and their efforts.

Photo by James Sutton on Unsplash.

For Developers For System Administrators New in Fedora

15 Comments

  1. I love that the comparison between ripgrep and GNU grep used Sherlock. <3
    Just for that I would install it. Thanks!

  2. Awesome! As far as I know, ripgrep has already been integrated into VS Code as the default search engine. Looks promising!

  3. Tobias

    The article could have also mentioned that Builder is a really good IDE to write rust. The integration is still at the beginning but together with the rust compilers helpful messages it is really easy to get started writing rust.

    • @Tobias, thanks for suggestion. I do agree that GNOME Builder is good IDE to write Rust code, but we’ve tried to focus this article just on applications written fully in Rust.

  4. Michael Christenson

    Is there any chance that we can have rustup so that we can do nightly builds? It’s fine having a stable rust, but a lot of frameworks like rocket.rs still need the macro changes that nightly gives us.

    • Packaging rustup doesn’t make much sense, it’s installable in very simple way from upstream. I was proposing to package nightly compiler, but supporting it is very hard.. But we could try to do this in COPR.

      • Michael Christenson

        Yeah, I imagine a nightly would take more effort. The trouble with going the rustup way without it being a part of the distro is possible collision with distro installed rust (provided it becomes a dependency some how). It’s kind of tricky.

  5. TomTr

    This is one useful and entertaining article again, thanks!
    That said, one case where ‘ripgrep’ needs some love is its ”rg –help’ output. It’s very hard to use it for a quick reference.
    Although I find it funny to pipe it through ‘grep’.

  6. Michael Christenson

    My ternimal is dancing proudly next to the real work 🙂

  7. Matthew Bunt

    Nice article.

    I would love to see an article about how to properly setup Rust and GNOME Builder on Fedora. I have never been able to get it to work quite right. Although it has been close enough to really want it to work well. Specifically there have been issues with syntax highlighting (partial highlighting) and linting (not detecting errors). I get the feeling not many people use Rust and Builder because there are few bug reports about Rust, but I would greatly prefer using Builder rather than VSCode. I have a feeling I just didn’t set up Rust or Builder correctly because I have not seen others reporting this problem and have not seen any tutorials on how to do it right.

  8. Alexander

    Someone should package fd (https://github.com/sharkdp/fd)!

    fd : find :: ripgrep : grep

  9. ifoolb

    What a coincidence, I’m looking into Rust these days too, hoping to develop some networking programs in future, as it is used for writing Servo.

Comments are Closed

The opinions expressed on this website are those of each author, not of the author's employer or of Red Hat. Fedora Magazine aspires to publish all content under a Creative Commons license but may not be able to do so in all cases. You are responsible for ensuring that you have the necessary permission to reuse any work on this site. The Fedora logo is a trademark of Red Hat, Inc. Terms and Conditions