In recent years, it has become increasingly important to develop software that minimizes security vulnerabilities. Memory management bugs are a common cause of these vulnerabilities. To that end, the Mozilla community has spent the last several years building the Rust language and ecosystem which focuses primarily on eliminating those bugs. And Rust is available in Fedora today, along with a few applications in Fedora 27 and higher, as seen below.
Introducing Rust
The Rust programming language bills itself as “a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.” But what does that mean?
Unlike most popular programming languages like Python, Ruby, and Java, Rust is compiled to machine code instead of being executed in a runtime environment. This means all code written in Rust must be processed long before execution. The Rust compiler checks the code as it compiles to ensure that problems are eliminated in the compiled code. Examples of such problems are:
- Data races: contention between threads accessing the same memory location while one is trying to write there
- Memory access violations: attempts to access/manipulate data in memory that aren’t permitted or no longer exist
Of course, checks alone aren’t enough to make this work. Rust features a long list of language enhancements specifically tailored to develop “safe” code in the first place.
If you want to learn more about the language, the Rust website offers excellent documentation. You can also read about Rust in this previous Magazine article.
Rust crates
Rust modules are also known as crates. The main index of crates is crates.io. Developers manage crates and their build and runtime dependencies using the cargo tool.
Today, Fedora includes over 300 Rust crates in Rawhide, which at the time of this writing will become Fedora 28. Over 120 Rust crates are included in Fedora 27. This selection includes nearly all of the popular crates that work with the Rust stable compiler on Linux. And even more crates will become available in Fedora over time!
Rust applications
Today, Fedora includes a few interesting Rust-based applications. All of these are available in Fedora 27 and newer and can be installed via DNF. Here are some examples:
ripgrep
ripgrep is a command line tool in the tradition of grep. It is optimized for searching large directories of files using parallelism out of the box. It recursively searches your current directory by default. This utility automatically skips files matching a pattern in your .gitignore files (this option can be disabled). ripgrep supports file type filtering and the full gamut of Unicode, while remaining fast. It can even automatically search compressed files. In most cases, it is faster than grep, ag, git grep, ucg, pt and shift.
For example, a simple comparison of searching through an 8GB file with ripgrep and then GNU grep:
$ /tmp env LC_ALL=C time rg -n -w 'Sherlock [A-Z]\w+' OpenSubtitles2018.raw.en | wc -l 2.16user 0.33system 0:02.50elapsed 99%CPU (0avgtext+0avgdata 8151736maxresident)k 0inputs+0outputs (0major+127911minor)pagefaults 0swaps 4148 $ /tmp env LC_ALL=C time egrep -n -w 'Sherlock [A-Z]\w+' OpenSubtitles2018.raw.en | wc -l 7.98user 1.21system 0:09.21elapsed 99%CPU (0avgtext+0avgdata 2624maxresident)k 752inputs+0outputs (3major+223minor)pagefaults 0swaps 4148
GNU grep takes 9.2 seconds whereas ripgrep takes only 2.5 seconds. Impressive, isn’t it? If you are a user of any of the mentioned tools, you might want to look at the feature comparison table of ack, ag, git-grep, GNU grep and ripgrep. To install ripgrep, run this command using sudo:
sudo dnf install ripgrep
exa
exa is a modern replacement for ls. It uses colors for information by default, which helps you distinguish properties such as file type and ownership. It also has extra features not present in the original ls, such as viewing git status for a directory, or recursively listing files through sub-directories with a tree view.
To install the exa utility, use this command:
sudo dnf install exa
tokei
Tokei is a tool that analyzes code in a project and offers project statistics. It shows you exactly how many code, comments, or blank lines you have. It was originally inspired by the Perl script cloc. However, cloc is slower and more error-prone as it encounters cases that cause it to miscount code as comments and vice versa. One of Tokei’s biggest selling points is speed. The following example shows how Tokei performs on Fedora’s source of Firefox 58. This codebase has over 175,000 files containing over 26 million lines of code.
$ time tokei firefox-58.0.1/ ------------------------------------------------------------------------------- Language Files Lines Code Comments Blanks ------------------------------------------------------------------------------- Ada 10 2840 1681 560 599 Assembly 497 281559 235805 15038 30716 Autoconf 417 59137 43862 7533 7742 BASH 3 342 257 43 42 Batch 48 4005 3380 101 524 C 3761 2567794 1864710 402801 300283 C Header 14258 3034649 1830164 782437 422048 CMake 72 10811 7263 2009 1539 C# 9 1615 879 506 230 C Shell 2 72 34 14 24 CoffeeScript 4 64 34 12 18 C++ 11055 5812950 4449843 607276 755831 C++ Header 92 41014 32622 4627 3765 CSS 1401 123014 95702 8047 19265 D 1 34 8 22 4 Dockerfile 76 1983 1290 320 373 Emacs Lisp 2 338 258 38 42 Elm 2 542 399 29 114 Fish 2 152 94 26 32 GLSL 2952 144792 57711 68029 19052 Go 2 485 314 101 70 Handlebars 17 212 211 0 1 Happy 3 2008 2008 0 0 HTML 62132 3479735 2955995 140901 382839 Java 2872 511312 324521 120016 66775 JavaScript 55028 5576166 3572186 1199464 804516 JSON 1078 803571 803571 0 0 JSX 6 886 706 62 118 Makefile 723 46698 25789 12197 8712 Markdown 572 62395 62395 0 0 Module-Definition 52 5118 3865 1173 80 MSBuild 3 223 165 48 10 Objective C 60 4055 2889 527 639 Objective C++ 238 73816 54479 8071 11266 Org 2 54 42 0 12 Pascal 5 1569 1122 210 237 Perl 96 21520 15188 2987 3345 PHP 2 864 440 284 140 Prolog 1 17 15 0 2 Protocol Buffers 24 5184 1988 2466 730 Python 4165 787017 592691 66138 128188 R 1 38 12 18 8 Rakefile 1 11 9 0 2 ReStructuredText 388 51423 51423 0 0 Ruby 4 181 153 5 23 Rust 3250 1095452 833476 163020 98956 Sass 6 215 157 16 42 Scala 1 195 164 2 29 Scons 1 25 18 1 6 Shell 652 91256 64023 15520 11713 SVG 3885 152642 126545 18540 7557 Swift 1 9 7 0 2 TeX 2 11081 6860 3236 985 Plain Text 2992 1444524 1444524 0 0 TOML 445 10738 8291 1102 1345 TypeScript 21 32983 28256 4544 183 Vim Script 5 5 5 0 0 XML 2259 225666 204439 6691 14536 YAML 154 34415 31155 560 2700 ------------------------------------------------------------------------------- Total 175813 26621471 19846093 3667368 3108010 ------------------------------------------------------------------------------- 22.36user 4.79system 0:07.61elapsed 356%CPU (0avgtext+0avgdata 136184maxresident)k 1864920inputs+0outputs (0major+48004minor)pagefaults 0swaps
Here’s the same exercise using the original cloc utility:
$ time cloc firefox-58.0.1/
220532 text files.
209900 unique files.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/^(.*?){ <-- HERE [^$]/ at /usr/bin/cloc line 4850.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/^(.*?){ <-- HERE [^$]/ at /usr/bin/cloc line 4850.
49681 files ignored.
github.com/AlDanial/cloc v 1.72 T=627.95 s (276.9 files/s, 40329.3 lines/s)
---------------------------------------------------------------------------------------
Language files blank comment code
---------------------------------------------------------------------------------------
C++ 10964 755278 602036 4450883
JavaScript 52752 795983 1200658 3557155
HTML 58973 374714 133514 2819513
C 3684 299543 406289 1857360
C/C++ Header 14119 415207 776222 1805495
Rust 3145 97425 160172 822947
JSON 1228 516 0 802139
Python 3886 124724 142265 495765
Java 2866 66713 120321 323965
Assembly 469 30686 32627 217460
XML 2190 14459 6621 202914
INI 8252 53608 159 202694
Bourne Shell 672 24242 26559 151447
IDL 1295 15749 0 119280
XHTML 2399 10646 4857 100383
CSS 1053 18805 7891 92729
Objective C++ 222 11245 8032 54356
NAnt script 1378 8371 0 48827
Markdown 569 17225 4 44998
MSBuild script 28 1 0 44320
GLSL 1827 12943 42011 42461
m4 77 4151 827 35890
YAML 296 3082 703 34810
make 634 7630 10330 27522
Perl 103 3632 3690 16208
DTD 176 3698 4696 14297
CMake 72 1539 2009 7263
TeX 2 985 3236 6860
Windows Module Definition 48 54 1161 3617
DOS Batch 44 492 87 3200
SKILL 4 68 2 2419
HLSL 33 409 285 2045
Protocol Buffers 24 730 2472 1982
Windows Resource File 48 442 575 1864
Objective C 37 459 514 1823
yacc 3 173 85 1750
Ada 10 599 560 1681
XSLT 26 168 142 1437
Pascal 8 260 504 1405
Cython 1 59 158 1310
Dockerfile 74 367 315 1266
Groovy 14 249 316 1194
lex 4 237 82 1074
diff 14 530 2038 963
C# 9 230 506 879
MATLAB 11 162 147 874
JSX 6 118 62 706
Bourne Again Shell 22 126 196 676
Jam 27 170 379 586
Korn Shell 5 83 165 526
Expect 6 105 164 506
PHP 2 140 288 436
Elm 2 114 29 399
Ant 2 27 107 389
Go 2 70 101 314
TypeScript 13 73 72 268
Lisp 2 42 38 258
Handlebars 12 1 0 199
Mako 3 14 0 168
Scala 1 29 2 164
Ruby 4 25 4 163
awk 2 41 8 154
Sass 4 36 15 144
Haxe 2 25 5 137
Vuejs Component 2 6 0 122
Visual Basic 1 17 15 84
sed 7 16 27 73
PowerShell 2 17 110 46
SAS 1 14 22 32
C Shell 2 13 7 28
CoffeeScript 3 13 8 25
Prolog 1 2 0 15
R 1 8 18 12
MUMPS 3 2 0 9
D 1 4 22 8
Mathematica 2 1 0 7
Swift 1 2 0 7
Freemarker Template 3 0 0 6
Stylus 1 2 0 5
vim script 1 0 0 1
---------------------------------------------------------------------------------------
SUM: 173892 3179844 3707542 18437397
---------------------------------------------------------------------------------------
266.51user 306.34system 10:28.37elapsed 91%CPU (0avgtext+0avgdata 446552maxresident)k
6704096inputs+0outputs (12major+44888834minor)pagefaults 0swaps
On average, tokei takes 8 seconds whereas cloc takes 12 minutes. To install tokei, use this command:
sudo dnf install tokei
ternimal
Ternimal is a program that draws a glowing, animated lifeform in the terminal using Unicode block symbols. It’s not a typical Linux utility but something like an experiment in digital art. It does have niche applications like benchmarking terminal emulators or making cool or scary SSH greeting messages by combining with timeout.
You can customize almost every aspect of the rendered “animal” with command line arguments. Currently, the only documentation available for those is the (well commented) source code and some examples. Ternimal is written in pure Rust and carefully profiled and optimized. It’s quite resource efficient and can render fluid, complex animations with minimal CPU usage.
To install Ternimal, run this command:
sudo dnf install ternimal
Fedora Rust SIG
Over the last year, a lot of work has gone into smoothly integrating the Rust ecosystem into Fedora. Fedora now has a special interest group (SIG) for supporting the Rust ecosystem. The Fedora Rust SIG is the general steward of Rust in Fedora, including crates and application packaging.
The Rust SIG’s approach towards packaging and supporting the Rust ecosystem is different from some SIGs in Fedora. As Rust was a brand new ecosystem, the Rust SIG was treading on new ground. The members decided early on to make this a truly cross-distribution effort, and harmonize packaging of Rust code across Linux distributions using the RPM package manager. The SIG worked closely with members of Mageia and openSUSE to ensure the tooling worked for everyone. As a consequence, today you can make Fedora, Mageia, and openSUSE compliant packaging of Rust crates and applications using rust2rpm in any of those distros.
Acknowledgements
Of course, none of this would have been possible without the efforts of several people:
- Without the assistance of Michael Schröder from openSUSE, many of the fundamental pieces we needed in the dependency resolver stack would not be possible. Florian Festi from the RPM team deserves special thanks for reviewing all of this work and making sure it is all sane in RPM.
- A lot of the refinement of Rust compiler packaging and finding/fixing weird bugs related to that can be credited to Josh Stone from Red Hat and Rémi Verschelde from Mageia. Both of them maintain the core Rust stack in their respective distributions (Fedora/EPEL and Mageia, respectively) and regularly share work with each other to ensure Rust and Cargo are available in the best form possible.
- Adam Miller (Fedora Release Engineering), Dusty Mabe (Project Atomic), Kushal Das (Fedora Cloud), Patrick Uiterwijk (Fedora Release Engineering), and Randy Barlow (Fedora Release Engineering) deserve a ton of credit for revamping the infrastructure to help support the new features that were needed to bring a first-class experience with the Rust ecosystem to Fedora.
- Last but not least, the Rust community has been fantastic. Engaging with them is a great experience and Rust and its ecosystem exist thanks to them and their efforts.
Photo by James Sutton on Unsplash.
crossingtheair.wordpress.com
I love that the comparison between ripgrep and GNU grep used Sherlock. <3
Just for that I would install it. Thanks!
Weihang Lo
Awesome! As far as I know, ripgrep has already been integrated into VS Code as the default search engine. Looks promising!
Tobias
The article could have also mentioned that Builder is a really good IDE to write rust. The integration is still at the beginning but together with the rust compilers helpful messages it is really easy to get started writing rust.
Igor Gnatenko
@Tobias, thanks for suggestion. I do agree that GNOME Builder is good IDE to write Rust code, but we’ve tried to focus this article just on applications written fully in Rust.
Michael Christenson
Is there any chance that we can have rustup so that we can do nightly builds? It’s fine having a stable rust, but a lot of frameworks like rocket.rs still need the macro changes that nightly gives us.
Igor Gnatenko
Packaging rustup doesn’t make much sense, it’s installable in very simple way from upstream. I was proposing to package nightly compiler, but supporting it is very hard.. But we could try to do this in COPR.
Michael Christenson
Yeah, I imagine a nightly would take more effort. The trouble with going the rustup way without it being a part of the distro is possible collision with distro installed rust (provided it becomes a dependency some how). It’s kind of tricky.
TomTr
This is one useful and entertaining article again, thanks!
That said, one case where ‘ripgrep’ needs some love is its ”rg –help’ output. It’s very hard to use it for a quick reference.
Although I find it funny to pipe it through ‘grep’.
Michael Christenson
My ternimal is dancing proudly next to the real work 🙂
Matthew Bunt
Nice article.
I would love to see an article about how to properly setup Rust and GNOME Builder on Fedora. I have never been able to get it to work quite right. Although it has been close enough to really want it to work well. Specifically there have been issues with syntax highlighting (partial highlighting) and linting (not detecting errors). I get the feeling not many people use Rust and Builder because there are few bug reports about Rust, but I would greatly prefer using Builder rather than VSCode. I have a feeling I just didn’t set up Rust or Builder correctly because I have not seen others reporting this problem and have not seen any tutorials on how to do it right.
Tobias
Rust and Builder should work together fine. Best to get the Nightly version of both. RLS using Rustup and Builder as a nightly Flatpak: https://wiki.gnome.org/Apps/Builder/Downloads – I get code completion at the moment, but no linting.
Alexander
Someone should package fd (https://github.com/sharkdp/fd)!
fd : find :: ripgrep : grep
Neal Gompa
fd is coming to Fedora 28: https://koji.fedoraproject.org/koji/packageinfo?packageID=25842
Igor Gnatenko
It is already packaged, but only for F28+.
ifoolb
What a coincidence, I’m looking into Rust these days too, hoping to develop some networking programs in future, as it is used for writing Servo.