- 2021-04-07: plocate
- 2021-03-05: ugrep
- 2021-01-26: diffoscope, procs, tokei, watchexec
- 2020-09-28: huniq, duf (Disk Usage/Free Utility)
- 2020-09-14: xoxide, starship, delta, bottom, loop
- 2020-05-03: xpipe
- 2019-05-23: moreutils
- 2019-01-09: exa, hyperfine, PathPicker, and svg-term-cli
- 2018-11-19: Pipe Viewer
People who use Unix since 1980s, or GNU/Linux since 1990s know that they can rely on the command line and many utilities for a lot of daily automation and data processing tasks. As someone who’s been using Unix and GNU/Linux based systems since 1994, I’m more than happy that I can count on these tools and the stable know-how built on them. Nevertheless, I think the command line and TUIs (Text-based User Interfaces) can be a bit better in 21. century. Therefore, in this post, I’ll list a few recent utilities that helps us to have a better command line experience.
Before you dive into the list, please be aware that I’m after the low-hanging fruit, that is, tools that can make an existing Unix / Linux command line environment a bit better with the least disruption. In other words, I will not touch on how to have brand new, GPU-powered terminal emulators such as alacritty and kitty, and neither will I talk about how nice it’d be if you only changed your shell from Bash into fish, or elvish. (If you really want to know about alternative shells, please read https://github.com/oilshell/oil/wiki/ExternalResources and http://www.oilshell.org/blog/2018/01/28.html.) I also won’t send you down the rabbit hole and make you spend countless hours customizing your shell prompt (that requires an article by itself, but in the meantime you can go and check Go Bullet Train (GBT), you’ve been warned!). Finally, no, I won’t be talking about
tmux neither, because it has books dedicated to it such as “The Tao of tmux” and “tmux 2: Productive Mouse-Free Development“.
If you think something is missing and fits within the context of “low-hanging fruit” (see above), please add a comment at the end of this post. Also, for a more specialized domain, see my recent post titled “Data Processing Resources: Command-line Interface (CLI) for CSV, TSV, JSON, and XML“.
So let’s start with
cd command, and how it can be enhanced with context and history, together with fuzzy matching:
Quickly finding files is also a very regular task, and we can always rely on the
find utility, but we can have a slightly more ergonomic version:
– plocate: a faster locate. It can completely replace mlocate with a much faster (and smaller) index. It is suitable as a default locate on your system. plocate does nearly all I/O asynchronously using io_uring if available (Linux 5.1+), which reduces the impact of seek latency on systems without SSDs.
Whenever I need to use a command line program that I haven’t used for a while, I need to refresh my memory, and probably read at least one man page. But even for commands such as
tar, reading a man page can trigger a TL;DR (Too Long; Didn’t Read) reaction. Therefore, tealdeer to the rescue:
– bat: A cat(1) clone with wings. I think this utility is good enough to move the official
cat to the attic and use in only rare circumstances. It supports syntax highlighting for a large number of programming and markup languages. It is Git-aware, supports automatic paging for long outputs, can show non-printable characters, and can integrate with other tiny little command line utilities such as fd, ripgrep, xclip, man, tail, and others. Did I say that you can use it as a drop-in replacement for cat in many cases?
Every now and then, I check how much disk space is left, or which directory is consuming the biggest amount. Back in the day, I used
du for this, and I also like the more interactive version that’s called ncdu, but nowadays I think the following tiny terminal file manager can also be used for that task:
– nnn: probably the fastest and most resource-sensitive file manager you have ever used. It integrates seamlessly with your DE and favourite GUI utilities, has a unique navigate-as-you-type mode with auto-select, disk usage analyzer mode, bookmarks, familiar navigation shortcuts, subshell spawning and much more. It also lets you preview a file using whichever previewer you want. You can use bat, or even emacsclient (that is, if you, like me, prefer to have your Emacs server started, first thing after booting).
– Another fine utility to see disk usage and free space at a glance is “duf: Disk Usage/Free Utility (Linux, BSD & macOS)“. It has a very nice human-readable output, but it can also output its reports in JSON:
One other frequent task, in addition to navigating around your directories, and files is searching for strings and patterns in them. You always have the venerable
grep tool, but nowadays, we have nice alternatives such as ripgrep:
– ripgrep: a faster grep / ag / git grep / ucg / pt / sift. See for yourself at https://github.com/BurntSushi/ripgrep. Also see: “Faster, RegEx! Match! Match! (Which Regular Expression Utility is the Fastest?)“
For those who are looking for something even faster than ripgrep, coupled with interactive search options, there’s also ugrep:
– ugrep: “Search for anything in everything… ultra fast. Search file systems, source code, text, binary files, archives (cpio/tar/pax/zip), compressed files (gz/Z/bz2/lzma/xz/lz4), documents and more. A faster, user-friendly and compatible grep replacement. New match algorithms and features beating grep, ripgrep, silver searcher, ack, sift, etc. A lot of optional capabilities to make searching very convenient.”
If you’re like me, you’ve used sort and uniq, and their combination for too many times. Therefore you might find the following utility useful (for more such utilities you can always visit “Data Processing Resources: Command-line Interface (CLI) for CSV, TSV, JSON, and XML“):
– huniq: Replacement for
sort | uniq optimized for speed (10x faster) when sorting is not needed.
sort | uniq (or
sort -u with gnu sort) and
huniq -c replaces
sort | uniq -c. The order of the output is stable when in normal mode, but it is not stable when in -c/count mode.
Another tool that I and many other people use very frequently is Git, a version-control system for tracking changes in computer files. And when using Git, as well as other similar tools, seeing what changed in a particular file is important. Normally you rely on your favorite Git tool to compare two versions of a file, but if you need to quickly compare two files on the command line, independent of your Git tool, you can rely on:
– delta: a viewer for git and diff output. It has support for
diff-so-fancy emulation modes, too.
– diffoscope: when you really need to see the difference between not only same type of files but all kinds of different artifacts. This program helps you for in-depth comparison of files, archives, and directories. diffoscope tries to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human-readable form to compare them. It can compare two tarballs, ISO images, or PDF just as easily.
Next in line, we have an alternative for the venerable
ping utility. I think having a bit more information from ping wouldn’t hurt:
– prettyping: runs the standard
ping in background and parses its output, showing ping responses in a graphical way at the terminal (by using colors and Unicode characters). It’s written in
awk, and should work out-of-the-box in most systems (Linux, BSD, Mac OS X, …). It is self-contained in only one file, so it is trivial to install and run. (You can also give gping a try).
– bmon: Bandwidth Monitor is a monitoring and debugging tool to capture networking related statistics and prepare them visually in a human friendly way. It features various output methods including an interactive curses user interface and a programmable text output for scripting. I think of this utility in the same category as iptraf and iptraf-ng.
Another relatively less known tool is
pv , the pipe viewer: it is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion:
One thing that I also do at the command line is to trigger an event whenever there’s some change to some files & directories. Of course, you can
watch for things, and even utilize inotify subsystem, but maybe you’ll find the following more convenient to quickly build a command line event triggering system:
– watchexec: a simple, standalone tool that watches a path and runs a command whenever it detects modifications. Among the use cases are automatically running unit tests and run linters/syntax checkers whenever there’s a pre-defined change in a directory and its sub-directories.
And finally, a few honorable mentions:
– bottom: A cross-platform graphical process/system monitor with a customizable interface and a multitude of features. Written in Rust, and supports GNU/Linux, macOS, and Windows. Inspired by both gtop and gotop.
– exa: a modern replacement for ls. It’s an improved file lister with more features and better defaults. It uses colours to distinguish file types and metadata. It knows about symlinks, extended attributes, and Git. And it’s small, fast, and just one single binary. Source code is also available at https://github.com/ogham/exa.
– PathPicker: accepts a wide range of input — output from git commands, grep results, searches — pretty much anything.After parsing the input, PathPicker presents you with a nice UI to select which files you’re interested in. After that you can open them in your favorite editor or execute arbitrary commands.
– svg-term-cli: Share terminal sessions as razor-sharp animated SVG everywhere. Useful for demonstrations and information sharing, error-reporting, etc.
– moreutils: “a growing collection of the Unix tools that nobody thought to write long ago when Unix was young.” Once you get used to them, you’ll wonder how you managed to live without them.
– loop: “UNIX’s missing
loop lets you write powerful, intuitive looping one-liners in your favorite shell! Finally, loops in Bash that make sense! Imagine combining
loop with moreutils and wielding all that automation power into one-liners!
– xpipe: “reads input from stdin and splits it by the given number of bytes, lines, or if matching the given pattern. It then invokes the given utility repeatedly, feeding it the generated data chunks as input. You can think of it as a Unix love-child of the split(1), tee(1), and xargs(1) commands.”
– starship: ☄🌌️ The minimal, blazing-fast, and infinitely customizable prompt for any shell. Great installation experience with sane out-of-the-box defaults. See the demo below:
tokei: count your code, quickly. This is a program that displays statistics about your code. Tokei will show the number of files, total lines within those files and code, comments, and blanks grouped by language. Tokei is very fast, and is able to count millions of lines of code in seconds.