Zstandard

Help Wanted

This section is in need of contributions. If you believe you can help, please see our Contribution Guide to get started as a contributor!

Zstandard is a compression algorithm developed by Facebook known for its extremely fast decompression speeds. It was released in early 2015 and is used in a variety of different contexts. It was designed to perform similarly to older Deflate-based compression algorithms like ZIP or gzip while being faster overall. In practice, it is said to compress similarly to pure LZMA (part of XZ & 7-zip) while being much faster.

While .tar.zstd archives aren't as popular as .tar.xz or .tar.gz, Zstandard is already a very popular tool for compression in the world of open-source software. It has been integrated into both the FreeBSD kernel & the Linux kernel and is available as a filesystem compression method for the btrfs, squashfs, bcachefs, & OpenZFS filesystems. Filesystem compression refers to a compression scheme that transparently compresses files stored on a filesystem at all times, leading to an overall reduction in storage used across the filesystem.

The command line zstd utility can compress to Zstandard at compression levels 1 through 19 by default. The upper bound is raised to 22 when passing the --ultra flag. All Arch Linux packages are compressed at zstd level 20, allowing Arch packages to be decompressed 14 times faster compared to XZ at the cost of an average 0.8% filesize increase across all packages. It is popular in the game emulation scene as well, as many game file formats for emulating console games support zstd compression. The ZIP file format standard actually supports Zstandard in compression level 93 since version 6.3.8, published in 2020. Content encoding using zstd is supported in chromium since Chromium 118 behind an experimental flag, meaning it might compete with Brotli on the web in the future. Apple's LZFSE algorithm is purportedly similar to Zstandard compression level 6.

Zstandard has the potential to effectively compete with nearly every modern compression method available across most modern use cases. In certain scenarios, if it takes off as a content delivery format, it could replace Brotli if the benefits of super-fast & super-light decode improve the responsiveness of web pages & are worth sacrificing a bit of compression ratio. When using the much higher effort settings, it often outcompetes Brotli for the archive size as well. In the future, .tar.zst could replace 7-zip, ZIP, or other archiving formats, making speedy decode a reality on systems featuring varying levels of compute horsepower.

Usage

note

This guide has been written for the zstd command-line utility, however GUI archivers such as peazip and 7zip have growing support for zstd.

Compress a file

Like many other compressing utilities, in order to compress mutliple files, one should probably archive them with tar.

zstd -# {file} -o {file}.zstd

-# is actually a number that represents the desired compression level, for example -3, -6. By default you can specify 1-19. By also passing -ultra, you can go up to compression level 22.

Decompress a file

zstd -d {file}.zstd -o file