xxHash … Fast, Fast, Fast

xxHash … Fast, Fast, Fast

For hashing, we take some data and create a hashed value of the data. It should then not be possible to reverse it back. In some applications, such as with Hashtables and Bloom filters, we often do not need the complexity of the cryptography methods (such as MD5, SHA-1 and SHA-256), and need methods that are fast and efficient. These point towards the non-cryptography methods, and which can be over a 100 times faster than the cryptography methods.

One of the fastest around is the xxHash and was created by Yann Collet. A significant speed improvement is achieved on processors that support SSE2, and which is an extension to the IA-32 architecture. This, of course, limits the architecture range for its implementation. Overall xxHash works at close RAM limits. In a recent test, xxH3 achieved a hashing rate of 31GB/s, and was especially efficient for a small amount of data (such as with text strings). We can see that xxh3 and xxh128 beat many of the other non-cryptography hashes, in both the performance and for overall quality:

Blake2 is seen to be one of the fastest cryptography methods, and beats SHA-1 for throughput. In this test, xxHash is shown to be around 30 times higher. One of the strengths of xxHash is that it has good hashing performance for short amounts of data (such as with strings). In the following, we see the xxh3 manages to keep a consistently good hash rate for small amounts of data input:

A sample of the code is:

https://asecuritysite.com/encryption/smh_xxhash

And here is the running code:

Hashtables are wonderful infrastructures, and where we can quickly identify our data using a hashed version. In large data infrastructures, especially ones that are distributed, we often need a fast method of hashing data that could be small strings or large data objects. For this, xxHash is a great contender for a fast and efficient hashing method.