eXtendable-Output Functions (XOFs)

eXtendable-Output Functions (XOFs)

Our existing hashing functions are split into two main categories. The first is the cryptography hashes, such as MD5, SHA-1, SHA-256 and SHA-3 [here]. These are typically fast methods and which can take any amount of data as an input and produce a fixed hash output — a fingerprint of the data. The other type of hash is the non-cryptographic hash and these tend to be even faster than the cryptographic hashes, but have a lesser security strength. Typical non-cryptographic methods are SipHash, xxHash and FarmHash [here]. These hashes are typically used for indexing such as with hash tables and Bloom filters. But, there’s another type of permutated output, and these are defined as eXtendable-Output Functions (XOFs).

XOFs differ from hashing functions in that they can produce any length of output — from one byte to an infinitely long byte stream. They can thus be used in applications where we need something larger or smaller than the 256-bit (16 bytes) or 512-bit (32 bytes) output value that our cryptographic hashing methods produce. For cryptographic methods, most modern applications now use SHA-256 (256-bit hash), SHA-512 (512-bit hash) or SHA-3 (256-bit).

In designing a modern hashing the focus is typically on a fixed hash output, but designers also implement a variable-length output for XOF. Overall, the main XOF methods are SHAKE128, SHAKE256, SHAKE128, SHAKE256, BLAKE2XB and BLAKE2XS. With the SHA-3 hashing method, we have four different cryptographic hashing methods (SHA3–224, SHA3–256, SHA3–384, and SHA3–512) and two XOF functions (SHAKE128 and SHAKE256). With BLAKE2b — one of the fastest cryptographic hashing methods — we have an XOF of BLAKE2XB, and for BLAKE2s we have an XOF of BLAKE2XS. With a string input, we will get a digest output of a given byte length. For a random input, we can generate a digest of the random values for a given output size.

The applications of XOF include creating hashes of variable lengths and in creating encryption keys based on passwords. While HKDF focuses on creating an encryption key from passwords, XOFs have the advantage of producing a variable output size for the key. As XOFs always produce the same output for a given input, it is strongly recommended that a salt value is used with the password (and then stored for recovery). There are also applications of XOF into stream encryption, where we can create a seed value and then produce an infinitely long encryption key. This input key can then simply be XOR-ed with the data stream to produce the cipher stream.

So let’s implementation some of the main methods using the Cloudflare CIRCL library. The following is an outline of the code [here]:

package main
import (
 "fmt"
 "os"
 "strconv"
 "github.com/cloudflare/circl/xof"
)
func main() {
 length := 32
 v := "The quick brown fox jumps over the lazy dog"
 meth := "BLAKE2XS"
 X := xof.BLAKE2XS.New()
 argCount := len(os.Args[1:])
 if argCount > 0 {
  meth = os.Args[1]
 }
 if argCount > 1 {
  v = os.Args[2]
 }
 if argCount > 2 {
  length,_ = strconv.Atoi(os.Args[3])
 }

 if meth == "SHAKE128" {
  X = xof.SHAKE128.New()
 }
 if meth == "SHAKE256" {
  X = xof.SHAKE256.New()
 }
 if meth == "SHAKE128" {
  X = xof.SHAKE128.New()
 }
 if meth == "SHAKE256" {
  X = xof.SHAKE256.New()
 }
 if meth == "BLAKE2XB" {
  X = xof.BLAKE2XB.New()
 }
 if meth == "BLAKE2XS" {
  X = xof.BLAKE2XS.New()
 }
 
 _, _ = X.Write([]byte(v))
 fmt.Printf("Input: %s\n", v)
 fmt.Printf("Method: %s\n", meth)
 fmt.Printf("Size: %d bytes\n", length)
 xof_out := make([]byte, length)
 _, _ = X.Read(xof_out)
 fmt.Printf("\nHash: %x\n", xof_out[:length])
}

A sample run for SHAKE128 for an output of 256 bytes is [here]:

Input: The quick brown fox jumps over the lazy dog
Method: SHAKE128
Size: 256 bytes
XOF: f4202e3c5852f9182a0430fd8144f0a74b95e7417ecae17db0f8cfeed0e3e66eb5585ec6f86021cacf272c798bcf97d368b886b18fec3a571f096086a523717a3732d50db2b0b7998b4117ae66a761ccf1847a1616f4c07d5178d0d965f9feba351420f8bfb6f5ab9a0cb102568eabf3dfa4e22279f8082dce8143eb78235a1a54914ab71abb07f2f3648468370b9fbb071e074f1c030a4030225f40c39480339f3dc71d0f04f71326de1381674cc89e259e219927fae8ea2799a03da862a55afafe670957a2af3318d919d0a3358f3b891236d6a8e8d19999d1076b529968faefbd880d77bb300829dca87e9c8e4c28e0800ff37490a5bd8c36c0b0bdb2701a

Conclusions

XOFs — they are fast, efficient, and all ready to build a new digital world. If you want to learn more about CIRCL, try here:

https://asecuritysite.com/circl

and for hashing methods:

https://asecuritysite.com/hash

and:

https://asecuritysite.com/hash2