Lably
Skip to content

Tabulation hashing

Hash functions computed by exclusive or

  • 13 min read
  • 5 chapters

Related on Lably

Through Computer science, Tuple and Polynomial

Despite its simplicity, tabulation hashing has strong theoretical properties that distinguish it from some other hash functions. In particular, it is 3-independent: every 3-tuple of keys is equally likely to be mapped to any 3-tuple of hash values. However, it is not 4-independent. More sophisticated but slower variants of tabulation hashing extend the method to higher degrees of independence.

Because of its high degree of independence, tabulation hashing is usable with hashing methods that require a high-quality hash function, including hopscotch hashing, cuckoo hashing, and the MinHash technique for estimating the size of set intersections.

Method

The basic idea is as follows:

First, divide the key to be hashed into smaller "blocks" of a chosen length. Then, create a set of lookup tables, one for each block, and fill them with random values. Finally, use the tables to compute a hash value for each block, and combine all of these hashes into a final hash value using the bitwise exclusive or operation.

More formally:

Let p be the number of bits in a key to be hashed, and q be the number of bits desired in an output hash function. Choose a block size r ≤ p; the choice of block size controls the tradeoff between time and memory usage, so it should be made so that the tables are not too large, e.g., so that the tables fit into the computer's cache memory. Smaller blocks use less memory but slow down the hash function. Compute t = ceil(p/r), the number of r-bit blocks needed to represent a key.

Create a two-dimensional 2r × t array, T, and fill it with random q-bit numbers. Now T can be used to compute the hash value h(x) of any given key x. To do so, partition x into r-bit values, where x0 consists of the lowest r bits of x, x1 consists of the next r bits, etc. For example, if r = 8, then xi is just the ith byte of x. Then, use these r-bit and position values as indices into T, and combine the results using the exclusive or operation:

h(x) = T[0][x0] ⊕ T[1][x1] ⊕ T[2][x2] ⊕ ... ⊕ T[t-1][xt-1].

Note that it is not valid to use the same table (e.g. T[0]) for each xi, since then the hash function would not be able to distinguish between strings with the same xis, but permuted differently.

Code for a typical example with r = t = 8 and q = p = 64 is given below.

<span id="mwcg"></span><span class="c1" id="mwcw">// Secret table of random numbers</span> <span class="kt" id="mwdA">uint64_t</span><span class="w" id="mwdQ"> </span><span class="n" id="mwdg">T</span><span class="p" id="mwdw">[</span><span class="mi" id="mweA">8</span><span class="p" id="mweQ">][</span><span class="mi" id="mweg">256</span><span class="p" id="mwew">];</span> <span class="k" id="mwfA">for</span><span class="w" id="mwfQ"> </span><span class="p" id="mwfg">(</span><span class="kt" id="mwfw">int</span><span class="w" id="mwgA"> </span><span class="n" id="mwgQ">i</span><span class="w" id="mwgg"> </span><span class="o" id="mwgw">=</span><span class="w" id="mwhA"> </span><span class="mi" id="mwhQ">0</span><span class="p" id="mwhg">;</span><span class="w" id="mwhw"> </span><span class="n" id="mwiA">i</span><span class="w" id="mwiQ"> </span><span class="o" id="mwig"><</span><span class="w" id="mwiw"> </span><span class="mi" id="mwjA">8</span><span class="p" id="mwjQ">;</span><span class="w" id="mwjg"> </span><span class="n" id="mwjw">i</span><span class="o" id="mwkA">++</span><span class="p" id="mwkQ">)</span> <span class="w" id="mwkg"> </span><span class="k" id="mwkw">for</span><span class="w" id="mwlA"> </span><span class="p" id="mwlQ">(</span><span class="kt" id="mwlg">int</span><span class="w" id="mwlw"> </span><span class="n" id="mwmA">j</span><span class="w" id="mwmQ"> </span><span class="o" id="mwmg">=</span><span class="w" id="mwmw"> </span><span class="mi" id="mwnA">0</span><span class="p" id="mwnQ">;</span><span class="w" id="mwng"> </span><span class="n" id="mwnw">j</span><span class="w" id="mwoA"> </span><span class="o" id="mwoQ"><</span><span class="w" id="mwog"> </span><span class="mi" id="mwow">256</span><span class="p" id="mwpA">;</span><span class="w" id="mwpQ"> </span><span class="n" id="mwpg">j</span><span class="o" id="mwpw">++</span><span class="p" id="mwqA">)</span> <span class="w" id="mwqQ"> </span><span class="n" id="mwqg">T</span><span class="p" id="mwqw">[</span><span class="n" id="mwrA">i</span><span class="p" id="mwrQ">][</span><span class="n" id="mwrg">j</span><span class="p" id="mwrw">]</span><span class="w" id="mwsA"> </span><span class="o" id="mwsQ">=</span><span class="w" id="mwsg"> </span><span class="n" id="mwsw">getRandomUInt64</span><span class="p" id="mwtA">();</span> <span class="c1" id="mwtQ">// Simple Tabulation Hash function</span> <span class="kt" id="mwtg">uint64_t</span><span class="w" id="mwtw"> </span><span class="nf" id="mwuA">hash</span><span class="p" id="mwuQ">(</span><span class="kt" id="mwug">uint64_t</span><span class="w" id="mwuw"> </span><span class="n" id="mwvA">x</span><span class="p" id="mwvQ">)</span><span class="w" id="mwvg"> </span><span class="p" id="mwvw">{</span> <span class="w" id="mwwA"> </span><span class="kt" id="mwwQ">uint64_t</span><span class="w" id="mwwg"> </span><span class="n" id="mwww">res</span><span class="w" id="mwxA"> </span><span class="o" id="mwxQ">=</span><span class="w" id="mwxg"> </span><span class="mi" id="mwxw">0</span><span class="p" id="mwyA">;</span> <span class="w" id="mwyQ"> </span><span class="k" id="mwyg">for</span><span class="w" id="mwyw"> </span><span class="p" id="mwzA">(</span><span class="kt" id="mwzQ">int</span><span class="w" id="mwzg"> </span><span class="n" id="mwzw">i</span><span class="w" id="mw0A"> </span><span class="o" id="mw0Q">=</span><span class="w" id="mw0g"> </span><span class="mi" id="mw0w">0</span><span class="p" id="mw1A">;</span><span class="w" id="mw1Q"> </span><span class="n" id="mw1g">i</span><span class="w" id="mw1w"> </span><span class="o" id="mw2A"><</span><span class="w" id="mw2Q"> </span><span class="mi" id="mw2g">8</span><span class="p" id="mw2w">;</span><span class="w" id="mw3A"> </span><span class="n" id="mw3Q">i</span><span class="o" id="mw3g">++</span><span class="p" id="mw3w">)</span> <span class="w" id="mw4A"> </span><span class="n" id="mw4Q">res</span><span class="w" id="mw4g"> </span><span class="o" id="mw4w">^=</span><span class="w" id="mw5A"> </span><span class="n" id="mw5Q">T</span><span class="p" id="mw5g">[</span><span class="n" id="mw5w">i</span><span class="p" id="mw6A">][(</span><span class="kt" id="mw6Q">char</span><span class="p" id="mw6g">)(</span><span class="n" id="mw6w">x</span><span class="w" id="mw7A"> </span><span class="o" id="mw7Q">>></span><span class="w" id="mw7g"> </span><span class="mi" id="mw7w">8</span><span class="o" id="mw8A">*</span><span class="n" id="mw8Q">i</span><span class="p" id="mw8g">)];</span> <span class="w" id="mw8w"> </span><span class="k" id="mw9A">return</span><span class="w" id="mw9Q"> </span><span class="n" id="mw9g">res</span><span class="p" id="mw9w">;</span> <span class="p" id="mw-A">}</span>

Sign in to keep reading

4 more chapters are free with a Lably account.

More in Computing and communication