Improve cacheTagsToHashes method and cache tag compression

Would it be possible make the hash length customizable or at least increase it 8? I have run into an issue where the first 4 digits of the hash match another and it ends up clearing a lot more pages than intended.

What a coïncidence, i just met the same issue.

My collegue and i are working a better solution but it s not easy.
We can't just increase hash and hoping for the best.
It will decrease duplicate x tag but not enough.

Also, we can't just add a long string for x tag.
X tag, as it s actually made, is really useful to prevent nginx to raise an exception for too big header.

If we can post better cache tags compression, we will do.

Hey guys!

We can't just increase hash and hoping for the best.
It will decrease duplicate x tag but not enough.

That's correct. The X tag was initially added to avoid issues with the too big header exceptions.

We can think of mirroring the solution from Acquia Purge, something like a

  /**
   * Create a hash with the given input and length.
   *
   * @param string $input
   *   The input string to be hashed.
   * @param int $length
   *   The length of the hash.
   *
   * @return string
   *   Cryptographic hash with the given length.
   */
  protected static function hashInput($input, $length) {
    // MD5 is the fastest algorithm beyond CRC32 (which is 30% faster, but high
    // collision risk), so this is the best bet for now. If collisions are going
    // to be a major problem in the future, we might have to consider a hash DB.
    $hex = md5($input);
    // The produced HEX can be converted to BASE32 number to take less space.
    // For example 5 characters HEX can be stored in 4 characters BASE32.
    $hash = base_convert(substr($hex, 0, ceil($length * 1.25)), 16, 32);
    // Return a hash with consistent length, padding zeroes if needed.
    return strtolower(str_pad(substr($hash, 0, $length), $length, '0', STR_PAD_LEFT));
  }

and make the $length configurable. So in case of the collisions you can configure the hash length as you wish.

#8 is a great idea.

But we have a better algorithm that's suppose to prevent any collision.
If it work's, x tag may be a bit longer for some occasion (for complexe cache tags like 'segment1:segment2:segment3:segment4').

But it's not the common case.

For exemple, 'node:76' enconding should on two caracters (if i remember correct).
One letter and on digit.

Here's some example of compression :

Cache tag : "agency:yvelines:houilles"
Custom compression : 1L2X2N
#8 solution : ikaa11ev

Cache tag : "node:76"
Custom compression : 1c1e
#8 solution : fgrjero9

Cache tag : "node:67"
Custom compression : 1c15
#8 solution : gn4fonhc

Cache tag : "abcd"
Custom compression : K
#8 solution : sbu72j27

Cache tag : "agency:yvelines:andagainoadsglkfghalkefgjkasdfbglkasdbglkabsdlkgjasdfkjgnasdg"
Custom compression : 1L2XgM
#8 solution : se47ecf1

Here's our full solution

This patch must be install with #5

Please let me know if you don't want our solution, we will move it in a custom CacheManager and override service injection (to prevent to keep so much change in a patch)

Small improvement to have only one patch and pass x-tag in lowercase because varnish refuse to invalidate uppercase x-tag

Automatically closed - issue fixed for 2 weeks with no activity.

I made a mistake in #15.
I erase the addition of letters.

With that, X-Tag value is nothing more than the value of last letter (a bit annoying).

Sorry about that.
Here's the fix.

@shumer can you publish the fix asap pls ?

The fun fact, we had no reports since the last release... which was a while ago :)
Let me publish fixed version