Hotjar and other random library cookie consent

Hi @joseph.olstad,

I'm on vacation currently, so just giving a quick first answer.
In general it should be possible and quite easy to provide hotjar support by a cookies_hotjar submodule (that should better live in the hotjar module than as further submodule here).

Take a look at the implementation of the serveral cookies_* submodules for GTM, Facebook Pixel, ... and also at the cookies_etracker submodule in the etracker module.

Cookies works a bit different from eu_cookie_compliance and needs some lines of code in a submodule to integrate with further services. But I think it's more straight-forward, flexible and reliable that way! (We used eucc for D7 and D8 but made the switch to COOKiES finally).

After reading through the submodules code and comparing them, I think you'll understand how it works. PHP does the server-side knockout and adds information on how to load the knocked-out libraries, once consent is given via JS.

I'm looking forward to your feedback and hotjar support. Feel free to open further issues for documentation and improvements as a first-time-user.

Sadly the original developer JFeltkamp is completely inactive since several months, seems like he quit Drupal. :( FYI

@joseph.olstad I added some related issues that I think will bring some light into this confusion. We didn't have the time & money to fix them yet, but reading them might help you.

I think especially #3323524: [META] Improve combination of COOKiES submodules and their services through "Drupal Plugins" → is where your confusion comes from. Looking forward to your feedback.
So these are especially historic reasons and 2.x should clean these up.

@Anybody, thank you very much for the response. This is very helpful. I will aim to get something prepared and ready shortly.

@joseph.olstad have you been successful or do you need any further help?

Looking forward to your feedback and reports! :)

There's an open patch for Cookies Google tag Manager

Which cookies module should model from? Or third party module?

@joseph.olstad; Yes, we're aware of that and hope we can fix that soon.

Well I think
https://git.drupalcode.org/project/cookies/-/tree/2.x/modules/cookies_ga
or
https://git.drupalcode.org/project/cookies/-/tree/2.x/modules/cookies_ma...

might be a good starting point for hotjar implementation.

Otherwise perhaps https://git.drupalcode.org/project/etracker/-/tree/8.x-3.x/modules/cooki...

Shell we close this issue then?

@Anybody,

I modelled off of cookies_etracker , with that said, they use an id called _htLoader ,

however hotjar does not use this approach.

What I need to do is knock out this javascript:
hotjar:public://hotjar/hotjar.script.js

it shows up in the head as follows:
<script src="/sites/default/files/hotjar/hotjar.script.js"></script>

We desperately are needing to figure out how to knock out this js.

The attachment name I'm unsure, it may be something like:
['attachment']['hotjar_snippet_path']

not sure.

I'll upload ASAP the cookies_hotjar module I created so far, it's not working of course because _hjLoader is just something I renamed from _etLoader and hotjar doesn't do this approach.

I think we have a similar submodule, where we simply replaced "src" with "data-src" to temporarily knock out the js.

If I remember correctly, "cookies_instagram", "cookies_facebook_pixel" etc. could be a good template.

@Grevil, thank you so much for the suggestions, I will have a look at those two, cookies_instagram and cookies_facebook_pixel soon

Ok I got it working, but none of those examples helped.

I had to adjust the javascript to blow away all hotjar cookies by setting an expired cookie of the same name and the same domain name.

See the module code.

so now when the consent is refused, the cookies are deleted as soon as they're created. None of the other stuff really matters at this point.

The submodule I just uploaded is functional. There's french text in the config yml, but otherwise, it seems to function well and blocks the hotjar cookies upon refusal and when consent is granted, they come back.

actually, I hard coded the id in the cookie names, they should be using the hotjar id configuration value

Hotjar ID

so, the cookies_hotjar module needs a bit of work, but it's almost there.

Ok I got it working, but none of those examples helped very much other than to get it started

Sorry to hear that, because of your comment in #11:

What I need to do is knock out this javascript:
hotjar:public://hotjar/hotjar.script.js

I thought, you simply want to knock out that js file, and exactly that is done in the mentioned submodule's. Note, that we are knocking out the scripts server-side via php!

Please do not upload your code changes inside a zip. You never know what is inside an unknown zip file!

Instead, please create an MR containing your code changes (aka your submodule).

Wouldn't it instead be correct to block the script tag from being attached?

This is added here in hotjar:

  /**
   * Add page attachments when Build mode is in use.
   */
  protected function pageAttachmentBuilt(array &$attachments) {
    $uri = $this->getSnippetPath();
    $query_string = $this->state->get('system.css_js_query_string') ?: '0';
    $query_string_separator = (strpos($uri, '?') !== FALSE) ? '&' : '?';

    $url = $this->fileUrlGenerator->generateString($uri);
    $attachments['#attached']['html_head'][] = [
      [
        '#type' => 'html_tag',
        '#tag' => 'script',
        '#attributes' => ['src' => $url . $query_string_separator . $query_string],
      ],
      'hotjar_script_tag',
    ];
  }

https://git.drupalcode.org/project/hotjar/-/blob/8.x-3.x/src/SnippetBuil...

I think that's the way other coookies_* modules do it and re-enable it via JS?
Knockout happens by setting a different type instead of script, see here:
https://git.drupalcode.org/project/cookies/-/blob/2.x/modules/cookies_ma...

Giving it an idea and type allows to later set "script" as type again so the browser loads and runs the JS.

Or am I missing something specific to hotjar (which is absolutely possible). Just wanted to let you know, how it works typically.

PS: You can identify it by 'hotjar_script_tag'

I've gotten it working through the javascript of cookies, these are all varnish cached pages so I'd imagine this PHP example in #21 would be cached and therefore this could break hotjar for others.

With that said, I've got to integrate the config lookup for the hotjar id because the cookienames use the hotjar id in the name of the cookie, once I do that, this cookies_hotjar module will be capable of functioning for anyone using hotjar with the cookies_hotjar module that is built on top of cookies.

I've gotten it working through the javascript that writes expired hotjar cookies (deletes them) with a refusal. Keep in mind, we're using varnish, these are all varnish cached pages so I'd imagine this PHP example in #21 would be cached and therefore this could break hotjar for others.

No I don't think so and that's why cookies only knocks out scripts server-side. Keeping them knocked-out (no consent given) or reviving them is all done in JS!

I also had to understand that concept first and remember why it's important to do it this way. So I still think that would be the correct way (of course I might be wrong). But I think the key problem here is to document and understand how COOKiES works, which is quite simple once you got it.

TL;DR:

1. Server-side knockout of Libraries / JS-loading
2. Client-Side reviving of the knocked-out libraries, if consent is given

btw. that's very similar to how other cookie consent tools, like Cookiebot, ... do it.

hmm, ok, the issue with this approach is, existing tracking cookies prior to the implementing of consent/refusal are still present after a refusal. Deleting the cookies seems like a clean approach from what I can see, guaranteeing that tracking cookies are blown away on refusal.

@joseph.olstad thank you! :)

Deleting the cookies indeed isn't something the module currently does. See #3217333: Accepted cookies are not deleted when consent is removed →
I'm not against adding that functionality, but it also means, we need to track the cookie names. Let's discuss the details in that issue, if you think we should implement that.
But some last words on that: I think the module already offers, what it needs to do that smart, with the JS callbacks. We just need to either add another optional callback on refusal, which deletes a list of cookies by their name or add a helper function that can be called in the existing callback... We can at least add that optionally. It has to be done in JS for the caching reasons.
I'd be happy to read your feedback in the other issue to implement a general solution.

Back to the other points:
A cookie consent, at least in the EU, needs both:

1. Protect the user from calling third party sources without the user consent (as already the call is illegal as it's transferring the users IP, which is seen as personal data)
2. Prevent cookies from being set by the third party sources without consent - this is typically combined with the first point, as typically the blocked sources / libraries are setting the cookie.

I know it can be confusing first, after using EUCC which is UI-centric and kind of magic, while COOKiES is more code (submodule) based and specific to the services and libraries used, but after using EUCC for many years, I think the EUCC way is the more hacky one... (not to blame, but that's why we finally left EUCC)

So to sum things up: After understanding how COOKiES works, we should now focus on two things as result of this issue:

1. Implement a convenient way to remove cookies on refusal: #3217333: Accepted cookies are not deleted when consent is removed →
2. Better document how COOKiES works and how to implement submodules for further services to onboard users like you

Do you agree or are there questions left? Would you be willing to help a bit, at least by adding some external feedback & thoughts?

Here's a working cookies_hotjar module that requires no special configuration, just requires hotjar to be configured and for cookies to be installed and configured. Installing this module should just work right out of the box unless I missed something in the configuration.

There's noise in the module code because I converted cookies_etracker to cookies_hotjar and haven't yet cleaned it up 100% yet.

With that said, the actual implementation is quite simple here, refusal will delete all cookies using the hotjar account id passed in as a drupalSetting to js and retrieve the session id from hotjar using js that exists when hotjar is installed so that all hotjar cookies are vapourized and no tracking occurs.

I expect this module to be Drupal 10.1.x+ compatible and it works on D9.5 also.

You can examine the delete cookies approach, it's pretty simple. I removed all hard coded ids, it's all dynamically figuring out which cookie names to delete and which domain name to use when deleting. The deleting is to set the expiry date to -1
so -1 means it's expired, the web browser automatically cleans up the mess.

Hada look at the patch, it works, however I think this js library can be removed, it's not needed with the delete cookie approach.

modules/cookies_hotjar/js/cookies_hotjar-block_cookies_true_without_consent.js

The good stuff that actually is needed is in this js file:

modules/cookies_hotjar/js/cookies_hotjar-knockout_without_consent.js

function setCookie(cname, cvalue, exdays, domain) {
  const d = new Date();
  d.setTime(d.getTime() + (exdays * 24 * 60 * 60 * 1000));
  let expires = "expires=" + d.toUTCString();
  document.cookie = cname + "=" + cvalue + ";" + expires + ";path=/;domain=" + domain;
};
function removeHotjarCookies(account) {
  var cookie_domain = getLastTwoSegments(window.location.origin);
  setCookie('_hjFirstSeen', 0, -1, cookie_domain);
  setCookie('_hjAbsoluteSessionInProgress', 0, -1, cookie_domain);
  setCookie('_hjIncludedInSessionSample_' + account, 0, -1, cookie_domain);
  if (typeof _hjSettings != 'undefined' && _hjSettings.hjid > 0) {
    setCookie('_hjSession_' + _hjSettings.hjid, 0, -1, cookie_domain);
  }
  setCookie('_hjSessionUser_' + account, 0, -1, cookie_domain);
  setCookie('_hjDonePolls', 0, -1, cookie_domain);
  setCookie('_hjid', 0, -1, cookie_domain);
};
function getHostName(url) {
  var match = url.match(/:\/\/(www[0-9]?\.)?(.[^/:]+)/i);
  if (match != null && match.length > 2 && typeof match[2] === 'string' && match[2].length > 0) {
    return match[2];
  }
  else {
    return null;
  }
}
function getDomain(url) {
  var hostName = getHostName(url);
  var domain = hostName;
  if (hostName != null) {
    var parts = hostName.split('.').reverse();
    if (parts != null && parts.length > 1) {
      domain = parts[1] + '.' + parts[0];
      if (hostName.toLowerCase().indexOf('.co.uk') != -1 && parts.length > 2) {
        domain = parts[2] + '.' + domain;
      }
    }
  }
  return domain;
}
function getLastTwoSegments(url) {
  var domain_name = getDomain(url);
  var strs = domain_name.split('.');
  var key_str = '.' + strs[0] + '.' + strs[1];
  return key_str;
}

This is the code that actually does the trick.

is called as follows:
removeHotjarCookies(settings.cookies_hotjar.account);
This hotjar account settings is passed into javascript in the cookies_hotjar.module

We're going to be putting this module through a plethora of additional testing this week and next week.

I'll provide an updated version maybe next week some time.

Thank you @joseph.olstad - let's proceed in the referenced issues afterwards!