/print/pdf content appears in search engines

Open on Drupal.org →

Created on 21 June 2024, 10 months ago

Problem/Motivation

I had many of my site's "print/pdf" URLs indexed by Google and Bing. In my attempts to remove them and stop this from re-occurring, I wonder if there are some features the module could make available, perhaps.

For example, could we add options in the UI to let users turn on things like:

- adding nofollow attribute to Entity-Print-generated links

- adding nofollow and noindex headers to Entity Print routes

- adding noindex meta tag to entity-print.html.twig (not sure if does anything)

I'm not sure this has happened to others but it could be a nice feature to let users enable these or have them enabled by default so that search engines don't view the PDFs.

Steps to reproduce

Enable the module and add links to PDF URLs on content pages without adding "nofollow" attribute to the links.

Proposed resolution

Add elements to the UI to allow users to enable noindex and nofollow attributes where needed. Or add these attributes as a default and add element to UI to allow them to be disabled.

I was able to use the patch from https://www.drupal.org/project/entity_print/issues/3202847 ✨ Possibility to alter http headers when we generate PDF Postponed: needs info to allow me to add the X-Robots-Tag HTTP header. Perhaps this could be something users could turn on via the UI?

✨ Feature request

Status

Active

Version

2.14

Component

Miscellaneous

Created by

🇨🇦Canada endless_wander

Live updates comments and jobs are added and updated live.

Sign in to follow issues

Comments & Activities

Issue created by @endless_wander
Comment 8 months ago →
🇺🇸United States routinet
Second this.

We're getting hammered from some AI/ML bots scraping our site. Our current approach is to use robots.txt to disallow the suspects. Adding the ability to specify arbitrary attributes to add to the links created by entity_print would be great. If not arbitrary attributes, at least the standards for "rel".
Comment 4 months ago →
🇨🇦Canada endless_wander
This is still happening for me and appears to be happening on a high-profile Drupal site as well london.gov.uk

https://www.google.com/search?q=site%3Alondon.gov.uk%2Fprint&rlz=1C1GCEA...

Any ideas for preventing this?
Comment 3 months ago →
🇫🇷France tednacip
I have the same issue. Did you find a solution ?
Thanks
Comment 2 months ago →
🇨🇦Canada endless_wander
@tednacip -- see custom code at https://www.drupal.org/project/entity_print/issues/3202847 ✨ Possibility to alter http headers when we generate PDF Postponed: needs info and not that it has to be X-Robot and not X-Robots

Would be awesome if this could be added directly to the module, but it seems like nobody is paying attention here.

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024