- Issue created by @robpowell
Our site is not behind a CDN and we are seeing a spike in traffic to our search page specifically with facet url parameters populated. My understanding is search pages cannot be cached so any hit to the search page is a full drupal boot and query to the search backend - but I wouldn't mind a verification of this. Also, since facets are rendered as links, bots seem to have a field day with them and can get locked into continuously crawling search results. We are still experiencing a spike in traffic even though links/facets are no follow and our robots.txt disallows traffic to our search page.
This problem is not limited to just our sites and other's in the community are experiencing it as well β .
* create a drupal search experience with facets
* publish/make live without any cdn
* take a look at access logs to review traffic.
I solved this by removing the facet that was being hammered by suspicious traffic. I also updated our .htaccess to redirect all traffic to our homepage if the url args did not match our approved facet list. The hope here is that our homepage has a better cache hit .
However, I think what would be more helpful is either a search page/search index specific facet limit or a per facet limit. Just to be clear, the limit here is the number of facets a user can select via the UI AND pass in the URL.
Here are some thoughts about approach:
Active
3.0
Code