Thanks. Yes all the data is being returned from Solr, I've confirmed using the page you provided. The problem is, when a search is performed, it caches that search in the 'cache_page' table, that table can at times get to GB in size in a matter of hours.
Example row you might see in that table (data truncated for illustration purposes):
If you're getting hit thousands of times per minute, there's a lot of potential SQL queries taking place.
All of my data is stored in Solr however what I've noticed is, when the site is being hit hard the MySQL usage will spike due to it writing and retrieving from the cache - I haven't found a way to turn this off for Solr search results returned in a view.
Thank you all, some really useful information. Unfortunately for the time being I'm unable to implement a WAF, however it is on my list of things to investigate.
What I ended up doing after interrogating the access logs was adding a few lines to my htaccess, the bot in question always seemed to pass a 'photo_id' parameter at the end of the facet parameters, so I wrote a few lines to drop into my htaccess:
RewriteCond %{QUERY_STRING} (^|&)photo_id= [NC]
RewriteCond %{HTTP_REFERER} ^$
RewriteRule ^search$ - [F,L]
I implemented that last Friday and could see the bot being served 403 instead of 200, by Sunday it had obviously got bored as my access logs slowed down significantly.
Thank you all again.
Thanks! I’ve already explored number 2 but unfortunately it’d didn’t do everything I required. That’s why I thought I’d explore creating a custom RDF mapping but can’t work out how to do it.
Is option 1 as simple as creating a yml file with the required Dublin Core mappings and dropping into the config file within the RDF module?
Thanks.
I've hit this problem as well, did you find a solution? Thanks.
My solution was to create a custom module and add:
$query->where[0]['conditions'][0]["value"][":node__field_creation_date_field_creation_date_value_month"] = date("m");