Adding a best bet term to a node instantly breaks that entire search

Created on 4 January 2023, almost 2 years ago
Updated 2 November 2023, about 1 year ago

Problem/Motivation

I have a Search API & Solr Search server working on my site. We wanted best bets functionality with our new search functions so we added this module.

When we add a best bet to any node, it instantly breaks that search returning "no results"

We have a bunch of modules on our site but only a few related to search / search api:

"drupal/search_api": "^1.27",
"drupal/search_api_best_bets": "^2.0",
"drupal/search_api_exclude_entity": "^1.3",
"drupal/search_api_solr": "^4.2",
"drupal/search_api_spellcheck": "^4.0",

Steps to reproduce

1. Create a solr search server & indexed on the site
2. Add and configure Search API Best Bets module
3. Perform a search that returns results with a solr site search query
4. Open up any node result below the first return result that you want to elevate with best bet (also occurs with first result tho)
5. Add the exact term or phrase from step 3 to the best bet to elevate that node
6. Save the node
7. Return to your search page and either refresh or re-search the same term
8. Observe that the result is now completely broken and returns "no results" no matter how many results were populated before
9. Remove the term from the node in step 4-6. Save
10. Attempt to search the same term from step 3 again, note the search is still broken

Proposed resolution

Best bets elevates queries and does not break them

Remaining tasks

User interface changes

API changes

Data model changes

💬 Support request
Status

Fixed

Version

2.0

Component

Code

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇺🇸United States smustgrave

    Tested this again today locally and still not hitting an error.

    Wonder if you were able to figure out the issue?

  • 🇨🇦Canada Brownell Toronto, Ontario

    I'm actually having the same issue. When I search for a term that has been added in the Elevate Query field, the search results for that term are always empty. When the search term is removed from the Elevate Query field, the results return fine.

    Some additional information:
    - Drupal v9.5.9
    - Solr v8.11.2
    - PHP v8.1.14
    - MariaDB v10.4.22
    - Apache v2.4.53

    I've added the Search API Best Bets field to each of my content types (one field used in multiple content types, not seperate fields). I've configured the field so that it's allowed an unlimited number of values, or just one; no difference.

    I've configured the Solr index to have the Search API Best Bets setting enabled, and in the setting tab at the bottom, I've selected the field (the one field used in multiple content types) to be used. I've selected the Solr query handler. I've tried both of the Elevated Flag options; no difference. On the Preprocess Query and Postprocess Query Processor Order lists I've tried having Search API Best Bets at the top and the bottom; no difference.

    I also have the Double Quote Workaround, HTML filter, and Ignore characters prosessors enabled. Though I've disabled them and there's no difference; the issue presists.

    After trying each of these options I've rebuilt the index. Nothing seams to make any difference. Is there anything else you can recommend that I try?

  • We have not been able to figure out this issue on our end and glad (sort of) to see that someone else is experiencing the same problem

  • @smutgrave Yeah just to be sure, i just reinstalled the entire module, configured the settings as directed in the configuration section

    Enabled the module, added the field to nodes (tried unlimited and limited in the field settings), added the field to the solr fields (tried full text and fulltext string), enabled the processor (selected the field, solr as query handler, tried both elevated flag settings). Reindexed (also after each setting change). Added the search term to a node, searched for that term, query instantly breaks with "no results".

    Essentially exactly the same as it was when i originally reported the issue, no change in the module or the outcome of attempting to use it.

    If theres other data needed to debug it let me know, the only other thing i can provide is the error listed in the log messages:

    Message { "error":{ "trace":"java.lang.NullPointerException\n\tat org.apache.solr.response.transform.BaseEditorialTransformer.getKey(BaseEditorialTransformer.java:79)\n\tat org.apache.solr.response.transform.BaseEditorialTransformer.transform(BaseEditorialTransformer.java:54)\n\tat org.apache.solr.response.transform.DocTransformer.transform(DocTransformer.java:85)\n\tat org.apache.solr.response.transform.DocTransformers.transform(DocTransformers.java:76)\n\tat org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:102)\n\tat org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:59)\n\tat org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:194)\n\tat org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:137)\n\tat org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:387)\n\tat org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:293)\n\tat org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)\n\tat org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)\n\tat org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)\n\tat org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:580)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)\n\tat org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\n\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\n\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n", "code":500}}

  • 🇫🇮Finland kekkis Pirkkala

    @sbrenner02, I'm basically running into exactly the same issue as you are. I wonder if @smustgrave could share his solrconfig*.xml files here for comparison. I'm not entirely sure the problem lies in those though, but I would like to see.

    For reference, my config files are attached and the versions I'm running are:

    Drupal core 10.1.4
    Search API 8.x-1.29
    Search API Solr 4.3.0
    Search API Best Bets 2.0.2
    Solr 8.11.2

    Installed search_api related modules:

      search_api: 0
      search_api_autocomplete: 0
      search_api_best_bets: 0
      search_api_solr: 0
      search_api_solr_autocomplete: 0
    

    I debugged the query so far that I got as far as a working and a non-working query. The difference between those is that the one that does not work has the elevateIds query parameter. An otherwise identical query does return results correctly, except of course with no elevation functionality. It does not matter what the value of the elevateIds parameter is: as long as the parameter is present, Solr returns HTTP 500 with the exact same stack as in @sbrenner02's comment. I'm adding that in as code for better readability:

    java.lang.NullPointerException
      at org.apache.solr.response.transform.BaseEditorialTransformer.getKey(BaseEditorialTransformer.java:79)
      at org.apache.solr.response.transform.BaseEditorialTransformer.transform(BaseEditorialTransformer.java:54)
      at org.apache.solr.response.transform.DocTransformer.transform(DocTransformer.java:85)
      at org.apache.solr.response.transform.DocTransformers.transform(DocTransformers.java:76)
      at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:102)
      at org.apache.solr.response.DocsStreamer.next(DocsStreamer.java:59)
      at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:194)
      at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:137)
      at org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:387)
      at org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:293)
      at org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
      at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
      at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
      at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:887)
      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:580)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
      at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
      at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
      at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
      at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
      at org.eclipse.jetty.server.Server.handle(Server.java:516)
      at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
      at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
      at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
      at java.base/java.lang.Thread.run(Unknown Source)
    
    
  • Status changed to Active about 1 year ago
  • 🇫🇮Finland kekkis Pirkkala

    Upon further exploration I discovered that the query was what was wrong!

    The original request, or its most relevant parts anyway, are here, split by ampersand:

    &forceElevation=true
    &enableElevation=true
    &fl=%5Belevated%5D
    &fl=ss_search_api_id%2Css_search_api_language%2Cscore%2Chash
    &elevateIds=eob6nl-training_search-entity:node/2019:fi
    

    Notice that neither of the generated fl parameters include the id field. When I added that in the query, Solr stopped complaining.

    To make this work in the module, I basically forced the addition of the id field whenever either an excludeIds or elevateIds was being added.

    And so as I was about to add a patch for this, I discovered commit f6896391 which already fixes the issue. So adding a related issue with this comment.

  • 🇺🇸United States smustgrave

    So are you saying the dev branches addresses this?

  • 🇫🇮Finland kekkis Pirkkala

    Yes, and that is why I encourage you to publish a new release with that fix in place.

  • Status changed to Fixed about 1 year ago
  • 🇺🇸United States smustgrave

    Just deployed 2.0.3

  • Confirming that the issue is fixed on my end! thanks!

  • 🇨🇦Canada Brownell Toronto, Ontario

    I can also confirm that I do not have this issue in version 2.0.3. Thank you!

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024