Supporting Solr 9 in 7.x version of Search API Solr module

Created on 8 June 2023, over 1 year ago
Updated 14 September 2024, 2 months ago

Setup

  • Solr version: 9.2.1
  • Drupal Core version: 7.96
  • Search API version: 7.x-1.28
  • Search API Solr version: 7.x-1.15
  • Configured Solr Connector: Standard (I think)

Issue

We have been using the 7.x version with Solr 8 for a while now and the Solr 7 XML config worked fine for Solr 8 as far as we can tell. For Solr 9, I was able to copy the Solr 7 XML config and modify it to work with Solr 9 and that seem to work so far locally. I will share a patch and a diff of the configs so folks can try it out.

Feature request
Status

Needs work

Version

1.0

Component

Code

Created by

🇺🇸United States dmundra Eugene, OR

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @dmundra
  • 🇺🇸United States dmundra Eugene, OR

    Here is a diff between solr 7.x XML config and solr 9.x XML config:

    diff solr-conf/7.x solr-conf/9.x
    diff solr-conf/7.x/schema.xml solr-conf/9.x/schema.xml
    13c13
    < <schema name="drupal-4.4-solr-7.x" version="1.5">
    ---
    > <schema name="drupal-4.4-solr-9.x" version="1.5">
    400c400
    <     <fieldType name="location" class="solr.LatLonType" subFieldType="tdouble"/>
    ---
    >     <fieldType name="location" class="solr.LatLonPointSpatialField" docValues="true"/>
    402,406d401
    <     <!-- A Geohash is a compact representation of a latitude longitude pair in a single field.
    <          See http://wiki.apache.org/solr/SpatialSearch
    <      -->
    <     <fieldtype name="geohash" class="solr.GeoHashField"/>
    < 
    623,624c618,620
    <     <dynamicField name="geos_*" type="geohash" indexed="true"  stored="true" multiValued="false"/>
    <     <dynamicField name="geom_*" type="geohash" indexed="true"  stored="true" multiValued="true"/>
    ---
    >     <!-- GeoHash fields are deprecated. LatLonPointSpatial fields solve all needs. But we keep the dedicated field names for backward compatibility. -->
    >     <dynamicField name="geos_*" type="location" indexed="true"  stored="true" multiValued="false"/>
    >     <dynamicField name="geom_*" type="location" indexed="true"  stored="true" multiValued="true"/>
    diff solr-conf/7.x/solrconfig.xml solr-conf/9.x/solrconfig.xml
    23c23
    < <config name="drupal-4.4-solr-7.x" >
    ---
    > <config name="drupal-4.4-solr-9.x" >
    50c50
    <   <luceneMatchVersion>${solr.luceneMatchVersion:LUCENE_60}</luceneMatchVersion>
    ---
    >   <luceneMatchVersion>${solr.luceneMatchVersion:LUCENE_80}</luceneMatchVersion>
    75a76
    >   <!--
    77a79,84
    >   -->
    >   <lib dir="/opt/solr/contrib/extraction/lib" regex=".*\.jar" />
    >   <lib dir="/opt/solr/contrib/langid/lib/" regex=".*\.jar" />
    >   <lib dir="/opt/solr/contrib/scripting/lib/" regex=".*\.jar" />
    >   <lib dir="/opt/solr/contrib/clustering/lib/" regex=".*\.jar" />
    >   <lib dir="/opt/solr/contrib/analysis-extras/lib" regex=".*\.jar" />
    499,500c506
    <     <filterCache class="solr.FastLRUCache"
    <                  size="512"
    ---
    >     <filterCache size="512"
    509,510c515
    <     <queryResultCache class="solr.LRUCache"
    <                      size="512"
    ---
    >     <queryResultCache size="512"
    520,521c525
    <     <documentCache class="solr.LRUCache"
    <                    size="512"
    ---
    >     <documentCache size="512"
    1725c1729
    <   <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
    ---
    >   <queryResponseWriter name="xslt" class="solr.scripting.xslt.XSLTResponseWriter">
    diff solr-conf/7.x/solrcore.properties solr-conf/9.x/solrcore.properties
    10c10
    < solr.luceneMatchVersion=6.0
    ---
    > solr.luceneMatchVersion=8.0
    

    Attached is the patch to add the Solr 9 XML config.

  • 🇺🇸United States dmundra Eugene, OR
  • 🇺🇸United States dmundra Eugene, OR

    Note, moving to Solr 9 definitely need a re-index. I set the luceneMatchVersion to LUCENE_80 to make it maybe easier to upgrade to.

  • Status changed to Needs review over 1 year ago
  • 🇦🇹Austria drunken monkey Vienna, Austria

    Thanks a lot for sharing this!
    If anyone else has had a chance to test this, please let me know whether it worked for you.

  • Status changed to Needs work over 1 year ago
  • 🇦🇹Austria drunken monkey Vienna, Austria

    At a quick test, it mostly seemed to work for me after a few modifications: You hard-coded the absolute path of the contrib directory, so this didn’t work for me. Also, the installation I downloaded had that directory renamed to modules, compared to contrib used in Solr 8 and before. Is this directory named contrib still in your Solr 9 installation?

    The only problem still remaining in the version uploaded here is that highlighting doesn’t work: For some reason, even though the unified highligher is used, Solr returns an error with this message: field 'tm_body$value' was indexed without offsets, cannot highlight. According to the documentation, however, the unified highlighter shouldn’t require offsets in the indexed data, so I don’t quite understand this. It works fine with Solr 8 and earlier, with the exact same request.
    Adding hl.method=original solves the problem, but I’d prefer to find the source of this discrepancy first before settling on a fix/workaround for it.

  • 🇺🇸United States dmundra Eugene, OR

    Thanks @drunken-monkey for updating the patch. I was using the Solr 9 docker image that happened to symlink the contrib folder hence why I kept it but forget about when I was sharing the code.

    Interesting, I hadn't tested that. I can give it a try.

  • 🇫🇷France luukyb

    I can confirm that the data is being indexed using @dmundra's patch for me. Didn't work with the patch from @drunken monkey, probably due to my local SOLR instance.
    @drunken monkey could you give more details on using hl.method=original ? I'm not sure I used it properly because it didn't work on my end. Thanks!

  • 🇦🇹Austria drunken monkey Vienna, Austria

    @luukyb: In SearchApiSolrService::getHighlightParams(), simply add $highlight_params['hl.method'] = 'original'; inside the if.
    If this still doesn’t work for you, does it produce the exact same error as without it (field 'tm_body$value' was indexed without offsets, cannot highlight)?

Production build 0.71.5 2024