Incompatibility with Solr 9.0? Solr 9.3 works.

Created on 14 September 2023, 10 months ago
Updated 16 May 2024, about 1 month ago

Setup
Solr version: 9
Drupal Core version: 9.5.10
Search API version: 8x-1.29
Search API Solr version: 4.3.0-alpha1
Configured Solr Connector: Basic Auth

Issue
Hello! I am u

nable to create a core in the solr server SSH terminal. The result shows HTML with a "ERROR 401 require authentication" message.

Background/Setup

  • I am using the README for instructions
  • Modules are installed in Drupal
  • SOLR Server Search API is created in Drupal and is reachable using Basic Auth
  • Attempting to create SOLR single core - the classic way
  • I downloaded the config.zip with the "future" core name in the Drupal server settings
  • I am able to access the server via SSH, using the root user
  • I uploaded the config.zip to /tmp
  • I also created a folder in /var/solr/data with the name of the "future" core
  • In this folder, I created a /conf folder, uploaded the config.zip into it and unzipped

The issue I'm having is running the create_core command. In my case, here's what I'm trying:
sudo -u solr /opt/solr/bin/solr create_core -c future_core -d /tmp/solr9config -n future_core

This fails and I get a message showing HTML.

ERROR: Failed to get system information from http://localhost:8983/solr due to: org.apache.solr.common.SolrException: Parse error : <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 401 require authentication</title>
</head>
<body><h2>HTTP ERROR 401 require authentication</h2>
<table>
<tr><th>URI:</th><td>/solr/admin/info/system</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>require authentication</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>

</body>
</html>

Incidentally, it appears I can't run a basic solr command:

/opt/solr/bin/solr status

I get the same error message.

Other Notes

  • I'm not sure WHO I'm supposed to be logged in with. I tried logging in with the same credentials used in Drupal's basic auth settings. This failed. Root is the only user I can log in with currently to the server.
  • I'm not sure WHERE I'm supposed to put the config files. They are in /tmp, per the README instructions, but drupalize.me says to create a folder with a /conf folder, then upload config.zip into this.
  • If I try a create_core command without setting the configset in the command, I get a message: WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.

Please let me know if you need any more information. I don't really know where else to go for support and I really appreciate your time and help.

πŸ“Œ Task
Status

Fixed

Version

4.3

Component

Solr Connectors

Created by

πŸ‡ΊπŸ‡ΈUnited States rraney

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @rraney
  • πŸ‡ΊπŸ‡ΈUnited States rraney
  • πŸ‡ΊπŸ‡ΈUnited States rraney
  • πŸ‡ΊπŸ‡ΈUnited States rraney

    Update
    Someone in Slack recommended attempting to access Solr server via browser. I was able to login using the "basic auth" credentials which are used to connect to the server in Search API. I tried adding a core. In the var/solr/data folder, I created the core directory. Inside this, I created a /conf with the config.zip contents. I also created a /data folder.

    It attempted create a core but gave an error. It said it failed to initialize the core.

    limestone_solr: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: JVM Error creating core [limestone_solr]: class org.apache.solr.schema.ICUCollationField tried to access protected field org.apache.solr.schema.IndexSchema.luceneVersion (org.apache.solr.schema.ICUCollationField is in unnamed module of loader java.net.FactoryURLClassLoader @6ac505d5; org.apache.solr.schema.IndexSchema is in unnamed module of loader org.eclipse.jetty.webapp.WebAppClassLoader @71c5b236)
    Please check your logs for more information

    I submitted this error to my hosting company to see if they know. I found something in online search suggesting to change the solrcore.properties file to include this solr.install.dir=/opt/solr. I did this before attempting to create another core but I'm getting the same error above.

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    Our hosting company was having the same issue with creating a core. They started testing the removal of lines from "schema_extra_fields.xml". I don't know how they came to this conclusion. They said it was outside the scope of their support to go through and remove problematic fields.

    I keep seeing "collation". Is there some sort of collation that might be misconfigured? Another thing I'm wondering is whether there are some missing dependencies? Another thing I'm wondering is whether there are issues now that we have multiple partial cores with the same config. If anyone knows how to delete ALL cores so we can start fresh, I'd appreciate that pointer.

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    Here is a more detailed message shown when I try to create core from command line:

    <h3>Caused by:</h3><pre>java.lang.IllegalAccessError: class org.apache.solr.schema.ICUCollationField tried to access protected field org.apache.solr.schema.IndexSchema.luceneVersion (org.apache.solr.schema.ICUCollationField is in unnamed module of loader java.net.FactoryURLClassLoader @4b2fa09d; org.apache.solr.schema.IndexSchema is in unnamed module of loader org.eclipse.jetty.webapp.WebAppClassLoader @2f7a7219)
            at org.apache.solr.schema.ICUCollationField.init(ICUCollationField.java:128)
            at org.apache.solr.schema.FieldType.setArgs(FieldType.java:202)
            at org.apache.solr.schema.FieldTypePluginLoader.init(FieldTypePluginLoader.java:149)
            at org.apache.solr.schema.FieldTypePluginLoader.init(FieldTypePluginLoader.java:43)
            at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:189)
            at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:531)
            at org.apache.solr.schema.IndexSchema.&lt;init&gt;(IndexSchema.java:188)
            at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:104)
            at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:342)
            at org.apache.solr.core.ConfigSetService.lambda$loadConfigSet$0(ConfigSetService.java:253)
            at org.apache.solr.core.ConfigSet.&lt;init&gt;(ConfigSet.java:49)
            at org.apache.solr.core.ConfigSetService.loadConfigSet(ConfigSetService.java:249)
            at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1550)
            at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1426)
            at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
            at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:389)
            at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:411)
            at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)
            at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:207)
            at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:941)
            at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:893)
            at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:584)
            at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:239)
            at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:207)
            at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257)
            at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227)
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
            at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
            at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
            at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
            at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
            at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
            at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
            at org.eclipse.jetty.server.Server.handle(Server.java:516)
            at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
            at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
            at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
            at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
            at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
            at java.base/java.lang.Thread.run(Thread.java:833)
    
  • πŸ‡ΊπŸ‡ΈUnited States rraney

    I have one functioning core on the server. I configured the Drupal solr search api to use this core. I downloaded the config.zip, uploaded to /var/solr/data/core_name/conf, then unzipped it. The message I'm getting in Drupal is "You are using an incompatible Solr schema. Please follow the instructions described in the README.md file for setting up Solr.".

    When I try to start all over with a new core, I cannot create a core with the /conf already placed in folders, nor by directing the create_core using the command in README. If I run this: sudo -u solr ./bin/solr create_core -c lime_solr -d ../../tmp/solr_9dotx_config I get this error: ERROR: "Error CREATEing SolrCore 'lime_solr': Couldn't persist core properties to /var/solr/data/lime_solr/core.properties : /var/solr/data/lime_solr/core.properties"

    FYI, this is set up in Drupal config and solrcore.properties: solr.install.dir=/opt/solr

  • πŸ‡©πŸ‡ͺGermany mkalkbrenner πŸ‡©πŸ‡ͺ

    This is not the Solr support channel ;-)
    I recommend to read the Solr docs.

    Anyway, ...

    I keep seeing "collation". Is there some sort of collation that might be misconfigured?

    Collations use the ICU Solr plugin/module which is part of the Solr installation. But I assume that the include path is wrong.

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    The problem relates to this module's requirement that Solr core be configured with a specific config set. Therefore, I think it's totally appropriate to seek help here. Other Drupal users or contributors may have run into similar issues and it might technically be a module issue. In fact, I'm told by my hosting company that it might have to do with settings in the module's schema_extra_fields.xml file.

    For example, this error:

    ERROR: Error CREATEing SolrCore 'core2': Unable to create core [core2] Caused by: Unknown fieldType 'collated_und' specified on field sort_*

    org.apache.solr.client.solrj.SolrServerException: Error CREATEing SolrCore 'core2': Unable to create core [core2] Caused by: Unknown fieldType 'collated_und' specified on field sort_*

    This points to a single conflict in the schema_extra_fields.xml file having to do with "collated_und". I'm not going to pretend to understand everything that's going on, that's why I'm seeking help. However, it seems entirely possible that it's not just a "solr" issue. It seems like it could be a module/solr conflict issue.

  • πŸ‡©πŸ‡ͺGermany mkalkbrenner πŸ‡©πŸ‡ͺ

    There's no error in the module. As already mentioned, the setup of your hoster isn't correct. Solr can't find the required modules to use the ICU stuff:
    https://git.drupalcode.org/project/search_api_solr/-/blob/4.x/jump-start...
    https://git.drupalcode.org/project/search_api_solr/-/blob/4.x/jump-start...

    Note that the path for these extensions changed between Solr 8 and 9.
    As you can see, the paths are configurable within the configs. You need to talk with your hoster about it.

    The required modules/plugins are part of Solr and should be somewhere in the filesystem:
    https://github.com/apache/solr/tree/main/solr/modules/analysis-extras

    Other hosters like Acquia, pantheon, SearchStax, hosted Solr, ... don't have that issue.
    You could also use the jump-start configs this module provides or even use docker-compose stuff to test it locally (or DDEV).

    Don't get me wrong. I want to help. But I can't solve your hoster's issue for free in my free time here in the issue queue.

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    I asked them to update to 9.3.x because they had 9.0.0 installed. The only hint I've been able to find on Solr forums and Google is something that may have been fixed in 9.1. I'm curious if there is legitimately an issue with your module and 9.0.0. I will try to learn about Solr not finding dependencies.

    I understand the nature of Drupal with regard to people offering their time, being open source. Thanks for your time.

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    I had the hosting company update to Solr 9.3.0. After a few issues, I got a clean core creation. I truly believe using Solr 9.0.0 was incompatible with this module, but I really don't know how to test this. I don't have a good local environment.

    It's too early to know whether I actually will get it working, but this is a positive step and I consider this specific issue resolved.

  • πŸ‡©πŸ‡ͺGermany mkalkbrenner πŸ‡©πŸ‡ͺ

    Good to here!

    It would be worth checking, if we have to declare Solr 9.0 as incompatible. But we also need to check 9.1 and 9.2.

  • πŸ‡§πŸ‡ͺBelgium Yanivs

    I also had the "ERROR 401 require authentication" when trying to create a core via terminal. D10, Search API Solr 4.3.0 and Solr 9.4.0.

    Followed the documentation and got stuck with the 401 when trying to create the core.

    To overcome this, I turned off the Solr basic authentication by editing security.json and changing:

    "blockUnknown": true

    to

    "blockUnknown": false

    Solr had to be restarted.

    Then, solr create -c my_solr_core worked.

    After that I enabled basic authentication again.

    This is not a solution, just a workaround. It should be possible to run commands on Solr with basic authentication enabled, but I could not find how.

  • πŸ‡©πŸ‡ͺGermany mkalkbrenner πŸ‡©πŸ‡ͺ

    It should be possible to run commands on Solr with basic authentication enabled, but I could not find how.

    @Yanvis, this is documented:
    https://solr.apache.org/guide/8_11/basic-authentication-plugin.html#usin...

    You have to set environment variables.

  • Status changed to Fixed about 2 months ago
  • πŸ‡©πŸ‡ͺGermany mkalkbrenner πŸ‡©πŸ‡ͺ
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.69.0 2024