Add more details on how to set up Solr Cloud on a server in README

Created on 8 June 2023, about 1 year ago
Updated 2 May 2024, about 2 months ago

Moving a locally developed "Solr (single core) - the classic way" configuration to a server, listing prerequisites such as Java, creating a core, moving the Solr configset in place, and so on is very well documented in the README file under Setting up Solr (single core) - the classic way.

Problem

It is only briefly documented how to set up Solr Cloud on a server, and importing the configset. It would be great if more detail were added to the steps required to set up Solr Cloud on a server in the README, to help move a Solr Cloud configuration from local development with ddev-solr to production.

Proposed solution

Expand documentation for setting up Solr Cloud on a server and importing the configuration in the README file, the same way Setting up Solr (single core) is thoroughly documented.

Remaining tasks

🔲 Expand Solr Cloud server set up documentation in the README file, under these sections:

💬 Support request
Status

Needs review

Version

4.0

Component

Documentation

Created by

🇩🇰Denmark ressa Copenhagen

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @ressa
  • 🇩🇰Denmark ressa Copenhagen
  • 🇩🇰Denmark ressa Copenhagen

    I realized that https://github.com/docker-solr/docker-solr-examples/blob/master/docker-c... is probably the best solution to use as a starting point, so I updated the Issue Summary.

  • 🇩🇰Denmark ressa Copenhagen

    Updating the Issue Summary to make it less vague, also adding link to https://github.com/mkalkbrenner/ddev-solr which has been created in the meantime.

  • 🇩🇰Denmark ressa Copenhagen

    I now see that there are Setting up Solr Cloud - the classic way and Using Jump-Start config-sets and docker images, which is great.

    But they are much more general, and not nearly as detailed as Setting up Solr (single core) - the classic way.

  • 🇩🇰Denmark ressa Copenhagen

    Updating title.

  • 🇬🇷Greece pinkonomy

    I have the same issue,I cannot make Solr Could work.

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    But they are much more general, and not nearly as detailed as Setting up Solr (single core) - the classic way.

    Setting up Solr Cloud locally is much more complex if you don't use tools like DDEV. For example you also need Zookeeper. And there're different ways to achieve that. Th README already contains the correct help:
    To do so you have to read the Solr handbook.

    Drupal doesn't document how to install MariaDB or Postgress. That's comparable.

  • 🇩🇰Denmark ressa Copenhagen

    I agree, setting up Solr Cloud for local development in DDEV easy.

    But this issue is about documenting how to set Solr up on a server, not for local development ... Getting Solr to run locally with DDEV is not a big deal, getting it on the server is hard.

    I think the word "local" may mean different things to us ... For me it means "local development". This is why I take care to use either "local development" or "server".

    Drupal doesn't document how to install MariaDB or Postgress. That's comparable.

    I don't think that's true. True, you may have to run apt install mysql and create a MySQL/MariaDB database and define users and set the password, but that's all very well documented:

    Again, I don't want to repeat myself, but the steps for setting up Solr on a server is very well documented, in the README section "Local Solr Instance: Solr (single core) - the classic way".

    The same detailed steps for setting up Solr using the new Cloud method on a server (not for local development) is a requirement to get the users to switch to using that method, as I see it.

    It not, the users will just keep using the Classic method.

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    There's a simple way to start Solr in cloud mode using an embedded zookeeper:
    https://solr.apache.org/guide/solr/latest/getting-started/tutorial-solrc...

    But this isn't recommended for production use! Just for local development.

  • 🇩🇰Denmark ressa Copenhagen

    That's great, that could be a start! We should link to that page from the README.

    Hopefully some day, someone will document how to do a basic set up of Solr Cloud with Drupal on a server. But just briefly: Which steps do you think should also be done for production?

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    We use docker or kubernetes in production. There're docker compose files for docker.
    If you host on kubernetes, you should use Solr Operator.

  • 🇩🇰Denmark ressa Copenhagen

    Thanks. Using Kubernetes is probably overkill for many use cases, but Docker might work.

    Let's hope it gets documented at some point, so that people can start using the new Cloud method.

  • Status changed to Needs review 5 months ago
  • 🇨🇳China skyredwang Shanghai

    I'd like to suggest to include a Docker instruction for the morden SolrCloud setup.

    Step 1, start a Zookeeper container:

    docker run -d --name zookeeper -p 2181:2181 zookeeper

    Step 2, save a copy of Solr security configuration in your project, e.g. DOCROOT/config/dev.security.json, whose content is below:

    {
      "authentication": {
        "class": "solr.BasicAuthPlugin",
        "credentials": {
          "solr": "IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="
        }
      },
      "authorization": {
        "class": "solr.RuleBasedAuthorizationPlugin",
        "permissions": [
          {
            "name": "security-edit",
            "role": "admin"
          }
        ],
        "user-role": {
          "solr": "admin"
        }
      }
    }
    

    Step 3, start a Solr container running in a cloud mode with basic authentication.

    docker run --name solr9 -p 8983:8983 -e ZK_HOST=zookeeper --link zookeeper:zookeeper -v "$PWD/config/solrcloud.security.json:/opt/solr/server/solr/security.json" -d solr:9 bash -c "docker-entrypoint.sh solr zk cp file:/opt/solr/server/solr/security.json zk:/security.json && exec solr-foreground"
    

    Step 4, Upload the config and create a collection

    On the Drupal side, visit /admin/config/search/search-api/server/[SERVER_NAME]/solr-admin/upload-configset, then click "Upload and Create Collection"

  • 🇩🇰Denmark ressa Copenhagen

    This looks interesting, thanks for sharing!

    I am testing this in DDEV (without the https://github.com/ddev/ddev-solr add-on) and it looks like the URL is http://0.0.0.0:8983/solr/#/ -- so I should enter 0.0.0.0 as "Solr node" under /admin/config/search/search-api/server/solr/edit?

    The username and password is solr and SolrRocks.

    I can access Solr at http://0.0.0.0:8983/solr/#/ via the browser, but when I click "Upload Configset" to submit the Search API Solr configuration, I get this message:

    Configset upload failed with error code 7: Solr HTTP error: HTTP request failed, Failed to connect to 0.0.0.0 port 8983: Connection refused (7)

    The set up:

    Status 	enabled (disable)
    Backend class 	Solr
    Solr connector plugin 	Solr Cloud with Basic Auth
    Solr server URI 	http://0.0.0.0:8983/
    Solr collection URI 	http://0.0.0.0:8983/solr/#/techproducts
    Basic HTTP authentication 	Username: solr | Password: *********
    Server Connection 	The Solr server could not be reached or is protected by your service provider.
    Collection Connection 	The Solr collection could not be accessed. Further data is therefore unavailable.
    Minimal required schema version 	4.3.0
    Preferred schema version 	4.3.2
    Configured Solr Version 	9.0.0
    Targeted content domain 	generic
    Targeted environment 	default
    
  • 🇨🇳China lawxen

    #16 help me realize why the example docker-compose.yml didn't work with security.json. Because it use solr start -c witch will start solr with an embedded ZooKeeper instance instance which started on Solr port+1000.

    This is my config:
    security.json:

    {
        "authentication": {
            "class": "solr.BasicAuthPlugin",
            "credentials": {
                "solr": "IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="
            },
            "": {
                "v": 0
            }
        },
        "authorization": {
            "class": "solr.RuleBasedAuthorizationPlugin",
            "permissions": [
                {
                    "name": "all",
                    "role": "admin"
                }
            ],
            "user-role": {
                "solr": "admin"
            }
        }
    }
    

    compose.yaml:

    version: '3'
    services:
      solr7:
        image: solr:7
        ports:
          - "8983:8983"
        volumes:
          - ./cloud-config-set:/opt/solr/server/solr/configsets/drupal/conf
        command: bash -c "docker-entrypoint.sh solr start -c; until solr status; do sleep 1; done; solr zk cp file:/opt/solr/server/solr/security.json zk:/security.json -z localhost:9983; solr create -c drupal -d server/solr/configsets/drupal/conf -n drupal; solr restart -c -f"
    
  • 🇳🇱Netherlands ekes

    I'm going to put this question here, as documentation, I couldn't find it anywhere else.

    What is the minimum set of permissions for a solr role that will work out-of-the-box with the module? For example including the upload config. I see by default:

    security-edit 
    security-read
    config-edit
    config-read
    collection-admin-edit
    core-admin-edit
    core-admin-read
    read
    collection-admin-read
    update
    health
    metrics-read
    filestore-read
    package-read
    package-edit
    schema-edit
    schema-read
    zk-read
    all
    

    I assume several of these that are needed could also be constrained to a collection.

  • 🇩🇪Germany mkalkbrenner 🇩🇪

    I just refactored the ddev integration. The permissions to be set in security.json changed in Solr 9:
    https://github.com/ddev/ddev-solr/pull/32

Production build 0.69.0 2024