Allow AWS Signature Connector to work with Opensearch Serverless

I think I can implement this. I think the aws signature connector is missing the SigV4 service definition shown here. here in the opensearch client builder on this line here

Having done some more research. I have managed to connect to the serverless collection with the base php opensearch client as it is supported. I have tried adding the extra method to the client instantiation in this module but still the serverless cluster is not available or able to connect according to the generic error message (this should also be showing the actual exception not defaulting to an unhelpful generic message). I can only assume that in order to check for cluster health/availability that this plugin is using an API call not supported in aws opensearch serverless therefore will look to be unavailable. https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ser...

+1. I can confirm there are two issues:
1. Reading credentials from AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable and get the key, secret, token & expire.
2. Service setup.

For the first issue, if we don't pass the details via the form or settings.php, it would use CredentialProvider::defaultProvider() and reads & fills the details. However for the service setup, there is no option at the moment. Here is the simple working script on ECS:

<?php

use Aws\Credentials\CredentialProvider;
use Drupal\Core\Utility\Error;
use OpenSearch\ClientBuilder;

$config = [
  'url' => 'https://[server-id].[region].aoss.amazonaws.com',
  'ssl_verification' => TRUE,
  'region' => '[region]',
  'service' => 'aoss',
];
$provider = CredentialProvider::defaultProvider();

$client = ClientBuilder::create()
  ->setHosts([$config['url']])
  ->includePortInHostHeader(TRUE)
  // To see the connection issues. can be removed once done with debugging the issues.
  ->setConnectionParams([
    'client' => [
      'curl' => [
        \CURLOPT_VERBOSE => 1,
      ]
    ]
  ])
  ->setSSLVerification($config['ssl_verification'])
  ->setSigV4Region($config['region'])
  ->setSigV4Service($config['service'])
  ->setSigV4CredentialProvider($provider)
  ->build();

try {
 $response = $client->indices()->create(['index' => 'projects-index-1']);
  print_r($response);
  $data = [
    'title'=> 'Drupal',
    'namespace' => 'drupal/drupal',
    'version' => '10.4.3'
  ];

  $response = $client->create([
    'index' => 'projects-index-1',
    'body' => json_encode($data),
    'id' => 't' . time(),
  ]);
  print_r($response);

}
catch (\Exception $e) {
  $error = Error::decodeException($e);
  print_r($error['@message']);
  print_r($error['@backtrace_string']);
}

Since we updated the opensearch-php library to 2.4.x you can use the \Drupal\search_api_aws_signature_connector\AwsSigningHttpClientFactory and pass your own Aws\Credentials\CredentialProvider in the service definition.

We might be able to make it an option in the settings form.

Thanks @kim.pepper. For our setup, we are just missing service, so doesn't make sense to have custom credential provider. I have attached the patch to introduce the service field in settings form.

+++ b/modules/search_api_aws_signature_connector/src/Plugin/OpenSearch/Connector/AwsSignatureConnector.php
@@ -91,6 +93,17 @@ class AwsSignatureConnector extends StandardConnector {
+    ];

Should we default this to 'es'?

Created a MR from the patch at #7

kim.pepper → changed the visibility of the branch 3479880-allow-aws-signature to hidden.

I think we need to handle existing sites a bit better.

• Should we have a config migration path?
• The wording on the form description isn't that helpful to those who don't know what the possible values are.
• Should we have an options list with just OpenSearch and OpenSearch Serverless?
• Can we just default to OpenSearch for existing users.

1. Should we have a config migration path?

you mean add an update hook?

2. The wording on the form description isn't that helpful to those who don't know what the possible values are.

Now this is a dropdown, so helps what services are available.

3. Should we have an options list with just OpenSearch and OpenSearch Serverless?

yeah, makes sense. updated.

4. Can we just default to OpenSearch for existing users.

Ideally yes as the library sets default to es.

P.S: I am pushing changes to both MR and patch as I need to apply on my project in latest 2.x.

1. Should we have a config migration path?

you mean add an update hook?

Yep.

Added update hook.

NW for the code style fails

rerolled against latest 2.x

rerolled and fixed PHPCS issues on the MR at 3.x as well.