Allow AWS Signature Connector to work with Opensearch Serverless

Created on 10 October 2024, 3 months ago

Problem/Motivation

Currently looking to add search/indexing to our drupal site. Deciding to experiment with AWS OpenSearch Serverless I came across this plugin however, the plugin is unable to connect to said serverless collection. Having tried both creating an iam user and using permanent creds from said iam user. Or grabbing session creds from the task role and neither result in a successful connection. We then created a standard aws opensearch domain and connection was successful immediately both through basic HTTP auth with a master user and with using the same permanent creds using the IAM user as master user. The motivation is to use serverless opensearch instead so we dont have fixed compute we wont need for a long while, while search gets established.

Steps to reproduce

Create an opensearch serverless cluster in a private vpc with an opensearch serverless vpc endpoint, create network, data-access and encryption policies allowing for ecs task role/iam user access for data. Have an ecs cluster/service/task running in the same vpc with the ecs task role being granted aoss:APIAccessAll and aoss:DashboardAccessAll. Allow connection access to the vpce via a security group with https allow inbound rule for said ecs service. Use session creds from curl http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI in the ecs container in the aws sig connector, should result in a could not connect to opensearch in plugin. otherwise create an iam user with the same policy for aoss and use permanent creds/keys in connector setup.

Proposed resolution

Allow aws sig connector or base connector to differentiate between aws opensearch and opensearch serverless.

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request

Status

Active

Version

2.1

Component

Code

Created by

dlund

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @dlund
Comment 3 months ago →
dlund
I think I can implement this. I think the aws signature connector is missing the SigV4 service definition shown here. here in the opensearch client builder on this line here
Comment 3 months ago →
dlund
Comment 3 months ago →
dlund
Having done some more research. I have managed to connect to the serverless collection with the base php opensearch client as it is supported. I have tried adding the extra method to the client instantiation in this module but still the serverless cluster is not available or able to connect according to the generic error message (this should also be showing the actual exception not defaulting to an unhelpful generic message). I can only assume that in order to check for cluster health/availability that this plugin is using an API call not supported in aws opensearch serverless therefore will look to be unavailable. https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ser...

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024