Use bulk endpoint in Elastic search destination plugin

This uses a bulk of 100 items at a time

The last submitted patch, 2: 3361760.patch, failed testing. View results →

Fixes tests

@larowlan. Looks good. I think we should make the batch/bulk size configurable as part of the plugin.

Re #5: I think we could do that maybe as a config of data_pipelines_elasticsearch instead but that can happen in a follow-up.

@jibran I'd rather it done now - It's easy to add. It just needs to be added to the plugin configuration:

https://git.drupalcode.org/project/data_pipelines/-/blob/1.x/modules/dat...

https://git.drupalcode.org/project/data_pipelines/-/blob/1.x/modules/dat...

Sure, let's do that.

Made this configurable

Added missing config schema as pointed out by Jibran

Thanks for adding that in @larowlan!

+++ b/modules/data_pipelines_elasticsearch/config/schema/data_pipelines_elasticsearch.schema.yml
@@ -14,3 +14,6 @@ data_pipelines.dataset_destination.destinationSettings.elasticsearch:
+    chunk_size:

FWIW, we also needed an post update hook to update the existing instances of ES destination instances. We only have one so we can do that manually.

The call to defaultSettings in the constructor resolves that

Automatically closed - issue fixed for 2 weeks with no activity.