Support Drupal's Entity Reference field type concept on external data source.

This sounds overly complicated. What about simply having a config tab for the entity that allows you do choose which fields are entity references? By default, it could translate the remote ID value into the Drupal entity ID, and use that to make the reference?

For more complicated values in the reference field, it could allow for hook implementations that provide a translation in code.

I didn't read everything in details (not enough time to be honest) but I leave this post as a note for people who might want to do something for that issue.

There is something that has not been implemented yet in the external entity API for entity queries: in StorageClientBase::testDrupalConditions(), there is currently no (or limited) support for complex filtering on fields using syntax with sub-fields (like it is the case here with entity references) or with '%delta' (eg. ->condition('tags.%delta.value', 'news'))). That might be the reason why it does not behave like other Drupal entities with references.

The issue description is a bit confusing.

What I'd like to do is have an entity reference field that can reference external entities using the external entity ID.

This seems similar to annotations? I just tried that and was not able to reference an external entity by ID using autocomplete. There are a list of items that I loaded so far if I use the select element, but it throws an error that I can't use them.

Quick note: I have several sites using external entities with references to other external entities (same xntt type as well as different xntt type). So it should work unless there is a regression somewhere I missed, that does not affect my sites.

I'll experiment some more next time I get a chance, but this was with an entity reference on a node, referencing an external entity.

I have not tried it between two external entities.

I had a look and I have a good and a "bad" news. The good news is that it IS possible to reference external entities using a regular entity reference field, without any modification or trick. It already works. That was the good news... :-)

The "bad" news is that the storage client(s) used to fetch external entities must support the 'IN' operator in query filtering. If it does not, there's no way for the default Drupal entity reference validator to fetch all the referenced external entities and validate their IDs. That's why @mortona2k got issue referencing external entities.

When you use REST services, you don't have a "standard" query that returns all the valid IDs, given a list of IDs. For the builtin REST client, there is a way to get around that but with a real cost. On the storage client settings, in the "filtering" section, you may check the box "Enable Drupal-side filtering". This will support Drupal filters such as "IN" as well as others. The problem will be that the REST client will need to fetch ALL the external entities from the REST service to filter them on the client side, which can be a real problem if there are many and/or if the service is slow. You may use cache (and pre-populate it using the XNTT manager) but it may not be an ideal solution.

Regarding tests, the external entity reference field is already tested appropriately (SimpleExternalEntityTest::testSimpleExternalEntity()). No need to add new tests.

So, I'll close that issue as "Working as designed". For the very specific case of REST services and the "IN" filter operator, one of you may create a new issue that would be a "feature request" for the REST client. I don't see so far an easy way to add that feature so propose solutions if you create that feature request! :)

Great analysis! Makes things much clearer.