Base field support?

Or is this a very special case that needs custom implementation in Drupal Commerce?

At the moment its just finding field configs: https://git.drupalcode.org/project/views_core_entity_reference/-/blob/1....

But if you could add some way to load up the various entity types available in a site, check their definitions for entity reference fields, similar auto-opting in could be made for base fields that are entity references.

Just to note base fields can be a bit more dangerous as its possible Core has replaced some base field filter handlers (e.g. language) with purpose-built filter handlers. So it may be that you'd want to opt-in to specific fields (see example on module homepage) rather than a blanket opt-in of base fields.