Problem
Entity reference and term reference in particular have the general problem of knowing on not knowing how many options they are referencing too. In case of a vocabulary, it might be 5,20 or hundreds of thousands.
Right now, we have OptionsProviderInterface which those field types implement to provide all possible options to callers, like select widgets. However, if the number of values becomes too high it's not feasible any more to use select widgets, views exposed filter forms or Rules data input widgets.
Also one could argue that having an OptionsProvider does not fit for an autocompleted tag-style vocabulary as usually they are only auto-completed and the set of options is not limitted.
Possible solutions
Let's discuss the approach taken to deal with this situation. I see the following options:
a) Make it configurable. Users can configure for entity reference fields and taxonomy terms whether they should be rendered as select lists, e.g. in views exposed forms. That's the approach taken by ER fields in d7. In addition to exposing new settings, we'd have to come up with an implementation that only provides the options provider for those fields if configured.
b) Extend the interface with a countOptions() method which provides callers with an easy way to check for the number of available options, so the caller can decide whether it's safe / reasonable to make use of the options. E.g., this could be used to select a fitting "default widget" in views exposed filter forms or Rules data input widgets. Also, we could modify the interface to limit the number of returned values by a safe value.
c) Do not deal with it and default to "safe" default widgets where unknown. Easy, but not nice when you get only an autocomplete for a controlled taxonomy vocabulary with 5 entries (e.g. views exposed filter forms).