Have a separate search input for selected items

The current implementation has 1 search input spanning both the unselected and the selected items lists:

I think there are two issues with this:
1) Usability: One expects the search two work for both underlying selects, as it spans both. It currently only searches the unselected items.
2) Practical: One needs to be able to search the selected items as well (to unselect specific items in a long list).

My proposal is to add an extra search input:

(Screenshot taken from my D7 implementation: https://www.drupal.org/project/improved_multi_select/issues/2592477 → )