You have requested a non-existent service ".abstract.instanceof.Drupal\entity_usage\EntityUsageBatchManager"

After upgrading code to newer versions, you always need to run drush updatedb before doing anything else. Can you confirm if doing this after bumping to the new version fixes the issue for you?
Or alternatively, can you reproduce this in an installation where this module has never been enabled?
Thanks!

#4

It's a new installation:

$ composer require drupal/entity_usage
...
www-data@2a078b834d81:/docroot$ drush en paragraphs_library
The following module(s) will be enabled: paragraphs_library, entity_usage
...

That's odd, I can't really reproduce it on a new installation. The first output you pasted in the issue summary made me think the code was being upgraded instead of installed for the first time.
Do you have anything particular about your project that may be interfering?