Notify only when subscribed to a term from two term reference fields (or vocabs)

I have been searching and can't seem to find an answer to this, so I hoping someone here can provide some guidance.

Ultimately I am trying to implement a subscription notification that only notifies when a user is subscribed to two terms (one from each vocabulary).

I have nodes with two taxonomy term reference fields for two different vocabularies.

Sample:
Vocab 1
* apples
* bananas
* carrots

Vocab 2
* fresh
* expiring
* rotten

With the documentation and example module, I was able to set up a term subscription which seems to be working okay, such that regardless of vocabulary, a user can subscribe to get notified when a node is submitted which contains any of their subscribed terms. However, users would like to be able to configure their subscription so that they only receive a notification if the node submitted has a subscribed term from vocab 1 AND a subscribed term from vocab 2.

Quick example: User subscribed to the terms 'apples', 'expiring' and 'rotten'. With the current implementation, a node having terms "bananas" and "rotten" would kick off a notification to the user. Further, a node having terms "apples" and "fresh" would also kick off a notification. Ultimately, the user only wants notifications for nodes containing the term "apples", and only if the node also contains the term "expiring" or "rotten".

I initially started playing with hook_message_subscribe_get_subscribers_alter(), however I quickly realized once I starting doing logic checks on each users that I seemed to be re-implementing the functionality that already exists in the module.

So my question is, is there a better way to implement than just running through the uids and manually checking subscriptions?