Views with Ajax pagers don't filter correctly

One problem with this module (both Drupal 6 and Drupal 7 versions), is that the filter only works for the current page request.
If a view has a pager, then on page 2 the filter won't work. Often this might not be an issues, but for one use case I have it is:
The page have some nodes loaded at the top, below these, there is a view with views_exclude_previous-filter.
This view has a pager and uses ajax.
So when you click for the next page, sometimes you will get the nodes that are loaded at the top.

The problem is with the static variable that is used by the filter:

static $nids = array();

This variable only lives for the current page load and is cleared for any ajax-calls that are done later on the same page.
I don't know if anyone else is bothered by this, but I am and I have started testing alternative solutions.
On is to use cache_set and cache_get to save previously loaded/viewed nodes.
The problem is where this cache should be cleared.

If I comes up with something good, I share the code here.
And if anyone have some good ideas, I will love to get some input.

Thanks!