Page access cache has to be dependent on conditions

Thanks for the review @jeni_dc but we need more reports here and a code review to make sure that this attempt to fix has no flaws or not a wrong approach on long run.

Apart from that, your initial review is over 4 years ago while your RTBC setting only 26 days ago. There is a great chance that this pathc needs a reroll ir still rewuired since latest dev of page_manager is younger then this patch.