Access policy API cache problem

Problem/Motivation

I've been looking into the new access policy API introduced in Drupal 10.3, and while it works for the most part, I am seeing some caching issues that I'm trying to figure out. My project now has a bunch of config overrides which assigns roles/permissions on the fly, but I've been looking into removing these overrides in favor of the access policy API.

This isn't my exact scenario...but for instance, say I have something like this:

public function calculatePermissions(AccountInterface $account, string $scope): RefinableCalculatedPermissionsInterface {
    $calculated_permissions = parent::calculatePermissions($account, $scope);

    if (!in_array('content_type_manager', $account->getRoles())) {
      return $calculated_permissions;
    }

    // Get a list of all available content types.
    $types = $this->entityTypeManager->getStorage('node_type')->loadMultiple();

    // Loop through each content type.
    foreach ($types as $type) {
      $calculated_permissions
      ->addItem(new CalculatedPermissionsItem(['create ' . $type . ' content']));
    }

    return $calculated_permissions;
  }

If a user has the content_type_manager role, I want them to be able to create content for any content type. If not, then I don't want that permission assigned to them.

This seems to work fine... however in my functional test, when I create a new content type:

$this->drupalcreateContentType([
  'type' => 'new_content_type',
  'name' => 'New content type'
]);

then attempt to log in with a user that has the content_type_manager role, I'm getting a 403 when I visit node/add/new_content_type. If I force my test to clear caches right before that (using something like $this->resetAll();), then it works fine. However, I obviously don't want to force a cache clear here to solve this problem.

I'm still learning the access policy API, but I seem to be missing something here. I never had problems assigning config overrides when it came to the example above, but I'm having difficulty figuring out what I need to do to get this to work.

Any suggestions would be greatly appreciated!

Thanks