Removing fields from a taxonomy causes an error

Problem/Motivation

Removing the fields allowed users or roles causes problems.

Steps to reproduce

remove the fields from a taxonomy vocab, then try to go load the term listing, you'll get an error.

Proposed resolution

public function canUserAccessTerm(int $tid, ?AccountInterface $account = NULL): bool {

    // If the account is not passed, use the currently logged-in user.
    if (!isset($account)) {
      $account = $this->account;
    }

    $term = $this->entityTypeManager->getStorage('taxonomy_term')->load($tid);
    if (!$term) {
      return FALSE;
    }
    if (!$term->hasField(self::ALLOWED_ROLES_FIELD) || !$term->hasField(self::ALLOWED_USERS_FIELD)) {
      return TRUE;
    }