🇺🇸United States @fathershawn

Resetting status and moving to "Fixed" at @jurgenhaas suggestion so someone can comment and assign issue credits. Thought we needed to commit something to do that :)

I feel like this issue did it's job. We are now taking the learning from this work into new child issues on 🌱 Gradually replace Drupal's AJAX system with HTMX Active

MR is updated. Leaving as "active" until we have the security policy resolved

fathershawn → created an issue.

The storage is implemented by your own plugin class as the kind of storage varies by use case. My suggestion in #78 was to implement the storage methods but within them use a dpm to see the values rather than store them so you could see what was returned.

Alternatively you can use the provided storage trait and put a dpm() or ksm() call in the trait.

The snippet of code provided in #75 is from the StateTokenStorage which itself is provided to reduce the code that developers need to implement in their plugins.

If the StateTokenStorage file is empty of code then something is wrong with your code base. If you mean that nothing has been stored in the State system see my next paragraph.

The interface requires that you implement public function storeAccessToken(AccessTokenInterface $accessToken) but while you are developing you don't have to store anything. You could simply print info from the token object to a message in your implementation of this method and not use the trait.

Linting complete.

Most tests fixed.

Only 1 failing test: ActiveLinkResponseFilterTest. Suggestions welcome from anyone else who has worked in this area. I'm going to fire up xdebug to step through and puzzle out what is needed

So apparently it always has worked but no message of getting the token was ever placed anywhere to inform of success as you stated is supposed to occur

I have not been able to pull and print it from token storage however

The message display is a conditional in the StateTokenStorage trait. I have just re-verified that it is functioning correctly.

  /**
   * Stores access tokens obtained by the client.
   *
   * @param \League\OAuth2\Client\Token\AccessTokenInterface $accessToken
   *   The token to store.
   */
  public function storeAccessToken(AccessTokenInterface $accessToken): void {
    $this->state->set('oauth2_client_access_token-' . $this->getId(), $accessToken);
    if ($this->displaySuccessMessage()) {
      $this->messenger->addStatus(
        $this->t('OAuth token stored.')
      );
    }
  }

I don't find any cloud based services for testing the client credentials flow, but you can configure and authorization code plugin to intereact with https://docs.wiremock.io/security/oauth2-mock#using-with-your-app

It was the last client that sends and gets a response

I have different results. In a clean install of Drupal, using latest version of Devel, the example plugin is both posted and uploaded in #66 along with inserting debugging functions in both sides of the code branch within getParsedResponse also posted in #66, I get an expected denial response from USPS, also as posted when using random strings as client id and client secret.

I can try to add some more debugging output built into the module, but you could one of Devel's other functions there such dpm() instead.

Rebased on this week's changes. Looking at test failures next

What I posted in #66 was the result of enabling the attached module in a new install of Drupal and altering the AbstractProvider code as shown.

I used the latest version of Devel which has this backwards compatibility code: https://gitlab.com/drupalspoons/devel/-/blob/5.x/devel.module?ref_type=h... but any of Devel's methods of outputing data would probably work. I use other methods to inspect values at runtime but this was my best idea.

I entered dummy client ID and secret and did a save and test. USPS responded with a message about improper credentials. What I posted works correctly, it just needs a real id and secret. If it is not working in your situation there must be some interaction with other code or the environment. I can't think of anything else.

If you have a ksm() call in both sides of the try/catch as shown in AbstractProvider then something is very much off with how the code is running in your setup. That's a dual path code branch and so one of them runs.

I can't tell more from here. Is there a Drupal community where you are? Someone you could pair with to debug this with you?

Making multiple large changes to your code is not the best way to debug. Single changes and test will get you where you want to be.

You didn't get a message in #62 likely because USPS returned an error. So let's move to this:

    public function getParsedResponse(RequestInterface $request)
    {
        try {
            $response = $this->getResponse($request);
ksm($response->getBody()->getContents());
        } catch (BadResponseException $e) {
            $response = $e->getResponse();
ksm($response->getBody()->getContents());
        }

        $parsed = $this->parseResponse($response);

        $this->checkResponse($response, $parsed);

        return $parsed;
    }

With this in place, and this plugin (Also uploaded in a zip file as example 3)

<?php

declare(strict_types=1);

namespace Drupal\usps_api_example\Plugin\Oauth2Client;

use Drupal\Core\StringTranslation\TranslatableMarkup;
use Drupal\oauth2_client\Attribute\Oauth2Client;
use Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginBase;
use Drupal\oauth2_client\Plugin\Oauth2Client\StateTokenStorage;
use League\OAuth2\Client\Provider\AbstractProvider;
use League\OAuth2\Client\Provider\GenericProvider;

/**
 * Client credentials for USPS test endpoint.
 */
#[Oauth2Client(
  id: 'usps_test_api',
  name: new TranslatableMarkup('USPS Test API'),
  grant_type: 'client_credentials',
  authorization_uri: 'https://apis-tem.usps.com/oauth2/v3/token',
  token_uri: 'https://apis-tem.usps.com/oauth2/v3/token',
)]
class UspsTestApi extends Oauth2ClientPluginBase {
  use StateTokenStorage;

  /**
   * Creates a new provider object.
   *
   * @return \League\OAuth2\Client\Provider\GenericProvider
   *   The provider of the OAuth2 Server.
   */
  public function getProvider(): AbstractProvider {
    return new GenericProvider(
      [
        'clientId' => $this->getClientId(),
        'clientSecret' => $this->getClientSecret(),
        'urlAuthorize' => $this->getAuthorizationUri(),
        'urlAccessToken' => $this->getTokenUri(),
        'urlResourceOwnerDetails' => $this->getResourceUri(),
      ],
      $this->getCollaborators()
    );
  }

}

I get this message after clicking "Save and request token"

You have two return statements in your getProvider code in #63

Maybe try inserting calls to ksm earlier in the stack after you fix that. You should be able to use that to find where it is failing

Please add the Devel module to your test site:

and enable the devel module

Then edit the code brought down by composer. Add a call to ksm() in the try block of this function:

    /**
     * Sends a request and returns the parsed response.
     *
     * @param  RequestInterface $request
     * @return mixed
     * @throws IdentityProviderException
     * @throws UnexpectedValueException
     * @throws GuzzleException
     */
    public function getParsedResponse(RequestInterface $request)
    {
        try {
            $response = $this->getResponse($request);
ksm($response);
        } catch (BadResponseException $e) {
            $response = $e->getResponse();
        }

        $parsed = $this->parseResponse($response);

        $this->checkResponse($response, $parsed);

        return $parsed;
    }

You will find this in your site at vendor/league/oauth2-client/src/Provider/AbstractProvider.php. This will output a drupal message on the page with the response recieved from USPS after you hit the "Save and request token" button on your plugin's form.

Merged

MR created

This module does provide the route described in those issues

entity.oauth2_client.edit_form:
  path: '/admin/config/system/oauth2-client/{oauth2_client}/edit'
  defaults:
    _entity_form: 'oauth2_client.edit'
    _title: 'Edit an oauth2 client'
  requirements:
    _permission: 'administer oauth2 clients'

Still and yet somehow on re install no entity is created for oauth2_client

Oauth2ClientListBuilder triggers discovery on load. I can't reproduce this and do not have any way to know why additional plugins you have created are not discovering properly. I'm assuming that you have tried clearing cache as the traditional cure-all.

Thanks for the offer but there's no need to expand the features of this module when simple_oauth has the Oauth server function well covered. Have a look over there!

Oh Rocket Chat seems to be turned around. This module is not an Oauth2 server. It provides Oauth2 clients for Drupal to connect via Oauth to other services.

I think they were looking for https://www.drupal.org/project/simple_oauth →

I'll reach out to them.

I don't know what to advise you about that error. This module does not alter any routes. I just uninstalled the module in my development site without error.

fathershawn → created an issue. See original summary → .

USPS told me anything more than what the example shows sent in the client_credentials request will cause a fail

There's the issue then and it is my code. We have

redirect_code<code> as required and it's sending that when along with your other data.  I'm going post below how you can work around it.  Or you can extend AbstractProvider and fully customize by using your provider class in this function.

Add the provider classes to your your <code>Oauth2Client

plugin class:

use League\OAuth2\Client\Provider\AbstractProvider;
use League\OAuth2\Client\Provider\GenericProvider;

The method below omits scope and all other body data except your credentials. urlResourceOwnerDetails is required by GenericProvider but will be dropped before the request if empty. Add it to your Oauth2Client plugin class to override behavior in Oauth2ClientPluginBase

  /**
   * Creates a new provider object.
   *
   * @return \League\OAuth2\Client\Provider\GenericProvider
   *   The provider of the OAuth2 Server.
   */
  public function getProvider(): AbstractProvider {
    return new GenericProvider(
      [
        'clientId' => $this->getClientId(),
        'clientSecret' => $this->getClientSecret(),
        'urlAuthorize' => $this->getAuthorizationUri(),
        'urlAccessToken' => $this->getTokenUri(),
        'urlResourceOwnerDetails' => $this->getResourceUri(),
      ],
      $this->getCollaborators()
    );
  }

This results in the following data prepared for the request in my local testing:

[
  "headers" =>  [
    "content-type" => "application/x-www-form-urlencoded"
  ]
  "body" => "client_id=foo&client_secret=bar&grant_type=client_credentials"
]

All caps is shouting in text interchange. I think you must know that, so please stop.

I'll explain in more detail about client id and secret.

You can see in my first code block in #50 that these values are put in an associative array and stored in the database as key-value pair based on your plugin.

My last code block shows these values being retrieved from this storage and cached locally in the credentials array. I said this was used by the getters, so let me post an explicit example for client id. There is a matching method for client secret.

  /**
   * {@inheritdoc}
   */
  public function getClientId(): string {
    $credentials = $this->retrieveCredentials();
    return $credentials['client_id'] ?? '';
  }

I previously posted where this method is called in the constructor within Oauth2ClientPluginBase::getProvider

        'clientId' => $this->getClientId(),

This gets stored by the constructor of GenericProvider in the clientId property.

public function __construct(array $options = [], array $collaborators = [])
    {
        $this->assertRequiredOptions($options);

        $possible   = $this->getConfigurableOptions();
        $configured = array_intersect_key($options, array_flip($possible));

        foreach ($configured as $key => $value) {
            $this->$key = $value;
        }

        // Remove all options that are only used locally
        $options = array_diff_key($options, $configured);

        parent::__construct($options, $collaborators);
    }

I posted the code for \League\OAuth2\Client\Provider\AbstractProvider::getAccessToken in #48 which shows that the clientId property is stored into the $params array. I also showed the code in #48 of how this value gets encoded and placed as the body of the request.

There is a line that sets a string for use to send as a header but right now that above I have raised four times to no address

You have posted multiple times about a basic authorization scheme class from the upstream library that is not used in this module. A developer could code one of our plugins to use but that class. That is not the case by default.

Your credentials are stored based on from submission. See \Drupal\oauth2_client\Form\Oauth2ClientForm::submitForm

 public function submitForm(array &$form, FormStateInterface $form_state): void {
    // Prepare the credential data for storage and save in the entity,.
    $values = $form_state->getValues();
    $provider = $values['credential_provider'];
    // Set default storage key.
    $key = $this->entity->uuid() ?? $form['#build_id'];
    switch ($provider) {
      case 'oauth2_client':
        $credentials = [
          'client_id' => $values['client_id'],
          'client_secret' => $values['client_secret'],
        ];
        $this->state->set($key, $credentials);
        break;

      case 'key':
        $key = $values['key_id'];
    }
    $form_state->setValue('credential_storage_key', $key);
    parent::submitForm($form, $form_state);
  }

And then \Drupal\oauth2_client\Service\CredentialProvider::getCredentials is used to retrieve those values.

  public function getCredentials(Oauth2ClientPluginInterface $plugin): array {
    /** @var \Drupal\oauth2_client\Entity\Oauth2Client $config */
    $config = $this->entityTypeManager->getStorage('oauth2_client')->load($plugin->getId());
    $credentialProvider = $config->getCredentialProvider();
    $storageKey = $config->getCredentialStorageKey();
    $credentials = [];
    if (empty($credentialProvider) || empty($storageKey)) {
      return $credentials;
    }
    switch ($credentialProvider) {
      case 'key':
        $keyEntity = $this->keyRepository->getKey($storageKey);
        if ($keyEntity instanceof Key) {
          // A key was found in the repository.
          $credentials = $keyEntity->getKeyValues();
        }
        break;

      default:
        $credentials = $this->state->get($storageKey);
    }

    return $credentials ?? [];
  }

This service is called in the base plugin getters for these values. For example:

  /**
   * Helper function to retrieve and cache credentials.
   *
   * @return string[]
   *   The credentials array.
   */
  private function retrieveCredentials(): array {
    if (empty($this->credentials)) {
      $this->credentials = $this->credentialService->getCredentials($this);
    }
    return $this->credentials;
  }

It also appears that this is still sent in a URL header string which will fail.

Can you point to code that supports this assertion? I've now twice posted the code showing that parameters are sent as form values in the body via POST.

Thank you, but there is nothing wrong with MY Drupal site. I have been working with Drupal since version 6. I am not to the point of debugging MY code, this is the module code not working. Simple oauth works, curl works, Postman works, oauth2_client does not.

That's not a respectful answer to my attempts to help you. This module provides tools for you to build oauth2 clients. Correctly building and configuring the oauth2 client is your code, that I can neither access nor debug. I've also been working in Drupal since D6 and am 100% certain that I will make an unintentional mistake or bug in software that I create and have to find it. That is true of all of us.

I'm going to walk you through how it uses the data that you have entered and the plugin definition that you have created to communicate. I hope that helps you focus your development efforts. I also want to assure someone else who come upon this issue that this module does what it sets out to do as you are implying that it does not in fact work properly.

I don't know what else to do to help you but I will continue to answer specific questions about the code because that is who we are as a community. However I'm not going to respond to general assertions that this module is completely broken. It's not true.

Walkthrough

For this discussion, I'll use code that I've sent you:

declare(strict_types=1);

namespace Drupal\usps_api_example\Plugin\Oauth2Client;

use Drupal\Core\StringTranslation\TranslatableMarkup;
use Drupal\oauth2_client\Attribute\Oauth2Client;
use Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginBase;
use Drupal\oauth2_client\Plugin\Oauth2Client\StateTokenStorage;

/**
 * Client credentials for USPS test endpoint.
 */
#[Oauth2Client(
  id: 'usps_test_api',
  name: new TranslatableMarkup('USPS Test API'),
  grant_type: 'client_credentials',
  authorization_uri: 'https://apis-tem.usps.com/oauth2/v3/token',
  token_uri: 'https://apis-tem.usps.com/oauth2/v3/v3/token',
)]
class UspsTestApi extends Oauth2ClientPluginBase {
  use StateTokenStorage;
}

See https://project.pages.drupalcode.org/oauth2_client/creating-plugins/

Without additional modules installed, an admin must browse to /admin/config/system/oauth2-client, enter the client id and client secret and save the values. These credentials are stored via the State system in your database.
See https://project.pages.drupalcode.org/oauth2_client/secrets/

This module provides a service which is used to obtain your token (See https://project.pages.drupalcode.org/oauth2_client/tokens/). Assume that $oauth2ClientService is an instance of Oauth2ClientService. This is not a resource owner grant so only the plugin id is needed as a parameter:

$access_token = $oauth2ClientService->getAccessToken('usps_test_api);

Here's the method being called:

public function getAccessToken(string $pluginId, ?OwnerCredentials $credentials = NULL): ?AccessTokenInterface {
    try {
      $client = $this->getClient($pluginId);
      return $client->getAccessToken($credentials);
    }
    catch (InvalidOauth2ClientException $e) {
      Error::logException($this->logger, $e);
      return NULL;
    }
  }

The plugin class above does not implement this method directly but depends on Oauth2ClientPluginBase. That code is below, I've added multiple comments for this walkthrough.

public function getAccessToken(?OwnerCredentials $credentials = NULL): ?AccessTokenInterface {
// First see if there is a token stored in the database.
    $accessToken = $this->retrieveAccessToken();
    if ($accessToken instanceof AccessTokenInterface) {
// Found one, now is it expired?
      $expirationTimestamp = $accessToken->getExpires();
      $expired = !empty($expirationTimestamp) && $accessToken->hasExpired();
      if (!$expired) {
        return $accessToken;
      }
// It is expired, do we also have a refresh token that we can use?
      $refreshToken = $accessToken->getRefreshToken();
      if (!empty($refreshToken)) {
        $accessToken = $this->refresh->getAccessToken($this);
        return $accessToken;
      }
// It's expired.  We can't make a direct request for an auth code grant and we don't store owner credentials.
      if ($this->getGrantType() === 'authorization_code' || $this->getGrantType() === 'resource_owner') {
        throw new NonrenewableTokenException($this->getGrantType());
      }
    }
// We didn't find a stored token, so get a new token based on grant type.
    if ($this->grantType instanceof GrantWithCredentialsInterface) {
      $this->grantType->setUsernamePassword($credentials);
    }
    $token = $this->grantType->getAccessToken($this);
    if ($token instanceof AccessTokenInterface) {
      $this->storeAccessToken($token);
      return $token;
    }
    return NULL;
  }

Your token is using client credentials grant so we look to

ClientCredentials::getAccessToken

  public function getAccessToken(Oauth2ClientPluginInterface $clientPlugin): ?AccessTokenInterface {
    $provider = $clientPlugin->getProvider();
    $optionProvider = $provider->getOptionProvider();
    // If the provider was just created, our OptionProvider must be set.
    if (!($optionProvider instanceof ClientCredentialsOptionProvider)) {
      $provider->setOptionProvider(new ClientCredentialsOptionProvider($clientPlugin));
    }
    try {
      return $provider->getAccessToken('client_credentials', $clientPlugin->getRequestOptions());
    }
    catch (\Exception $e) {
      // Failed to get the access token.
      Error::logException($this->logger, $e);
      return NULL;
    }
  }

The ClientCredentialsOptionProvider adds scope, we'll come back to it when it actually gets used.

The first line above calls the ::getProvider method in the plugin. The one we are using depends on the base class so what's called is

  public function getProvider(): AbstractProvider {
    return new GenericProvider(
      [
        'clientId' => $this->getClientId(),
        'clientSecret' => $this->getClientSecret(),
        'redirectUri' => $this->getRedirectUri(),
        'urlAuthorize' => $this->getAuthorizationUri(),
        'urlAccessToken' => $this->getTokenUri(),
        'urlResourceOwnerDetails' => $this->getResourceUri(),
        'scopes' => $this->getScopes(),
        'scopeSeparator' => $this->getScopeSeparator(),
      ],
      $this->getCollaborators()
    );
  }

Here your plugin retrieved the client id and client secret from the database, and the urls and scopes from your plugin definition and passes all of that into an instance of GenericProvider from the league/oauth2-client library.

GenericProvider uses the parent AbstractProvider::getAccessToken

public function getAccessToken($grant, array $options = [])
    {
        $grant = $this->verifyGrant($grant);

        if (empty($options['scope'])) {
            $options['scope'] = $this->getDefaultScopes();
        }

        if (is_array($options['scope'])) {
            $separator = $this->getScopeSeparator();
            $options['scope'] = implode($separator, $options['scope']);
        }

        $params = [
            'client_id'     => $this->clientId,
            'client_secret' => $this->clientSecret,
            'redirect_uri'  => $this->redirectUri,
        ];

        if (!empty($this->pkceCode)) {
            $params['code_verifier'] = $this->pkceCode;
        }

        $params   = $grant->prepareRequestParameters($params, $options);
        $request  = $this->getAccessTokenRequest($params);
        $response = $this->getParsedResponse($request);
        if (false === is_array($response)) {
            throw new UnexpectedValueException(
                'Invalid response received from Authorization Server. Expected JSON.'
            );
        }
        $prepared = $this->prepareAccessTokenResponse($response);
        $token    = $this->createAccessToken($prepared, $grant);

        return $token;
    }

Here your data gets prepared. This helper method gets called:

protected function getAccessTokenRequest(array $params)
    {
        $method  = $this->getAccessTokenMethod();
        $url     = $this->getAccessTokenUrl($params);
        $options = $this->optionProvider->getAccessTokenOptions($this->getAccessTokenMethod(), $params);

        return $this->getRequest($method, $url, $options);
    }

The default method is POST

    protected function getAccessTokenMethod()
    {
        return self::METHOD_POST;
    }

The url is the token url that was set from your plugin definition.
The options are set in this case by the ClientCredentialsOptionProvider::getAccessTokenOptions which adds scope to the parameters, and calls \League\OAuth2\Client\OptionProvider\PostAuthOptionProvider::getAccessTokenOptions

    public function getAccessTokenOptions($method, array $params)
    {
        $options = ['headers' => ['content-type' => 'application/x-www-form-urlencoded']];

        if ($method === AbstractProvider::METHOD_POST) {
            $options['body'] = $this->getAccessTokenBody($params);
        }

        return $options;
    }

Which sets the content type and encodes the values for the request body.

This is all sent to a request factory and sent back from getAccessTokenRequest as a request object.

We are now to ::getParsedResponse in the code above which calls ::getResponse which uses a Guzzle client to send the request.

    public function getResponse(RequestInterface $request)
    {
        return $this->getHttpClient()->send($request);
    }

If JSON is returned it is parsed and turned into an AccessToken object. Otherwise errors are thrown.

Also, we just launched more extensive documentation at https://project.pages.drupalcode.org/oauth2_client/

You do not need to rewrite the module code. You do not really need to verify that the upstream package AbstractProvider actually sends the correct data because you can see that nearly 1,500 other Drupal sites are happily using it and its basic functions work. In addition, we use this upstream package because it is highly regarded, has been required via composer more than 88 million times in the PHP ecosystem and has thousands of stars on packagist.org.

You need to be sure that you have configured and implemented your custom plugin correctly. You don't need to use Xdebug. You can also use the Devel or Kint module to display values from your code in the browser.

You also should consider using a local development environment so that you can iterate more quickly on your code. There are several video tutorials. Here's one: https://www.youtube.com/watch?v=8TaL6UmOohc and documentation here of d.o: https://www.drupal.org/search/site/ddev →

Added info on advanced customization

I've given this some thought and want to make this easier for users to implement.

Migrating to Gitlab pages

Initial setup complete at https://project.pages.drupalcode.org/oauth2_client/

That display has no relation to the function of this module. It is telling you how to obtain data about the configuration entities that this module creates using the REST API.

I've traced the code for you showing that the token request method is a POST. I've now added a line to the automated test for that flow to the dev branch of this module to further verify:

if ($request instanceof Request) {
      $this->assertEquals('POST', $request->getMethod());
      $body = [];
      parse_str($request->getBody()->getContents(), $body);
      $this->assertArrayHasKey('scope', $body);
      $this->assertEquals('test-1,test-2', $body['scope']);
    }

This test passes with the additional assertion.

I gave guidance on how and where to inspect what you are sending to USPS in #41. Did you try that?

There are a variety of ways that to customize aspects of the flow. If you don't want to post publicly what it is that you need to change, perhaps you could DM me in Drupal Slack.

Sounds good. Please do report back on what you find.

Do you know how to use Devel module functions → or even better Xdebug to inspect values as the code executes?

The request/response exchange you will want to examine is at vendor/league/oauth2-client/src/Provider/AbstractProvider.php:645-646

I don't have a USPS account and I can't directly debug this. I don't have any other reports of client credentials not working. If it still doesn't work then either the URLs have to be wrong or you may have a typo in the client id or client secret. I can't think of anything else

I just do not see where it is put into or called into the body as a parameter.

You don't see it here as it is in the upstream library. Here's a GitHub link since I can't link to your code to the default options for POST, which is the default method: https://github.com/thephpleague/oauth2-client/blob/8cc8488e627ab17b71238ef4c09235e95f4d5068/src/OptionProvider/PostAuthOptionProvider.php#L30

The upstream library expects Guzzle. You may be able to use custom code to implement something with curl but that's not the standard practice in PHP now. Guzzle is pretty ubiquitous. I've removed the scopes since that's optional and corrected the test plugin id.

That is what I last had in #24 calling by variable

Your metadata is not correct in #24.

believe it is continuing to send data in a header

Can you point to where that is happening in my code or the upstream library? I'm not aware of such logic and will need to fix it if true.

I have attached a module with example plugins for your use case. Please let me know if they work, and this back and forth is not producing a conclusion.

Looks like a Guzzle issue, but why is it trying to call "OwnerCredentials" ???

OwnerCredentials is a value object that contains the client id and secret. It is used to pass those values so that they don't get captured by logs unintentionally revealed.

You are actually successfully communicating with USPS: https://developer.usps.com/oauth#tag/Resources/operation/post-token invalid_request is from them and the exception is thrown because an HTTP code 200 is not received.

This module should log the exception in a way that includes the return code and that will give you more of a clue.

But you may not need this last round of customizations. The documentation page I found and linked above shows application/json as a option not a requirement! In the right sidebar one can switch between that and the standard application/x-www-form-urlencoded so they do accept standard Oauth2.

Scope is listed as optional in the standard, but maybe not for USPS?

Note that they include scope in their example values:

urlencoded:

grant_type=client_credentials &client_id=123456789 &client_secret=A1B2C3D4E5 &scope=ResourceA+ResourceB+ResourceC

json:

{
  "grant_type": "client_credentials",
  "client_id": "123456789",
  "client_secret": "A1B2c3d4E5",
  "scope": "ResourceA ResourceB ResourceC"
}

So try adding a scope value or values to your plugin annotation.

Do you have database logging enabled? If not do you have access to PHP error logs? Can you post any errors shown in your logs?

The \League\OAuth2\Client\OptionProvider\PostAuthOptionProvider enforces the content type and encoding in the Oauth2 standard. We need to override that to set the json option, so you will want your own options provider class. Place this class in your module at src/OAuth2/Client/OptionProvider/UspsClientCredentialsOptionProvider.php, and be sure to set the namespace to the right value for your module.

declare(strict_types=1);

namespace Drupal\your_module\OAuth2\Client\OptionProvider;

use Drupal\oauth2_client\OAuth2\Client\OptionProvider\ClientCredentialsOptionProvider;
use Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginInterface;

/**
 * An option provider which alters the token request content type.
 */
class UspsClientCredentialsOptionProvider extends ClientCredentialsOptionProvider {


  /**
   * A string of scopes imploded from the Oauth2ClientPlugin.
   */
  private string $scopeOption;

  /**
   * {@inheritdoc}
   */
  public function __construct(Oauth2ClientPluginInterface $clientPlugin) {
    $scopes = $clientPlugin->getScopes();
    if (!empty($scopes)) {
      $this->scopeOption = implode($clientPlugin->getScopeSeparator(), $scopes);
    }
  }

  /**
   * {@inheritdoc}
   */
  public function getAccessTokenOptions($method, array $params): array {
    if (!empty($this->scopeOption)) {
      $params['scope'] = $this->scopeOption;
    }
    return ['json' => $params];
  }

}

Next, add an override in your plugin class. First add the proper use statement for your new options provider:

  use Drupal\your_module\OAuth2\Client\OptionProvider\UspsClientCredentialsOptionProvider;

Then override the getProvider method.

  /**
   * {@inheritdoc}
   */
  public function getProvider(): AbstractProvider {
    $provider =parent::getProvider();
    $provider->setOptionProvider(new UspsClientCredentialsOptionProvider($this));
    return $provider;
  }

I think I found the disconnect! The Oauth2 standard for client credentials states:

The client makes a request to the token endpoint by adding the
following parameters using the "application/x-www-form-urlencoded"
format per Appendix B with a character encoding of UTF-8 in the HTTP
request entity-body

However, your reference to Postman prompted me to go look at the USPS examples in GitHub again. Both the Postman file and the curl example use a non-standard json content type:

curl -X 'POST' 'https://apis.usps.com/oauth2/v3/token' \
     --header 'Content-Type: application/json' \
     --data '{
		"client_id": "{{CLIENT_ID}}",
		"client_secret": "{{CLIENT_SECRET}}",
		"grant_type": "client_credentials"
		}'

This has to be the source of your failure and I'll post back code in a bit you can add to your plugin to override the standard setting.

So does the client credentials flow send only a POST by default??? If so then the modification is only to send the data params in the body and not the header like the last thing I asked about the $options section I posted of the example using your previous code solution.

All of the token requests are POST by default.

All of the POST requests send the data in the body, content 'application/x-www-form-urlencoded' by default. That is to say the request sends data as if it were submitting an HTML form. This is default Oauth behavior.

I don't know why your plugin implementation is not working, but neither request method nor non-standard data transmission are not the reasons. You've talked about changing the corresponding configuration entity form. Have you done so? If so please revert your version of this module to the released state before we continue.

Please post the current state of your plugin class.

Tests passed

Thanks for the report. Found the issue and creating a fix

fathershawn → made their first commit to this issue’s fork.

Good changes to the summary @catch!

I think the simplest way is to use this module's original default method on the /donate route as that establishes a dedicate checkout flow to build and process the donation transaction. We provide a field for monthly here, but you would need to investigate how to use that with the Commerce integration for Stripe:

• https://www.centarro.io/drupal-commerce/integrations/stripe
• https://www.drupal.org/project/commerce_stripe →

You might find that you need to extend our dedicated checkout flow plugin to do more work in the submit method. If you let me know what you find I'm happy to continue to advise/help. This issue hints at the configuration for repeating payments: 📌 Make sure recurring payments are working properly Fixed

It could. I ported this code to a contributed module from an unreleased client project. In that project the recurring payment was a feature of their processor. One route would be to complete integration with commerce_recurring if that supports what you need. Another is to customize the submission logic in the checkout process.

If you let me know what direction you would like to go and which checkout route you would be using I can give more advice.

I'm happy to keep chatting. This felt like a conclusion to me:

OK, thanks I will see what I can do in the client form with a for example

if ($grantType == 'client_credentials') {
$form['oauth2_client']['data in body'] = [
'#type' => 'checkbox',
'#title' => $this->t('Enabled'),
'#default_value' => $this->entity->point to the code needed(),
];

As needed et cetera and it will be a stretch to my skills to cobble together but there is other code helpful on the internet on the issue as it is a common problem one

However my support for you is challenging each post from you has multiple topics/threads and often I can't see how it connects to what I last said. I'm gathering that you are fully focused on getting connected to USPS and aren't willing to take the interim step of getting Oauth working in your site in general to known test service, so let's stay with that, and I'll try to explain what you are seeing in their examples and how it relates to what I have written.

I wrote

The initial auth code request is a GET to a url on the remote service where the user logs in, authorizes the connection. This generates a code which is sent in a query parameter to the original site (Drupal) which then makes a POST to a token url with that code to get the token.

And your reply is

It sounds like you are saying it sends a GET request first, which the server you say sends back a response and then this client software sends a POST. Well the first GET sent will cause a FAIL - period - there will be no further communication as USPS has set it up

If you look at the Example OAuth Client Credentials Token request on the USPS example page onb Github, there is one POST to the /token URL ad that is all - with the data sent in the body. That is what works, and all that works - so how is this module stopped from sending the intial GET request in favor of the POST wit the data in the body

I am describing the process for the authorization code flow and you are responding about the client credentials flow. This module will not send the client credentials token request, or any token request in a GET since I am not overriding the default in the upstream library.

It seems like you keep switching between grant flows. Let's pick one, stick with it, and get it working for you.

The token request is normally sent in the body here.

Look at \Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginBase::getProvider.

This method instantiates \League\OAuth2\Client\Provider\GenericProvider

Look at \League\OAuth2\Client\Provider\GenericProvider::getAccessTokenMethod which if there is no override value created above calls \League\OAuth2\Client\Provider\AbstractProvider::getAccessTokenMethod which returns POST.

The initial auth code request is a GET to a url on the remote service where the user logs in, authorizes the connection. This generates a code which is sent in a query parameter to the original site (Drupal) which then makes a POST to a token url with that code to get the token.

This module would allow you to completely customize all the available options in GenericProvider or AbstractProvider by overriding ::getProvider in your plugin and changing how the provider is instantiated.

Yes, you can install the Devel Kint module and use the dpr() function to print any variable to data to the browser.
See https://www.drupal.org/docs/extending-drupal/contributed-modules/contributed-module-documentation/devel/introduction →

I still recommend you simplify your problem and get an auth code flow working against a reliable testing service and then pivot to USPS.

The plan was to use the Oauth2Client module for token management purposes for access only, Otherwise yes I would need far too many plugins. My other file is set up for CURL calls that would handle the API calls. I would only need the initial client credentials for authcode file, and the access file, and a refresh file I have not built yet

That's a good plan, and this module is not only for getting the initial token but for managing the refresh token. It allows a developer to just use Oauth and not mess with the details. With this module you don't have to manage the details of an auth code flow, neither the code nor the state parameter, nor storing the refresh token, nor using the refresh token to get a fresh token. All of that is what this module is for. It's not clear from this thread why the USPS service isn't working for you. I don't have access to it so can't debug it.

This module is built to integrate Drupal with the excellent Oauth2 Client PHP library, and both it and the upstream library conform to the Oauth2 standards.

It seems like you are facing two challenges. One is implementing Oauth in your Drupal code and the other is connecting with USPS. My only remaining suggestion is to simplify the problem and just work at implementing Oauth in your code. The folks at Okta provide an Oauth playground. They don't offer client credentials but they do offer Auth code workflow. You can register an account and get oauth working in your code. Then you can repoint your working code at USPS, and if it doesn't work then something is off and you should reach out to them.

fathershawn → created an issue.

Are you saying to add the code from #11 to the plugin or elsewhere or is it already existing code elsewhere in the module ???

It's in the trait that you have included in the plugin already.

As to the "scope" suggestion, it is my understanding until an initial token is obtained there is no API access and then the scope access request issues are then needed. The initial token is critical to moving forward at all. You cannot jump into scope issues at all.

This is not in keeping with the spec: https://datatracker.ietf.org/doc/html/rfc6749#section-3.3 explained in more plain language at https://oauth.net/2/scope/. The scope is encoded in your token.

You've added an unsupported property into your plugin definition, which is probably benign but I'm not sure:

 * state = "nonce=abscdefg#",

This module takes care of the state management.

You also have removed the two essential urls from the plugin definition for the authorization code flow. This module includes a default redirect url but you need these two as the module has no idea how to communicate with USPS.

 *   authorization_uri = "https://apis.usps.com//oauth2/v3/authorize",
 *   token_uri = "https://apis.usps.com/oauth2/v3/token",

Changing the file and rebuilding the cache does not show up changes in the config page for me - only a fresh file and flushing caches to rediscover works.

That's my mistake, forgetting a recent architecture improvement. I'll add an improvement ticket that makes it easier to clear. What's going on there is the UI is showing a set of configuration entities, which match 1:1 with the plugins, and are created when plugins are discovered. But plugin definitions are 100% cleared when the cache is rebuilt and any new plugin ids get paired with a config entity in \Drupal\oauth2_client\Plugin\Discovery\Oauth2ClientDiscoveryDecorator::getDefinitions

if I cannot get and store a token for access , that is moot.

These plugins will handle requesting and storing your token. Since you have success_message set to true in your definition, the code I posted at the end of #11 will tell you that a token was stored.

Plugin definitions are cached in Drupal, but a cache rebuild will refresh the definition. To verify that this is working, change the name property from "Code grant" to something else, maybe "USPS", and rebuild your cache. You should see the name change on the configuration page.

I think the source of your failure is missing scope in your client definition. Looking at the USPS API documentation, scope is required and it differs between APIs. For example, the address api (https://developers.usps.com/addressesv3#tag/Resources) within the Authorize details has Required scopes: addresses It looks to me like you will want a Drupal plugin for each of these API endpoints in which you set the scope property in your plugin to the required value. You could try a list of scopes, the most common separator is comma but the Oauth2 spec is space delimited and that API is already being obstinate so I'd at least start with just one scope and use it for one endpoint.

So does editing the file after discovery do NOTHING?

I can see that you are frustrated, and I am trying to help, so please try to be patient and don't shout. There are a lot of questions in your last comment. It will be much easier in this format if we go step by step, but I'll try to answer all your questions and then go back to the step 1 I requested in #9.

The documentation shows the " * " comment frame removed for the "Instagram" access example - while it ony works leaving it in place. I renamed the files to single word titles as classes

I see the difference now between attribute and annotation. The examples are attribute, and the Instagram example is annotation.

The code examples in README.md are meant to illustrate both the new Attribute based discovery/metadata and the retiring Annotation based approach. A made up example for Instagram is given in both syntaxes. Both are supported for now and Attribute is the long term approach. Attributes use a PHP8 language feature and Annotations use comments.

The examples sub-module still uses the older Annotation syntax.

You still did not say where does and how does the

$access_token = Drupal::service('oauth2_client.service')->getAccessToken($client_id);
$token = $access_token->getToken();

go in the code??

The purpose of this module is to make it easy for you to get the token value in your own code, without needing to build an oauth client. When you make a request to an Oauth2 protected api, such as the USPS api that you referer to, you need this token in your request header to authorize the request. Your api give a curl based example;

curl	-X 'GET' 'https://apis.usps.com/addresses/v3/address?streetAddress=3120%20M%20St&secondaryAddress=NW&city=Washington&state=DC&ZIPCode=20027&ZIPPlus4=3704' \
	--header 'accept: application/json' \
	--header 'authorization: Bearer $TOKEN' \

PHP commonly uses Guzzle for such requests and Drupal provides a factory for Guzzle clients, \Drupal\Core\Http\ClientFactory which has a means for setting these headers.

I have edited the client form to include the CRID and MID.

I don't recommend this as I have not had time to implement plugin specific forms. I don't see anywhere in the api documents that you linked that these values are needed to get a token, which is our goal here. There's a way to include them in a custom Key file but let's set that aside for now.

Now your custom plugin looks like it's account for all 4 of my steps. This looks right to me, but you should remove the blank line between the annotation and class declaration.

declare(strict_types=1);

namespace Drupal\xxxxxxxxxxxxx\Plugin\Oauth2Client;

use Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginBase;
use Drupal\oauth2_client\Plugin\Oauth2Client\StateTokenStorage;

/**
* Auth code
*
* @Oauth2Client(
* id = "code",
* name = @Translation("Code grant"),
* grant_type = "client_credentials",
* authorization_uri = "https://apis.usps.com/oauth2/v3/token",
* token_uri = "https://apis.usps.com/oauth2/v3/token",
* success_message = TRUE
* )
*/
class Code extends Oauth2ClientPluginBase {
  use StateTokenStorage;
}

I don't recommend storing your secrets in config, but for testing it's okay. If you remove the changes you made to my form, enter your client id and secret and test your plugin. If it works, you will get a message

/**
   * Stores access tokens obtained by the client.
   *
   * @param \League\OAuth2\Client\Token\AccessTokenInterface $accessToken
   *   The token to store.
   */
  public function storeAccessToken(AccessTokenInterface $accessToken): void {
    $this->state->set('oauth2_client_access_token-' . $this->getId(), $accessToken);
    if ($this->displaySuccessMessage()) {
      $this->messenger->addStatus(
        $this->t('OAuth token stored.')
      );
    }
  }

I recommend that you start over. The examples are intended to document how you would create a plugin class for your use case. None of them will work as is for your use case as your grant type is not used in any of the examples.

I recommend that you remove all of your copies and start fresh. If you need step by step guidance in building a plugin class, we can do that here, just post your code after each step below. In general you need to:

1. Create your own plugin class that extends \Drupal\oauth2_client\Plugin\Oauth2Client\Oauth2ClientPluginBase
2. Decide on the appropriate token storage for your use case and use the provided trait for that storage
3. Decide on Attribute → or Annotation → for your plugin definition method.
4. Add the proper data using Attribute/Annotation to setup the plugin for your use case. Possible values are in \Drupal\oauth2_client\Annotation\Oauth2Client or \Drupal\oauth2_client\Attribute\Oauth2Client.

Super news!

If you don't see your plugin listed at /admin/config/system/oauth2-client then your plugin is not being discovered.

You should see it listed like this:

And when you edit it you will see a credential section. With Key module installed you will see a selector for provider and when Key is selected a select to choose your key.

Double check the placement of your plugin class in your custom module. It should be in directory src/Plugin/Oauth2Client

Are you using Attribute or Annotation for metadata?

You will see the form when you configure the client. I know the documentation is minimal but have you followed https://www.drupal.org/docs/contributed-modules/oauth2-client/oauth2-cli... → ?

Can you be more specific about what credential data you need beyond client id and secret?

There has been no response from the project owner for 28 days

Do you have steps to reproduce?

fathershawn → made their first commit to this issue’s fork.

All good ideas. When I have time to refactor I'll review this for features in the next version.

Biweekly progress check-in meetings are now happening the htmx channel desecribed in #85

@greggles Absolutely happy to continue the basic maintenance.

@cmlara is also correct and change logs on maintainer permissions would definitely have made this issue trivial to fix and false claims easy to verify.