New, long keys can't be stored properly

Created on 2 August 2024, 10 months ago

This is my new API key, I upgraded to next billing plan. The length is 132. https://imgur.com/5MJ1Ddz
Of course I got error: Incorrect API key provided: sk-proj-********************************************************************************************************************MSO8.
Because textfield in config form is trimmed my new key.

๐Ÿ› Bug report
Status

Active

Version

1.0

Component

Code

Created by

๐Ÿ‡บ๐Ÿ‡ฆUkraine arialblack

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @arialblack
  • ๐Ÿ‡บ๐Ÿ‡ฆUkraine arialblack
  • Status changed to Needs work 10 months ago
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom scott_euser

    Thanks for flagging the issue. Few things here:

    1. Can you say the length of your key? Text area probably not right, probably should be a text field still with a higher #size
    2. Please create as a merge request for easier review and merge
    3. If storage size in database table needs to change, needs update hook otherwise existing sites will have entity definition warnings in admin status report. Something like this https://agaric.coop/blog/change-text-field-maximum-length-drupal-8

    .

  • ๐Ÿ‡บ๐Ÿ‡ฆUkraine arialblack

    The length of my key 132.

  • ๐Ÿ‡ฏ๐Ÿ‡ดJordan rahaf albawab Amman

    Rahaf Albawab โ†’ made their first commit to this issueโ€™s fork.

  • Pipeline finished with Success
    9 months ago
    Total: 140s
    #251624
  • ๐Ÿ‡ฏ๐Ÿ‡ดJordan rahaf albawab Amman

    Hi @scott_euser.
    I'm facing the same issue, and my key length is also 132. I created a merge request; you can check it.

  • ๐Ÿ‡น๐Ÿ‡ญThailand AlfTheCat

    I tried #7 but no luck yet.

  • ๐Ÿ‡ฎ๐Ÿ‡ณIndia pinesh

    Instead of increasing the maxlength for the API key field, I recommend using the Key module for securely storing API keys. This allows flexibility and better security without worrying about arbitrary field length limits.

    if (\Drupal::moduleHandler()->moduleExists('key')) {
      $key = \Drupal\key\Entity\Key::load('openai_api_key');
      $api_key = $key ? $key->getKeyValue() : '';
    } else {
      $api_key = $this->config('openai.settings')->get('api_key');
    }
    
  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom scott_euser

    Yep that's how we are doing it in the ai module which is slowly taking this over. https://www.drupal.org/project/ai โ†’

  • ๐Ÿ‡น๐Ÿ‡ญThailand AlfTheCat

    Hope I'm not overstepping my boundaries here but I think a bump in priority is called for. This module is now not available to new projects or projects using a newer api key.

    I've got one old key in my possession that's now in use on a handful of sites and that is not ideal, but for those without access to older keys this module is unusable at the moment.

    Migrating to the new ai module is definitely the way to go in general but it's going to take a lot of time in many cases so we don't have a viable work around right now leaving some people in the dust.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States shasha821110

    same issue here. Newer API Key seems make error and all the function not working in openAI module.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium stijnstroobants Leuven

    Had the same issue, my API-key had 163 characters. Forcing the field to allow 255 characters fixed the issue in my case.
    Patch attached.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom ujdave

    Had the same issue, but our key was even longer and we use they key module anyway, so on top of increasing the field size to 256 I added a bit more logic to cover the use case for they key module as well.
    Patch attached.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom ujdave
  • Pipeline finished with Failed
    7 months ago
    Total: 152s
    #326106
  • Status changed to Needs review 6 months ago
  • ๐Ÿ‡น๐Ÿ‡ผTaiwan charlesc

    same issue here.
    New/long API Key made error so the OpenAI module cant work as usually

  • ๐Ÿ‡ฒ๐Ÿ‡พMalaysia ckng

    Patch #13 works for us.
    Patch #15 using deprecated API and breaks the /admin/config/openai/chatgpt page.

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires
  • Merge request !100increase length in API field key โ†’ (Open) created by MariaB
  • Pipeline finished with Success
    5 months ago
    Total: 138s
    #366487
  • Patch #13 tested and works as expected.
    Created MR !100

  • mariab โ†’ changed the visibility of the branch bugfix/3465426-increase-maxlength to hidden.

  • Pipeline finished with Success
    5 months ago
    Total: 180s
    #366634
  • Patch #15 is a new feature and provides extra functionality which is out of scope. So, I opened a new issue for this.
    https://www.drupal.org/project/openai/issues/3493573#comment-15899026 โœจ Integration with key module Active

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    I can't turn https://git.drupalcode.org/project/openai/-/merge_requests/98 to "ready" state but it does what it promises, related to the specific issue description. I set it as RTBC.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States MattDanger

    MR #98 resolved this issue for me.

  • Status changed to RTBC 4 months ago
  • ๐Ÿ‡ฟ๐Ÿ‡ฆSouth Africa Gomez_in_the_South

    I'm also happy with this patch.
    +1 for getting it into a new release, please.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    Patch #13 but only after I uninstalled/installed the module and added the new key again. Using Drupal 10.3.11. Patch #14 didn't work for me at all

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    @danrod, based on the change of the MR or the patch in #13, I can't guess why a reinstallation of the module would be required. A cache clear wouldn't suffice?

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    @vensires I just applied the patch #13 on a clean Drupal 10 installation and I had the same issue, these were the steps.

    1. Installed Drupal 10.3.11
    2. Installed module OpenAI
    3. Added new OpenAI keys then went to /admin/config/openai/chatgpt and got the same error
    4. Applied patch #13, cleared cache, had the same issue.
    5. Installed/uninstalled module, didn't have any issues after that.

    Perhaps I'm missing another step?

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    I don't doubt you but is does seem strange to me... It's just a simple maxlength attribute change. I also checked the schema.yml just in case and there isn't anything related in there either. Something is escaping us...
    Setting this to "Needs work" until we find this and fix this.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    @vensires I think setting this issue to "Needs work" is a bit too mucho, I'll do some extra tests and post a video later today, I am sure I skipped something.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    @vensires I tested this patch again on a clean D11 installation, I installed the openapi module, enabled the modules, applied the patch then cleared cache and it worked this time, but I had to click on "Save configuration" twice for some reason.

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    Thank you for the test @danrod. I can't personally understand why the double submission or the clear cache is required; I really think it's RTBC but let's keep it under "Needs Review" for a while more.

  • ๐Ÿ‡ช๐Ÿ‡จEcuador rvntone

    Hi everyone,

    I fixed adding #maxlength as 164 for the field api_key on the ApiSettingsForm class, maybe should be higher (not sure) but that did the trick for me.

  • ๐Ÿ‡ช๐Ÿ‡จEcuador rvntone

    Hi everyone,

    I fixed adding #maxlength as 164 for the field api_key on the ApiSettingsForm class, maybe should be higher (not sure) but that did the trick for me.

  • Status changed to Needs review 25 days ago
  • ๐Ÿ‡ฉ๐Ÿ‡ฐDenmark nicklasmf

    Patch #13 works here as well.

  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    Based on the previous two comments, I would take the chance to set it as RTBC but... Has anyone else experienced the I had to click on "Save configuration" twice for some reason issue described previously by @danrod?

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    Hello @vensires, I don't see that error anymore, but I had to save twice in order to get rid of the red warning at the top.

    I believe that needs to be deleted because it's confusing since you already provided the keys.

    #15 breaks the whole thing, don't use it.

    Screenshots:

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    And on top of that, I tried to use the Chat Explorer and I got this nice WSOD:

    The website encountered an unexpected error. Try again later.
    
    OpenAI\Exceptions\ErrorException: Incorrect API key provided: org-eRv4****************OAmH. You can find your API key at https://platform.openai.com/account/api-keys. in OpenAI\Transporters\HttpTransporter->throwIfJsonError() (line 133 of /var/www/html/vendor/openai-php/client/src/Transporters/HttpTransporter.php).
    OpenAI\Transporters\HttpTransporter->requestObject() (Line: 28)
    OpenAI\Resources\Models->list() (Line: 86)
    Drupal\openai\OpenAIApi->getModels() (Line: 130)
    Drupal\openai\OpenAIApi->filterModels() (Line: 58)
    Drupal\openai_chatgpt\Form\ChatGptForm->buildForm()
    call_user_func_array() (Line: 528)
    Drupal\Core\Form\FormBuilder->retrieveForm() (Line: 279)
    Drupal\Core\Form\FormBuilder->buildForm() (Line: 73)
    Drupal\Core\Controller\FormController->getContentResult()
    call_user_func_array() (Line: 123)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 593)
    Drupal\Core\Render\Renderer->executeInRenderContext() (Line: 121)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext() (Line: 97)
    Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 183)
    Symfony\Component\HttpKernel\HttpKernel->handleRaw() (Line: 76)
    Symfony\Component\HttpKernel\HttpKernel->handle() (Line: 53)
    Drupal\Core\StackMiddleware\Session->handle() (Line: 48)
    Drupal\Core\StackMiddleware\KernelPreHandle->handle() (Line: 28)
    Drupal\Core\StackMiddleware\ContentLength->handle() (Line: 32)
    Drupal\big_pipe\StackMiddleware\ContentLength->handle() (Line: 116)
    Drupal\page_cache\StackMiddleware\PageCache->pass() (Line: 90)
    Drupal\page_cache\StackMiddleware\PageCache->handle() (Line: 50)
    Drupal\ban\BanMiddleware->handle() (Line: 48)
    Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle() (Line: 51)
    Drupal\Core\StackMiddleware\NegotiationMiddleware->handle() (Line: 36)
    Drupal\Core\StackMiddleware\AjaxPageState->handle() (Line: 51)
    Drupal\Core\StackMiddleware\StackedHttpKernel->handle() (Line: 709)
    Drupal\Core\DrupalKernel->handle() (Line: 19)
    

    I'll fire up a new Drupal 11 instance and try again, not sure if I have old codebase lying around or the module is just broken.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa
  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    Still the same issue, the strang this is that I'm using the same keys in the AI module and it works like a charm.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa
  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa
  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    Since you provided the patch, I set it as Needs Review for someone else to approve this and set it as RTBC.
    Can you also update the MR with the changes?

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    Hello @vensires I will, should I put my changes in the the MR 100 (which is hidden for some reason) ? seems the one with the more accurate changes.

  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    danrod โ†’ changed the visibility of the branch bugfix/3465426-increase-maxlength to active.

  • Pipeline finished with Success
    11 days ago
    Total: 151s
    #489705
  • ๐Ÿ‡ฌ๐Ÿ‡ทGreece vensires

    Thank you @danrod! I see you haven't added the following line of code in the MR though. Is it intentional?

    \Drupal::service('page_cache_kill_switch')->trigger();
    
  • ๐Ÿ‡จ๐Ÿ‡ฆCanada danrod Ottawa

    Hi @vensires , where is that line is supposed to be added? right in the submitForm() method? I am kinda confused:

      /**
       * {@inheritdoc}
       */
      public function submitForm(array &$form, FormStateInterface $form_state) {
        $this->config('openai.settings')
          ->set('api_key', $form_state->getValue('api_key'))
          ->set('api_org', $form_state->getValue('api_org'))
          ->save();
        parent::submitForm($form, $form_state);
    
       \Drupal::service('page_cache_kill_switch')->trigger();
      }
    }
    

    Also, I would like to know the OpenAIEventSubscriber Event Subscriber, because I believe that is causing the red box to appear again even if the credentials were provided. I'm guessing, at that point the openai.settings configuration object is not available yet and that's why you'll get an empty or null value when trying to get the value of $config->get('api_key')

      public function onKernelRequest(RequestEvent $event): void {
        $config = $this->configFactory->get('openai.settings');
    
        if ($this->adminContext->isAdminRoute() && empty($config->get('api_key'))) {
          $message = $this->t('You have not provided an OpenAI API key yet. This is required for its functionality to work. Please obtain an API key from <a href=":account">your OpenAI account</a> and add it to the <a href=":settings">OpenAI settings configuration here</a>.',
            [
              ':account' => 'https://platform.openai.com/',
              ':settings' => '/admin/config/openai/settings',
    
Production build 0.71.5 2024