TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map()

Created on 5 November 2023, about 1 year ago
Updated 20 February 2024, 9 months ago

Problem/Motivation

After an apache restart, I'm suddenly unable to edit fields and the following PHP error is in the log:

TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 46 of /var/www/***/vendor/openai-php/client/src/Responses/Models/RetrieveResponse.php).

Also, a warning:

Warning: Undefined array key "permission" in OpenAI\Responses\Models\RetrieveResponse::from() (line 46 of /var/www/***/vendor/openai-php/client/src/Responses/Models/RetrieveResponse.php)

It seems like a 'permission' attribute is expected by the 'from' function in the openai library but the module does not send it. I'm not sure if that was introduced in an update to the library and the conflict didn't appear until I restarted my web server, or if it's something else. It all worked fine on Friday :|

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Fixed

Version

1.0

Component

Code

Created by

🇹🇭Thailand AlfTheCat

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Issue created by @AlfTheCat
  • Assigned to chetan 11
  • Issue was unassigned.
  • First commit to issue fork.
  • Assigned to marcus_johansson
  • 🇩🇪Germany marcus_johansson

    Thanks for the report, could you check in your composer.lock file which versions you have of openai-php/client and drupal/openai and I will see if I can replicate it. It shouldn't be connected to your server restart I think, unless you use something like DDEV or Docksal and have a composer update inside the restart.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, thanks for the reply.

    I'm using v0.5.3 of the openai-php library.

    If it's not the library, then I think it has something to do with the queueing mechanism. Here are my steps that probably led to the issue:

    • Yesterday I configured a field to interpolate via a queue. I want to run it via drush so I installed https://www.drupal.org/project/mob_queue
    • I disabled the queue in my cron settings, as it should only run via drush. I updated a field so there was something in the queue.
    • I selected the interpolator queue to run via drush and ran the mob queue command. Nothing got executed so I figured it didn't work.
    • I re-configured cron and using queueu_ui, I batch executed the queue and it processed the interpolation tasks.

    I did some research earlier today and I found a fixed drupal issue that unfortunately now I can't find anymore. In that issue a module was trying to queue something or there was an empty item in the queue (don't remember exactly), and it mentioned that in PHP 8.1 this resulted in an empty value being passed into an array (queue).

    Hope this helps :)

  • 🇩🇪Germany marcus_johansson

    So I can't really figure out why this happened, but I have a solutionish to it. There is no signs of deprecation here (https://platform.openai.com/docs/deprecations), but the OpenAI client from the OpenAI module was outdated.

    I have pushed and update to force over 1.0.0-alpha15 from the OpenAI module, instead of general Alpha, which will update the client library.

    I have pushed a the AI Interpolator OpenAI module to alpha5 and you would have to update to this.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, awesome, I will update and report back.

  • 🇩🇪Germany marcus_johansson

    Seems like this is not the only instance of this missing at least: https://community.openai.com/t/missing-permission-inside-model-response/...

    Weird that they don't write deprecation messages when they change the API.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, updating the module did not resolve the issue in my case. Still get a crash when I try to edit any field that uses openai via Interpolator.
    Is there anything in addition to updating the module that I should do?

  • 🇩🇰Denmark lennart

    I have the same issue - I am currently using ai_interpolator_openai (1.0.0-alpha4)

  • 🇩🇪Germany marcus_johansson

    hmm, I'm not sure why its not updating to a later version - the latest is 1.0.0-alpha7, but the fix should be on 1.0.0-alpha5.

    Could you specify that specific version and see if it updates, so

    composer require 'drupal/ai_interpolator_openai:^1.0.0-alpha7'

    and the flush caches

  • 🇹🇭Thailand AlfTheCat

    I forced an update of the openai module and I also upgraded specifically to the 1.0.0-alpha7 module of the Interpolator openai submodule.
    Flushed caches but still no luck.

    Below is my terminal input/output:

    composer require 'drupal/openai:^1.0@alpha' --with-all-dependencies
    ./composer.json has been updated
    Running composer update drupal/openai --with-all-dependencies
    Gathering patches for root package.
    Loading composer repositories with package information
    Updating dependencies
    Lock file operations: 0 installs, 8 updates, 0 removals
    - Upgrading drupal/openai (1.0.0-alpha14 => 1.0.0-alpha15)
    - Downgrading openai-php/client (v0.5.3 => v0.4.2)
    - Downgrading psr/http-message (2.0 => 1.1)
    - Upgrading symfony/http-foundation (v6.3.6 => v6.3.7)
    - Upgrading symfony/http-kernel (v6.3.6 => v6.3.7)
    - Upgrading symfony/serializer (v6.3.6 => v6.3.7)
    - Upgrading symfony/validator (v6.3.6 => v6.3.7)
    - Upgrading symfony/yaml (v6.3.3 => v6.3.7)
    Writing lock file
    Installing dependencies from lock file (including require-dev)
    Package operations: 0 installs, 8 updates, 0 removals
    - Downloading symfony/yaml (v6.3.7)
    - Downloading symfony/validator (v6.3.7)
    - Downloading symfony/serializer (v6.3.7)
    - Downloading symfony/http-foundation (v6.3.7)
    - Downloading psr/http-message (1.1)
    - Downloading symfony/http-kernel (v6.3.7)
    - Downloading openai-php/client (v0.4.2)
    - Downloading drupal/openai (1.0.0-alpha15)
    Gathering patches for root package.
    Gathering patches for dependencies. This might take a minute.
    - Upgrading symfony/yaml (v6.3.3 => v6.3.7): Extracting archive
    - Upgrading symfony/validator (v6.3.6 => v6.3.7): Extracting archive
    - Upgrading symfony/serializer (v6.3.6 => v6.3.7): Extracting archive
    - Upgrading symfony/http-foundation (v6.3.6 => v6.3.7): Extracting archive
    - Downgrading psr/http-message (2.0 => 1.1): Extracting archive
    - Upgrading symfony/http-kernel (v6.3.6 => v6.3.7): Extracting archive
    - Downgrading openai-php/client (v0.5.3 => v0.4.2): Extracting archive
    - Upgrading drupal/openai (1.0.0-alpha14 => 1.0.0-alpha15): Extracting archive
    Cleaning: symfony/yaml
    Cleaning: symfony/validator
    Cleaning: symfony/serializer
    Cleaning: symfony/http-foundation
    Cleaning: symfony/http-kernel

    and

    composer require drupal/ai_interpolator_openai:^1.0.0-alpha7
    ./composer.json has been updated
    Running composer update drupal/ai_interpolator_openai
    Gathering patches for root package.
    Loading composer repositories with package information
    Updating dependencies
    Lock file operations: 0 installs, 1 update, 0 removals
    - Upgrading drupal/ai_interpolator_openai (1.0.0-alpha3 => 1.0.0-alpha7)
    Writing lock file
    Installing dependencies from lock file (including require-dev)
    Package operations: 0 installs, 1 update, 0 removals
    - Downloading drupal/ai_interpolator_openai (1.0.0-alpha7)
    Gathering patches for root package.
    Gathering patches for dependencies. This might take a minute.
    - Upgrading drupal/ai_interpolator_openai (1.0.0-alpha3 => 1.0.0-alpha7): Extracting archive

    Hope this helps :)

  • 🇩🇪Germany marcus_johansson

    @AlfTheCat It looks like the OpenAI module can allow two complete different minor versions of the official PHP package.

    I have released version alpha8, try if that one works. If not I will change the module so there is no dependency on the OpenAI module.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, thanks for the new release.

    Unfortunately, composer won't let me update because of the library version required by the openai module, it seems.

    I think

    If not I will change the module so there is no dependency on the OpenAI module.

    is maybe a better approach in general, also for those that don't need the features the openai module introduces.

    Here is the terminal output:

    composer require drupal/ai_interpolator_openai:^1.0.0-alpha8 --with-all-dependencies
    ./composer.json has been updated
    Running composer update drupal/ai_interpolator_openai --with-all-dependencies
    Gathering patches for root package.
    Loading composer repositories with package information
    Updating dependencies
    Your requirements could not be resolved to an installable set of packages.

    Problem 1
    - Root composer.json requires drupal/ai_interpolator_openai ^1.0.0-alpha8 -> satisfiable by drupal/ai_interpolator_openai[1.0.0-alpha8, 1.0.x-dev].
    - drupal/ai_interpolator_openai[1.0.0-alpha8, ..., 1.0.x-dev] require openai -php/client ^0.7.8 -> found openai-php/client[v0.7.8] but these were not loaded, likely because it conflicts with another require.

    Installation failed, reverting ./composer.json and ./composer.lock to their original content.

  • 🇩🇪Germany marcus_johansson

    Could you check if you have some other project that requires the opeai-php/client in your composer.lock file or by the output of composer depends openai-php/client

    The OpenAI module should currently not stop the latest version from loading, so you might have something else installed which required an older version.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, thnaks for that, indeed another module requires the openai library.

    drupal/tmgmt_openai 1.0.x-dev requires openai-php/client (^0.4|^v0.5.2)

    I've created an issue in the queue of that module requesting an upgrade: 📌 Upgrade openai client to version to v0.7.8 Active .

  • 🇩🇰Denmark lennart

    for me upgrading like so:

    drupal/ai_interpolator_openai (1.0.0-alpha4 => 1.0.0-alpha9)

    fixed this problem

  • 🇹🇭Thailand AlfTheCat

    I've manually edited my composer files, an upgraded to alpha10 of the interpoloator_openai module. I'm still getting the error, though. Can't edit any field that uses the openai interpolator.

    I also can't add any fields to the taxonomy term that has openai interpolated fields present, that action too results in "The website encountered an unexpected error. Please try again later." in the front-end and in the log:

    TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 46 of /var/www/xxx/vendor/openai-php/client/src/Responses/Models/RetrieveResponse.php).

    Again, in my case, this started happening after I tried switching from direct interpolation to queue-based interpolation, and then switched back. Maybe something got stuck there and is causing this.

  • 🇩🇪Germany marcus_johansson

    Could you again check which version of openai-php/client is in your composer.lock file or check the output of composer depends openai-php/client?

    If that is correct, if possible, could you give me a list of all projects that has dependency to openai-php/client and I will try to replicate the issue.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, thank you.

    Composer.lock file has 0.7.8

    "name": "openai-php/client",
    "version": "v0.7.8",

    and

    composer depends openai-php/client
    drupal/ai_interpolator_openai 1.0.0-alpha10 requires openai-php/client (^0.7.8)
    drupal/openai 1.0.0-alpha15 requires openai-php/client (^0.4|^v0.7.4)
    drupal/tmgmt_openai 1.0.x-devrequires openai-php/client (^0.4|^v0.5.2)

    I have the prompt module installed, other than tmgmt_openai and prompt, no other modules are enabled that use the openai library.

  • 🇩🇪Germany marcus_johansson

    Thanks for the list - could you write how you could force the file to 0.7.8? I can't install tmgmt_openai at the same time as the other two modules - it should not really be possible. Even with "--with-all-dependencies", it should not be able to install it. I have tried and if I install tmgmt_openai, I can't install ai_interpolator_openai and the other way around.

    Just to verify that you have 0.7.8 installed, could you check if you have this file:
    vendor/openai-php/client/src/Testing/Responses/Fixtures/FineTuning/ListJobEventsResponseFixture.php

    Sorry, for all the requests, but it's hard to fix when I can't replicate it.

  • 🇹🇭Thailand AlfTheCat

    Hi Marcus,

    No problem at all, regarding the requests. I very much appreciate the help and am equally eager to help get this fixed :)

    I don't have the file you mentioned. I tried the upgrade by manually editing my composer.lock file because the tmgmt_openai module prevented the update. It did seem to update when I did that, I was able to upgrade the interpolator_openai module and the composer output seemed to report a success. But maybe it didn't work as well after all.

    Is there another way I can get around the issue of composer blocking the update? I need the tmgmt_openai module but I'm happy to hack it a bit to bypass this roadblock, especially if it gives me back control of my fields :). Or somehow escape from composer's grip on the process...

  • 🇩🇪Germany marcus_johansson

    Thanks for the checking, that means that the problem is basically tmgmt_openai not being possible to combine with AI Interpolator. Because I'm trying to keep up with the latest changes I want to be on the latest clients for now until there is a stable version of the AI Interpolator - it has to move as fast as the space moves, so stuff like GPT vision and agents can be added. So, a fix to work with an older version of the library will not happen, since it would remove features.

    Because the issue is in their composer file, and not the code a patch wouldn't help you in this case, since it applies after loading dependencies.

    There are some solutions I could think about, depending on if you want quick results or good maintainability:

    1. You fork tmgmt_openai on a github repo and change the composer file in the repo and change your composer file to use this.
    2. You download the module locally, not using composer and add all dependencies of it manually. (Really not recommended, unless you just want to try something)
    3. You ask if you can become maintainer and help @minnur with the project. Or maybe at least send a patch.
    4. You ask the maintainer like you have now in your ticket and wait. A suggestion is that any module doing OpenAI stuff should depend on the OpenAI module , so they only need to keep one module up to date and store the api keys at one place (that's the reason for AI Interpolator to require it).

    But from my side, there is unfortunately nothing I can do to fix this ticket, except for creating a older version of the module that lacks features, which I won't.

  • Status changed to Needs review 10 months ago
  • 🇩🇪Germany marcus_johansson

    Any update here @AlfTheCat?

  • Status changed to Fixed 9 months ago
  • 🇹🇭Thailand AlfTheCat

    Hi Marcus, yes, I installed ai_interpolator on a fresh project without using tmgmt_openai and there it works without issues.

    There is a patch available now in the tmgmt_openai queue. I think we can mark this as fixed :)

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024