Do upgrade testing 1.1.0 with possible combinations

Created on 3 June 2025, 3 days ago

Problem/Motivation

Currently all upgrade testing has been with either Drupal Standard installation with all the modules installed or Drupal CMS with the agents installed and it has assumed that you update AI, AI Agents and the provider to 1.1.x branch. This has been working, however there are combinations that has not yet been tested that are possible.

  • AI 1.1.0, AI Agents 1.0.2 and AI provider 1.0.x is possible, if you specifically target to update just AI module.
  • AI 1.1.0, AI Agents 1.1.0 and AI provider 1.0.x is even probable for any provider being used that hansn't updated to function calling.

VDB providers we can assume 1.1.x should be added since its a breaking change. We should test these upgrade paths so:

  1. At least they do not break your website.
  2. Better - everything that isn't the assistant, agents, tool explorer or chat explorer with tools works well.
  3. Event better - you have graceful fallback on assistants and agents that already exists.
  4. Best - you have information that you should update (<- this would require updated code)

How to test

Step #1:

  1. Install Drupal Standard installation.
  2. Fetch AI 1.0.5, AI Agents 1.0.2, OpenAI Provider 1.0.2 (Its the most used provider), Milvus Provider 1.0.2-beta2,
  3. Install AI Core, AI API Explorer, AI Logging, AI Automators, AI CKEditor integration, AI Content Suggestions, AI External Moderation, AI Translate, AI Search, OpenAI Provider, AI Agents, AI Assistant API, AI Chatbot, Milvus VDB Provider. We skip the AI Validations module, since we haven't done any work and since the Field Validation module will break on 11.2.
  4. Setup OpenAI
  5. Setup Milvus
  6. Visit filter formats (/admin/config/content/formats) and add the AI CKEditor button to the full html. Setup Summarize.
  7. Visit AI logging settings (/admin/config/ai/logging/settings) and enable Automatically Log Requests and Automatically Log Responses.
  8. Visit AI Content Suggestions Settings (/admin/config/ai/suggestions) and enabled Summarise text (we should be consistent with summarise/summarize here :) )
  9. Visit AI Assistant configuration (/admin/config/ai/ai-assistant) and setup an assistant using the content type agent - you can give the simple instruction "Forward all calls to the Content Type Agent"
  10. Setup a Chatbot, make sure its not streaming, use the assistant.
  11. Visit Search APi (/admin/config/search/search-api) and setup an server and index pointing at your Milvus and using Articles.
  12. Create a new content type called "Test Automators", where you add a String Long text and enable LLM Text Simple and base it on the title and ask it to write a two sentence story.
  13. Visit Automator chain settings (/admin/config/ai/automator_chain_types) and add an chain called "Story teller" and add a text field and a long text and make the long text write a story of the text field. Set the text field to required.
  14. Visit filter formats (/admin/config/content/formats) again and connect this automator chain where the text field is the input and story field is the output.
  15. Add 10 articles - when creating them try the CKEditor assistants and the Content Suggestions.
  16. Run Search API Indexing if needed.
  17. Setup a view using the index.
  18. Try searching.
  19. Try to create using the Test Automators by just filling in the title.
  20. Try asking the Chatbot "What content type exists on the system?"
  21. Take a database dump
  22. Add a language
  23. Try to translate an article using AI translate

Test #1

  1. Upgrade the AI module to 1.1.x-dev
  2. Run drush updb or the update.php
  3. Flush caches
  4. Try to add an article with the CKEditor assistants tools and Content Suggestions.
  5. Try to translate an article using AI translate
  6. Try to index it.
  7. Check the view if its there and search for it.
  8. Try Test Automators
  9. Try asking the Chatbot "What content type exists on the system?
  10. Visit /admin/config/ai/automator_chain_types
  11. Visit /admin/config/ai/automator_chain_types/add
  12. Visit /admin/config/ai/ai-automators/automators-tool
  13. Visit /admin/config/ai/ai-automators/automators_tool/add
  14. Visit /admin/config/ai/ai-assistant/add
  15. Visit edit page of the assistant you setup
  16. Visit /admin/config/content/formats/manage/full_html
  17. Visit /admin/config/ai/agents
  18. Check /admin/reports/status for status errors
  19. Check /admin/reports/dblog for errors that might be connected to the upgrade

Test #2

  1. Upgrade the AI Agents module to 1.1.x-dev
  2. Run drush updb or the update.php
  3. Flush caches
  4. Try to add an article with the CKEditor assistants tools and Content Suggestions.
  5. Try to translate an article using AI translate
  6. Try to index it.
  7. Check the view if its there and search for it.
  8. Try Test Automators
  9. Try asking the Chatbot "What content type exists on the system?
  10. Visit /admin/config/ai/automator_chain_types
  11. Visit /admin/config/ai/automator_chain_types/add
  12. Visit /admin/config/ai/ai-automators/automators-tool
  13. Visit /admin/config/ai/ai-automators/automators_tool/add
  14. Visit /admin/config/ai/ai-assistant/add
  15. Visit edit page of the assistant you setup
  16. Visit /admin/config/content/formats/manage/full_html
  17. Visit /admin/config/ai/agents
  18. Check /admin/reports/status for status errors
  19. Check /admin/reports/dblog for errors that might be connected to the upgrade

These tests can later be created into recipes and kernel tests.

📌 Task
Status

Active

Version

1.1

Component

AI Core module

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @marcus_johansson
  • 🇩🇪Germany marcus_johansson

    I will set it as release blocker, however the "normal" upgrade path is tested and it is the most important. We will have RC before stable as well.

  • 🇮🇳India annmarysruthy

    Tried test setup. Assuming the Milvus Provider in step 2 is 1.0.0-beta2. Completed till step 16, On adding a new view with the index, got an error

    The submitted value search_api_data in the output as element is not allowed.
    The submitted value search_api_data in the Feed row style element is not allowed.

    This might be related to core issue https://www.drupal.org/project/facets/issues/3446040 🐛 The submitted value [value] in the [name] element is not allowed. Active

  • 🇮🇳India annmarysruthy

    After upgrading the AI module to 1.1.x-dev, Faced a fatal error:

    Fatal error: Declaration of Drupal\ai_vdb_provider_milvus\Plugin\VdbProvider\MilvusProvider::vectorSearch(string $collection_name, array $vector_input, array $output_fields, mixed $filters = '', int $limit = 10, int $offset = 0, string $database = 'default'): array must be compatible with Drupal\ai\AiVdbProviderInterface::vectorSearch(string $collection_name, array $vector_input, array $output_fields, Drupal\search_api\Query\QueryInterface $query, string $filters = '', int $limit = 10, int $offset = 0, string $database = 'default'): array in /var/www/html/modules/contrib/ai_vdb_provider_milvus/src/Plugin/VdbProvider/MilvusProvider.php on line 597

    This is mentioned in https://www.drupal.org/project/ai/issues/3515861 🐛 MilvusProvider.php on line 597 Active .

    @marcus_johansson Kindly let me know the milvus db provider version to be used for testing. in test setup it is mentioned "Milvus Provider 1.0.2-beta2" . But there is no version 1.0.2-beta2 available. Currently I have used "1.0.0-beta2"

  • 🇩🇪Germany marcus_johansson

    Oh, sorry - the update of any VDB provider has to be 1.1.x. This should be automatic on release, see: https://www.drupal.org/project/ai_vdb_provider_milvus/issues/3528067 📌 Pin version 1.0.0 to AI 1.0.x Active

  • 🇮🇳India annmarysruthy

    In Test #1:

    4. Try to add an article with the CKEditor assistants tools and Content Suggestions.

    Result: Unable to get response from ai when using CKEditor assistants "Story teller" created in test setup step 13. Story teller was working correctly before update. However summarize plugin from ckeditor module and summarize in content suggestions works perfectly. error in story teller automator chain:
    InvalidArgumentException: Field selected_text is unknown. in Drupal\Core\Entity\ContentEntityBase->getTranslatedField() (line 617 of /var/www/html/core/lib/Drupal/Core/Entity/ContentEntityBase.php).

    5. successfully translated article node with ai translate.
    6. On creating any new content/translation, the node is getting indexed.
    7. was unable to create view in test setup. So Unable to test step 7.
    8. Successfully created test automators by filling in only title field and long string generated from AI.
    9. Response from chatbot:

    The content types available on this Drupal instance are:
    
        Article
        Basic page
        Test Automators
    

    10,11,12,13: Visited the paths and no error observed
    14, 15: Unable to access the path. Fatal error:

    The website encountered an unexpected error. Try again later.
    
    Drupal\Component\Plugin\Exception\PluginNotFoundException: The "ai_agent" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 139 of core/lib/Drupal/Core/Entity/EntityTypeManager.php). 

    16,17: Visited the paths and no error observed
    18, 19: No errors observed

  • 🇩🇪Germany marcus_johansson

    Second follow up issue 🐛 AI Automators CKEditor plugin is broken Active

  • 🇮🇳India annmarysruthy

    In test #2: observed results are similar to that observed in test #1. Completed testing.

    4. Unable to get response from ai when using CKEditor assistants "Story teller" created in test setup step 13. Already reported in the above comment https://www.drupal.org/project/ai/issues/3528446 🐛 AI Automators CKEditor plugin is broken Active and fixed.

    summarize plugin from ckeditor module and summarize in content suggestions works perfectly.

    5. successfully translated article node with ai translate.
    6. On creating any new content/translation, the node is getting indexed.
    7. was unable to create view in test setup. So Unable to test step 7.
    8. Successfully created test automators by filling in only title field and long string generated from AI.
    9. Obtained correct response from chatbot.
    10,11,12,13: Visited the paths and no error observed
    14,15,16,17: Visited the paths and no errors observed.
    18, 19: No errors observed

  • 🇮🇳India anjaliprasannan

    Retested and found no issues other than those reported above.

Production build 0.71.5 2024