Create separate issue for the text format issue
Ah the issue with Anthropic is max tokens being capped at 2048 when returning values that was the reason for the issues.
For documentation:
- We want to test and add Drupal 10 instructions (if different)
- Add steps on creating the migration yml file
Yes using the stable versions:
"drupal/ai_provider_anthropic": "^1.1",
"drupal/ai_provider_openai": "^1.1"Tried Anthropic and it failed on both migrations with time outs but still create one page with the same text format mapping issues.
OpenAI gpt-4.1 did better but noticed that for one post it set the text format to plain_text and the other one to full_html. I presume prompt adjustments will be needed per model.
Going to try anthropic one more time.
Thank you @yce (not a big deal but can I trouble you to update the contribution record). Thank you!
Trying Anthropic and model Claude 3.7 Sonnet got errors curl and overloaded errors. Will try tomorrow and maybe another model.
gpt-4.0 was looking better but generated these errors when trying to migrate the HTML
TypeError: Drupal\ai_migration\Service\AiMigrationCacheBinProvider::setPromptResponse(): Argument #1 ($response) must be of type array, false given, called in /var/www/html/src/AiMigrator.php on line 212 in Drupal\ai_migration\Service\AiMigrationCacheBinProvider->setPromptResponse() (line 52 of /var/www/html/src/Service/AiMigrationCacheBinProvider.php)
#0 /var/www/html/src/AiMigrator.php(212): Drupal\ai_migration\Service\AiMigrationCacheBinProvider->setPromptResponse()
#1 /var/www/html/src/Plugin/migrate_plus/data_parser/Ai.php(116): Drupal\ai_migration\AiMigrator->convert()
Failed to decode cleaned AI JSON response: { "data": { "type": "node--simple_content_migration", "id": "prioritizing-accessibility-bugs-for-maximum-impact", "attributes": { "title": "Prioritizing accessibility bugs for maximum impact", "created": 1604321962, "changed": 1604321962, "field_author": ["Jack Haas"], "field_post_date": { "value": "2025-04-01" }, "field_post_content": { "value": "<p>By Jack Haas, Front End Engineer</p><p><img src='https://example.com/max/1024/1*Ss7GsqaiWJYjA_4fkagEVQ.jpeg' alt='Close-up of a computer screen displaying lines of color-coded programming code.' /></p> <p>Juggling competing priorities, tight deadlines, and endless iterations can make it easy to deprioritize accessibility considerations. In our work, we've seen how neglecting accessibility can lead to frustrating user experiences and other consequences. That's why we decided to <a href='https://github.com/readme/guides/fix-accessibility-bugs'>treat accessibility issues with the same urgency as any other bug.</a></p> <p>Enter the Accessibility Bug Classification Matrix, which we created to help us quickly prioritize issues based on their impact. It provides a clear framework for promptly identifying and addressing the most critical accessibility barriers.</p> <h3 id='features-of-our-bug-classification-matrix'>Features of our bug classification matrix</h3> <ul> <li>Adapted from industry best practices</li> <li>Covers a wide range of Web Content Accessibility Guidelines (WCAG) criteria</li> <li>Prioritizes issues based on user impact severity</li> <li>Provides clear action steps for each priority level</li></ul> <h3 id='overview'>Overview</h3> <p>As our team grows and projects become more complex, we've realized the value of having a centralized process and systemic approach to prioritizing accessibility bugs as they are discovered.</p> <p>Inspired by the impact classifications developed by accessibility experts at Deque, we adapted their framework to fit our workflows and priorities. The result is a helpful tool that helps our teams quickly assess and prioritize accessibility issues based on their true severity and impact on users.</p><p><strong>This matrix isn't just for accessibility experts.</strong> We designed it to be a resource for all team members, regardless of the accessibility knowledge. By providing clear definitions and examples for each priority level, we're empowering everyone on our team to make informed decisions about what needs to happen when a bug is discovered.</p><p>It's important to note that this isn't meant to be a rigid, exhaustive list of every possible accessibility issue. We recognize that the field of accessibility is constantly changing, and new challenges emerge.</p> <h3 id='accessibility-as-a-team-sport'>Accessibility as a team sport</h3> <p>We're fostering a culture of shared responsibility by making this resource available to the whole team. Accessibility shouldn't be limited to a few experts. Whether you're a project manager triaging a backlog of issues, a QA team member doing release testing, or a designer who has noticed something odd when reviewing how their work was implemented, our <a href='https://accessibility.civicactions.com/guide/defect-priority'>Accessibility Bug Classification Matrix</a> provides a common language and clear guidance.</p> <h3 id='assessing-failure-levels-and-issue-priorities'>Assessing failure levels and issue priorities</h3> <p>We've divided the matrix into five levels, each with its own set of criteria and corresponding remediation priority.</p> <h3 id='level-1-page-level-interference-single-a-wcag-criteria'>Level 1: Page-level interference (Single A WCAG criteria)</h3> <p>Think of this as the 'drop everything and fix it now' category. Issues at this level are the most severe, as they fundamentally block users with disabilities from accessing essential content or functionality.</p><p><img src='https://cdn-images-1.medium.com/max/696/0*lQ7GlQN8eGmycoN2' alt='Table of criteria showing page-level interference: the highest level of user impact which should be addressed immediately.' /> See the full table of criteria in the <a href='https://accessibility.civicactions.com/guide/defect-priority'>Accessibility Bug Classification Matrix</a>.</p> <p>When we encounter issues like these,
Looks like for OpenAI sort of worked but the HTML was stripped out for one post and the other post was truncated. Here are the outputs:
AI migration response for url https://accessibility.civicactions.com/posts/delivering-digital-first-turning-21st-century-idea-into-action:
Array
(
[title] => Delivering Digital First: Turning 21st Century IDEA into Action
[created] => 1664534400
[changed] => 1664534400
[langcode] => Array
(
[value] => en-US
[language] => English
)
[status] => 1
[promote] => 1
[sticky] =>
[default_langcode] => 1
[revision_default] =>
[revision_translation_affected] =>
[path] => Array
(
[alias] => /posts
[pid] =>
[langcode] => en
)
[field_author] => Array
(
[0] => Mike Gifford
[1] => Emily Ryan
)
[field_post_content] => Array
(
[value] => What 21st Century IDEA Means to CivicActions Our team has embraced many of the key elements of The 21st Century Integrated Digital Experience Act (IDEA), which became a US law 5 years ago (December 2018). This was a bipartisan act to encourage government agencies to build a framework and requirements for a digital-first public experience, with an emphasis on accessibility, building mobile-friendly user experiences, digitizing services and forms, and improving the overall customer experience. The legislation was designed with specific recommendations for a variety of senior executive agency roles including Chief Information Officers. At a high level, the guidance was designed to push agencies and the overall administration to modernize the federal government. Included in the IDEA act is a handful of strategic goals calling on agencies to: Modernize their websites (which includes being mobile-friendly) Digitize services and forms Transition to standardized and centralized shared services Improve overall customer experience (CX) Additionally, the Office of Management and Budget (OMB) Memo M-23–22, Delivering a Digital-First Public Experience, came out this past fall (September 2023), which expanded on the 21st Century IDEA act. The OMB memo clarified what the 21st Century IDEA act meant by the concept of modernization. The OMB memo affords greater insight into what OMB and the federal government expects agencies to deliver to the public. This OMB Memo gives specific mechanisms by which agencies should be improving their CX. Specifically, agency sites should: Provide a feedback mechanism for users to report satisfaction or dissatisfaction with each web page or piece of web content Test online content with the intended target audience before and after publishing Examine websites and digital services to ensure content is written and implemented so that Limited English Proficiency (LEP) users can meaningfully access those services CivicActions has extensive experience working within these strategic goals found in the IDEA act and the OMB memo, and we have brought this expertise to agencies such as the US Department of Veterans Affairs (VA), Centers for Medicare and Medicaid Services (CMS), the National Science Foundation (NSF), and the Department of Education (ED). The work that we do is accessible, responsive, mobile performant, and frequently exceeds the requirements of Section 508, thereby ensuring the solutions create the best customer experience while adhering to the needs of the public. Modernizing websites We understand that a modern digital experience fully incorporates smartphones. Our team works with our clients to build a good mobile first experience. We know from the US Census data that millions of Americans do not have a desktop computer. Data from Analytics.USA.gov shows that more than half of the traffic to the sites monitored is coming from either iOS or Android users. To ensure we're building the best mobile-first experience, CivicActions works extensively with Drupal and the US Web Design System (USWDS). We maintain the Drupal USWDS Theme, and also implement it for our clients. This allows us to confidently build modern digital experiences for multiple devices, screen sizes and assistive technology, through a proven mobile-first strategy. By leveraging the USWDS federal design system and open source platforms such as Drupal, we improve the customer experience (CX) of our clients' sites. In addition, we bring content strategy, user research, and visual design into our processes to ensure they meet the needs of the public and the agencies we serve. Our practitioners are well-versed in using human-centered design (HCD) approaches and we balance this with addressing the needs of each agency and their goals. Finally, our focus on accessibility allows us to build sites that are future compatible, providing an experience that works across all of the public. Digitizing services and forms There is an ongoing challenge to digitize government services. Agencies are struggling to move beyond legacy PDF forms. At CMS and VA, we have done extensive work on building customized web forms. These forms follow best practices, and provide a user-friendly interface for citizens to provide information to the government. Such approaches also help to ensure information gathering is done in an accessible and usable manner, creating time-saving data collection methods for agencies and reducing mistakes and processing time. Transitioning to standardized and centralized shared services (including open source) CivicActions has extensive experience in open source communities, like the USWDS. We are connected into the USWDS community, and our teams contribute to improving the USWDS framework by actively engaging with the community through collaboration in GitHub and Slack as well as through annual conferences, trainings, newsletters, podcasts, webinars and other opportunities. Where we can, we bring our deep Drupal experience and solutions to our client's challenges back into the USWDS. This is a critical part of seeing that we continue to provide robust digital services in the future. This work with Drupal supports the goal of future transitions to centralized shared services. Building open source solutions with Drupal create structured content which lends itself to standalone and/or federated systems. Content can be published once, and blended into the pages of multiple sites, thus reducing the cost of maintaining repetitive content. Governments around the world are seeing the benefits of centralizing on mature open source technology like Drupal. Working with Drupal also provides us an advantage both in accessibility and mobile support. Drupal has been built to exceed the needs of Section 508 for both the public facing and author facing interfaces. With 25% of the population having a disability, we know that authors also need accessible interfaces. Because of our work in the Drupal community, we are able to engage with global experts on a range of topics. Improve overall customer experience (CX) As mentioned above, we have an experienced team with an understanding of user research, content design and service design work. CivicActions understands the need for user research and testing. Our human- centered and data-driven design team works to align with the USWDS and the specific design systems and style guides of our government clients. CivicActions' content design team is also working to establish plain language text that is easy to understand and more accessible to the average user. We know people come to government websites in order to find information and we work with our clients to build an experience that allows users to quickly find what they need. The OMB's policy guidance (M-23–22) embraces the idea that search is an important piece of a digital first public experience. It is important that government pages are optimized for search engines, but also that they have a meaningful internal search capacity. Our team has experience customizing the search experience in Drupal for our clients. There is a lot of customization available within Drupal, but we have also leveraged Search.gov services. We are working with our clients to find ways to embed dynamic experience for our users in their sites. We understand that government often requires users to complete complex tasks and we continue to utilize best practices that align to the IDEA act and the OMB's memo in order to simplify their workloads. Accessibility and Digital-First Public Experiences Finally, for the first time, per the OMB memo, agencies are directly encouraged to do the following: "Apply the most current Web Content Accessibility Guidelines (WCAG) published by the World Wide Web Consortium (W3C) to websites and web applications, where possible." "Include automated scanning, manual testing of websites, and usability testing with people with disabilities, as well as testing with users of adaptive technologies." "Incorporate the needs of individuals with disabilities into the design and development of websites and digital services, and should include individuals with disabilities in usability testing of new tools or features." The guidance from the OMB re-enforces the need for government agencies to focus more on accessibility. The Spring 2023 Section 508 Program Maturity Report, demonstrated accessibility is not being consistently well managed across government agencies. Citizens with disabilities should be able to expect that they can engage with government agencies. This has been understood as a right for citizens for over 30 years, yet it is still a challenge. The OMB Memo provides very specific guidance on how agencies should be addressing accessibility. Accessibility is at the heart of CivicActions mission; our team is taking a leadership role in Drupal's accessibility, and is engaged in several other open source projects that focus on accessibility as well. Our staff includes people with disabilities, and through our Champions Program and Accessibility Onboarding, we ensure that this value is understood by every team member. Where We Are Going Our team is continuing to innovate in digital government. Our work with the open source communities, like the USWDS & Drupal, allow us to regularly engage with other teams, projects, and agencies. By engaging with others, we learn and contribute to the development of best practices. We can then bring these approaches back to our clients, improving the experience for all. Reach out if you are interested in learning more about our approach — we'd love the opportunity to talk about what we can do together.
[format] => text
)
[field_post_date] => Array
(
[value] => 2024-01-31
[format] => date
)
[field_publishing_information] => Array
(
[value] => Delivering Digital First: Turning 21st Century IDEA into Action was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.
[format] => text
)
[type] => simple_content_migration
)AI migration response for url https://accessibility.civicactions.com/posts/prioritizing-accessibility-bugs-for-maximum-impact:
Array
(
[langcode] => Array
(
[value] => en-US
[language] => English
)
[status] => 1
[title] => Prioritizing accessibility bugs for maximum impact
[created] => 1659302400
[changed] => 1659302400
[promote] => 1
[sticky] =>
[default_langcode] => 1
[field_author] => Array
(
[0] => Jack Haas
)
[field_post_content] => Array
(
[value] => Juggling competing priorities, tight deadlines, and endless iterations can make it easy to deprioritize accessibility considerations...
[format] => text
)
[field_post_date] => Array
(
[value] => 2025-04-01
)
[field_publishing_information] => Array
(
[value] => Prioritizing accessibility bugs for maximum impact was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.
)
[type] => simple_content_migration
)Working on testing this by setting API keys for both OpenAI and Anthropic
To build out the readme file, I am installing Drupal vanilla site and then installing the module. We do require drupal/ai:^1.2@alpha and I tried installing Drupal CMS but that requires 2.x version that is getting complicated to install. So I am going try again with just vanilla Drupal 11 (instead of Drupal CMS) and install the module that way.
Started documentation here https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... → and created pages for
https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →
- links to contributing.md
https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →
https://www.drupal.org/docs/extending-drupal/contributed-modules/contrib... →
Should the how to use it instructions be in the readme and also on the module page? readme would be good.
Looking good @majorrobot. I do like the prompt details in the field descriptions. For the byline should we grab the title if it exists? E.g. https://accessibility.civicactions.com/posts/prioritizing-accessibility-... has “By Jack Haas, Front End Engineer” (but that is for the future)
Thanks @majorrobot, looks interesting.
I get this error when trying to re-enable the module
dmundra in ~/workspace/community/ai_migration on branch 3545204-adjust-easy-migration > ddev drush en -y ai_migration_example
In UnmetDependenciesException.php line 100:
Configuration objects provided by <em class="placeholder">ai_migration_example</em> have unmet dependencies: <em class="placeholder">field.field.node.simple_content_migration.field_author (field.storag
e.node.field_author), field.field.node.simple_content_migration.field_post_content (field.storage.node.field_post_content), field.field.node.simple_content_migration.field_post_date (field.storage.node
.field_post_date), field.field.node.simple_content_migration.field_publishing_information (field.storage.node.field_publishing_information)</em>
Failed to run drush en -y ai_migration_example: exit status 1I did uninstall the module. Ah I believe you are missing the field.storage config files for those new fields.
Looks good @majorrobot approving and merging
dmundra → created an issue. See original summary → .
Thank you @swirt. Sorry for the delay in my sharing. I used https://www.drupal.org/docs/user_guide/en/language-add.html → to setup the second language and then test the changes. I agree that the changes look good and worked for me. Schema issue needs fixing and also tweaking some of the strings to make sure they are going through the t() function so they can be translated (what Steve said)
I can take a shot at reviewing this.
@codebymikey I am not a maintainer of this module but would be interested in being maintainer? I presume the other maintainers would be happy to make you one.
Test needs work so switching status to that after leaving comments.
Thank you @majorrobot, tested following your steps and I did get a JSON schema in the log. Did notice a failing test in the MR.
Ran the following to test it.
- ddev poser
- ddev drush cim
- ddev drush migrate:import slice1_content_with_stubs
- Log in and confirm there are 2 new pieces of content.
- ddev drush migrate:rollback --group=slice1 to roll it back.
Merging the changes and closing issue as fixed.
Approved the code changes
Hi @charlliequadros,
Note that patch did not work for us so here is a new patch just in case someone finds (sorry to update a closed ticket).
Confirmed the update fixes the JS error. Thank you @muriqui
Attaching tugboat screenshots from 🐛 Invalid boolean values in recurring_events_create_form.js Active where it is failing and from here where it is passing.
Failing:
Passing:
Removed the config.yaml file and updated the contributing instructions. The test is to pull down the branch locally (ideally in a fresh repo) and follow the steps in CONTRIBUTING.md to install ddev and get a site running.
Thank you @webbywe. Approved and merged.
I updated the tests and left comments on a few things I noticed.
No worries. Merging.
Looks good @majorrobot, anything specific I should test other then the migration works?
I agree @majorrobot
This MR is available to use for testing. Setting it postponed for now.
I think either add it to ✨ Support tools calling Active or copy the changes from that ticket to ✨ Upgrade to 2.0 version of google-gemini-php Active and then add it there.
I followed https://jaybeaton.com/blog/how-use-version-module-drupal-issue-fork-comp... to install the issue fork https://git.drupalcode.org/issue/gemini_provider-3519469 code as a package to the composer.json so it can be installed locally for testing.
Adding tests/ai-test-drush-scripts/GeminiFunctionCallTester.php to test function calls.
QA steps:
- Checkout the branch from this issue's MR.
- Run ddev like normal
- Run ddev poser to install different version of the gemini_provider module and the dependencies google-gemini-php/client and drupal/ai_agents
- Configure Gemini AI following contributing steps (if haven't done before)
- Run
ddev drush php-script tests/ai-test-drush-scripts/GeminiFunctionCallTester.php - Confirm it outpus 140. Change the prompt in the test file and run it again to confirm the calculator function works.
Looking good @majorrobot. Approved and merged.
We mob-programmed on this PR so credited all the folks. Thank you!
For the HTML and other content type issues that can be handled in future issues like <#3539369>
@majorrobot migration working well. Noticed that the example content-type didn't display fields on node view or edit so after enabling those I can see the values. I will share comparing AI response with what is being migrated.
No worries @majorrobot. Reviewing now
Ran into some bugs trying to enable the example module after re-doing my local site.
This solution https://www.drupal.org/project/ai/issues/3528549 ✨ Allow tool calling in streamed chat Active might make this redundant so something to watch as well.
I submitted a commit that captures the output of the function call if it is an instance of ExecutableFunctionCallInterface. Attaching new patch and inter diff.
Thank you @swirt. Looks good.
I have this script that I am testing with drush and it calls the function but doesn't generate an output. I left a comment on the MR
<?php
use Drupal\ai\OperationType\Chat\ChatInput;
use Drupal\ai\OperationType\Chat\ChatMessage;
use Drupal\ai\OperationType\Chat\Tools\ToolsInput;
$messages = new ChatInput([
new ChatMessage('user', 'What is 20 * (4 + 3)?'),
]);
$provider = \Drupal::service('ai.provider')->createInstance('gemini');
// Set a function call.
$functions = [];
$function_instances = [];
$function_call = \Drupal::service("plugin.manager.ai.function_calls")->createInstance('ai:calculator');
$function_instances[$function_call->getFunctionName()] = $function_call;
$functions[] = $function_call->normalize();
// Set chat tool.
$messages->setChatTools(new ToolsInput($functions));
/** @var \Drupal\ai\OperationType\Chat\ChatOutput $response */
$response = $provider->chat($messages, 'gemini-2.5-flash', ['my-custom-call']);
/** @var \Drupal\ai\OperationType\Chat\ChatMessage $return_message */
$return_message = $response->getNormalized();
echo $return_message->getText() . "\n";
// Returns something like "140"Actually the signature has not changed between version 1 and 2, here is version 1 line https://github.com/google-gemini-php/client/blob/1.0.15/src/Data/Generat... (still an ?int)
An update, after enabling AI agents module and seeing that I can reference other tools like 'Calculator', I tried /admin/config/ai/explorers/chat_generator once again with xdebug and I can see that the code from the MR is calling the Calculator code it is just not outputting it to the screen as tool output. Maybe that is missing?
Trying to test this with /admin/config/ai/explorers/chat_generator and the existing HtmlToMarkdown function. I ran into an error with the generator not supporting the 'Top K' being a float. Here is the error
TypeError: Gemini\Data\GenerationConfig::__construct(): Argument #6 ($topK) must be of type ?int, float given, called in /var/www/html/web/modules/custom/gemini_provider/src/Plugin/AiProvider/GeminiProvider.php on line 260 in Gemini\Data\GenerationConfig->__construct() (line 39 of /var/www/html/vendor/google-gemini-php/client/src/Data/GenerationConfig.php)
Switching that argument type, I was able to generate code but not execute the function. I will continue to try to test but if someone has a better approach please do share.
Attaching a patch file for the current MR to test with.
Thank you @jibla for linking the issue. I can try to test that.
Content-type looks good. Thank you majorrobot
Taking a look.
Looking good @majorrobot. Thank you for the details and it makes sense.
Looking good @majorrobot and thank you for the easy to follow documentation. Left a few comments and here my tests:
Working URL:
dmundra in ~/workspace/community/ai_migration on branch 3537651-confirm-minimal-ai > ddev drush php-eval "print_r(\Drupal::service('ai_migration.ai_migrator')->convert('https://ndhurandhar.art/art/2025-06/artist-impression-rex'));"
```json
{
"title": "Artist impression of Rex"
}
```%
Non-existent URL:
dmundra in ~/workspace/community/ai_migration on branch 3537651-confirm-minimal-ai > ddev drush php-eval "print_r(\Drupal::service('ai_migration.ai_migrator')->convert('https://ndhurandhar.com'));"
[error] Failed to fetch content from URL https://ndhurandhar.com: cURL error 6: Could not resolve host: ndhurandhar.com (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://ndhurandhar.com
Thank you @majorrobot. Skeleton looks good and straightforward right now. Tests are passing. I left a comment about the convert function test.
Thank you @majorrobot. Approved and merged.
Thank you @kducharm. Merging.
Ready for review.
Ready for review.