Provide development environment for AI development using DDEV

Additional information: At 1xINTERNET we use an bash script to setup a local development environment. For reference I added the docker compose file for Milvus vector database → and the relevant parts of the installation script.

# Add RAG support.
if [ "${rag}" == "true" ]; then
  echo "Enable Milvus vector database"
  mv .ddev/example.docker-compose.milvus.yaml .ddev/docker-compose.milvus.yaml
  ddev restart
fi

I uploaded the same for pgvector, it needs the following command to be run inside the web container to work:

PGPASSWORD=db psql -U db -h localhost -p 5432 -d db -c "CREATE EXTENSION IF NOT EXISTS vector;"

If its for people contributing, the drupal/core-dev should be installed by default and we should also create ddev commands that people that do not know phpunit well, can run tests. So you can run "ddev test-ai-module" or something similar.

I would argue that we could also install git hooks for commit, that runs against our grumphp. I'm not sure if that is too "serious"? It would be good though, that stuff like linting gets cought before pushing. Its annoying that I have to merge and fix that a lot of the time.

We should add so we have one container that sets up mkdocs by default (https://github.com/Metadrop/ddev-mkdocs maybe?), so people can change and watch the documentation.

All the things I write here does not need to be in version 1.0.0 of the dev environment, but they would all be good things to have.

Where should the repo be hosted for this or is the idea that its part of the AI module?

sets up mkdocs by default

Any reason for that? if you have LLM it knows more then generated docs (which require regular updates and will bring more collisions to MR)

Any reason for that? if you have LLM it knows more then generated docs (which require regular updates and will bring more collisions to MR)

We already have the documentation in the repo, so nothing changes there in terms of MRs.

It's just to make it easy to contribute code without having to learn how to install and start mkdocs on different operating systems. We can then just say - visit https://mkdocs.ai-playground.ddev.site and any changes to your markdown documentation will autoupdate there on save.

One thing we could also think about is having Selenium + VNC server on a web url like https://test-viewer.ai-playground.ddev.site, where its possible to watch them play out. Meaning you will see the test start the website, click around and what happens.

We also have FFMpeg recording possibilities on the Gitlab test suite, so we could add it here as well, though its probably overkill.

Note: ai-playground is just a placeholder.

This is just a raw dump of some of the commands I used to create my Drupal AI Ddev sandbox.

- DDEV
% mkdir dai-sandbox && cd dai-sandbox
% ddev config --project-type=drupal11 --docroot=web
% ddev composer create “drupal/recommended-project:^11.0.0” --no-install
% ddev composer require drush/drush:^13.6
% ddev composer install
% mkdir -p config/sync
% cp web/sites/example.settings.local.php web/sites/default/settings.local.php
- configure settings.php: config_sync_directory, hash_salt, trusted_host_patterns
% ddev start
% ddev drush site:install --locale=en --account-name=admin --account-pass=admin -y
% git init
- update .gitignore as required e.g. *.sql, *.zip, *.mysql.gz, etc.
- add useful modules; customize for your requirements
% ddev composer require drupal/admin_toolbar drupal/pathauto drupal/redirect drupal/token drupal/ctools
- add ai related modules; customize for your requirements
% ddev composer require drupal/ai:1.1.x-dev@dev drupal/ai_agents:1.1.x-dev@dev drupal/ai_provider_anthropic:1.1.x-dev@dev drupal/ai_provider_openai:1.1.x-dev@dev
- to test patches in composer.json
% ddev composer require cweagans/composer-patches

- DDEV EXTRAS:
- Adminer (like PhpMyAdmin but smaller/nicer)
% ddev get ddev/ddev-adminer
- Mailpit: captures outgoing emails; it’s bundled in Ddev, you can open it with this command;
% ddev mailpit

- PHPUNIT (setup)
% ddev composer require --dev drupal/core-dev
% ddev composer update --with-all-dependencies
% cp web/core/phpunit.xml.dist web/core/phpunit.xml
- configure as follows

<env name=“SIMPLETEST_BASE_URL” value=“https://dai-sandbox.ddev.site”/>
	<env name=“SIMPLETEST_DB” value=“mysql://db:db@db/db”/>
	<parameter name=“outputDirectory” value=“sites/simpletest/browser_output”/>

- then you should be able to run tests, for example:
% ddev exec “cd web/core && ../../vendor/bin/phpunit tests/Drupal/Tests/Core/Routing/UrlGeneratorTest.php”
% ddev exec “cd web/core && ../../vendor/bin/phpunit ../modules/contrib/pathauto/tests/src/Unit/VerboseMessengerTest.php”

- PHP_CodeSniffer / phpcs (setup)
% ddev composer require --dev drupal/coder
- register the Drupal coding standards:
% ddev exec vendor/bin/phpcs --config-set installed_paths vendor/drupal/coder/coder_sniffer
- check what standards are available (should include “Drupal”, “DrupalPractice”)
% ddev exec vendor/bin/phpcs -i
- check a specific file or directory:
% ddev exec vendor/bin/phpcs --standard=Drupal web/modules/custom/your_module
% ddev exec vendor/bin/phpcs --standard=Drupal,DrupalPractice web/modules/custom/your_module
- add a phpcs.xml file in your project root

<?xml version=“1.0"?>
	<ruleset name=“Custom Drupal Standards”>
	  <description>Custom coding standards for this project</description>
	  <rule ref=“Drupal”/>
	  <rule ref=“DrupalPractice”/>
	 
	  <!-- Paths to check -->
	  <file>web/modules/custom</file>
	 
	  <!-- Exclude certain files -->
	  <exclude-pattern>*.min.js</exclude-pattern>
	  <exclude-pattern>*.min.css</exclude-pattern>
	</ruleset>

- then you can run this command
% ddev exec vendor/bin/phpcs

This will need its own place to live, since it’s not a module. Should it be a separate project? I’d volunteer to create a repo for it in the Drupal Forge organization on GitHub, but first I need to hear what everyone else thinks.

@darren oh

On Drupal.org you can add general projects. Here is the documentation for general projects → .

These can contain arbitrary code as for example DDEV installation scripts.

They are also used for recipes (for example this one → which we contributed earlier this year).

Amazee only has chat + embeddings. However, we can work with that to start with and have requirements of adding other providers for other task.

What other capabilities would you like to see for the amazee AI provider module? Are you looking for image or video generation? We can add additional models.

@matthews - in the end we need providers listed for the different model types on /admin/config/ai/settings (Image Classification, Image to Video, Speech to Text, etc.).

@metthews & @breidert - we have some of them being very niche and I do not think those are needed. Speech-to-Speech being only Elevenlabs, Audio-to-Audio being only Auphonic. In both those cases the underlying services do not allow for shared accounts, so there is no way Amazee can provide this without breaking the terms and conditions. Also these operation types are niche and the usages for them are for very specific products.

I think the following are general operation types that are generally useful:

• Chat (with tools and vision) - already available in Amazee
• Embeddings - already available in Amazee
• Moderation
• Text-To-Image
• Text-To-Speech
• Speech-To-Text

With all these, the more languages they handle, the better. OpenAI is the golden standard handling around 58 languages - but having the major Drupal languages markets would be good I assume?

I think another thing we need to add is a document loader - I have attached an Unstructured docker composer file that works with ddev, but we could also use Roberto's extension here: https://github.com/robertoperuzzo/ddev-unstructured

This might need to be discussed, because ✨ Create Document Loader Normalization Layer Active is still not done, these are at the moment not interchangeable, like the vector database or AI provider is. Meaning you create specific recipes/solutions that rely on a specific service.

I would however argue that Unstructured is the best, because:

1. It is open source and can be self hosted.
2. It handles almost any file format - see: https://docs.unstructured.io/ui/supported-file-types
3. Its model is best in class at understanding how PDF's are structured and showing it in the right order in markdown.
4. The Drupal solution is pretty extensive and have more maintainers then me: https://www.drupal.org/project/unstructured →
5. Unstructured offers SaaS as well, which only requires you to add a api key to use. So people using simple hosters can still use them - just not in a privacy compliant manner.
6. Any solution will require server applications/changes, unless we solve each of these file formats in PHP - something that will require maintainers we don't have right now.

Here are other future proofing applications that should be added to webimage_extra_packages:

• ffmpeg
• chromium

oh, I think its already there - but python3 and pip3 we should have.

I think that https://github.com/ddev/ddev-drupal-contrib can be a good starting point for this issue.
Then we can add all required ddev addons and initial packages.
Finally, we need a way to install Drupal and install the required recipes

I can work on it :-)

I do run nginx with "fastcgi_buffering off;" to make streaming work in all cases. I'm not sure how important that is for a development environment though?

@lussoluca - just so its confirmed here as well, that's a great starting point!

I've prepared a quick recording of a first draft implementation to check if I'm in the right direction.

I'm using the drupal_cms_ai recipe, but maybe it's too much; should we create an ad hoc recipe for this task?

(I still have to check why I have a PHPUnit failure...)

One thing in regards for mkdocs - we are adding some underlying python project for it to look better. You can see it here for gitlab: https://git.drupalcode.org/project/ai/-/blob/1.2.x/.gitlab-ci.yml?ref_ty...

We should put this in a script/file so we only have to change this in one place, so any changes to gitlab, changes the local dev environment and the opposite.

I rarely recommend adding a .ddev/config.yaml, and never recommend adding the `name` element, since it means you've wired the name of the project. (It can be omitted)

The best approaches are DDEV add-ons like https://github.com/ddev/ddev-drupal-contrib

Just committing everything like this is really inflexible, compared to an add-on where it can be maintained properly and updated easily.

Related conversation in search_api: https://www.drupal.org/project/search_api/issues/3520659#comment-16149749 ✨ Enabled local development env with DDEV Active

Added a comment to the MR to remove the `name` key

Thanks!

I think that we have to decide which pattern we want to implement:

• AI module in the root and then symlinked to web/custom, like https://github.com/ddev/ddev-drupal-contrib
• Drupal in the root and AI module cloned into web/custom, like https://github.com/drupal-xb/ddev-drupal-xb-dev

At first, I thought that the first pattern was the one the community was going towards (for example, the Devel module uses it).
However, it is more complex to maintain and adds several unrelated files to the project. So now I'm more in favor of the second.

We can develop a new addon, like ddev-drupal-ai-dev, and add a CONTRIBUTING.md file to this repo. explaining how to use it.

Started adding comments, but saw that you were just asking for information about if its on the right way.

I think its on the right way - some things are missing, but that was mentioned.

I realized for pgvector and unstrcutred we need to set this up as well via the recipe.

I think the Drupal CMS AI might be overkill, we need something that sets up providers and the underlying structure - and I think the other functionalities should be fetched from external playground recipes if you want examples on what you can setup with the AI module.

I came up with a solution that is more maintainable and powerful (and more in line with what Randy says).

I created a new DDEV add-on for that, but it's designed to be used by every contrib module, not only by the ai module: https://github.com/lussoluca/ddev-drupal-contrib-root (of course, the namespace is temporary; we can place it where we want).

With this new add-on, the only change in the ai module is a CONTRIBUTING.md file: https://git.drupalcode.org/project/ai/-/merge_requests/745/diffs

Pros:

• No need to commit anything in this repo, we can update the stack without the need for a new AI release
• Support for working on multiple modules. Useful when a dev needs to work on both ai and ai_agents at the same time (for example)
• Support for adding temporary modules or recipes without impacting the composer.json file of the ai module
• We can then write a specific DDEV add-on to support specific use-cases, like the need for a vector database

I've to finish it, but I think this can be the solution.

Throwing my hat in the ring for supporting a Drupal-centric AI DDEV addon that brings in Milvus by default and can optionally be configured to bring in additional Docker services that can provide, for example, Llama, and can be used by the new AI Docker Provider. DDEV supports direct download of distribution packages:

e.g. ddev add-on get https://github.com/ddev/ddev-drupal-solr/archive/refs/tags/v1.2.3.tar.gz

@lpeabody - the biggest issue I found with Milvus is that its very hard to export and import data. Postgresql is just a sql.gz dump that you can move to the next machine. So if you embed 10000 items on one dev environment you just dump the normal database and the vector database and import it on the other, to not have to do hours of embeddings calls.

I have one more ask when this is done Luca, but its a big one and not crucial, so I will do a follow up issue. Mockoon so we can mock OpenAI or some other providers, when developing.

I've spun up https://github.com/lpeabody/ddev-ai. It re-uses a Docker Compose template that Marcus provided, I found a while back in https://www.drupal.org/project/ai/issues/3479943#comment-15809663 💬 Local Milvus Database with lando or ddev Active

@lpeabody - the biggest issue I found with Milvus is that its very hard to export and import data. Postgresql is just a sql.gz dump that you can move to the next machine. So if you embed 10000 items on one dev environment you just dump the normal database and the vector database and import it on the other, to not have to do hours of embeddings calls.

I'll research options for managing Milvus data. I don't know much about it, but if it doesn't have a native backup/import mechanism then an alternative could be using Docker volumes where you want to use Milvus in an other project, but initialize with Milvus data from another by copying the volume used in the other project. Something to experiment with. I'll add a tracking issue.

Also managed Postgres hosting is easy to come by - with Milvus you kind of need Zilliz or use cloud vendors.

I'm really approaching the add-on from a local-only perspective. There's no implication that because you're using Milvus locally that you also have to use Zilliz in production. I usually use Milvus locally and Pinecone or OpenSearch in production, with the Search API backend configuration difference being handled by Configuration Split or hardcoded configuration overrides in a settings.php file.

@lpeabody - currently we have a problem with that, but that should be fixed by 1.2.0. We can't build recipes that are VDB provider agnostic. But that should be fixed around the same time that this comes out. Issue here: https://www.drupal.org/project/ai/issues/3531028 ✨ Make vector databases abstracted and installable for recipes - Part 2 Active

But there is another reason - one of the reasons to have this development environment is also to be able to quickly spin up recipes with added database dumps for Drupal Forge and Amazee.io for one-click "test out the AI module with free credits" and we have mainly used Postgres there because it was easy to create provisioning scripts for this and it also as mentioned makes the import/export.

But maybe we could make it a possible to switch using a config value, similar to how you swap it out easily with the normal database?

Sorry, just saw your comment - nvm the comments I added.

+1 for having the development environment outside of the AI module. And the idea is very sound.

I don't think we should use recipes there, but a ddev command that can clone, install and setup the most popular providers and helper modules like Unstructured.

Recipes should then be used for adding AI functionality.

@lpeabody cool,
We can add the instruction to apply the DDEV AI add-on here: https://git.drupalcode.org/project/ai/-/merge_requests/745/diffs#3f454a9...

lussoluca → closed merge request !714

Can I recommend https://github.com/ddev/ddev-drupal-contrib as a worthwhile consideration. It does a lot of the effort it setting up an environment that can be flexibly used across different core versions and maintain relative parity with Drupal's CI testing.

It's slightly geared towards having a single primary module, though I think there are ways that could be handled by making use of require-dev and installer paths to put all the "active" projects into web/modules/custom so the commands like ddev phpunit behave as expected.

I think that would allow us to do all the config in the main AI module, rather than needing additional projects etc.

https://github.com/ddev/ddev-drupal-contrib is a very good project and I've used it in all my contrib modules (monolog, webprofiler, ...).

We need to think about pros and cons (in my opinion):

Pros:

• Well maintained
• Parity with Drupal's CI testing

Cons:

• ddev-drupal-contrib suppose you clone only one project at a time. At the moment, cloning multiple modules is not supported (you have to do it manually, by going into web/modules/contrib and run a git clone)
• if you need some other modules (not the source, just to be installed), you can't, unless they're listed in the require or require-dev section of the module's composer.json
• ddev-drupal-contrib works better if you commit all the DDEV artifacts it creates. The AI ecosystem is huge, here we're thinking about the AI module, but then we should configure every other module in the same way
• If we commit DDEV artifacts created by ddev-drupal-contrib, then we have to maintain them (update them every time they change on the source add-on)

Of course, we can find a way to resolve all those issues, but I think that, in the end, this solution will be more complex to maintain and provide a worse DX.

For example, one request from Marcus was:

Its very common that you do not just work on the AI module, but also some contrib module. It would make it easier if I for instance want to work on the OpenAI provider that it gets it from repo sources, so you don't have to uninstall, git clone and reinstall it

The solution proposed in this MR resolves all the issues and will allow us to release new versions of the stack without the need to release a new version of the AI module.

Just to recap, my proposal is:

• Develop a new generic DDEV add-on (that can be used by any contrib modules) that simplifies contributing to multiple modules at the same time: https://github.com/lussoluca/ddev-drupal-suite
• Develop a new specific DDEV add-on with all the components required by the AI ecosystem (vector db, ffmpeg, ...): https://github.com/lpeabody/ddev-ai
• Develop a Drupal recipe that installs and configures a minimal set of modules: https://www.drupal.org/project/ai_dev_recipe →
• Add to this repo (and maybe the repo of every other AI module) a CONTRIBUTING.md file with a set of instructions

In the end, for me, it's the same. I'll let Marcus and the community decide :-)

Firstly to say that I don't mean to be putting down the work you've done! I agree that the community, and in particular the maintainers who will end up responsible for it, are the ones to decide. I think both options have good merit and I'm introducing this only to help us find the best overall solution. It may be that we're best to bring in some of the good bits from ddev-drupal-contrib into our own 'custom rolled' stuff.

I've opened https://github.com/ddev/ddev-drupal-contrib/issues/135 to discuss where I feel the limitations of ddev-drupal-contrib lie. If I've missed anything, do say!

Your first two cons are the first two points of consideration on there.

ddev-drupal-contrib works better if you commit all the DDEV artifacts it creates. The AI ecosystem is huge, here we're thinking about the AI module, but then we should configure every other module in the same way

I'm leaning the other way - we only commit the .ddev configuration for the AI module, and everything else gets worked from in that project.

If we commit DDEV artifacts created by ddev-drupal-contrib, then we have to maintain them (update them every time they change on the source add-on)

We only need to update if there is something we want to update for. Once added to a project, the add-ons can be used "as is" without updates/changes.

An update is just ddev add-on get ddev/ddev-drupal-contrib and then review/commit any changes. Where as the alternative is the AI maintainers taking on not just the maintenance of 3 additional projects (plus documentation), but also the development to provide the functionality we'd want.

1. Production dependencies - these go into require, and we can use installer-paths if we want them in web/modules/custom so we can develop them.
2. Non-production dependencies - these go into require-dev and likewise can use installer-paths if we want.
3. Things we will always want when developing AI module - these could go in require-dev, but that would mean they also get installed in Drupal CI. This may be an acceptable trade-off, or we may want to find an alternative approach (git submodules could even be an acceptable approach).
4. An easy way to add ad-hoc dependencies in a specific instance - this could be solved with require-dev and no committing the changes, but that doesn't feel ideal. You could just check them out (and it'd be easy to provide some commands to make that easier).

I think looking at how preferred-install could be used to make it easier to get working repos would be worthwhile.

I do think a lot of those could be solved by having a single AI development project, which I think wouldn't need to be more than .ddev and composer.json (and probably a README.md).

@andrewbelcher, this is an interesting approach!

I'm pushing a new MR (https://git.drupalcode.org/project/ai/-/merge_requests/762) with ddev-drupal-contrib artifacts committed and a new command that uses composer require to add a new module to the codebase (similar to ddev poser), for now it is called ai-get-module.

After setting up an environment:

1. clone ai module
2. ddev start
3. ddev poser
4. ddev symlink-project

If I want to add ai_agents (as a git cloned repo) to work on it, I can run:

ddev ai-get-module drupal/ai_agents:1.1.x-dev@dev from-source

If I just need the module:

ddev ai-get-module drupal/ai_agents:1.1.1

Hi all. Lots of good discussion here. I'm happy to discuss and hopefully find a way for this ecosystem to build on top of ddev-drupal-contrib (I'm the maintainer).

The latest MR from @lussoluca looks great to me.

I think .ai-get-module could go into ddev-drupal-contrib in some fashion. Maybe it becomes `ddev poser require`. If no second argument is provided, we assume install for backwards compat with current poser. Until then, its fine for ai-get-module to live in this module.