Project/issue nodes are knowledge that can be graphed

Created on 15 May 2024, 2 months ago
Updated 13 July 2024, 6 days ago

Problem/Motivation

This article describes how to take bulk text and convert it to a network graph of concepts and then graph it. The immediate take away for me is that we are on the brink of having practical ways of taking a sum of knowledge (like durpal.org projects and issues) and leveraging the natural relationships between content through tagging, metadata and hyperlinks.

The risk is that we're creating a project browser that as of yesterday is already much worse than ChatGPT 4o - but I can also see how someone can use Predicive AI tooks alongside project browser.

Steps to reproduce

Proposed resolution

🌱 Plan
Status

Active

Version

1.0

Component

User experience

Created by

πŸ‡¦πŸ‡ΊAustralia sime Canberra

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @sime
  • πŸ‡¦πŸ‡ΊAustralia sime Canberra
  • πŸ‡¦πŸ‡ΊAustralia sime Canberra
  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Meta: I've created this issue because it's better on paper than in my head. I don't expect this to be on the roadmap since there are massive improvements in Project Browser already happening, and it would take a gigabrain to action which I'm not, or at least I have too much context switching to effectively scratch the itch.

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    Great idea. First it can be done as an experiment and then see the results.
    It looks like a very interesting post, I will read it.

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    One of the big problems we have in contrib is duplicate modules that don't really explain that they are duplicates. And there are more than a few half-finished module comparison pages. If someone gets to a Geofield type module, then they should be suggested other modules and be able to see quickly what the relative stablity and popularity of these modules is. I don't think we should sweat out fixing the metadata issues, I think this will be solvable with some sort of machine learning, because if you look in those modules you do find issues like "How does this differ to ..." and "Is there going to be a D10 version?" which I use to do a deep evaluation.

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    Yes! Another example of this I described here: #3241544-7: Add filter by project dependencies (ecosystem) β†’

    (P.S. I had a lengthy talk with AI about that ctools question, and discovered in detail that its many features listed in description was mostly incoporated in Drupal 8/9/10 core, and that module is just mysterious remnant from the past), but forgot again why it is still exist and most popular with 113,003 D8 installs.....)

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Ctools is used with a few legacy projects that have some utility. You can use page_manager module for example, it does a thing.

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    Task can be simplified.

    1. Do experiment only
    2. Visualize modules only (without issues for now)
    3. Use hosted AI (for example, openAI or other hosted service like HuggingFace) instead of local AI like described in the article.
    4. Use Drupal instead of other many fancy tools like described in the article

    I drawed a mindmap implementing this with Drupal:

    It will probably not work out-of-the-box for this usecase, but it means it is an opportunity to improve Drupal modules where features are missing for this.

    Links:
    1. Videos about AI interpolator capabilities https://www.youtube.com/@drupal_ai/videos
    2. https://www.drupal.org/project/huggingface β†’
    3. https://www.drupal.org/project/aiprompt β†’

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Drupal doesn't feel like the right tool for doing ML. But i kinda dig that there is a community forged pathway so I suspend my judgement :D

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    With AI, a lot of new things become possible :-)

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Let me try that another way... Firefox is a good tool but you don't need drupal to run it :D

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    You are right about using many different tools in general.
    But If I would do this experiment, I would do it with Drupal, because it is simpler and faster (for me and possibly for other Drupalists). And AI accelerate missing features development.

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    The future: we ask AI and it builds the tools we need. At least that is my version of the future.

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Do you have examples of a drupal site is fitted out with AI tools? For examples i can run `composer create-project foo/drupal_ai_demo`?

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    @Marcus_Johansson started working on one https://www.drupal.org/project/woa_simplytest β†’
    With Drupal Recipe initiative and Starshot β†’ , there should be possible to quickly spin up various workflows:
    https://kanopi.com/blog/the-comprehensive-guide-to-drupal-recipes/
    https://workflows-of-ai.com/

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Well, this is the thread where you can post examples (gitpod, or ddev quickstarts) that people can test. Your challenge is offering solutions which are palatable and fit into how the Drupal project operates (CI, ethics, etc).

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    Oh nice!

  • πŸ‡ΊπŸ‡ΈUnited States chrisfromredfin Portland, Maine

    I see the value here in asking an LLM "what modules should I use to accomplish X?" - and I think it would be a great thing to bolt on to PB someday, maybe.

    Note that I use AI/ML (an actual custom clustering model - not just "ask the LLM") to actually feed the descriptions of the Top 100 modules into an LLM and have it come up with word clusters as one of the ways we came up with our new list of 19 categories. πŸ˜‰

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    "what modules should I use to accomplish X?" - and I think it would be a great thing to bolt on to PB someday, maybe

    For this particular feature, FreelyGive team have built a working demonstration here: http://project-browser.freelygive.io/

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    @chris - I intentionally tried to keep the title loose. There is a tendency for people to say "plug in to Open AI" but I don't think integration into a commercial service is the way to go and a lot of the drupal modules I've seen are basically doing just that. But it would be silly not to have a place to discuss ideas.

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    @mindaugasd love the examples, but i think the API integration is just not going to happen within the actual Project Browser code base.

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    New API integration is https://www.drupal.org/project/ai β†’ , which enforces implementations to use key module by default :-)

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    I think you misunderstand me, it looks like your demo is connected to OpenAI. Project Browser wouldn't do that out of the box.

    Do any of these integrations you work with allow you to embed a "small gpt" and avoid connecting to commercial services?

  • πŸ‡±πŸ‡ΉLithuania mindaugasd

    That is one of the reasons AI abstraction layer was created.

    Drupal being a truly open project has to ensure users can choose which AI service they want to use instead of depending (hard-coding) a single one, therefore Drupal needs an LLM abstraction layer separating and decoupling Drupal features from AI services, enabling to quickly switch service provider when needed without any hassle.

    More at πŸ“Œ Create LLM abstraction layer Active

    I think AI module is well place to be part of Drupal CMS within starshot initiative, so module has to be not too much bloated and bundled with features people most commonly use, and also solid features prepared to be in Drupal CMS.

    More at ✨ Create AI ecosystem "add-ons" page Active

    In general, people show demand to experiment with local AI. So if there is demand, it could be included.
    [...]
    Another question is how to make it easy enough and accessible to regular Drupal CMS users. How does one install it on the server actually. How much knowledge, investment and experience does it require to do it in the proper way.

    More at πŸ“Œ Ollama LLM Provider Active with actual experiments/process to run it,

    As per πŸ“Œ [meta] Discussion: what LLM providers to include Active AI module currently have support for these options to run local or open AI:
    Ollama πŸ“Œ Ollama LLM Provider Active , LM Studio πŸ“Œ LM Studio LLM Provider Active , Hugging face πŸ“Œ Huggingface LLM Provider Active , discussion about jan.ai πŸ’¬ Can a module be provided for use with Jan.ai as a provider? Closed: won't fix .

Production build 0.69.0 2024