Add image Chat input possibility with Ollama

I have pushed it to a branch, where you can test it if you can checkout branches?

It also fixes: https://www.drupal.org/project/ai/issues/3474112 ✨ Add streaming and system role to Ollama provider. Active

The reason I'm not merging it is that I am 99% sure I have implemented it correctly, but when I use Llava 13B, it hallucinates a lot more then Llava 13B with Fireworks AI.

Please review.

Merge request and branch here: https://git.drupalcode.org/project/ai/-/merge_requests/95

@marcus Thank you for your marvelous diligence!
I test the latest dev version after drush updb && drush cr, the same problem still exists as below:

11 Add to Default shortcuts
Open configuration options
Prompt
user
Please extract text from this image:

Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
[{"value": "requested value"}]

Operation Type
chat
Provider
ollama
Model
llava:latest
Tags
chat
Response
{"id":"chatcmpl-0","object":"chat.completion","created":1727097978,"model":"llava:latest","systemFingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":" [[{\"value\": \"101-2021-03-31T10:00:00Z\"}]]] ","toolCalls":[],"functionCall":null},"finishReason":"stop"}],"usage":{"promptTokens":57,"completionTokens":34,"totalTokens":91}}
Configuration
max_tokens:
1024

temperature:
1

top_p:
1

Extra Data
[]
Authored on
Mon, 09/23/2024 - 21:26

Ah, it's not in merge, I didn't want to merge it before someone tried it. It has to be gotten from the branch in the merge request.

If this is not feasible for you and no one else tests it in the next week, I'll merge it and you can test it that way.

@marcus, your merge request works now! Thank you!

Cool, getting merged

Automatically closed - issue fixed for 2 weeks with no activity.

We are doing some issue management housekeeping and adding/removing components.

The "Providers" component will be removed because providers have been moved into their own projects, so this issue is being moved to "Miscellaneous".

If we have a new project to move this to, we can also do that as a secondary step.

See 📌 Update AI module project components Active for more details.