- Issue created by @andrewbelcher
- Merge request !792Issue #3538027: Support Fibers for collaborative multitasking on LLM io waiting β (Merged) created by andrewbelcher
- π¬π§United Kingdom andrewbelcherNeeds a provider extending OpenAiBasedProviderClientBasethat supports streaming to test. Can be tested with:phpunit --configuration phpunit.ai.xml --filter FiberTestWhere phpunit.ai.xmlhas a correctly configured OpenAI key.
- π¬π§United Kingdom andrewbelcherAlso run a straight Fibers vs non-Fibers for 5 LLM calls each generating 15 haiku: i$ ddev drush scr scripts/fibers.php Without fibers -------------- 5/5 [ββββββββββββββββββββββββββββ] 100% [OK] Without fibers: 56.567 seconds With fibers ----------- 5/5 [ββββββββββββββββββββββββββββ] 100% [OK] With fibers: 14.601 seconds Comparison ---------- [OK] Performance improvement: 3.9
- π¬π§United Kingdom andrewbelcherThis also includes a fix for #3531134-31: Create Base Class for OpenAI based clients β , though I'm not sure if TRUEis the correct default...
- π©πͺGermany marcus_johanssonI added some comments, I think its also good if we add a flag for this - so anyone setting up a third party app, can check if the provider supports fibers. We need documentation fort this as well, but let me know if you need help with writing that. 
- πΊπΈUnited States Kristen Pol Santa Cruz, CA, USASwitching to the correct tag 
- π©πͺGermany marcus_johanssonGetting merged - I have added the provider capability flag for if Fiber is supported in a provider, so a third party app can choose to only show models/providers that would work. I've also added a short documentation and merged with the improved structure. The actual code needed in the (any) provider is quite streamlined now - the FiberTests have been run on the new structure. 
- 
            
              marcus_johansson β
             committed 1cb2e289 on 1.2.x authored by 
            
              andrewbelcher β
            
Issue #3538027: Support Fibers for collaborative multitasking on LLM io... 
 
- 
            
              marcus_johansson β
             committed 1cb2e289 on 1.2.x authored by 
            
              andrewbelcher β
            
- Automatically closed - issue fixed for 2 weeks with no activity.