- Issue created by @seogow
- 🇬🇧United Kingdom yautja_cetanu
Ok firstly this is amazingly cool!!!
So the good things we need to do are:- Maybe create a video explaining it.
- Maybe a blog post showing screenshots of each page and why its useful and how to use it.
- Would love to bring emma into it and see what she thinks
- Tried downloading a prompt and used it in playground and it gave roughly the same answer as in drupal. Super cool for figuring out what is going on.
- Connect this with Evaluations.
- Have a button on DeepChat (There needs to be a setting to turn this on) that allows you to highlight an assistant message and go to the log directly.
Couple of points:
- What happens if people don't want to spend money on API calls to summarise any log. Is there a setting to turn this off and is there a fall back?
- It would be good to be able to export the log itself and see a similarly well structured log file that is human readable but also machine readable. I'd like it so that multiple front-ends could visualise the logs differently. (For example a react tree of agents, minikanban could replay the logs)
- I'm personally finding myself lost with the "Child" logs and where the one I'm looking at fits in and relates to the child logs. Don't have an immediate solution for it yet
- 🇬🇧United Kingdom yautja_cetanu
Definitely todo:
Fallback for no LLM summaries.
- Requested Actions - Goes to> "First few lines of last user prompt"
- Performaed action- Goes to> "First few lines of that is the body of the response"Follow up issues:
Explore how good smaller cheaper models are
Explore embedding a small opensource llm to handle this. - First commit to issue fork.
- 🇬🇧United Kingdom justanothermark
- Merge AI Logging LLM settings into the AI logging settings
- Allow a setting that turns off AI summarisation of logs
- If the setting is turned off, put a warning on the report page "This log overview page is significantly improved with AI summaries that can be turned on here"Pushed these changes to a new branch
3493277-extend-logging-capabilities-v2
- 🇩🇪Germany marcus_johansson
First off, it looks really nice! I added some comments outside of the stuff phpcs etc. found.
The biggest issue right now seems to be that all other operation types logging are gone? While some of them do not make sense at the moment, moderation and embeddings for isntance probably could do, to find out why something got banned or if something was searched for.
Also it adds Performed Actions and Last Performed Actions on any chat log, which also makes little sense, even if it doesn't show it?
Most of my comments are NIT - but I think we need to have a bundle for other operation types in there, otherwise we don't have feature parity?