- Issue created by @yautja_cetanu
- 🇬🇧United Kingdom yautja_cetanu
The post action prompt has this text:
"Start the message with the following information:
Thank you for your question. I am looking up the answer.
"Where does that copy come from? It's not in the assistant message?
- 🇬🇧United Kingdom yautja_cetanu
- The AI Agents don't have the context provided by default. So if a User says can you change this page to be called Wine Tours and we are looking at events. It will only know if the Assistant looks at the context, decides this information is important and passes it onto the agent. One test showed it did do that. But it does mean if there is any contextual information important for the agents the assistant didn't know matters than it won't have it. This is probably fine.
- 🇬🇧United Kingdom yautja_cetanu
The post-action prompt should likely provide the LLM information about what has actually been performed.
Currently:
user Can you change the content type called "Hard Cheeses" to be called "Soft Cheeses" instead? assistant The following are the results the different actions from the node_content_type_agent action: The node type <em class="placeholder">Soft Cheeses</em> (<em class="placeholder">page</em>) has been updated.
Provides the answer:
Thank you for your question. I am looking up the answer. - The content type has successfully been changed from "Hard Cheeses" to "Soft Cheeses."
So it infers from the User question to change it from Hard to Soft and the response which "Soft has been updated" that Hard has changed to soft.
But actually even the LLM doesn't really know if Hard has been changed to soft. Soft Cheeses could have been updated some other way (a settings change) and maybe it already existed before the User asked for "Hard" to be changed to "soft". So the LLM has been essentially told to guess what it has done which may make it more prone to hallucination.
- 🇬🇧United Kingdom yautja_cetanu
Suggested changes to the structure of the pre-prompts:
https://docs.google.com/document/d/1ixlRmMjj_Kc51F02oV-vhBaEGmT7Ivkc75eU...
- First commit to issue fork.
- 🇬🇧United Kingdom MrDaleSmith
Approving this will immediately break the Starshot Recipes for the AI module, so this will need to be handled carefully. I'm not convinced I've picked up on all the nuance of the suggested changes, so I'd request a careful review.
Automatically closed - issue fixed for 2 weeks with no activity.