"BatchAPI gives a 50% discount on regular completions and much higher rate limits (250M input tokens enqueued for GPT-4T). Results guaranteed to come back with 24hrs and often much sooner."
For more details, visit the docs: https://platform.openai.com/docs/api-reference/batch
If a delay between content generation en retrieval is acceptable, we could provide a feature to enable BatchAPI and to reduce costs.
In order to do so we should implement a delayed batch process.
Postponed
1.0
OpenAI Embeddings
At the moment I believe batch is just for completions and not embeddings. I don't think we can use this yet (at least not for embeddings sub-module).