AI Evaluations to extend AI Logging functionality

Created on 11 December 2024, 12 days ago

Problem/Motivation

Currently, AI Evaluations duplicates much of the functionality already present in AI Logging module, specifically around storing AI interaction data (prompts, responses, provider information, metadata).

Proposed Change: Update AI Evaluations to extend AI Logging functionality rather than duplicate it.

This would mean:

  • Remove duplicate entities and code from AI Evaluations
  • Extend AI Logging entities with evaluation-specific fields
  • Update existing hooks and templates to use AI Logging data
  • Provide migration path for existing evaluation data

Benefits

  • Eliminates data redundancy
  • Reduces maintenance overhead
  • Improves system performance
  • Creates clearer separation of concerns
  • Single source of truth for AI interaction data

Migration considerations

Need to ensure smooth transition for sites already using AI Evaluations, with clear upgrade path and data preservation.

🌱 Plan
Status

Active

Version

1.0

Component

Code

Created by

🇬🇧United Kingdom seogow

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024