Logging output doesn't work for AWS Bedrock provider

Created on 7 February 2025, about 2 months ago

Problem/Motivation

The Bedrock provider sets the ChatOutput $rawOutput param with the direct response from the AWS SDK, which means it's an Aws\Result. This then fails to get json_encoded in the logger, so you end up with {} in the output.

Steps to reproduce

Enable the AWS Bedrock provider and AI logging with output logging enabled, and trigger a request. View the log, the output will always be {}.

Proposed resolution

Not quire sure, thoughts are one of:
1. AWS Bedrock Provider should set $response->toArray() as the raw output. This might not be ideal for direct use though, as the AWS response object may have a use.
2. Logger should do some detection on the data type and figure out ways to handle types that don't directly encode. This isn't ideal as it makes them dependent on lots of other types etc...
3. Logger should use normalised output for logging, but this will lose additional provider specific metadata...
4. We should add an extra property to ChatOutput that always contains the raw data in a consistent data type. This means the provider is responsible for conversion, but means storing yet another copy of data.
5. AWS could extend ChatOutput to store the AWS DTO, expose is via a new method, and do a toArray for getRawOutput.

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task

Status

Active

Version

1.1

Component

Providers

Created by

🇬🇧United Kingdom andrewbelcher

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @andrewbelcher
Comment about 2 months ago →
🇩🇪Germany marcus_johansson
So, #1 is tested and works, but its not great to change if someone is actaully using the raw output and the objects for their business logic. However the module is not stable yet, so we could do that. OpenAI, FireworksAI and Ollama does exactly that at the moment.

I think #4 is a good way to go, but for all operation types for 2.0 release. We currently log the rawoutput for any operation type, but should instead always have an array version of the output.

We could also do a mix where we add the properites now, and its optional for providers to implement and then the logging method checks if the property is set, otherwise it tries to use the raw output. And then make it required in the interface for 2.0 release. We are thinking about doing the same thing for output tokens handling before 2.0.

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024