[WIP] Support tensorzero_extra_content_experimental for OpenAI-compatible endpoint
#5992
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e02ab4728
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ContentBlockChunk::Thought(thought) => { | ||
| let len = thought_id_to_index.len(); | ||
| let index = *thought_id_to_index.entry(thought.id.clone()).or_insert(len); | ||
| thoughts.push(OpenAICompatibleThoughtChunk { | ||
| index, |
There was a problem hiding this comment.
In process_chat_content_chunk the thought index is derived solely from the count of distinct thought IDs, but the struct comment and openai_messages_to_input semantics treat this index as the position in the full content array (text/tool calls included). This means any stream where a text/tool-call block appears before the first thought will produce index = 0 even though the actual content position should be >0, so clients that round-trip tensorzero_reasoning_content will mis-order thoughts relative to text/tool calls (e.g., a text chunk followed by a thought chunk will be reconstructed as thought-first). Consider tracking indices based on overall content block order (like the non-streaming path) rather than only thought ID order.
Useful? React with 👍 / 👎.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| #[expect(clippy::ref_option)] | ||
| fn is_none_or_empty<T>(v: &Option<Vec<T>>) -> bool { | ||
| v.as_ref().is_none_or(Vec::is_empty) | ||
| } |
There was a problem hiding this comment.
Duplicated is_none_or_empty helper function across modules
Low Severity
The is_none_or_empty function is being added to chat_completions.rs but an identical function already exists in streaming.rs. Both functions have the same signature, implementation (v.as_ref().is_none_or(Vec::is_empty)), and even the same #[expect(clippy::ref_option)] attribute. This duplication creates a maintenance burden where any future bug fix would need to be applied to both locations.
Additional Locations (1)
GabrielBianconi
changed the title
tensorzero_reasoning_content for OpenAI-compatible endpointtensorzero_extra_content_experimental for OpenAI-compatible endpoint
Fix #5970
Note
Medium Risk
Changes the OpenAI-compatible API response/streaming shape by emitting a new
tensorzero_extra_content_experimentalfield and by omittingtool_callswhen empty, which may impact downstream clients expecting previous serialization behavior.Overview
Adds experimental support for non-text assistant content (TensorZero
ThoughtandUnknownblocks) to the OpenAI-compatible chat completions API.Non-streaming responses now surface these blocks via
message.tensorzero_extra_content_experimentalwithinsert_indexmetadata (instead of dropping them), and streaming deltas emitdelta.tensorzero_extra_content_experimentalwhile maintaining stable indices across chunks via an ID→index map. Assistant input messages can also includetensorzero_extra_content_experimental, which gets merged into internal message content using optional index-based insertion.Also adjusts serialization to omit empty
tool_calls/ extra-content arrays (returningNoneinstead of[]) and adds targeted tests for ordering and (de)serialization.Written by Cursor Bugbot for commit 48e0fae. This will update automatically on new commits. Configure here.