rhesis

Security

• Resolved security advisories across Python and npm dependencies (#3106).

Fixed

• Fixed end-to-end testing workflow compatibility by pinning CI runners to ubuntu-24.04 (#3102).

Platform Release

This release includes the following component versions:

• Backend 0.20.0
• Frontend 0.20.0
• SDK 0.20.0

Summary of Changes

Backend v0.20.0:

• Added S3-compatible blob storage support with strict startup configuration validation.
• Enhanced metric evaluations to pass full trace history and test expectations, enabling targeted span evaluations and tool call trajectory checks.
• Redesigned the Insights registry to add trace latency, token usage, cost, and execution duration analytics alongside new query-shaping parameters.
• Introduced ClickUp MCP tool integration, added a fromjson Jinja filter for structured inputs, and moved auth email sending to background tasks.

Frontend v0.20.0:

• Introduced interactive charts (line, bar, heat tables) to Architect responses with export options (CSV, SVG, PNG) and layout fixes.
• Redesigned and compacted the trace drawer with timing bars, filterable span trees, and a new Traces tab in test results.
• Enhanced the Endpoint Wizard to allow editing test payloads as JSON and creating endpoints without blocking on failed tests.
• Refreshed sidebar navigation with scroll cues, improved dark mode styling, and updated Insights grouping.

SDK v0.20.0:

• Refactored metric signatures to accept unified trace and expectations parameters (breaking change), allowing metrics to evaluate full trace spans or build traces in-memory.
• Introduced metric targets for evaluating specific parts of a trace and added the ToolCallTrajectory metric to compare agent execution against expected tool calls.
• Enhanced Architect agent capabilities with interactive charting tools (ShowChartTool), improved tool failure/retry handling, and expanded Insights query shaping (time_grain, order_by, limit).
• Added ClickUp MCP integration, parameter-name recording on tool spans, and improved cache token calculations for streamed LangChain calls.

See individual component changelogs for detailed changes:

• Backend Changelog
• Frontend Changelog
• SDK Changelog

Added

• Tool Call Trajectory Metric: Introduced ToolCallTrajectory to evaluate agent tool call sequences against expected tool calls defined in test configurations (Test(expected_tool_calls=[...])).
• Metric Targets: Added metric targeting support, enabling metrics to evaluate specific sub-paths or target scopes within a trace.
• In-Memory Trace Construction: Added Trace.from_response() and Trace.from_conversation() to allow trace and metric evaluations without requiring active background telemetry tracing.
• Trace Access for Custom Metrics: Code metrics (@metric) can now declare a trace parameter to inspect detailed trace spans, kinds, inputs, outputs, tool calls, and metadata.
• Agent Charting Capabilities: Added ShowChartTool allowing agents to construct charts from insights queries with custom focus filters and chart types.
• Enhanced Insights Query Shaping: Added time_grain, order_by, limit, and min_count parameters to Insights queries, alongside an Insights.describe() method for registry schema introspection.
• Named Tool Parameters in Telemetry: Tool spans now capture positional and keyword parameters by parameter name under ai.tool.arguments.

Changed

• Metric Evaluation Signature (BREAKING CHANGE): Standardized evaluate() and a_evaluate() signatures across all metrics to accept (input, output, trace, expectations).
  • Individual evaluation parameters (expected_output, context, tool_calls, metadata, conversation_history, goal, instructions, contract) have been removed in favor of expectations and trace / conversation.
  • @metric functions using legacy parameter names must update their signatures to use trace and expectations.
• Retrieval Telemetry Format: Telemetry hooks for LangChain, Haystack, and @observe.retrieval now record full document lists as JSON objects rather than truncated preview strings.
• Architect Approval & Retry Logic: Tightened agent execution by preventing infinite retry loops on failing tool calls, validating placeholder IDs before dispatch, and strictly scoping user approvals (Accept) to specific targeted calls.

Fixed

• LangChain Streamed Cache Token Calculations: Corrected input token pricing on streamed LangChain calls by subtracting cache_read and cache_creation details from the total input token count.
• Tool Routing in Multi-Provider Agents: Fixed an issue in BaseAgent where named tool execution failed when routed across multiple tool providers.
• Score & Choice Validation: Added strict validation checks for numeric judge scores and categorical decision model outputs.
• Architect Chart Rendering: Resolved issues with grouped bar focus matching, area/stacked bar constraints on non-count metrics, and chart title formatting.

Platform Release

This release includes the following component versions:

• Backend 0.19.1
• Frontend 0.19.1
• SDK 0.19.1

Summary of Changes

Backend v0.19.1:

• Added created_at and updated_at timestamp fields to API task responses.
• Fixed an issue where SDK multi-turn conversation runs split turns into separate traces.
• Updated backend dependencies (certifi, nltk, uv, pillow) to resolve security advisories.

Frontend v0.19.1:

• Redesigned endpoint creation into a four-step guided wizard with step navigation, connection testing, and raw JSON response previews.
• Added turn navigation to conversation traces, enabling users to step through multi-turn conversations and open specific turns from replies.
• Restored the dark mode toggle to the user menu with support for automatically matching browser theme preferences.
• Enhanced UI components by adding empty states for unmatched metric filters and moving tags to the bottom of the metric detail page.

SDK v0.19.1:

• Updated dependencies including nltk, certifi, uv, and pillow to address security vulnerabilities.
• Fixed an issue where authentication fields were not preserved when caching settings.
• Improved theme handling to automatically follow browser dark mode preferences.
• Resolved an issue to ensure the trace metrics grid automatically refreshes on change.

See individual component changelogs for detailed changes:

• Backend Changelog
• Frontend Changelog
• SDK Changelog