-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Feature Request
Problem
When calculating workspace build times using Prometheus metrics, there's a significant gap in observability. The current metrics:
coderd_provisionerd_workspace_build_timings_seconds- measures provisioner job duration (init, plan, graph, apply stages)coderd_agentstats_startup_script_seconds- measures startup script execution timecoderd_agents_connection_latencies_seconds- measures network latency to DERP relay (not connection time)
However, none of these capture the agent first connection duration — the time from when a workspace agent is created until it first connects to coderd. This can be a significant portion of the overall build time (several minutes in some cases), especially when users mount persistent home directories or have complex infrastructure provisioning.
The build timeline UI shows this as the "connect" stage under the agent, and the data is available via the /api/v2/workspacebuilds/{id}/timings endpoint (as agent_connection_timings), but it's not exposed as a Prometheus metric.
Proposed Solution
Add a new Prometheus metric to capture agent first connection duration, for example:
coderd_agent_first_connection_seconds
This metric could be a histogram (similar to other timing metrics) with labels such as:
template_nameagent_nameworkspace_name(optional, may be high cardinality)
The timing calculation already exists in the codebase:
StartedAt: agent.CreatedAt,
EndedAt: agent.FirstConnectedAt.Time,Use Cases
- Calculate accurate P90/P95 workspace build times that match what users actually experience
- Identify and alert on slow agent connection times
- Compare build performance across different templates or infrastructure configurations
- Debug workspace startup issues by correlating with other metrics
Additional Context
This timing data is already tracked in the database (workspace_agents.first_connected_at - workspace_agents.created_at) and exposed via the API, so this would primarily be about adding the Prometheus instrumentation.