> ## Documentation Index > Fetch the complete documentation index at: https://docs.interhuman.ai/llms.txt > Use this file to discover all available pages before exploring further. # Realtime Analyze > Analyze a realtime video stream. **Beta** — this API is under active development and may change without notice. ## AsyncAPI ````yaml realtime_analyze_v1 id: realtime_analyze_v1 title: realtime_analyze_v1 description: Analyze a realtime video stream. servers: - id: production protocol: ws host: api.interhuman.ai bindings: [] variables: [] address: /v0/realtime/analyze parameters: [] bindings: - protocol: ws version: 0.1.0 value: headers: type: object properties: X-Client-Request-Id: type: string description: >- Optional identifier supplied by the client to correlate this request with their own logs. When provided, the value is recorded alongside the server-assigned correlation ID in Interhuman logs to aid lookup and support investigations. This header is not echoed back in the response; the server returns its own correlation ID in the `X-Correlation-ID` HTTP response header. Sec-WebSocket-Protocol: type: string description: >- Authentication transport for clients that cannot set an `Authorization` header on a WebSocket connection, such as the browser `WebSocket` API. Send the literal subprotocol marker `access_token` followed by your credential, as a comma-separated subprotocol list: Sec-WebSocket-Protocol: access_token, In browser JavaScript, pass the same pair as the constructor's subprotocol array: new WebSocket(url, ["access_token", credential]) `access_token` is the required literal marker and `` is your bearer credential. In a browser, this should be a short-lived access token minted server-side by the client tokens endpoint (`POST /v1/client_tokens`) — never your API key. The server selects and echoes `access_token` as the negotiated subprotocol, never the credential. Treat the credential as a secret: never log it. examples: &ref_0 - access_token, schemaProperties: - name: headers type: object required: false properties: - name: X-Client-Request-Id type: string description: >- Optional identifier supplied by the client to correlate this request with their own logs. When provided, the value is recorded alongside the server-assigned correlation ID in Interhuman logs to aid lookup and support investigations. This header is not echoed back in the response; the server returns its own correlation ID in the `X-Correlation-ID` HTTP response header. required: false - name: Sec-WebSocket-Protocol type: string description: >- Authentication transport for clients that cannot set an `Authorization` header on a WebSocket connection, such as the browser `WebSocket` API. Send the literal subprotocol marker `access_token` followed by your credential, as a comma-separated subprotocol list: Sec-WebSocket-Protocol: access_token, In browser JavaScript, pass the same pair as the constructor's subprotocol array: new WebSocket(url, ["access_token", credential]) `access_token` is the required literal marker and `` is your bearer credential. In a browser, this should be a short-lived access token minted server-side by the client tokens endpoint (`POST /v1/client_tokens`) — never your API key. The server selects and echoes `access_token` as the negotiated subprotocol, never the credential. Treat the credential as a secret: never log it. examples: *ref_0 required: false operations: - &ref_48 id: v1_realtime_receive_handler title: V1_realtime_receive_handler description: '' type: receive messages: - &ref_63 id: stream_video_message payload: - format: binary x-parser-schema-id: name: Video (bytes) description: >- Binary video segment sent by the client for analysis. Each segment must not exceed 32MB. Accepts the following formats: mp4, avi, mov, mkv, mpeg-ts, mpeg-2-ts, webm. headers: [] jsonPayloadSchema: format: binary x-parser-schema-id: title: Video (bytes) description: >- Binary video segment sent by the client for analysis. Each segment must not exceed 32MB. Accepts the following formats: mp4, avi, mov, mkv, mpeg-ts, mpeg-2-ts, webm. example: '[]' bindings: [] extensions: - id: x-parser-unique-object-id value: stream_video_message - id: x-parser-message-name value: stream_video_message bindings: [] extensions: &ref_1 - id: x-parser-unique-object-id value: realtime_analyze_v1 - &ref_49 id: v1_realtime_session_close_handler title: V1_realtime_session_close_handler description: '' type: receive messages: - &ref_64 id: session_close_v1_message payload: - name: Session Close description: Inbound ``session.close`` text frame. type: object properties: - name: type type: string title: Type description: >- Discriminator identifying the inbound message kind. For a close request it is always ``session.close``. required: true headers: [] jsonPayloadSchema: additionalProperties: false description: >- Inbound ``session.close`` text frame. Tells the server the caller is done sending video and wants a graceful end-of-analysis handshake: the server acknowledges with ``session.closing`` (carrying ``max_drain_seconds``), rejects any further binary video frames, finishes analyzing the already-accepted video and emits its normal result envelopes in order, emits final lifecycle envelopes for still-open analysis state, sends ``session.ended``, and closes the WebSocket. Unknown fields are rejected (with a non-fatal ``error`` envelope; the session stays open) so a caller typo cannot be silently misread as a close request. Only binary video is refused after acceptance. Text messages remain accepted while the session drains: a session-config update is applied and acknowledged as usual and governs the analysis still draining, a duplicate ``session.close`` is ignored, and (on the realtime endpoint) a ``transcript.updated`` still lands and is rendered into any realtime recommendations that runs during the drain. Malformed text frames received during the drain are rejected with a non-fatal ``error`` envelope rather than closing the connection early, so the caller always gets its ``session.ended``. examples: - type: session.close properties: type: const: session.close description: >- Discriminator identifying the inbound message kind. For a close request it is always ``session.close``. title: Type type: string x-parser-schema-id: required: - type title: Session Close type: object x-parser-schema-id: SessionCloseV1 title: Session Close description: Inbound ``session.close`` text frame. example: |- { "type": "session.close" } bindings: [] extensions: - id: x-parser-unique-object-id value: session_close_v1_message - id: x-parser-message-name value: session_close_v1_message bindings: [] extensions: *ref_1 - &ref_50 id: v1_realtime_session_config_handler title: V1_realtime_session_config_handler description: '' type: receive messages: - &ref_65 id: realtime_session_config_v1_message payload: - name: Realtime Session Config description: Caller-supplied session configuration for the realtime endpoint. type: object properties: - name: realtime_recommendation_instructions type: string title: Realtime Recommendation Instructions description: Instructions for the periodic recommendation step. examples: &ref_2 - >- Goal: help me close a sales call. Keep the guidance warm and concise. required: false - name: realtime_recommendation_frequency type: string title: RealtimeRecommendationFrequency description: >- How often the realtime recommendation step runs, measured in analyzed video. - ``high``: every 10 seconds of analyzed video - ``medium``: every 20 seconds - ``low``: every 30 seconds enumValues: - high - medium - low required: false - name: analysis_groups type: array title: Analysis Groups description: >- Top-level analysis surfaces to run for this session. When omitted, the default is audio + visual. The values the server allows are advertised on session.ready under supported_session_config_options.analysis_groups. Available options: audio, visual Example: ["audio","visual"] examples: &ref_3 - - audio - visual required: false headers: [] jsonPayloadSchema: additionalProperties: false description: >- Caller-supplied session configuration for the realtime endpoint. Send this message at any time during the session; the most recent message wins. Recommendations are opt-in: supply a non-empty ``realtime_recommendation_instructions`` to enable them, and ``realtime_recommendation_frequency`` to pace them. ``analysis_groups`` selects which top-level analysis surfaces are active for the session. examples: - analysis_groups: - audio - visual realtime_recommendation_frequency: medium realtime_recommendation_instructions: >- Goal: help me close a sales call. Keep the guidance warm and concise. properties: realtime_recommendation_instructions: anyOf: - type: string x-parser-schema-id: - type: 'null' x-parser-schema-id: description: Instructions for the periodic recommendation step. examples: *ref_2 title: Realtime Recommendation Instructions x-parser-schema-id: realtime_recommendation_frequency: &ref_32 description: >- How often the realtime recommendation step runs, measured in analyzed video. - ``high``: every 10 seconds of analyzed video - ``medium``: every 20 seconds - ``low``: every 30 seconds enum: - high - medium - low title: RealtimeRecommendationFrequency type: string x-parser-schema-id: RealtimeRecommendationFrequency analysis_groups: anyOf: - items: &ref_34 description: >- Realtime analysis surfaces that the caller can switch on or off. Selected per session, for the caller's own ``WS /v0/realtime/analyze`` session. Available options: visual, audio enum: - visual - audio title: AnalysisGroup type: string x-parser-schema-id: AnalysisGroup minItems: 1 type: array x-parser-schema-id: - type: 'null' x-parser-schema-id: description: >- Top-level analysis surfaces to run for this session. When omitted, the default is audio + visual. The values the server allows are advertised on session.ready under supported_session_config_options.analysis_groups. Available options: audio, visual Example: ["audio","visual"] examples: *ref_3 title: Analysis Groups x-parser-schema-id: title: Realtime Session Config type: object x-parser-schema-id: RealtimeSessionConfigV1 title: Realtime Session Config description: Caller-supplied session configuration for the realtime endpoint. example: |- { "analysis_groups": [ "audio", "visual" ], "realtime_recommendation_frequency": "medium", "realtime_recommendation_instructions": "Goal: help me close a sales call. Keep the guidance warm and concise." } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_session_config_v1_message - id: x-parser-message-name value: realtime_session_config_v1_message bindings: [] extensions: *ref_1 - &ref_51 id: transcript_handler title: Transcript_handler description: '' type: receive messages: - &ref_66 id: transcript_updated_v1_message payload: - name: Transcript Update description: Inbound ``transcript.updated`` text frame. type: object properties: - name: type type: string title: Type description: >- Discriminator identifying the inbound message kind. For a transcript update it is always ``transcript.updated``. required: false - name: transcript type: array title: Transcript description: >- Caller-supplied transcript as a list of ``{start, end, text, speaker}`` segments, in spoken order. The most recent value is stored for the session and rendered into the recommendation prompt. required: true properties: - name: start type: number title: Start description: Start time of the utterance, in seconds. examples: &ref_4 - 0.44 required: true - name: end type: number title: End description: End time of the utterance, in seconds. examples: &ref_5 - 2.28 required: true - name: text type: string title: Text description: The spoken text of this segment. examples: &ref_6 - Yeah, yeah, exactly. required: true - name: speaker type: integer title: Speaker description: Zero-based index identifying the speaker of this segment. examples: &ref_7 - 0 required: true headers: [] jsonPayloadSchema: description: >- Inbound ``transcript.updated`` text frame. Lets the caller provide (and update) a transcript during the session. The ``transcript`` payload is a list of :class:`RealtimeTranscriptSegment` (``{start, end, text, speaker}``) entries; the most recent value is stored for the session and rendered into the recommendation prompt. examples: - transcript: - end: 2.28 speaker: 0 start: 0.44 text: Yeah, yeah, exactly. - end: 5.88 speaker: 0 start: 3 text: Um I think that that makes sense. type: transcript.updated properties: type: const: transcript.updated default: transcript.updated description: >- Discriminator identifying the inbound message kind. For a transcript update it is always ``transcript.updated``. title: Type type: string x-parser-schema-id: transcript: description: >- Caller-supplied transcript as a list of ``{start, end, text, speaker}`` segments, in spoken order. The most recent value is stored for the session and rendered into the recommendation prompt. items: description: >- One segment of a realtime transcript: a single diarized utterance. The same shape is used everywhere a transcript appears — the caller-supplied ``transcript.updated`` frames, the server-streamed ``transcript.generated`` events, and the transcript handed to recommendation — so the transcript has a single representation throughout. properties: start: description: Start time of the utterance, in seconds. examples: *ref_4 title: Start type: number x-parser-schema-id: end: description: End time of the utterance, in seconds. examples: *ref_5 title: End type: number x-parser-schema-id: text: description: The spoken text of this segment. examples: *ref_6 title: Text type: string x-parser-schema-id: speaker: description: Zero-based index identifying the speaker of this segment. examples: *ref_7 title: Speaker type: integer x-parser-schema-id: required: - start - end - text - speaker title: Transcript Segment type: object x-parser-schema-id: RealtimeTranscriptSegment title: Transcript type: array x-parser-schema-id: required: - transcript title: Transcript Update type: object x-parser-schema-id: TranscriptUpdatedV1 title: Transcript Update description: Inbound ``transcript.updated`` text frame. example: |- { "transcript": [ { "end": 2.28, "speaker": 0, "start": 0.44, "text": "Yeah, yeah, exactly." }, { "end": 5.88, "speaker": 0, "start": 3, "text": "Um I think that that makes sense." } ], "type": "transcript.updated" } bindings: [] extensions: - id: x-parser-unique-object-id value: transcript_updated_v1_message - id: x-parser-message-name value: transcript_updated_v1_message bindings: [] extensions: *ref_1 - &ref_52 id: v1_realtime_output_coverage_degraded title: V1_realtime_output_coverage_degraded description: Coverage Degraded type: send messages: - &ref_67 id: coverage_degraded_v1_message payload: - name: Coverage Degraded description: >- Reports that one or more analysis windows were analyzed with partial visual coverage. Emitted when a window decodes materially less video than the window span while its audio runs to the end — typically a stream whose keyframe interval exceeds the analysis window (a static screen share, a long-GOP encoder). This is an informational notice, not an error: the session stays open, the windows were analyzed (audio plus whatever video decoded) and are billed normally. ``data.ranges`` lists the affected time ranges in absolute session-cumulative seconds; ``data.reason`` is currently always ``video_gap``. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_8 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``coverage.degraded``. required: false - name: data type: object title: CoverageDegradedV1Data description: >- Payload for the ``coverage.degraded`` event. Lists time ranges whose analysis windows carried materially less video than the window span — for example a screen share whose keyframes are sparser than the analysis window, where the audio decodes fully but the video decodes only a fraction. Unlike ``coverage.dropped``, these windows **were analyzed** (and are billed): the audio and any decodable video informed the analysis; only the visual coverage was partial. required: true properties: - name: ranges type: array title: Ranges description: >- Disjoint, ascending time ranges whose windows were analyzed with partial visual coverage. examples: &ref_9 - - end: 12 start: 8 required: true properties: - name: start type: number title: Start description: >- Start of the degraded range, in seconds, in absolute session-cumulative time. examples: &ref_10 - 8 required: true - name: end type: number title: End description: >- End of the degraded range, in seconds, in absolute session-cumulative time. examples: &ref_11 - 12 required: true - name: reason type: string title: Reason description: >- Why the visual coverage was partial. Currently always ``video_gap``: the windows' video packets covered materially less than the window span (e.g. a keyframe interval longer than the analysis window). required: false headers: [] jsonPayloadSchema: description: v1 ``coverage.degraded`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_8 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: coverage.degraded default: coverage.degraded description: >- Discriminator identifying the message kind. For this message it is always ``coverage.degraded``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the ``coverage.degraded`` event. Lists time ranges whose analysis windows carried materially less video than the window span — for example a screen share whose keyframes are sparser than the analysis window, where the audio decodes fully but the video decodes only a fraction. Unlike ``coverage.dropped``, these windows **were analyzed** (and are billed): the audio and any decodable video informed the analysis; only the visual coverage was partial. properties: ranges: description: >- Disjoint, ascending time ranges whose windows were analyzed with partial visual coverage. examples: *ref_9 items: description: One time range whose visual coverage was partial. properties: start: description: >- Start of the degraded range, in seconds, in absolute session-cumulative time. examples: *ref_10 title: Start type: number x-parser-schema-id: end: description: >- End of the degraded range, in seconds, in absolute session-cumulative time. examples: *ref_11 title: End type: number x-parser-schema-id: required: - start - end title: CoverageDegradedRangeV1 type: object x-parser-schema-id: CoverageDegradedRangeV1 title: Ranges type: array x-parser-schema-id: reason: const: video_gap default: video_gap description: >- Why the visual coverage was partial. Currently always ``video_gap``: the windows' video packets covered materially less than the window span (e.g. a keyframe interval longer than the analysis window). title: Reason type: string x-parser-schema-id: required: - ranges title: CoverageDegradedV1Data type: object x-parser-schema-id: CoverageDegradedV1Data required: - timestamp - data title: CoverageDegradedV1Message type: object x-parser-schema-id: CoverageDegradedV1Message title: Coverage Degraded description: >- Reports that one or more analysis windows were analyzed with partial visual coverage. Emitted when a window decodes materially less video than the window span while its audio runs to the end — typically a stream whose keyframe interval exceeds the analysis window (a static screen share, a long-GOP encoder). This is an informational notice, not an error: the session stays open, the windows were analyzed (audio plus whatever video decoded) and are billed normally. ``data.ranges`` lists the affected time ranges in absolute session-cumulative seconds; ``data.reason`` is currently always ``video_gap``. example: |- { "type": "coverage.degraded", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "ranges": [ { "start": 8, "end": 12 } ], "reason": "video_gap" } } bindings: [] extensions: - id: x-parser-unique-object-id value: coverage_degraded_v1_message - id: x-parser-message-name value: coverage_degraded_v1_message bindings: [] extensions: *ref_1 - &ref_53 id: v1_realtime_output_coverage_dropped title: V1_realtime_output_coverage_dropped description: Coverage Dropped type: send messages: - &ref_68 id: coverage_dropped_v1_message payload: - name: Coverage Dropped description: >- Reports that analysis coverage was reduced under backpressure. Emitted when the analysis pipeline saturates and has to skip buffered video. This is an informational notice, not an error: the session stays open, subsequent windows continue uninterrupted, and the dropped portions of the video are not billed. ``data.ranges`` lists the skipped time ranges in absolute session-cumulative seconds. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_12 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``coverage.dropped``. required: false - name: data type: object title: CoverageDroppedV1Data description: >- Payload for the ``coverage.dropped`` event. Lists the time ranges that the analysis pipeline skipped when it saturated and shed buffered video. Dropped portions of the video are expected, recoverable degradation — the session stays open and continues analyzing subsequent windows — and they are **not billed**. required: true properties: - name: ranges type: array title: Ranges description: >- Disjoint, ascending time ranges that were dropped without analysis. These portions of the video are not billed. examples: &ref_13 - - end: 10 start: 8 required: true properties: - name: start type: number title: Start description: >- Start of the dropped range, in seconds, in absolute session-cumulative time. examples: &ref_14 - 8 required: true - name: end type: number title: End description: >- End of the dropped range, in seconds, in absolute session-cumulative time. examples: &ref_15 - 10 required: true headers: [] jsonPayloadSchema: description: v1 ``coverage.dropped`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_12 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: coverage.dropped default: coverage.dropped description: >- Discriminator identifying the message kind. For this message it is always ``coverage.dropped``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the ``coverage.dropped`` event. Lists the time ranges that the analysis pipeline skipped when it saturated and shed buffered video. Dropped portions of the video are expected, recoverable degradation — the session stays open and continues analyzing subsequent windows — and they are **not billed**. properties: ranges: description: >- Disjoint, ascending time ranges that were dropped without analysis. These portions of the video are not billed. examples: *ref_13 items: description: >- A single contiguous time range skipped under analysis backpressure. Times are in seconds, in absolute session-cumulative time, matching the timestamps carried by every other v1 envelope. properties: start: description: >- Start of the dropped range, in seconds, in absolute session-cumulative time. examples: *ref_14 title: Start type: number x-parser-schema-id: end: description: >- End of the dropped range, in seconds, in absolute session-cumulative time. examples: *ref_15 title: End type: number x-parser-schema-id: required: - start - end title: CoverageDroppedRangeV1 type: object x-parser-schema-id: CoverageDroppedRangeV1 title: Ranges type: array x-parser-schema-id: required: - ranges title: CoverageDroppedV1Data type: object x-parser-schema-id: CoverageDroppedV1Data required: - timestamp - data title: CoverageDroppedV1Message type: object x-parser-schema-id: CoverageDroppedV1Message title: Coverage Dropped description: >- Reports that analysis coverage was reduced under backpressure. Emitted when the analysis pipeline saturates and has to skip buffered video. This is an informational notice, not an error: the session stays open, subsequent windows continue uninterrupted, and the dropped portions of the video are not billed. ``data.ranges`` lists the skipped time ranges in absolute session-cumulative seconds. example: |- { "type": "coverage.dropped", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "ranges": [ { "start": 8, "end": 10 } ] } } bindings: [] extensions: - id: x-parser-unique-object-id value: coverage_dropped_v1_message - id: x-parser-message-name value: coverage_dropped_v1_message bindings: [] extensions: *ref_1 - &ref_54 id: v1_realtime_output_error title: V1_realtime_output_error description: Error type: send messages: - &ref_69 id: stream_error_v1_message payload: - name: Error description: >- Reports an error encountered while processing the stream. ``data.code`` carries the machine-readable error id (sub-type), and ``data.segment`` identifies the incoming caller segment when the failure maps to a specific chunk (size validation, quota); analysis-time failures that do not map to one chunk carry ``data.segment: null``. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_16 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``error``. required: false - name: data type: object title: StreamErrorV1Data description: Payload for the ``error`` event. required: true properties: - name: code type: string title: Code description: Machine-readable error code. examples: &ref_17 - ih6002 required: true - name: message type: string title: Message description: >- Error explanation. Contains any additional details about the specific error that was encountered. required: true - name: link type: string title: Link description: URL to additional information about this error. required: false - name: segment type: integer title: Segment description: >- The incoming caller segment associated with this error, when applicable. ``null`` for errors that are not tied to one specific caller segment. required: false headers: [] jsonPayloadSchema: description: v1 ``error`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_16 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: error default: error description: >- Discriminator identifying the message kind. For this message it is always ``error``. title: Type type: string x-parser-schema-id: data: description: Payload for the ``error`` event. properties: code: description: Machine-readable error code. examples: *ref_17 title: Code type: string x-parser-schema-id: message: description: >- Error explanation. Contains any additional details about the specific error that was encountered. title: Message type: string x-parser-schema-id: link: anyOf: - type: string x-parser-schema-id: - type: 'null' x-parser-schema-id: description: URL to additional information about this error. title: Link x-parser-schema-id: segment: anyOf: - type: integer x-parser-schema-id: - type: 'null' x-parser-schema-id: description: >- The incoming caller segment associated with this error, when applicable. ``null`` for errors that are not tied to one specific caller segment. title: Segment x-parser-schema-id: required: - code - message title: StreamErrorV1Data type: object x-parser-schema-id: StreamErrorV1Data required: - timestamp - data title: StreamErrorV1Message type: object x-parser-schema-id: StreamErrorV1Message title: Error description: >- Reports an error encountered while processing the stream. ``data.code`` carries the machine-readable error id (sub-type), and ``data.segment`` identifies the incoming caller segment when the failure maps to a specific chunk (size validation, quota); analysis-time failures that do not map to one chunk carry ``data.segment: null``. example: |- { "type": "error", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "code": "ih6002", "message": "WebSocket message too large. Individual video chunks must not exceed 32 MB.", "link": "https://docs.interhuman.ai/api-reference/error-handling#ih6002-message-too-large", "segment": 2 } } bindings: [] extensions: - id: x-parser-unique-object-id value: stream_error_v1_message - id: x-parser-message-name value: stream_error_v1_message bindings: [] extensions: *ref_1 - &ref_55 id: output_realtime_recommendation_generated title: Output_realtime_recommendation_generated description: Recommendation Generated type: send messages: - &ref_70 id: realtime_recommendation_generated_v1_message payload: - name: Recommendation Generated description: >- Reports a periodic recommendation of the conversation so far. Recommendation is emitted only when all three conditions hold: the caller supplies ``realtime_recommendation_instructions`` in ``session.config``, at least one signal has been detected, and at least one transcript has been received since the previous recommendation. Once enabled it is paced by the session-config ``realtime_recommendation_frequency`` (``high``/``medium``/``low`` mapped to 10/20/30 seconds of analyzed video). type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_18 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``realtime_recommendation.generated``. required: false - name: data type: object title: RealtimeRecommendationGeneratedV1Data description: >- Payload for the ``realtime_recommendation.generated`` event. Carries the generated text returned by the recommendation LLM plus the start time of the analyzed interval the recommendation covers, in absolute session-cumulative seconds. required: true properties: - name: text type: string title: Text description: >- The guidance text returned by the recommendation model — one short recommendation for the responding speaker. examples: &ref_19 - >- Speaker 0 agreed but sounds unsure — invite them to name what still feels unresolved. required: true - name: start type: number title: Start description: >- Start of the analyzed interval this recommendation covers, in seconds, in absolute session-cumulative time. examples: &ref_20 - 0 required: true - name: end type: number title: End description: >- End of the analyzed interval this recommendation covers, in seconds, in absolute session-cumulative time. examples: &ref_21 - 20 required: true headers: [] jsonPayloadSchema: description: >- v1 ``realtime_recommendation.generated`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_18 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: realtime_recommendation.generated default: realtime_recommendation.generated description: >- Discriminator identifying the message kind. For this message it is always ``realtime_recommendation.generated``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the ``realtime_recommendation.generated`` event. Carries the generated text returned by the recommendation LLM plus the start time of the analyzed interval the recommendation covers, in absolute session-cumulative seconds. properties: text: description: >- The guidance text returned by the recommendation model — one short recommendation for the responding speaker. examples: *ref_19 title: Text type: string x-parser-schema-id: start: description: >- Start of the analyzed interval this recommendation covers, in seconds, in absolute session-cumulative time. examples: *ref_20 title: Start type: number x-parser-schema-id: end: description: >- End of the analyzed interval this recommendation covers, in seconds, in absolute session-cumulative time. examples: *ref_21 title: End type: number x-parser-schema-id: required: - text - start - end title: RealtimeRecommendationGeneratedV1Data type: object x-parser-schema-id: RealtimeRecommendationGeneratedV1Data required: - timestamp - data title: RealtimeRecommendationGeneratedV1Message type: object x-parser-schema-id: RealtimeRecommendationGeneratedV1Message title: Recommendation Generated description: >- Reports a periodic recommendation of the conversation so far. Recommendation is emitted only when all three conditions hold: the caller supplies ``realtime_recommendation_instructions`` in ``session.config``, at least one signal has been detected, and at least one transcript has been received since the previous recommendation. Once enabled it is paced by the session-config ``realtime_recommendation_frequency`` (``high``/``medium``/``low`` mapped to 10/20/30 seconds of analyzed video). example: |- { "type": "realtime_recommendation.generated", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "text": "Speaker 0 agreed but sounds unsure — invite them to name what still feels unresolved.", "start": 0, "end": 20 } } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_recommendation_generated_v1_message - id: x-parser-message-name value: realtime_recommendation_generated_v1_message bindings: [] extensions: *ref_1 - &ref_56 id: v1_realtime_output_session_closing title: V1_realtime_output_session_closing description: Session Closing type: send messages: - &ref_71 id: session_closing_v1_message payload: - name: Session Closing description: >- Acknowledges that an inbound ``session.close`` text frame was accepted. From this point the server rejects new binary video frames, finishes analyzing the video it already accepted (emitting the normal result envelopes in order), emits final lifecycle envelopes for still-open analysis state, sends ``session.ended``, and closes the WebSocket. ``data.max_drain_seconds`` is the maximum time the caller should wait for ``session.ended`` after this acknowledgment; the session closes earlier when the accepted work finishes sooner. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_22 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``session.closing``. required: false - name: data type: object title: SessionClosingV1Data description: >- Payload for the ``session.closing`` event. Carries the maximum time the caller should expect to wait between this acknowledgment and the final ``session.ended`` message + socket close. required: true properties: - name: max_drain_seconds type: integer title: Max Drain Seconds description: >- Maximum number of seconds the server will spend draining already-accepted analysis work before it closes the session. The session closes earlier when the accepted work finishes sooner; work still unfinished at this deadline is abandoned so the close is never delayed past it. Use this as the upper bound for a client-side wait on ``session.ended``. examples: &ref_23 - 60 required: true headers: [] jsonPayloadSchema: description: v1 ``session.closing`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_22 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: session.closing default: session.closing description: >- Discriminator identifying the message kind. For this message it is always ``session.closing``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the ``session.closing`` event. Carries the maximum time the caller should expect to wait between this acknowledgment and the final ``session.ended`` message + socket close. properties: max_drain_seconds: description: >- Maximum number of seconds the server will spend draining already-accepted analysis work before it closes the session. The session closes earlier when the accepted work finishes sooner; work still unfinished at this deadline is abandoned so the close is never delayed past it. Use this as the upper bound for a client-side wait on ``session.ended``. examples: *ref_23 minimum: 0 title: Max Drain Seconds type: integer x-parser-schema-id: required: - max_drain_seconds title: SessionClosingV1Data type: object x-parser-schema-id: SessionClosingV1Data required: - timestamp - data title: SessionClosingV1Message type: object x-parser-schema-id: SessionClosingV1Message title: Session Closing description: >- Acknowledges that an inbound ``session.close`` text frame was accepted. From this point the server rejects new binary video frames, finishes analyzing the video it already accepted (emitting the normal result envelopes in order), emits final lifecycle envelopes for still-open analysis state, sends ``session.ended``, and closes the WebSocket. ``data.max_drain_seconds`` is the maximum time the caller should wait for ``session.ended`` after this acknowledgment; the session closes earlier when the accepted work finishes sooner. example: |- { "type": "session.closing", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "max_drain_seconds": 60 } } bindings: [] extensions: - id: x-parser-unique-object-id value: session_closing_v1_message - id: x-parser-message-name value: session_closing_v1_message bindings: [] extensions: *ref_1 - &ref_57 id: v1_realtime_output_session_ended title: V1_realtime_output_session_ended description: Session Ended type: send messages: - &ref_72 id: session_ended_v1_message payload: - name: Session Ended description: >- The final message of a gracefully closed session. Emitted after every already-accepted analysis window has drained (or the ``max_drain_seconds`` deadline advertised on ``session.closing`` expired) and after the final lifecycle envelopes for still-open analysis state. No further analysis messages follow; the server closes the WebSocket (close code 1000) immediately after sending it. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_24 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``session.ended``. required: false - name: data type: object title: SessionEndedV1Data description: Payload for the ``session.ended`` event. required: true properties: - name: reason type: string title: SessionEndedReason description: Why the server ended the analysis session. enumValues: - client_shutdown required: true headers: [] jsonPayloadSchema: description: v1 ``session.ended`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_24 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: session.ended default: session.ended description: >- Discriminator identifying the message kind. For this message it is always ``session.ended``. title: Type type: string x-parser-schema-id: data: description: Payload for the ``session.ended`` event. properties: reason: description: Why the server ended the analysis session. enum: - client_shutdown title: SessionEndedReason type: string x-parser-schema-id: SessionEndedReason required: - reason title: SessionEndedV1Data type: object x-parser-schema-id: SessionEndedV1Data required: - timestamp - data title: SessionEndedV1Message type: object x-parser-schema-id: SessionEndedV1Message title: Session Ended description: >- The final message of a gracefully closed session. Emitted after every already-accepted analysis window has drained (or the ``max_drain_seconds`` deadline advertised on ``session.closing`` expired) and after the final lifecycle envelopes for still-open analysis state. No further analysis messages follow; the server closes the WebSocket (close code 1000) immediately after sending it. example: |- { "type": "session.ended", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "reason": "client_shutdown" } } bindings: [] extensions: - id: x-parser-unique-object-id value: session_ended_v1_message - id: x-parser-message-name value: session_ended_v1_message bindings: [] extensions: *ref_1 - &ref_58 id: v1_realtime_output_session_ready title: V1_realtime_output_session_ready description: Session Ready type: send messages: - &ref_73 id: realtime_session_ready_v1_message payload: - name: Session Ready description: >- Acknowledges that the v1 realtime session is established. Emitted exactly once per session, immediately after the server accepts the WebSocket handshake. Carries the session-level contract (idle / max-duration timeouts, segment size constraints, supported session-config options) so the caller can adapt its producer side without round-tripping rejections. ``supported_session_config_options`` advertises the values the server accepts for ``realtime_recommendation_instructions``, ``realtime_recommendation_frequency``, and ``analysis_groups``. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_25 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``session.ready``. required: false - name: data type: object title: RealtimeSessionReadyV1Data description: >- Payload for the realtime ``session.ready`` event. Carries the connection-level limits and advertises the realtime session-config options the server accepts. required: true properties: - name: session_idle_timeout_seconds type: integer title: Session Idle Timeout Seconds description: >- Number of seconds the server allows the session to stay idle (no inbound frames) before it closes the connection with ``ih6003``. A value of ``0`` indicates the idle timeout is disabled for this session. examples: &ref_26 - 300 required: true - name: session_max_duration_seconds type: integer title: Session Max Duration Seconds description: >- Maximum total duration in seconds the server allows the session to remain open before closing the connection with ``ih6004``. A value of ``0`` indicates the maximum duration is disabled for this session. examples: &ref_27 - 3600 required: true - name: max_segment_duration_seconds type: number title: Max Segment Duration Seconds description: >- Maximum probed duration in seconds of a single inbound video chunk, or ``null`` when no fixed duration cap applies (the size cap in ``max_segment_size_bytes`` is the binding limit for long chunks). required: false - name: min_segment_size_bytes type: integer title: Min Segment Size Bytes description: >- Minimum size in bytes of a single inbound video chunk. Zero-byte frames are rejected and never reach analysis. examples: &ref_28 - 1 required: true - name: max_segment_size_bytes type: integer title: Max Segment Size Bytes description: >- Maximum size in bytes of a single inbound video chunk. Chunks larger than this are rejected with ``ih6002``. examples: &ref_29 - 33554432 required: true - name: supported_session_config_options type: object title: Realtime Supported Session Config Options description: >- Supported session-config options advertised on the realtime ``session.ready``. Mirrors the field names of the Realtime Session Config message. required: true properties: - name: realtime_recommendation_instructions type: string title: Realtime Recommendation Instructions description: >- Accepted type for the realtime_recommendation_instructions field in the Realtime Session Config message. examples: &ref_30 - string required: false - name: realtime_recommendation_frequency type: array title: Realtime Recommendation Frequency description: >- Supported values for the realtime_recommendation_frequency field in the Realtime Session Config message. examples: &ref_31 - - high - medium - low required: true properties: - name: item type: string title: RealtimeRecommendationFrequency description: >- How often the realtime recommendation step runs, measured in analyzed video. - ``high``: every 10 seconds of analyzed video - ``medium``: every 20 seconds - ``low``: every 30 seconds enumValues: - high - medium - low required: false - name: analysis_groups type: array title: Analysis Groups description: >- Analysis surfaces the caller may select for the analysis_groups field in the Realtime Session Config message. Example: ["visual","audio"] examples: &ref_33 - - visual - audio required: true properties: - name: item type: string title: AnalysisGroup description: >- Realtime analysis surfaces that the caller can switch on or off. Selected per session, for the caller's own ``WS /v0/realtime/analyze`` session. Available options: visual, audio enumValues: - visual - audio required: false headers: [] jsonPayloadSchema: description: v1 realtime ``session.ready`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_25 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: session.ready default: session.ready description: >- Discriminator identifying the message kind. For this message it is always ``session.ready``. title: Type type: string x-parser-schema-id: data: description: |- Payload for the realtime ``session.ready`` event. Carries the connection-level limits and advertises the realtime session-config options the server accepts. properties: session_idle_timeout_seconds: description: >- Number of seconds the server allows the session to stay idle (no inbound frames) before it closes the connection with ``ih6003``. A value of ``0`` indicates the idle timeout is disabled for this session. examples: *ref_26 minimum: 0 title: Session Idle Timeout Seconds type: integer x-parser-schema-id: session_max_duration_seconds: description: >- Maximum total duration in seconds the server allows the session to remain open before closing the connection with ``ih6004``. A value of ``0`` indicates the maximum duration is disabled for this session. examples: *ref_27 minimum: 0 title: Session Max Duration Seconds type: integer x-parser-schema-id: max_segment_duration_seconds: anyOf: - type: number x-parser-schema-id: - type: 'null' x-parser-schema-id: description: >- Maximum probed duration in seconds of a single inbound video chunk, or ``null`` when no fixed duration cap applies (the size cap in ``max_segment_size_bytes`` is the binding limit for long chunks). title: Max Segment Duration Seconds x-parser-schema-id: min_segment_size_bytes: description: >- Minimum size in bytes of a single inbound video chunk. Zero-byte frames are rejected and never reach analysis. examples: *ref_28 minimum: 1 title: Min Segment Size Bytes type: integer x-parser-schema-id: max_segment_size_bytes: description: >- Maximum size in bytes of a single inbound video chunk. Chunks larger than this are rejected with ``ih6002``. examples: *ref_29 minimum: 1 title: Max Segment Size Bytes type: integer x-parser-schema-id: supported_session_config_options: description: >- Supported session-config options advertised on the realtime ``session.ready``. Mirrors the field names of the Realtime Session Config message. properties: realtime_recommendation_instructions: const: string default: string description: >- Accepted type for the realtime_recommendation_instructions field in the Realtime Session Config message. examples: *ref_30 title: Realtime Recommendation Instructions type: string x-parser-schema-id: realtime_recommendation_frequency: description: >- Supported values for the realtime_recommendation_frequency field in the Realtime Session Config message. examples: *ref_31 items: *ref_32 title: Realtime Recommendation Frequency type: array x-parser-schema-id: analysis_groups: description: >- Analysis surfaces the caller may select for the analysis_groups field in the Realtime Session Config message. Example: ["visual","audio"] examples: *ref_33 items: *ref_34 title: Analysis Groups type: array x-parser-schema-id: required: - realtime_recommendation_frequency - analysis_groups title: Realtime Supported Session Config Options type: object x-parser-schema-id: Realtime Supported Session Config Options required: - session_idle_timeout_seconds - session_max_duration_seconds - min_segment_size_bytes - max_segment_size_bytes - supported_session_config_options title: RealtimeSessionReadyV1Data type: object x-parser-schema-id: RealtimeSessionReadyV1Data required: - timestamp - data title: RealtimeSessionReadyV1Message type: object x-parser-schema-id: RealtimeSessionReadyV1Message title: Session Ready description: >- Acknowledges that the v1 realtime session is established. Emitted exactly once per session, immediately after the server accepts the WebSocket handshake. Carries the session-level contract (idle / max-duration timeouts, segment size constraints, supported session-config options) so the caller can adapt its producer side without round-tripping rejections. ``supported_session_config_options`` advertises the values the server accepts for ``realtime_recommendation_instructions``, ``realtime_recommendation_frequency``, and ``analysis_groups``. example: |- { "type": "session.ready", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "session_idle_timeout_seconds": 300, "session_max_duration_seconds": 3600, "min_segment_size_bytes": 1, "max_segment_size_bytes": 33554432, "supported_session_config_options": { "realtime_recommendation_instructions": "string", "realtime_recommendation_frequency": [ "high", "medium", "low" ], "analysis_groups": [ "visual", "audio" ] } } } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_session_ready_v1_message - id: x-parser-message-name value: realtime_session_ready_v1_message bindings: [] extensions: *ref_1 - &ref_59 id: v1_realtime_output_session_updated title: V1_realtime_output_session_updated description: Session Updated type: send messages: - &ref_74 id: realtime_session_updated_v1_message payload: - name: Session Updated description: >- Acknowledges that an inbound ``realtime_session_config_v1`` text frame was accepted. Carries the consolidated post-apply realtime config (``realtime_recommendation_instructions`` / ``realtime_recommendation_frequency`` / ``analysis_groups``). type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_35 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``session.updated``. required: false - name: data type: object title: RealtimeSessionUpdatedV1Data description: >- Payload for the realtime ``session.updated`` event. Carries the consolidated realtime session config that was just accepted, so the caller has a single source of truth for what the server will apply to subsequent recommendation runs. Each successful inbound Realtime Session Config message fully replaces the active config, so these values reflect the post-apply state of the session, not a delta against the previous config. required: true properties: - name: realtime_recommendation_instructions type: string title: Realtime Recommendation Instructions description: >- The ``realtime_recommendation_instructions`` after the config was applied, echoed back verbatim — ``null`` when none was set. Recommendation is enabled only by a non-empty value, so a ``null``, empty, or whitespace-only value means recommendations are disabled and no ``realtime_recommendation.generated`` messages are emitted. examples: &ref_36 - >- Goal: help me close a sales call. Keep the guidance warm and concise. required: false - name: realtime_recommendation_frequency type: string title: RealtimeRecommendationFrequency description: >- How often the realtime recommendation step runs, measured in analyzed video. - ``high``: every 10 seconds of analyzed video - ``medium``: every 20 seconds - ``low``: every 30 seconds enumValues: - high - medium - low required: false - name: analysis_groups type: array title: Analysis Groups description: >- The analysis surfaces active for the session after the config was applied. Available options: visual, audio Example: ["audio","visual"] examples: &ref_37 - - audio - visual required: false properties: - name: item type: string title: AnalysisGroup description: >- Realtime analysis surfaces that the caller can switch on or off. Selected per session, for the caller's own ``WS /v0/realtime/analyze`` session. Available options: visual, audio enumValues: - visual - audio required: false headers: [] jsonPayloadSchema: description: v1 realtime ``session.updated`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_35 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: session.updated default: session.updated description: >- Discriminator identifying the message kind. For this message it is always ``session.updated``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the realtime ``session.updated`` event. Carries the consolidated realtime session config that was just accepted, so the caller has a single source of truth for what the server will apply to subsequent recommendation runs. Each successful inbound Realtime Session Config message fully replaces the active config, so these values reflect the post-apply state of the session, not a delta against the previous config. properties: realtime_recommendation_instructions: anyOf: - type: string x-parser-schema-id: - type: 'null' x-parser-schema-id: description: >- The ``realtime_recommendation_instructions`` after the config was applied, echoed back verbatim — ``null`` when none was set. Recommendation is enabled only by a non-empty value, so a ``null``, empty, or whitespace-only value means recommendations are disabled and no ``realtime_recommendation.generated`` messages are emitted. examples: *ref_36 title: Realtime Recommendation Instructions x-parser-schema-id: realtime_recommendation_frequency: *ref_32 analysis_groups: description: >- The analysis surfaces active for the session after the config was applied. Available options: visual, audio Example: ["audio","visual"] examples: *ref_37 items: *ref_34 title: Analysis Groups type: array x-parser-schema-id: title: RealtimeSessionUpdatedV1Data type: object x-parser-schema-id: RealtimeSessionUpdatedV1Data required: - timestamp - data title: RealtimeSessionUpdatedV1Message type: object x-parser-schema-id: RealtimeSessionUpdatedV1Message title: Session Updated description: >- Acknowledges that an inbound ``realtime_session_config_v1`` text frame was accepted. Carries the consolidated post-apply realtime config (``realtime_recommendation_instructions`` / ``realtime_recommendation_frequency`` / ``analysis_groups``). example: |- { "type": "session.updated", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "realtime_recommendation_instructions": "Goal: help me close a sales call. Keep the guidance warm and concise.", "realtime_recommendation_frequency": "medium", "analysis_groups": [ "audio", "visual" ] } } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_session_updated_v1_message - id: x-parser-message-name value: realtime_session_updated_v1_message bindings: [] extensions: *ref_1 - &ref_60 id: v1_realtime_output_signal_detected title: V1_realtime_output_signal_detected description: Signal Detected type: send messages: - &ref_75 id: realtime_signal_detected_v1_message payload: - name: Signal Detected description: >- Reports that a social signal has transitioned from inactive to active. Emitted once when the signal type is first detected for the session, and again only when the signal becomes active after a prior ``signal.ended``. ``data.start`` uses absolute session-cumulative time. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_38 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``signal.detected``. required: false - name: data type: object title: RealtimeSignalDetectedV1Data description: >- Payload for the realtime ``signal.detected`` event. Carries only a start time — no window end — because the envelope marks the moment the signal transitions from inactive to active. The matching ``signal.ended`` envelope reports the end-time when the signal stops being detected. required: true properties: - name: signal_type type: string title: SignalType description: Enumeration of supported social signals. enumValues: - agreement - confidence - confusion - disagreement - disengagement - engagement - frustration - hesitation - interest - skepticism - stress - tension - uncertainty required: true - name: start type: number title: Start description: >- Time at which the signal first becomes active, in seconds, in absolute session-cumulative time. examples: &ref_39 - 3 required: true - name: probability type: string title: Probability description: Signal probability levels. enumValues: - high - medium - low required: false - name: modality type: array title: Modality description: >- Analysis modalities that detected this signal, naming the source of the evidence: one or both of `audio` and `visual`. When both detect the same signal, both are included. examples: &ref_40 - - audio - visual required: false properties: - name: item type: string required: false headers: [] jsonPayloadSchema: description: v1 realtime ``signal.detected`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_38 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: signal.detected default: signal.detected description: >- Discriminator identifying the message kind. For this message it is always ``signal.detected``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the realtime ``signal.detected`` event. Carries only a start time — no window end — because the envelope marks the moment the signal transitions from inactive to active. The matching ``signal.ended`` envelope reports the end-time when the signal stops being detected. properties: signal_type: &ref_42 description: Enumeration of supported social signals. enum: - agreement - confidence - confusion - disagreement - disengagement - engagement - frustration - hesitation - interest - skepticism - stress - tension - uncertainty title: SignalType type: string x-parser-schema-id: SignalType start: description: >- Time at which the signal first becomes active, in seconds, in absolute session-cumulative time. examples: *ref_39 title: Start type: number x-parser-schema-id: probability: &ref_46 description: Signal probability levels. enum: - high - medium - low title: Probability type: string x-parser-schema-id: Probability modality: description: >- Analysis modalities that detected this signal, naming the source of the evidence: one or both of `audio` and `visual`. When both detect the same signal, both are included. examples: *ref_40 items: type: string x-parser-schema-id: title: Modality type: array x-parser-schema-id: required: - signal_type - start title: RealtimeSignalDetectedV1Data type: object x-parser-schema-id: RealtimeSignalDetectedV1Data required: - timestamp - data title: RealtimeSignalDetectedV1Message type: object x-parser-schema-id: RealtimeSignalDetectedV1Message title: Signal Detected description: >- Reports that a social signal has transitioned from inactive to active. Emitted once when the signal type is first detected for the session, and again only when the signal becomes active after a prior ``signal.ended``. ``data.start`` uses absolute session-cumulative time. example: |- { "type": "signal.detected", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "signal_type": "agreement", "start": 3, "probability": "high", "modality": [ "audio", "visual" ] } } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_signal_detected_v1_message - id: x-parser-message-name value: realtime_signal_detected_v1_message bindings: [] extensions: *ref_1 - &ref_61 id: v1_realtime_output_signal_ended title: V1_realtime_output_signal_ended description: Signal Ended type: send messages: - &ref_76 id: signal_ended_v1_message payload: - name: Signal Ended description: >- Reports that a previously active social signal is no longer active. Emitted when a signal type that was active in the previous analyzed window is not present in the current window. ``data.end`` uses absolute session-cumulative time. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_41 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``signal.ended``. required: false - name: data type: object title: SignalEndedV1Data description: >- Payload for the ``signal.ended`` event. Carries only an end time — no start or window — because the envelope marks the moment the signal transitions from active to inactive. The matching ``signal.detected`` envelope reported the start-time when the signal became active. required: true properties: - name: signal_type type: string title: SignalType description: Enumeration of supported social signals. enumValues: - agreement - confidence - confusion - disagreement - disengagement - engagement - frustration - hesitation - interest - skepticism - stress - tension - uncertainty required: true - name: end type: number title: End description: >- Time at which the signal stopped being active, in seconds, in absolute session-cumulative time. examples: &ref_43 - 14 required: true headers: [] jsonPayloadSchema: description: v1 ``signal.ended`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_41 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: signal.ended default: signal.ended description: >- Discriminator identifying the message kind. For this message it is always ``signal.ended``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the ``signal.ended`` event. Carries only an end time — no start or window — because the envelope marks the moment the signal transitions from active to inactive. The matching ``signal.detected`` envelope reported the start-time when the signal became active. properties: signal_type: *ref_42 end: description: >- Time at which the signal stopped being active, in seconds, in absolute session-cumulative time. examples: *ref_43 title: End type: number x-parser-schema-id: required: - signal_type - end title: SignalEndedV1Data type: object x-parser-schema-id: SignalEndedV1Data required: - timestamp - data title: SignalEndedV1Message type: object x-parser-schema-id: SignalEndedV1Message title: Signal Ended description: >- Reports that a previously active social signal is no longer active. Emitted when a signal type that was active in the previous analyzed window is not present in the current window. ``data.end`` uses absolute session-cumulative time. example: |- { "type": "signal.ended", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "signal_type": "agreement", "end": 14 } } bindings: [] extensions: - id: x-parser-unique-object-id value: signal_ended_v1_message - id: x-parser-message-name value: signal_ended_v1_message bindings: [] extensions: *ref_1 - &ref_62 id: v1_realtime_output_signal_updated title: V1_realtime_output_signal_updated description: Signal Updated type: send messages: - &ref_77 id: realtime_signal_updated_v1_message payload: - name: Signal Updated description: >- Reports that an already-active social signal experienced a change in probability. Emitted when an active signal type's ``probability`` has changed from the previous value reported for that signal type (either ``signal.detected`` or a prior ``signal.updated``). ``data.start`` uses absolute session-cumulative time. type: object properties: - name: timestamp type: string title: Timestamp description: ISO 8601 timestamp identifying when the event occurred. examples: &ref_44 - '2025-01-01T00:00:00.000000Z' required: true - name: correlation_id type: string title: Correlation Id description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. required: false - name: type type: string title: Type description: >- Discriminator identifying the message kind. For this message it is always ``signal.updated``. required: false - name: data type: object title: RealtimeSignalUpdatedV1Data description: >- Payload for the realtime ``signal.updated`` event. Emitted while the signal remains active and its probability or set of modalities has changed since the most recently emitted envelope for the same ``signal_type``. Carries the same fields as ``signal.detected``: ``start`` is the moment from which the new public state applies. required: true properties: - name: signal_type type: string title: SignalType description: Enumeration of supported social signals. enumValues: - agreement - confidence - confusion - disagreement - disengagement - engagement - frustration - hesitation - interest - skepticism - stress - tension - uncertainty required: true - name: start type: number title: Start description: >- Time at which the new public state begins to apply, in seconds, in absolute session-cumulative time. examples: &ref_45 - 6 required: true - name: probability type: string title: Probability description: Signal probability levels. enumValues: - high - medium - low required: false - name: modality type: array title: Modality description: >- Analysis modalities that detected this signal, naming the source of the evidence: one or both of `audio` and `visual`. When both detect the same signal, both are included. examples: &ref_47 - - audio - visual required: false properties: - name: item type: string required: false headers: [] jsonPayloadSchema: description: v1 realtime ``signal.updated`` envelope (chanx routing wrapper). properties: timestamp: description: ISO 8601 timestamp identifying when the event occurred. examples: *ref_44 title: Timestamp type: string x-parser-schema-id: correlation_id: default: '' description: >- Unique identifier assigned to this WebSocket connection. Use it when contacting support to locate the relevant records. title: Correlation Id type: string x-parser-schema-id: type: const: signal.updated default: signal.updated description: >- Discriminator identifying the message kind. For this message it is always ``signal.updated``. title: Type type: string x-parser-schema-id: data: description: >- Payload for the realtime ``signal.updated`` event. Emitted while the signal remains active and its probability or set of modalities has changed since the most recently emitted envelope for the same ``signal_type``. Carries the same fields as ``signal.detected``: ``start`` is the moment from which the new public state applies. properties: signal_type: *ref_42 start: description: >- Time at which the new public state begins to apply, in seconds, in absolute session-cumulative time. examples: *ref_45 title: Start type: number x-parser-schema-id: probability: *ref_46 modality: description: >- Analysis modalities that detected this signal, naming the source of the evidence: one or both of `audio` and `visual`. When both detect the same signal, both are included. examples: *ref_47 items: type: string x-parser-schema-id: title: Modality type: array x-parser-schema-id: required: - signal_type - start title: RealtimeSignalUpdatedV1Data type: object x-parser-schema-id: RealtimeSignalUpdatedV1Data required: - timestamp - data title: RealtimeSignalUpdatedV1Message type: object x-parser-schema-id: RealtimeSignalUpdatedV1Message title: Signal Updated description: >- Reports that an already-active social signal experienced a change in probability. Emitted when an active signal type's ``probability`` has changed from the previous value reported for that signal type (either ``signal.detected`` or a prior ``signal.updated``). ``data.start`` uses absolute session-cumulative time. example: |- { "type": "signal.updated", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "signal_type": "agreement", "start": 6, "probability": "medium", "modality": [ "audio", "visual" ] } } bindings: [] extensions: - id: x-parser-unique-object-id value: realtime_signal_updated_v1_message - id: x-parser-message-name value: realtime_signal_updated_v1_message bindings: [] extensions: *ref_1 sendOperations: - *ref_48 - *ref_49 - *ref_50 - *ref_51 receiveOperations: - *ref_52 - *ref_53 - *ref_54 - *ref_55 - *ref_56 - *ref_57 - *ref_58 - *ref_59 - *ref_60 - *ref_61 - *ref_62 sendMessages: - *ref_63 - *ref_64 - *ref_65 - *ref_66 receiveMessages: - *ref_67 - *ref_68 - *ref_69 - *ref_70 - *ref_71 - *ref_72 - *ref_73 - *ref_74 - *ref_75 - *ref_76 - *ref_77 extensions: - id: x-parser-unique-object-id value: realtime_analyze_v1 securitySchemes: - id: bearerAuth name: bearerAuth type: http description: >- API key authentication. Include your API key in the Authorization header as 'Bearer '. scheme: bearer extensions: [] ````