# Authentication Source: https://docs.interhuman.ai/api-reference/authentication Interhuman now supports direct API-key authentication for API requests. Use your API key in the `Authorization` header: ```http theme={null} Authorization: Bearer ``` If you still need a key, start with [Get an API key](/how-to/get-api-key). ## Migration from the legacy flow We migrated from a two-step authentication flow to a one-step flow: * **Legacy flow:** send `key_id` and `key_secret` to `/auth`, receive a short-lived token, then call API endpoints with that token. * **Current flow:** call API endpoints directly with your API key in `Authorization: Bearer `. # Client Tokens Source: https://docs.interhuman.ai/api-reference/client-tokens post /v1/client_tokens Mint a short-lived, capped client token for direct browser use. Call this server-side with your API key; the returned token grants the requested scopes (default 'interhumanai.stream') and can be handed to a browser to call the upload, stream, and realtime endpoints — e.g. POST /v1/upload/analyze, or open wss://.../v1/stream/analyze or wss://.../v0/realtime/analyze with the 'Sec-WebSocket-Protocol: access_token, ' subprotocol pair (a supported alternative to the 'Authorization' header, for browsers that cannot set it). The embedded caps are enforced by the API across all three (the video budget spans upload, stream, and realtime). #### Response Headers Unique identifier for the request. Include this when contacting support. Example: `f47ac10b-58cc-4372-a567-0e02b2c3d479` # Error handling Source: https://docs.interhuman.ai/api-reference/error-handling Common error formats and troubleshooting tips. **Interhuman APIs return structured JSON error responses so you can detect, log, and recover from failures programmatically.** If you run into issues you cannot explain, please contact us at [support@interhuman.ai](mailto:support@interhuman.ai) and include the `error_id` and `correlation_id` from the response, the endpoint you called, an approximate timestamp, and if possible a redacted example of the request. This information lets us quickly trace the failing request in our systems. ## Structured error shape ```json theme={null} { "error_id": "ih2005", "correlation_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479", "link": "https://docs.interhuman.ai/api-reference/error-handling#ih2005-missing-key-secret", "message": "The `key_secret` field is absent from the request body. This field is required to authenticate your API key." } ``` Each field has a specific purpose: * `error_id`: A stable machine-readable error code (for example, `ih2001`). You can use this to branch your client logic. * `correlation_id`: A unique identifier for the failing request. Log this value and include it in any support requests so we can trace the request in our systems. * `link`: A URL that points directly to the documentation section for this specific error. You can open this link from your terminal, logs, or browser to jump to the docs for that error and see more context and remediation steps. * `message`: A human-readable explanation you can show in logs or user-facing error messages. ## Error catalog The catalog below lists all current error IDs (`error_id` values), grouped by category. You can search this page for a specific `error_id` (for example, `ih2001`) to jump straight to its description. ### General #### ih1001 - Internal Server Error This error indicates that our service encountered an unexpected condition while processing your request. This means the request was received and understood, but we were unable to complete it due to an internal issue. In most cases, this is temporary and may be resolved by retrying the request after a short delay. We recommend implementing retry logic with exponential backoff. If you encounter this error, it generally reflects a problem on our side rather than an issue with your request, unless otherwise indicated in the response body. #### ih1002 - Service Temporarily Unavailable This error indicates that our service is currently unable to process your request due to temporary conditions such as maintenance, system overload, or a dependency outage. This status is typically transient. We recommend retrying the request after a short delay and implementing retry logic with exponential backoff. When provided, please respect the Retry-After header, which indicates how long to wait before attempting the request again. If the issue persists beyond a reasonable period, contact support so we can investigate further. #### ih1003 - Dependency Unavailable A required backend dependency (such as our database) is temporarily unavailable. The request was received and authenticated, but we could not reach the dependency needed to complete it. This status is typically transient. We recommend retrying the request after a short delay and implementing retry logic with exponential backoff. If the issue persists beyond a reasonable period, contact support so we can investigate further. *** ### Authentication and authorization #### ih2001 - Invalid Credentials The provided credentials are invalid. The API key submitted in `Authorization: Bearer ` does not match an active key. For security reasons we do not reveal whether the key was unrecognized, disabled, or revoked. ##### How to fix 1. Verify your API key is correct. 2. Ensure you are not using a rotated, disabled, or revoked key. 3. If you have recently rotated your credentials, use the new value. 4. Contact support if the problem persists. #### ih2002 - Missing Credentials No authentication credential was found in the `Authorization` header (for HTTP requests) or the `Sec-WebSocket-Protocol` header (for WebSocket connections). ##### How to fix 1. Include your API key in the `Authorization` header as `Bearer `. 2. WebSocket connections accept the same `Authorization` header, or — for clients that cannot set it, such as browsers — the `Sec-WebSocket-Protocol: access_token, ` subprotocol pair. Both schemes are supported, and either carries a JWT or a client token in place of the API key. #### ih2003 - Insufficient Scope The API key is valid but is not provisioned with the scope required by this endpoint. ##### How to fix * Have the key reissued or updated with the required scope. #### ih2013 - Origin Not Allowed The client token restricts which browser origins may use it, and this connection's `Origin` is absent or not on the token's allow-list. ##### How to fix * Connect from an origin included in the token's `allowed_origins`. * When minting the token, include every origin the browser will connect from in `allowed_origins` (or omit it to allow any origin). #### ih2014 - Token Revoked The client token was revoked by the backend that minted it, so it can no longer be used. ##### How to fix * Mint a new client token and reconnect with it. *** ### Rate limits and quotas #### ih3001 - Rate Limit Exceeded You have exceeded the configured requests-per-minute limit for your account. ##### How to fix 1. Reduce the frequency of your requests. 2. Implement retry logic with exponential backoff. 3. Respect the `Retry-After` response header. 4. Contact support if you need a higher rate limit. #### ih3002 - Concurrent Request Limit You have exceeded the maximum number of concurrent in-flight requests allowed for your account. ##### How to fix * Wait for one or more active requests to complete before submitting another. Consider implementing a request queue to manage concurrency. #### ih3003 - Quota Exceeded Your account has exhausted its analysis quota (e.g. total video-minutes for the current billing period or credit allowance). ##### How to fix * Contact support to upgrade your plan, purchase additional credits, or wait for the quota to reset at the start of the next billing period. #### ih3004 - Daily Quota Exceeded Your account has exhausted its daily analysis quota. ##### How to fix * Wait for the quota to reset (the `reset_time` is included in the response) or contact support to increase your daily limit. #### ih3005 - Inactive Subscription Your subscription is inactive or your billing information needs to be updated. ##### How to fix * Contact support to reactivate your subscription or update your billing information. #### ih3006 - Token Video Budget Exhausted The client token has a per-token limit on the total seconds of video it may process (across upload, stream, and realtime), and that budget is now used up. ##### How to fix * Mint a new client token, optionally with a larger `max_video_seconds` budget, and reconnect with it. *** ### Input validation #### ih4001 - Missing Video File No video file was included in the multipart form data, or the field name is incorrect. ##### How to fix * Include the video file as a `multipart/form-data` field named `file`. Ensure the `Content-Type` header is set to `multipart/form-data`. #### ih4002 - Unsupported File Type The file extension or detected content type is not in the list of supported formats. ##### How to fix * Re-encode or convert the video to a supported format. Supported formats include: MP4, AVI, MOV, MKV, WebM. #### ih4003 - File Too Large The uploaded file exceeds the maximum upload size. ##### How to fix * Reduce the file size by compressing the video, lowering the resolution, or trimming the duration. #### ih4004 - Video Too Long The video duration exceeds the maximum allowed length for a single request. ##### How to fix * Split the video into shorter segments and submit each one separately. #### ih4005 - Invalid Parameter A query parameter or body field has a value that is syntactically valid but outside its allowed range or fails semantic validation. ##### How to fix * Review the parameter value mentioned in the error message and ensure it falls within the documented range. #### ih4006 - Invalid JSON The request body or a WebSocket control message could not be parsed as valid JSON. ##### How to fix * Ensure the `Content-Type` header is `application/json` and the body is well-formed JSON. Validate your JSON payload with a linter before sending. #### ih4007 - Video Too Short The video is shorter than the minimum supported duration (3 seconds) or contains too little data to be analyzed. Recordings below this threshold cannot be analyzed. ##### How to fix * Ensure the video is at least 3 seconds long. *** ### Content and media issues #### ih5001 - Empty Video There is no video for the analysis to read. On an upload, the file reports a duration of zero or no determinable duration at all, which typically means an empty file or a container with no media tracks. On a Stream or Realtime session, the media carries an audio track but no video track, so no window has a picture to analyze; the session stays open and reports this once, then analyzes nothing further until a video track arrives. On Realtime the message names the enabled analysis that requires a picture, since a session keeping the `visual` group selected cannot be served by audio alone. ##### How to fix * On an upload, verify the video file plays correctly in a media player before uploading. Re-record or re-encode the file if necessary. * On a stream, check that the recording was started from a source that includes video — for example a `getUserMedia` call whose constraints request `video`, and a camera whose permission was granted. Restarting the recorder with a video track resumes analysis. * On Realtime, either send video as above, or send a session config narrowing `analysis_groups` to the audio-only selection the error message names. Analysis resumes from the next window. #### ih5002 - Corrupted Video The video file reports a negative duration or the container structure cannot be inspected, indicating corruption or an unrecognizable format. ##### How to fix * Re-encode the video using a standard tool (e.g. ffmpeg) with a supported codec and container format. #### ih5003 - Unsupported Codec The video codec or container format is not supported and cannot be decoded. ##### How to fix * Re-encode the video using a widely supported codec and container, such as H.264 video in an MP4 container with AAC audio. #### ih5004 - Malformed Video Segment A video segment is structurally invalid: it is missing required container atoms and cannot be processed. ##### How to fix * Verify the video file is complete and not truncated. Re-encode the file using a tool like ffmpeg with the `-movflags +faststart` flag. #### ih5005 - Video Segment Too Large A video segment exceeds the per-segment size cap enforced before analysis. The maximum segment size is 32MB. ##### How to fix * Reduce the video resolution or bitrate before uploading, or split the video into shorter segments. #### ih5006 - Video Segment Too Short A video segment is shorter than the minimum supported duration (3 seconds) or contains too little data to be analyzed. ##### How to fix * Send chunks that each contain at least 3 seconds of media. #### ih5007 - Content Rejected by Model The analysis service rejected the video content. This may occur due to content policy filtering or an unsupported media payload. ##### How to fix 1. Verify the video is valid and does not violate the service's usage policies. 2. Try re-encoding the video if the issue persists. *** ### Session issues #### ih6001 - Invalid Session Config Message A text frame sent on `/v1/stream/analyze` did not match the documented session-config schema. The endpoint accepts inbound JSON text frames carrying caller-supplied session config (`include` and future extensions); the client may send a new config at any time and the most recent successful frame wins. The server closes the connection on a rejected frame so an invalid config cannot silently downgrade the session. The error message describes which part of the payload failed validation (for example, an unknown field or a value outside the documented enum). ##### How to fix 1. Confirm the text frame is a JSON object matching the documented session-config schema. 2. Verify every field name and value is in the documented set. 3. Reconnect and resend the corrected config. #### ih6002 - Message Too Large A single binary video chunk sent over the WebSocket connection exceeds the maximum permitted message size. The maximum message size is 32MB. ##### How to fix 1. Send smaller video chunks. 2. Reduce the chunk duration or video bitrate. #### ih6003 - Session Idle Timeout The connection was idle (no data received from the client) for longer than the configured session timeout. The maximum idle time is 5 minutes. ##### How to fix 1. Send video data regularly to keep the session active. 2. If you need a longer idle period, reconnect when ready to resume. #### ih6004 - Session Duration Exceeded The total elapsed time for the WebSocket session has exceeded the maximum allowed session duration. The maximum session duration is 60 minutes. ##### How to fix 1. Start a new WebSocket session. 2. If you need longer sessions, contact support to discuss your use case. #### ih6005 - Server Going Away The server is performing a scheduled update and has gracefully closed your WebSocket session. ##### How to fix 1. Reconnect to the WebSocket endpoint to start a new session. 2. This is a transient condition — the service will be available again shortly. #### ih6006 - Shutdown Warning The server is warning that it will shut down soon for a scheduled update. The connection remains open until the countdown expires, at which point the session will be closed with IH6005. Generally, an advance notice is sent 5 minutes before shutdown. The `seconds_remaining` field in the message indicates how long until the connection closes. A final warning is sent 30 seconds before shutdown. ##### How to fix 1. Prepare to reconnect when the session is closed. 2. This is an advance notice — you can continue sending data until the connection closes. #### ih6007 - Unreadable Video Segment A video segment you sent could not be decoded and was skipped when measuring its duration. It may be malformed or incomplete — for example a truncated final segment produced at the end of a recording. This notice is non-fatal: the connection stays open, the segment is still included in the session where possible, and analysis continues with the readable segments. The `segment` field identifies which segment was affected. ##### How to fix 1. Ensure each segment sent over the WebSocket is a well-formed, decodable video fragment (for MediaRecorder, that media clusters are not split or truncated across segment boundaries, especially on the final flush). 2. If the notice only appears for the last segment of a session, verify the recorder is flushed cleanly before the stream is closed. #### ih6008 - Session Byte Limit Exceeded The session reached the maximum cumulative upload size allowed by the client token (its `max_bytes` cap), so the connection was closed. ##### How to fix * Mint a new client token to start a fresh session. * When minting, set a `max_bytes` budget appropriate for the expected session length, or omit it to rely only on the duration and idle limits. #### ih6009 - Session Closing A binary video chunk arrived after you requested a graceful session close with the `session.close` message, so it was rejected. Once the close request is acknowledged (`session.closing`), the API accepts no new video: it finishes analyzing the video it already accepted, emits the final lifecycle messages, sends `session.ended`, and closes the connection. This notice is non-fatal and is sent at most once per session: further chunks are dropped silently while the session drains. ##### How to fix 1. Stop sending video before (or immediately after) sending `session.close`. 2. Keep the connection open and keep reading messages until `session.ended` arrives, then treat the session as complete. # Overview Source: https://docs.interhuman.ai/api-reference/overview Detect and interpret engagement, signals, and quality. # Detect and interpret conversational behavior. Interhuman is the social intelligence API for developers to detect and interpret engagement state, social signals, and conversation quality in real-time or post-processing. ## How to integrate Upload video files for comprehensive behavioral analysis. Stream raw video data for real-time interpretation. ## Capabilities The API provides access to multimodal models that produce: * **Engagement State**: Time-bounded states such as engaged, disengaged, and neutral states. * **Social Signals**: [Agreement, confusion, engagement, and more](/explanations/social-signals) with timing, probability, and rationale. * **Conversation Quality**: Aggregate and timeline scores for energy, rapport, authority, learning, and clarity. ## Authentication Use your API key directly when calling endpoints by sending `Authorization: Bearer `. See [Authentication](/api-reference/authentication) and [Get an API key](/how-to/get-api-key). # Realtime Analyze Source: https://docs.interhuman.ai/api-reference/realtime-analyze Analyze a realtime video stream. **Beta** — this API is under active development and may change without notice. # Streaming Analyze Source: https://docs.interhuman.ai/api-reference/stream-analyze Analyze a video stream. # Analyze uploaded video Source: https://docs.interhuman.ai/api-reference/upload-analyze post /v1/upload/analyze Analyze a video file in upload mode. #### Response Headers Unique identifier for the request. Include this when contacting support. Example: `f47ac10b-58cc-4372-a567-0e02b2c3d479` # Conversation quality Source: https://docs.interhuman.ai/explanations/conversation-quality How Interhuman quantifies behavioral interaction quality using CQI, dimension scores, and event metrics. # Conversation Quality Conversation quality is Interhuman’s way of measuring how well an interaction unfolds behaviorally over time — separate from what was discussed or whether a desired outcome was achieved. We compute a **Conversation Quality Index** plus a set of **dimension scores** derived from automatically detected social signals (e.g., engagement, stress, disagreement). ## What conversation quality is (and isn’t) ### What it captures * The behavioral quality of an interaction (how it is handled), grounded in observable social signals over time. * A diagnostic profile across stable dimensions like clarity, authority, energy, rapport, and learning. ### What it does not capture * Whether the conversation “worked” (deal outcomes, persuasion success, interview pass/fail). * Whether the right topics were covered (sales-stage or checklist frameworks). * A semantic evaluation of the content itself. ## Inputs Conversation quality is computed from Interhuman’s detected behavioral outputs: * `engagement_state`: the person’s attention level to the interaction at a given moment, expressed as engagement, disengagement, or neutral * `signals`: specific social behaviors detected in the interaction, such as hesitation, agreement, confusion, etc. See: [Social signals](/explanations/social-signals) ## Outputs (two-layer structure) Conversation quality follows a layered structure (diagnostic first, summary second): 1. **Quality Index** — a single headline score (0–100) 2. **Dimension scores** — 0–100 scores for a small set of stable behavioral dimensions ## Layer 1 — Conversation Quality Index The Conversation Quality Index is a single summary score answering: “How good was this interaction, behaviorally speaking?” * Range: **0–100** * The Conversation Quality Index is always shown alongside the underlying dimension scores * The Conversation Quality Index is computed as the arithmetic mean of the five dimension scores. ### Qualitative interpretation bands * **0–30**: Weak * **30–50**: Below average * **50–65**: Moderate * **65–80**: Good * **80–100**: Excellent ## Layer 2 — Dimension scores A dimension score shows how strongly a conversation expressed a given quality overall, based on the balance of helpful versus harmful behavioral signals. ### Core dimensions * **Clarity / Structure** — how easy it is to follow the message (order, concision, clarity) * **Authority / Credibility** — decisiveness and credibility (certainty vs. hedging/hesitation) * **Energy / Presence** — vitality, responsiveness, participatory energy * **Rapport / Relational Safety** — warmth, respect, trust, emotional safety * **Learning / Exploration** — curiosity, reflection, experimentation, openness to learning # Model Improvement Partnership Program Source: https://docs.interhuman.ai/explanations/model-improvement-program Learn how Interhuman AI trains its social intelligence models and how you can control your data's role in that process. ## Overview The Interhuman Model Improvement Partnership Program (MIP) gives you a meaningful role in shaping the future of social intelligence AI — while saving up to **50% on usage costs**. When you opt in to the MIP, anonymized data from your API requests may be used to improve Interhuman AI's models. In return, you receive significantly reduced pricing and contribute to more accurate, fair, and representative signal detection across accents, cultures, and interaction styles. Participation is **voluntary**, **transparent**, and **revocable at any time.** ## How It Works ### Two pricing tracks | Track | Data used for model improvement | Pricing | | ----------------- | ------------------------------- | ------------------------------ | | **Standard** | No | List price per your plan tier | | **Partner (MIP)** | Yes, with controls | **50% discount** on list price | * **Standard** — Your data is processed and returned to you. It is not used for model training. * **Partner (MIP)** — You opt in to share anonymized data under strict controls. Interhuman uses this data solely to train and evaluate its own models. You pay half the standard rate. ### Your controls * **Account or project-level toggle** — Enable or disable MIP from your dashboard at any time. ## How We Improve Our Models Interhuman AI uses multimodal deep learning to detect and interpret social signals — engagement, hesitation, confidence, skepticism, and more — from video and audio data. Our models learn the subtle, context-dependent patterns of human behavior that make up social communication. By incorporating real-world interaction data from MIP participants during training, we produce models that: * **Generalize better** across diverse accents, cultural norms, and interaction styles * **Detect signals more accurately** in the specific domains and scenarios our customers care about * **Reduce bias** by ensuring our training data reflects the full diversity of human communication After training, our models are mathematical representations of behavioral patterns — they have no rote memory of any individual data point used to train them. There is no risk of data leakage when the model is used in production. ## Data Handling, Security & Privacy We take data privacy seriously. Here is how MIP data is handled: * **Purpose-limited** — Data is used exclusively to improve Interhuman AI's own models. We never resell data or use it for advertising. * **Retention** — Training data is retained for the next model training cycle, with full audit logs. Source data is minimized after preprocessing. * **EU data residency** — Available via our European infrastructure to support GDPR-first requirements. * **Encryption** — All data is encrypted in transit (TLS 1.3) and at rest. * **Access controls** — Role-based access, audit logs, and strict internal policies govern all access to customer data. * **Contracts** — Data Processing Addendum (DPA) including Model Improvement language with explicit lawful basis and consent language available for enterprise customers. ## Why Participate * **Lower cost today** — Halve your per-credit cost immediately while keeping full API functionality and enterprise-grade controls. * **Higher accuracy tomorrow** — Your anonymized edge cases help our models adapt to your specific accents, cultures, and scenarios — directly benefiting your end users. * **Faster roadmap** — MIP data accelerates model evaluations and releases. Participants get access to new capabilities and signal types sooner. * **Fairness by design** — We prioritize representativeness across accents, genders, cultures, and interaction contexts. MIP participants help us measure and improve fairness. * **No lock-in** — Opt in or out at any time. Keep your plan tier and volume pricing either way. *** ## FAQ ### Does opting out change pricing for that request? Yes. Opted-out requests are billed at **Standard** price. All other requests on MIP-enrolled accounts are billed at Partner price. ### Can I revoke participation entirely? Yes, at any time. Revocation stops new data from being ingested. ### What if I'm in a regulated environment? Choose **Standard** to ensure no data is used for training. We also support EU data residency and can accommodate custom retention windows for enterprise customers. ### How do you handle PII? We apply automated redaction to transcripts and metadata before any data enters training pipelines. You can also limit your API usage to specific data types if needed. ### Will my data be shared with third parties? No. Data is used solely to train and evaluate Interhuman AI models. We never resell or redistribute raw data. ### What's the retention policy for MIP data? Training corpora are retained for up to **18 months**. Source data is minimized and trimmed after preprocessing. Full audit logs are maintained. ### Can I get a custom DPA or Model Improvement Addendum? Yes. Enterprise customers can request legal review and tailored contract annexes. ### Can I use Standard and still get volume discounts? Yes. Volume pricing and plan tiers apply regardless of MIP participation. *** ## Get Started 1. **Join Model Improvement** in your [dashboard](https://platform.interhuman.ai), or talk to Sales for an enterprise rollout. 2. Prefer a slower ramp? Start in **Standard**, then enable MIP for selected projects. **Questions?** Contact [support@interhuman.ai](mailto:support@interhuman.ai) or your account team. # Social signals Source: https://docs.interhuman.ai/explanations/social-signals Understand each social signal detected by the Interhuman API. The Interhuman API returns two related but distinct behavioral outputs: * `engagement_state`: the person’s attention level to the interaction at a given moment, expressed as engagement, disengagement, or neutral * `signals`: specific social behaviors detected in the interaction, such as hesitation, agreement, confusion, etc. Use these outputs to adapt your application's responses, route conversations, or score interaction quality without manual labeling. ## Probability and Rationale Alongside the signal label and its timestamps, Inter-1 will also return **`probability`** and **`rationale`**. * `signals[].probability` is a simple likelihood rating (`low` | `medium` | `high`). It helps you judge how reliable a detection is and decide how to use it (for example, you might only act on `high` signals, or show `low` signals as a “maybe”). * `signals[].rationale` is a short explanation of *what the model noticed* (in video, audio, and text) and how it connects to the signal. It’s designed to be human-readable, so you can quickly sanity-check the detection and understand what drove it. ## Engagement state These values belong to `engagement_state[].state` and are separate from `signals[].type`. ### Engaged Engaged reflects sustained focus and active participation in the interaction. An engaged person maintains eye contact, sits upright or leans forward, offers timely nods and short backchannels like "mm-hmm" or "I see," and keeps their body oriented toward the speaker. They might ask brief, task-focused questions that show they are following along. ### Disengaged Disengaged signals a reduction in attention, involvement, or investment in the interaction. A disengaged person appears mentally or emotionally withdrawn, contributes minimally, and shows limited responsiveness to what is happening. This often indicates that the interaction no longer feels relevant or worth sustained effort. ### Neutral Neutral reflects a baseline state where a participant is present and following along without strong positive or negative engagement markers. It often appears between more pronounced shifts toward engaged or disengaged behavior. ## Social signals (10 types) ## Agreement Agreement reflects alignment with another person's position, intent, or understanding. It appears when someone signals that they are on the same page and moving in the same direction as the speaker. This can take the form of affirming responses or moments where the listener reinforces or builds on what has just been said, indicating shared understanding. ## Confidence Confidence captures how firmly and assuredly someone communicates their position or decision. A confident person presents ideas with clarity and conviction, without excessive qualification or visible doubt. Their delivery suggests comfort with the stance they are taking and readiness to stand behind it. ## Confusion Confusion indicates a breakdown or gap in understanding during an interaction. It emerges when someone struggles to follow the content, structure, or implications of what is being discussed. This state often leads to pauses, clarification attempts, or visible effort to reorient and regain clarity. ## Disagreement Disagreement reflects an active divergence from another person's viewpoint or proposal. It arises when someone challenges, rejects, or pushes back against what is being presented. Rather than passive misalignment, disagreement involves a clear stance that the current direction or claim is not accepted. ## Frustration Frustration emerges when progress toward a goal feels blocked or repeatedly unsuccessful. It reflects mounting tension or irritation as expectations are unmet. This state is often associated with a sense of effort without payoff and can precede disengagement or escalation if unresolved. ## Hesitation Hesitation captures uncertainty or delay before committing to a response or action. It reflects moments where someone is weighing options, searching for the right wording, or holding back before moving forward. Hesitation often signals that a decision or stance is not yet fully formed. ## Interest Interest reflects positive engagement driven by relevance or curiosity. An interested person appears drawn into the topic and motivated to continue the exchange. This state is marked by attentiveness and signals that the content resonates or holds value for them. ## Skepticism Skepticism represents a questioning or doubtful stance toward a claim, proposal, or explanation. Rather than outright rejection, it involves cautious evaluation and a desire for justification or evidence. Skepticism often surfaces when information conflicts with prior beliefs or seems insufficiently supported. ## Stress Stress indicates heightened pressure or cognitive load during an interaction. It reflects a state where demands feel intense, time-sensitive, or difficult to manage. Under stress, people may prioritize speed or coping over depth, signaling strain rather than deliberate engagement. ## Uncertainty Uncertainty reflects low confidence in one's own knowledge, judgment, or decision. It appears when someone is unsure about the correctness or appropriateness of what they are saying or choosing. This state often involves tentative positioning and openness to revision or guidance. # Introduction Source: https://docs.interhuman.ai/getting-started/introduction Add a social-intelligence layer to your AI application. ## See Interhuman in action Understand engagement, hesitation, frustration, and other [observable cues](/explanations/social-signals) across any interaction. Interhuman captures behavioral patterns and returns structured outputs your application can use to adapt responses: engagement state, social signals, and conversation quality metrics. ## Quickstart Start analyzing multimodal context in minutes. You'll need an API key before you begin. Learn more in the [API key guide](/how-to/get-api-key). Upload a video through one request and get engagement state, social signals, and quality metrics your application can use immediately. Stream from your camera with MediaRecorder over WebSocket, or send file segments, and receive typed events for each chunk. ## Join Our Community Connect with other developers, get help, and share your projects. } > Join our Discord community to connect with other developers, get help with integration, and share your projects. ## Codealong tutorials Learn Interhuman by building real apps alongside a step-by-step video walkthrough. These codealong tutorials pair live coding with the core APIs so you can see how everything fits together in practice. A guided walkthrough where you'll build a simple application that uploads a video, processes it through the Upload & Analyze API, and displays engagement state, social signals, and conversation quality outputs. # Stream analysis Source: https://docs.interhuman.ai/getting-started/stream-analyze-quickstart Codealong: connect over WebSocket, capture camera and microphone, and stream segments to Interhuman. Stream analysis from a **live camera**: connect to Interhuman over WebSocket, capture media, and send **binary** chunks as they are recorded. In this codealong, you will: 1. **Connect** to `wss://api.interhuman.ai/v1/stream/analyze` 2. **Get camera** (and microphone where your stack supports it) 3. **Send** each recorded segment to Interhuman and read typed server events Wire the three steps together in your app. Use **JavaScript** in the browser (`getUserMedia` + `MediaRecorder`) or **Python** on the desktop (`opencv-python` for video; see notes below for audio). You’ll need an API key. Follow the [API key guide](/how-to/get-api-key) for details. ## 1) Connect to the WebSocket Open a TLS WebSocket to the stream endpoint. On connect, send **session config** as a **text** frame (UTF-8 JSON), then listen for **text** replies and parse JSON. Branch on `type` (`signal.detected`, `engagement.updated`, `conversation_quality.updated`, `error`). ```javascript JavaScript icon="square-js" theme={null} const WS_URL = "wss://api.interhuman.ai/v1/stream/analyze"; const apiKey = "YOUR_API_KEY"; // In production, do not hardcode—use your app's auth flow. const ws = new WebSocket(WS_URL, apiKey); ws.binaryType = "arraybuffer"; ws.addEventListener("open", () => { const sessionConfig = { include: [ "conversation_quality_overall", "conversation_quality_timeline", ], }; ws.send(JSON.stringify(sessionConfig)); }); ws.addEventListener("message", (event) => { if (typeof event.data !== "string") return; const payload = JSON.parse(event.data); console.log(payload.type, payload); }); ``` ```python Python icon="python" theme={null} import asyncio import json import os import websockets WS_URL = "wss://api.interhuman.ai/v1/stream/analyze" async def connect(): api_key = os.environ["API_KEY"] headers = {"Authorization": f"Bearer {api_key}"} ws = await websockets.connect( WS_URL, additional_headers=headers, max_size=None, ) session_config = { "include": [ "conversation_quality_overall", "conversation_quality_timeline", ], } await ws.send(json.dumps(session_config)) return ws # Example: ws = asyncio.run(connect()) ``` Reference: [Stream & analyze](/api-reference/stream-analyze) ## 2) Get camera and microphone ```javascript JavaScript icon="square-js" theme={null} const mediaStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true, }); const preview = document.querySelector("#preview"); preview.srcObject = mediaStream; preview.play(); ``` ```python Python icon="python" theme={null} import cv2 cap = cv2.VideoCapture(0) if not cap.isOpened(): raise RuntimeError("Could not open camera (device index 0)") # Optional local preview while you build: ok, frame = cap.read() if ok: cv2.imshow("preview", frame) ``` ## 3) Send segments to Interhuman Send each non-empty recording as a **binary** WebSocket frame. Start recording **after** the WebSocket is open and session config is sent. ```javascript JavaScript icon="square-js" theme={null} const SEGMENT_MS = 3000; const mimeType = ["video/webm;codecs=vp9,opus", "video/webm;codecs=vp8,opus", "video/webm"].find( (m) => MediaRecorder.isTypeSupported(m) ) || ""; const recorder = new MediaRecorder( mediaStream, mimeType ? { mimeType } : undefined ); recorder.addEventListener("dataavailable", async (event) => { if (!event.data || event.data.size === 0) return; if (ws.readyState !== WebSocket.OPEN) return; const buffer = await event.data.arrayBuffer(); ws.send(buffer); }); // Call once the WebSocket is open and session config is sent: recorder.start(SEGMENT_MS); ``` ```python Python icon="python" theme={null} import asyncio import time import cv2 SEGMENT_SECONDS = 3 SEGMENT_PATH = "segment.mp4" def record_segment(cap: cv2.VideoCapture, path: str, seconds: float) -> None: fps = cap.get(cv2.CAP_PROP_FPS) or 20.0 width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) fourcc = cv2.VideoWriter_fourcc(*"mp4v") writer = cv2.VideoWriter(path, fourcc, fps, (width, height)) deadline = time.time() + seconds while time.time() < deadline: ok, frame = cap.read() if ok: writer.write(frame) writer.release() async def send_segment(ws) -> None: record_segment(cap, SEGMENT_PATH, SEGMENT_SECONDS) with open(SEGMENT_PATH, "rb") as f: await ws.send(f.read()) msg = await asyncio.wait_for(ws.recv(), timeout=60.0) print(msg) # After connect(): asyncio.run(send_segment(ws)) ``` When the user stops, release the camera and close the connection: ```javascript JavaScript icon="square-js" theme={null} if (recorder && recorder.state !== "inactive") { recorder.stop(); } ws.close(); mediaStream.getTracks().forEach((track) => track.stop()); ``` ```python Python icon="python" theme={null} cap.release() await ws.close() cv2.destroyAllWindows() # if you opened a preview window ``` ## 4) Read server envelopes Every server message shares the same outer shape: `type`, `timestamp`, `correlation_id`, and `data`. Narrow on `type` before reading fields inside `data`. ### `signal.detected` ```json theme={null} { "type": "signal.detected", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "signals": [ { "type": "agreement", "start": 3.0, "end": 11.0, "probability": "high", "rationale": "Subject nodded repeatedly while maintaining eye contact." } ] } } ``` Each entry in `data.signals[]` uses the same shape as upload responses: `type`, `start`, `end`, `probability`, and `rationale`. ### `engagement.updated` ```json theme={null} { "type": "engagement.updated", "timestamp": "2025-01-01T00:00:00.000000Z", "correlation_id": "550e8400-e29b-41d4-a716-446655440000", "data": { "state": "engaged", "start": 3.0, "end": 11.0 } } ``` ### `conversation_quality.updated` (when opted in) When your session config `include` lists `conversation_quality_overall` and/or `conversation_quality_timeline`, you may receive `conversation_quality.updated` with `data.overall` and/or `data.timeline` for the window that was just processed. See [Conversation quality](/explanations/conversation-quality). ### `error` Errors use the same envelope with `type: "error"` and structured fields under `data` (for example `code`, `message`, `link`, and `segment` when applicable). See [Error handling](/api-reference/error-handling). ### How to interpret it quickly * **`signal.detected`**: `data.signals[]` lists moment-level [social signals](/explanations/social-signals) for the segment; each `rationale` explains that detection. * **`engagement.updated`**: attention level for a time window within the segment (`start` / `end` are seconds within that segment). * **`conversation_quality.updated`**: optional overall and per-window quality metrics when requested. ## Next steps * [Stream & analyze](/api-reference/stream-analyze) — full AsyncAPI channel, headers, and message schemas. * [Authentication](/api-reference/authentication) — API key usage; browser WebSockets use the subprotocol as above. * [Error handling](/api-reference/error-handling) — structured error codes and recovery. * [Video upload quickstart](/getting-started/video-upload-quickstart) — one-shot `POST /v1/upload/analyze` flow. * [Agent Skills](/how-to/agent-skills) — installable skills that wrap upload and stream calls. * [Social signals](/explanations/social-signals) and [Conversation quality](/explanations/conversation-quality) — meaning of outputs. # Video Analysis Source: https://docs.interhuman.ai/getting-started/video-upload-quickstart Use this quickstart to get your first successful analysis response in a few minutes. In this guide, you will: 1. Upload a local video file to `POST /v1/upload/analyze` 2. Read `engagement_state`, `signals[]` (including per-signal `rationale`), and optional `conversation_quality` outputs You’ll need an API key. Follow the [API key guide](/how-to/get-api-key) for details. You’ll also need a video file. You can download an example from here. ## 1) Upload and analyze a video Use one of the requests below to send a local video file (mp4, avi, mov, mkv, mpeg-ts, webm; minimum 10 KB, maximum 32MB) to `POST /v1/upload/analyze`. ### Video and audio content **Include both video and audio.** Black video, silent audio, and muted screen captures may upload successfully but return unreliable results. The API returns core analysis by default. You can optionally request conversation-quality sections by passing `include[]` flags (shown below). ```bash cURL theme={null} export API_KEY="YOUR_API_KEY" export VIDEO_PATH="path_to_your_video.mp4" curl -X POST https://api.interhuman.ai/v1/upload/analyze \ -H "Authorization: Bearer ${API_KEY}" \ -F "file=@${VIDEO_PATH};type=video/mp4" \ -F "include[]=conversation_quality_overall" \ -F "include[]=conversation_quality_timeline" ``` ```javascript JavaScript icon="square-js" theme={null} import fs from "fs"; import FormData from "form-data"; import fetch from "node-fetch"; const formData = new FormData(); formData.append("file", fs.createReadStream(process.env.VIDEO_PATH)); formData.append("include[]", "conversation_quality_overall"); formData.append("include[]", "conversation_quality_timeline"); fetch("https://api.interhuman.ai/v1/upload/analyze", { method: "POST", headers: { Authorization: `Bearer ${process.env.API_KEY}` }, body: formData }) .then((res) => res.text()) .then((text) => console.log("Response:", text)) .catch((err) => console.error(err)); ``` ```python Python icon="python" theme={null} import os import requests api_key = os.environ["API_KEY"] video_path = os.environ["VIDEO_PATH"] with open(video_path, "rb") as f: files = {"file": (os.path.basename(video_path), f, "video/mp4")} data = [ ("include[]", "conversation_quality_overall"), ("include[]", "conversation_quality_timeline"), ] response = requests.post( "https://api.interhuman.ai/v1/upload/analyze", headers={"Authorization": f"Bearer {api_key}"}, files=files, data=data, timeout=300, ) print("Status code:", response.status_code) print("Response:", response.text) ``` If you want only core outputs, remove the `include[]` lines. Reference: [Upload & Analyze API](/api-reference/upload-analyze) ## 2) Read the response After your upload is processed, the API returns a structured response with three complementary outputs: * **engagement\_state**: Time-bounded labels such as `engaged`, `disengaged`, or `neutral`. * **signals\[]**: Time-bounded [social signals](/explanations/social-signals), each with `type`, `probability`, and `rationale`. * **conversation\_quality** (optional): Reuses the `conversation_quality_values` shape in both `overall` and each timeline window's `values` object. Time fields (`start`, `end`) are expressed in seconds from the start of the uploaded video. `conversation_quality_values` shape (reused by `conversation_quality.overall` and `conversation_quality.timeline[].values`): ```json theme={null} { "quality_index": 45, "energy": 53, "rapport": 50, "authority": 49, "learning": 50, "clarity": 48 } ``` Here’s an example of what the API returns: ```json theme={null} { "engagement_state": [ { "start": 0, "end": 10, "state": "engaged" }, { "start": 10, "end": 20, "state": "disengaged" } ], "signals": [ { "start": 0, "end": 10, "type": "agreement", "probability": "high", "rationale": "The speaker provides a quick affiliative nod while the partner is speaking." }, { "start": 5, "end": 15, "type": "confidence", "probability": "medium", "rationale": "The speaker maintains upright posture and responds with steady, fluent delivery." } ], "conversation_quality": { "overall": { "quality_index": 45, "energy": 53, "rapport": 50, "authority": 49, "learning": 50, "clarity": 48 }, "timeline": [ { "start": 0, "end": 10, "values": { "quality_index": 72, "energy": 80, "rapport": 75, "authority": 68, "learning": 70, "clarity": 67 } } ] } } ``` ### How to interpret it quickly * `signals[]` gives moment-level events; `rationale` explains why each signal was inferred. * `engagement_state` shows attention level over contiguous windows. * `conversation_quality.overall` is a single interaction summary. * `conversation_quality.timeline[]` shows how quality changes over time. ## Next steps * [Build a Video Analyzer App](/how-to/build-video-analyzer-app-codealong) — full UI codealong. * [Social signals](/explanations/social-signals) — meaning of each signal type. * [Conversation quality](/explanations/conversation-quality) — quality dimensions and interpretation. * [Upload & Analyze API](/api-reference/upload-analyze) — full request/response and error details. # Agent Skills Source: https://docs.interhuman.ai/how-to/agent-skills Install Interhuman Agent Skills so coding agents can call the API using packaged workflows. Skill names and behavior are maintained in the public repository. # Agent Skills Interhuman **Agent Skills** are installable packages that teach compatible agents (Cursor, Claude Code, Codex, OpenCode, and others) how to call Interhuman API workflows with consistent prompts and guardrails. The set of skills, their names, and what each one covers **change over time**. Treat the public repository as the catalog and changelog—this page explains how to install and use the skill set without duplicating the per-skill README. Use the repository to: * See **which skills exist today** and what each one wraps * Copy **up-to-date** install examples and options (`--list`, `--skill`, `--global`, and so on) * Align agent behavior with **current** request shapes and response handling ## Install skills Install the full Interhuman skill set from GitHub: ```bash theme={null} npx skills add InterhumanAI/skills ``` Or from the repository URL: ```bash theme={null} npx skills add https://github.com/InterhumanAI/skills ``` Authoritative list of skills, install commands, and per-skill behavior. Check here whenever you add or update skills in your project. ## Common install patterns List what the repository offers without installing: ```bash theme={null} npx skills add InterhumanAI/skills --list ``` Install specific skills only (use the names listed in the repository README): ```bash theme={null} # Example skill name — see GitHub for current options npx skills add InterhumanAI/skills --skill interhuman-stream-analyze ``` Install every skill from the repository: ```bash theme={null} npx skills add InterhumanAI/skills --skill '*' ``` Install globally for Cursor (example): ```bash theme={null} npx skills add InterhumanAI/skills -g -a cursor -y ``` ## Recommended workflow 1. Open [github.com/InterhumanAI/skills](https://github.com/InterhumanAI/skills) and pick the skill that matches your integration (upload, stream, or others listed there). 2. Configure your **API key** in the environment your agent uses (for example `export API_KEY="YOUR_API_KEY"`). Skills expect direct API-key usage: `Authorization: Bearer ` unless the repo documents otherwise. 3. Let the agent follow the skill instructions and treat outputs as **raw API JSON**—compose app logic in your own code. ## API reference (stable docs) HTTP upload analysis and WebSocket streaming are documented here; use these pages alongside the skill README for parameters, envelopes, and error fields: * [Upload & analyze](/api-reference/upload-analyze) * [Stream & analyze](/api-reference/stream-analyze) * [Authentication](/api-reference/authentication) * [Video upload quickstart](/getting-started/video-upload-quickstart) # Build a Video Analyzer App Source: https://docs.interhuman.ai/how-to/build-video-analyzer-app-codealong Follow a step-by-step video tutorial to build a simple video analyzer app using Interhuman's upload analysis API. Build a small app that uploads a video from your computer to Interhuman and returns structured analysis results. In this codealong, you will: * upload a local video to `POST /v1/upload/analyze` * optionally request conversation-quality outputs * render returned `signals[]` (including per-signal `rationale`) and quality metrics in your UI This page is written so you can follow the build without the video, while the video helps with pacing and implementation details. ## Watch the codealong video