Web SDK

Use the Web SDK in browser applications with a short-lived tokenProvider flow from your backend. If you want to evaluate the runtime before you wire in browser auth, start with the guided guest Workbench demo. This page is the integration path once you are ready to connect your own frontend.

Step 1: Install

npm install @kynasmith/web-sdk

The @kynasmith/web-sdk package is the intended public browser integration path. If the install command currently returns a 404, check the changelog for registry availability before treating it as an integration issue.

Step 2: Set up a backend token endpoint

Your browser application must not embed long-lived API keys. Instead, create a backend endpoint that exchanges your API key for a short-lived access token:

# Example backend endpoint (using any Python web framework)
import httpx

async def exchange_token():
    """Mint a short-lived Kynasmith access token for the browser client."""
    async with httpx.AsyncClient() as http:
        resp = await http.post(
            "https://api.kynasmith.dev/api/auth/access-tokens",
            json={
                "api_key_id": "key_123",
                "api_key_secret": "secret_123",
                "scopes": ["sessions:write", "movespecs:read"],
            },
        )
        resp.raise_for_status()
        return resp.json()

See Authentication for more details on the token exchange flow and available scopes.

Step 3: Initialize the client

import { createClient } from "@kynasmith/web-sdk";

const client = createClient({
  tokenProvider: async () => {
    const response = await fetch("/api/kynasmith/token", { method: "POST" });
    const { access_token } = await response.json();
    return access_token;
  },
});

Your tokenProvider is called before each authenticated request, so it must always be able to return a fresh token.

Step 4: Create and release a MoveSpec

const movespec = await client.movespecs.create({
  name: "Squat counter",
});

await client.movespecs.updateDraft(movespec.movespec_id, {
  yamlSource: `
name: squat
version: "1.0"
detection:
  type: repetition
  movement:
    name: squat
    phases:
      - name: standing
        criteria:
          - joint: knee
            angle_min: 160
            angle_max: 180
      - name: bottom
        criteria:
          - joint: knee
            angle_min: 60
            angle_max: 100
    sequence:
      - standing
      - bottom
      - standing
`,
});

const validation = await client.movespecs.validateDraft(movespec.movespec_id);
if (!validation.ok) {
  throw new Error("Draft validation failed");
}

const version = await client.movespecs.createVersion(movespec.movespec_id);
await client.movespecs.releaseVersion(
  movespec.movespec_id,
  version.movespec_version_id,
);

See MoveSpec for the full YAML authoring guide.

Step 5: Replay a browser video file

const fileInput = document.querySelector("#video") as HTMLInputElement;
const videoFile = fileInput.files?.[0];

if (!videoFile) {
  throw new Error("Choose a video first");
}

const result = await client.processVideo(
  {
    movespecVersionId: version.movespec_version_id,
    mode: "replay",
  },
  videoFile,
  { batchSize: 15 },
);

console.log(`Accepted frames: ${result.acceptedFramesTotal}`);
console.log("Completed events:", result.completedEventCounts);

Step 6: Run a live camera session

const liveResult = await client.startCamera(
  {
    movespecVersionId: version.movespec_version_id,
    mode: "live",
  },
  { maxFrames: 300 },
);

startCamera() processes local camera frames until maxFrames is reached or the run is aborted, then finalizes the session and returns the result.

Manual session control

Use the lower-level session helpers for full control over realtime flow:

const session = await client.detection.createSession({
  movespecVersionId: version.movespec_version_id,
  mode: "live",
});

const realtime = client.detection.connectSession(session);

for await (const event of realtime.iterateEvents()) {
  console.log(event.type);
}

Alternative configuration

Static access token

If you already have a short-lived bearer token:

const client = createClient({
  accessToken: "ks_at_...",
});

When using a static accessToken, you are responsible for refresh and rotation.

Explicit project ID

When your credential is scoped to multiple projects, pass projectId in request payloads:

const result = await client.processVideo(
  {
    movespecVersionId: version.movespec_version_id,
    mode: "replay",
    projectId: "proj_456",
  },
  videoFile,
);

Typed errors

The Web SDK exposes typed error classes for structured error handling:

AuthError — authentication failures
AuthorizationError — permission or scope errors
CompatibilityError — SDK version is not compatible (update your SDK)
RealtimeProtocolError — connection or session communication failure
SessionStateError — session lifecycle violations
ValidationError — request or MoveSpec validation failures
RateLimitError — rate limit exceeded (HTTP 429)

Browser requirements

tokenProvider must be able to mint or fetch fresh short-lived bearer tokens repeatedly
Camera helpers require navigator.mediaDevices.getUserMedia()
Replay and camera helpers need a browser environment with fetch, WebSocket, and media APIs
If you keep the default MediaPipe asset URLs, your CSP and network path must allow Google Cloud Storage and jsDelivr

Compatibility

@mediapipe/tasks-vision 0.10.32
Default WASM root: @mediapipe/tasks-vision@0.10.32/wasm
Default model asset: pose_landmarker_full/float16/1/pose_landmarker_full.task

Quickstart

SDKs

Web SDK

Web SDK

Step 1: Install

Step 2: Set up a backend token endpoint

Step 3: Initialize the client

Step 4: Create and release a MoveSpec

Step 5: Replay a browser video file

Step 6: Run a live camera session

Manual session control

Static access token

Explicit project ID

Typed errors

Browser requirements

Compatibility

​Web SDK

​Step 1: Install

​Step 2: Set up a backend token endpoint

​Step 3: Initialize the client

​Step 4: Create and release a MoveSpec

​Step 5: Replay a browser video file

​Step 6: Run a live camera session

​Manual session control

​Static access token

​Explicit project ID

​Typed errors

​Browser requirements

​Compatibility

Web SDK

Step 1: Install

Step 2: Set up a backend token endpoint

Step 3: Initialize the client

Step 4: Create and release a MoveSpec

Step 5: Replay a browser video file

Step 6: Run a live camera session

Manual session control

Static access token

Explicit project ID

Typed errors

Browser requirements

Compatibility