Hex Project Orchestration

Overview

Trigger Hex project runs from external orchestration tools (Airflow, Dagster, cron) with input parameters, status polling, and error handling. This is the primary integration pattern for embedding Hex in data pipelines.

Instructions

Step 1: Parameterized Project Runs

import 'dotenv/config';
const TOKEN = process.env.HEX_API_TOKEN!;
const BASE = 'https://app.hex.tech/api/v1';

interface RunConfig {
  projectId: string;
  inputParams?: Record<string, any>;
  updateCache?: boolean;
  killRunning?: boolean;
}

async function triggerRun(config: RunConfig) {
  const response = await fetch(`${BASE}/project/${config.projectId}/run`, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${TOKEN}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({
      inputParams: config.inputParams || {},
      updateCacheResult: config.updateCache ?? true,
      killRunningExecution: config.killRunning ?? false,
    }),
  });
  if (!response.ok) throw new Error(`Trigger failed: ${response.status} ${await response.text()}`);
  return response.json();
}

Step 2: Synchronous Run Helper

async function runAndWait(config: RunConfig, timeoutMs = 600000): Promise<any> {
  const { runId, projectId } = await triggerRun(config);
  const startTime = Date.now();

  while (Date.now() - startTime < timeoutMs) {
    const res = await fetch(`${BASE}/project/${projectId}/run/${runId}`, {
      headers: { 'Authorization': `Bearer ${TOKEN}` },
    });
    const status = await res.json();

    switch (status.status) {
      case 'COMPLETED': return { success: true, runId, duration: Date.now() - startTime };
      case 'ERRORED': throw new Error(`Run ${runId} errored: ${status.statusMessage || 'unknown'}`);
      case 'KILLED': throw new Error(`Run ${runId} was killed`);
      default: await new Promise(r => setTimeout(r, 5000));
    }
  }
  throw new Error(`Run ${runId} timed out after ${timeoutMs}ms`);
}

Step 3: Pipeline Orchestration

// Run multiple Hex projects in sequence (data pipeline)
async function runPipeline(steps: RunConfig[]) {
  const results = [];
  for (const step of steps) {
    console.log(`Running: ${step.projectId}`);
    const result = await runAndWait(step);
    console.log(`Completed in ${result.duration}ms`);
    results.push(result);
  }
  return results;
}

// Example: ETL pipeline
await runPipeline([
  { projectId: 'extract-project-id', inputParams: { date: '2025-01-01' } },
  { projectId: 'transform-project-id' },
  { projectId: 'load-project-id', updateCache: true },
]);

Step 4: Cancel Long-Running Projects

async function cancelRun(projectId: string, runId: string) {
  const response = await fetch(`${BASE}/project/${projectId}/run/${runId}`, {
    method: 'DELETE',
    headers: { 'Authorization': `Bearer ${TOKEN}` },
  });
  console.log(`Cancelled run ${runId}: ${response.status}`);
}

Error Handling

Prerequisites

• A named project owner, environment allowlist, approved parameter schema, and a sandbox project with fictitious or approved test data.
• Execution/cancel authority scoped to one project, a correlation convention, and a rollback or cancel procedure.

Output

Return an orchestration receipt with opaque project/run IDs, parameter revision, trigger identity class, start/terminal state, aggregate assertions, cancellation result, and rollback reference. Do not store SQL, cell output, or credentials.

Examples

project=proj-sandbox-12; params=r3; trigger=ci-service; run=complete; assertions=pass; cancel=not-needed; rollback=run-r5 records a controlled project run.

Resources

• Run Project API
• Orchestration Blog

Next Steps

For scheduled runs, see hex-core-workflow-b.