Skip to main content
Use the masterprompt below to guide AI agents through an end-to-end Emerge integration without hardcoding API specs. It keeps identifiers private and instructs the agent to read the live docs for exact endpoints, parameters, and response formats.

Masterprompt (copy/paste)

You are building a full-stack app that integrates Emerge Link + Query end-to-end.

Rules:
- Do NOT hardcode Emerge endpoints or parameter lists. Always read the live docs via MCP for exact references.
- Keep Link (consent) and Query (data retrieval) separate.
- Before coding, ask the user for config fields and confirm any URLs/params against the docs. At minimum ask for: client_id, signing_secret, api_token, redirect_uri, flow_config name (if any), and the current Link start URL + Query endpoint URLs + export/consent status URLs from the docs.
- Generate signed links server-side. Never expose signing_secret or API tokens in the frontend.
- Generate a random state, store it server-side, and verify it on callback.
- Never ask end-users for uid. Use your internal user id (server-side) or omit uid and store the callback uid.
- Poll export readiness using the Link export status endpoint: https://link.emergedata.ai/export/status/{uid}.
- Provide TypeScript and Python examples with async/await and error handling.
- Use privacy-first messaging and explain why each step matters when it is not obvious.

Required references (read before coding):
- /link/create-link (signature rules + link parameters)
- /link/callbacks (status + error handling)
- Link API export status endpoint (GET /export/status/{uid})
- Link API consent status endpoint (GET /consent/status/{uid})
- /query/overview (sync vs async + auth)
- /query/pagination (cursor + delta sync)

Output requirements:
1) Direct answer
2) Code examples (TypeScript + Python)
3) Links to the docs pages you used
4) Edge cases/gotchas

Flow summary (spec-free)

  1. Create a signed consent link server-side and redirect the user.
  2. Handle the callback by verifying state and storing the returned uid.
  3. Poll export status via https://link.emergedata.ai/export/status/{uid} before querying.
  4. Query data using the stored uid, with pagination and delta sync where needed.
  5. Check consent status via https://link.emergedata.ai/consent/status/{uid} and stop processing if revoked.
Why this matters:
  • The uid is the canonical user scope. Keeping it server-side prevents wrong-user data access.
  • Polling export status avoids empty results and improves UX when data is not ready.
  • Keeping secrets off the client prevents data leakage.

Integration scaffold (server-side)

import express from 'express';
import crypto from 'crypto';

const app = express();

const stateStore = new Map<string, { internalUserId: string; createdAt: number }>();
const uidMap = new Map<string, string>();

function loadConfig() {
  const linkStartUrl = process.env.EMERGE_LINK_START_URL;
  const querySearchUrl = process.env.EMERGE_QUERY_SEARCH_URL;
  const exportStatusUrl = process.env.EMERGE_EXPORT_STATUS_URL;
  const consentStatusUrl = process.env.EMERGE_CONSENT_STATUS_URL;
  const signingSecret = process.env.EMERGE_SIGNING_SECRET;
  const linkParamsJson = process.env.EMERGE_LINK_PARAMS_JSON;

  if (!linkStartUrl || !querySearchUrl || !exportStatusUrl || !consentStatusUrl || !signingSecret || !linkParamsJson) {
    throw new Error('Missing EMERGE_LINK_START_URL, EMERGE_QUERY_SEARCH_URL, EMERGE_EXPORT_STATUS_URL, EMERGE_CONSENT_STATUS_URL, EMERGE_SIGNING_SECRET, or EMERGE_LINK_PARAMS_JSON');
  }

  return { linkStartUrl, querySearchUrl, exportStatusUrl, consentStatusUrl, signingSecret, linkParamsJson };
}

function signParams(params: Record<string, string>, signingSecret: string): string {
  const signatureBase = Object.keys(params)
    .sort()
    .map(key => `${key}=${params[key]}`)
    .join('&');

  return crypto
    .createHmac('sha256', signingSecret)
    .update(signatureBase)
    .digest('hex');
}

function buildSignedLinkUrl(internalUserId: string): { url: string; state: string } {
  const { linkStartUrl, signingSecret, linkParamsJson } = loadConfig();

  const state = crypto.randomBytes(16).toString('hex');
  const timestamp = new Date().toISOString();

  const params: Record<string, string> = JSON.parse(linkParamsJson);
  params.state = state;
  params.timestamp = timestamp;
  params.uid = internalUserId;

  const signature = signParams(params, signingSecret);
  const finalParams = new URLSearchParams(params);
  finalParams.append('signature', signature);

  return { url: `${linkStartUrl}?${finalParams.toString()}`, state };
}

app.get('/connect-data', async (req, res) => {
  try {
    const internalUserId = String(req.query.user_id || '');
    if (!internalUserId) {
      return res.status(400).send('Missing user_id');
    }

    const { url, state } = buildSignedLinkUrl(internalUserId);
    stateStore.set(state, { internalUserId, createdAt: Date.now() });

    return res.redirect(url);
  } catch (err) {
    console.error('Failed to create link', err);
    return res.status(500).send('Server error');
  }
});

app.get('/emerge/callback', async (req, res) => {
  try {
    const { status, state, uid, error_code } = req.query as Record<string, string>;
    const record = stateStore.get(state);

    if (!record) {
      return res.status(400).send('Invalid state');
    }

    stateStore.delete(state);

    if (status === 'success' || status === 'reauthorized') {
      if (!uid) {
        return res.status(400).send('Missing uid');
      }
      uidMap.set(record.internalUserId, uid);
      return res.redirect('/dashboard?connected=true');
    }

    const errorMessage = encodeURIComponent(error_code || 'unknown_error');
    return res.redirect(`/connect?error=${errorMessage}`);
  } catch (err) {
    console.error('Callback error', err);
    return res.status(500).send('Server error');
  }
});

function buildStatusUrl(template: string, uid: string): string {
  if (template.includes('{uid}')) {
    return template.replace('{uid}', encodeURIComponent(uid));
  }
  return template;
}

async function pollExportReady(uid: string, provider: 'google_data' | 'gmail' = 'google_data'): Promise<void> {
  const { exportStatusUrl } = loadConfig();
  const token = process.env.EMERGE_API_TOKEN;
  if (!token) {
    throw new Error('Missing EMERGE_API_TOKEN');
  }

  const maxAttempts = 10;
  const baseDelayMs = 1500;

  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
    const url = buildStatusUrl(exportStatusUrl, uid);
    const response = await fetch(url, {
      headers: { Authorization: `Bearer ${token}` }
    });

    if (!response.ok) {
      const body = await response.text();
      throw new Error(`Export status failed (${response.status}): ${body}`);
    }

    const status = await response.json() as {
      sources?: Array<{ provider: string; data_ready: boolean }>;
    };
    const source = status.sources?.find((item) => item.provider === provider);
    if (source?.data_ready === true) {
      return;
    }

    const delay = baseDelayMs * attempt;
    await new Promise(resolve => setTimeout(resolve, delay));
  }

  throw new Error('Export not ready after polling');
}

app.get('/query-search', async (req, res) => {
  try {
    const internalUserId = String(req.query.user_id || '');
    const emergeUid = uidMap.get(internalUserId);
    if (!emergeUid) {
      return res.status(404).send('User not connected');
    }

    const exportProvider = (process.env.EMERGE_EXPORT_PROVIDER as 'google_data' | 'gmail' | undefined) || 'google_data';
    await pollExportReady(emergeUid, exportProvider);

    const { querySearchUrl } = loadConfig();
    const url = new URL(querySearchUrl);
    url.searchParams.set('uid', emergeUid);

    const response = await fetch(url.toString(), {
      headers: { Authorization: `Bearer ${process.env.EMERGE_API_TOKEN}` }
    });

    if (!response.ok) {
      const body = await response.text();
      throw new Error(`Query failed (${response.status}): ${body}`);
    }

    const data = await response.json();
    return res.json(data);
  } catch (err) {
    console.error('Query error', err);
    return res.status(500).send('Server error');
  }
});

app.get('/consent-status', async (req, res) => {
  try {
    const internalUserId = String(req.query.user_id || '');
    const emergeUid = uidMap.get(internalUserId);
    if (!emergeUid) {
      return res.status(404).send('User not connected');
    }

    const { consentStatusUrl } = loadConfig();
    const token = process.env.EMERGE_API_TOKEN;
    if (!token) {
      return res.status(500).send('Missing EMERGE_API_TOKEN');
    }

    const url = buildStatusUrl(consentStatusUrl, emergeUid);
    const response = await fetch(url, {
      headers: { Authorization: `Bearer ${token}` }
    });

    if (!response.ok) {
      const body = await response.text();
      throw new Error(`Consent status failed (${response.status}): ${body}`);
    }

    const data = await response.json();
    return res.json(data);
  } catch (err) {
    console.error('Consent status error', err);
    return res.status(500).send('Server error');
  }
});
app.listen(3000, () => {
  console.log('Server running on http://localhost:3000');
});

Edge cases / gotchas

  • state must be verified on callback or you risk account linking attacks.
  • Querying before export is ready can return empty results; poll export status and retry with backoff.
  • If a user revokes consent, stop processing and delete data to stay privacy-first; use the consent status endpoint to detect changes.
  • Never prompt the user to type a uid — it must stay server-side.