fal-js,
from the inside

Start here because everything else in fal-js borrows from this one object. Get it wrong and the rest of the library reads as unrelated pieces; get it right and the queue, the streaming client, the WebSocket and the extension kernel are all obviously the same thing wearing different transports.

Why this comes first

The tempting mental model is that fal is an HTTP wrapper: you call fal.run(), it does a fetch, you get a result. That model survives about ten minutes. It cannot explain why a queue poller and a WebSocket connection both know your credentials, why setting one proxyUrl silently reroutes uploads you never configured, or why a realtime extension can make an authenticated call to a host that isn't a fal endpoint.

The better model: the client is a resolved configuration with methods attached. createFalClient resolves a config once, then hands the same frozen object to every sub-client it builds. Auth, retries, the fetch implementation, and the request middleware are decided in one place and read everywhere else.

Resolution is a phase, not a getter

Two types matter and the distinction is easy to skim past: Config is what you pass, with almost everything optional. RequiredConfig is what the library uses, with nothing optional. createConfig is the function that turns the first into the second, and it does four things worth knowing about:

// config.ts — createConfig, compressed to its decisions
let configuration = {
  ...DEFAULT_CONFIG,          // credentialsFromEnv, defaultResponseHandler, retry
  ...config,                  // your overrides win
  fetch: config.fetch ?? resolveDefaultFetch(),
  retry: { ...DEFAULT_RETRY_OPTIONS, ...(config.retry || {}) },
};

if (config.proxyUrl) {
  configuration.requestMiddleware = withMiddleware(   // composed, not replaced
    configuration.requestMiddleware,
    withProxy({ targetUrl: proxy.url, when: proxy.when }),
  );
}

First, defaults are merged under your config, so an omitted field is not undefined later — it is the default. Second, retry is merged as an object rather than replaced, so passing { maxAttempts: 2 } keeps the default backoff instead of erasing it. Third, fetch is resolved eagerly. Fourth — the one with real consequences — proxyUrl is not a mode, it is middleware, composed onto whatever middleware already existed.

That fourth point explains a behaviour that surprises people. Setting proxyUrl does not add a special case to fal.run(); it wraps the request-rewriting step that every request passes through. So storage uploads, queue polls, and anything a realtime extension sends all become proxied too, without any of them knowing a proxy exists. Powerful, and the reason a misconfigured allowlist breaks features you never touched.

Credentials can be a function

credentials is typed string | CredentialsResolver | undefined, where the resolver is () => string | undefined. The default is itself a resolver that reads FAL_KEY from the environment.

This is not decoration. A string is captured once at construction; a function is called per request. That is the difference between a client that dies when a short-lived token expires and one that picks up the new one — and it is the seam that Lucy's token provider and any credential rotation scheme hang off. If you only remember one thing about the config's shape, remember that this field is lazy on purpose.

The trap: fetch cares who calls it

Here is a detail that looks like trivia and is not. resolveDefaultFetch() returns the global fetch unbound:

// config.ts
export function resolveDefaultFetch(): FetchType {
  if (typeof fetch === "undefined") throw new Error(...);
  return fetch;   // <- no .bind(globalThis)
}

It is then stored as config.fetch. In a browser, native fetch checks its receiver — a WebIDL brand check, not a JavaScript rule — so it must be called with this as the global or as a bare function, never as a method of some object. config.fetch(url), which reads like the obvious thing to write, throws. The library's own request path avoids it by destructuring first, which looks like style and is load-bearing:

The detail that makes this genuinely nasty: Node does not enforce it. Undici's fetch accepts any receiver, so the same line runs fine in a Node process and throws in Chrome. A test suite running under Node cannot catch this even without mocks — which is a stronger statement than "the mocks hid it", and worth sitting with, because it means the platform your tests run on is part of your coverage story.

What this buys the layers above

Because resolution happens once and is shared, everything downstream gets auth, proxying, retry policy and a working fetch for free — including code that hasn't been written yet. When module 7 introduces the extension kernel, the reason an extension can be handed run(), connect() and fetch() with the parent client's credentials already attached is entirely this module. There is no second auth path to build, because there was never a first one.

Config field	Resolved how	Who reads it later
credentials	string, or a resolver called per request	every transport; token providers
fetch	eagerly, unbound global	request pipeline, context.fetch
requestMiddleware	composed with withProxy if proxyUrl set	every outbound request, incl. uploads
responseHandler	defaults to defaultResponseHandler	run, queue, storage
retry	merged with defaults, not replaced	request pipeline

Where this leads

We now know what is decided and where it lives. The next question is what actually happens to a single request as it travels: the order of middleware, why the auth header is applied after the URL is rewritten rather than before, and how a response becomes either a typed result or a thrown ApiError. That is module 2 — the request pipeline — and it is the last stop before transports start to differ.

Check yourself — you set proxyUrl and never touch storage. Uploads start failing. Why?

Because proxyUrl is not a flag on run() — it is composed into requestMiddleware, and every outbound request passes through that. Your storage uploads are now being rewritten to your proxy too.

If the proxy's allowlist doesn't include the storage endpoints, they are refused — which is precisely why the shipped defaults name two rest.fal.ai/storage/upload/* URLs explicitly. The failure is in a feature you never configured, caused by a field that reads like it only concerns inference calls.

The general lesson: in this library, config fields are rarely local. Ask "what reads this?" rather than "what did I set this for?"

One request, start to finish. The order of the steps is the whole lesson — change it and either your key leaks to your own proxy or your retries stop working.

Why order is the interesting part

Every fal call funnels through dispatchRequest. It looks like a thin wrapper until you ask when the auth header is attached relative to when the URL is rewritten. The answer is what makes a browser-side proxy safe at all.

// request.ts — dispatchRequest, in order
const credentials = typeof credentialsValue === "function"
  ? credentialsValue() : credentialsValue;          // 1. resolve, per request

const { method, url, headers } = await requestMiddleware({
  method: (params.method ?? "post").toUpperCase(),
  url: targetUrl, headers: params.headers,
});                                                  // 2. rewrite (proxy lives here)

const authHeader = credentials ? { Authorization: `Key ${credentials}` } : {};
const requestHeaders = {
  ...authHeader,                                     // 3. auth AFTER the rewrite
  Accept: "application/json",
  "Content-Type": "application/json",
  ...userAgent,
  ...(headers ?? {}),                                // 4. middleware headers win
};

Step 2 before step 3 is the load-bearing choice. The proxy middleware rewrites the URL to your server and stashes the real destination in x-fal-target-url. Because auth is attached afterwards and the proxy path resolves credentials on the server, a browser using proxyUrl never holds a fal key at all. Flip those two steps and the whole proxy feature becomes theatre.

Step 4 matters too: middleware headers are spread last, so middleware can override anything the pipeline chose — including Content-Type. That is deliberate extensibility, and also the reason a careless middleware can silently break a request body.

A response is not a value, it is a decision

defaultResponseHandler does more than parse JSON. It reads x-fal-request-id off the headers so a failure is traceable, and it picks the error class from the status:

const ErrorType = status === 422 ? ValidationError : ApiError;

422 gets its own type because a schema rejection is a different thing from an outage: one means your input is wrong, the other means try again. That distinction is what lets calling code branch on instanceof instead of string-matching a message — and it is the same instinct as module 8's rule about failures reporting what was observed.

Retry knows about a Node quirk

Defaults: 3 retries, 1s base delay, ×2 backoff, 30s ceiling, jitter on. The unusual part is what counts as retryable. Undici — Node's fetch — wraps a transport-level SystemError inside a plain TypeError("fetch failed") and hides the real code on .cause. A naive retry policy sees TypeError, decides it is a programming error, and gives up on a connection reset it should have retried. fal-js unwraps .cause to find the code.

That is the second time this doc has hit "Node's fetch differs from the browser's" — first the receiver check, now error shape. Worth generalising: fetch is a shared name over two different implementations, and this library is full of code that knows it.

Check yourself — why can a browser using proxyUrl hold no fal key?

Because middleware rewrites the request before auth is attached, and the rewritten request goes to your server. The browser's config has no credentials, so step 3 adds no Authorization header — nothing to leak. Your server receives the call, reads x-fal-target-url, and attaches the key on its side.

If auth were applied first, the key would already be on the request the browser sends, and the proxy would be forwarding a secret the page had to know.

fal-js can move work four different ways. They are not tiers of the same thing — they are answers to different questions about when you find out the result.

The distinction that matters

People reach for "realtime" as though it means "fast". In this library it names a specific transport: a persistent WebSocket for inference, where you push inputs and receive outputs on the same socket. It is not for media, and it is not the same thing as realtime.open(), which module 7 covers and which negotiates something else entirely.

Why they all know your credentials

This is module 1 paying off. None of these four contains an auth implementation. Each is built by a factory that receives the same RequiredConfig, so credentials, retry policy and middleware arrive already resolved. Adding a fifth transport costs nothing in auth code, which is exactly why a realtime extension could later be handed the same primitives without inventing a parallel path.

Check yourself — you need a 90-second video generation in a browser tab. Which transport, and why not the others?

subscribe(), the queue. run() holds one HTTP request open for 90 seconds, which proxies, load balancers and mobile radios all feel entitled to kill — and if it dies you have no handle to recover with.

The queue gives you a requestId immediately, so the work survives a dropped connection, a page reload, or a user switching apps. stream() is right only if the model emits partial output worth showing; realtime.connect() is for many small round-trips on one socket, not one long job.

The queue is the transport most worth understanding, because it is the one whose failure modes are about time rather than correctness.

Three calls wearing one method

subscribe() reads like a single operation. It is three: submit returns a requestId, status polls that id, and result fetches the payload once status reports COMPLETED. Each is separately callable, and that is the point — the id is a durable handle, so the work outlives the page that started it.

Polling, and why not webhooks

Default interval is 500ms. That number is a compromise nobody loves: shorter and you spend requests on nothing, longer and a fast job feels slow. The obvious objection — "use a webhook" — cannot work here, because a browser tab has no address a server can call back. Webhooks are available for server-side work; polling exists because the browser is a client and only a client.

The non-obvious cost: each poll is a real authenticated request through the whole pipeline from module 2, middleware and all. With a proxy configured, every poll traverses your server too. A 5-minute job at 500ms is roughly 600 proxied round-trips, which is worth knowing before you set pollInterval to 100.

Terminal states are detected, not awaited

There is no "done" event. The loop asks for status and checks whether it reads COMPLETED; anything else means poll again. Add a state server-side that the client does not recognise and a naive loop spins forever, which is why the check is for the terminal value rather than against a list of in-flight ones.

Check yourself — why does submit() returning an id matter more than it looks?

Because it decouples the work from the connection. Once you hold an id, every subsequent step is a fresh short request — so a dropped socket, a backgrounded tab, or a full page reload costs you nothing but the polling loop, which you can restart.

run() has no such handle: its only reference to the work is the open connection, so losing the connection loses the job. That is the same argument the WMA bridge makes with its session_id in module 6.

Streaming looks like the easiest transport and contains the subtlest bug class in the library: the boundary between "bytes arrived" and "a message arrived".

Why you cannot just split on newlines

Server-Sent Events look trivially parseable — data: {...} per line, blank line ends an event. The trap is that TCP does not deliver lines. A single read() can hand you half a JSON object, or three events plus a fragment of a fourth. Splitting each chunk on \n and parsing works in development, against a fast local server, and fails under load or over a slow network.

fal-js does not hand-roll this. It uses eventsource-parser, whose entire job is to hold partial frames across chunk boundaries and emit only complete events. Recognising that this is a solved problem — rather than "just split the string" — is the difference between a stream that works and one that fails intermittently in ways users cannot reproduce.

Two abort controllers, on purpose

FalStream creates its own AbortController and, when the caller also passes a signal, mirrors the caller's aborts onto it:

// streaming.ts
private abortController = new AbortController();
...
if (options.signal) {
  options.signal.addEventListener("abort", () => {
    this.abortController.abort();     // mirror, do not adopt
  });
}

Why not just use the caller's signal? Because the stream needs to abort itself — on timeout, on a terminal event, on teardown — and it has no right to abort a signal it does not own. The caller's controller might be shared across several operations; firing it would cancel unrelated work. Owning an inner controller and treating the outer one as an input keeps cancellation flowing one way only.

Check yourself — your SSE parser works locally and drops events in production. First suspicion?

Chunk boundaries. Locally, responses arrive fast and whole, so nearly every read happens to contain complete events and naive splitting appears correct. In production — slower links, larger payloads, intermediary buffering — reads land mid-frame, and any parser without cross-chunk state silently discards the fragments.

The fix is not more careful splitting; it is a parser that carries state between chunks. That is the whole reason eventsource-parser is a dependency rather than fifteen lines of local code.

Every transport so far shares one shape: your process asks, a server answers. Live media cannot be expressed that way, and understanding why is what makes the rest of this doc necessary.

The thing request/response cannot do

A generated video stream needs frames arriving continuously, in order, with latency low enough to feel immediate, and it must tolerate losing a frame rather than stalling to retransmit it. HTTP gives you ordering and reliability whether you want them or not; a dropped packet blocks everything behind it. For media that is exactly backwards — a late frame is worthless, and waiting for it makes the next one late too.

WebRTC exists for this: UDP-based, loss-tolerant, peer-to-peer. But it comes with a cost that shapes everything downstream — before two peers can exchange media, they must agree how to reach each other. That agreement is called signalling, and WebRTC deliberately does not specify how it happens. You bring your own channel.

Signalling versus media

This is the split that confuses people, so state it plainly. Signalling is a small amount of text — an SDP offer describing what you can send and receive, an SDP answer, and a list of candidate network paths. It happens once, at the start, and can travel over anything: HTTP POST, a WebSocket, a carrier pigeon. Media is the continuous flow afterwards, and it never touches the signalling channel.

So "which transport does realtime use?" has two answers, and that is precisely why the three fal extensions differ. Lucy signals over a fal WebSocket. Happy Oyster signals through a vendor SDK. The WMA raw path POSTs one complete offer to a bridge. All three then carry media over the same peer connection primitive.

ICE, and the trap that eats afternoons

Candidate gathering is the browser discovering routes to itself: host (your LAN address), srflx (your public address, learned from a STUN server), and relay (a TURN server that forwards traffic when no direct path exists). Modern WebRTC "trickles" these — sends each as it is found. The WMA bridge does not accept trickle: it takes one complete SDP, so the offer can only be sent once gathering has produced a set worth sending.

Which raises a question with three plausible answers and only one right one.

The third strategy is the one that works: wait until the set is sufficient — at least one server-reflexive candidate, and a relay candidate too if TURN is configured — then wait a short quiet period for the set to stop changing, under a hard timeout as a backstop. "Sufficient" has to require a relay when TURN is present, because a relay is the entire reason TURN was configured.

Check yourself — why can't the signalling channel also carry the media?

It could, technically, and it would be terrible. A WebSocket runs over TCP: ordered and reliable, so one lost packet stalls everything behind it while it is retransmitted. For video that trade is inverted — you want the newest frame now, and a frame from 200ms ago is worthless even if it arrives perfectly.

The deeper point: signalling and media have opposite requirements. Signalling is small, must arrive intact, and happens once — TCP is ideal. Media is large, continuous, and loss-tolerant — UDP is ideal. Using one channel for both means being wrong about one of them.

Three protocols, no shared wire format, and a client that cannot invent one. The kernel's answer is to standardise ownership instead.

The insight

fal cannot define one protocol for every world model — customers host their own, and a new one may arrive tomorrow with its own handshake. But every one of them opens resources, may be cancelled halfway, and must be torn down exactly once. That part is identical, and it is the part applications keep getting wrong.

So the rule is: fal owns the session lifecycle, an extension owns negotiation, the application owns presentation. An extension is ordinary installed JavaScript with one method that matters:

interface RealtimeExtension<Options, Session> {
  readonly id: string;
  supports(endpointId: string): boolean;
  open(context: RealtimeExtensionContext, options: Options): Promise<Session>;
}

The returned session may expose anything — steer(), roar(), send({type:"keys"}). Deliberately not standardised: a driving model and a character model have nothing useful in common at that level, and pretending otherwise would produce a lowest-common-denominator API nobody wants.

What the kernel guarantees

Situation	Behaviour
Caller aborts before opening	the extension is never invoked
Caller aborts mid-negotiation	signal propagates, registered resources released
open() throws halfway	cleanups run in reverse order
close() called twice	teardown runs once
Two installed extensions match	the client reports ambiguity rather than guessing

Reverse order is not decoration. Resources acquired later often depend on earlier ones — a data channel on a peer connection, a heartbeat on a session id — so releasing forwards means tearing down a dependency while something still holds it. Reverse is the same reason a stack unwinds the way it does.

Two details worth stealing

First, the returned session is wrapped in a Proxy. Reading close gives you the kernel's idempotent teardown, not the extension's — so an extension cannot accidentally offer a close() that runs twice. Reading state gives the kernel's value, so an extension cannot claim to be live while the kernel is still opening.

Second, ambiguity is an error, not a resolution. If two installed extensions both claim an endpoint, the client refuses. The alternative — pick the first, or the last registered — would make behaviour depend on import order, which is the kind of bug that appears only after a bundler upgrade.

Check yourself — why does the kernel own state rather than the extension?

Because only the kernel knows the whole truth. It is the thing that sees an abort arrive before open() was called, an open() that threw halfway, and a second close() that must do nothing. An extension sees its own protocol and nothing about cancellation semantics.

If both could report state, they could disagree — and the caller would have no way to know which to trust. Making it kernel-owned means an extension's optimism cannot contradict the kernel's teardown.

Everything above is the setup. Writing a third extension against that contract — as a customer, with no ability to change the client — found four places it came up short. None of them is a matter of taste.

1 · The context could not make a credentialed request to non-endpoint infrastructure

The context offered run() for fal endpoints and connect() for the fal WebSocket. The WMA bridge is neither: POST wma.fal.run/session, a different host with the app id in the body. And per module 1, an extension cannot reach resolved credentials — config.ts exports no accessor. So the only way to ship was for the application to inject a credentialed fetch:

// what every consuming app had to write
bridgeFetch: (url, init) => fetch(url, {
  ...init, headers: { ...init.headers, Authorization: `Key ${key}` },
}),

That hands auth for one leg of the connection back to the caller — the exact thing fal.realtime.open() exists to prevent. A server-side proxy hides the key and leaves the ownership. The fix is context.fetch(), delegating to the parent client's credentials and middleware, returning a raw Response because this reaches services that do not speak fal's result envelope.

2 · No common lifecycle, so no shared status UI

close() as the only universal member reads clean until an application offers two models. Lucy reported onConnectionStateChange; the WMA adapter onConnectionState; Happy Oyster a "world status". Moving a callback between two of them was a compile error, and a page offering three protocols needed a branch per protocol to render one status pill.

Fix: state and onState with four values — opening | live | failed | closed. Four, deliberately: anything finer is protocol detail. negotiating means something in Lucy and nothing in a world that spends thirty seconds building.

3 · Diagnostics were reinvented per extension, and one of them lied

The WMA adapter had invented three reporting callbacks because a failed connect otherwise says nothing. The fix is one onDiagnostic channel, deliberately not protocol-shaped — useful progress is "world building" for one model and "3 of 4 TURN servers answered" for another, so a phase plus a free-form detail bag carries both.

One convention travels with it, and it is a rule rather than a type:

A failure reports what was observed, never what was inferred. The first message cost three rounds of debugging on a router that was innocent. The second named the cause in one.

4 · Every WebRTC extension rewrites ICE gathering

Module 6's strategy is roughly sixty lines with four tuning constants, and none of it is specific to one protocol. Lucy trickles and does not need it — which is precisely why it belongs in the kernel rather than in whichever adapter met the problem first, or the next one writes it again, slightly differently.

And a fifth the work exposed

context.fail(). close() could not distinguish a transport that died from a user who disconnected; both arrived as closed, and those are the two cases a status UI most needs to separate. Now a dead peer connection reports failed — with a diagnostic — and then tears down.

Check yourself — why is context.fetch() better than a first-class context.wmaSession(offer)?

Because it solves the class rather than the instance. A WMA-specific method on a generic context is a layering inversion: the kernel would then know about one product's bridge, and the next protocol needing non-endpoint infrastructure — a regional relay, a control plane addressed by body rather than path — would need its own method too.

The counter-argument is real: a general credentialed fetch is a wider escape hatch, and every future extension can reach for it. That is a genuine cost, accepted deliberately, because the alternative grows the kernel one product at a time.

One last layer, because module 8's first gap has a sting in the tail: once the extension makes its own credentialed request, that request goes through your proxy — and the proxy had an opinion nobody had noticed.

Two allowlists, two questions

@fal-ai/server-proxy attaches FAL_KEY on your server so the browser never holds it. Without allowlists it is an open relay to your whole account, so it asks two questions: allowedUrlPatterns — which URLs may I forward to? — and allowedEndpoints — which of your apps may be called?

The second is skipped for *.fal.ai hosts and enforced everywhere else. And an empty allowedEndpoints means "allow everything". Put those two facts together and you get a genuinely perverse outcome.

Why the asymmetry existed

The exemption tested *.fal.ai. Storage uploads escaped the endpoint check for free because they live on rest.fal.ai; the signalling bridge did not, because it lives on wma.fal.run. Same category of thing — fal's own infrastructure, carrying no customer app — and opposite treatment, decided entirely by which domain it happened to sit on.

The fix names fal's service hosts explicitly and exempts them from both checks. Two details that look small and are not: it must bypass rather than appear in the defaults, because supplying allowedUrlPatterns replaces the defaults — a default only helps callers who never narrow the list, and narrowing it is the careful thing to do. And it must be an enumerated set, not a suffix rule on .fal.run, because fal.run and queue.fal.run serve customer apps and are what allowedEndpoints governs. Exempting the domain would leave that option restricting nothing.

Three rejections, one string

All three failure paths returned "Invalid request". A missing header, a disallowed host and a disallowed path were indistinguishable — which is what turned a configuration mistake into a source-reading exercise. Each now names the option to change, and none echoes the configured patterns: which check rejected you is something a blocked caller can already infer, while the pattern list would hand them a map of what the proxy may reach.

Check yourself — why not simply add the bridge to DEFAULT_ALLOWED_URL_PATTERNS?

Because supplying allowedUrlPatterns replaces the defaults rather than extending them. A default entry helps only callers who never narrow the list — and the caller most likely to narrow it is the one being careful, who is exactly the caller you do not want to break.

Bypassing the check for enumerated fal service hosts means the careful caller keeps a list that means only what it says — "my apps" — and signalling works regardless.