Choose the right test boundary

Use the smallest boundary that can expose the defect in the claim. localhost2137 is not the default for pure application logic or one fixed response.

Run the same app at two boundaries

The crash-course application calls Slack-shaped users.list. This first test owns a one-response HTTP stub:

import { execFile } from "node:child_process";
import { createServer } from "node:http";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";

const execFileAsync = promisify(execFile);

it("reads one scripted Slack-shaped response without service state", async () => {
	const requests: Array<Readonly<{ authorization: string | undefined; url: string }>> = [];
	const server = createServer((request, response) => {
		requests.push({
			authorization: request.headers.authorization,
			url: request.url ?? "",
		});
		response.writeHead(200, { "content-type": "application/json" });
		response.end(
			JSON.stringify({
				members: [{ id: "U_STUB", name: "Scripted Ada" }],
				ok: true,
			}),
		);
	});
	await new Promise<void>((resolve, reject) => {
		server.once("error", reject);
		server.listen({ host: "127.0.0.1", port: 0 }, resolve);
	});

	try {
		const address = server.address();
		if (!address || typeof address === "string") throw new Error("Expected a TCP address.");
		const appPath = fileURLToPath(new URL("../src/read-workspace.ts", import.meta.url));
		const { stderr, stdout } = await execFileAsync(process.execPath, [appPath], {
			env: {
				...process.env,
				SLACK_API_URL: `http://127.0.0.1:${address.port}/api/`,
				SLACK_BOT_TOKEN: "xoxb-scripted",
			},
		});

		expect(stderr).toBe("");
		expect(JSON.parse(stdout)).toEqual([{ id: "U_STUB", name: "Scripted Ada" }]);
		expect(requests).toEqual([{ authorization: "Bearer xoxb-scripted", url: "/api/users.list" }]);
	} finally {
		await new Promise<void>((resolve, reject) => {
			server.close((cause) => (cause ? reject(cause) : resolve()));
		});
	}
});

The second test runs the unchanged application against a seeded emulator world:

import { execFile } from "node:child_process";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { createTestRuntime } from "localhost2137/testing";
import { expect, it } from "vitest";
import config from "../localhost.config.js";

const execFileAsync = promisify(execFile);

it("reads one seeded world through the provider-shaped HTTP API", async () => {
	const runtime = await createTestRuntime({ config, port: 0, storage: "temporary" });

	try {
		const instance = await runtime.createInstance({ seed: true });
		try {
			const appPath = fileURLToPath(new URL("../src/read-workspace.ts", import.meta.url));
			const { stderr, stdout } = await execFileAsync(process.execPath, [appPath], {
				env: { ...process.env, ...instance.env },
			});

			expect(stderr).toBe("");
			expect(JSON.parse(stdout)).toEqual([
				{ id: "U000000", name: "localhost2137-bot" },
				{ id: "U_ADA", name: "Ada" },
			]);
		} finally {
			await instance.destroy();
		}
	} finally {
		await runtime.close();
	}
});

pnpm exec vitest run test/read-workspace-stub.test.ts test/read-workspace.test.ts

Both tests prove that the application builds the expected URL and authorization header and parses a successful response. Their additional evidence differs:

The stub is the better test when one request/response contract is the whole claim. The emulator earns its runtime when the defect needs service-owned state or behavior the stub would have to reimplement.

Read the evidence ladder horizontally

This is an evidence ladder, not a quality ladder. Replacing every unit test with a local service can make a suite slower without improving a pure logic claim. Replacing every provider boundary with a mock can make it fast while leaving serialization, state, callbacks, and SDK behavior untested.

Keep the application interaction named by the claim

Use operations to arrange prerequisites or represent actors outside the application. Exercise the application through its normal SDK, HTTP client, or callback handler when that integration is the subject. Observe application behavior through an application effect; inspect emulator state through a documented operation when that is useful evidence.

For example, arranging an existing product through an operation is honest setup for a test about how the application reads products. Creating a subscription through an operation does not prove that the application can create it through its provider SDK. Operations and emulated APIs shows those roles in one callback scenario.

Add a stateful emulator only when it changes the evidence

A plugin is a candidate when the claim needs at least one of these and a smaller boundary cannot falsify it:

• coherent state across provider-shaped requests;
• a local actor causing a signed callback or webhook;
• a controlled time transition or documented retry;
• privileged arrangement or inspection that the provider API should not expose;
• isolated service worlds without separate provider accounts.

The installed plugin must implement the required route, event, encoding, error, and sequence. A stateful runtime cannot turn an omitted capability into compatibility.

Keep external checks for external claims

No local emulator establishes real credentials, account permissions, dashboards, DNS, TLS, internet routing, regional behavior, production rate limits, scale, latency, or availability. A provider sandbox may cover some of these, but only for the scenario and account it actually exercises.

A useful suite often has three different jobs:

1. Function-level tests cover business rules and edge cases without a service process.
2. Focused localhost2137 tests cover supported protocol, state, callbacks, and controllable time.
3. A smaller provider-facing suite checks external credentials, configuration, and critical drift.

Duplicate a scenario at another layer only when the second boundary answers a different question. For instance, a unit test can prove an invoice-state rule, a local plugin test can prove that an SDK read and renewal callback apply that rule, and a provider check can prove deployed webhook wiring.

Stop before adding localhost2137 when behavior is wholly inside the application, one stub exposes the relevant mistake, the needed provider behavior is unsupported, an exact lightweight service already exists, or the goal is load/security/resilience testing. Using plugins gives the compatibility checklist; Getting started gives the complete local loop.