Authoring a tool

The walkthrough for a first tool, then the manifest, inputs, results and fixtures in full.

docs/authoring-a-tool.md Edit this page on GitHub

Writing, testing and shipping a tool, start to finish. Read architecture.md if you want to know why any of this is shaped the way it is.

1. What a tool is

A directory with four files:

tools/base64/  tool.json      what it is, what it takes, what it can return  index.ts       the function  cases.json     inputs with known answers  README.md      help text, shown next to the tool

The function is the whole tool:

run(input, ctx) -> Output

Three rules, and none of them are style preferences:

1. No DOM. run executes in Node during tests, in a worker where no DOM exists, and possibly during a site build. Reaching for document breaks all three.
2. A function of its inputs. No clock, no unseeded randomness, no network, no files. This is what makes a fixture mean anything. Randomness is fine if it comes from a seeded generator whose seed is an input; tools/queue-explorer does exactly that.
3. Cancellable if it can be slow. Any loop whose length comes from an input checks ctx.signal.

2. Walkthrough: build one

pnpm new-tool base64

writes tools/base64/ with the four files, already green under pnpm check: one input, one field of output, one fixture, and an error return with a fixture for it. Those last two are the parts people skip, which is why the template includes them rather than leaving them as an exercise.

The rest of this section is what you replace that starter with. A Base64 decoder is a good first tool because it has real errors, and it wants two result shapes at once. This is where it ends up:

Step 1: the manifest

// tools/base64/tool.json{  "sdk": 1,  "id": "base64",  "name": "Base64 decoder",  "blurb": "Decodes Base64 and shows what the bytes actually are, including where it breaks.",  "version": "1.0.0",  "capabilities": ["pure"],  "runtime": { "entry": "index.ts", "thread": "main" },  "kinds": ["group", "fields", "table", "error"],  "card": "live",  "cardFields": 3,  "inputs": [    {      "id": "source",      "type": "textarea",      "label": "Base64",      "description": "Standard or URL-safe alphabet. Padding optional.",      "primary": true,      "dir": "ltr",      "rows": 3,      "default": "SGVsbG8sIHdvcmxkIQ=="    },    {      "id": "view",      "type": "select",      "label": "Show bytes as",      "default": "hex",      "options": [        { "value": "hex", "label": "Hex" },        { "value": "ascii", "label": "ASCII" }      ]    }  ]}

sdk is 1 on purpose, because a v1 manifest still being valid is the compatibility promise made concrete: the current contract is 6 (§3), and declaring 1 opts out of everything added since (bytes in v2, samples in v3, through select presets and highlighted textareas in v6).

Declare the version of the newest feature you use, not the newest one there is. Validation refuses a manifest that uses a key, input type or kind newer than its sdk, and the fixture harness refuses a case or a sample whose result does, with the field, the version that added it and the sdk to declare: a runtime of the version you declared would load the tool and quietly ignore what it does not know.

dir: "ltr" matters more than it looks. Base64 is not prose, and inside a right-to-left page it would otherwise be laid out backwards.

Step 2: the function

// tools/base64/index.tsimport type { Output, Tool } from "@toolbench/sdk"; // A type alias, not an interface. An interface does not satisfy the SDK's index-signature// constraint, and the error you get for that is not obvious.type Input = { source: string; view: string }; const ALPHABET = /^[A-Za-z0-9+/\-_]*={0,2}$/; export default {  run({ source, view }): Output {    const trimmed = source.replace(/\s+/g, "");    if (trimmed.length === 0) {      return { kind: "error", message: "Nothing to decode.", input: "source" };    }    if (!ALPHABET.test(trimmed)) {      const at = trimmed.search(/[^A-Za-z0-9+/\-_=]/);      return {        kind: "error",        message: `"${trimmed[at]}" is not a Base64 character.`,        input: "source",        at,        len: 1,      };    }     const bytes = decode(trimmed);    return {      kind: "group",      parts: [        {          kind: "fields",          fields: [            { label: "Bytes", value: String(bytes.length) },            { label: "Padding", value: trimmed.endsWith("=") ? "present" : "absent" },            { label: "Alphabet", value: /[-_]/.test(trimmed) ? "URL-safe" : "standard" },          ],        },        {          kind: "table",          caption: "Sixteen bytes per row",          columns: [            { label: "Offset", align: "end" },            { label: view === "hex" ? "Hex" : "ASCII", mono: true },          ],          rows: rowsOf(bytes, view),        },      ],    };  },} satisfies Tool<Input>;

decode and rowsOf are left out; they are ordinary functions in the same file with nothing Toolbench-specific about them.

Four decisions worth copying:

• error for bad input, with input naming the control. That is what marks the textarea invalid and attaches the message to it for a screen reader. at and len say where in the string, so the host can point at the exact character.
• group because the summary and the bytes are both the answer. Not one or the other.
• align: "end" on the numeric column, because misaligned numbers cannot be scanned.
• mono: true on the byte column, because proportional hex is unreadable.

Step 3: fixtures

// tools/base64/cases.json[  {    "name": "decodes the canonical example",    "input": { "source": "SGVsbG8sIHdvcmxkIQ==", "view": "ascii" },    "expect": {      "kind": "group",      "parts": [        {          "kind": "fields",          "fields": [            { "label": "Bytes", "value": "13" },            { "label": "Padding", "value": "present" },            { "label": "Alphabet", "value": "standard" }          ]        },        {          "kind": "table",          "caption": "Sixteen bytes per row",          "columns": [            { "label": "Offset", "align": "end" },            { "label": "ASCII", "mono": true }          ],          "rows": [["0", "Hello, world!"]]        }      ]    }  },  {    "name": "rejects a character outside the alphabet, and says where",    "input": { "source": "SGVs*bG8=", "view": "hex" },    "expect": { "kind": "error", "message": "\"*\" is not a Base64 character.", "input": "source", "at": 4, "len": 1 }  },  {    "name": "empty input is an error, not an empty result",    "input": { "source": "   ", "view": "hex" },    "expect": { "kind": "error", "message": "Nothing to decode.", "input": "source" }  }]

Exact matching by default is the point. If the tool starts emitting a field twice, or reorders them, or returns something plausible alongside a correct error, the fixture fails. A "contains what I expected" check passes all three.

Step 4: run it

node --test tools/cases.test.ts

That file walks every tool directory, so a new tool is picked up with no registration. It checks the manifest validates, that id equals the directory name, that cases.json exists and is not empty, that every case's kind is declared in kinds, and then runs the cases and every sample the manifest declares.

On your own site, the same suite is three lines, because the walk is exported:

import { describe, it } from "node:test";import { checkToolDirectory } from "@toolbench/sdk/fixtures"; checkToolDirectory(new URL("../src/tools/", import.meta.url), { describe, it });

describe and it are passed in rather than imported, so the SDK keeps its zero dependencies and the same call works under Vitest. It lives on the /fixtures subpath because it reads the filesystem, and the main entry deliberately does not: @toolbench/runtime imports the SDK and runs in a browser.

Step 5: see it

pnpm bench

The bench globs tools/*, so the new tool appears in the gallery on the front page and at /tool.html?id=base64. Click through all three modes. Every visual bug worth mentioning in this repository was found by looking at the thing, not by reasoning about it.

3. The manifest, field by field

3b. Retiring a tool

Prefer retired over deleting. A URL someone has bookmarked, or that is linked from a post, should explain what happened rather than return 404.

deprecated still runs. The element shows a visible marker. A host styles it from outside the shadow root: data-status="deprecated" is set on <tool-host>, and the marker colours read --tb-mark-fg and --tb-mark-bg. It is not a live card. A landing page must not present a tool on the way out as current. The page and embed still work, so a reader who already has the URL can finish what they were doing.

retired does not run. The element refuses to activate: no form, no Run, and no tool code is fetched. It explains, and it renders links as the way onward. If there is a replacement, put it in links. That is what a bookmarked URL is for: this moved, and here is where to go, rather than a silent 404 or a tool that claims to be gone and then runs anyway.

{  "status": "retired",  "links": [    { "label": "Use the new converter", "href": "/tools/new-converter/" }  ]}

Do not keep card: "live" on either. Validation refuses it. A compact slot that opens a retired tool, or that presents a deprecated one as current, is a field that lies.

See it live: Deprecated and retired tools

4. Inputs

Ten types. Each generates a labelled control with its description wired to aria-describedby.

{ "id": "text",  "type": "text",     "label": "Pattern", "default": "^a.*z$", "maxLength": 200 }{ "id": "body",  "type": "textarea", "label": "Input",   "default": "",       "rows": 6 }{ "id": "n",     "type": "number",   "label": "Samples", "default": 1000, "min": 1, "max": 100000, "step": 1, "unit": "samples" }{ "id": "mode",  "type": "select",   "label": "Mode",    "default": "fast",  "options": [{ "value": "fast", "label": "Fast" }, { "value": "exact", "label": "Exact" }] }{ "id": "trace", "type": "toggle",   "label": "Show trace", "default": false }{ "id": "day",   "type": "date",     "label": "Day",     "default": "2026-10-25", "min": "2020-01-01" }{ "id": "tint",  "type": "color",    "label": "Tint",    "default": "#1f5aa8" }{ "id": "key",   "type": "secret",   "label": "API key", "minLength": 16 }{ "id": "types", "type": "multiselect", "label": "Record types", "default": ["A", "MX"], "min": 1,  "options": [{ "value": "A", "label": "A" }, { "value": "MX", "label": "MX" }, { "value": "TXT", "label": "TXT" }] }{ "id": "limits", "type": "rows", "label": "Limits", "maxRows": 8, "minRows": 1,  "columns": [{ "id": "path", "type": "text", "label": "Path" }, { "id": "limit", "type": "number", "label": "Limit", "min": 1, "max": 10000, "unit": "req/s" }],  "default": [{ "path": "/api", "limit": 500 }] }

Things the validator will hold you to, and why:

• min and max are required on a number. They are the only thing between a bounded computation and a hang, and the element refuses a value outside them before your function is called. That means run can trust its range and skip the defensive check.
• A select needs two or more options, and the default must be one of them. One option is a constant.
• unit on any number that has one. 1000 alone is a riddle; 1000 samples is not.
• "control": "slider" on a number the reader explores by dragging (contract version 4, "sdk": 4). A native range control appears beside the number box, which stays editable for exact values. It uses min, max and step, so a slider can never produce a value the manifest calls invalid. Pair it with autoRun when the tool is instant, or the answer will not follow the thumb.
• "control": "segmented" on a select of two to five short choices (contract version 9, "sdk": 9). The options become one row of segments, the chosen one filled with the accent, which a reader sees all at once instead of opening a dropdown. They are native radios in a fieldset whose legend is the label, so a reader tabs in once and the arrow keys choose. When a segment would cut its label, measured as the tool resizes, the row becomes stacked radios, so long labels still read. run gets the option value and presets behave as on a dropdown. Any other control on a select is refused.
• A date input for a day, a time of day, or both (contract version 9). mode is date, the default, time or datetime, and run gets YYYY-MM-DD, HH:MM or YYYY-MM-DDTHH:MM, local, with no time zone: convert in run if you need one. min, default and max are written in the same format, each must parse, a day must exist, and min ≤ default ≤ max. The control is the browser's own picker, which lets a reader type a day past max and hands over an empty string while a date is half typed, so the runtime checks the value first and refuses it on the input, naming it by its label, before run is called. Samples and presets are held to the same check. mode on any other input is refused.
• A color input for a colour (contract version 9). The reader types six hex digits in a box, the labelled control, or picks from the system's picker in the swatch beside it; the two follow each other. run gets a lowercase #rrggbb, never alpha or shorthand, and the default must already be in that form. What is typed is trimmed and lowercased, and anything else is refused before run.
• A secret input for a password, a token or a key (contract version 9). A password field with an eye button to show it. It has no default, and a sample or a preset that sets it is refused, because a manifest is published: every reader would have the value. The form opens empty, and seed and a sample run see "". Its share is false unless you say true, so a link neither carries it nor fills it. minLength refuses a shorter value before run, by its length, and minLength on any other input is refused. ⚠️ Do not return the value in a result, not even as an echo: a result is on screen, announced to a screen reader, kept in the seed a host may store, and in the tool-run event a page listens to. Return what you did with it.
• A multiselect for several choices from a fixed list (contract version 9, "sdk": 9). run gets a string[] of the chosen option values, in the options' order, so declare it as type Input = { types: string[] }. Up to six options are checkboxes and more are chips, a row of toggle buttons that wraps; "control": "checkboxes" or "chips" picks one. min (default 0) and max (default every option) bound the list: at max the rest are marked unavailable, with the reason in the count line under the group, and below min the reader's Run is refused with "Record types: pick at least one." under the input, so run never sees too few. The options follow a select's rules, without sets. default, a sample and a preset's sets give the list as a JSON array of option values, each once and within min..max, and so does a case in cases.json; anything else is refused naming the item. A shared link writes one parameter per choice, ?dns.types=A&dns.types=MX, and reading a list with an unknown option, a repeat or more than max opens on the whole default instead.
• rows for a list the reader edits (contract version 9): hosts to check, or paths and their limits. run gets a Row[], one object per row keyed by column id, even with one column, so declare type Input = { limits: Row[] } with Row from the SDK, and read limits.map((r) => r.path). columns takes one to four, each a text, number, select or toggle with that type's own keys (maxLength, min/max/step, options), a label and an optional unit, and no default: a new row's cell is empty, the number nearest 0 in range, the first option, or off. maxRows is required, at most 50, for the reason a number's max is; minRows defaults to 0, and below it Run is refused with "Limits: add at least one row." under the input. The reader adds a row with Add, removes one with its cross, and moves one with the arrow keys on its handle, Alt and an arrow from a cell, or by dragging the handle. At maxRows Add stays focusable, marked unavailable, with the reason beside it. Number cells arrive in range: one typed or linked outside it, or emptied, is refused before Run under its row, "Limits, row 2: Limit must be between 1 and 10000, got 9999999.", and a host's write is clamped. default, samples, presets and cases give rows as JSON objects with a value for every column and nothing else, checked cell by cell. A link writes one key per column, zipped by position, ?rl.limits.path=%2Fapi&rl.limits.limit=500, and a link whose columns differ in length, or with more than maxRows, opens on the whole default. Refuse a row from your own code with row on the error, its index from 0, and the message is shown under that row: "Limits, row 2: Path "/api" is already in row 1."
• "lang" on a textarea whose text has structure (contract version 6, "sdk": 6): a DSL, a config, a log. The host's highlight(source, lang) hook, the one that paints code results, paints the textarea too, behind the real one, so the caret, selection, undo and screen reader stay the browser's. Treat it as a hint: a host with no hook, or a hook that does not know your language, gives the reader the plain textarea, so the help text still has to explain the syntax. The paint catches up 250 ms after typing stops and is skipped past 20,000 characters. Pick the name your host's highlighter uses (json, yaml); it must be one word, and lang on any other input type is refused.
• A grammar beside lang for a notation no host knows (contract version 7): a state-machine syntax, a packet filter. The runtime tokenises it and paints it the same way, behind the real textarea:
  {}JSONCopy JSON code

  "lang": "doors","grammar": { "keywords": ["state", "on"], "comment": "#", "operators": ["->", ":"], "strings": "\"" }

  Four keys and nothing else. keywords are whole words, case-sensitive, at most 64 of up to 32 letters, digits or _. comment starts a comment that runs to the end of its line. operators are literal, the longest match winning, at most 32 of up to 4 characters. strings lists the characters that open a string, each closed by itself, a backslash escaping. Comment, operators and string delimiters are ASCII punctuation. There is no pattern key, and any other key is refused, because a regular expression a tool ships could backtrack for seconds on a line the reader pasted and freeze the page it is embedded in. A host's hook still goes first: when it paints your lang, its paint is used, and when it hands back plain text, which is how a hook says it does not know the language, or there is no hook, the grammar paints. The tokeniser lives in the same lazily fetched chunk, so a page without a painted textarea pays nothing for it. grammar without lang, or off a textarea, is refused.

• description is not a placeholder. Placeholder text vanishes when someone types, which is when they most want it.
• description takes light markup, for help that is a list rather than a run-on paragraph:

  Write	Renders
  A blank line	A new paragraph
  Lines starting with -	A bulleted list
  `text`	Code
  **text**	Bold
  {good|text}, {warn|...}, {bad|...}, {muted|...}, {accent|...}	The text in that tone's colour, bold except for muted

  It is built as elements, never parsed as HTML, and there are no links. Anything else is text, a marker left open included, so a description with none of it renders exactly as before. A tone colours words that already say the thing: "{bad|refused}", not a red word standing for refusal. On a runtime older than this, the characters show as written.

• "reset": false at the top level of the manifest (contract version 8) hides the Reset control, which puts every input back to the values the tool opened with (#208), exactly as reset="none" on the element does. For a tool with one input, where retyping it is the reset. Any value other than a boolean is refused.
• details for the rest of the help (contract version 8). Keep description to the one-line hint and put every case, format and reason in details: it is drawn under the hint behind a closed "More" that the reader opens, it takes the same markup, and it is not part of what a screen reader announces on focus, which stays the hint. A 600-character description was up to 13 lines on a phone and read in full every time the control took focus.
• Mark the inputs a card cannot do without as primary. A compact card shows every primary input and nothing else, so mark as many as the question genuinely needs and no more: a capacity planner offering a rate without a duration is not a smaller version of itself, it is a broken one. Mark none and a card shows the first input. Every input is shown on the full tool either way.
• dir: "ltr" on anything that is not prose: bytes, code, patterns, identifiers.
• role on an input a host may want to style by name (contract version 8): "credential", "advanced". It is rendered as data-role on the input's row, beside data-input and data-type, and the runtime gives it no meaning. It must be a non-empty string. See STYLING.md.
• "share": false on an input whose value must never be in a URL (contract version 8): a password, a token, a private key, or a textarea a reader pastes private material into. A host building a link from host.values leaves it out, and does not read it back from one (shareableInputs(manifest) in the SDK lists the rest), because a URL ends up in browser history, server logs, chat previews and referrer headers. It is a hint to hosts, not access control: the runtime builds no links, so it enforces nothing, and host.values still fills the input. With no share, an input is shareable, as every input was before version 8. The value must be true or false.
• maxLength bounds typing and samples, not default. The browser maxlength attribute stops a reader typing past it, and validateSamples refuses an over-long sample because assigning value walks past that attribute. The same check is not applied to default: adding it would reject manifests contract v3 still accepts, which an additive change may not do. A long default still loads. Keep it inside maxLength yourself if you set both.

  ⚠️ Why that gap stays open while duplicate sample values are refused, since the two look like opposite decisions taken in the same breath. A refusal is only worth its cost when the thing refused is never what anybody wanted. Two buttons filling identical values are in that category: there is no tool for which that is the intended behaviour, so refusing it can only catch a copy-paste. An over-long default is not: an author may genuinely want the form pre-filled with text the reader is expected to trim, and refusing it would break a manifest whose author chose it deliberately. The test is not whether a rule is tighter, it is whether a correct tool could ever trip it.

4b. Sample inputs

Nobody arrives at a tool knowing what to put into it, and the defaults can only demonstrate one thing. samples is a list of labelled example inputs. The runtime draws them as a row of small buttons under the form, introduced by a visible Try:, and a click fills the form with that sample's values.

// tools/percentiles/tool.json, three of the four samples it ships"samples": [  {    "label": "Two clusters",    "input": { "values": "2 2 3 3 3 4 4 4 5 5 5 6 6 7 7 180 190 195 210 240 260 300 340 900" }  },  {    "label": "Definitions disagree",    "input": { "values": "3 4 4 5 6 7 9 14 40 260", "method": "linear" }  },  {    "label": "Not a number",    "input": { "values": "12, 14, 15, 18ms, 21, 24, 31, 44" }  }]

input is keyed by input id, and it is partial on purpose. Two of the three above set only values and say nothing about method, so whichever definition the reader had chosen stays chosen. That is the usual shape and usually the right one: change the one thing the example is about, and leave everything else where the reader put it. "Definitions disagree" sets both, because the point it makes needs both. Under the linear definition those ten measurements give a p90 of 62.0, and nearest rank gives 40 for the same data; a sample that quietly reset the definition to the default would put half of that comparison out of reach.

Choosing them

Picking the examples is an authoring skill rather than a step in a procedure, so it deserves more thought than the JSON does.

Ship the malformed one. A sample whose input the tool rejects is the most useful example a tool can offer. It is the one a reader will never type by hand, because nobody sits down to invent a broken input, and it is the only way they see how the tool behaves when something is wrong before it happens to them for real. It also says something about the tool: that its failure path was designed and is being shown on purpose, rather than discovered by accident at the least convenient moment. percentiles ships "Not a number", where a stray 18ms sits in the middle of an otherwise clean list. A tool with no such sample hides its failure path until a reader stumbles into it.

Three or four, not eight. A row of eight buttons stops being a set of examples and becomes a menu, and a menu is one more thing to read before the tool can be used. Each sample should show something none of the others do: the canonical case, one that exposes a distinction the tool exists to explain, an edge, and the one that fails. If two samples make the same point, keep the clearer one and delete the other.

A label says what the reader is about to see, in two or three words. It sits on a small button in a row, so there is no room for a sentence, and "Two clusters" or "Definitions disagree" tell a reader what they are choosing. "Example 2" and "Test data" tell them nothing. Two samples may not share a label: identical buttons that do different things are not something anyone can choose between. They may not share an input either: two labels for the same values are two buttons that do the same thing.

What a click does

A click fills the form and stops there, for the same reason typing does not run the tool: Run is the trigger, and a sample is an input change like any other. An existing result goes stale, dims, and waits, which is what lets a reader hold the answer they already have beside the example they just loaded. A tool that set autoRun: true re-runs instead, again exactly as it would for typing. Samples are not an exception to the rule about Run, they follow it.

Two details are deliberate rather than incidental. Values are written into the live controls rather than by rebuilding the form, so focus stays on the button that was pressed and a reader can move along the row from the keyboard. And the status region announces that the form was filled, naming the sample, and that Run is next: for somebody who cannot see the form change, that announcement is the only evidence the click did anything at all.

A card shows no samples. A compact card has room for one input and a Run button, and a row of buttons there would crowd out the result the card exists to show. Samples appear in page and embed mode only, so do not design a tool whose form makes no sense without them.

A host page can still put its own example buttons next to a card, or prefill a deep link, without those examples living in the manifest. <tool-host> has a values setter and a run() method for that. values is partial and does not run the tool. Ship samples for examples that belong to the tool. Leave page-specific ones to the host.

What validation will hold you to

validateManifest refuses the manifest, naming the field, when:

• a sample names an input the tool does not declare. The message lists the ids that do exist, because this is almost always a typo;
• a value's type does not match its input: a string where a number is declared, a non-boolean for a toggle;
• a number falls outside that input's min and max;
• a string is longer than that input's maxLength;
• a select value is not one of that input's options;
• a sample's input sets nothing at all;
• samples is present but empty, which is a key that says nothing;
• two samples share a label. The message names the second colliding sample, not the list;
• two samples fill the same input values, even with different labels. Comparison sorts the keys, so { a: 1, b: 2 } and { b: 2, a: 1 } are the same payload. Two buttons that do the same thing are the copy-paste the label check misses.

⚠️ The runtime marks a reader's out-of-range number, and validation rejects an author's. A reader who types 9999 into a field that stops at 1000 keeps 9999 on screen, marked with "must be between 1 and 1000, got 9999." under it, and Run refuses with that sentence, so run keeps its guarantee that values are in range. A shared link carries the 9999 and opens marked the same way. It used to clamp, which ran and shared 1000 while the box still said 9999 (#262). A host's values write is still clamped. An author's sample is static data that CI reads before any reader sees the tool, so it can be a build error naming the field, fixed once. Clamping it instead would ship a button whose label promises one thing and whose value quietly does another, and nobody would ever notice.

maxLength is the sharpest case. It reaches the browser as the maxlength attribute, which constrains typing and nothing else. Filling a control from a sample assigns its value directly and walks straight past it, so an over-long sample would be the one route to a value the field itself says is impossible.

An input's own default is not checked against maxLength. That is deliberate. Closing the gap would narrow a rule existing tools already pass, and this SDK has no warning path. maxLength bounds typing and samples. It does not bound default. Keep the default inside the bound yourself if you set both.

What the build will hold you to

checkToolDirectory runs every sample a tool declares, so a declared example that crashes the tool fails the build with the sample's label in the message. An { kind: "error" } result is a pass: the malformed sample is the one that earns the feature, and rejecting bad input is the tool working. A sample that overruns its budget is reported as slow rather than as a crash, so the two diagnoses stay apart.

See it live: Sample inputs

4c. Presets on a select

When the right values for several inputs depend on one earlier choice, make that choice a select whose options carry sets. Choosing an option writes those values into the form, and the reader edits from there. Contract version 6, so declare "sdk": 6.

{  "id": "stack", "type": "select", "label": "Library", "default": "a",  "options": [    { "value": "a", "label": "Library A", "sets": { "read": 30, "retries": 2 } },    { "value": "b", "label": "Library B", "sets": { "read": 60, "retries": 3 } },    { "value": "custom", "label": "Custom" }  ]}

A preset or a sample? A sample answers "show me an interesting case" and sits under the form. A preset answers "start from what I actually run" and sits where the choice is made. If the reader picks it before anything else, it is a preset.

What a choice does

Exactly what a sample click does: the named inputs are written, the others keep what the reader left in them, an existing result goes stale (or re-runs, under autoRun), and the status line names the option. run receives the select's own value too, so a tool can branch on it.

The select follows the form

The select never names a preset the values have left. After any change to an input a preset names, by typing, a sample or the host, it shows the option whose sets all match the form, else the first option with no sets, else it stays where it was. So ship a "Custom" option: without one, an edited form keeps saying "Library B" over values that are not Library B's. Editing an input no preset names never moves the select, and when it does move, the status line says so, because a screen-reader user has no other way to hear that a control they did not touch changed.

A host writing the select through values is taken literally: { stack: "b" } selects Library B and fills nothing. A host that wants the values writes them too.

On a card, a primary preset select fills inputs the card may not draw. The select's label then describes what Run will use, which is why this is allowed where a sample row is not.

What validation will hold you to

sets is checked with the same rules as a sample's input: every key names a declared input, every value has that input's type, numbers sit inside min..max, strings inside maxLength, select values are options. Clamping is not on offer, for the same reason. On top of those:

• a preset may not name its own select;
• a preset may not name another select that has presets. Chained presets would let one choice rewrite another's values;
• two options of one select may not set the same values, compared with keys sorted, because the select could not tell which one the form matches;
• the default option's sets must equal those inputs' defaults. The form opens on that option;
• sets with no keys is refused, and sets written on the input instead of an option is refused with a message saying where it belongs.

See it live: A select that fills other inputs

5. Results

{ kind: "fields", fields: [    { label: "p50", value: "120", unit: "ms" },    { label: "p99", value: "1450", unit: "ms", tone: "warn", note: "tail is 12x the median" },    { label: "Arrival rate", value: "800", group: "Inputs" },]}

tone is "good" | "warn" | "bad", for when a number's meaning is not obvious from the number, or "muted" (below). note is the sentence a reader would otherwise have to work out. group puts fields under a heading, which is how one result holds two comparable sets: tools/queue-explorer uses "Formula" and "Simulation" side by side.

A field's value is set in the monospace face, which is right for an address, a hex string or a header value the reader compares character by character. mono: false sets it, and its note, in the text face instead, for a sentence or a plain number with a unit, which in monospace reads like terminal output. One list can hold both: { label: "Mapped address", value: "192.0.2.10:54321" } beside { label: "Integrity", value: "No integrity attribute present", mono: false }. Contract version 8.

A table column can say how its text wraps, so the table survives a phone: wrap: "none" for a short token kept on one line (a hash, a date, a status word, a row's name), wrap: "prose" for sentences that need a reading width. A cell's own wrap overrides its column's. Say nothing and nothing is claimed, as before. It is rendered as data-wrap on the header and every cell. Contract version 8.

rowHeader: true on a column makes its cells each row's name: they render as <th scope="row">, so a screen reader announces the name with every cell of the row, and on a narrow tool the column stays in view while the rest scrolls. Mark the column that names the row, usually the first. Contract version 8.

With wrap on its columns, a table has a narrow layout built in: on a tool narrower than 34rem it keeps its columns and scrolls, with the tokens on one line and the prose at a reading width.

{ kind: "table",  columns: [{ key: "n", label: "Bucket", align: "end" }, { key: "hits", label: "Hits", align: "end", mono: true }],  rows: [[{ value: "0-10" }, { value: "812" }], [{ value: "10-20" }, { value: "39", tone: "warn" }]],  caption: "Latency distribution" } { kind: "text", text: "Converged after 4 iterations.", mono: false } { kind: "code", lang: "json", source: '{"ok":true}' } `code` is readable preformatted text with `data-lang` set. The runtime does not highlight it: ahighlighter would roughly double `@toolbench/runtime`, and a host usually already has one. Pass`highlight?: (source: string, lang: string) => Node` to `defineToolHost` to replace the text nodeinside the block. The return type is a `Node` on purpose. A string would need `innerHTML`, which[SECURITY.md](../SECURITY.md) forbids. Omit the hook and the source stays plain text. { kind: "bytes",  bytes: [72, 195, 169, 108],  offset: 0,                    // what to LABEL the first byte, for a window into something larger  caption: "Highlighted runs are one character each",  highlight: [{ at: 1, len: 2, label: "U+00E9 é", tone: "normal" }] } { kind: "group", parts: [ /* any of the above */ ] }

Four notes on bytes, added in contract version 2 for wire formats. Every highlight needs a label, because colour alone tells a reader something is interesting without saying what, and tells a screen reader nothing at all: the labels become a legend under the dump. at is always relative to bytes, never to offset, which only changes what the first row is labelled. And tone is for real severity only: the default palette resolves warn and accent to the same colour, so a tone used decoratively implies a distinction the reader cannot see. And you do not need a tone to tell ranges apart: a range without one is drawn in the next chart series colour, --tb-s1 to --tb-s6 in order, coming round to the first again after six, and its legend swatch matches. A toned range keeps its tone and takes no series colour, so the next untoned range takes the next one. Past six ranges two share a colour, which is one more reason the labels carry the identity rather than the colour.

Two notes on group. Order matters: a card renders only the first part, so put the part that works alone at the front. And declare every kind you use, "group" included.

Naming parts for a host's stylesheet and links

Contract version 5 ("sdk": 5) adds three things a host can use. None changes how anything looks on a host that styles nothing.

• role on any part, or on any field: a name such as "verdict" or "stats", rendered as data-role. A host can then draw "the verdict" large without counting rows, which breaks the day you add a field above it. Name what a part is, not how it should look: "verdict", not "big-red".
  TSTypeScriptCopy TypeScript code

  { kind: "fields", role: "stats", fields: [{ label: "Checked", value: "8" }, { label: "Risky", value: "2" }] }{ kind: "fields", fields: [{ label: "Verdict", value: "2 fields accept any text", tone: "bad", role: "verdict" }] }

• id on any part: a name unique within the result, rendered as data-part, so a page can link to that one part with host.reveal(id). Where role says what kind of part it is, id says which one. Keep it stable across inputs, because a page's link is written once. A fixture fails if a result uses one id twice.
  TSTypeScriptCopy TypeScript code

  { kind: "series", id: "trade", chart }

• tone: "muted" on a field or a table cell, for the routine majority of a dense table, so the cells that matter are not buried among cells coloured like them. Charts and byte ranges do not take it.

The names a host may select by, and what counts as breaking them, are in STYLING.md.

6. Errors

Two different situations, and a reader can tell them apart only if you keep them apart.

Bad input. Return an error. The tool worked.

return {  kind: "error",  message: `"${token}" is not a number.`,  input: "values",   // marks that control invalid, attaches the message to it  at: offset,        // character offset, if you know it  len: token.length,};

The input named by input is marked invalid and the message is shown under it, after the field's name: "Measurements: No measurements yet. Paste some numbers." So write the message to follow a name and a colon, and do not repeat the field's name in it. For a rows input, row names the row at fault, counted from 0, and the message is shown under that row as "Limits, row 2: ...".

Write the message so it says what is wrong, where, and what would be right. "nope" is not a number beats Invalid input. Never abbreviate an error; a browser test asserts that an error never ends in an ellipsis, because the truncated half is the half that says what to fix.

A bug in the tool. Throw. The runtime says "This tool hit a bug and stopped", puts the stack in the console, and does not pretend the input was at fault. Do not catch your own bugs and report them as input errors; that sends the reader off to fix something that was never broken.

See it live: Failure modes

7. Long-running tools

If the running time depends on an input, declare a worker and honour the signal:

"runtime": { "entry": "index.ts", "thread": "worker" },"timeoutMs": 8000

run({ samples }, ctx) {  for (let i = 0; i < samples; i++) {    if ((i & 0x3ff) === 0) {      if (ctx.signal.aborted) throw new DOMException("Aborted", "AbortError");      ctx.progress(i / samples);    }    // ... work  }}

• Check every 1024 iterations, not every one. Reading signal.aborted a million times is itself the slow part.
• ctx.progress(fraction) takes 0 to 1. The runtime coalesces calls to one animation frame, so calling it often is cheap, and the progress bar only appears if the run outlasts 400 ms.
• ctx.progress(fraction, partial) can pass a partial Output to draw as the run proceeds. Useful for a converging simulation. The final result always wins, whatever order the frames arrive in.
• Worker mode is a stability boundary, not a security one. It exists so a runaway loop can be terminated without freezing the page. It shares the page's origin and cookies.
• autoRun is refused here. A tool that declared a worker did so because it can be slow.

See it live: A worker, with a timeout

8. Fixtures

cases.json is an array of cases. Fields:

Subset matching is for results that are mostly computed numbers:

{  "name": "simulation lands near the closed form at rho = 0.8",  "input": { "lambda": 8, "mu": 10, "samples": 20000, "seed": 42 },  "match": "subset",  "why": "the simulated numbers are checked for convergence in convergence.test.ts; this case pins the shape and the analytic half",  "fieldCount": 8,  "expect": { "kind": "fields", "fields": [{ "group": "Formula", "label": "Utilisation", "value": "0.800" }] }}

Note where the work is split. The fixture pins the shape and the exactly-known numbers. The claim that the simulation is right lives in a unit test that can say why: it asserts the simulation converges on the closed form, that the same seed gives the same answer, and that Little's law holds. Pinning two hundred computed digits in a fixture would test that nothing changed, which is not the same as testing that it is correct.

Aim for four to six cases: the canonical example, an edge (empty, one element, the maximum), a bad-input error, and anything that was once a bug.

The harness enforces four things a tool cannot enforce about itself: that the manifest is valid and its sdk is one this SDK can read, that cases.json exists and is not empty, that no case expects a kind kinds failed to declare, and that every sample in the manifest runs. The fourth is there because a sample is the first thing a reader clicks, so an example that crashes stays invisible until one of them finds it. See §4b for what counts as running.

9. Charts

{ kind: "series", chart: {    title: "Wait time against utilisation",    x: { label: "Utilisation", min: 0, max: 1, format: "fixed2" },    y: { label: "Wait", unit: "ms", format: "compact" },    series: [      { name: "M/M/1", shape: "line", points: [{ x: 0.1, y: 11 }, { x: 0.5, y: 100 }] },      { name: "Simulated", shape: "line", axis: "right", points: [/* ... */] },    ],    annotations: [{ x: 0.8, label: "knee" }],}}

• shape is "line", "area", "bar" or, from contract version 4, "points".
• x can be text, from contract version 4: ["Mesh", "MCU", "SFU"]. Text values are categories, drawn evenly spaced in the order you give them, never sorted, and labelled with their text (cut short if the slot is narrow; the full text is in the data table and the readout). A bar chart's axis always starts at zero, since a bar's length is its value.
• When x is a set of separate items (frames, requests, runs) rather than a continuous quantity, a line between two of them invents a value that does not exist. Use markers: true on a line or area to draw every real point, or shape: "points" to draw only the points. Each series gets its own marker shape, so series with equal values stay distinguishable, and the legend keys each one by its marker. Both need "sdk": 4.
• Every chart answers a pointer and the keyboard with a readout: a crosshair at the nearest x and a card of every series' value there. You get it for free. To say what a point means, set readout.titles (a heading per x) and series.notes (the text for one series at one point, instead of its value), null wherever the default is right. readout.mode: "point" shows only the nearest series, and "none" turns it off. Contract version 4.
• Link a chart to tables. readout.highlightTable: true lights the chart's own data-table row as the readout moves. To link a table part in the same result, give the chart an id and the table link: { chart: "<id>", keys }, where keys[i] is the x that row i describes (null for a row with none, a total say). Pointing at the chart then lights the matching row, and pointing at a row moves the chart's readout to it. Links never reach outside one result.
• thresholds: [{ y, label, tone }] draws a labelled horizontal rule at a y value: a budget, an SLO, a limit. It is not a series, so it adds no legend entry, no table column and no readout row, and the scale widens to show it. tone is warn, bad or good for a limit that should read as one.
• stack: "<id>" on bar series stacks them at each x, in series order, bottom to top: each part and the total in one bar. Positive and negative values stack away from zero separately, unstacked bars sit beside the stack, and the data table gains a total column per stack.
• orientation: "horizontal" draws bars growing rightward, one row per x: for long names, and for a budget read left to right. Grouping, stacking, thresholds (as vertical rules) and the readout all carry over, and the down arrow steps through the rows. Bars only; other shapes are not drawn there.
• shape: "step" holds each value until the next x and then moves straight to the next one: for state sampled on events, a buffer depth or a configured rate, where a sloped line invents a gradual change. markers: true marks the start of each run.
• yScale: "log" or xScale: "log" draws that axis in powers of ten, for values across orders of magnitude. A log axis cannot show zero or a negative, so such points are left out and the data table says how many; bars are not drawn on a log y axis.
• shape: "band" with lower fills the region between lower and points, the upper edge: a p5 to p95 spread, a min-to-max envelope. It is drawn beneath every other series, a null in either edge breaks it, the readout reads it as a range, and the data table gives it low and high columns.
• x on a series gives it its own x values, one per point, for measurements that share no x: a scatter. Use it with shape: "points". The x scale covers every series, the readout snaps to the nearest point rather than the nearest x, and the data table lists one row per point.
• shape: "box" with boxes draws a box plot per x: whiskers low to high, a box q1 to q3, the median, and outliers. You compute the summary, so the whiskers mean what you say they mean; put it in the axis label. Set points to the medians. The readout and the data table show all five numbers.
• mirror: true on a horizontal bar chart draws series given as negative values to the left and writes every number as its magnitude: back-to-back comparisons and pyramids. Put both halves in one stack so they share a row.
• sizes on a points series draws each point as a bubble whose area is its size, scaled across the whole chart, with a key of reference sizes underneath. sizeLabel names the measure. By area, not radius, so a value twice as large looks twice as large.
• axis: "right" for a second scale. The renderer reserves the extra padding for it, so labels are not clipped.
• format is per axis, not per value. A mix of 20, 15, 10, 5.00 on one axis is a formatting bug, and the reason formatting is chosen once per axis.
• Six series colours, then they repeat. A chart needing seven series needs rethinking.
• Every chart also renders as a table inside a <details>, automatically. A chart is an image, and a screen reader gets nothing from an image. You do not have to do anything to get this, but it is worth knowing that your axis labels and series names are read aloud.

Heatmaps and pies

Two more chart kinds, from contract version 4, each its own output kind and its own lazily loaded chunk, so a page fetches the code for the kinds it draws and no others. Declare them in kinds.

{ kind: "heatmap", heatmap: {    xLabel: "Hour", yLabel: "Day", label: "Packet loss", unit: "%",    x: ["00:00", "02:00", /* ... */], y: ["Mon", "Tue", /* ... */],    values: [[0.2, 0.3, /* ... */], /* one row per y */],   // null for a cell with no value    scale: "sequential",                                   // or "diverging", with midpoint}} { kind: "pie", pie: {    label: "One voice stream, 48 kb/s", unit: "kb/s",    slices: [{ label: "Opus audio", value: 32 }, { label: "Headers", value: 16 }],    donut: true, total: "48 kb/s",}}

• A heatmap's colour is its encoding, so it is never the only channel: a key shows the scale, every cell answers the readout and the arrow keys, and the data table has every value. null cells are hatched. sequential is one hue for "how much"; diverging is two hues either side of midpoint, for "which side of a baseline". Hosts theme the ramps with --tb-heat-lo, --tb-heat-hi, --tb-heat-neg, --tb-heat-mid and --tb-heat-pos.
• A pie is drawn the way pies are hard to misread: in the order you give the slices, from 12 o'clock clockwise, every slice labelled with its share. More than six fold into "Other" and the data table says so; a negative value or a zero total draws a message instead of a chart. If your parts are many, or close in size, a bar chart reads better.
• explode: true on a slice pulls it out a little, to point at the part your text is about. A slice pulled out looks bigger than it is, so use it on one slice, not for decoration.

Radar charts, treemaps and Venn diagrams

Three more kinds, each its own output and its own chunk, from contract version 4:

{ kind: "radar", radar: { label, axes: [{ label: "Latency" }, /* ... */], series: [{ label: "Fibre", values: [9, /* ... */] }] } }{ kind: "treemap", treemap: { label, unit, root: { label: "All", children: [{ label: "Media", children: [/* ... */] }] } } }{ kind: "venn", venn: { label, sets: [{ label: "Web", size: 100 }, { label: "Mobile", size: 100 }], overlaps: [{ sets: [0, 1], size: 50 }] } }

• Radar: one spoke per axis, clockwise from 12 o'clock, on one scale from zero unless an axis gives its own max. At most four series are drawn; a null breaks a shape rather than pulling it to the centre.
• Treemap: each rectangle's area is its value; a branch's value is the sum of its children. Labels draw where they fit, and every node is in the readout and the data table. Three levels are drawn at most.
• Venn: two or three sets. Two are drawn exactly. Three cannot in general be drawn exactly with circles, so every region carries its count and the caption says the areas are approximate. Give each overlap as the full intersection: [0, 1] includes what is also in set 2.

See it live: every chart kind

9b. Graphs

Contract version 7 ("sdk": 7) adds graph: you say what is connected, and the runtime decides where everything goes. tools/build-order is the example.

{ kind: "graph", graph: {    label: "Package dependencies",          // the figure's name, and the table's caption    direction: "BT",                        // LR (default), RL, TB or BT    nodes: [{ id: "core", badge: "1" }, { id: "http", badge: "2" }, { id: "app", badge: "3" }],    edges: [{ from: "core", to: "http" }, { from: "http", to: "app", label: "uses" }],    highlight: ["core", "http", "app"],     // a path to emphasise, as node ids in order} }

What you choose, with its default:

• layout: layered (ranks along direction, for anything with a flow), tree and radial for a hierarchy, force for a network with no direction, manual for nodes you place with x and y.
• routing: orthogonal for layered and tree, straight for force and manual, spline for radial. A straight line that would cross a node is drawn as a polyline instead.
• Nodes: label (the id if you give none), a muted note, a shape (rounded by default; rect, pill, circle, ellipse, diamond, hexagon, parallelogram, cylinder, document, point, or record with rows), a tone for status or a color for category, a short badge, a group, and in a layered layout a lane or a fixed rank.
• Edges: a label and where it sits (labelAt), line (dashed, dotted), width (thin, thick), tone or color, a marker at each end (head, tail: arrows, UML ends, or crow's feet for entity-relationship diagrams), a side to leave or arrive on (fromSide, toSide), and on a record the row (fromPort, toPort).
• Groups nest through parent, and a reader can close one (collapsed: true starts it closed); lanes are straight bands across a layered graph. merge: "source" lets the edges leaving one node share a trunk, the bus of an org chart; otherwise no two edges ever share a stretch of line.

What the runtime does for you, so you do not have to:

• Every edge is also a row in a table, under the picture in a <details>, and a node with no edges is a row saying so. A reader who cannot see the picture still has the whole graph.
• The picture is one tab stop. The arrow along the flow follows an edge out of the focused node, the arrow against it goes back, the arrows across it choose among the edges, and a live region says where focus is. Space on two nodes highlights the shortest path between them.
• The same graph gives the same picture in every engine and in Node, so a screenshot or a test never depends on the browser.
• A graph past a hundred nodes is laid out off the main thread when the host configured a worker: its table shows at once and the picture follows.

The structural rules hold in your fixtures as well as on the page: ids are unique, every edge, port, group, lane and highlighted node names something that exists, groups do not nest in a circle, and a group's members share one lane. A case whose output breaks one fails, because the runtime would list that graph rather than draw it. Nothing about size is refused unless you say so: limits: { nodes: 200 } makes a larger graph a table and a sentence saying why.

See it live: graph layouts and families

10. Help text

README.md in the tool directory, referenced by help. The host renders it beside or below the tool. What earns its place:

• what the tool computes, in one paragraph;
• the formula, the algorithm, or the spec section, with a link;
• what it does not handle. This is the most useful part and the most often missing;
• where the numbers come from, if any are hard-coded.

10b. Seeding

If a host puts your tool on a card, it will compute a seed: your tool run against its manifest defaults, embedded in the page so the card shows a real result before any JavaScript loads.

Two consequences for you as the author, and both are easy to get wrong:

• Your defaults must produce a real result. seed() throws if the tool rejects its own defaults, so a default that returns error fails the host's build. Pick defaults that demonstrate the tool.
• Your defaults must be fast. Seeding runs at build time with a 5 second budget, and it is one tool among however many the host has. A default that takes seconds is a default nobody should ship.

You do not have to do anything to support seeding. It is worth knowing it happens, because it makes your choice of defaults visible to every reader who never presses Run.

11. Checklist

• id matches the directory name
• blurb is one sentence under 200 characters, and says what the tool does
• Every number input has min, max and a unit
• Every input has a description that is not a restatement of its label
• Exactly one input is primary, and it is the interesting one
• Non-prose inputs are dir: "ltr"
• kinds lists every kind run can return, "error" included
• Bad input returns error with input naming the control
• run touches no DOM, no clock, no network, no unseeded randomness
• Any input-dependent loop checks ctx.signal, and the tool declares thread: "worker"
• Four or more cases: canonical, edge, error, past bug
• Every subset case has a why that says what is deliberately not pinned
• Defaults produce a real result, quickly: they are what a seeded card shows
• If the tool ships samples: three or four, one of them malformed, every label two or three words saying what the reader will see, and no two fill the same values
• README.md says what the tool does not handle
• node --test tools/cases.test.ts passes
• Looked at it in pnpm bench, in all three modes

12. Common mistakes