The weft book
Weft is a language and a runtime for programs that combine AI models, people and tools.
Running an agent puts one model in charge of the whole job, and you wait while it works. Weft goes the other way. You break the job into a graph of small pieces, called nodes. A node can run a model, open a webhook, ask a person, call an API or keep a database. Each one sees only what you wired to it, and the graph does the coordinating. The compiler checks every connection before the run, and every value that passes between nodes is written down where you can read it afterwards.
Let Tangle show you
Two things are yours to do, and both are quick. Install weft, which is mostly waiting, and decide which services you want weft to keep keys for. Then make a project:
weft new hello --assistant claude-code --remember
That puts Tangle in the project, and --remember makes Claude Code the default
so your next weft new needs no flag. Tangle is our weft specialist: it knows the
language, reads the catalog on your disk before it wires anything, writes any
node you are missing, and will build you a web frontend if you ask for one.
It also tests what it builds. It runs one piece at a time on a real input, looks
at what came out, and goes back to fix the node that got it wrong. You can hand
it a job and find the program working when you come back.
Open the project in your assistant and ask it to show you around, or just say what you want built. It walks you through everything below, pointing at your own screen while it does. You do not need the rest of this book.
Or read it yourself
If you would rather build with your own hands, or you just prefer reading, two sections get you to the same place.
Set it up gets you to a running program: install, your first project, the three things in a weft project that are alive on their own clocks, and how to talk to Tangle.
Build something real is where you take the wheel: editing the graph with a mouse, running one part of it on its own, connecting an account, giving your machine an address the internet can reach, and putting a person in the middle of a run.
After that the book stops being a path and becomes a reference. The language, writing your own nodes, connections, and what the runtime is doing underneath. Read a section when you need it.
The last part, why it is shaped this way, is the thinking behind the design: what weft handles for you, and how we decide what belongs in the language.
If something here is wrong
These docs are AI-written and the code moves fast, so mistakes get through. If you find one, or an explanation you cannot follow, tell us or come and say so on Discord. You do not need the correction worked out before you raise it.
Install
This gets you a weft runtime on your own machine, the weft command, and the
graph editor in VS Code. It is mostly waiting, so keep reading while it runs.
Docker has to be there first. weft’s runtime is a program on your machine. Its database, its file store and the infrastructure your programs ask for (a database, a model server) run in Docker containers next to it, and your programs themselves run as plain processes, each in its project’s folder. If Docker is missing, the installer stops and prints the link to install it.
You also need Rust whenever weft has to compile
something on your machine: a project with nodes of its own or with a
standard node you edited (under nodes/base_catalog/), a project you build
with --referenced, a live node test, and ./setup.sh --rebuild. The
install itself needs it on a branch’s install, which always builds the CLI
and the standard worker from source. The default install needs it whenever
it cannot use what CI published for your exact checkout: you have local
changes, CI has not finished building the commit you just pulled, you passed
--debug or --from-source, or the release has nothing for your kind of
machine. The installer says which one it hit, then compiles what it could
not download.
For which projects compile and which share the standard worker, go and read
the worker.
The installer is a bash script, so on Windows run it inside WSL.
Install
curl -fsSL https://weft.weavemind.ai/install.sh | bash
It downloads weft into ~/weft and runs the installer from there. To put it
somewhere else, start the line with WEFT_DIR=<folder>. Leave that folder
where it is afterwards: weft reads its standard library and its assistant
templates from it.
On a fresh machine the installer asks no questions: the first run starts the
database and pulls images, then sets the runtime up as a service of your user (systemd on Linux, launchd on macOS), so it comes
back on its own after a reboot. Each long step shows a spinner with the last
line it printed; if you want to see everything as it happens, add
bash -s -- --verbose in place of the final bash.
Put your keys in
Fill in the services you expect to use often, and leave the rest alone. We recommend at least OpenRouter, because a single key there reaches most models.
A service you set up here becomes one click forever after: a node that calls it picks the key up on its own, and your AI assistant can wire one in without stopping to ask you for anything. A service you skip asks you for a credential the first time you need it, which is fine for the ones you touch once.
cd ~/weft
cp access-apps.example.json access-apps.json
Open it and edit in place, leaving the services you are not using on their placeholder values. weft ignores an entry it cannot use, and reads the file each time it needs a key, so a change takes effect at once.
"openrouter": [{
"kind": "api_key",
"label": "Runtime key",
"key": "sk-or-REPLACE_WITH_YOUR_KEY"
}],
The file also ships entries for Slack, Google, Notion, Airtable, fal, ElevenLabs, Exa, Firecrawl and Mistral. Most of them just need a key pasted in. Slack and Google want you to register an application with that provider first, which is a longer job, so do those only if they are services you will live in. For the whole format, go and read the apps file.
Check it came up
weft daemon status
A healthy answer counts your projects:
weft: running at http://127.0.0.1:14111 (install 'default'); 0 project(s)
public address: closed
closed there is the normal state and not a fault. It means nothing of yours
is reachable from the internet, which is what you want until you need
a public address.
If it says weft: unreachable at ..., run weft daemon logs to see how the
runtime’s log ends (the line also names the log file), then run the install
line again and read the last block it prints. If that block has scrolled away, every run writes the
same thing to ~/.local/share/weft/setup-runs.log.
Two things that can go differently.
If weft is not found, the installer warned that its directory is not on
your PATH and printed an export PATH=... line. Put that line in your
shell’s startup file (~/.bashrc for bash, ~/.zshrc for zsh) and open a new
terminal.
If VS Code was closed, or code is not on your PATH, the installer built
the editor extension but could not install it, and printed the exact
code --install-extension ... command to finish the job. Without the extension
everything the weft command does still works; you just will not see the
graph.
The installer also needs Rust when it cannot download a prebuilt weft for
your machine, and Node 22.12 or newer with pnpm when it cannot download a prebuilt editor
extension. It tells you which, up front, and stops rather than half-installing.
Add the browser extension
When a program stops to ask a person something, the question shows up in a browser extension called Weft tasks. Install it now, from Firefox Add-ons or the Chrome Web Store, and close it again. It has nothing to show you yet. You connect it to your runtime in putting a person in the loop.
Keep your projects elsewhere
Your projects do not belong inside ~/weft. Go wherever you keep code:
cd ~/code
Then build your first program.
When you come back to this page
To change your keys, edit ~/weft/access-apps.json; the change takes effect at once
(the apps file).
To update, run weft maintenance on first, which takes down every
trigger that is on and stops your programs’ infrastructure, then run the
install line again, then weft maintenance off, which builds those projects again on the new
weft and brings back exactly what on took down. The install line updates ~/weft and runs the
installer, the same as git pull and ./setup.sh in that folder, and the
installer refuses to replace the running weft while something is on. Your
projects and their history stay where they are. Before you do, check the
changelog for
anything that needs you. For what each step does, go and read
updating weft.
If something in a project does not come back with weft maintenance off,
everything else is back anyway, and off says what went wrong. If the
message says the project’s copy of the standard nodes is older than this
weft, run weft catalog update in that folder (it replaces
nodes/base_catalog/, so any edits you made in there are lost); otherwise
fix what it says. Then run weft resync there, which builds the project and
brings back what the update took down.
If you installed weft before it stopped running on Kubernetes, that update is
the exception: its database cannot come along. ./setup.sh finds the old
install and asks before wiping it. The wipe deletes the old cluster, your run
history, versions and stored connections, and every container and image weft
made, along with every named install on the machine (the question lists them).
Your project folders stay, and weft run registers them again. If
./setup.sh has no terminal to ask on, run scripts/scrub-old-install.sh
yourself first.
If your install’s database was built before executions were renamed runs,
that update starts it over too. ./setup.sh asks before wiping that one
install’s database: its run history, versions and stored connections go,
and so does everything only that database gave a meaning to (the files the
install stored, and the workers and infra of its projects). Your project
folders, the install’s settings and every other install stay. Without a terminal to ask on, run ./setup.sh --yes.
To remove it, run ./setup.sh --uninstall in ~/weft. It stops the runtime and
removes the CLI and the editor extension. Your database, stored connections
and your programs’ infrastructure stay, so a later ./setup.sh brings
everything back. Adding --purge deletes all of that too, your programs’ own
databases included.
Your first program
Making a project takes one command, and it has one choice in it: which AI
assistant you use. If you run weft new hello with no flag, weft asks you,
listing the assistants it found on your machine first. Or pass --assistant
with your row from this table. Either way weft copies Tangle into the project. Tangle is a persona that teaches your assistant
the language, and it reads the catalog on your disk rather than trusting what
it remembers.
| You use | Pass |
|---|---|
| Claude Code | --assistant claude-code, or cc |
| Kilo Code | --assistant kilo-code, or kc |
| Cursor | --assistant cursor, or cu |
| Codex | --assistant codex, or cx |
| GitHub Copilot | --assistant github-copilot, or gh |
| Gemini CLI | --assistant gemini-cli, or gc |
| Cline | --assistant cline, or cl |
| OpenCode | --assistant opencode, or oc |
| Devin Desktop | --assistant devin-desktop, or dd |
| Junie | --assistant junie, or ju |
| Something else | --assistant agents, which writes a plain AGENTS.md |
| Nothing | --assistant none |
If weft is run from a script or by an AI agent, there is nobody to ask, so it
stops and lists these values: pass one. If you use more than one assistant,
pass --assistant once for each. When weft asks, it also asks whether to keep
your answer for later projects. A flag is for this project only, so if you
want your next weft new to install the same one with no flag, add
--remember, as here:
weft new hello --assistant claude-code --remember
cd hello
weft run
weft run says it is building, then prints the project’s name and id, then
the run’s id and how many nodes it runs, then a line for every node as it
starts and finishes, then a tick:
building hello
registered hello (3f2a1c4e-9b81-4d0a-8e55-7c2f1a6b30de)
started run 9b81d0a2-4e17-4a33-bc90-51d8e2f4a7c1 on version a1b2c3d4
2 nodes
→ started run=9b81d0a2 entry=greeting
…
✓ completed run=9b81d0a2
hello uses only the standard library’s nodes, so on your machine it runs
the worker program weft ships ready-made (the standard worker), and nothing
compiles. Once a
project has nodes of its own, or infrastructure, its build prints two lines
for each worker it compiles and each infrastructure image it builds,
indented under the rest:
building <image> as <build>; its log: <url>
built <image> in about <time>
The ; its log: <url> part shows only when the build has a log to point at.
That long id after started run is the run’s id, and it is how you
point at this one run later. Everything that reads a run takes it, and the
first eight characters are enough as long as they name only one.
If weft is not found, open a new terminal; the installer’s PATH line only
takes effect in a fresh one. If the run stops saying it cannot reach the
runtime, run weft daemon status and go back to Install.
See it as a graph
Open the hello folder in VS Code, click the Weft icon in the bar down the
left edge, and click your project under Projects. The source opens on the
left and the graph opens beside it. (If you opened src/main.weft yourself,
click Open Graph to the Side in the file’s title bar.)
Type in either pane and the other updates.
This is what weft new wrote, and what you just ran:
greeting = Text { value: "Hello, World!" }
out = Debug { data: greeting.value }
Two boxes and an arrow. A box is a node. The dots down its left edge are what it takes in, the dots down its right edge are what it sends out, and each dot is coloured by the kind of value it carries, so two dots of the same colour fit together.
The arrow is greeting.value sitting where out’s data input goes. Writing
a source and a port where a value belongs is how you connect two nodes, and it
is the only wiring syntax there is.
While a run is going, each box glows: amber while it works, cyan while it waits for something, grey while it is paused only because its run paused, green when it finished, red when it failed. Click the magnifying glass on a box to see exactly what went in and what came out.
For adding a box and drawing an arrow yourself, go and read working in the graph.
What weft new made
| File or directory | What it is |
|---|---|
src/main.weft | The program |
weft.toml | Its name and its permanent id |
nodes/base_catalog/ | Every node weft ships, copied in so the build never reaches outside your project. weft catalog update replaces this folder wholesale after you update weft, so keep nothing of your own in here |
nodes/ (anywhere else) | Nodes you write yourself. A node can also sit under src/, next to the module that uses it |
CLAUDE.md, .claude/ | Tangle, for whichever assistant you picked |
.weft/ | Build state, generated |
Two more folders turn up later rather than now: layouts/, the first time you
drag a box, and assets/, when something needs a prompt or a script kept in a
file of its own.
weft new also runs git init and writes a .gitignore. Tangle’s files are
ordinary project files, so they commit with the rest and anyone who clones your
project gets Tangle without installing weft.
Next, the three lifecycles: what starts on its own and what does not.
The three lifecycles
Three things in a weft project are alive on their own clocks. Infrastructure and triggers start only when you (or your program) start them, and runs also start when a trigger fires or a caller arrives. The one exception: if infrastructure stops or breaks, the triggers that read it go down with it, and weft turns them back on by itself once it runs again (unless the node’s own health rules say otherwise).
| What it is | You start it with | |
|---|---|---|
| Infrastructure | Containers your program asked for: a database, a model server, anything that has to keep running | weft infra start |
| Triggers | The listening: a webhook waiting, a timer ticking, a form open for answers | weft activate |
| Runs | One go through the graph, from start to finish | weft run, or a trigger firing, or a caller arriving |
Hitting Run never starts your database and never puts your webhooks live.
Your hello program has neither of the first two. It is two boxes and no
listening, so weft run is the whole story until you build something that
talks to the world.
Why they are separate
Infrastructure costs money while it runs, so weft will not start a container because you clicked something that did not mention one. If you run a program whose containers are down, it refuses up front and names them:
infra not running for: pg. Run `weft infra start` first.
Activating reaches out to other people’s systems. A trigger that waits for Slack messages registers a subscription with Slack during activation. A trigger holding a socket open dials it there and then. That is a real thing happening in somebody else’s account, which is why it is a separate decision from running your program once.
It also has a consequence you will meet: activation freezes the trigger’s
inputs, because those values went to the provider and the provider is holding
them. Edit one afterwards and your live listener carries on with what it
registered, until you run weft resync.
A run can wait as long as it needs. It ends completed, failed or
cancelled, or it parks at waiting_for_input while a node waits on a person
or a service. Parking costs nothing: weft lets the run’s worker shut down, and
a new worker picks the run back up when the answer lands, however many days
later. Once everything in the run waits, the worker first stays up a few
seconds (the trigger’s pauseAfterSecs, 10 unless you set it), so an answer
that comes quickly carries the run on right there, before it parks.
Otherwise the worker stays up only while the run cannot pause: a caller is still
on the line, or the trigger has recorded off. Such a run holds the wait in its
worker instead. Without a caller, if nothing moves in the run for its
trigger’s holdSecs (60 seconds unless you set it), the waiting node fails;
while a caller is on the line, the wait holds for as long as the caller
stays, unless holdSecs is 0, which fails it at once.
The rest
For every verb, what each status means, the difference between a run that is waiting and one that is stuck, and which command to run after you edit something, go and read starting and stopping things.
Next, talk to Tangle and have it build you something with all three.
Talking to Tangle
Tangle is the weft expert that weft new --assistant copied into your project.
Open the project in your assistant (the last lines weft new printed are the
command that does it) and it is already there. Describe what you
want in plain words and it builds it. You do not have to learn this language.
Say what it should do, not how
I want a WhatsApp bot I can forward voice notes to, and it replies with what was said in the audio. If I send it text, it just ignores it.
Tangle turns that into a graph and tells you the shape in a sentence or two before it builds anything. Say so then if the shape is wrong.
Notice what you did not say. Not which model transcribes the audio, not how the bot receives a message, and not how to tell a voice note from a text. That last clause, the one about ignoring text, is a branch in the finished graph.
What it does without being asked
It looks things up rather than remembering. The catalog on your disk is the truth, and Tangle reads a node’s real inputs and outputs before wiring it. The language moves fast, so anything the model remembers is probably stale.
It writes missing nodes. A node’s own logic is Rust, and if nothing in the catalog does what you need, Tangle designs the node’s inputs and outputs and hands it to a helper agent that writes that Rust.
It writes the prompts. When a model’s answer decides what happens next, Tangle hands that prompt to a helper that does nothing but write prompts, rather than typing one out as it goes.
It builds you a frontend. Ask for a page, an app or a site and Tangle puts
one in front/, talking to your program through the program’s own HTTP routes.
The default stack is SvelteKit, PostgreSQL, BetterAuth and shadcn-svelte, and
if you name a different one, yours wins.
It tests as it goes. Tangle runs one piece of the program on its own, with values it makes up or a real input you gave it. It reads what came out, fixes the node, then runs only what changed. When a case comes out right, Tangle freezes it, and after a later change it runs that case again and reads what moved. You do not have to be there for any of that.
It offers guard rails on risky programs. If strangers can post into your program, or it sends, deletes or spends money on its own, Tangle names the protections that slow things down (a check, a gate, a person who has to approve) and asks you before it builds. When the program is finished, it attacks it before handing it over.
Give it one real case
Paste in a real input: an actual email, an actual message. Then read what came out, and when it is wrong, say what is wrong with it rather than restating the goal.
This email is asking for a replacement, but the extracted request says the customer wants a refund. Fix the prompt for that node.
Tangle builds one stage, runs it on your case, and shows you the result before writing the next one. For the longer version of that habit, go and read Sequential Diffusion Programming.
When you want to do it yourself
The .weft file is the source, and the graph edits that same file. For every
command, go and read the CLI.
That is the first ten minutes. Next, work in the graph with your own hands.
Working in the graph
Everything here works with the mouse. Whatever you change on the canvas is
written into your .weft file, so the two never disagree. The one exception is
where a box sits: positions are kept in layouts/, beside your source.
Getting around
Scroll to move around. Drag empty canvas to move around faster. Zoom is the
+ and - buttons at the bottom left, with Fit View beside them for when
you drag something into the distance and lose it. Scrolling does not zoom, on
purpose, so a stray trackpad gesture cannot throw your whole graph off screen.
Hold Shift and drag to select several boxes at once.
Add a node
Press Ctrl+P (Cmd+P on a Mac) and search. Right-clicking empty canvas and picking Add Node… opens the same list.
The list holds every node your project knows about, and Group and Loop are
in there too.
It also holds the plain actions, so you can find them without remembering a chord: Undo, Redo, Duplicate Selected, Delete Selected, Select All Nodes, Fit View, and Auto Organize Layout, which tidies the whole canvas.
If you already know what should come next, drag out of a dot and let go over empty canvas. The list opens, and the node you pick arrives with the arrow already drawn. One Ctrl+Z takes away both.
If the text of the file changes outside the graph (you type in the code tab, or an AI writes the file), the graph forgets its undo history, because what it remembered no longer matches the text. Undo those earlier graph edits in the text editor instead. The graph never writes over a change made to the text.
Connect two nodes
Drag from a dot on one box’s right edge to a dot on another box’s left edge. The line takes the colour of the dot it started from.
Three things will refuse the drop, and each tells you which:
- The types do not fit. Same colour is the quick check. The full rules are in Types.
- The two boxes are in different groups. A value crossing that line has to go through the group’s own dots. That is what a group is for.
- The input already has a value written into it. You get
'<port>' is driven by a config assignment; unset it first to drive it with an edge.Clear the field, then draw the arrow.
Near the top of each node’s right edge, set apart from its output ports, there
is a small red circle with a ! in it. That is _error, where the node’s
failure comes out, and you drag a wire out of it as you would from any output. While nothing is wired
to it, it is an outline and a failure stops the run. Once a wire leaves it, it
fills in red, and a failure goes down that wire instead. For which failures go
there, go and read when a node fails.
To remove an arrow, grab it near one end and drop it on empty canvas.
To take one field out of a value instead of the whole thing, right-click the arrow. You get the keys inside it with their types, and clicking one narrows the arrow to that key. Up one level goes back and Read the whole value undoes it.
Values written by metadata ports
A metadata port is a value fixed in the program, which the compiler copies into every input the code wires from it, before anything runs. For what metadata ports are for, go and read metadata ports.
The graph draws neither the ports nor the wires from them: both are written
in the code (tools.search = search.tool, or a value gathered from several,
self.tools = [...screen.tools, pause.tool]). What it shows is where each
value comes from, and it lets you follow it:
- An input a metadata port fills shows the value, read-only, with a link
such as from
calls.toolsbeside its name. Click it and the editor takes you to that box, opening the groups it sits in. - A group’s metadata ports show in the Config strip at the top of the open group. A value written out in full is a field you can edit there. A value gathered from others is read-only, written the way the code writes it, with each port it names as a link: follow the links down until you reach the box that writes the text itself.
Change what a node does
The small arrow in a box’s header opens its body, where its fields are: text boxes, dropdowns, tickboxes, code editors, file pickers. A field fed by an arrow hides itself, because the arrow is now the source of that value.
Under the inputs there is a + input button, and output + under the outputs. A port you add with no type takes its type from what you wire to it. If you want to set one yourself, right-click the new dot and pick the Type: row.
That same right-click menu makes an input required or optional, removes it, or puts a catalog port back the way it was.
Double-click a box’s name to rename it. Right-click a box for Duplicate, Delete and Tags….
On an infra node, the same menu has One per instance: each instance of your program then gets its own copy of it, and it shows a per instance badge. Share across instances takes the mark off.
If you want each instance to get its own value for a field (an account, a spreadsheet, a schedule), click the ⧉ button beside the field’s name, or right-click the field and pick Let each instance fill this (the same entry is on the right-click menu of the field’s input port). The field then shows Filled per instance instead of a control, and the node a filled per instance badge. If you had already written a value, an instance that gets none gets that one. Click ⧉ again, or pick Stop letting each instance fill this, and that value becomes the one value for everyone again, or the field goes back to empty if there was none. A field with a wire into it offers neither, because its value comes from the wire.
A box that reads from a per-instance node or an instance-filled field runs once for each instance too, and shows a dashed per instance (follows) badge. For what all of this means, go and read programs with instances.
Run part of the graph
Select the boxes whose answers you want (click one, or hold Shift and drag around several). The selection bar appears at the bottom center of the canvas with a big RUN 2 TARGETS button (RUN 1 TARGET if you picked one box). Click it, or press Ctrl+Enter, and those boxes run together with everything they need. That is the same run as:
weft run --target dailyReport --target weeklyReport
To work out what a box needs, weft follows the wires back up from it and
stops at any trigger. That trigger does not fire in this run, so a box only a
trigger feeds is skipped. If you want that box to run, fire the trigger with a
made-up event from the CLI (weft run --fire). A box that two branches share
does not drag the other branch in. If the box sits inside a group, only the
boxes of that group it needs run, plus whatever feeds the switch that decides
whether the group runs at all (its _should_flow or _should_not_flow). If weft reaches a loop on the way, the whole loop comes along. A
box inside a loop needs what the loop hands its body, so picked on its own it
reaches the loop’s start without its end, and Run stays grey saying so. If
you want to run part of a loop’s body, start the run inside it from the
command line (weft run --from ... --target ..., see running part of a
loop’s body).
Triggers and infra have no Run button: select one and the bar offers its own buttons instead (Turn on, Turn off, Resync for a trigger; Start, Stop, Upgrade, Terminate for infra; covered below). If you select them together with other boxes, RUN N TARGETS counts and runs the other boxes. Pressing Ctrl+Enter with only triggers or infra selected tells you Run leaves them out.
While weft works out what the selection needs, the button is greyed out and
says Checking what this run needs... on hover. If the run cannot start,
the button stays grey and its tooltip gives weft’s reason: the refusal
weft run --check prints, or the infra the run reads that is not running
(Start db first: this run reads it, and it is not running.). Several boxes
selected are one run, so one blocked box blocks it: the tooltip names each
blocked box with its reason, and the boxes that could run on their own.
If you want the whole program, open the Weft icon in the side strip and
click Run Project on the project’s row. That is a plain weft run.
The editor has no button for starting partway in with values you supply,
running one group on its own, firing a trigger with a made-up event, running a
saved example (examples/*.json) or reusing what did not change since the last
run. For those, use the CLI: --from,
--group, --fire, --seed, or an example’s name after weft run. For the
rules behind them, go and read
versions, seeds and frozen examples.
Watch it go
The status bar, at the very bottom of the canvas, is the same bar as the
selection bar above: what is happening now comes first, then what the
selected boxes offer. With nothing selected and nothing going
on, it is not drawn (the eye buttons that show only the trigger or the infra
subgraph stay). While the run on the graph is going, it shows Stop and how far the run
got (run 3f2a... 4/9 nodes). Which run that is depends on the toggle in
look at an older run. The bar also shows work started
elsewhere: an
infra start typed in a terminal, or asked for by Tangle, shows as
Starting infra…, an activation as Turning triggers on…, a build as
Building…. If that work can still be cancelled, the label ends in
(Cancel), and clicking the label cancels it.
If a command fails, a red banner sits above the bar with the error. Click it for the whole message, and the × to dismiss it. A dark line above the bar shows the last thing the command printed, while it runs and, if it failed, for as long as the banner is up. Clicking it shows everything it printed.
Each box gets a glow and a small mark:
| The box | What happened |
|---|---|
Amber glow, ● | Running now |
Cyan glow, ◉ | Waiting for an answer from outside |
Green glow, ✓ | Finished |
Red glow, ✕ | Failed |
Grey ■ | You stopped it |
Grey ⊘ | Skipped |
| Purple glow | Reused from an earlier run instead of run again |
| Dimmed, no glow | Outside the part of the graph this run covered |
Click the magnifying glass in a box’s header, Inspect run, for what
went in, what came out, how long it took and what it cost. A failed box shows
its error there. A skipped box says why in words, such as
its _should_flow said no or the required input 'x' closed, and you follow
the arrows backwards from there to find the node that decided. A bus’s
messages end with * the bus closed here, or with * the bus failed here:
and the reason when it broke off (go and read how a bus
ends). If a node
changed a stored file in place (added a turn to a conversation, say), its panel
lists each change under Files edited: the file, the versions it moved
between, and the lines added and removed.
A box that ran several times, which happens inside a loop, gets a ‹ 2/5 › in
its header to step through the firings.
Look at an older run
The toggle at the top left decides which run the graph shows. Hover any of its three parts to see what it does:
- Following puts every run that starts on the graph. It is the default, and clicking Run in the editor always comes back to it.
- Locked keeps the graph on the run it shows, so you can read it while others start.
- Off takes the run away and leaves just the program.
While locked or off, runs that start are counted in a button beside the toggle, like 2 new runs · Follow, and clicking it shows the newest and follows again. A run started from a terminal, yours or an assistant’s, never changes the mode. Each project remembers whether you left it following.
For a run from yesterday, click the Weft icon in the side strip. Runs lists every run of the project you last opened, newest first, and each row has View in Graph, which puts that run back on the canvas with its values in place and locks the toggle onto it. A replayed run looks exactly like a live one.
If your project deploys to a cloud install, the switch at the top right shows the graph as that install runs it. For how it works, go and read Look at prod from the editor.
Turn triggers and infra on and off
Select a trigger or an infra node, or several (with the graph focused,
Ctrl+P opens its list, and Select All Triggers there picks every
trigger). The selection bar then offers what each can do right now, with a
count when a button covers more than one of the selected nodes:
Turn on, Turn off and Resync for triggers, Start, Stop,
Terminate and Upgrade for infra. These are weft activate,
weft deactivate, weft resync and the weft infra commands.
If you want to choose how triggers go down, options… (shown when a
selected box has something to choose) asks you three
things. What happens to work that arrives while they are down: Hold it,
Hold it, hidden (held, but not listed for the people who answer your
program’s questions), or Refuse it. If you hold it, you can also say “then
refuse after N minutes”. What happens to work already waiting: Keep it or
Drop it. And what happens to the runs going now: Wait for them or
Cancel them.
Each starts on the default the weft commands use: Hold it, Keep it
and Wait for them. If you are stopping, terminating
or upgrading infra, the panel asks the same three questions again for the
triggers that go down with it, and names them. For turning triggers on, it asks
what to do with what they held while they were down: Run it or Drop it.
For what each answer does, go and read
taking triggers down.
If a selected node exists once per instance, the bar shows an instance: picker. It starts on shared, the copy that belongs to no instance; you can pick all instances, or find or type one instance’s id. The pick applies to every action on the bar and is remembered until you close the editor.
Each trigger and infra node carries a small badge with its state, and hovering it says more:
| Badge | What it means |
|---|---|
on, off, turning on... | The trigger’s state |
held 3 | Off, with three arrivals or paused runs waiting for it |
behind | On, but firing an older version of the program, or infra running from older source. Resync or Upgrade brings it up to date |
held back | Taken down by the health check or with its infra, and it cannot come back by itself because other triggers are on another build. The hover names them and says weft resync --all |
running, stopped, starting..., stopping..., terminating..., flaky, failed | The infra node’s state |
3 of 5 on, 3 of 5 running | A node that exists once per instance. The hover lists each instance. An infra node no instance has a copy of yet reads no copies; such a trigger reads off |
If the health check took a trigger down, the hover says so. With the built-in health rules, the trigger comes back on its own once its infra is healthy. If the infra node’s own rules never bring it back, click Turn on. For what the health check watches, go and read when a unit stops being ready.
If a resync stopped partway with triggers still down, the status bar shows Resync stopped, with Resync to pick it up where it stopped and Give up to leave those triggers off.
Fold a big program down
Every group has Collapse group in its header, and Expand group when it is shut. The wiring outside does not change either way. Groups start shut.
If you are zoomed in when you open a group, the view zooms out around your mouse until about a dozen of its boxes fit on the screen. If closing one leaves five boxes or fewer on screen, the view zooms back in. Each zoom takes under half a second, and scrolling or dragging during it stops it where it is.
An included file’s box has an Open button that shows that file’s own graph, where you can edit it. There, the group holding the whole file starts open. Return at the top left takes you back.
To make a group, add a Group from the list (Ctrl+P) and drag boxes into it. weft decides where
a box landed from its centre, so drag it until its middle is inside. To undo
that, delete the group: every box inside climbs out one level and keeps its
position.
Moving a box in or out of a group changes your program, because it changes what
that box can reach. Dragging a box anywhere else does not: positions live in
layouts/, beside your source, not in it.
To show your program to somebody, there is a Simplified view switch at the top right. Every box becomes a plain square with one dot on each side. You can still move boxes and fold groups, and you cannot wire anything, which is the point.
Next, connect an account.
Connecting an account
A node that sends a Slack message needs a Slack account. You connect it once, weft keeps the credential, and your program passes around a reference to it rather than the secret itself.
Do it in the graph
A node that talks to an outside service has a blue Connect Slack… button in its body, named after whatever service it needs. Until you pick something, that node will not fold shut: the fold arrow is greyed out, and hovering it says Pick a connection first.
Click it and you get your existing connections for that service, then, under + Add a connection, the ways to make a new one:
- Use App name (one click), one row per application your runtime was configured with, each showing the permissions it asks for. Those permissions are fixed per application, so you pick an application rather than ticking boxes.
- Use ours (uses your credits), when the runtime holds a key for that
service. This is the row that appears because you filled in
access-apps.jsonbefore installing. - Your own…, always. Tick the permissions you want, follow the guide for registering an application with that provider, or just paste a key you already have.
Sign-in opens your browser for the provider’s consent screen and the panel waits, then flips to a green Connected as your name with Change and Disconnect beside it.
If you take the shared door, weft warns you once that somebody else in the same workspace using that same application may disconnect you, and that your own application avoids it.
Or from the terminal
weft connect
Run it in the project folder, pick the node, and answer the prompts. weft connect --list prints what is already stored and changes nothing. weft connect account goes straight to one node: any node that takes a connection
of its own, an access node or a route that checks its callers. --disconnect clears a
node’s pick and leaves the stored account alone, and --forget <id> deletes
the stored account for good.
Every project on the install can pick any stored connection, so if you keep
two keys for one service, their names are how you tell them apart: the list
shows each one’s account and its name. When you connect a new one your own
way, with your own key or your own application, weft connect asks for a name
(--label <name> answers without asking), and pressing Enter names it after
the service. The editor’s Name field does the same when you leave it empty.
If you sign in through a shared application, or through the one your project
declares, that application names the connection (a shared key is named after
the service), so nothing is asked and --label is refused. A name with the project in it, like
acme-scraper, suits a key only that project should use.
Picked on each install
Which connection a node uses is kept by the install you picked it on, never
written in your .weft files. A connection lives in one install, so its id
means nothing on another: your machine and your cloud each have their own
accounts, and each keeps its own pick. If you want a node connected on your
cloud install, run weft connect --on prod there (it is the same
walkthrough). A node inside a file you include twice is two nodes, one per
place, and each place gets its own pick.
If you run a program with a node nobody connected on this install, the run
is refused before it starts, naming the node and the weft connect that
fixes it. weft activate and weft resync check every node of the program
the same way before any trigger moves, so a missing pick is caught there and
not at the first call. Forgetting a stored connection takes every pick of it
with it.
A pick is kept under the node’s full address, and the file a node is written
in is part of that address. So if you move chat.weft into a studio/
folder, the node that was chat.send.agent is now studio.chat.send.agent,
and its pick stays behind under the old name. weft never guesses where a node
went. If a node in the program can hold every field the old one kept (the same
field, picked the same way, on the install or @instance_filled, and
connecting to the same service; extra fields are fine) and nothing is stored
for those fields there yet, activating refuses and names both addresses:
'chat.send.agent' has connections or values stored on this install and is no
longer in the program; did you move it to 'studio.chat.send.agent'?
`weft connect --move chat.send.agent studio.chat.send.agent` carries them across
weft connect --move <old> <new> carries the program’s pick and every
instance’s values for that node across in one go. It refuses if nothing is
stored at the old address, if the new address is not in the program this
install last built, if the new node already has something stored, or if it
cannot hold every field the old one kept.
If a build refuses a line like account: {"id": "...", "identity": "..."}
with “that was the old way of connecting a node”, the connection was written
into the source by an older weft. Erase that line and connect the node again;
the install keeps the pick from then on.
Never paste a key into a node’s field
Values written into a node travel through the journal and render in the inspector as plain text, where anybody who can open the project reads them. That is also true of anything you paste into a chat with your assistant.
A connection is the other thing: the secret goes into weft’s own store, and what moves along the wire is a small reference with no secret in it.
{
"__weft_access__": {
"accessId": "connection-id",
"service": "slack",
"identity": "Acme Corp",
"requiresPermissions": ["chat:write"]
}
}
When a node consumes it, the runtime adds what that node needs onto it. When the node opens it, a missing stored value is refused, and so is a missing permission on a verified connection, rather than failing halfway through a call.
What weft can tell you before a call fails
Providers differ in how honest they are. Some state exactly what they granted, and weft blocks a node whose permission is missing. Others only confirm the credential is alive, or say nothing at all until a real call comes back refused, and then weft lets the call through because it has nothing to block on.
So a bad credential sometimes shows up for the first time when a node tries to use it. For the whole picture, go and read how connections work.
Next, put a person in the loop.
A public address
If you want Slack, Stripe or Telegram to push events into a program running on your laptop, they need somewhere to push them. Your laptop is behind a router and has no address they can reach, so weft opens one for you:
./setup.sh --public-url
That opens a tunnel from a public address to a port of its own on your machine
(14112 on the default install), where weft serves a short list of paths. The
choice sticks: later runs of ./setup.sh keep the tunnel open until you pass
--no-public-url.
What it exposes, and what it does not
These paths, and nothing else. Everything else on that address answers 404.
| Path | What arrives there |
|---|---|
POST /events/<service>/<topic> | A provider pushing you an event. A GET there answers 404 |
POST /signal/<token> | Somebody answering a waiting node through its link |
DELETE /signal/<token> | A client cancelling the run waiting behind that link, and every other wait of that run with it. It needs a signal token minted without --tags: either a token for whole projects, or an instance token (weft token mint --instance), which can only cancel its own instance’s waits. On a trigger’s link, it drops that trigger’s signal instead |
POST /signal/<token>/skip | Somebody skipping that waiting node |
GET /signal-token/signals | A client holding a signal token, listing the nodes waiting on it |
DELETE /signal-token/signals | That client cancelling every run it can see waiting on anything (a form, a timer, a poll), and dropping every trigger signal it can see, listed or not: a webhook’s or a schedule’s too. It needs a token that covers whole projects (minted with weft token mint without --tags, not an instance token) |
GET /signal-token/signals/<signal>/files/<field> | That client asking for a fresh link to a file a listed form shows |
GET /signal-token/health | That client checking its token still works |
GET /signal-token/displays | That client listing what your nodes are showing (a QR code, say) |
GET /signal-token/displays/<project>/<node> | That client reading one node’s display |
POST /signal-token/displays/<project>/<node>/action | That client pressing a button on it |
/instance/... | Somebody holding an instance token for one of your programs: that instance’s fields, its connections, its runs (a failed run shows only Something went wrong; the reason stays on the run’s record, where weft logs <run id> shows it to you) |
GET, HEAD or PUT /files/runtime/... | Somebody opening a file link your program handed out, or a page sending a part of an upload. The link’s own signature is what lets it in |
POST /files/upload/<step> | A page uploading a file through an upload slot your program handed it |
/connect/<tenant>/<project id>/<path> | A caller of one of your routes or sockets |
/infra/<project>/<copy>/... | A caller of an infrastructure endpoint your node opened to the public |
GET /access/oauth/callback | A provider sending you back after you approve a connection |
/access/picker/<state> | The page where you pick files for a connection, and its answer |
POST /<tenant>/<project id>/<path> | A trigger of yours that fires on a plain POST to its own path |
/, /index.html, /logo.png | A small page saying this address is a weft install |
Every /signal-token/... request, and a DELETE /signal/<token>, carries the
signal token in its Authorization: Bearer header, so it stays out of proxy
logs. The <token> in a /signal/ path is a different one: the link’s own
token, which is made to be pasted.
Your dispatcher’s real API, your projects, your journal and your runs are not
on that port at all, so nothing arriving through the tunnel can reach them.
They stay on 127.0.0.1:14111 (the default install’s address; weft daemon which gives another’s), which listens on your machine only, asks
every caller for an operator key and sends no CORS headers. For
how that key works, go and read the key the install’s API asks
for.
Find your address
weft daemon status
weft: running at http://127.0.0.1:14111 (install 'default'); 1 project(s)
public address: https://something.example (./setup.sh --no-public-url closes it)
The installer prints it too, when it is open.
Hand a provider the full path for what it is sending, not the bare address:
https://something.example/events/slack/messages. Each trigger’s connect panel
shows you the exact one it wants.
It changes when the tunnel restarts
This is the thing that bites. The address is not yours to keep. When the tunnel comes back up with a different one, anything you registered against the old one is pointing at nothing, and events stop arriving without an error anywhere: the provider is posting into the void.
So after a restart, check weft daemon status and re-register the address with
any provider you configured by hand. A Slack app’s Request URL and a webhook
you pasted into somebody’s dashboard both need it.
A stable address of your own
If you want an address that never changes, run the tunnel under a hostname you own. You need a Cloudflare account with that domain on it.
-
In the Cloudflare dashboard, create a tunnel (Zero Trust, then Networks, then Tunnels) and copy its token.
-
Add a public hostname to that tunnel, say
weft.example.com, and set its service to the port weft serves those paths on: on Linux,http://127.0.0.1:14112, because the tunnel shares your machine’s network; on macOS,http://host.docker.internal:14112. 14112 is the default install’s port. A branch’s install has its own: theoutsidevalue inports.json, in the folderweft daemon whichprints. -
Install with both set:
WEFT_PUBLIC_TUNNEL_TOKEN=<token> WEFT_PUBLIC_TUNNEL_HOSTNAME=https://weft.example.com ./setup.sh --public-urlThe hostname is the bare
https://address, with no port and no path.
Once the tunnel is up, weft daemon start fetches the page at your address
through it. If the service in step 2 is wrong, the start fails and says so,
naming the value it expects.
The install remembers that tunnel, so a later weft daemon start or weft daemon restart from a shell that sets neither variable opens the same one
again. If you set both, they win, and the install remembers the new ones. To
go back to a quick tunnel, run ./setup.sh --no-public-url, then
./setup.sh --public-url with neither variable set.
Close it
./setup.sh --no-public-url
The tunnel comes down and the address stops answering.
Devices on your own network
If you want a phone or another computer on your wifi to open what your program hands out, without the internet in between, give weft this machine’s address on that network:
WEFT_NETWORK_ADDRESS=192.168.1.20 weft daemon start
network address: http://192.168.1.20:14112 (the doors outside callers use, for devices on this machine's network; a start with WEFT_NETWORK_ADDRESS=off closes it)
weft then answers the paths in the table above on that address too, on the
port the tunnel uses (14112 on the default install), and nothing else: your projects and runs stay on 127.0.0.1. weft
remembers the address for every later start. If you want to close it, start
once with WEFT_NETWORK_ADDRESS=off.
It takes the bare IP (ip -brief address lists yours), and weft adds the
scheme and the port itself. The start refuses a value with a scheme, a port or
a path. It also refuses a loopback address (no other device can reach
it), 0.0.0.0 or :: (that would open these paths on every network this
machine is on), a link-local IPv6 one starting fe80:, and one this machine
does not have. Each refusal names
what to give instead.
Your own site in front
If your project has its own site and you want links to its stored files to
go through that site, tell weft the address where the site passes its
visitors’ calls on to weft (the weftPassThrough helper from @weft/connect/server,
mounted at /weft on the site):
weft frontend visitor-address shop https://shop.example/weft
shop is the name you gave the frontend with weft frontend add. If you are
making the frontend now, weft frontend add shop --visitor-address https://shop.example/weft sets it in the same step. To clear it, give off in
place of the address. One
frontend per project can hold it: asking for a second is refused, naming the
frontend that has it.
Which address a file link gets
When your program hands somebody outside a link to a stored file (a file in a
route’s answer, say), the link goes through this tunnel only on a local install
whose project has no visitor address. With the tunnel closed too, it goes to
the install’s own address: the network address if you set one, which only
devices on your network can open, otherwise 127.0.0.1, which only this
machine can open. For the full order and what a node sees, go and read
handing a file out.
If your tunnel is the one whose address changes on restart (./setup.sh --public-url without a hostname of your own), weft asks it for its address each time it makes a
link, so a new link always carries the current address. A link handed out
before the tunnel restarted still points at the old one. If the tunnel cannot say its address at that
moment, the link is refused:
the install's tunnel does not say its address right now; `weft daemon start` opens it again
When you do not need it
Triggers that go and fetch rather than wait to be pushed work with no public address at all: a timer, a poller, anything weft opens an outgoing connection for. So do runs you start yourself, and so does a person answering through the browser extension, because that talks to your runtime directly.
You need this page when a provider insists on pushing. For which triggers those are and how a service declares its event transport, go and read events from a service.
Putting a person in the loop
A node can stop and ask somebody a question. The run parks, the worker shuts down, and when the answer arrives a fresh worker picks the run back up exactly where it was. Waiting costs nothing, so a program can sit on a question for a week, as long as the run can pause (for when it cannot, go and read when a run cannot pause).
This is HumanQuery, and it looks like this:
review = HumanQuery {
title: "Send this answer?"
fields: [
{ "kind": "display", "key": "question" },
{ "kind": "display", "key": "answer" },
{ "kind": "approve_reject", "key": "send" }
]
question: ask.content
answer: draft.answer
}
The fields list builds both the form and the node’s ports. A display field
takes a value in and shows it to the reader. An approve_reject field gives
you two boolean outputs, sendApproved and sendRejected, so you gate what
happens next on the one you want:
# ask is the BaileyReceive trigger the message came in on, whatsapp its BaileyBridge
reply = BaileySend {
_should_flow: review.sendApproved
bridge: whatsapp.bridge
to: ask.chatId
message: draft.answer
}
Where the question shows up
Not in VS Code. The graph shows you that something is waiting, with a cyan ring
round the box and the word Suspended in its body, and the inspector says
Waiting for input.... Answering happens in the Weft tasks browser
extension you installed earlier.
Give the extension a way in by minting a token from your project folder:
weft token mint --name "my laptop"
A signal token lets an outside client list a project’s waiting nodes and answer
them. The full value is printed once, at mint, and never again, so copy it now.
--projects and --tags narrow what it can reach, and weft token ls and
weft token revoke <id> handle the rest.
Open the extension, go to its settings, and add your runtime: its address and that token. The browser will ask you to allow access to that host, because the extension holds no permission to reach anything until you give it one.
What the person sees
One card at a time. Your title, the values you wired into the display fields,
and the buttons or boxes your other fields asked for. The primary button says
Submit for a question inside a running program, and Fire for a trigger
that starts one.
Beside it, Skip declines the question: the node that asked ends skipped, every output of it closes, and the nodes reading them skip in turn while the rest of the run carries on. Cancel run ends the whole run. Neither shows up on a trigger, which has no question to decline: weft refuses a skip there.
After a submit the card says Submitted and moves to the next one. Back in the
graph, the cyan ring goes green and the rest of your program runs.
Starting a run from a person
HumanTrigger is the same form pointed the other way: instead of parking a run
that already exists, it starts a new one when somebody fills it in. It shows up
in the extension under trigger tasks rather than resume tasks, and it only
appears once you have run weft activate, because a trigger is only listening
after you arm it.
For why activating is a separate step from running, go and read the three lifecycles.
When nobody answers
A parked run waits, with no deadline. HumanQuery has no timeout setting and
weft will not invent one, because a question that expires on its own leaves you
with a run that ended for a reason nobody wrote down.
So ending it is a decision somebody makes. The person looking at the card can
press Cancel run, and you can run weft stop <run-id>. To find the ones
still sitting there:
weft runs --status waiting_for_input
That lists only the runs parked on a wait. weft runs --status running lists
them too, beside the ones still working, and the status column tells them
apart.
How a program runs
Do not read a weft program top to bottom. A node can send its answer before it has finished, or sit waiting for a week while the rest of the program carries on. If you want to predict what happens, look at what arrives at each node’s inputs.
Values arrive one at a time
When a node emits, the value travels along every arrow leaving that output. Each delivery is a pulse, aimed at one input, carrying a tag saying which run it belongs to. One output wired to three inputs is three deliveries.
A node waits until every input with an arrow into it has either a value or a closure. Optional inputs count too: an arrow is an arrow. An input can also be satisfied by a value written into the node or by a default, so not everything needs an arrow.
Once every wired input has settled, weft decides whether the node runs or is skipped. A node with nothing wired into it has nothing to wait for, so weft starts it when the run begins.
How a branch stops the nodes after it
Here is the problem. A node decided not to send anything. Everything wired to it is now waiting for a value that is never coming. How does the program know the difference between “not yet” and “never”?
A closure is the answer: it is a pulse that says nothing will ever arrive here, for this run. It is how a node says no rather than saying nothing.
It is not null. null is a value, and a port whose type allows it takes it
like any other.
Here is what a closure does to the node it reaches:
| What happened | What the node does |
|---|---|
| An input it needs closed | Skips |
An input you declared x?: T closed | Nothing. It runs without that input |
Every @require_one_of input closed | Skips |
It takes the first of several branches, like FirstInOrder, and every one of them closed | Skips |
_should_flow is false, or closed | Skips |
| It was waiting on a person, who skipped the question | Skips |
By default a node needs every input that is wired, and writing ? on one lets
it run without it. For which inputs a node needs, with an example of each, go
and read optional and required inputs.
If every input wired into a node is written with ?, nothing says when it
should run, and the compiler stops with run-undecided: gate it with
_should_flow, drop the ? from the input it cannot do without, or add
@require_one_of. A node that joins branches, like FirstInOrder, needs none
of these. Neither does a node whose input comes from a port that always
carries something when its group or loop runs, such as a port of a group whose
gate is wired, a group port with a value written on it (g.x = "hi"), the
item a loop iterates over (a port named in its over), or self.index: the
way its value arrives already says when it runs.
A skipped node closes its own outputs, which can skip the next node, and the next, until the closures reach something that can carry on with what it has.
flowchart LR
choice[Switch] -->|true| send[Send]
choice -. closed .-> review[Review skipped]
review -. closed .-> archive[Archive skipped]
Streams close differently. A closure on a stream is the stream’s end rather than a skip, so a reader whose stream carried nothing still runs and sees that nothing came.
A failed node closes whatever it had not already sent. Anything it did send is not taken back. For how a program carries on past a failure, go and read when a node fails.
Choosing a branch, and joining back up
Every node has _should_flow, and it defaults to true. Wire a value into it to
decide whether that node runs.
Only a literal false says no. Any other value lets the node run, and so does
anything the node’s own code would have considered falsy. The node never sees
this port: weft checks it before calling in.
Switch gives you one output per case. It tries them in the order you wrote
them, and the first one that matches emits true while every other one closes.
Wire a case’s output into a node’s _should_flow and that node runs on that
case only. If nothing matches and you gave no otherwise, they all close.
To bring two alternative answers back into one arrow, use FirstInOrder. It
waits for its wired inputs to settle, then takes the first one that actually
supplied a value, in written order. Moving those lines changes which answer
wins. Which one arrived first makes no difference.
_should_flow takes one wire, so “only if the model said it was rude and
said it was sure” is one gate fed by two conditions. All is the AND: wire
both conditions into it and gate on what it emits. It emits when every input
arrived and none of them is false, and a branch that closed never arrives at
all, so a closed input is not a false.
Running on the thing that did not happen
Everything above skips when its inputs close. So how do you run a node because something did not arrive?
Wire that something into _should_not_flow instead. Every answer flips: a
value arriving means the node stays off, and a closure, the structural “nothing
is coming”, is what runs it.
route = Route -> (body: { title: String, photo?: File }) { path: "cards", method: "POST" }
# A card sent with no picture: the wire reading `photo` closes, so this runs.
defaultArt = FetchToStorage {
url: "https://example.com/blank.png"
_should_not_flow: route.body.photo
}
This is the one port in weft that starts a node on a closure. Reach for it when the absence is data, like a key the caller did not send. When the absence is a decision your own node made, have that node say so on a second output and gate on that, because the wire then reads forwards.
A failure is not an absence. When the node it watches fails, its ports close
too, but that closure carries which node broke and its error, and the gate
reads it: the node stays off, skipped with the reason what its _should_not_flow watches did not finish ('query' failed: the database is down), and the run reports the failure. So a
“nothing there” branch never runs over a database that is down. The same holds
through a group or a loop: a failure inside closes the scope’s outputs with the
failure on them, so a watcher outside reads it as one (and a loop whose
carried value is lost to a failure stops and fails, see
loops). And it holds through a
skip: a node that skipped because its input closed on a failure did not decline
either, so its own ports close with that same failure on them (its skip reason
ends in because 'query' failed: ...), and a watcher two nodes down still
reads a failure, not an absence, and still sees query named as the node that
broke rather than the node that skipped. The one exception is a node whose
_error is wired: it handled its failure, so it counts as finished and its
ports close as plain absences (below).
A node has one gate. Wiring both spellings is the two-gates error.
When a node fails
Left alone, a node that fails fails the whole run. If you want the program to
carry on instead, wire the node’s _error output. Every node has one, given by
weft. When it is wired and the node fails, the reason comes out on _error as
text, the node counts as finished, and every output it had not sent closes.
So to run a backup only when something broke, gate the backup on _error:
fetch = HttpRequest { url: "https://example.com/prices", method: "GET" }
backup = HttpRequest { url: "https://mirror.example.com/prices", method: "GET" }
backup._should_flow = fetch._error
When fetch works, its _error closes and backup skips. When the host does
not answer, the reason arrives on _error and backup runs. The failure still
shows in the node’s log, as failed, handed to the '_error' output: ..., so
weft logs and the inspector tell you it happened.
Only a failure of the node’s own work goes to _error: a service refusing or
not answering, a script raising, a wait that gave up, a node whose worker died
in the middle of a durable run, or a value that does not fit the type you
declared on its output (a model’s answer missing a field of the record you
declared on LlmInferenceJson, or a value a Cast cannot convert). A mistake
in the program, like a bad setting or an input the node cannot read as the
type that input declares, fails the run even with _error wired, because only
changing the program fixes it. Scripts are the exception to the output case:
a script’s code is part of your program, so a script returning the wrong type
for an output is a mistake in the program and fails the run. For what a script may return, go
and read declaring ports, and for which nodes
fail and which run again when a worker dies, how a run is
kept. A cancel, or a node pausing
to wait for an answer, is not a failure, so it never comes out on _error
either.
Once a node has started a stream, its failure fails the run. As soon as
anything went out on a Bus or Generator output, the readers already hold
part of the answer, and a backup gated on _error would send a second answer
on top of it.
The stream ends failed instead, and its readers are told (go and read how a
bus ends). A producer can open its bus
on the first piece, the way LlmStream does. So if a model refuses a prompt
before writing a word, the refusal still comes out on _error. The node’s bus
port then closes like any output it never sent: a reader that needs the bus
skips, and the backup can answer.
A group, a loop or an included file has no _error of its own. If you want to
handle a failure from inside one, take the failing node’s _error out through
one of the group’s outputs, like any other value.
_error belongs to weft. A program that declares an output called _error on
a node is refused, while error, without the underscore, is a name like any
other.
One node, several firings at once
A firing is one go at one node. Three things say which firing you are looking at: the run, the node, and the loop iterations it sits inside. Inside a parallel loop the same node can have several firings alive at the same time.
Those iteration numbers are called frames. Work at the top level has none, and work inside two nested loops carries one frame for each. weft only ever combines inputs whose run and frames match, which is what stops iteration three eating iteration four’s answer.
An ordinary output emits at most once per firing, though it can emit on one output now and another later. Whatever it never mentions is closed when it finishes. Generators and buses have their own rules, in streams and buses.
Groups and loops at run time
When the program runs, no node stands for a group, and a loop becomes two
nodes, a LoopIn and a LoopOut. For the details, go and read no node
stands for a group
and loops.
What happens when you hit run
A manual run starts every node at the top level that has nothing wired into it, plus every trigger in the project, wired or not. Triggers get no event on a run like that, so they close their outputs, and only the paths that do not need an event go anywhere.
To run part of a project:
weft run --target dailyReport --target alert
weft takes those nodes and everything feeding into them, stopping the walk when
it reaches a trigger. A node that several branches share does not drag those
other branches in. A target inside a group brings only the work needed through
it, and the group’s _should_flow comes along too, unless it is wired straight
from a trigger the run does not fire. A loop the walk reaches from outside
comes along whole: a target after a loop runs every iteration of it. If you
want to run part of a loop’s body, start and end the run inside the body
(see running part of a loop’s body).
What runs is a cut: a smaller program holding only what the walk kept, with every node under its own name, so the journal and the inspector line up with your source.
You can also start in the middle, handing values in where the walk begins:
weft run --from classify='{"text":"..."}' --target reply
weft run --group triage='{"text":"..."}'
A value you hand in is that port’s value: in this run the wire into that port delivers nothing there, so nothing upstream can replace it. For which flag picks which part of the graph, go and read versions, seeds and frozen examples.
--group runs the group’s own nodes and nothing that feeds it, so a group
port wired from outside gets only what you hand it. If a cut leaves an input
with nothing, weft looks at who needs it. When a node would skip without it
(it is an input written without ?, default or not, other than an input of
a join node such as FirstInOrder, which is optional anyway; or the only member
of a @require_one_of set that could get a value; the node’s own, or one inside the group it enters, followed
through every group, loop and included file on the way), the run is refused
before it starts, naming the input and what it feeds:
triage.text gets nothing in this run: ask.text is outside it, and
triage.classify.text cannot run without it. Hand it a value at this start
(--group triage='{"text": ...}'), run what feeds it with --feed triage, or
start further up so ask runs too.
The flag in the message is the one the start was named with: --group for a
--group start. An input no start lies in front of is named where the wire
lands, and the way to hand it a value is to start there. When the value could
only come from a trigger the run does not fire, the message says so and
offers --fire on that trigger as well as handing the value in.
The input is named at the start you gave, even when the missing value would
pass through more group ports on the way (a group inside an included file), and a value you
hand there feeds everything behind it. A trigger’s own inputs never count: a
run that does not fire it never reads them. An input that only feeds inputs written with ?
just closes, with a warning.
A start is where the walk upstream stops, for --from and --group alike, so
a start’s own ports get only what you hand them. To have weft run what feeds
them instead, add --feed:
weft run --from 'hear.note={}' --feed hear.note --target hear.note
For each input of hear.note you did not hand a value, that runs the node
feeding it and nothing above that node. The value is followed through every
group port it passes, however deeply the groups nest, and a feeder shared by several
inputs runs once. A loop’s result comes from the whole loop, so a loop feeding
the start runs whole. You can also
name the feeders as starts yourself (--from db --from accounts --from hear.note): when one start lies upstream of another, the run keeps both, and
--target still leaves out every branch that target does not need.
A gate only holds a start back when something in the run decides it. The walk
upstream stops at a start, so the start’s own _should_flow, and that of each
group it sits inside, stays out of the cut unless another start, or the
trigger you fire, runs whatever feeds it. A group’s gate wired straight from a trigger the run
does not fire stays out too, for a start or a target. A wired gate left out
acts as if nothing were wired to it, so the node or group runs (a gate written
in the braces still applies): that is how a node inside a group gated by a
Route runs when you start it by hand. If you want the condition checked,
start further up instead, so whatever decides it runs too, or drop the start
and --fire the trigger with an event.
Without --target, a run goes everywhere downstream of its starts, so
--from db alone runs every branch db feeds. --target narrows that to
what the target needs, starting no further up than the starts. It is one
shape, not a run that goes until it reaches the target.
If you want everything a node needs but not the node itself (say a node
called post that publishes the result), end with --before post instead of
--target.
If you want to get
past a node without running it (a trigger you have not activated, an infra
node, a call you do not want to pay for twice), stand in for it with
--emit: --emit fetch='{"body": "..."}' says what fetch would have sent,
and the run starts from there, the way it starts at a --from.
Any of these flags can name a node or a group inside a loop’s body, as long as the run does not cut a loop in half: it holds both ends of a loop (the loop runs whole, every iteration) or neither (only the part of its body you picked runs, once). For what that part gets and what is refused, go and read running part of a loop’s body.
On a --seed run the check waits for the seed, since a saved result can
supply the input. It only can when the run it reads completed that node and
the node’s code has not changed since; a refused seeded run names the run it
read.
A trigger firing picks the work downstream of that trigger, plus whatever that work needs upstream, again stopping at other triggers. Only the trigger that fired gets the event; any others close. Two triggers can share the nodes in the middle without becoming one run.
In a program with instances, a run is also for one instance or for none
(weft run --instance user-42 picks one). For what is checked before such a
run starts, go and read
programs with instances.
How a run ends
A run is finished when there is no work left and nothing still in flight. It
does not need an end node. It can also fail, when a node failed and nothing was
wired to its _error, or when weft could get no further. A person or another
run can also cancel it.
A run can pause without ending. Waiting for input means a node is parked on an answer from outside, and the run carries on when that answer comes.
If work is left that can never proceed, weft reports the run as stuck and names the nodes and inputs involved. A run parked on a person’s answer is waiting, not stuck.
What survives a restart
As a run goes, the worker writes down each thing that happened, in a log called the journal. Those records are what the graph shows you.
If its worker dies, what happens to a run depends on its trigger’s durable
setting: for that, go and read how a run is
kept. Before you put side effects
around a wait, go and read surviving a restart.
What the build catches
The compiler checks that every arrow’s types fit, that every required input is covered, that no wire crosses a scope boundary it should not, that there are no cycles, and that the loop and channel restrictions hold. It also runs whatever validation rules a node declares for itself.
Some things are checked when you build and some only when you run, so a build can succeed and a value can still be rejected later, when a node emits something its declared output type does not allow.
It cannot check whether a node does useful work. A model can return a perfectly typed answer that is wrong. For every rejection by name, go and read what the compiler refuses. To start writing weft yourself, carry on to syntax.
Syntax
A .weft file names some nodes and connects them up. This program puts HELLO
in front of you:
message = Text { value: "hello" }
shout = ExecPython(text: String) -> (text: String) {
text: message.value
code: "return {'text': text.upper()}"
}
show = Debug { data: shout.text }
message, shout and show are names you choose, unique where they sit.
Text, ExecPython and Debug are node types, from the project’s nodes/
folder.
Naming a node
name = NodeType, and braces if you have anything to put inside:
message = Text { value: "hello" }
A node with nothing to configure can drop the braces, so Debug and Debug {}
mean the same. You still have to supply whatever that node requires.
A name the catalog does not know is unknown-type. Writing a name does not
bring a node into existence; the declaration has to be there.
Connecting things
Write source.output where a value would go:
show = Debug { data: message.value }
Or put it on its own line, which is the same connection:
show.data = message.value
Values travel right to left, into the input on the left, and the type has to
fit. Write the wire in the braces whenever you can. The line of its own is for
the ports a group, a loop or an @include declares, which you set from
outside because their braces hold something else or there are none.
To take one field out of a value, keep going with dots:
show = Debug { data: reader.profile.stats.wpm }
That reads stats.wpm off reader.profile. For the type rules, go and read
types.
Writing values in
Inputs take strings, numbers, true, false, null, lists and objects, as
long as the type allows it. Lists and objects can spread over several lines,
and commas between fields are optional.
message = Text {
value: "hello"
}
Some inputs only take arrows and some only take written values; the
input-accepts diagnostic tells you which. Anything the compiler has to read
to work out the node’s shape, such as a form’s field list, has to be written
inline: an arrow, @file or @asset cannot supply it.
For text kept in another file, go and read files and reuse.
Triple backticks hold code, or anything with newlines in it:
answer = ExecPython -> (value: Number) {
code: ```
return {'value': 42}
```
}
You can write a value into an input. Outputs are filled by the node
when it runs, so answer.value = 42 in that example is the value-on-output
error. The same holds inside a group: connect a child’s output to
self.result, and if you want a constant to come out, use something like
Text to produce it.
Declaring ports
Inputs in brackets, outputs after ->:
calc = ExecPython(a: Number, b: Number) -> (sum: Number) {
a: 7
b: 2
code: "return {'sum': a + b}"
}
ExecPython hands your code a and b and expects a dictionary back, keyed
by output name. A key left out sends nothing on that output, and return {}
sends nothing at all. A key set to None sends a real null on an output whose
type takes Null (last: JsonDict | Null), and nothing on any other output.
If you want the program to carry on when the script raises (a request it
makes fails, or it raises on purpose), wire the node’s _error output: the
exception’s type, message and traceback come out there, and the other outputs
send nothing. Left unwired, the exception fails the run. A mistake in the
program fails the run either way: code that does not compile, a script that
ends without returning a dictionary, a key that is not one of its outputs, or a
value of the wrong type for its output. _error counts as a key it may not
return, since weft fills it; to fail on purpose, raise. _error is not only
for scripts: every node has one. For which failures reach it, go and read
when a node fails.
You can declare any of the node’s own inputs, with its type or a narrower
one, and add new inputs on a node that takes them. An output you can declare only where the node leaves it open: one it lets
you add, one typed with a type variable (a stand-in like T that takes its
type where the node is used), or one it lets you narrow. Declaring an output
looks like this:
answer = LlmInferenceJson -> (response: { label: String })
convert = Cast -> (value: Boolean)
Those are fragments: the nodes still need their other required inputs before
they can run. For narrowing an output like response, go and read saying
what a broad output holds.
A group or a loop can also list a metadata port among its outputs, with a *
before the name: -> (results: String, *tool: LlmTool). Its value is written
in the program instead of made by a run. For what that is for, go and read
metadata ports.
Optional and required inputs
A node’s metadata says which of its inputs it can run without. Where you use
the node, you can declare the input yourself: without ? it is required, and
with ? the node runs without it. If the metadata requires that input, the ?
is a compile error.
| What the node’s metadata says | What the program writes | Result |
|---|---|---|
| required | nothing | always required |
| required | x?: T | compile error naming the port: the node’s code needs it |
| optional | nothing | unwired: runs without it. Wired: waits for it, and skips if the wire closes |
| optional | x: T (no ?) | always required |
| optional | x?: T | runs without it even when wired and the wire closed, whether nothing arrived or the source failed |
A wire closes when its source finished without sending on that port, or
failed. A node that skips closes its own outputs in turn. A ? lets a node run
without a failed source’s value, but the failure still fails the run unless
that source’s _error is wired (go and read when a node
fails).
One exception to the table: an input in one of the node’s oneOfRequired
sets (or in a @require_one_of you wrote on it) is decided by its set. The
node skips only when every member of the set closed, unless you declared that
input without ?.
If you want the build to refuse a node until something fills an input,
declare that input without ?. SlackSendMessage takes text or blocks, so neither is
required on its own, but this message must have text:
send = SlackSendMessage(text: String) {
account: slack.access
channel: "#support"
text: draft.answer
}
Leave text out and weft validate refuses send: required input 'send.text' has no driver (no edge, no body value, no default).
If you want a node to run when a wire into it brings nothing, write the ?.
Here ask is a SlackReceiveMessage, whose thread sends nothing when the
message was not in a thread. With threadTs?, such a message gets its answer
at the top of the channel. Without the ?, answer waits for threadTs and
skips:
answer = SlackSendMessage(threadTs?: String) {
account: slack.access
channel: "#support"
text: draft.answer
threadTs: ask.thread
}
Extra inputs
Some nodes, ExecPython and FirstInOrder among them, let you invent input
names. Writing a key in the body creates that input and works out its type from
what you put there:
summary = ExecPython -> (out: String) {
text: draft.answer
limit: 3
notes?: review.notes
code: @file("assets/scripts/summary.py")
}
limit becomes a Number. text takes whatever type draft.answer has. A
bare null gives it nothing to work from, so that fails.
The ? on notes? works like x?: T in the table above: the node runs
without notes even when its wire closes. Without the ?, a port you create
this way is required. That spelling only works when you are creating the port;
for a port the node already has, write the ? in the signature, like
(threadTs?: String).
Extra inputs stay in the order you wrote them, which matters for
FirstInOrder, since it takes the first one that supplied a value.
The four underscore keys
| Key | What it does |
|---|---|
_label: "Review the draft" | Puts a label on the box. A string literal only; it cannot come over a wire |
_tags: ["support"] | Tags the node, including for deciding which runs a stop reaches |
_should_flow: decision.approved | Decides whether the node runs. Takes true, false, or an arrow |
_should_not_flow: decision.sent | The same decision inverted: the node runs when the thing wired here did NOT happen |
Any other key starting with _ is an error.
send = SlackSendMessage {
_should_flow: review.approved
channel: "#support"
text: draft.answer
}
A node has one gate, and wiring both spellings is the two-gates error. For
what each one does to the nodes after it, go and read
how a program runs.
Groups and loops
A group’s braces hold its children, and its signature says what the outside can see:
shout = Group(text: String) -> (result: String) {
# Uppercase the message
convert = ExecPython(text: String) -> (text: String) {
text: self.text
code: "return {'text': text.upper()}"
}
self.result = convert.text
}
shout.text = "hello"
Set a group’s inputs from outside, as shout.text does. self means this
group’s own interface. _should_flow can go inside the braces too, and when it
skips a group it skips everything in it. The only literal you can write on
self is a metadata port’s value: self.tool = { "name": "search" }.
Loops work the same way with iteration settings of their own. Go and read groups and loops.
Declaring a node where you use it
You can write a node straight into the place its value is wanted, ending with
.port to say which output you mean:
show = Debug {
data: Text { value: "hello" }.value
}
That makes the node and the arrow together, exactly as a named declaration
would. The .port on the end is required even when there is only one output.
Needing one of several inputs
@require_one_of says a node needs at least one out of a list:
lookup = SlackFindUser {
@require_one_of(email, id)
account: slack.access
email: "person@example.com"
}
Every name in there has to be a real input. Supply none of them and the
compiler says require-one-of-unmet. At run time, the node skips if they all
close without a value.
It goes in a node’s body or its inline signature. A node’s author can bake the
same requirement into its metadata as oneOfRequired. Groups and loops reject
it: put it on the child that needs the value.
One copy per instance, one value per instance
@per_instance on its own line in an infra node’s body gives each instance of
the program its own copy of that node (its own container):
bridge = BaileyBridge {
@per_instance
}
It takes no arguments.
@instance_filled goes where a field’s value would, and makes the value each
instance’s own: @instance_filled alone, or @instance_filled(<value>) with
the value an instance that was given none gets. A connection is a value too:
google = GoogleAccess { account: @instance_filled }
digest = Cron { cron: @instance_filled("0 0 8 * * *") }
The fallback can also be a file, read the way a written one is:
personality: @instance_filled(@file("prompts/default.md")).
For what both do, go and read programs with instances.
Spelling a name
Every name you choose in a program is camelCase: what you call a node, a
group, a loop or an @include (sendReply), and every port you declare on a
node, a group or a loop, or in an included file’s header
(maxRetries: Number). A key you write in a node’s body that creates a port
counts too (userId: 3 on ExecPython). A node type is spelled exactly as
the catalog spells it, which is PascalCase (ExecPython).
If you spell a name you chose any other way, the program still runs, and the
compiler warns not-camel-case. When it can respell the name, the message says how:
node 'send_reply' is not camelCase: write 'sendReply'
The gates, _should_flow and _should_not_flow, are the language’s own names,
so this warning skips them.
If the ports you add are named after data from outside weft, you can keep that
data’s own spelling on a node built for it: body = JsonObject { user_id: ask.id } draws no not-camel-case warning for user_id, though the name you
give the node, body, is still checked. A few nodes that ship with weft let you do this (Route for its path’s captures, JsonObject for the keys
of the object it builds, among others), because each one sets
addedPortsAreDataKeys in its metadata.json. For the full list, go and read
the addedPortsAreDataKeys row in the features table.
Comments
# starts a comment. A plain comment as the first thing inside a group or loop
body becomes its description, which is what the graph shows when it is folded
shut, as in the shout example above. A comment anywhere else is just a
comment.
The parser keeps your whitespace and comments, so the editor can change one field without reformatting everything around it. There is no enforced formatting style.
Types
Every port has a type, and the compiler checks every arrow before anything runs. Same colour on two dots is the quick version of what this page explains.
What you can write
String
Number
Boolean
Null
List[Number]
Dict[String, Number]
String | Number
{ role: String, name?: String }
{ role: String, .. }
Generator[String]
| Type | What it holds |
|---|---|
String, Number, Boolean | The obvious |
Null | Ordinary data. A port whose type allows it takes it like any value |
Image, Video, Audio, Blob | A stored file. Blob is anything that is not the other three: a PDF, a zip |
Media | Short for Image | Video | Audio |
File | Short for those three plus Blob |
List[T] | Any number of T |
Dict[String, V] | Text keys, and values all of type V. Dict[Number, String] is refused, and the message suggests List[{key: Number, value: String}] |
JsonDict | An object whose values nobody checked |
Json | Any JSON value: text, a number, a boolean, null, or a list or object of those, at any depth. For a value whose shape you cannot know, like what a device answers to an arbitrary command. A stored file, a stream or a connection is never one |
{ a: String, b?: Number } | A record, with ? marking a field that may be missing |
{ a: String, .. } | An open record: a record that also takes keys it does not name, and drops them |
A | B | Either one |
Access[service] | A connection to one service, named the way its access node names it (Access[slack], Access[google_drive]). A bare Access is a connection to any service. Go and read how connections work |
Generator[T] | A stream of T. Go and read streams and buses |
Bus | A live channel. Same page |
Secret[T] | A T the run’s record never holds. See secrets below |
Empty | Nothing can be one. See below |
T, or T and a number (T1, T2) | A type variable. The node works with whatever you connect |
A stored file on a wire is never the bytes. It is a small marker saying where
the file is, which is why a String holding a URL is not an Image.
What fits into what
| From | Into | Fits? |
|---|---|---|
String | String | Yes, and the same for every other primitive |
List[A] | List[B] | If A fits B |
List[T] | T | No. To go through a list one at a time, put the consumer in a Loop |
Record | JsonDict | Yes. A record forgets itself |
JsonDict | Record | No. Use a Cast, which checks the shape |
Dict[String, V] | JsonDict | Yes, and back |
Anything made of JSON (text, numbers, booleans, null, JsonDict, and lists, dicts and records of those) | Json | Yes, at any depth: List[List[Number]] fits |
A file, Bus, Access, Infra, Generator[T], or a type holding one | Json | No |
Json | anything but Json | No. Use a Cast, which checks the value |
Record | Record | Every field you have must be declared on the target (unless the target ends with ..), every required field it wants must be required on yours, and the types must match |
Access[slack] | Access[slack] or Access | Yes |
Access | Access[slack] | No. Type the port with its service |
A | A | B | Yes |
A | B | A | Only if both A and B fit A |
Generator[A] | Generator[B] | Only if they fit both ways. A stream is invariant |
Secret[A] | Secret[B] | If A fits B |
Secret[T] | T, Json, or a JsonDict | No. Wire it to a node whose port takes a Secret; that node reveals it |
A record or a Dict holding a Secret | JsonDict | No. A JsonDict would carry the secret on to a port that reads it plain |
T | Secret[T] | No. Only a node makes a secret |
Empty | anything | Yes |
| anything | Empty | No |
Records are strict. A field the target does not declare is a mismatch, not
something quietly carried along. If you want a record that takes keys it does
not name, end it with ..: { email: String, .. } takes any record with an
email. At run time, a value checked against it (a request’s body, a model’s answer,
what a Cast makes) loses every key the type does not name, so what flows on
carries only email. A missing required field is refused either way.
A field that can be null says so: extra: JsonDict | Null. A JsonDict alone
refuses a null, so if you declare a query’s rows as
List[{ id: String, extra: JsonDict }] and a row comes back with extra
null, the node fails with ... does not accept: [0].extra: expected JsonDict, got Null, and the fix is the | Null on that field.
Empty is the type of a value that cannot exist. An empty list [] is
List[Empty], because there is no item to learn from, and that goes into a
List[String] or a List[Number] without complaint. Inside a union it
disappears: Number | Empty is just Number.
A port typed with a type variable (T, T2) gets its type where the node is
used: from the type your program declares on it, from what is wired to it, or
from a value written into it. Cast’s output is the type variable T2 (its
input is T), so it takes the type you declare on it (-> (value: Boolean)),
or else the type of the port it feeds. A port you declare in a header with no
type, like (x), gets a type variable of its own, so for a group
g = Group(x) -> (y: String), writing g.x = 3 makes x a Number. A type variable nothing pins is the unresolved-typevar error.
A type variable can be limited to some types. A group says so right after
the word Group, blur = Group[T: Media](file: T) -> (out: T), and so can a
node you declare inline (ExecPython[T: Media](f: T)). A node’s author does
it in its metadata.json (KeepFile only takes a file); for how, go and read
typeParams. Wire a String into that
group’s file and the build stops with type-bound: `T` has to fit in Media, and what is connected makes it String.
Saying what a broad output holds
Some outputs carry a shape only your program knows, like the JSON a model
answers with or the body of a request. Their node types them broadly, usually
JsonDict, and lets you declare what they hold on the node, like this:
type Verdict = { label: String, score: Number }
judge = LlmInferenceJson -> (response: Verdict) {
prompt: "Label this review and score it from 0 to 1"
provider: model.provider
}
From there on the wire carries a Verdict, so judge.response.score reads
as a Number. At run time every value is checked against Verdict, and one
that does not fit fails the node, which _error catches when it is wired.
You can only do this on an output whose node allows it, and only with a type
that fits inside the widest one the node accepts there. Declaring
-> (response: Number) on LlmInference, whose answer is always text, is
refused when you build: ... output port 'response' is declared Number, but LlmInference always gives it String: only an output its node marks narrowable takes the type the program declares.
Giving a type a name
type CustomerId = String
type Ticket = { id: CustomerId, body: String, priority?: Number }
A declaration goes at the top of a scope, meaning the file, or a group’s or a
loop’s body. Not inside a node’s braces. Order does not matter. A name is
visible in its scope and every scope inside it, included files too: for how a
type reaches into an @include, go and read
files and reuse.
Names are compared as names. CustomerId and String are different types
even though one is the other underneath. That is the point: a customer id
cannot be wired into a port that wanted any old string by accident.
A name flows into its underlying shape for free, because the name just
falls away. Going the other direction needs a Cast, which checks the value
really has that shape.
Nothing shadows. A name already visible cannot be redeclared, even to the
same thing. Two catalog packages shipping an identical type is fine (for two that
disagree, go and read custom types). And a port that restates a declared name with a different body is the
named-type-conflict error, because a named type has one body everywhere.
Reading a key off a wire
speed.wpm = reader.profile.stats.wpm
The dotted path goes on the source side only. A wire lands on a port, never on a key inside one.
Each step of the path has to name a field of a record. A JsonDict has no
declared fields, so reading a key off one is the deref-path error telling you
to declare the type on the source port, or Cast it first.
The interesting case is an optional field. If you read ?-marked key and it is
missing, or it is null, the wire closes. It does not deliver null. So
the node you wired it into applies its own rules: it skips, unless you declared
that input with ? (x?: T), and then it runs without it. For what a closure
does, go and read
how a program runs.
A required key that is missing is an error naming the key, never a quiet null.
Cast
Cast is the node for converting a value and checking the result. You declare
the type you want on its output, and the compiler tells you whether that
conversion exists at all.
| From | Into | What happens |
|---|---|---|
String | Number, Boolean | Parsed |
String | List, Dict, JsonDict, a record, a named type | Parsed as JSON, then held to the target’s shape |
| Anything not a file, a live handle or a secret | String | Turned into text: a list or an object becomes its JSON |
Json | Anything made of JSON | Checked at run time: a Json holding a list casts into List[Number] when every item is a number |
Number | Boolean | 1 and 0 |
| Any object shape | A record or a named type | Checked against the declared structure at run time |
| Any object shape | A file type | The marker is checked, including that its kind matches |
List[A] | List[B] | If A casts to B. Same for Dict values |
If an ordinary arrow is refused and a cast would work, the type-mismatch
message says so and tells you to wire it through a Cast.
At run time, a value that does not convert fails the node naming the field
that was wrong: cannot cast the value into Verdict: not a valid Verdict: field "score": "high" is not a number. If you want to handle that instead of
failing the run, wire the Cast’s _error.
Secrets
Secret[T] is a T that flows through a run like any value while the run’s
record holds only a marker naming it. A caller’s Authorization header on a
Route or a Socket comes out this way, on credentials, and so does a token
MintInstanceToken makes. The record, the inspector, weft events and search
show the marker. The value is kept sealed beside the record, and goes when the
record goes.
A secret turns back into its value only inside a node that asks for it, and
only where the type of what you wired in says a secret is. Some of weft’s own
do it for you: HttpRequest sends a secret in its headers or its body
revealed, Reply and Stream send one to their caller revealed, and
ExecPython hands its script the value of an input you declared as a secret.
If a secret reaches one of them somewhere its input’s type does not declare
one (inside a JsonDict, or in a Stream’s bus message, which carries no
type), the node fails and names the input and where the secret is, rather
than sending the marker. Declare the type, like an object’s fields, and it
goes out revealed. If you want an HttpRequest body to carry a token, declare
the object you build for it:
mint = MintInstanceToken { instance: "user-42", expiresInHours: 24 }
handoff = JsonObject -> (object: { instance: String, token: Secret[String] }) {
instance: "user-42"
token: mint.token
}
notify = HttpRequest {
url: "https://partner.example/tokens"
method: "POST"
body: handoff.object
}
Left as JsonObject { token: mint.token }, the same body builds, and the
HttpRequest fails when it runs, naming body and /token.
A body with no secret in it needs no declared type: a plain object, written or wired, goes out as it always did. For writing a node that makes or reveals one, go and read the ctx.
Only a node makes a secret, so nothing written in a .weft file can be one.
If you need to make one without writing a node, declare an ExecPython
output as Secret[String], and what the script returns there becomes a
secret. The build refuses a value written on a Secret port, a secret’s
marker written anywhere (secret-literal: it would name a secret of another
run), a Cast into or out of a secret, and an @instance_filled secret
(instance-filled-secret). A wire cannot read a key inside one either: if
node s has an output session typed Secret[{ token: String }],
s.session.token is the deref-path error, since what flows is the marker.
What an unwired port starts as
A port nobody wired and nobody wrote a value into has a zero value, which is what seeds a loop’s carry when you did not give it a starting value.
| Type | Starts as |
|---|---|
Number | 0 |
String | "" |
Boolean | false |
List[T] | [] |
Dict, JsonDict | {} |
| A union | The zero of its first variant that is not Null |
| Everything else | null |
Groups
A group holds several nodes under one name and folds shut like a folder. Its signature says what the outside can see.
credit = Group(db: Access[postgres], user: String) -> (paid: Boolean, refusal: String) {
# Take one credit off this account, or say why we cannot
debit = PostgresExecuteQuery(telegram_id: String) {
query: @file("assets/sql/spend_credit.sql")
account: self.db
telegram_id: self.user
}
read = ExecPython(rows: List[JsonDict]) -> (paid: Boolean, refusal: String) {
code: @file("assets/scripts/outcome.py")
rows: debit.rows
}
self.paid = read.paid
self.refusal = read.refusal
}
credit.db = pg.access
credit.user = ask.user
self is the group’s own interface. On the right of an arrow it means the
values coming in from outside; on the left it means the values going out. It is
always the innermost scope, so inside a loop inside a group, self is the
loop.
Set the group’s inputs from outside, on its name, as credit.db does. You
cannot write a literal onto self.port: a group’s input is given its value
from outside. The one exception is a metadata port, whose
value you write in the body.
A plain comment as the first thing in the body becomes the group’s description, which is what the graph shows when it is folded shut.
No node stands for a group when the program runs
A wire through a group’s ports is one wire, from the node that sends to the
node that reads, and the group’s ports change nothing on the way. The group
itself is a scope: the runtime decides, once for each copy of it (one per
iteration of a loop around it, one per @include of a file), whether it opens
or skips, and everything inside follows.
So folding a group in the editor costs nothing, and neither does nesting five deep.
When a group starts
A group with a gate wired to it (_should_flow or _should_not_flow) waits
for that value and opens or skips on what it says. One with a gate written in its braces decides at once.
One with no gate opens on the first value that reaches any node inside it from
outside, or at once when nothing comes in from outside; if every wire coming in
closes with nothing on it, it skips. An included file is a group, so each
@include starts the same way.
When it opens, every node inside that no wire feeds starts with it. So a node
with no inputs needs no _should_flow to get its turn. Every other node inside
waits on its own inputs, and every output leaves the moment it is made.
It only runs in a run that takes it along, though. A trigger’s run holds what
the trigger feeds plus what those nodes read, so a node with no inputs inside a
group comes along when something that run uses reads it, or when the run feeds
the group’s _should_flow, which brings the whole group in. If several routes
feed one include, a route that did not fire never holds it up: it is either
left out of the run, or it runs only to close its outputs, and a closure counts
as arrived.
A group input passes through
This is the part that surprises people.
You give a group a db: Access[postgres] input, then the thing feeding it closes. You
would expect the whole group to skip.
It does not. A group input holds nothing: whatever reaches it, a value or a closure, goes straight inside, and whichever node actually needed that value skips there, and the skip cascades from that node outward. Without a gate, the whole group skips only when every wire coming into it closes with nothing on it.
That is deliberate. A group is usually several jobs, and one missing input often only kills one of them. Skipping the whole group would throw away work that had everything it needed.
So a group input is never required or optional itself, and a ? on one is
refused with optional-pass-through-port. The editor draws these ports with
their own marker, a ring with a dot in the middle. Whether a node runs without
the value is said on that node’s own input. It also means @require_one_of is
refused on a group: put it on the node inside that needs one of those values.
A loop is the exception, for the ports it iterates or threads: with no list to
walk, there is no iteration to launch. Those ports keep their ?, and every
other port of a loop passes through like a group’s.
When a group input has to be connected
What the compiler checks is whether a node inside needs the port.
It follows the port inward, through any group, loop or included file nested
behind it, until it reaches the nodes that read it. If one of them takes it
into an input it needs (one written without ?, see optional and
required inputs; a default does not
help, since it only fills an input no wire reaches, and an input of a join node
such as FirstInOrder is optional without any ?), or into the only
member of a @require_one_of set that could get a value, and nothing outside
connects or writes the group’s port, the build fails with
required-port-unmet, naming both ends:
input 'g.b' has nothing connected, and inside it feeds 'g.f.y', a required
input, so 'g.b' is required too: wire or write it
A port that only inputs written with ? read may stay unconnected: the closure goes
inside and each of them handles it the way it handles any closure. An included
file’s ports and a loop’s other ports follow the same rule.
Skipping the whole thing
_should_flow goes inside the braces, and when it skips a group it skips
everything in it, however deep, including nested groups. One line guards a
whole subgraph.
brief = Group(request: String) -> (prompt: String) {
write = Format(request: String) {
template: "Draw this: {{request}}"
request: self.request
}
self.prompt = write.text
}
brief.request = ask.text
brief._should_flow = credit.paid
The gate is consumed at the boundary. The nodes inside never see it, and they do not each get to decide: they take the group’s decision as a whole.
Only the incoming side takes a gate. The outgoing side has nothing to decide, because a group that did not run has nothing to forward.
What crosses the boundary
| Thing | Crosses? |
|---|---|
| Ordinary values, connections, file markers, bus markers | Yes |
| A closure | Yes, and it cascades to whatever inside needed the value |
_should_flow and _should_not_flow | No. Read by the group itself; no node inside sees it |
A stream (Generator[T]) | No |
| A wire from inside to outside that skips the interface | No |
A stream is a live edge between two running nodes, so both ends have to be in
the same scope. If you need a stream to cross, either move the consumer in, or
have a node read the stream and emit ordinary values. A loop is the one thing
that takes a stream across its edge, through its own over.
That last row is the scope-reachability error. A value crossing the line has
to appear on the interface, where somebody reading the group can see it.
Metadata ports
Say a group is a tool a model can call. The model has to be told about the tool (its name, what it does, what arguments it takes) before it runs, and the group only runs once the model has asked for it. The description belongs to the group, but the node that needs it sits before the group in the graph. An ordinary wire from the group back to the model would be a cycle.
A metadata port holds a value you write in the program, which no run
computes. You declare one among the group’s outputs with a * before its name,
and write its value in the body, on self:
search = Group(query: String) -> (results: String, *tool: { name: String, description: String }) {
self.tool = { "name": "search", "description": "Search the web." }
echo = Text {}
echo.value = self.query
self.results = echo.value
}
search.query = "cats"
tools = List {}
tools.search = search.tool
The value is JSON, so its keys are quoted. The port needs a type, and the type
cannot hold a type variable like T, because nothing wired to the port could
tell what T stands for. (A tool a real model reads also needs parameters:
see the LlmTool further down.)
A wire that leaves a metadata port is a metadata wire, and it never runs.
When the program compiles, the compiler writes the value into the input the
wire points at, as if you had typed tools.search = { "name": "search", ... }
yourself. So search does not have to run for tools to have its value.
That is why a metadata wire may point backwards. This compiles, and with an
ordinary wire from g into tools it would be a graph-cycle:
g = Group(items: List[{ name: String }]) -> (*tool: { name: String }) {
self.tool = { "name": "g" }
shown = Debug { data: self.items }
}
tools = List {}
tools.g = g.tool
g.items = tools.list
Where the value lands
The value ends up where the wire ends, and from there on it is a value you wrote:
- On a node’s input. The input takes the metadata port’s type, so in the
first example the List’s output is
List[{name: String, description: String}]. If the value does not fit the input, that isconfig-type-mismatch. If the input already has a value written on it, or a second metadata wire, that isdouble-driven-port. An input that takes only wires refuses it withinput-accepts. - On a group’s input, from outside (
h.t = search.tool). The value is written on the group’s port, and the nodes inside that readself.tget it when the group opens, like any value written there.
If the wire reads one key of the value (search.tool.name), that key’s value
is what gets written, and a key the value does not hold is deref-path.
Taking the value from other metadata ports
Instead of writing the value out, a group can take it from other metadata
ports. Name one where a value goes, as member.port, and the compiler puts
that port’s value there. A name reads the way it does where you write it: a
member of the group first, else something of that name in an enclosing
group, nearest first, else something at the top of the file. That is how a
value gets out of a nested group:
outer = Group() -> (*tool: { name: String }) {
inner = Group() -> (*tool: { name: String }) {
self.tool = { "name": "deep" }
}
self.tool = inner.tool
}
A name can also sit inside the JSON, and inside a list ... puts a list’s
items in place of the name, so a group can gather what its members describe:
toolbox = Group() -> (*tools: List[{ name: String }]) {
one = Group() -> (*tools: List[{ name: String }]) {
self.tools = [{ "name": "one" }, { "name": "uno" }]
}
two = Group() -> (*tool: { name: String }) {
self.tool = { "name": "two" }
}
self.tools = [...one.tools, two.tool, { "name": "three" }]
}
toolbox.tools is the four tools, in that order. A name reads keys off a
value the way a wire does (act.tools.anthropic), and ... on a value that
is no list is type-mismatch. What comes out is held to the type the port
declares, item by item, so one spliced item of another shape is
config-type-mismatch.
Only another metadata port can fill one this way. If made is a node,
self.tool = made.value is metadata-value-not-literal, because made.value
only exists once the program runs. A name that is nothing declared is
unknown-source-node, a port that member does not have is
unknown-source-port, and two metadata ports that take their value from each
other are metadata-reference-cycle. A group’s own ports are no value of its
own: self.x inside the value is a parse error.
If you write a tool once in its own file and @include it in several places,
every include hands out the same value, the one the file works out.
What is refused
- A metadata port with no value is
metadata-port-not-written, and the message shows the line to add. - A node inside the group cannot read the group’s own metadata port:
shown.data = self.tool, written insidesearch, ismetadata-port-read-inside. If a node inside needs the same value, write it on that node’s input too, or keep it on a member’s metadata port and name that port in both places. - Nothing goes into a metadata port from outside. A wire into
search.toolismetadata-port-as-target, and a value written there (search.tool = ...) is refused with the same message. - The value cannot leave a group through an ordinary output:
self.out = search.tool, whereouthas no*, ismetadata-into-output. Declareoutas a metadata port instead.
A few mistakes are caught while the file is read, and come back as a parse
error: a * in the input list, a * on a node (a node type declares its own,
in its metadata.json), a metadata port
with no type or with a type variable, a metadata port with the name of an
input or an output, two self.tool = ... lines for one port, a value
written on a key of one (self.tool.name = ...), an inline node as a
metadata port’s value (it computes its value during the run), and a value
that does not read. For why a name may not
repeat an input’s, go and read names you cannot use.
A model that calls tools
Here is one round of tool calls: the model is asked once, and each tool call
it makes is answered. It compiles as written, with model standing in for
whichever provider node you use. The loop runs once per call: because of
over, self.calls holds one call each time round, and carry hands the
conversation file from one call to the next. For how over and carry work,
go and read Loops.
model = CustomProvider { baseUrl: "http://127.0.0.1:9/v1", model: "m" }
handle = Loop(calls: List[ToolCall]) -> (historyFile: File, *dogTool: LlmTool) {
over: ["calls"]
carry: ["historyFile"]
dog = Group(call: ToolCall) -> (callId: String, result: String, *tool: LlmTool) {
self.tool = { "name": "dogName", "description": "The dog's name.", "parameters": { "type": "object" } }
work = ExecPython(call: ToolCall) -> (callId: String, name: String) {
code: "return {'callId': call['id'], 'name': 'Biscuit'}"
}
work.call = self.call
self.callId = work.callId
self.result = work.name
}
self.dogTool = dog.tool
route = Dispatch
route.value = self.calls
route.dog = dog.tool
dog.call = route.dog
unknown = Debug
unknown.data = route.otherwise
answer = ChatHistoryAppend { role: "tool" }
answer.historyFile = self.historyFile
answer.toolCallId = dog.callId
answer.text = dog.result
self.historyFile = answer.historyFile
}
tools = List
tools.dog = handle.dogTool
ask = LlmInference { prompt: "What is my dog's name?" }
ask.provider = model.provider
ask.tools = tools.list
handle.calls = ask.toolCalls
handle.historyFile = ask.historyFile
done = Debug { data: handle.historyFile }
The tool is the group dog. Its *tool port describes it as an LlmTool,
{ name: String, description?: String, parameters: JsonDict }, where
parameters is a JSON Schema for the tool’s arguments ({ "type": "object" }
is a tool that takes none). Each call the model makes is a ToolCall,
{ id: String, name: String, arguments: String }, with arguments holding
the model’s JSON as text.
Three metadata wires carry that description. Two take it to the model:
self.dogTool = dog.tool hands it out of the loop on the loop’s own metadata
port, and tools.dog = handle.dogTool puts it in the List that feeds the
model’s tools. The model runs before the loop, so that wire points
backwards.
The third, route.dog = dog.tool, adds a branch named dog to the
Dispatch node, which sends each call to the branch whose name matches the
call’s. route.dog is written twice, and the two are different ports: on the
left of = it is the branch’s input, which takes the description, and on the
right (dog.call = route.dog) it is the output of the same name, which the
compiler makes and which carries the calls that branch claims. A call that
matches no branch leaves on route.otherwise. For how a node asks for that
pairing, go and read pairsAddedPorts. The
tool’s name is written once, on its own group, and both the model and the
routing read it from there.
If you want a second tool, write a second group beside dog with its own
*tool, add *catTool: LlmTool to the loop’s outputs, and give cat the
same wires as dog: self.catTool = cat.tool fills that port,
tools.cat = handle.catTool adds it to the List, route.cat = cat.tool adds
its branch, and cat.call = route.cat takes its calls. answer would then
get two wires into toolCallId and two into text, and an input takes one.
So read the id straight off the call instead of dog.callId
(answer.toolCallId = self.calls.id),
and take whichever result came back with FirstInOrder
(answer.text = FirstInOrder { dog: dog.result, cat: cat.result }.value).
For a model that keeps calling tools until it answers, this round goes inside
a loop of turns that carries the conversation file. For that shape, read the
LlmInference node’s description: weft describe-nodes --node LlmInference.
Names you cannot use
Group, Loop, LoopIn, LoopOut and self are the
language’s. So is any name containing a double underscore, which is the
compiler’s own separator.
A metadata port’s name may not be the name of an input or an output of the
same group, so self.tool and search.tool only ever mean one port. An input
and an output may still share a name.
Naming a group or a node after a node type is also refused, because
Ticket.body would then read as an inline node rather than a reference. That
is the reserved-name error.
How big a level should be
Six items on one level is what the language asks for. The compiler warns past fifteen, names the level, and tells you to group.
A group counts as one item on its parent’s level, whatever is inside it. So a program grows downward into nested folders rather than sideways into a wall, and every level stays something you can hold in your head.
A file’s top level is counted per connected part. Everything a wire links counts together, a database or a key set that several pieces use included, so the warning reads “one connected part of the top level holds N items”. Only pieces that share no wire at all are counted apart. The inside of a group, a loop and an included file is counted whole (“this included file holds N items”).
Loops
A loop runs what is inside it once per item of a list, or until the body says stop.
grade = Loop(cards: List[JsonDict], rubric: String, provider: LlmProvider) -> (scores: List[Number | Null]) {
over: ["cards"]
ask = Format(card: JsonDict, rubric: String) {
template: "{{rubric}}\n\nThe card: {{card}}\n\nAnswer with one number."
card: self.cards
rubric: self.rubric
}
mark = LlmInference {
prompt: ask.text
provider: self.provider
}
self.scores = Cast { value: mark.response }.value
}
grade.cards = pull.rows
grade.rubric = "Score this flashcard from 0 to 10 for clarity."
grade.provider = model.provider
Unlike a group, a loop becomes nodes: the compiler turns it into a LoopIn and
a LoopOut, and the runtime drives the iterations between them.
The five settings
They go in the body as ordinary lines. Anything else starting with a letter is
a node, and anything else that looks like a setting is a hard error, because a
typo like maxItres: 10 silently running your loop uncapped is exactly what
the language refuses to allow.
| Setting | Default | What it does |
|---|---|---|
over: ["cards"] | [] | Which inputs to iterate. Each has to be a List[T] or a Generator[T] |
carry: ["total"] | [] | Values threaded from one iteration to the next |
parallel: true | false | Launch every iteration at once, rather than each once the one before it reaches the loop’s edge |
maxIters: 50 | no cap | A hard limit. 0 is legal and means no iterations |
trimOnMismatch: false | true | With several over lists of different lengths: trim to the shortest, or fail |
The four kinds of port
Which kind a port is depends on where it is and whether you named it.
| Kind | Outside the loop | Inside the loop |
|---|---|---|
Iterated, named in over | List[T] or Generator[T] | One T, a different one each iteration |
| Broadcast, any other input | T | The same T, every iteration |
Gathered, any output not in carry | List[T | Null], one slot per iteration | You write one T per iteration |
Carried, named in carry | The seed goes in, the final value comes out | The running value in, the new one out |
A gathered output has to allow null, because an iteration that sends
nothing on it (it failed, or a branch inside it was not taken) leaves a null
in its slot. If you declare it without | Null, the build stops with
gather-output-must-be-nullable.
The list that comes out is built for every gathered port, even one that no
iteration ever wrote, because a port that produces nothing at all would leave
whatever is downstream waiting forever. A port nothing inside the loop writes
at all is refused when you build (loop-output-unwritten): it could only
gather nulls.
A carried value that an iteration does not write keeps what it had, as long as nothing broke: a branch not taken or a gate that said no is fine.
A carried value needs a seed from outside, wired or written
(myLoop.total = 0 for a loop named myLoop); without one, the build fails
with required input 'myLoop.total' has no driver. The one case it can go
without: you declare it among the loop’s inputs with a ? (total?: Number)
and nothing inside wires it into an input written without ?. Then it
starts at the zero for its type: 0 for a number, "" for a string, false
for a boolean, [] for a list, {} for a dict, and null for a type with no
empty value, such as a File or an Image. A seed that is wired but arrives
as null starts at the zero too, and for a type like Number | Null that zero
is 0, not null.
A loop can also declare a metadata port
(*tool: LlmTool), which is none of the four: it holds one value written
in the program, never a list.
Two implicit ports
self.index is a Number, readable inside, and it is which iteration you are
in, counting from zero.
self.done is a Boolean you can write inside, and it is how the body votes
to stop.
Both only appear if you did not declare a port of that name yourself. And
index as an input name, or done as an output name, is the
reserved-port-name error.
One at a time, or all at once
Sequential is the default. The next iteration launches once this one’s values
reach the loop’s edge: its carried values, what it gathers, and its
self.done. That is what makes carry possible: the next iteration reads the
value the last one wrote. Nothing else holds the next iteration back, so a
loop with no outputs launches its iterations back to back. If one item’s work
has to finish before the next item starts (a database write the next item
reads), carry a value out of that work into the next iteration.
A loop launches its iterations and does not own them. Work inside an iteration that does not feed the loop’s edge keeps running after the loop has moved on, and after the loop has ended. That is what lets a loop start twenty agents, each holding a bus, gather the buses and exit, so the nodes after the loop can talk to every agent while they all keep running.
parallel: true launches every iteration up front. Three things it refuses:
- with
carry, because carrying means each turn feeds the next, and that is sequential by definition - without
over, because there is nothing to spread out - with
self.done, because a vote to stop means nothing when everything already started
How a loop ends
- It runs out of items in
over. - It hits
maxIters. - The body writes
self.done = true, on a sequential loop. - A carried value is lost to a failure, and then the loop fails (see below).
A sequential loop with none of those is the
loop-unbounded-no-termination error, refused at compile time rather than
left to run forever.
With several over lists of different lengths, the default trims to the
shortest. Set trimOnMismatch: false and a mismatch fails the run instead,
naming the lengths.
When an iteration fails
If you only gather, the run carries on. That iteration’s slot in every
gathered list is null, so a loop over a hundred rows where three fail gives
you a hundred slots with three nulls in them. Whatever reads that list decides
what a null means.
If the failure means a carried value never got written, the loop stops there
and fails. The next iteration would start from the same value and meet the
same failure: an agent loop whose model call broke would otherwise pay for
the same doomed call up to maxIters times, then hand on a stale history as
if all went well. The loop fails with
loop 'agent' stopped at iteration 4: its carried value 'history' could not be updated because 'ask' failed: <the error>
where ask is the node that actually broke, even when other nodes between it
and the loop’s end skipped because of it. Everything reading the loop’s outputs
sees a failure, the same as for any failed node, and ask shows as failed in
that iteration too.
A carried value that closed with no failure behind it, because a branch was not taken or a gate said no, still keeps what it had.
What cannot go in one
A trigger and an infra node are both refused inside a loop, with
trigger-in-loop and infra-in-loop. A trigger registers once for the
project, and infrastructure is provisioned once for the project, so a
per-iteration one is not a thing.
Running part of its body
If you want to prove one stage of a loop’s body without running the loop over
every item, run that stage on its own. Every way of cutting a run (--group,
--from, --emit, --target, --before, --feed) works inside a body,
loops inside loops included. Say the loop clips runs a group edit on each
item, reading the item and self.index, then caption and upper after it:
weft run --from 'clips.edit={"clip": "intro.mp4", "at": 0}' --target clips.upper
The part you pick runs once, as if it were the first time round: inside
iteration 0 of each loop around it, so weft runs, weft events and the
graph show its nodes at #0. Nothing of those loops runs: no other
iteration, and nothing reaches the loop’s outputs. A loop that sits wholly
inside the part you picked runs normally, every iteration of it.
What the loop would have handed the part (the item, self.index, a carried
value) is a value from outside the run like any other: you hand it at the
start, as edit’s clip and at above, or the run is refused before it
starts, naming the input that would get nothing.
What is refused is cutting a loop in half. A loop has two ends, its start (where its inputs go in) and its end (where its outputs come out), and a run holds both or neither. Ending the run inside the body with nothing started there reaches back to the loop’s start for the item:
the run holds loop 'clips''s start but not its end: include both to run the loop, or neither to run part of its body
Starting inside the body with no end there reaches the loop’s outputs, which runs the loop whole, every iteration, around a start that cannot apply to all of them:
'clips.edit' is inside loop 'clips', and the run holds both of the loop's ends, so it runs the loop
whole: a run cannot start, stand in or stop inside a loop it runs whole. To run part of the loop's
body, end the run inside the body too (--target a node in it), so the run holds neither end; to run
the loop, start or end at the loop
In both cases you either start and end inside the body, or run the whole loop.
A loop in a file you include at several places is one loop at all of them, so one run takes it the same way everywhere: whole at every place it holds it, or part of its body at every place. A run that holds all of it at one place and part of its body at another is refused, naming both; run them in two runs.
Streams and buses
Most wires carry one value. Two things carry more than one, and they are for different jobs.
A stream is one node handing items to one other node, in order, until it
ends: the numbers Range counts out, or the messages a Socket hands on as
its caller sends them.
A bus is several nodes that are all running at once talking to each other:
a model writing its answer, a Stream passing the words to the caller, and
a voice reading them aloud as they come.
Streams
A stream is a port typed Generator[T]. Range hands out numbers that way, and
a loop over it takes one per iteration:
numbers = Range { to: 1000 }
double = Loop(n: Generator[Number]) -> (doubled: List[Number | Null]) {
over: ["n"]
times = ExecPython(n: Number) -> (out: Number) {
n: self.n
code: "return {'out': n * 2}"
}
self.doubled = times.out
}
double.n = numbers.values
The | Null is required, because an iteration that sends nothing leaves a
null in its place in the list. For the other kinds of loop port, go and read
the four kinds of port.
Items travel as ordinary pulses, each checked against the element type. The consumer starts when the first item arrives and pulls the rest as it goes, so it is running while the producer is still producing.
A closure on a stream is its end, not a skip. This is the one place where a closed wire does not make the consumer skip. It runs, sees the stream is over, and acts on it. A stream that carried nothing is an empty stream, which is a result, not an absence: zero rows has to behave like one row, not like a branch that never happened.
If the producer failed, the consumer finds out through an error rather than a clean end, so nothing mistakes a broken stream for an empty one.
What a stream will not do
| Why | |
|---|---|
| Feed two consumers | Two readers on one stream is a question with no answer: do both see every item, or do they race? Fanning out is what a bus is for |
| Cross a group boundary | It is a live edge between two running nodes, so both ends have to be in the same scope. A loop takes one through its own over and nowhere else |
| Be optional, or have a default | There is no such thing as an empty stream sitting unwired. A stream input is required |
| Sit inside a list, dict, record or union | It is a port type, not a value |
Be a loop’s carry | A carried value goes through the journal as JSON between iterations, and a stream is a live thing |
| Be stored, cast, or turned into text | Same reason |
A producer that runs far ahead of its consumer will fill the buffer: 4096 un-taken items, and then it fails and tells you the three ways out, which are to yield instead of pulse so each item waits to be taken, to let the consumer catch up, or to raise the cap.
Buses
A bus is a channel any number of nodes join. One node makes it and emits a marker, and anything that gets the marker can join.
ask = Route -> (body: { question: String }) { path: "ask", method: "POST" }
answer = LlmStream {
prompt: ask.body.question
provider: model.provider
}
toCaller = Stream { bus: answer.stream }
voice = ElevenLabsSpeak {
account: eleven.access
text: answer.stream
voice: "JBFqnCBsd6RMkjVDRZzb"
}
LlmStream makes the bus and sends the model’s answer on it as it is
written. Stream passes it to the caller, and at the same time
ElevenLabsSpeak turns the same words into audio on its own audio bus (its
voice is an ElevenLabs voice id). model and eleven are the model and
the ElevenLabs connection, set up elsewhere in the program.
The marker is an ordinary value. It crosses group boundaries, goes into loop bodies, sits inside records, and travels like anything else. Only the live channel is special, and it lives in the worker rather than on the wire.
Every participant registers a name, which is the stamp on everything it sends and what other nodes wait for. Registering is the “I am here and ready” moment, so a node with a slow warmup holds the bus and registers when it is genuinely ready rather than when it starts.
Each reader has its own position. Reading does not consume, so ten readers all see everything.
Two modes
| Journaled, the default | Ephemeral | |
|---|---|---|
| What the record keeps | Every message | Counts per sender, and nothing else |
| A reader that falls behind | Can reach back as far as the window | Skips to the oldest message still held |
| If the record cannot be written | The send is refused before anything is appended | Nothing is refused |
Ephemeral is for a firehose. Video frames should not stall because something downstream is slow, and they should not end up in permanent storage either.
A node that streams lets you pick the mode with two settings:
journalEphemeral (on keeps payloads out of the record, off by default) and
journalWindowMs (a whole number from 100 to 60000, 1000 by default). Every
message sent within one window lands in the same row of the record. A shorter
window shows messages in the inspector sooner and gives a row per message on a
slow stream. A longer one writes fewer rows for a fast stream (audio frames,
tokens), and the inspector trails the live stream by up to that long. A row
also closes early once it holds 2 MB, and at once when the bus ends or the run
pauses. Route and Socket have both settings,
and so do LlmStream, ElevenLabsTranscribe and PostgresWatchQuery. Other
nodes have them only if their author turned them on.
The window is 64 messages either way. What is always kept, in both modes, is who joined and who left, so nothing a reader needs can disappear into a gap.
The record exists so you can read the conversation in the inspector afterwards, and so a paused run gets its buses back: the worker that picks the run up opens each one again from it (below).
How a bus ends
Picture a reader at the end of a model’s answer. Did the model finish, or did it stop halfway through a sentence? A bus says which. It ends when the node that made it is done, or when a node that joined it is, and it ends one of two ways:
- finished, when that node’s work completed, or when the node closed the bus early on purpose;
- failed, with the reason, when that node failed or was cancelled. weft also ends a bus failed when the run got stuck, when it ended while the bus was still open, and when a node was still running as its worker died.
Each reader decides what a failed end means for it. Stream fails the answer
to its caller (what the caller
sees), even on a
route whose whenChunkFails is drop_chunk, because that setting skips one
message and a failed bus has no next one to go on to. Debug logs
a warning naming the reason, and the bus’s panel in the inspector ends with
* the bus failed here: <reason> instead of * the bus closed here.
If a producer fails before its first piece, it may not have opened its bus yet. For what the program can do then, go and read when a node fails.
A bus pauses with its run
A bus is memory in the worker running the run, and it does not stop the run from pausing. A node can wait a week on a question while a bus between two other nodes stays open: once every node still running waits, the run pauses, buses and all, and the worker lets go. Nobody on the bus sees anybody leave.
When the run picks up, the new worker opens every bus that was open again,
with the same id, so every marker still names it, and its messages carry on
numbering from where they stopped. A journaled bus comes back with its last
JSON messages, as many as its window holds, so a reader can still reach back;
the record keeps a message over 100 KB trimmed, so that message and every one
before it are left out. An
ephemeral bus, or one that carries bytes, comes back empty. A reader that was
waiting for the next message had read everything and loses nothing. One that
stopped to wait on something else carries on where it stopped, and if the
messages it had not read did not come back, its first read fails, saying
so. A node that must not stop reading at all holds the run
instead (ctx.hold()), which keeps the run from pausing while it reads.
A worker that dies is different, because the last messages it carried may
never have reached the record. If a durable run carries on in another worker
and any node was still running when the first one went, each bus that was
open ends failed, with the worker holding this bus went away, so messages it carried may be lost.
A seeded run cannot reuse a node that emitted a bus, since that bus belonged to the earlier run: weft leaves the producer and its readers out rather than handing you a marker pointing at nothing.
What a bus does not check
Payloads. A stream checks every item against its element type; a bus carries whatever you send. You pick JSON or bytes when you make it and that is frozen, but what is inside is between the nodes talking.
Which one
| You want | Use |
|---|---|
| One node’s output, arriving in pieces | A stream |
| Several live nodes talking to each other | A bus |
| To fan one thing out to several readers | A bus |
| Ordering and type checking on every item | A stream |
| To survive its worker dying | Neither. Use storage, or a node that ends |
For the Rust on both sides, go and read streams and buses in Rust.
Triggers and routes
A trigger is how a program starts without you. A message arriving, a schedule coming round, a form somebody fills in, a request hitting an address.
ask = TelegramReceiveMessage { account: telegram.access }
Wire it like anything else. What comes out of it is the event.
Nothing listens until you run weft activate. Go and read
the three lifecycles for why that is its own step.
What a run started by a trigger covers
Everything downstream of the trigger that fired, plus whatever that work needs upstream. Any other trigger in the project closes its outputs, so paths that needed it skip.
A trigger’s inputs are frozen
Whatever fed a trigger’s ports when you activated is what it uses on every firing. The nodes upstream do not run again per event.
Everything upstream of every trigger runs at activation as one program, so two
nodes that do not depend on each other run at the same time. Setup that several
triggers share, like the schema all their tables live in, is written once and
wired to each of them; two copies of create extension if not exists racing
each other is how one of them fails on a duplicate key.
If you want something done once before the triggers listen, like creating the
table they write to, wire it in front of them. A gate is enough, since any
value but false lets the trigger through:
pg = PostgresDatabase {}
schema = PostgresExecuteQuery {
account: pg.access
query: "create table if not exists cards (id text primary key, title text)"
}
add = Route { path: "cards", method: "POST" }
add._should_flow = schema.rows
list = Route { path: "cards", method: "GET" }
list._should_flow = schema.rows
schema runs once, at weft activate, before either route listens, and if it
fails the activation is cancelled. A request never runs it again.
So after changing anything in front of a trigger, weft resync. Until then
weft status shows the trigger as behind, and your listener carries on with
what it registered. weft tells you when a bake is stale:
the code changed since trigger 'inbound' was last prepared, so its bake is
stale. Prepare it again: `weft bake` does it and changes nothing that listens;
if the trigger is off, `weft activate` does it and turns it on; if it is on,
`weft resync` brings it onto the new build and prepares it there
Answering on a URL
Route claims an address and starts a run when somebody calls it.
type Card = { title: String, color?: String }
post = Route -> (body: Card) {
path: "cards"
method: "POST"
}
answer = JsonObject {
title: post.body.title
status: "saved"
}
reply = Reply { body: answer.object }
The caller is held open while your program works, and Reply is what answers
them. For streaming an answer as it is produced, and for the Rust side, go and
read talking to a live caller.
If only some callers may call it, set its auth (auth: "api_keys", say)
and pick the connection holding the keys with weft connect post. For every
scheme, go and read checking a signed request.
The request’s body comes out whole on body. Left undeclared, it is a
JsonDict. If you declare a type on it, as Card above, a call whose body
does not fit is answered 400 naming what is wrong, and no node after post
runs. A body carrying a key that Card does not declare is answered 400
too. If callers may
send more than you read, end the type with ..; for what that does, go and
read what fits into what. If callers send JSON
of any shape, a list or a bare number as well as an object, declare
-> (body: Json). A request with no body, like a GET, closes body,
whatever the route’s dataType. A json body that is null is a value: it
comes out on a body whose type takes Null (Json does), and closes
body on any other, the way a null counts as nothing on a port that does
not take it.
If you want part of the path as a value, capture it with {name} and declare
an output of the same name:
card = Route -> (id: String) {
path: "cards/{id}"
method: "GET"
}
A call to cards/42 puts "42" on card.id. Every other output you add to a
Route has to be a capture of its path, typed String, or the build stops
with route-output-not-a-capture. A capture can’t share a name with one of
Route’s own outputs (body, method, path, query, headers,
credentials, caller, fired), so cards/{body} is the route-capture-shadows-output error,
asking you to rename the capture.
A caller’s credentials come out on credentials, never on headers. The
headers that carry one (authorization, proxy-authorization, cookie,
weft-instance-token, and any header the route’s auth reads, like the
X-Api-Key that auth: "api_keys" checks) are
each made a secret the moment the call arrives, before anything of the run is
written, so the run’s record, the inspector, weft events and search hold a
marker in their place and never the value. The value is still there for the
run, and for the same run picked up after a pause. If you want to pass them
on, wire them into a port that takes secrets, like an HttpRequest’s headers, which
sends them revealed. Here a route hands its caller’s credentials on to your
own backend:
me = Route { path: "me", method: "GET" }
profile = HttpRequest {
url: "https://api.internal.example/me"
method: "GET"
}
profile.headers = me.credentials
A plain String port refuses a secret: wired into a Dict[String, String],
me.credentials is a type-mismatch telling you the node has to take the
secret and reveal it. For what a secret is, go and read
secrets.
Two routes that one call could reach, where neither is more specific, is an
error. cards/count beside cards/{id} is fine, because the first is more
specific. Two spellings that genuinely overlap are the route-overlap error,
because a call arriving would have no defined answer.
You know a route’s address before you activate: on your machine it is
http://127.0.0.1:14111/connect/local/<project id>/cards (the install’s
address, then /connect/local/, then your project’s id, then the path). The
project id is the id under [package] in your weft.toml, so it is the
same on every install, and two projects can each serve a cards route without
getting in each other’s way. On a cloud install it is the install’s own
address, then /connect/local/<project id>/, then the path. weft activate
prints the id (activated cards (<project id>)), and the route’s trigger
shows its full address in the editor.
If you want the fastest way in, or a frontend wants your routes at the root of
an address, call the project’s own address, where your program answers with
nothing of weft’s in between. On your machine that is a free port the project
gets the first time you activate it and keeps from then on
(http://127.0.0.1:14200/cards); if another program takes that port, the
project moves to another free one. On a cloud install it is the project’s own
Cloud Run address. On your machine, if you want a port of your choosing, weft activate --port 8080 opens that one, and the project keeps it for later
activates. If another program or another of your projects holds that port,
weft activate fails and nothing is activated.
weft activate prints the address, and weft status shows it later or says
why it is unavailable. Your routes keep answering under /connect/local/<project id>/ as well.
If a call arrives while the route’s trigger is down or still being set up, it
does not wait for the trigger: it is answered 503 at once, with a Retry-After of 5 seconds,
so a client that retries gets through once the trigger is back on. The
exception is a trigger taken down with weft deactivate --door refuse --held drop, which takes its route away with it, so a call is answered 404,
the same as a call to a path no route serves. For what a take-down does with
other work, like a webhook or a form, go and read starting and stopping
things.
How a run is kept
Your runs happen inside a worker: a copy of your compiled program, running as a process on your machine, or as a Cloud Run service on a cloud install. A worker keeps the runs it is working on in its memory. A trigger has inputs that decide what happens to those runs: whether they survive their worker dying, whether they are written down, how long they are kept once they end, and how long a wait holds when the run cannot pause. You write them in the trigger’s braces like any other input:
pay = Route -> (body: JsonDict) {
path: "pay"
method: "POST"
durable: true
}
| Input | Default | Change it when |
|---|---|---|
durable | off | a run has to carry on after its worker dies, instead of ending |
recorded | on | a page asks a route for something every few seconds (turn it off) |
outlivesCaller | off | a run answers its caller early and keeps working (only on a trigger that holds a caller on the line, like Route and Socket) |
keepRunsFor | the project’s, else a week | you want its runs kept longer or shorter once they end (12h, 30d, forever). For the project-wide default, go and read how long a run is kept |
journalEphemeral | off | what streams through the route should not be written to the run’s record, only counted. For what that keeps, go and read two modes |
journalWindowMs | 1000 | you want the record’s rows to cover less or more time, from 100 to 60000 ms. Same page |
holdSecs | 60 | a run that cannot pause should wait longer for an answer (up to 30 days), or not at all (0). See when a run cannot pause |
pauseAfterSecs | 10 | a run fed by frequent messages from outside should stay on its worker longer before it pauses (up to an hour), or pause at once (0). See when a run pauses |
By default (durable off), a run is fast: it only waits for the database
when it has to. That is when it pauses (a timer, a form), and in a few
other places (for the full list, go and read when a run waits for
its writes). The
rest of the time its work lives in the worker’s memory, and its record (the
journal that weft events and the editor read) is written a moment behind
it, as it goes. If its worker dies, the run ends
cancelled and is not run again. For why, and for what happens when the
platform shuts a worker down on purpose, go and read what happens when
something dies.
If a run must not end when its worker dies (it moves money, say), turn
durable on. Before each node starts, everything the run did so far is
written to the database, and a route’s answer is written before it is sent.
The exception is a pure node
(Text, JsonObject, Switch, Reply), which does nothing outside the run
but answer its caller and handle the run’s own files, and starts without
waiting.
If the worker dies mid-run, another worker picks the run up where it stopped:
every finished node stays finished, and a node that was in the middle of its
work is failed with a message saying it may have partly happened, so you know
which node to check. Two kinds of node run again instead: one that saved a
state (ctx.keep), from its last save, and a pure node that takes no stream
and had not passed anything on yet. If a caller
started the run, its connection was on the old worker and is gone: the run
carries on in the new worker with nobody on the line, so a node that answers
the caller fails, saying no live caller is attached. If the platform stops the
worker instead, a run that can pause gets 5 seconds for the nodes already
running to finish, and a node still running after that runs again from its
last save if it saved a state, or is failed if it did not. A run
that cannot pause (below) keeps running on the
stopping worker for as long as the platform lets it, and if the worker goes
before the run ends, that is the same as the worker dying. If you want to know
what a wired _error does with that failure, or what happens to a node that was
waiting on an answer or reading a stream, go and read surviving a
restart.
Even with durable off, a paused run survives its worker. A run that cannot
pause is the exception, below.
When a run pauses
A run pauses in one way: every node running in it waits (on a timer, a form, a
message, a bus or a stream), at least one of those waits needs an answer from
outside, and the run has stayed like that for the trigger’s pauseAfterSecs
(10 seconds unless it says otherwise). Then every node saves where it is, all
together, the record is written, and the worker lets go.
During those seconds each wait stays open in its node. If its answer arrives,
it goes straight into the node, which carries on from that very line: nothing
pauses and nothing starts again. So a run fed by a steady trickle of messages
is not paused and picked up again for every one of them. Each time it goes
quiet again, the count starts over. With pauseAfterSecs: 0 it pauses as soon
as everything waits.
The run does not stay when weft needs it off its worker: if the worker is stopping, or the run has to move to a fresh one, it pauses straight away and the next worker picks it up. On a cloud install it also never stays past its hour on the worker, since reaching that would end it.
When a run cannot pause
Two kinds of run cannot pause: one whose caller is still on the line (the
caller cannot follow it to another worker, whatever outlivesCaller says),
and one with recorded off (there is no record to pick it back up from). A
node test cannot pause either, since it has no program for another worker to
pick up.
When such a run reaches a timer or a form, the wait holds in the node’s call
for as long as it needs. The wait is registered as usual, the run stays on
its worker, and the answer carries it on right there. The hold clock only runs while the run
is quiet, meaning every node still running is waiting, on an answer or on a
bus, and never while a caller is on the line. Anything that moves (a bus
message, a node finishing) starts it over. If
the run stays quiet for holdSecs (60 seconds unless the trigger says
otherwise), the call that was waiting fails, saying it gave up, and the wait is
withdrawn, so a form is no longer offered. The node’s code can handle that
error like any other, and if it does not, the program can: it comes out on the
node’s _error when that is wired (when a node
fails). With holdSecs: 0, such a
wait fails at once.
A node that holds the run (its code called ctx.hold(), because its work
cannot be picked up from the middle) works differently. While it holds, none
of its waits pauses, a timer or a form as much as a read, and the run keeps
its worker as a run that cannot pause does: for as long as a caller is on the
line, otherwise until nothing has moved in it for holdSecs, when the waiting
call gives up with WaitGaveUp, saying the node holds its run. The other
nodes’ waits stay open in their calls meanwhile.
If the run becomes able to pause while a wait holds (its caller hangs up, on
a route with outlivesCaller on), it pauses the way any other run does, once
it has been quiet for its pauseAfterSecs. A run whose caller leaves while outlivesCaller is off is cancelled,
as always. And a fast run holding a wait still ends with its worker.
If a page asks a route for a status every few seconds, turn recorded
off, or weft runs fills with hundreds of runs. weft keeps no history
of such a run, not even in its worker’s memory. If it fails, weft runs
lists it with the node that failed and why, and nothing of what ran before. If
it reports a cost, or asks weft for something on its behalf that its worker
does not already hold (a stored file, a new connection or infrastructure
address, a task such as starting or stopping its infra, a stop through
ctx.stop_tagged, a tag through ctx.tag_run, a wait on a timer or a form), it is listed with its costs
and how it ended. Otherwise it leaves nothing. If you want to check that a
trigger with recorded off is being called at all, weft status shows how
many runs each trigger started in the last minute or two, and how many of them
failed.
If you only want a key kept out of the record, you do not need recorded
off. A value that arrives as a secret (a caller’s credentials, a node’s
Secret output, an ExecPython output you declared Secret[String]) is
already kept out. For more, go and read secrets.
An unrecorded run cannot pause, so a timer or a form in it holds
(when a run cannot pause). A tag works on it (a
label you later find or stop runs by, ctx.tag_run): the run is listed, as
above, and a stop by that tag reaches it while it runs. The compiler also
refuses durable: true with recorded: false.
On a cloud install, one stretch of a run lasts an hour at most. Cloud Run cuts a request at 60 minutes, so a run stops itself 59 minutes after it started on its worker. It ends cancelled with a message saying so, instead of being cut off while a node is still running, with no ending written. A run that pauses starts a fresh hour when it picks back up; one branch waiting while another runs does not. So if a job can take longer than an hour, split it with a pause. On your machine no run is cut.
A run that answers a caller lives on the worker the caller reached, and stays
there while the caller is on the line. If the caller is still there at 59
minutes, the route’s outlivesCaller decides. Off, the run stops, with a
message telling you to have the client reconnect. On, the run moves to a fresh
worker, with a fresh hour. It starts no new node, and the nodes it is running
get 5 seconds to end, while the caller’s connection stays open for what they
still send. Once they have ended, or the 5 seconds are up, the caller gets a
504 with the usual failure (Something went wrong and the run id; a socket
closes with 1011), and the run carries on in the fresh worker. Why goes on
the run’s record, where weft logs <run id> shows it: this connection ended 60 seconds before the 60 minute limit this platform puts on one connection; run <id> goes on without it (its route outlives its caller), so reconnect to talk to the program again. A node still running after the 5 seconds stops
where it is, and the worker that picks the run up runs it again from its saved
state, or fails it as cut short if it keeps none.
Once the caller has gone, a route with outlivesCaller on keeps its run
going. It pauses if everything in it is waiting, and otherwise, on a cloud
install, it moves: it starts no new node on that worker, and once the nodes it
was running end or settle into a wait, it carries on under a call weft holds
open for it, with a fresh hour. Buses between its nodes come along: the worker
that picks it up opens them again where they stopped. Moving works from the run’s record, so the compiler
refuses outlivesCaller: true with recorded: false.
When a run ends before it answers
Every answer a route’s run gives, good or bad, carries a Weft-Run-Id header
holding the run’s id.
If a run ends before it has finished answering its caller, the caller gets one of these statuses instead, and this JSON body:
{"error": "Something went wrong", "runId": "9b81d0a2-..."}
| Status | When |
|---|---|
500 | the program failed, got stuck, or ended without answering |
504 | a time limit ran out (the four limits are listed below the table) |
503 | weft, a person, or another run (ctx.stop_tagged) stopped the run, weft could not keep it going, or it carries on in another worker without this caller |
A 504 means one of four limits ran out: on a cloud install, the run’s
hour; the route’s or socket’s maxSessionSecs (the longest one connection
may stay open, off unless you set it); a wait that held for its whole
holdSecs; or, on a cloud install, the hour a worker’s connection to weft
may last, when it runs out while every run on that worker only waits (for
why, go and read
the broker).
The caller never sees why the run ended: the reason can name your nodes, your
files and a provider’s words, and those are yours to read. The whole reason
stays on the run’s record, and you open it with weft logs <run id> or in the
graph. The id is safe to hand out, because opening a run takes your own
credential. If you want a caller to get a message of your own instead, wire
_error on the nodes that may fail and gate a Reply of yours on it; the run
then no longer fails (when a node
fails).
A 503 here carries no Retry-After, because calling again would start a
second run beside the one that may still be going. If the answer had already
started, its status was already sent, so the error comes at the end of the
body instead, in the answer’s own format:
| The answer is | It ends with |
|---|---|
| JSON lines | one more line, {"error": "Something went wrong", "runId": "...", "status": 504}, where status is the one the call would have got |
| Server-sent events | an error event whose data is {"error": "Something went wrong", "runId": "..."} |
| Anything else | a line [error] Something went wrong (run <id>) |
A socket closes with 1011 where a call would get 500 or 504, and with
1013 where it would get 503, and the close reason is that same
Something went wrong (run <id>).
A Stream answers this way too when the bus it reads ended failed: a 500 if
nothing was sent yet, the marker above if it was, 1011 on a socket. Its
done output then stays absent.
Answering on a socket
Socket is the same idea for a WebSocket: the caller connects, your program
runs, and the two talk until one of them stops.
A client that can send headers (a server, a script, an app) opens its socket at
the route’s address and is checked like any other call. A browser can’t put a
credential on a socket’s opening request, so on a route that checks callers it
asks first with a plain GET, which carries the credential, and gets back
{"url": "...", "protocol": "websocket"}; it opens its socket at that url
within a couple of minutes.
On a cloud install, one connection lasts an hour at most. Google cuts any
request to weft at 60 minutes, and an open socket is one long request, so a
socket (or a streaming answer) that is still open then is closed. A connection
can also drop sooner for ordinary reasons: a phone changing networks, a laptop
going to sleep. When it does, the route’s outlivesCaller decides what
happens to the run: off (the default), the run stops; on, it carries on and
what it sends goes nowhere.
So if you want a conversation to last longer than one connection, keep what it needs outside the run, and have the client reconnect:
- the client picks a session id once (or the program hands it one in its first message) and sends it on every connection, as a query parameter or in the first message;
- the program keeps the conversation’s state in its own storage, keyed by that id (a table in the project’s Postgres, a file), and reads it back when a connection arrives with an id it knows;
- the client reconnects whenever its socket closes, with the same id.
Each connection is then its own run, and losing one, at the hour or before,
loses nothing the conversation needs. If your client is a web page, the
connect library’s openLiveSocket does the client’s half: it keeps the
session id, sends it as the session query parameter on every connection
(your program reads it off the trigger’s query port), and reconnects until
the page closes the socket, your program closes it with code 4000, which
means the conversation is over (a run that simply ends closes with 1000,
and that ends only its connection), or, when the page passes credentials,
the route refuses them. A browser is told nothing about a socket refused at
its opening (it sees only that it closed), so a socket opened without
credentials keeps trying, at most every 15 seconds.
Watching a folder
If you want a run each time a file lands in a folder on your machine (a scan,
a download, an export from another program), use FolderWatch:
invoice = FolderWatch {
folder: "/home/me/Invoices"
pattern: "*.pdf"
events: ["created"]
}
Each run gets the file’s absolute path on invoice.path, what happened to it
on invoice.event (created, changed or removed), and its size on
invoice.sizeBytes, which is absent for a removed file. The worker runs on
the same machine, so a node downstream opens the path itself.
folder has to be an absolute path to a folder that exists when you run
weft activate. recursive: true watches the folders under it too, however
deep. pattern matches the file’s name only (*.{pdf,docx} for either).
If you leave events out, all three fire.
A save or a slow copy that writes one file many times starts one run, sent once the file has been quiet for a second. To decide what happened, weft compares the file with what it last saw rather than trusting the operating system’s event, because systems report a save differently: as a write, as a rename over the old file, or as a removal followed by a creation.
weft keeps what it last saw in its database, so a file that arrives, changes
or goes while weft is stopped, while it is in maintenance for an update, or
while your machine is off still fires once the trigger is back up, the same
as if weft had been watching. While you have the trigger down behind a door
that takes or hides, weft keeps watching, and what fires waits for the
trigger to be back on. Behind a door that refuses, or once a door’s
--door-for window ended, nothing is watched: what happened in the folder
meanwhile never fires, and turning the trigger back on starts it as if for
the first time. A poll keeps its place in its feed by the same rule; for
the rule itself, go and read taking triggers
down.
The first time the trigger starts, the files already in the folder fire
nothing. Changing folder, pattern or recursive counts as a first time;
changing events does not. If weft stops just after a file’s run started and
before it wrote that down, that file fires once more when weft is back. If a
file’s run could not be handed over at all, weft offers it again the next
time that file changes or the trigger starts.
It only works on a local install, since that is where your folders are. On a
cloud install, weft activate refuses it, with a message that starts a folder watch needs the install to run on the machine the folder is on. On WSL, a folder under
/mnt/<drive> is refused too, because Windows tells Linux nothing about the
changes made there: move it under ~/.
Slow callers and messages that cannot be made
Picture a live price page streaming from a Route, and a phone on a bad
connection reading it. Your program writes faster than the phone reads, so
messages pile up for it. Up to 256 wait; past that, the route’s
whenCallerIsSlow decides what gives:
whenCallerIsSlow | What happens when 256 are waiting | Set it when |
|---|---|---|
block (the default) | Your program’s next send waits for room, so a slow caller slows the program down and gets every message | The caller has to read the whole thing: a model’s answer, a conversation |
drop_oldest | The oldest waiting message is thrown away to make room, so the caller always gets the latest | Only the newest value matters: live video, a price feed |
drop_newest | The new message is thrown away, and what was already waiting goes out in order | The messages already waiting matter more than the ones arriving while the caller is behind |
db = PostgresDatabase { database: "shop" }
prices = Route -> (symbol: String) {
path: "prices/{symbol}"
method: "GET"
whenCallerIsSlow: "drop_oldest"
}
latest = PostgresWatchQuery(symbol: String) {
account: db.access
symbol: prices.symbol
query: "SELECT price FROM quote WHERE symbol = $symbol"
channel: "quote_changed"
}
feed = Stream { format: "ndjson", bus: latest.row }
The other setting, whenChunkFails, is for a message your program cannot turn
into the route’s dataType at all: a value that is not a String on a text
route, a file that cannot be read.
whenChunkFails | What happens | Set it when |
|---|---|---|
surface (the default) | The node sending it fails and the caller is told, the way a run that ends before it answers tells them | Every message counts |
drop_chunk | That message is skipped, the skip goes on the run’s record, and the answer goes on | One missing message is fine, like a frame of a live feed |
A Socket has both settings and they mean the same, for every message a
Reply or a Stream sends on it; under surface the socket closes with
1011. A run that fails, or a bus that ends failed, is told to the caller
whatever whenChunkFails says.
If a caller says part of a stream never arrived, the run’s record shows
which. The inspector strikes each lost message through and says why, and
weft events prints them on the caller’s rows as dropped=<offsets> (<why>),
an offset being the message’s place in the conversation. The messages still
waiting when the caller left are named the same way, whatever
whenCallerIsSlow says. A message skipped under drop_chunk shows as
an error row saying a chunk was dropped and the exchange went on, then why.
Which triggers need a public address
Only the ones a provider pushes to.
| The trigger | Needs a public address? |
|---|---|
| A timer or a schedule | No |
| Something weft polls | No |
| A socket weft dials out to | No |
| A form somebody fills in | No |
| A folder on your machine | No |
| A provider pushing events at you | Yes |
A Route or Socket that strangers call | Yes |
./setup.sh --public-url opens one. Go and read
a public address.
What a trigger cannot do
| Why | |
|---|---|
| Be wired from another trigger | A trigger’s inputs are frozen at setup, so another trigger’s output could never reach it |
| Have an infra node downstream | Provisioning happens before any event exists |
| Sit inside a loop | It registers once for the project, not once per item |
All three are compile errors, named trigger-into-trigger,
trigger-into-infra and trigger-in-loop.
Firing one by hand
While you are building, you do not want to expose anything:
weft bake
weft run --fire inbound='{"chatId":"123","text":"hello"}'
weft bake prepares every trigger’s settings and starts no listeners.
--fire then runs exactly one, with an event you typed.
The payload is checked against what that trigger declared it fires with, both ways: a missing field is refused, and so is one you invented.
A route fired this way cannot carry authorization, proxy-authorization,
cookie, weft-instance-token, or a header its auth reads a credential
from (an API key’s X-Api-Key, a signature’s header), nor anything on its
credentials key: a route hands its caller’s credentials to the program as
secrets, which only a real call makes, so the fire is refused, naming the
header. If you need one, call the route for real (weft activate
prints its URL), or leave the header out.
A run you fire this way follows the trigger’s durable and keepRunsFor, and
it is always recorded, even when the trigger has recorded off, so you can
find it in weft runs. For the flags that change how one run is kept, go and read the weft run flags.
For writing a trigger of your own, go and read writing a trigger.
Files and reuse
Three markers pull something in from outside the line you are writing.
| What it brings in | Writes back? | |
|---|---|---|
@include("triage.weft") | Another weft file, as one group | No. The editor opens that file instead |
@file("assets/prompts/triage.md") | A file’s text, as a value | Yes. Editing the field edits the file |
@asset("assets/logo.png", Image) | A file, a URL or a stored file, as a value | No. Pull only |
Where a path is measured from
This catches people, so it is worth stating on its own.
@file and @asset are measured from the project root, wherever you write
them. @file("assets/prompts/triage.md") means the same file in
src/main.weft, in src/billing/charge.weft, and in a file included from
either.
@include is measured from the file that writes it, like an import in any
other language. So @include("../lib/clean.weft") inside
src/billing/charge.weft climbs one level from src/billing/.
Both refuse a path that climbs out of the project, and @include refuses it on
the spelling before it even looks, so a ../ that escapes is refused whether
or not anything is there.
A path starting with __catalog/ is measured from neither place: it is
measured from the project’s nodes/ folder, where the standard library and
your shared nodes live, and it may not climb out of nodes/. If you want to
include nodes/my_pkg/triage.weft, write
@include("__catalog/my_pkg/triage.weft"), and that spelling reaches it from
any file in the project. In @file and @asset, which are already measured
from the root, __catalog/x is nodes/x, so a file under nodes/ can name
its neighbours the same way in all three.
If you put a .weft file under nodes/ for others to include, make its
first line a # comment saying what it does: weft describe-nodes --list
ends with every such file, spelled the way you include it, with that line
beside it.
@asset is the one exception to the rule that a path stays inside the
project: it may name a file anywhere on your machine by an absolute path, or
under ~. For when that works and when it is refused, see
where an @asset is read.
@file
write = LlmInference {
prompt: @file("assets/prompts/image_brief.md")
}
The type is String unless you say otherwise, and a String is taken
verbatim: no parsing, no trimming. That is the common case, which is a prompt
or a document. Give it another type and the text is parsed and checked:
@file("limit.txt", Number).
It goes both ways. The editor shows that field as backed by a file, and
editing it writes the file rather than pasting the text into your .weft. So a
prompt lives in a real file that a person can open, and the graph is still the
place you edit it.
That is also why a @file cannot name a URL or a file type. There would be
nothing to write back to. The error points you at @asset.
The built program does not carry the text. It carries the file’s name and a hash of what the file holds, and the version stores the file, so a 2 MB SQL script costs the program a few bytes, and a file two versions share is stored once. The worker that runs the program reads the file and gives the node the value, as if you had typed it in.
@asset
terms = Text {
value: @asset("https://example.com/terms.txt", String)
}
fal = FalAccess
sharper = FalUpscaleImage {
account: fal.access
image: @asset("assets/logo.png", Image)
}
The type is required, because weft will not guess it from the extension
or from the bytes. For text, it is the type to read the text as
(String, Number, JsonDict, a record). For a file, it names one concrete
kind: Image, Video, Audio or Blob. File and Media are refused:
each covers several kinds, and weft has to know which one the file is.
It is resolved when you build. If it is a URL with a text type, the build
fetches it once and stores the text, which the program names the way it
names a @file’s. If it is a URL with a file
type, the node gets the address, and the file is fetched when the run reads it.
Where an @asset is read
The install builds from a version, so it looks for the file there first. A file the version holds
(anything under assets/, prompts/ and the other folders a
version covers) is already stored with the
version, so nothing is read off your disk or uploaded for it.
If the version does not hold the file (an absolute path, ~/..., or a project
file outside those folders), an install on your machine reads it off your disk
(a relative path from your project folder) and stores it, with a file type and
a text type alike. A cloud install
cannot reach your machine, so it refuses the build:
assets that cannot be used:
@asset("/home/me/logo.png", Image): '/home/me/logo.png' is not in the version, and this install runs in the cloud, so it cannot read files off a machine: put the file in the project, under `assets/`, and name it from there
On an install on your machine, a program building a project that has no
folder can still use an absolute or ~ path, but a relative one has nowhere
to be read from, so that build is refused.
@file in your program can only name a file the version holds, because the
install compiles the version.
In the editor, a file you choose with the file picker is used where it is,
and a file you drag onto a node is copied into your project’s assets/ (with
-1, -2 added if the name is taken).
@include
An included file is one group, and its header is the interface:
# triage.weft
Group(email: JsonDict) -> (severity: String) {
# Classify an inbound ticket
classify = ExecPython(email: JsonDict) -> (severity: String) {
email: self.email
code: "return {'severity': 'high' if 'urgent' in str(email) else 'low'}"
}
self.severity = classify.severity
}
# main.weft
triage = @include("triage.weft")
triage.email = inbox.message
The file has to be exactly one anonymous group, with nothing beside it. No loose nodes, no top-level connections, no top-level includes, not two groups, and not a named one. The group’s header is the file’s interface and the file name is its identity, so anything else would be quietly thrown away, which is why weft checks rather than assumes.
Includes inside that group’s body are fine, and they resolve against that file’s own folder. A file that ends up including itself, through however many hops, is a compile error naming the cycle.
Ten includes, one body
A file is compiled once, however many places include it. Each @include is
a call: the values on its ports go into the one body under a frame naming the
call site, the body runs, and the results come back to that site alone.
Ten includes of the same file are one body and ten frames, the way ten iterations of a loop are one body and ten frames. The two nest freely, so a loop inside an include inside a loop is just a deeper stack.
Only the type check runs once per include. If blur.weft holds
Group[T: Media](file: T) -> (out: T) ([T: Media] limits T to kinds of
Media; without it, T takes any type),
you can include it once fed an Image and once fed a Video, and each out
keeps the type that went in. That holds however deep you go: for a group
nested inside the file, for a node’s type variable, and for another file the
file includes. If img and vid are your two includes of blur.weft, and it
includes a file as inner, img.inner and vid.inner each get their own
types. The nodes and their code stay one copy each.
Whether a node runs once per instance is decided per include too. A file included under a per-instance part and also somewhere shared is per instance only under the first; for what that means, go and read programs with instances.
Naming a node inside one
An included file has no name you write. Its nodes are named through the site:
triage.classify is the node classify of the file that triage includes,
and only that use of it.
That spelling is the same everywhere. It is what weft events prints, what
--node triage.classify filters on, what weft run --group triage runs, and
what the box header shows in the editor. Include the same file twice and the
two read apart, as triage.classify and again.classify.
Cutting a run works inside a call too, so --target triage.classify runs
inside that one call.
Types reach down into an included file
A type declared above an @include is visible inside the file it pulls in, the
same way it is visible inside a group written in place. Declare Settings once
at the top of main.weft, and bot.weft and facts.weft can both use it. It
keeps going down: a file bot.weft includes sees it too.
It only goes down. A type declared inside a group reaches the includes inside that group and nothing beside it, and a type an included file declares stays in that file. Since nothing shadows, an included file can’t declare a name its including file already has.
A file included from several places has to find each type it uses at every one of them. If one place doesn’t have it, the build says so on the file’s own line. When you open an included file on its own, weft reads it with the types of the first place that includes it, so it looks the way the build compiles it.
The including file doesn’t see an included file’s types, but the call site’s ports still type-check, because a named type carries its shape along with its name.
If two files use the same type name with different shapes, that is the
named-type-conflict error. A named type has one body everywhere.
Where each belongs
Put a prompt, a SQL query or a script in assets/ and pull it in with @file,
so somebody can open it, read it and edit it without reading your graph.
Put a picture, a sound or a document in with @asset.
Pull out a group into its own file with @include when it gets big enough to
stop fitting on a screen, or when two places need it. For what that costs at
run time, which is nothing, go and read groups.
What the compiler refuses
Every compiler finding carries a message saying what went wrong, usually with
the fix, and a slug naming the rule. The editor shows the slug beside the
message, and weft validate puts it under code. The slug stays the same when
the wording of a message changes, so look yours up here.
Everything here is checked when you build, except rule-runtime, which is
only checked by weft validate and by the editor just before you Run, Activate
or Resync, so a half-wired sketch whose provider has no model named yet still
builds while you work on it.
A row marked (warning) is reported and the build goes on, unless the row says
otherwise. Every row with no mark is an error, except rule-structural and
rule-runtime, which take whatever severity the node’s own rule sets:
error, warning, info or hint.
Before the rest can run
| Slug | What it found | What to do |
|---|---|---|
parse | The source did not parse | Read the message; it names the span and the syntax that failed |
enrich | A node’s ports could not be worked out: a bad type, or a custom port on a node that does not take them | Fix the port declaration on the node named |
unknown-type (warning) | A node names a type the catalog does not have. A warning in the editor; when you build, in weft validate, and in the editor’s check before you Run, Activate or Resync, the same node is an enrich error | Check the spelling, or add the node under nodes/, or next to the module that uses it under src/. If the node is there but failed to load, or its folder has a metadata.json and no mod.rs, the message says so and why |
parse and enrich are buckets: they carry the underlying message rather than
a slug per cause.
Wires
| Slug | What it found | What to do |
|---|---|---|
unknown-source-node | A wire reads from a node that does not exist, or a metadata port’s value names one | Fix the name |
unknown-target-node | A wire writes to a node that does not exist | Fix the name |
unknown-source-port | A wire reads a port the source does not declare, or a metadata port’s value names a metadata port its member does not have | Fix it; the message lists the real ports |
unknown-target-port | A wire writes a port the target does not declare | Same |
duplicate-input-port | Two wires land on one input | Remove one; the message names the first one’s line |
double-driven-port | An input is both wired and written into the body, or takes a written value and a metadata wire, or two metadata wires | Remove one of the two |
scope-reachability | A wire crosses a group or loop boundary without going through its ports | Route it through the group’s own ports |
deref-path | The dotted path on a wire cannot be read off the source’s type, or, on a metadata wire, the value written does not hold that key | Fix the path, or change the source type or the value |
type-mismatch | The source’s type does not fit the target’s, or a metadata port’s value puts ... before a value that is no list | Change one side, or cast. The message suggests a cast where one exists |
unresolved-typevar | A connected port still has a type variable nothing pinned down | Connect it to something concrete, or declare the type |
type-bound | A type variable limited to some types got another one: a String into a group declared Group[T: Media], or into KeepFile, which takes only a file | Connect a value of a type the limit takes; the message names the limit and what you connected |
cast-not-allowed | A cast node’s two ports are a pair the conversion table does not allow | Pick types the table allows |
graph-cycle | The wires form a cycle | Iterate with a Loop, or exchange feedback over a bus. If what goes back is a value you can write in the program, like a tool’s description, make it a metadata port |
required-port-unmet | A required input has no wire, no written value and no default. On a group, loop or included file: a port nothing outside connects, whose value a node inside cannot run without (a required input, or the last possible member of a @require_one_of set) | Give it one of the three; for a group port, connect or write it from outside |
optional-pass-through-port | A ? on a port of a group or an included file, or on a loop port the loop neither iterates nor carries. Such a port passes what reaches it to the nodes inside, so it is never optional itself (a group input passes through) | Remove the ?; to let a node inside run without the value, write the ? on that node’s input |
Values you wrote
| Slug | What it found | What to do |
|---|---|---|
config-type-mismatch | A written value does not fit the input’s type, including one a metadata wire writes there, or a metadata port’s value does not fit the port’s own type | Fix the value, or the declared type |
input-accepts | The port does not take this kind of driver: a wire on a literal-only port, a written value on a wire-only port | Use the one it takes; the message lists them |
gate-not-boolean | A written _should_flow or _should_not_flow is not a Boolean | Write true or false, or wire it |
literal-out-of-range | A written number is outside the input’s min and max, or off its whole-number step, or a written date on a datetime input is not a moment with its zone (2026-09-03T11:00:00+02:00) | Write a value inside the declared range, or the date with its zone |
literal-not-an-option | A written value is not one of the input’s dropdown options (for a multiselect, one item is not). A dropdown marked freeText never raises it | Write one of the options; the message lists them |
secret-literal | A value written in the program is a secret’s marker. A secret is made by a node while the program runs, so a program cannot write one | Wire the value from a node’s Secret output |
value-on-output | You wrote a value for an output port | Remove it. Outputs are filled by the node, and you read them with node.port |
undeclared-port-no-custom | A key names no input, on a node that does not take added inputs | Fix the key, or use a node with canAddInputPorts |
config-null-literal (warning) | A key is written key: null, which does nothing | Leave the key out and let the default apply |
require-one-of-unknown-port | @require_one_of(...) names a port the node does not have | Fix it; the message lists the real inputs |
require-one-of-unmet | None of the ports in a @require_one_of group is driven | Drive at least one |
placeholder-not-an-input | A query or template you wrote names a placeholder ($userId in a Postgres query, {{name}} in a Format template) that is not an input of the node | Declare the input (PostgresExecuteQuery(userId: String)), or fix the placeholder; the message suggests a close input, lists the inputs you added, or, if you added none, shows how to declare one |
input-not-a-placeholder | You added an input to a node that takes a query or template, and the text never uses it | Use it in the text, or drop the input |
Names and shape
| Slug | What it found | What to do |
|---|---|---|
duplicate-node-id | Two nodes share an id once the graph is flattened | Rename one; the message names the first one’s line |
duplicate-port | A node has two ports of the same name on one side, or a port the program adds to a node has the name of one of its node type’s metadata ports | Rename one |
unpaired-port | On a node that pairs each added input with an output of its name, like Dispatch, one of three things. An added output has no input of its name: add the input, or remove the output. Nothing reads an added output, so what it gets would be dropped: wire it to what handles it. An added input is named after one of the node’s own outputs: rename it | The message says which of the three |
reserved-name | A node or group is named after a node type, so name.port would read as an inline node | Rename it |
two-gates | A node carries both _should_flow and _should_not_flow | Keep the one that says what you mean |
run-undecided | Every input wired into a node is declared with ?, so nothing says when it runs | Gate it with _should_flow, drop the ? from the input it cannot do without, or add @require_one_of |
level-too-large (warning) | One level of the graph holds more than fifteen items | Group the nodes that cooperate on one job. Nesting a group costs nothing at run time |
not-camel-case (warning) | A name you chose in the program is not camelCase | Rename it to camelCase. The message gives the spelling when it can work one out. For which names are checked, go and read Spelling a name |
Ports that come from a list
For a node whose ports are built from entries you write, like a form’s fields or a switch’s cases.
| Slug | What it found | What to do |
|---|---|---|
config-ports-not-a-list | The key holding those entries is not a list | Write it as a list |
config-entry-not-an-object | One entry is not an object | Write it as an object |
unknown-config-entry-kind | An entry’s kind is not one this node offers | Use one the message lists |
config-entry-without-a-port | An entry does not say which port it adds | Add the key naming the port |
unknown-config-entry-key | An entry carries a key that kind does not take | Remove it, or use one the message lists |
config-entry-bad-value | An entry’s value is the wrong shape, or a value the matched input could never hold | Fill in a value of the right shape |
config-entry-missing-value | An entry is missing a value that kind requires | Add the key the message names |
duplicate-catch-all | Two entries match anything, so the second is unreachable | Keep one |
catch-all-not-last | Entries sit after a catch-all, where nothing can reach them | Move the catch-all last |
Metadata ports
For what a metadata port is, go and read metadata ports.
| Slug | What it found | What to do |
|---|---|---|
metadata-port-not-written | A group’s metadata port has no value | Write it in the body (self.tool = { ... }), or take it from another metadata port |
metadata-value-not-literal | A metadata port’s value names a node’s output, which only exists once the program runs | Write the value itself, or take it from another metadata port |
metadata-reference-cycle | Metadata ports take their value from each other, so none has one | Write the value on one of them |
metadata-port-as-target | A wire goes into a metadata port from outside | Wire from it instead (other.input = g.tool) |
metadata-port-read-inside | A node inside a group is wired from that group’s own metadata port (self.tool). Like an output, a metadata port is read from outside its group | Read it from outside the group, or take the value inside from where the port gets it |
metadata-into-output | A metadata port’s value goes into an ordinary output of a group | Make that output a metadata port too (-> (*out: Type)) |
Loops
| Slug | What it found | What to do |
|---|---|---|
loop-unknown-config-field | A key that is not parallel, over, carry, maxIters or trimOnMismatch | Fix the name |
loop-parallel-not-boolean | parallel is not true or false | Write one of them |
loop-max-iters-not-integer | maxIters is not a whole number of zero or more | Write one |
loop-output-unwritten | A loop declares an output nothing inside it writes: a gathered one would be a list of nothing but nulls, a carried one would never change | Write self.<output> inside the loop, or remove the port |
loop-trim-not-boolean | trimOnMismatch is not a boolean | Write true or false |
loop-over-unknown-port | over names an input that does not exist | Fix the name |
over-not-a-list | An over port is neither List[T] nor Generator[T] | Declare it as one |
loop-carry-unknown-port | carry names an output that does not exist | Fix the name |
carry-port-type-mismatch | A carry port’s two sides have different types | Make them the same |
over-and-carry-overlap | A port is in both over and carry | Pick one role |
parallel-with-carry | parallel: true with carry ports. Carrying means each turn feeds the next, which is sequential | Drop the carry, or drop the parallel |
parallel-without-over | parallel: true with nothing to spread out | Give it a non-empty over |
parallel-with-done | A parallel loop writes self.done | Remove it, or make the loop sequential |
loop-unbounded-no-termination | A sequential loop with no over, no maxIters, and no self.done. Nothing would ever stop it | Iterate a list, cap the count, or wire self.done |
gather-output-must-be-nullable | A gather output is not List[T | Null]. An iteration that failed leaves an empty slot | Declare it List[T | Null] |
reserved-port-name | A loop boundary declares index or done, which the language owns | Rename the port |
loop-boundary-unpaired | A LoopIn without its LoopOut, or the reverse | This one is a compiler bug, not yours. Please report it |
loop-config-missing-parallel | The loop’s config did not come through lowering | Also a compiler bug. Please report it |
Streams
| Slug | What it found | What to do |
|---|---|---|
generator-in-container | Generator[T] nested inside a list, dict, record or union | A stream is a port type. Declare it at the top level of the port |
generator-input-must-be-required | A stream input is optional or has a default. A stream has no empty value | Mark it required and drop the default |
generator-multiple-consumers | A stream output feeds more than one input. A stream has one taker | Wire it to exactly one. To broadcast, use a Bus |
generator-through-group | A stream crosses a group or loop boundary | Keep the producer and the consumer in the same scope. A loop takes a stream only through its over |
generator-into-generic-port | A stream is wired into a port whose type is unresolved and not a stream | Declare the target Generator[T], or consume the stream and emit values |
generator-not-iterated | A loop takes a stream that is not in over. A stream cannot broadcast into a body | Add it to over |
generator-not-carriable | A carry port is a stream. A stream is a live edge and cannot cross iterations | Carry a value instead |
over-stream-not-alone | A stream in over beside other over ports | Make it the only one, and zip upstream if you need more |
Triggers, infrastructure and addresses
| Slug | What it found | What to do |
|---|---|---|
run-settings | A trigger’s durable, recorded or outlivesCaller is not true or false, or it is durable or outlives its caller and is not recorded | Write true or false; turn recorded back on, or the other one off |
trigger-into-trigger | A trigger is wired into another trigger | Remove the wire |
trigger-into-infra | An infra node sits downstream of a trigger | Remove the wire |
trigger-in-loop | A trigger inside a loop body | Move it out of the loop |
infra-in-loop | An infra node inside a loop body | Move it out of the loop |
route-path-missing | A node claiming a public address has none, or an empty one | Give it a path, like hooks/stripe or chat/{room} |
route-path-invalid | The path is not a servable pattern | The message says what is wrong |
route-method-unknown | That is not an HTTP method | Fix it |
route-capture-shadows-output | A path captures a name that is also one of the node’s own outputs (cards/{body} on a Route), so the capture could never be read | Rename the capture |
route-output-not-a-capture | An output added to a Route names no capture of its path, or is not typed String | Add {name} to the path or remove the port, and type every capture String |
route-overlap | Two nodes claim addresses one call could reach, and neither is more specific | Make one spell out what the other captures, or change a path or a method |
For why a trigger cannot be wired into another trigger or have an infra node after it, go and read what a trigger cannot do. For why neither a trigger nor an infra node can sit inside a loop, go and read what cannot go in a loop.
Instances
| Slug | What it found | What to do |
|---|---|---|
per-instance-ineligible | @per_instance on a node that runs no container of its own. Only an infra node can carry it | Remove it; for a value each instance gets (its connection included), write @instance_filled on the field |
instance-filled-wired | An @instance_filled field also has a wire into it | Remove the wire, or the marker |
instance-filled-place-types | An @instance_filled field of a node in an included file takes a different type at each place the file is included, and one instance’s value cannot be both | Declare the field’s type on the node |
instance-filled-secret | An @instance_filled field holds a Secret. An instance’s value comes from outside every run, and a secret is made by a node inside one | Remove the marker and wire the field from a node’s Secret output |
instance-filled-boundary | @instance_filled on a group’s or included file’s own port | Write it on the input of the node inside that reads the value |
instance-filled-not-an-input | @instance_filled on a key that is not one of the node’s inputs | Put it on an input |
shared-trigger-reaches-per-instance | A trigger every instance shares (so its runs are for no instance) starts runs that reach something per instance, so every fire would be refused. A route or a socket is not checked: each call can name its instance | Make the trigger per instance too (wire it from what is per instance, or write one of its fields @instance_filled), or keep what is per instance out of what it runs |
For what @per_instance and @instance_filled do, go and read Programs with instances.
Types you named
| Slug | What it found | What to do |
|---|---|---|
named-type-conflict | A named type appears with two different bodies | Make every use of the name carry the same body |
Rules a node declared for itself
| Slug | What it found | What to do |
|---|---|---|
rule-structural | A rule the node’s own author wrote, checked on every build | Whatever its message says |
rule-runtime | The same, but only checked at the moments listed at the top of this page | Fill what the message names. A node with no connection picked is not one of these: the install refuses the run when it starts, and the message gives the command that picks one, weft connect <node> |
A rule whose answer depends on a value that arrives on a wire cannot be decided here, so it is checked again when the node runs, on the value that arrived, and fails the node with the same message. For how, go and read rules on a value that arrives on a wire.
Things that have no slug
Some refusals happen before the compiler gets a graph at all, so there is nothing to look up.
A broken metadata.json fails when the catalog loads, with the path and a
sentence. A fire payload that does not match a trigger’s firesWith is refused
at run time, not at build time. And in the editor, a node whose metadata is
mid-edit is skipped with a warning rather than blanking your graph.
What a node is
A folder with two files in it.
word_count/
metadata.json what it takes and what it gives back
mod.rs what it does
The metadata.json is what makes the folder a node. Put the folder anywhere
under your project’s nodes/ or src/ folder and weft finds it. The two
folders make one catalog, so a name (a node type, a service, a package) has to
be unique across both.
| File | Needed | What it is |
|---|---|---|
metadata.json | Yes | The declared surface: ports, types, settings. Go and read metadata.json |
mod.rs | Yes, to run | The Rust: one impl Node |
deps.toml | No | Extra crates, system packages, build environment |
tests.rs | No | Its own tests. Go and read testing a node |
images/<name>/Dockerfile | Infra nodes only | A container this node needs built |
A folder with metadata and no code
That is not an error. weft finds it, warns that it has no mod.rs yet, and leaves it
out of the build. A program that never names it builds fine, and a program that
does is told which folder is waiting for its Rust:
node type 'WordCount' is not ready yet: node 'WordCount' at <project>/nodes/word_count
has no mod.rs yet; it is left out until it does
So you can write the metadata.json first and the Rust later.
Packages
Drop a package.toml in a folder and it becomes a package. Every subfolder
holding a metadata.json is then one of its nodes, found automatically. There
is no list to maintain.
slack/
package.toml [package] name = "slack", plus shared dependencies
api.rs shared code, reached from a node as `super::api`
metadata.json optional defaults every node here inherits
access/
metadata.json
mod.rs
send_message/
metadata.json
mod.rs
For the shared code, the defaults and the dependencies, go and read packaging.
The trait
#[async_trait]
pub trait Node: NodeManifest + Send + Sync {
async fn run(&self, ctx: RunContext) -> WeftResult<()>;
// everything below has a default
fn node_type(&self) -> &'static str;
async fn provision_infra(&self, ctx: InfraProvisionContext, input: ValueBag) -> WeftResult<InfraSpec>;
async fn setup_trigger(&self, ctx: RunContext) -> WeftResult<()>;
fn tests(&self) -> Vec<NodeTest>;
}
run is the only one you have to write.
provision_infra and setup_trigger have defaults that fail loudly, naming
your node, if your metadata said you implement them and you did not. So
requiresInfra: true without a provision_infra is an error you get told
about when the infra is provisioned, not a silent nothing.
You never check which phase you are in. The engine reads your metadata and
calls the right body. For a plain node that means run is called in every
phase, including while a trigger is being set up, because a value feeding a
trigger’s settings has to be produced then too. For a trigger, run only fires
on a real event, with the payload on ctx.wake.
How it gets into the binary
There is no registration file: the compiler generates the lookup table from the folders it finds.
It only compiles the packages your program uses, each as its own cargo crate. For what that buys you, go and read shared code.
Your program’s definition is not baked into the binary. The worker fetches it by hash at run time, which is why rewiring your graph without touching any Rust reuses the worker already compiled (a program on your machine, an image on a cloud install).
The one macro
#[derive(NodeManifest)]
pub struct TextNode;
That reads the metadata.json sitting beside your file and attaches it to your
type. It reads the package defaults too, so what your code sees at run time is
the same merged document the compiler saw.
It checks at compile time that the file is there, is valid JSON, is an object,
and has a string type. Editing the JSON rebuilds the crate.
The struct’s name is how it finds the file, so TextNode has to live in the
same folder as its metadata. By convention the struct is the node’s type with
Node on the end.
Next, write one.
Your first node
Here is Text, one of the smallest nodes in the catalog, with the lines that
hook up its tests left out. Text takes a string you typed and sends it on.
catalog/basic/text/metadata.json:
{
"type": "Text",
"label": "Text",
"description": "Emit a literal string (a prompt, a label, a config value).",
"tags": ["literal", "string"],
"icon": "Type",
"color": "#64748b",
"inputs": [
{ "name": "value", "type": "String", "widget": { "kind": "textarea" },
"required": true, "label": "Value" }
],
"outputs": [ { "name": "value", "type": "String" } ],
"features": { "pure": true }
}
catalog/basic/text/mod.rs:
//! Text: emit a literal string configured at design time.
use async_trait::async_trait;
use weft::{RunContext, Node, NodeManifest, WeftResult};
use weft::node::NodeOutput;
#[derive(NodeManifest)]
pub struct TextNode;
#[async_trait]
impl Node for TextNode {
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let value: String = ctx.inputs.get("value")?;
ctx.pulse_downstream(NodeOutput::new().set("value", value)).await
}
}
Read an input, send an output. That is the shape of every node.
Write one
Make a folder anywhere under your project’s nodes/ (except
nodes/base_catalog/), or anywhere under src/, for example next to the
.weft file that uses it. nodes/base_catalog/ holds the standard library, and it is a
folder of nodes like any other: weft loads and builds it the same way, and
you can delete it if you do not use it. The one thing particular to it is
that weft catalog update replaces it with the installed weft’s copy, so
anything of yours in there would be lost.
nodes/word_count/
metadata.json
mod.rs
metadata.json:
{
"type": "WordCount",
"label": "Count words",
"description": "Count the words in a piece of text.",
"tags": ["text"],
"inputs": [
{ "name": "text", "type": "String", "required": true, "label": "Text" }
],
"outputs": [
{ "name": "count", "type": "Number" },
{ "name": "longest", "type": "String" }
]
}
mod.rs:
//! Count the words in a piece of text, and find the longest one.
use async_trait::async_trait;
use weft::{RunContext, Node, NodeManifest, WeftResult};
use weft::node::NodeOutput;
#[derive(NodeManifest)]
pub struct WordCountNode;
#[async_trait]
impl Node for WordCountNode {
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let text: String = ctx.inputs.get("text")?;
let words: Vec<&str> = text.split_whitespace().collect();
let longest = words.iter().max_by_key(|w| w.len()).copied().unwrap_or("");
ctx.pulse_downstream(
NodeOutput::new()
.set("count", words.len() as f64)
.set("longest", longest)
).await
}
}
Now use it. The type name from your metadata is the name you write:
words = WordCount { text: draft.answer }
show = Debug { data: words.count }
weft run compiles your node into the project’s worker program, on your
machine with your own Rust, and runs it. If you have not installed Rust yet,
go and read the install page.
Five things that would have bitten you
Do not check your inputs. No asserting that text is really a string, no
unwrapping something you were already promised. Every wire was checked before
anything ran. A wrong value fails before it reaches your code.
Do not use .get(...).unwrap_or(...). That turns a real type error into
your default and you never find out. ctx.inputs.get_or("limit", 10) is the
same shape and fails honestly.
Emit each port at most once. Several calls are fine if they touch different
ports, which is how you release a value early and a done flag at the end.
Touching one port twice is an error naming it. Anything you never mention is
closed for you when the body returns, and that closure is what tells everything
downstream to stop waiting.
Keep values small. A value over 100 KB fails the emit, naming the port. Bytes go in storage, and what travels the wire is the marker saying where they are. If a big answer is not a file, go and read emitting for how to send less of it.
If your code does heavy work, hand it to spawn_blocking. Your body shares
a few threads, one per CPU, with the calls arriving at the worker and with its
other runs, and a thread only changes hands at an .await. Parsing a big
document, resizing an image or hashing a large file in one go keeps the thread
until it is done, and everything waiting for that thread waits with it. On a
worker with one CPU, that is everything. The worker keeps its hold on your run
from a separate thread, so a slow body never loses the run; it only slows the
rest down.
let rows = tokio::task::spawn_blocking(move || parse_csv(&text))
.await
.node_err("parsing the export")?;
Failing well
let body = response.text().await.node_err("reading the reply")?;
.node_err("...") wraps somebody else’s error with what you were doing. On an
Option, the message is what the reader sees when it was None, so say what
was missing.
weft::node_bail!("pick a destination: a channel or a user");
node_bail! is for something your own code worked out. Write the message as an
instruction, because it is going in front of whoever is building the program.
Those two, plus ? on anything the ctx hands back, cover almost every
failure. The exception is a mistake in the program that your code spots, like
SQL that does not parse: return Err(WeftError::Input(..)) for that. For why,
and for which failures a program can catch on _error (an output weft gives
every node), go and read which failures a program can
handle.
Then a test
#[cfg(feature = "node-tests")]
mod tests;
and in your impl Node:
#[cfg(feature = "node-tests")]
fn tests(&self) -> Vec<weft::NodeTest> {
tests::tests()
}
tests.rs:
pub fn tests() -> Vec<NodeTest> {
vec![NodeTest::fake("counts_words", counts_words)]
}
async fn counts_words(rig: FakeRig) -> WeftResult<()> {
let outcome = rig.run(&WordCountNode, json!({ "text": "one two three" })).await.ok()?;
assert_eq!(outcome.outputs["count"], json!(3.0));
assert_eq!(outcome.outputs["longest"], json!("three"));
Ok(())
}
weft test-node WordCount
For the tiers, the rig and what a live test costs, go and read testing a node.
metadata.json
Every node has one, beside its mod.rs. It declares the node’s type, its ports,
and everything the editor and the compiler need to know about it without
reading your Rust.
If you write a key weft does not define, the node fails to load: it is left out
of the catalog, with a warning naming the error, and a program that uses it
fails to compile, saying the node failed to load and why. Places that hold
someone else’s names, like authParams or an input’s default, take any key.
Under accessApps, an entry takes label, client_id and the registration
keys its service declares; for how that is checked, go and read
accessApps. A port’s type is
parsed when the file loads, so a misspelled type (Strng) fails then too.
Three keys are required: type, label and description.
Spelling
The node’s type and any types you declare under types are PascalCase
(WordCount), and a type variable under typeParams is T, or T and a
number (T1). Every key weft defines (requiresInfra) is camelCase, and so
is every port, setting and record field you name. So are the fields under firesWith, and the names under each
event’s fields in a service’s events (for what those are, go and read
Events from a service).
The names of the values a connection stores (imapHost) are camelCase too,
both where a service declares them (the name of each entry in fields, captures and
registrationFields) and where an input asks for them in requiresValues.
Two kinds of key are snake_case: a service’s event topics (drive_changes)
and the keys under accessApps (google, the service’s name). They are
snake_case because a node also writes them as values (a trigger picks a topic
by name, and a node names its service as "service": "google"), and every
value from a fixed list is snake_case. A name somebody else chose keeps their
spelling, like an OAuth parameter under authParams (access_type), or a
field of a request body you send to a provider. Inside an accessApps entry,
client_id keeps OAuth’s spelling, and any other registration key is spelled
exactly as the service names it in its registrationFields.
If you write a key weft renamed in its old snake_case spelling
(requires_infra), the node fails to load, and the error names the new
spelling: “requires_infra is spelled requiresInfra now … Rename it”. In
a package root’s metadata.json (the defaults its members inherit), the error
is reported against the root’s file, and the package’s members are left out.
For how those defaults work, go and read
Package defaults.
If any other name in a metadata.json breaks these rules, the node still loads
and you get a catalog warning. Your project’s own nodes get this check
too. It also covers the field names and port templates a portsFromConfig
spec declares. The check never reads a node’s data, so it says nothing about a
select’s options or an input’s default.
If you have the graph open in the editor, these warnings show up in the graph
view’s problems list as “node spelling warning”. On the command line,
weft validate < src/main.weft and weft describe-nodes --list print them on
stderr, and the JSON from weft validate and from bare weft describe-nodes
lists them under warnings, each as {"kind": "spelling", "message": ...}.
On stderr a warning looks like this, and the JSON message is the same line
without warning: :
warning: <NodeType> (<path to metadata.json>): <message>
If a name is off in a package root’s metadata.json, including a port it gives
its members and that port’s record fields, the warning names the root, not
each member, and starts package '<name>' defaults (<path>) instead of a node
type. A declared type’s fields are checked once, where it is declared.
If you are changing a node that ships with weft, the catalog’s casing test
fails on any such warning. For the command, go and read the Code style section
of CONTRIBUTING.
The top level
| Key | Type | Default | What it does |
|---|---|---|---|
type | String | required | The node’s catalog id. Has to be a valid Rust identifier |
label | String | required | The name shown in the editor |
description | String | required | One line, shown to people and given to the AI builder |
tags | List[String] | [] | Search words for the node picker |
icon | String | none | A lucide icon name |
color | String | none | Hex, a CSS variable, or a palette token |
inputs | List | [] | Everything the node takes, wired data and written settings alike |
outputs | List | [] | The ports it emits on |
metadataPorts | List | [] | Ports whose value you write here, which no run computes. Go and read metadataPorts |
requiresInfra | Boolean | false | The node provisions containers and must implement provision_infra |
images | List[String] | [] | Directories under the package root, each holding a Dockerfile to build |
publishes | String | none | The service whose connection this node hands out itself. Needs an Access[<that service>] output too |
features | Object | all off | Small flags about what the node is |
display | Object | none | What to render on the node’s body after a firing |
validate | List | [] | Rules the compiler checks against the graph |
portsFromConfig | Object | none | Says this node’s ports come from a list in its own config |
types | Map | {} | Named types this node contributes to the project. Go and read custom types |
typeParams | Map | {} | What each type variable of the ports may stand for. Go and read typeParams |
placeholdersAreInputs | Object | none | Which text setting’s placeholders name the node’s inputs. Go and read placeholdersAreInputs |
firesWith | Map | {} | For a trigger: the shape of the event it wakes with |
claimsRoute | Object | none | Which config fields hold the public address this node claims |
accessApps | Map | {} | Public OAuth apps this node ships, by service |
service | Object | none | The connect recipe, on an access node. Go and read declaring a service |
inputs
| Key | Type | Default | What it does |
|---|---|---|---|
name | String | required | The port name, unique among inputs |
type | type string | required | What it takes |
required | Boolean | false | Write it only when true. true: the node cannot run without it, and a program cannot make it optional. Left out: unwired, the node runs without it; wired, it waits for the wire and skips if the wire closes, and a program can change this where it uses the node. For how, go and read optional and required inputs |
accepts | List | both | ["wire"] refuses a written value, ["literal"] refuses an arrow. A Bus or Generator port is wire-only whatever you say |
widget | Object | from the type | The editor control |
default | any | none | What the runtime supplies when nothing drives the input. Never written into source |
label | String | the name | The field’s name on the form |
placeholder | String | none | Hint text in the empty field |
description | String | none | A sentence next to the field |
requiresScopes | List[String] | none | Permissions this consumer needs on the wired connection. Connection inputs only |
requiresValues | List[String] | none | Stored values it needs. Connection inputs only |
keepsBlank | Boolean | false | On an optional input that takes text, a blank text is a value your node uses (a separator, a preview of what arrived). Without it, a blank counts as no value; see below |
A blank text counts as no value
If you take an optional input that takes text (String, or a union with
String in it, like String | List[String]), a blank text on it (empty, or
only spaces) reaches your node as nothing at all, whether it was written in the
program or arrived on a wire. ctx.inputs.opt::<String>("title")? gives
None, a default fills it as it would fill any missing input,
oneOfRequired counts it as not given, and the rules below see it as not set.
So read it plainly, with no filter of your own:
let title = ctx.inputs.opt::<String>("title")?;
This holds for the inputs your metadata.json declares. A port the program
adds to your node keeps whatever arrives on it, a blank included, because it
is the program’s data, and so does a required input. If a
blank is a real value for your node, set "keepsBlank": true on the input; on
a required input or one that does not take text, the node fails to load. A
list is still a value, even an empty one: only a blank text counts as
nothing.
outputs
| Key | Type | Default | What it does |
|---|---|---|---|
name | String | required | The port name |
type | type string | required | What it emits |
description | String | none | A sentence next to the port |
narrowable | true or a type string | none | For an output whose shape only the program knows (a model’s JSON answer, a request’s body, a query’s rows): type it broadly and set this, and a program declares what it holds (-> (response: Verdict)). true lets a program declare any type that fits inside type. A type string lets it declare anything that fits inside that type instead, which may be wider than type. Each value is checked against the declared type at run time, and one that does not fit fails the node, a failure the program can catch on _error. Without it, a program can only restate the output’s own type |
baked | Boolean | false | On an infrastructure node: the value is worked out when the node’s infra is applied, and saved, so a run that reads nothing else from the node uses the saved value instead of running the node. For how, go and read baking |
An output has no required. A firing that emits nothing on a port closes it,
which is what downstream reads.
Do not declare _error: weft gives every node that output for its failures,
and the loader refuses a metadata.json that declares it. For what
comes out on it, go and read which failures a program can
handle.
metadataPorts
A metadata port holds a value written here, by hand, and the node never computes it. A wire a program draws from it is a metadata wire: before the program runs, the compiler writes the value into the input the wire points at. For what that is for, go and read metadata ports.
"metadataPorts": [
{ "name": "tool", "type": "{ name: String }", "description": "What this node is, for a model", "value": { "name": "pick" } }
]
| Key | Type | Default | What it does |
|---|---|---|---|
name | String | required | The port name. It may not be the name of another of the node’s ports: an input, an output, another metadata port, _error, _should_flow or _should_not_flow |
type | type string | required | What the value is. No type variable: nothing a program wires can tell what one would stand for |
description | String | none | What the port holds, for people and for the AI builder; weft describe-nodes shows it |
value | any | required | The value, which has to fit type |
If one of these rules is broken, the node fails to load, and the message names the port.
features
| Key | Type | Default | What it does |
|---|---|---|---|
isTrigger | Boolean | false | This node starts runs from outside instead of running inside one. Weft gives it callsPerMinute, callsAtOnce, durable, recorded, keepRunsFor, holdSecs and pauseAfterSecs inputs (and outlivesCaller too when it sets liveConnection), so do not declare those |
liveConnection | "http" or "websocket" | none | On a trigger whose run answers a caller holding the connection open: "http" for a request waiting on its response (a route), "websocket" for a socket. It also gets callsPerMinutePerCaller, journalEphemeral and journalWindowMs |
journalSettings | Boolean | false | Set it on a node that opens a bus and wants its users to choose how the run’s record keeps it: the node gets the journalEphemeral and journalWindowMs inputs, and reads them with ctx.journal_policy()?. Weft refuses it together with liveConnection (which gives the same two inputs) and refuses a node that declares either input itself. For what they do, go and read two modes |
checksCallers | Boolean | false | On a liveConnection trigger with an auth setting: weft checks every caller against it, with a connection of the node’s own picked on each install (weft connect <node>), before a run starts. Route and Socket set it. For the schemes, go and read checking a signed request |
answersCaller | "whole", "stream" or "end" | none | This node answers its run’s live caller. "whole": the response in one go (a Reply). "stream": the response head, then the body piece by piece (a Stream). "end": the end of the exchange (a Close). Set it on your own node too if it answers through the ctx, so a route’s “never answers its caller” warning counts it |
calledFromOutside | Boolean | false | On a trigger somebody outside calls whose call ends there (a form somebody submits). It also gets callsPerMinutePerCaller. A liveConnection trigger is called from outside already, so it never sets this; a trigger that picks its events up itself (a schedule, a feed, a provider’s push) has no caller and does not either |
pure | Boolean | false | Set it when the node’s body does nothing outside its run that weft does not put on record first: no network, no connection, no wait on a signal (state.wait), no tag, no storage beyond the run’s own files. It may answer its caller (a durable run’s answer leaves only once its record is written), read stored files, store files for this run alone, hand its caller links to them, use a bus to another node of the run, and keep a state (ctx.keep), which goes to the run’s own record. A durable run then starts it without waiting for the database, and may run it again after a crash (for when, go and read surviving a restart). A pure node that reaches outside through the ctx anyway fails at that call, naming this flag; weft cannot see a body that opens a socket or a file on its own, so if you mark one of those pure, a durable run may do what it does twice after a crash. Text, JsonObject, Switch, Route and Reply are pure; HttpRequest and PostgresExecuteQuery are not, because each one reaches outside the run on its own |
machineInputs | {cpu, memory} | none | On an infrastructure node whose users pick its machine size: gives it cpu, memory and machineType inputs, cpu and memory defaulting to the sizes you write. Needs requiresInfra. For how to read them, go and read the spec |
oneOfRequired | List[List[String]] | [] | Each inner list is a group where at least one port must arrive, or the node skips. A blank text on an optional input that takes text counts as not arrived (a blank text counts as no value) |
canAddInputPorts | Boolean | false | Source may declare extra inputs on it |
canAddOutputPorts | Boolean | false | Source may declare extra outputs |
addedPortsAreDataKeys | Boolean | false | Set it when the ports a program adds are named after data from outside weft: a path’s captures, a query’s parameters, a template’s ids, the keys of a JSON object it builds. The compiler’s not-camel-case warning then leaves every port a program adds to the node alone, and still checks the name the program gives the node. The nodes that ship with weft and set it are Route, JsonObject, ElevenLabsRunTemplate, PostgresExecuteQuery, PostgresWatchQuery and PostgresUpdateRows |
join | Boolean | false | Set it when the node meets several branches and goes on with whichever arrived, like FirstInOrder. The inputs a program adds are optional (a closure on one does not skip the node), it runs once everything it waits on arrived or closed, and when nothing arrived it closes its output itself. The compiler’s run-undecided check skips it, so it needs no _should_flow even when every wired input is optional |
customInputType | type string | none | The type every added input takes. Naming one variable makes them all one type |
customOutputType | type string | none | The type every output a program adds takes, when the program does not give it one. If you name a type variable that one of the node’s own inputs uses (T on Dispatch’s value), each added output carries whatever that input received: a router’s branches carry what it routes. Needs canAddOutputPorts |
pairsAddedPorts | Boolean | false | Set it when each input a program adds should come with an output of the same name: the compiler makes that output, so the program does not have to declare it. An added output with no input of its name, an added output nothing reads, and an added input named after one of the node’s own outputs are each the unpaired-port error. Dispatch sets it: route.search = search.tool adds a branch, and route.search is where it sends what that branch claims. Needs canAddInputPorts and canAddOutputPorts |
showDebugPreview | Boolean | false | Render the last output as JSON on the node body |
liveEndpoint | String | none | The declared endpoint the dispatcher proxies /live to |
castPorts | {input, output} | none | Declares the node a checked cast between those two ports |
hidden | Boolean | false | Keep it out of the picker and out of describe-nodes |
display
| Key | Type | What it does |
|---|---|---|
kind | "media" or "link" | media renders by type: images inline, a real player for audio and video, a file card otherwise. link is always a file card with a download button |
input | String | Show this input’s value |
output | String | Show this output’s value |
Exactly one of input and output, never both and never neither.
widget
Every widget, by its kind:
kind | Other keys | What it draws |
|---|---|---|
text | One line of text. The default for String | |
textarea | A wrapping box. The default for anything JSON-shaped | |
code | language | A code box with highlighting. language is one of python, javascript, sql, json, and any other word fails to load. json also fits a structured input (JsonDict, a list), edited as JSON text; the others edit a String |
number | min, max, step | A number box. min and max are real rules the runtime enforces, and a whole-number step ("step": 1) means the input takes whole numbers: a written value is checked when the program compiles, a wired one when it arrives. A fractional step is only the arrow keys’ increment |
checkbox | A tick box. The default for Boolean | |
datetime | A date and time picker. It is a real rule too: the value must be a moment written with its zone (RFC 3339, like 2026-09-03T11:00:00+02:00 or 2026-09-03T09:00:00Z), checked when the program compiles for a written value (literal-out-of-range) and when it arrives for a wired one. Your node reads it as a date, ctx.inputs.get::<chrono::DateTime<chrono::Utc>>("at")?, once it declares chrono = { version = "0.4", features = ["serde"] } in its deps.toml (weft does not hand chrono on) | |
select | options, freeText | A dropdown. options cannot be empty, and they are a real rule: a written, wired or instance-provided value outside them is refused, and so is a default. If the list is only suggestions (model ids, voices, anything the provider adds to), set "freeText": true: the editor still offers the list, lets the user type anything else, and nothing is refused |
multiselect | options, freeText | Multiple choice over a List[String], each item held to options the same way, unless freeText is set |
text_list | Short strings added one at a time. The default for List[String] | |
password | A masked box | |
file_drop | accept, type, multiple | A file picker that writes an @asset(...) reference |
entry_list | Builds the config list this node’s ports come from. Only legal on the input portsFromConfig names | |
access | The connection picker. Write just {"kind": "access"}: the service and whether it is optional are filled in from your service recipe | |
remote_select | access, sources, dependsOn, freeText | Pick a resource on the connected service |
Leave widget out and weft picks one from the type: a file type gets
file_drop, Boolean gets checkbox, Number gets number, List[String]
gets text_list, String gets text, and everything else gets textarea.
remote_select sources
kind | Keys | Where the options come from |
|---|---|---|
granted | from, label, value | The connection row itself, recorded at sign-in. Costs no call |
list | a Lookup, plus requires | Calling the service and enumerating |
picker | script, code, grants, mimeTypes | The provider’s own chooser, opened on a weft page |
from_url | pattern | The person pastes a link and the first capture group is the id |
A Lookup takes get, items, label and value, and optionally page (an
object holding cursorParam and cursorPath), public, where, errorPath
and okPath. get is the URL to
call. items is the dotted path to the array in the answer (data.channels for
{"data": {"channels": [...]}}), or "" when the answer is the array itself.
label and value are dotted paths inside one item: the text the person sees
and the id that gets saved.
If you want the service to do the searching, put {query} in get, and it
becomes what the person typed in the search box. If get has no {query}, the
service never sees the search text, and the list only shows the items whose
label or id contains it, ignoring case. If you want another input’s value
in the URL, list that input in the widget’s dependsOn and write its name in
braces: {team} becomes whatever is set on team. Weft percent-encodes every
value it puts into the path or the query. A value before the path, like a
{base} holding https://..., goes in as it is. If a brace in get is never
closed, the node fails to load and says so.
If the endpoint is https and needs no credential, add "public": true. The list
then works before anyone picks a connection, and the call never carries the
connection’s credential, even after they pick one.
If you only want some of the items the service sends back, add where. Each
key is a dotted path inside one item, and its value is what the item must hold
there. An item is kept only if every path matches, type included: true does
not match "true". An item with nothing at a path is dropped, so null only
matches a field the service sends as null.
ElevenLabs’ models endpoint answers with a bare array of every model, each
carrying flags like can_do_text_to_speech, so a text-to-speech node sets
"items": "" and keeps the ones that can speak:
{ "kind": "list", "get": "https://api.elevenlabs.io/v1/models", "items": "",
"label": "name", "value": "model_id", "where": { "can_do_text_to_speech": true } }
If the lookup also has a page, where filters each page as it arrives: a page where
nothing matches adds nothing to the list, and the editor still shows “More…”
for the next one.
If the service answers a failed lookup with a 200 and says so in the body, like
Slack’s {"ok": false, "error": "missing_scope"}, add errorPath and okPath
so the lookup fails with the service’s own words. For what they read, go and
read refusals answered with a 200.
validate
Each rule is { "when": <condition>, "then": <diagnostic> }.
The diagnostic takes message (with {id}, {port} and {field} filled
in, plus the ones the conditions below name),
level (structural, the default, or runtime), severity (error by default, or warning, info, hint), and
optionally port or field to say what the finding is about.
A structural rule is checked every time the project is parsed or built. Pick
runtime for a rule that only matters once the project runs, like a
provider with no model named yet. The build skips it; for when it is checked
instead, go and read What the compiler refuses.
The conditions are a closed list:
kind | Keys | What it asks |
|---|---|---|
input_satisfied | port | Does this input have a value, from a wire or written in |
input_wired | port | Does it have a wire specifically |
output_wired | port | Does anybody downstream read this output |
value | field, and the checks below | Is this field set to anything non-null and, when checks are given, does it pass every one |
custom_inputs_declared | Did the source add inputs beyond the metadata’s own. {customInputs} in the message names them | |
custom_outputs_declared | Did the source add outputs beyond the metadata’s own. {customOutputs} in the message names them | |
output_type_fits | port, within | Does this output, as the program declared it (or as your metadata types it, if the program declared nothing), fit in the type within. Use it when the types that make sense on a narrowable output depend on a setting: a route whose dataType is text takes only a String body. False when the node has no such output |
custom_inputs_fit | oneOf | Does every input the source added beyond the metadata’s own fit in one of the types oneOf lists. Each input is held to one of them, never to their union, so your node can count on each added input being one of the shapes it handles. Use it when your node sends its added inputs somewhere that takes only some shapes: a model that reads a JSON value or a file lists `[“Json”, “File |
run_reaches | direction, with | Does this node’s run hold a node with a feature. downstream: every run this node starts holds one. upstream: this node is in the run of one |
downstream_of | with | Is a node with a feature wired upstream of this one, so it runs first. {with} in the message names those nodes |
per_instance | Does this node exist once per instance: marked @per_instance, with an @instance_filled field, or reading one of those (a group that receives a per-instance value on any port counts for everything reading its ports). {perInstanceReason} in the message says why, path included: “it reads ‘blender’”, or “it sits inside group ‘work’, which receives ‘blender’” | |
input_names | port, names | Does every name written on this input name a node or a field of this program. If the input holds a String, that is the name; if a list, each String in it; if an object, each of its keys. names says what they name: {"node": {}} is a node spelled the way the program writes it (bridge, one.bridge), narrowed with role (infra or trigger) and perInstance (true or false); {"field": {}} is a node’s field written node.field (send.model), narrowed with instanceFilled (true or false). Nothing written is fine. A wired or @instance_filled input holds names only known at run time, so it is not checked. {names} in the message lists the ones that fail |
all / any | of | Every one, or at least one |
not | of | The opposite |
A value condition takes any of these checks, and all the ones you write must
hold. In each { "min", "max" }, both bounds are optional and both included.
| Check | Holds when |
|---|---|
"nonempty": true | The value is not blank or empty |
"equals": <value> | It equals this exact value |
"oneOf": [...] | It is one of these values, compared as JSON (a string against strings) |
"matches": "<regex>" | It is text and matches |
"count": { "min", "max" } | It holds that many items. A list counts its items, a dict its entries, any other value one |
"range": { "min", "max" } | It is a number in that range |
"length": { "min", "max" } | It is text of that many characters |
"unique": true | The items of the list are all different |
"unique": "<key>" | The records of the list all hold a different <key> |
"each": { <checks> } | Every item of the list passes the checks inside, a single value that is not a list counting as one item. With "key" inside, the checks read that key of each record. An item of the wrong kind for a check passes, since the type check owns kinds, and so does a record without the key. each may hold another each |
{ "kind": "value", "field": "voice", "oneOf": ["alloy", "verse"] }
{ "kind": "value", "field": "schedule", "each": { "range": { "min": 50, "max": 500 } } }
{ "kind": "value", "field": "messages", "each": { "key": "role", "oneOf": ["user", "assistant"] } }
A check meeting the wrong kind of value is false (matches on a number,
range on text), and a field that is absent or null fails every check. So a
rule that refuses a bad value with not around a check refuses a missing one
too. If the field is optional, put the check inside all with a bare value.
Here an optional list that must hold at least two items when it is given:
{ "when": { "kind": "all", "of": [
{ "kind": "value", "field": "options" },
{ "kind": "not", "of": { "kind": "value", "field": "options", "count": { "min": 2 } } } ] },
"then": { "message": "Pick '{id}' needs at least two `options` to choose between.", "field": "options" } }
On a required input, the not alone does it.
with picks nodes by one feature they declare, never by type, so your own
node counts the same as a catalog one: {"answersCaller": true} is any node
that answers its caller, {"answersCaller": ["stream"]} only a Stream-like
one, {"liveConnection": ["http"]} a trigger with an HTTP caller.
Two examples from the catalog. A route warns when nothing in its run answers its caller, and refuses to become per instance, since its public address is one for everybody:
{ "when": { "kind": "not", "of": { "kind": "run_reaches", "direction": "downstream", "with": { "answersCaller": true } } },
"then": { "message": "Route '{id}' starts a run that never answers its caller...", "severity": "warning" } }
{ "when": { "kind": "per_instance" },
"then": { "message": "Route '{id}' would exist once per instance, because {perInstanceReason}..." } }
A node that starts an instance’s copy of an infra node, when it is given an
instance, refuses a name that is no such node, and SetInstanceValues refuses
a key that is no field an instance fills:
{ "when": { "kind": "all", "of": [{ "kind": "input_satisfied", "port": "instance" }, { "kind": "not", "of": { "kind": "input_names", "port": "node", "names": { "node": { "role": "infra", "perInstance": true } } } }] },
"then": { "message": "StartInfra '{id}' names {names} in `node`...", "port": "node" } }
{ "when": { "kind": "not", "of": { "kind": "input_names", "port": "values", "names": { "field": { "instanceFilled": true } } } },
"then": { "message": "SetInstanceValues '{id}' gives {names} in `values`...", "port": "values" } }
Rules on a value that arrives on a wire
If a rule’s answer depends on a value a wire brings, the compiler cannot
decide it. It records the rule on the node in the compiled program, and the
rule is checked again with the values that actually arrived, before your
node’s body runs. If it fires, the node fails with the rule’s own message.
That failure is an input error, so a wired _error does not catch it: the fix
is in the program. A rule asking whether an optional wired input is set
(input_satisfied) is one of these too, because the wire may close empty.
So a rule you write here covers every value, written or wired, and your
node’s Rust never checks it again. A node with no undecided rule pays
nothing. Only rules with severity: error are recorded; a warning the
compiler could not decide is dropped, because it could not fail a firing
anyway.
Which connection a node takes
If your node takes a connection, its input’s type names the service:
"type": "Access[slack]". The compiler then refuses any other service’s
connection on that wire, so a program cannot wire a Mistral connection into
your node and send the Mistral key to your provider. If the node works with
several services, write a union, Access[openai] | Access[anthropic]. If
your node declares its own service, the input it takes that connection on
(account, or connection) is typed with that service, Access[google] on
GoogleAccess, and the catalog refuses the node when it names another one. A
bare Access takes any service, so write it only where the service is not
known until the program is written (a Route’s connection, whose service
follows its auth). When the node opens the connection, weft checks the
service again and fails the node naming both if they differ.
portsFromConfig
For a node whose ports come from a list the author writes, like a form’s fields or a switch’s cases.
| Key | Default | What it does |
|---|---|---|
field | required | The config key holding the list. Must be a declared input |
matchInput | none | The input the entries are matched against, when only one entry wins |
specs | required | The entry kinds you accept, and the ports each one adds |
Each spec takes kind (required), keyField (default "key", the entry
key holding the port name), label, render, fields, catchAll (at most
one, and it goes last), addsInputs and addsOutputs (at least one of them).
A port template is { "nameTemplate": "...", "portType": "..." }, with {key}
replaced by the entry’s key.
A spec field is { "key", "label", "required", "shape", "valueType", "widget" }, where shape is one of typed (needs valueType), value,
value_list, number, element or regex. Every shape but typed needs
matchInput set. You can also write a field as a bare string, such as
"email": that is a typed String field with a text widget, labelled from
its key.
types
A flat map from a name to a type, usable by every node in the project:
"types": { "ChatMessage": "{ role: String, content: String }" }
For how the names are shared and compared, go and read custom types.
typeParams
If a port is typed with a type variable (T, or T and a number), it takes
whatever type is wired to it. If you want to limit what that can be, give the
variable a bound here:
"typeParams": { "T": "File" }
KeepFile does this with "T": "File". Its file input and its file output are
both T, so a Video in gives a Video out. Wire a String into it and the
build stops with type-bound. A bound never picks the type: the compiler works
the variable out from what is connected, then checks the result against the
bound. A bound may name another variable that one of your ports uses
("T": "List[T2]"). The file fails to
load if a variable here is used by no port, or if two bounds name each other
in a cycle.
placeholdersAreInputs
If your node takes a text setting whose placeholders name inputs the program
adds to it (a query’s $userId, a template’s {{name}}), say so here, and
the compiler checks that the placeholders and those inputs match. The node
needs features.canAddInputPorts, or the file fails to load.
"placeholdersAreInputs": { "in": "template", "pattern": "\\{\\{\\s*(?<name>[A-Za-z_][A-Za-z0-9_]*)\\s*\\}\\}" }
in is the input holding the text, and it must hold a String. pattern is
a regular expression finding one placeholder, whose group named name is the
input it stands for. If a match leaves the name group unmatched, weft ignores
it, which is how a pattern skips text that only looks like a placeholder: the
pattern '[^']*'|\$(?<name>[A-Za-z_]\w*) skips '$userId' inside a string literal and finds the bare $userId.
Back-references are allowed (\k<tag>), so a pattern can skip everything up
to the delimiter that matches its opening one, like SQL’s $tag$ ... $tag$.
When the program writes the text itself, in the node’s settings or through
@file (files and reuse),
every placeholder in it must be an input the program added
(placeholder-not-an-input), and every input it added must appear in it
(input-not-a-placeholder). Text that arrives on a wire or through @asset
is only known when the node runs, so your node checks it then: find the
placeholders in the text yourself and pass their names to
ctx.inputs.for_holes (it calls them holes), which matches them to the inputs
and fails on either mismatch. For its arguments, go and read reading
inputs.
firesWith
firesWith says exactly what a trigger’s event carries: a flat map from field
name to type. Only a trigger may have one; on any other node, the file is
refused when it loads. Each type is parsed when the file loads, so a misspelled
type fails then too. A ?
goes on the name, not the type:
"firesWith": { "chatId": "String", "caller?": "JsonDict" }
The payload has to match it exactly. Whenever the trigger fires, whether a
listener sends the event or you fire it by hand with weft run --fire, the
payload is checked before the trigger’s body runs. A payload missing a field
you declared without ? is refused, and so is one carrying a field you never
declared, whether at the top level or inside a field whose type is a record. The refusal
names the field and ends with It wants { ... }, the shape you declared. If a
trigger has no firesWith, its payloads go through unchecked.
claimsRoute
pathField names the config field holding the route pattern, and the optional
methodField names the one holding the HTTP method. Leave the method out, or
leave it empty on the node in your program, and the node claims every method.
If your node puts the path’s captures on outputs a program adds (-> (id: String) on cards/{id}), set capturesAreOutputs: true; the node also needs
features.canAddOutputPorts. The compiler then refuses an added output the
path does not capture, or one not typed String, with
route-output-not-a-capture, and a capture named like one of the node’s own
outputs with route-capture-shadows-output. Filling each port is your node’s
code: emit the capture under the port’s name.
accessApps
If you want people to connect your node’s service without registering an OAuth
application of their own, ship a public one here, keyed by service: when
somebody connects through the own door (the “Your own” page, where people
paste their own app) and fills in none of the app fields, it signs in with this
one. For what the doors are, go and read
How connections work.
Each app needs a label (the name shown in the connection list) and a
client_id, plus every registrationFields entry the service does not mark
optional:
"accessApps": { "google": { "label": "Acme", "client_id": "1234.apps.googleusercontent.com" } }
Every check on an app here runs when the catalog loads, and an app that fails one leaves its node out of the catalog, with a message naming what is wrong:
- the key under
accessAppsnames no service any node of the project declares (no node in this project declares a service named 'slak'); - the service connects without an app, because it does not sign in through OAuth;
labelorclient_idis missing or empty, a required registration field is missing or empty, or the app has a key the service does not declare (the message lists the keys it knows).
An app a package root gives its members is reported once, against the root’s
file, unless it leaves out label or client_id entirely: then every member
that inherits it fails to load, each against its own file. A project’s catalog holds its copy of the standard nodes
(nodes/base_catalog, which weft new copies in), so the standard services
count as declared.
An app here is a public client, one that signs in with no secret (OAuth’s PKCE
flow). A metadata.json ships with the node to everyone who uses it, so a
secret written here would go to all of them: a client_secret is refused, and
so is any registration field the service marks secret (a field is secret
unless the service marks it "secret": false). A service that needs a secret
field in every app cannot ship a public app at all. An app with a secret goes in
the install’s apps file (access-apps.json),
or is pasted on the “Your own” page when somebody connects.
Package defaults
A package root can hold a partial metadata.json whose keys every member
inherits. Members win, key by key, at the top level only, with no deep merge: a
member’s types replaces the package’s rather than adding to it.
Three keys can never be defaults, because they are one node’s identity:
package-level metadata.json must not set `type`: it is one node's identity, not a package default
Keys that no longer exist
If you are reading older node code, the loader names these rather than saying “unknown field”:
| What you wrote | What to do now |
|---|---|
inputs[].exposure | Use accepts. ["wire"] refuses written values, ["literal"] refuses arrows, and leaving it out allows both |
inputs[].requiredWhenWired | Remove it. An input without required already waits for its wire once it is wired, and runs without it when it is not |
outputs[].required | Remove it. An output has no optionality |
features.optionalCustomInputs | Write features.join instead: a node whose added inputs are optional is a join, which goes on with whichever of several branches arrived |
features.isOutputDefault | Remove it. Every node runs when it is reached, and weft run --target <node> narrows a run |
features.catchErrors | Remove it. Every node has the _error output, and a failure goes there when a program wires it |
The ctx
ctx is everything your node’s code can reach: its inputs, and every service
weft provides. The runtime hands it to run, and to a trigger’s
setup_trigger.
Every call here returns WeftResult, and your node never names a weft error
type. Wrap an outside error with .node_err("what you were doing"), and fail
on something you detected yourself with node_bail!. If what you detected is
a mistake in the program (a value of the wrong shape, SQL that does not
parse), fail with input_bail!("...") instead. A wired _error catches what
node_bail! raises but never this: the fix is in the program, so the run
fails.
Five rules that save you a debugging session
Calls on a connection go through ctx.open or ctx.client. That is what
signs the request and records what it cost. ctx.http() is for everything
else. A reqwest client you built yourself skips both.
If your node can pause, keep a state that says where it is. After a pause
the body runs again from its first line, and ctx.keep hands it back the
state it saved: a once whose field is filled runs nothing, a wait whose
field is filled waits for nothing. A port it already sent on sends nothing a
second time. For the whole story, go and read surviving a
restart.
A file in StorageScope::Run goes with its run, when the run’s record is
deleted (its keep time runs out, weft clean, weft rm).
Right for what one run makes, wrong for something a project keeps using: store
that in StorageScope::Project.
ctx.stop_tagged returns when the stop is queued, not when the other runs
are gone. Do not write a node whose next step assumes they stopped.
ctx.register_signal is once per node per setup. A second call is a loud
error rather than a trigger that quietly never fires.
Which firing am I
| Field | Type | What it is |
|---|---|---|
run_id | RunId (a UUID) | This run’s id |
project_id | Uuid | The project it belongs to |
node_id | String | This node’s id in the graph |
node_type | String | Its catalog type, such as ExecPython |
node_label | Option<String> | The title shown on the box, if it has one |
frames | LoopFrames | Which loop iterations this firing sits inside |
instance() | Option<&InstanceId> | Which instance this run is for, when it is for one instance of the program (see programs with instances) |
Reading inputs
ctx.inputs holds this firing’s inputs, whether they arrived on a wire, were
written into the body, or came from a declared default. ctx.wake holds the
event payload, and only on the trigger that actually fired. Both are a
ValueBag with the same methods.
| Call | What you get | It fails when |
|---|---|---|
get::<T>("name") | A required value | It is absent, or it does not fit T |
opt::<T>("name") | Option<T>; absent or null gives None | A value is there and does not fit T |
get_or::<T>("name", default) | The value, or your default | A value is there and does not fit T |
list::<T>("name") | Vec<T>; absent gives [], and one value gives a one-item list | Any item does not fit T |
access("name") | The connection picked on an access input | Nobody picked one and the service requires it |
raw("name") | Option<&Value>, the JSON as it arrived | never |
object() / record() | The whole bag as one record | On a wake bag whose event was missing or not an object |
nested("name") | The object under name, as its own bag | The value is there and is not an object |
iter() | Every named value | never |
custom() | Only the ports this node added in the program, not the type’s own settings | never |
declared() | Only the type’s own declared settings | never |
in_order() | Every input that arrived, in source order | On a wake or nested bag |
declares("name") | Whether the node declares this input at all | never |
for_holes(holes, what, spell, declare) | Values matched to named holes in a text, such as $userId in SQL | A hole names no port, or a wired port the text never reads |
Never write get(...).unwrap_or(...). It turns a real type error into your
default and you will not find out. get_or is the same shape and fails
honestly.
Emitting
| Call | What it does | It fails when |
|---|---|---|
pulse_downstream(output).await | Sends values on | A port is mentioned twice, is not declared, gets a value its type does not allow or one over 100 KB, or is _error |
yield_downstream(output, &mut state).await | The same, but waits until the value was taken. The run can pause there, so it takes your state (surviving a restart) | Same, plus a delivery that can never happen, and an empty output |
close_port("port").await | Closes one output. On a stream port this is the end of the stream | The port was already mentioned this firing, or is _error |
set_max_buffered_items("port", n) | Raises the un-taken cap on a stream port above 4096 | The port is not a stream output, or n is 0 |
fan_declared(&value) | Builds an output by matching an object’s keys to your declared ports, skipping the rest. It never fills _error: a key called _error in the data is data | never |
data_outputs() | Your declared ports minus _error: the ones a node fills from data (a row’s columns, a response’s fields) | never |
output_type("port")? | The resolved type of one output | The node does not declare that output: a type error, so it fails the run even with _error wired |
declared_outputs() / declared_inputs() | Every port this node declares in the program, with its type | never |
is_output_wired("port") | Whether anything downstream reads that output in this run | never |
An ordinary port emits at most once per firing. Whatever you never mention is
closed for you when the body returns. A task you spawned with a clone of ctx
can emit only while the body is running: an emit from it after the body
returned, or after it paused, fails with an error naming
the node and the port. A Generator[T] port emits as often as
you like until you close it.
NodeOutput is the payload you hand those calls:
| Call | What it does |
|---|---|
NodeOutput::new() | An empty one |
.set("port", value) | Sets one port |
.set_serialized("port", &value)? | Sets one port to any Serialize value (a struct, a list of them). Fails naming the port when the value has no JSON form |
.extend_from_object(&value) | Fans every top-level key onto a same-named port. Last write wins |
NodeOutput::stored_file(stored) | The four ports a stored file travels as: file, filename, mimeType, sizeBytes |
.get("port") | Reads back what you already set |
Waiting, and surviving a restart
| Call | What it does | It fails when |
|---|---|---|
keep(initial).await | The state this firing keeps across a pause: initial on its first run, the last one it saved on every run after | Called a second time in one firing, or the saved state no longer reads as your type |
state.once(|s| &mut s.field, async |st| work).await | The value in the field if it is filled, running nothing; otherwise runs work, which gets the state as st (so it can read a bus or a stream with it), stores its result there and saves | work fails, or its result saves as JSON null (a (), a None) |
state.read(|s| &mut s.field, &mut reader).await | The item in the field if it is filled, reading nothing; otherwise reads the next one off a bus reader (a BusEntry) or a stream (its item), stores it there and hands it back, None at the end. Saves nothing by itself: the field goes on record at the next pause, save(), or end of a once or wait (reading across a pause) | The read fails |
state.wait(|s| &mut s.field, signal).await | The answer in the field if it is filled; otherwise waits on the signal (pausing the run when it can), stores the answer there and saves. signal.cancel_with(call) also sends call once if the run is cancelled while it waits | The wait was given up (WeftError::WaitGaveUp), the answer is JSON null or does not read as the field’s type, or the body read a bus or a stream directly before reaching this wait the first time in its run (reading across a pause). A wait somebody skipped ends the firing skipped |
state.save().await | Writes the state now | The write fails |
hold() | A Holding to hand bus reads, a bus’s wait_for, stream pulls and yields instead of your state, for work that cannot be picked up from the middle: while it exists, no wait of your node pauses (its state.waits and helpers’ waits included), and once it is dropped the node can pause again. It saves nothing (holding the run) | never |
register_signal(kind).await | Sets up a trigger that starts a new run on every fire. Called during setup | Called twice for one node in one setup |
after_node(call, &access).await | Puts call on the run’s record, to send through access (None: plain) when this node ends: however it ends with .always(), only on a cancel without it (cleaning up) | The node is pure, or the call’s address is empty or not http(s) |
For how the state carries a node through a pause, and through its worker dying, go and read surviving a restart.
If your node waits on a job a provider runs, go and read waiting on a job a service runs, which also covers the cancel call.
Connections
| Call | What you get | It fails when |
|---|---|---|
open(&access).await | The connection, leased for this firing, with its token refreshed | It needs reconnecting, the service does not match, or the door refuses |
open_within(&access, window).await | The same, with a window you choose instead of the default 15 minutes | Same |
client(access).await | Straight to the signed-in HTTP client. Takes None and gives you a plain one | Same, when it is Some |
http() | The shared plain HTTP client, no credentials | never |
publish_access(values).await | Publishes a connection to something this node runs itself, such as a database it provisioned | A field the service does not declare, or a required one missing |
published_access().await | What this node published before, or None | The broker cannot answer |
endpoint("name").await | A handle on one of this node’s declared infra endpoints, once something answers there | The endpoint is not declared, or the infra is not running |
On an opened connection: client(), credential(), value(name),
identity(), owner(). On an endpoint handle: url(),
host_and_port(), and call(method, path, body), whose path starts with
/.
Putting a value into an address
If an id or a search term goes into a URL, build the URL with client::url,
so that an id holding /, ? or # stays one piece of the address you
meant instead of changing which address you call:
use weft::access::client;
let url = client::url(API, &["files", &id, "copy"])?
.query("q", &search)
.query("pageSize", 50)
.query_opt("pageToken", page_token.as_deref());
let answer = client::get_json(&http, url.as_str(), "copy the file").await?;
Every path segment, and both halves of every query pair, are percent-encoded:
anything but letters, digits and -_.~ (a space becomes %20). The base is
taken as written, with a trailing / dropped, so it has to be a constant of
your node or an address weft handed you, never a value. A segment that is
empty, . or .. fails with an input error, because an address reads even
the encoded %2E%2E as .., so a wired id of .. would climb to another
address; that is why url returns a Result. .segment(s)? adds one more
segment, before the first query pair only (after one, it panics). If a
value’s / must stay, like an object store key (photos/2024/a.jpg),
add it with .key_path(key)?, which encodes each part between the / and
refuses a . or .. part. .query_opt adds the pair only when the value is Some, for an optional
input. Read the result with .as_str(), format!("{url}") or
String::from(url).
For an endpoint, start from its address, client::url(endpoint.url(), &["media", &id])?, or build only the path for call with
client::url("", &["media", &id])?, which gives /media/<id>.
Secrets
If your node hands out a value the run’s record must never show, like a session key a voice app gets for a few minutes, make it a secret. What flows on the wire is a marker naming it. For what the record keeps and for how long, go and read secrets.
let session = ctx.secret(&key)?;
ctx.pulse_downstream(NodeOutput::new().set("session", session)).await
Declare that output as Secret[String] in your metadata.json. A node that
needs the value declares a Secret[String] input and asks for it:
let session: weft::secret::Secret = ctx.inputs.get("session")?;
let key: String = ctx.reveal(&session).await?;
| Call | What you get | It fails when |
|---|---|---|
secret(&value) | A Secret to put on an output typed Secret[T] | The value is not JSON |
reveal::<T>(&secret).await | The value, as T | The record of the run that made it is gone, or the value is not a T |
reveal_input(port, value).await | The value of your input port with every secret its declared type says it holds revealed, for handing a whole value outside weft (a script, a request, a caller) | As reveal_in, naming the input |
reveal_in(value, &ty).await | The same for a value of type ty that is no input of yours | A secret sits where ty declares none, a place ty says is a secret holds a plain value, or the record of the run that made one is gone |
secret_in(value, &ty) | value with every plain value at a place ty says is a secret made one, for a node whose code hands back plain values | never |
A secret this run made, or revealed already, comes back from the worker’s
memory. Any other comes from its sealed copy, which any run of the same
project can read: the same run picked up after a pause, a seeded run, a
marker your program stored somewhere and reads back later. A run with
recorded off writes nothing down, so only that run reveals its secrets.
If a value holds a secret its type does not declare (a token inside a
JsonDict), reveal_in and reveal_input fail rather than hand it on as
its marker, saying where the secret is and that the program should declare
the type, like { token: Secret[String] }. If you emit a revealed value on a
plain output, the record holds it like any other value. For which types fit a secret, go and read what fits into
what.
Sharing something between runs
If your node builds something on every run that is slow to build, let the worker’s runs share it instead. There are two calls for that:
- if each run needs one to itself for a while (a database connection a run
may hold a transaction on, a process that runs one script at a time), take
one from a pool with
ctx.pool; - if runs can all use the same one at once (a client, a cache, a pool library
you already use), ask for it with
ctx.shared.
One each, from a pool
use std::time::Duration;
let mut conn = ctx
.pool(&access, |opened| async move { connect(&opened).await })
.max(20)
.idle_close(Duration::from_secs(5))
.check(|c| !c.is_closed())
.reset(|c| async move { clear_session(&c).await.map(|()| c) })
.take()
.await?;
You write only how to make one (connect here is your own function), and
weft does the rest. take hands you the newest idle one that is still fresh
and passes your check, letting go of every one that fails on the way, and
calls your closure to make a new one when none is left. What comes back is a
weft::shared::pool::Lease<T>: it derefs to the item, and goes back to the
pool when you drop it.
The first argument is the key, the same as for ctx.shared:
a connection (your closure gets it opened, and the pool is built again once
the connection’s values change) or a name of your own (your closure gets
()).
| Setting | Set it when | What it does |
|---|---|---|
.max(n) | The other end only serves so many at once | At most n items out at once on this worker. A take past that waits for one to come back, and the wait ends with the cancelled error if the run is cancelled. 0 is refused. Without it, nobody waits |
.idle_close(d) | The other end closes what sits idle, or should not be held idle | Lets an item go once it sat idle for d, and never hands out one idle for longer, even when the timer that closes it ran late. Without it, an idle item is kept for as long as the pool is |
.check(|c| ..) | An idle item can die on its own (a closed connection, a process that exited) | Says whether an idle item is still good. One that is not is let go, and the next is tried |
.reset(|c| async move { .. }) | A run can leave something on the item (a session setting, an open transaction) | Runs when a lease is dropped, before the item goes back. Hand the item back if it is fit for the next run, or an error to let it go (the error goes to the worker’s log). The item keeps its place under max while this runs |
Every ask of one pool has to give the same max and the same idle_close.
An ask that gives others fails with the pool '<key>' is asked for with max .. and idle close .., and it is held with max .. and idle close ..: every ask of one pool says the same, so keep the settings in one function your nodes
call, the way the Postgres nodes do.
If an item is no good for the next run (a process that stopped answering, a
connection whose state your run cannot vouch for), call lease.discard()
and weft lets it go instead of giving it back. weft does the same on its own
for a lease dropped while its run is cancelled, since whatever the run was
doing may still be under way on it. A make that fails takes no place under
max.
One for everybody
let client = ctx.shared(&access, |opened| async move { Client::new(&opened) }).await?;
The closure is async and its result is a WeftResult of the thing (Client
here is your own type, whose new returns a WeftResult<Client>).
The first argument is the key weft keeps the thing under. If you pass a
connection (&access), the closure gets the opened connection, and once one
of its values changes (a new password), the next run that asks gets your
thing built again from the new values. To notice the change, weft opens the
connection every time a run calls ctx.shared (or ctx.pool). For a
connection a person made or an infra node published, the worker answers
that open from memory, so a reused thing costs no extra call. A connection
set to Use ours (weft’s own key) costs one network call to weft each
time.
If you pass a name of your own instead, the closure gets ():
let python = ctx.shared("exec_python", |()| async { Ok(Interpreters::new()) }).await?;
On one worker, every node that asks under the same name for the same Rust
type gets the same thing, so pick a name that is your node’s own, the way
ExecPython uses "exec_python". A worker serves one project, so nothing
another project builds can collide with yours.
ctx.shared hands back a weft::shared::SharedHandle<T>, which derefs to the
thing. If two runs ask at the same moment, it is built only once. A build that
fails keeps nothing, so the next run tries again.
If the thing can only serve so many runs at once, say so with with_limit:
let client = ctx.shared(&access, |opened| async move { Client::new(&opened) }).with_limit(100).await?;
Then at most that many handles to it exist at once on one worker, and the
next run that calls ctx.shared waits until one is dropped. A run that asks again while it still
holds a handle can end up waiting on itself once the limit is reached, so
keep one handle per run. Every call for one shared thing has to pass the
same limit: a call that passes another, or none, fails, naming both. The limit counts the handles to one thing on one worker: two workers can go
past it between them, and so can the old thing and the new one for a moment
after a connection’s values change, while runs still hold handles to the old
one.
If a task you spawn needs the thing after your run returns, move the handle
into the task: it keeps its place under with_limit until the task drops
it. If your task should not count against the limit, take handle.arc()
instead: the Arc keeps the value alive but holds no place under
with_limit, and weft can let go of the thing while you hold it, so the next
run may build a new one beside yours.
How long weft keeps them
By default, weft lets go of a shared thing (a pool is one too) about five
minutes after the last run asked for it or dropped its handle, once no
handle is left. Your thing’s Drop runs once weft has let go of it and the
last handle and Arc are gone, which is when a pool closes its connections.
When the worker stops, it lets go of every shared thing as soon as its last
run has ended, used recently or not, so a pool’s connections go back to the
database while the worker is still writing its last records.
If you want a quiet stretch not to cost a rebuild, raise the five minutes
with weft workers set --shared-idle-seconds <n> (or lower it to let go
sooner). It is one setting for the whole project, covering every
shared thing on its workers. The change applies from the next run: weft starts workers with the
new value (a new revision on Cloud Run), and the old ones finish what they
are running and stop.
When the other end only serves so many
max and with_limit only cap what one worker takes, so that its runs
never leave its sibling workers without a connection. They cannot keep a
database under its connection limit, because they cannot see how many
workers there are. If your node talks to something that only serves so much
(a database with a fixed number of connections, an API with a rate limit),
assume any number of workers run your node at once: Cloud Run starts copies
up to the project’s --max-instances (set with weft workers set), and a
change of program or worker settings runs the old copies beside the new
ones for a while. Two things follow:
- Hold nothing you are not using. Close an idle connection within
seconds, because one a copy keeps between requests is one no other copy can
have. On Cloud Run a copy has no CPU between requests (unless the
project turned on
weft workers set --cpu-always-allocated true), so a timer that closes it may not fire for minutes, which is whyidle_closealso refuses to hand out an item idle for longer. The Postgres nodes use anidle_closeof 5 seconds and amaxof 20. - Let the other end do the counting. It is the one place that sees every
copy, so that is where a client past the limit should wait its turn, or be
refused with a clear answer. Your node then waits, or fails with a message
that says why. A PostgresDatabase does this with a pooler in front of
Postgres, where a client that finds every connection busy waits for one,
and Postgres closes a session that sat idle for a minute. If your node
keeps a connection idle on purpose (one that
LISTENs), lift that limit for its own session when it connects, the way PostgresWatchQuery sendsSELECT set_config('idle_session_timeout', '0', false) FROM pg_settings WHERE name = 'idle_session_timeout'(theFROM pg_settingspart makes it do nothing on a Postgres that has no such setting, instead of failing).
Storage
ctx.storage(scope) gives you a handle. The scope decides where new files go,
how long they live, and what list sees; get, delete, keep and the links
act on whatever scope the key itself belongs to. Any run of any project in your
account can read any stored file it holds a reference to, for as long as the
file exists.
For every scope, go and read Storage.
| Call | What it does |
|---|---|
identified(id) | Names what the next put is a copy of, so storing it twice stores it once |
link_lifetime(duration) | Makes every link to the files this handle stores live that long instead of 15 minutes (at most seven days). The value carries it as linkSecs |
put(bytes, mime, filename, keep) | Stores bytes and gives back the stored-file value |
put_stream(stream, mime, filename, keep) | The same without holding the file in memory |
put_response(resp, what, mime, filename, keep) | Streams an HTTP response you already made into storage |
put_from_url(url, mime, filename, keep) | Fetches a URL straight into storage, as mime when you pass one |
copy(&file, keep) | Copies a file into this handle’s scope, leaving the original |
edit(&file, |old| ...) | Changes the file’s content in place: reads it, runs your function on the bytes, writes back what it returns. Another write of the same file is never lost (it reads again and reruns the function), so your function must use nothing but the bytes it is given. Same key, name, type and lifetime, new size and version |
replace(&file, bytes) | Overwrites the file’s content in place, whatever it holds now |
get(&file) | The file’s bytes, as a stream |
get_range(&file, range) | Part of the file |
get_bytes(&file) | The whole file in memory. Only for small ones |
delete(&file) | Deletes it. Stored files only, not URL-backed ones |
list() | Everything under this scope |
keep(&file, ttl) | Sets the file’s deadline from now on (Never takes it away). It does not move the file, so a run file still goes with its run |
internal_link(&file) | A FileLink { url, public } for code inside the install (a script your node runs), straight to storage |
external_link(&file) | A FileLink { url, public } for anything outside the install. public: false means only this machine (or its network) opens it, so an internet service needs the bytes instead |
external_file(&file) | The external link when it is public, else a data: URL of the bytes, with the file’s mime type and filename |
upload_slot(max_bytes, accepts, minutes, keep) | A weft-upload:... string a page’s visitor uploads one file through, with uploadFile from @weft/connect |
claim_upload(slot, &ty), claim_uploads(&[(slot, &ty)]) | The file uploaded through each slot, received once into a place of type ty. Several slots are received together or not at all, which is what a Route does for the file fields of one call |
externalize(&value, &ty, policy) | Turns every file inside a typed value into a link or inline bytes, for handing out |
internalize(&value, &ty, keep) | The reverse: pulls URLs and inline data into stored files |
keep takes KeepTtl::Default (30 days, pushed back each time the file is
read), Secs { secs }, or Never (no deadline), in every scope but Asset. For what it
means in each scope, go and read how long a file lives.
What comes out of externalize is no longer a value of that type, because the
links expire. Hand it to whoever asked and do not store it.
Streams and buses
| Call | What it does | It fails when |
|---|---|---|
journal_policy() | What your node’s users set on journalEphemeral and journalWindowMs, as the journal option of a bus. Only a node with features.journalSettings (or a live-connection trigger) has them; for any other node it is the default | A setting is out of range or the wrong shape |
create_bus("port", opts) | Makes a bus whose marker goes out on that output, and gives you a handle and the marker to emit. A body run again after a pause gets the same bus back. The bus ends when your body does | port is not one of your Bus outputs, the options do not make sense, or opts.journal has a window outside 100 to 60000 ms |
bus(&marker) | Turns a marker value back into a live handle | The bus is gone, or that is not a marker |
open_bus("port", opts, "name").await | The whole producer move: make it, emit the marker, register your name. The bus ends when your body does | Emitting or registering fails |
pending_bus("port", opts, "name") | A bus that opens only on your first open().await, so the work that can fail runs first and its failure can still reach _error | never; open() fails as open_bus does |
join_bus("port", "name") | The consumer twin: registers your name, and the bus ends when your body does | The input is not a live bus |
bus_from_input("port") | A handle whose bus does not end with your body. For an observer | Same |
On the bus handle:
| Call | What it does | It fails when |
|---|---|---|
cursor()?, cursor_from_start()?, cursor_at(offset)? | Your reader on the bus: from now (from your own join, once you registered), from the oldest message the bus still holds, or from that offset. A body run again after a pause gets its reader back where it stopped | Your body already opened a reader on this bus (one reader per bus) |
wait_for(name, &mut state).await | Waits until that name has joined the bus (the run can pause there). A body run again after a pause gets past it at once: the list of who joined comes back with the bus | The bus closes first |
reader.next(&mut state).await, next_json(kind, &mut state), next_message(..), next_bytes(..) | The next entry, or the next message of that kind, waiting (the run can pause there). None once the bus ends | The payload is the other shape (next_json on a bytes bus), the read breaks the rule of reading across a pause, or the bus came back from a pause without messages this reader had not read yet |
entry.as_json() | On an entry a reader handed you: a JSON message’s kind and value, or None for a join, a leave or a bytes message | never |
For how a bus ends and what a reader does about it, go and read streams and buses in Rust.
Read a stream input like any other value: ctx.inputs.get::<Generator<Row>>("rows")?.
| Call | What it does |
|---|---|
next(&mut state).await | The next item, waiting (the run can pause there). None once, at a clean end |
try_next(&state) | Item, Empty or Finished, without waiting |
drain(&mut state).await | Everything, waiting for the end (the run can pause there). A failed stream errors instead of returning a partial list |
end() | Whether the producer finished or failed, or None while it is open |
Live callers
| Call | What you get |
|---|---|
is_api_call() / is_websocket() | Whether a caller of that kind is attached |
caller_data_type() | Whether this connection speaks json, text or bytes (the trigger’s dataType), or None when nobody is attached |
caller() | The caller as a handle, or None |
http_caller().await | The HTTP caller, attached and connected, or a loud error |
ws_caller().await | The same for a WebSocket |
live_caller().await | Whichever one is connected |
caller_request() | What the caller sent to open the exchange, without waiting for the connection |
Logging, tags, cancelling
| Call | What it does | It fails when |
|---|---|---|
log(level, message).await | Writes a line into the run’s log. Trace, Debug, Info, Warn, Error | The write fails |
tag_run(["user_7"]).await | Tags this run. Additive and safe to repeat. Any string works: one that is not already a tag is turned into one (see below) | The list is empty, or a tag is empty |
stop_tagged("user_7", StopSelf::Keep).await | Queues a stop for every live run of this project carrying that tag, turned into a tag the same way | The tag is empty |
is_cancelled() | Whether this run was cancelled. Cheap enough to poll | never |
cancellation() | The flag itself, to end your own work with the run (below) | never |
StopSelf::Keep only reaches runs that took the tag before this one did, so
two runs racing to stop each other leave the later one alive. StopSelf::Include
ends this run too.
When the run is cancelled
You do not have to watch for a cancel around weft’s own calls. When the run is cancelled, each of these ends at once with the cancelled error, even in the middle of waiting on a slow server:
- a request sent through a client from
ctx.http(),ctx.client(..)or an opened connection, and the read of its answer’s body; - an endpoint handle’s
call; - a provider socket’s
sendandrecv; - a storage read, and a write of bytes or a stream (the calls listed in
Storage;
put_from_urlandcopyare not among them); - a bus reader’s read.
Pass that error up with ? and the node ends cancelled instead of failed.
If your node waits on something of its own (a child process, a timer, a
library with its own client), wrap it in race, which hands back the work’s
own output, or the cancelled error the moment the run is cancelled:
let status = ctx.cancellation().race(child.wait()).await?.node_err("wait for the script")?;
The work is dropped where it stands, so a child process you spawned with
kill_on_drop(true) is killed with it.
If you turn a request’s error into a weft error yourself, check
weft::cancellation::is_cancel(&e) first: .node_err makes any outside error
a failure, a cancel included. client::send(req, what) keeps a cancel a
cancel and hands back any status, for a node whose answer is the status
itself. checked_send does the same and also refuses a non-success status,
quoting the provider, and get_json and post_json add the parse.
client::body_error(what, e) does it for a failed read of a body.
Your project, and the owner’s other projects
A node can do from code what a person does with the CLI: build and run a project, write its files, turn its triggers on, start its infra, read its runs. The runtime carries each call out on behalf of this run, with the same parameters and the same refusals the CLI gets. Each call is numbered within the firing; for what a body that runs again gets back from one, go and read the ctx helpers that wait.
The calls below act on this run’s own project. If you want the same call on
another project of the same owner, put ctx.project(id) in front of it, id
being the project’s id (the one in its weft.toml):
ctx.project(shop_id).infra("bridge").start(&mut state).await. A project of another
owner is answered as if it did not exist. ctx.projects().list() lists the
owner’s projects. If you want a new, empty one, call
ctx.projects().declare("shop") and take its id from the answer (the id comes from the call’s number in the firing, so if the worker dies
after the declare and before the node’s next save, the body run again finds
the project the first call made; every other time the body runs again, the
declare is a new call and makes a new project, so put it in a state.once or
keep the id in your state), then write its
files: at least a weft.toml naming it and carrying that id, and
nodes/base_catalog/ too if it uses the standard library, since a build only
sees the files the project holds. If you want to copy files the owner
already holds, such as the standard library of this run’s own project, read
their hashes with state() and name each one with edit().blob(path, hash):
nothing is downloaded or stored again, so the copy is one read and one write
however many files it holds.
let shop = ctx.projects().declare("shop").await?;
let shop_id: uuid::Uuid = shop.id.parse().node_err("read the new project's id")?;
let weft_toml = format!("[package]\nname = \"shop\"\nid = \"{shop_id}\"\n");
let source = "# your program here\n";
let own = ctx.code().state().await?;
let mut edit = ctx.project(shop_id).code().edit()
.text("weft.toml", weft_toml)
.text("src/main.weft", source);
for (path, hash) in own.working.iter().filter(|(path, _)| path.starts_with("nodes/base_catalog/")) {
edit = edit.blob(path.clone(), hash)?;
}
edit.apply().await?;
Three things a program cannot do to its own project, each refused with the reason:
| Refused on its own project | Why |
|---|---|
Starting a run (ctx.code().run()...start(&mut state)) | Every run of it would start the next, with nothing to end the chain. It is refused before anything is recorded as a version or built. Start it from another project, or reach the part you want with a trigger |
resync | The resync waits for the runs on the old version to end, and this run is one of them, so it would wait on itself |
remove | The removal cancels every run of the project, this one among them, before it finishes |
If you want to hand out a token, mint an instance token with
tokens().mint_for_instance, which hands the value to the run alone. Only a
person can mint a signal token or a frontend’s new token, because its value
would sit in the journal.
The project’s infra, triggers, instances and settings
Most of these take an optional .instance(id), which picks that instance’s
copy instead of the shared one; values(), connections() and
tokens().instance(..) always name an instance.
| Call | What it does |
|---|---|
infra("bridge").start(&mut state).await | Brings the shared copy up. Returns once it runs, and the run can pause between looks; fails with the reason if it does not come up, or if somebody stops it while this waits |
infra("bridge").request_start(&mut state).await | Asks for the copy to come up and returns once weft has accepted, without waiting for it to run. If a route should answer its caller at once (a 202), call this and let a later status() say whether the copy came up; until then it reads provisioning |
infra("bridge").stop(take_down, stop_self).await | Stops the copy, keeping its disks (a unit whose onStop keeps it running stays up until terminate). It returns once the stop is under way, and status() reads stopping until the copy is down. Answers an InfraDownAnswer: TakenDown; AlreadyDown when the copy was already stopped or stopping; or NoCopy (a copy never started, a mistyped instance id, or a copy terminated earlier) |
infra("bridge").terminate(take_down, stop_self).await | Deletes the copy and its disks, except the ones the node lists in keep_on_terminate. Answers as stop does, except that AlreadyDown means it was already being terminated |
infra("bridge").instance(id).wipe(take_down, stop_self).await | Deletes the instance’s copy and every one of its disks, the kept ones too, even if the copy was already terminated. Always answers TakenDown |
infra("bridge").upgrade(take_down, stop_self).await | Moves a running copy onto the last build weft made (a stop, then a start; nothing is built). A copy already on that build, stopped or missing is left as it is. It returns once the upgrade is under way; its command_ids hold one command per owner (none when nothing was upgraded), and all_infra().follow_command(id, &mut state) waits for one to end |
infra("bridge").status().await | The copy’s state, None when there is none. A start or stop on its way reads provisioning or stopping at once, the same answer weft status gives |
infra("bridge").copies().await | Every copy: the shared one and each instance’s |
infra("bridge").logs(tail), .show(), .press(kind, payload), .rebake(&mut state) | weft infra logs, show, press and rebake for that node |
all_infra().cancel(instance), .look(), .doors(), .logs(tail), .command(id) | What reaches every copy at once: weft infra cancel, look, list-doors and logs, and where an upgrade’s command stands |
trigger("receive").activate(held).await | Turns one trigger on if it is off. While it was down it may have held work that arrived: ReleaseHeld::Run runs that work, at half the trigger’s own limits so new arrivals still get through, and ReleaseHeld::Drop deletes it and cancels the runs paused on it |
triggers().deactivate(take_down, stop_self).await | Turns off every shared trigger of the program, or an instance’s triggers with .instance(id) (chain it again for each further instance), or every instance’s with .all_instances(), which you cannot combine with .instance(..). If you want only some triggers, name them with .only([..]). It returns once they are down, without waiting for the runs they started, even with Running::Wait |
triggers().bake(), .resync(take_down, &mut state), .cancel_activate() | weft bake, weft resync and weft cancel-activate. A resync returns once it has ended, and the run can pause between looks |
status(), cancel_build(), remove(&mut state) | weft status, weft cancel-build and weft rm |
values().instance(id).get().await | The instance’s values for the program’s @instance_filled fields, by node and field |
values().instance(id).set(node, field, value).clear(node, field).apply().await | Sets and clears values in one change, all or none, each checked against its field and its node’s rules first. Any live trigger of the instance that reads one is set up again, and apply returns those triggers’ names |
values().instance(id).forget().await | Forgets every value the instance was given |
connections().instance(id).list() / forget() | Lists an instance’s connections, or forgets all of them (the values naming them go too) |
instances().list().await | Every instance weft holds anything for, one entry each: how many values, connections and live tokens, its infra copies, and its triggers, each with the events waiting on a field not yet filled and why |
costs().instance(id).service(s).since(t).list().await | The cost records, each saying whose credential paid, narrowed by instance, node, service, run, paid_by or since (unix seconds). A run’s costs are deleted whenever the run is, whether its keep time ran out or something cleaned it (for how long runs are kept, go and read how long a run is kept) |
runs().instance(id).status(s)?.older_than(d).clean(running, stop_self).await | Deletes the matching runs and answers how many it deleted, cancelled and left running. If a run is still going, StillGoing::Leave leaves it alone and StillGoing::Cancel cancels it (its rows go at the next clean). This run is never deleted. If you pass StopSelf::Include and this run matches the filters, it is stopped once the rest is cleaned, and the call does not return |
runs()...list(limit).await / .count().await | The newest limit matching runs (1 to 200) with how many match in all, or just the count; same filters as clean |
tokens().mint_for_instance(id, expires_in).await | An instance token, value shown once. Made again after a crash, from a state saved before it, it mints a fresh value for the same token, and the value handed out first stops working |
tokens().instance(id).revoke().await | Revokes an instance’s tokens |
tokens().list(), .revoke(id) | weft token ls and weft token revoke |
frontends(), workers(), picks() | weft frontend, weft workers and the picks weft connect makes |
take_down is a TakeDownChoice and stop_self a StopSelf. For what each
one can say, with an example, go and read taking something down from a
run.
If you want status(), cancel_build(), get(), tree(),
running_source(), prune(), working_copies(), forget_working_copy() or set_run_example() on
this run’s own project, go through ctx.own_project() (for example
ctx.own_project().status()), because ctx has no short form for them.
One run, by its id
ctx.run_by_id(run) reaches any run of the owner’s, this one included
(ctx.run_id), and a run whose project was removed since.
| Call | What it does |
|---|---|
stop() | weft stop. Called on this run, it stops it: the call does not return |
wake(node) / wake_in(node, &[3]) | weft wake: resolves a time wait now, and refuses one that expects a value |
logs(lines), events()...read() | weft logs and weft events |
outputs(&[..]), detail() | Everything the run produced, as weft freeze reads it, and the run with the waits it is parked on |
node_state(node) | Where one node of the run stands: pending, running, waiting (with the last answer its wait got), paused (no wait of its own: its run paused while it was inside it, and it carries on when the run does), completed, failed (with the error), skipped or cancelled. Name the node the way weft run --from takes it: render, work.render inside a group, one.render inside the file the site one includes. A node inside a loop is refused (name the loop), except in a run of part of that loop’s body, where it ran once |
delete() | weft clean <run>. A run still going is refused, and so is this one |
ctx.resolve_run("3f2a") finds the one run whose id starts that way, the way
weft logs 3f2a does.
A project’s code
ctx.project(id).code() reads and writes a project’s files and builds and runs
them, the way you do from a folder. A program writes into a working copy (for
what that is, go and read your folder is a
copy). If you name none, it
writes into the project’s most recently changed folder: on an install on that
person’s machine the files land on their disk right away, and on a cloud
install at their next command that syncs. When the project has no
folder, it writes into the program copy. If you want a particular working
copy, ctx.project(id).working_copies() lists them, each with its id, and
ctx.project(id).code().working_copy(copy) acts on that one. If a prune is
refused because of a copy’s head, move it with
.code().working_copy(copy).switch_to(version) to a version outside that
history, or call ctx.project(id).forget_working_copy(copy), which records
the copy’s files as a version (named in saved, None when it held no file)
and removes the copy, as weft forget-copy does. A folder that still exists
registers again at its next command that syncs it.
let shop = ctx.project(shop_id).code();
shop.write_text("prompts/tone.txt", "Answer in two sentences.").await?;
let started = shop.run().example("angry-customer").start(&mut state).await?;
ctx.log(LogLevel::Info, format!("run {} on version {}", started.run_id, started.version)).await?;
| Call | What it does |
|---|---|
state() | The working copy, and every file it holds by path and hash |
read_text(path) / file(path) | One file’s text, or the file as a stored file to read with ctx.storage(..); None when there is none |
write_text(path, text) / remove(path) | Writes or removes one file. Text is capped at 4 MiB; a larger or binary file goes through edit().stored(path, key) |
edit().text(..).stored(..).blob(..)?.remove(..).apply() | Several files changed together, all or none. .blob(path, hash) puts a file the owner already holds (a hash from state() or a version’s files) at path without downloading it; a hash that is no sha256 fails the blob call, and one the install does not hold refuses the write. .expecting(Some(hash)) after a change (None: no file there) lands it only while the file still holds what you read, and answers Applied::Stale naming the paths otherwise, so two writers never overwrite each other unseen |
checkpoint(Some(label)) | weft checkpoint |
switch_to(version) / switch_to_run(run) | weft branch, given a full version or run id. What the copy held is recorded as a version first, named in saved |
build().start(&mut state) | weft build, returning once the build registered. Between looks at it the run can pause on a timer |
run()...start(&mut state) | weft run: .example(name) is the example weft run <name> takes, .seed_run(run) is --seed=<run>, and each other setter is the flag of the same name (.from(..), .target(..), .seed(), .instance(..)?, .durable()…). It builds the code, checking the run against the compiled program before any image is built, so a run that could not start is refused before you wait on images. It returns once the run has started |
If you want a project’s version tree, ctx.project(id).tree() answers what
weft tree --json prints, without workingCopy (the CLI adds that to say
which folder it ran in). If you want what the project
runs, .running_source() answers the version of its build and that build’s
files by path and hash.
.prune(version, frozen, plan_only) is weft prune. In frozen, pass the
frozenFrom.version of every examples/*.json in the project (read each with
read_text): the prune is refused if it would delete a version a frozen
example came from, or an ancestor of one, and an origin you leave out has no
such protection. Set plan_only to see what it would remove without removing
anything. If you froze a run as an example yourself,
.set_run_example(run, Some(name)) records it the way weft freeze does, and
None forgets it.
What is not here
ContextHandle, the trait underneath, is the seam the runtime implements. Your
node only ever sees the RunContext wrappers on this page.
Reading inputs, emitting outputs
What the methods are is on the ctx. This page is what happens around them: how a value gets into your bag, and what the runtime does with what you send.
Where an input’s value comes from
Wired data and design-time settings live in one bag, under the port’s name. Three things can fill a port, in this order:
| Rank | Source |
|---|---|
| 1 | A value that arrived on a wire |
| 2 | A value written into the node’s body, if nothing arrived |
| 3 | The default from the node’s metadata, if the port is still empty |
Four rules sit around that.
A closed wire kills the default. If a wire closes without carrying a value (weft calls that a closure: upstream finished without sending on that port, or failed), the default does not fill in, because that would hide that something did not happen.
A delivered null is a value only where the type allows it. On a port
whose type admits Null, null is data. Anywhere else, a null arriving on a
wire fails the run, unless the program marked that input optional (x?: T
where it uses your node): then it reads as nothing arrived, and the default
fills.
A wrong type fails the run, required or not, and the program cannot catch it (which failures a program can handle). It is never dropped so the default can take over.
A number outside its number widget’s min and max fails the run too
(widget), and so does a fraction where the widget’s
step is whole: a poll interval of 0 on a port with "min": 1 fails rather
than becoming the port’s default. So does a value that is not one of a
select widget’s options, unless the widget sets freeText.
An input your metadata leaves optional can be empty when your body runs: nobody
wired it and it has no default, or the program wrote ? on it and its wire
closed. Read such an
input with opt or get_or (reading inputs). Whether
your node runs at all then is up to the program. For how a program decides, go
and read optional and required
inputs.
Two ports get special handling before your code sees them. A connection input
arrives as a full Access value carrying the service, rather than the id the
editor stored. And the gate ports are stripped out entirely, so your node never
finds a _should_flow it did not declare.
A trigger is different again: on a real firing, its ports replay the values that were frozen when the trigger was set up, and written literals fill whatever is left.
Emitting
ctx.pulse_downstream is the only way out.
Once per port, per firing. Emitting or closing the same port twice is an error naming it:
node 'reply' touched port 'text' twice in one firing. Each output port can be
emitted or closed AT MOST ONCE per firing
If your node emits, pauses on a wait, and its body runs again from the top when the answer comes, the emit made again does not count as a second one: it does nothing, and the first value stands. For why the body runs again, go and read surviving a restart.
Several calls are fine if they touch different ports. That is how you
release a bus marker early and a done flag at the end. If any port in one
call collides, nothing in that call is recorded, so a failed call is a clean
no-op rather than half an emission.
A stream port is the exception. A Generator[T] output takes as many
emissions as you like, one per item, until you close it. After the close,
nothing can follow.
Anything you never mention is closed for you when the body returns. That closure is the signal everything downstream is waiting for. A node that decides it has nothing to say does not need to do anything special: it just returns, and the branch behind it skips.
A node that fails closes whatever it had not already sent. What it did send stays sent. weft cannot unsend a message your node already put on Slack.
An undeclared port is caught first, before the once-only rule, so a typo reads as the real problem:
node 'reply' tried to emit on undeclared output port 'txet'. Declare it in
metadata.json's outputs list, or correct the port name in the node body.
A value that does not fit its port’s type fails the call, naming the port,
the type and what did not fit. The type is the one in your metadata, after the program filled in any type
variable. If your metadata marks the output narrowable
(outputs), it is the type the program declared on it,
like -> (response: Verdict). If the type is an open record, extra keys are
dropped instead of failing the call (records that take extra
keys). A refused call sends nothing and leaves the port open, so if your
code handles the Err instead of returning it with ?, it can emit on that
port again with something else.
A value over 100 KB fails the call, naming the port. The weight is the value written as JSON, checked on what you emit, so a node one hop later cannot trim it for you. There are two ways out, and the refusal names both: put the data in storage and send the stored file, whose value weighs a few hundred bytes whatever the file holds, or send less. Sending less means sending one page of it along with where the next page starts, or cutting it short and saying so in another output.
If you want to bound a value yourself, weft::storage measures it the way the
wire does, against MAX_WIRE_VALUE_BYTES:
| You want | Call |
|---|---|
| What a value weighs on a wire | wire_weight(&value) |
The longest start of a text that weighs at most budget, never splitting a character | cut_to_weight(text, budget) |
A text cut to max_chars characters, then to what one port carries, and whether anything was cut | cap_text(text, max_chars) |
FetchPage uses cap_text this way: a maxChars input, and a truncated
output that tells the program the page was cut.
Waiting for the value to be taken
pulse_downstream sends and carries on. yield_downstream waits until the
value was actually taken, which is what you want when you are producing faster
than the consumer reads.
It fails rather than waiting forever when delivery becomes impossible, for instance because the consumer skipped or the run finished. An unwired port counts as delivered immediately.
Building the output
NodeOutput::new()
.set("count", words.len() as f64)
.set("longest", longest)
set takes anything that turns into JSON. If you already have a Value, it
passes through rather than being wrapped again.
For a dynamic object whose keys should land on same-named ports,
ctx.fan_declared(&value) matches them against the ports your node actually
declares and skips the rest. That matters after a paid call: an extra key in a
provider’s response should not fail your node when you already spent the money.
NodeOutput::stored_file(stored) fills the four ports a stored file travels
as: file, filename, mimeType and sizeBytes.
Errors
| You want | Write |
|---|---|
| To wrap somebody else’s error | result.node_err("reading the reply")? |
To say what was missing when an Option was None | option.node_err("the response had no body")? |
| To fail on something you worked out | weft::node_bail!("pick a channel or a user") |
| The same, as an expression | node_error(format!("...")) |
Those, plus ? on anything the ctx returns, cover every failure except a
mistake in the program itself, which you return as Err(WeftError::Input(..))
(see which failures a program can handle).
Write the message as an instruction to whoever is building the program, because that is who reads it. “pick a destination: a channel or a user” beats “invalid configuration”.
Which failures a program can handle
Every node has an _error output, which weft adds when it loads the node. If
a program wires it, your node’s failure comes out there and the node counts as
done; if nobody wired it, the failure fails the run. For what that looks like
from the program, go and read when a node
fails. Your body writes
no handling for it: it returns its errors with ? as usual.
_error is weft’s, so a body that emits on it or closes it fails. error,
without the underscore, is yours to use like any other name.
What your node does decide is which kind of failure it returns. A failure of
the work itself can be caught on _error: anything from node_err or
node_bail!, a value your node emits that does not fit its port (like a
model’s answer in the wrong shape), and a wait that gave up (go and read when
the run cannot pause). A
mistake in the program (a bad setting, or a value of the wrong type arriving
on an input) always fails the run, even when the program wired _error, because the fix is
in the program. If your node asks ctx.output_type about a port it does not declare,
that fails the run too, and _error never sees it.
So when your node spots a mistake in the program itself (SQL that does not parse or names a table that is not there, an
input value of the wrong shape, an empty field it needs), fail with
Err(WeftError::Input(..)). If a program could sensibly retry or route around
the failure while it runs, use node_bail! instead. If your node talks to a
server, decide the same way for each kind of refusal: the Postgres nodes fail
the run on a syntax error or a missing table, and put a broken constraint on
_error.
If your node produces a stream, its failure stops reaching _error once
something went out on a Bus or Generator output. So do the work that can
fail before you send the first piece. For a bus, ctx.pending_bus waits to
open it until you have the first piece (go and read streams and buses in
Rust).
In a test, rig.wire_output("_error") wires _error the way a program would,
and outcome.output("_error") reads what came out. Without it the rig wires
nothing, so the same node fails loudly.
Custom types
Give a shape a name in a node’s metadata.json, and every node in the project
can use it.
"types": {
"ChatMessage": "{ role: String, content: String }",
"ChatHistory": "List[ChatMessage]"
}
Then in the ports of any node in the project, including nodes from other packages:
"inputs": [ { "name": "messages", "type": "ChatHistory", "required": true } ]
A named type is only equal to itself. Declare "CustomerId": "String" and a
port that wants a CustomerId refuses a plain String unless the program puts
a Cast in between, while a CustomerId goes into a port that wants a
String with nothing in between. For how names compare and what fits into what, go and read
types.
One table for the whole project
Every metadata.json in the project adds its types to one table before any
port is read, and the program’s own .weft files can use these names too. If
several nodes in a package share a type, declare it once in the package root’s
metadata.json.
Two files may declare the same name with the same shape: the two declarations
count as one. If two metadata.json files declare it with different shapes,
the nodes of both are left out of the catalog, each with an error saying the
name is declared twice with different bodies.
A type written in a .weft file is stricter: it may never reuse a name the
program can already see, including the names metadata.json files declare,
even to declare the same shape.
Records that take extra keys
A record refuses any key it does not declare. If your node’s type describes
data that comes from outside and may carry more than you read (a provider’s
answer, a webhook’s body), end it with ..:
"types": { "Charge": "{ id: String, amount: Number, .. }" }
Such a record accepts a value with extra keys. weft drops them when your node
emits the value and when a Cast produces it, so what flows on carries only
id and amount. A missing field or
a field of the wrong type still fails.
What travels
The name carries its shape with it. A port typed ChatMessage is stored as
ChatMessage={role: String, content: String}, so anything reading a saved
project or a journal row understands it with no table to consult, and a run
that started yesterday keeps the shape it started with even if you edit the
declaration today.
If you write that stored form into a port’s type yourself and its shape
differs from the declaration ("type": "ChatMessage={role: String}"), the
build fails with named-type-conflict.
Making your node a cast
If your node converts its input into whatever type the program gives its
output, the way Cast does, name the two ports here:
"features": { "castPorts": { "input": "value", "output": "value" } }
Where the node is used, the pair of types its two ports end up with has to be
a conversion Cast knows (for the table of those, go and read
Cast), and any other pair is the cast-not-allowed error when the program compiles:
no cast from JsonDict into Number; a cast parses text, stringifies data, or
validates an object shape against a declared type
Your node does the conversion itself when it runs, the way Cast does:
let target = ctx.output_type("value")?;
let cast = target
.cast_value(&value)
.node_err(format!("casting the value into {target}"))?;
Showing something in the graph
Every value on every wire is already in the inspector, so your node never has to send anything just so a person can see it.
Two things go further, and they share a word, so keep them apart.
A file, rendered on the node
Name one port in your metadata and the editor renders its value on the box after each firing.
"display": { "kind": "media", "output": "image" }
kind | What you get |
|---|---|
media | The file rendered by its own type: a picture inline, a real player for audio and video, and the file card for anything that will not play |
link | Always the file card, with its details and a download button |
Exactly one of input and output, naming the port. The value has to be a
file.
Use link for something somebody downloads and media for something they
look at or listen to.
A live panel
This is the other one, and it is a different thing: a panel that updates while something runs, rather than a value from a firing.
Two kinds of thing produce one, and only one of them is your job.
An infrastructure node’s container. Your container serves /live, and you
opt in by naming the endpoint:
"features": { "liveEndpoint": "api" }
Declaring it is the opt-in. There is no separate flag. The dispatcher proxies the panel to your endpoint, and a node that names none answers 404.
A trigger’s signal kind. Written by weft, inside the listener, not by you. A node that declares a trigger writes no display code at all, and those panels are read-only: the listener refuses to serve one carrying a button rather than show a button that cannot work.
What a panel holds
A list of items, each with a label, and at most one action.
type | data | Drawn as |
|---|---|---|
text | a string | A box you can copy from |
image | a string an <img src> takes, so a URL or a data: URI | The picture |
progress | a number from 0 to 1 | A bar |
secret | a string | Masked behind dots until somebody clicks the eye. Copy gives the real value either way, and weft infra env writes it into an env file without anyone seeing it |
An answer that is not a list of items comes back to the reader as an error naming what you sent, rather than an empty panel. One item it cannot read becomes a line saying so, in its place, and the rest of the panel still draws.
A number outside 0 to 1 on a progress bar is shown as given rather than clamped, because clamping would hide the bug in whatever produced it.
Who can see it
Nobody, by default. A signal token reads no display at all unless it was minted
with --display <node> naming that node, or --displays for all of them.
That is on purpose. A panel can be a credential: a bridge showing a QR code to scan, a database showing the password it just minted. So the scope grants nothing until you say which node.
Storage
A value on a wire is capped at 100 KB. Anything bigger goes into storage, and what travels the wire is a small marker saying where it is.
let storage = ctx.storage(StorageScope::Run);
let stored = storage.put(bytes, "image/png", "chart.png", None).await?;
ctx.pulse_downstream(NodeOutput::stored_file(StoredFile::from_value(&stored)?)).await
The five scopes
Any run of any project in your account can read any stored file, whatever its scope, for as long as the file exists. The scope decides only where a new file goes and how long it lives.
| Scope | Holds | Gone when |
|---|---|---|
Run (the default) | What this run made | Its run is deleted, that is when the run’s record goes (its keep time runs out, weft clean, weft rm). The run ending does not delete it |
Project | Files of this project | The project is deleted (weft rm) |
Shared { name } | Files several projects meet in by naming the same space | Somebody deletes them (weft files rm) |
Asset | The project’s @asset copies | No version of the project names it any more, after a countdown. Your node reads it, and the worker refuses writes to it |
Instance { of } | One instance’s files in this project (StorageScope::instance() for the run’s own instance, instance_of(id) for any) | Somebody deletes them (weft files rm, or wiping the instance with WipeInstance), or the project is removed. For more, go and read an instance’s files |
The scope governs writes and lists. Reading, deleting, keeping and linking
act on whatever scope the key itself belongs to, so a later node, or a later
run that kept the reference, can get a file without knowing where it came
from. copy is the one that does both, reading from the key’s scope and
writing into yours, which is how a file moves to a scope that lives longer.
How long a file lives
A file can also have a deadline. It is deleted when the deadline arrives, whatever its scope, or with its owner (its run, its project) if that comes first. Every read, link or replace pushes the deadline back, so a file something still uses does not vanish underneath it. A file with no deadline lives as long as its scope says.
storage.put(bytes, "image/png", "chart.png", Some(KeepTtl::Default)).await?;
KeepTtl | Meaning |
|---|---|
Default | 30 days since it was last touched |
Secs { secs } | That long since it was last touched |
Never | No deadline: it lives as long as its scope says |
None gives no deadline either. A cache of fetched pages that should go once
nobody has read it for a week, say:
ctx.storage(StorageScope::Project)
.put(page, "text/html", "page.html", Some(KeepTtl::Secs { secs: 7 * 24 * 3600 }))
.await?;
For a file that already exists, keep(&file, ttl) sets its deadline from now
on, and keep(&file, KeepTtl::Never) takes it away. Neither moves the file, so
a run file still goes with its run. If you want a file to outlive the run that
made it, copy it into the project (the KeepFile node with scope: project
does that). An asset lives as long as a version of the project still names it
(once none does, it expires on its own), so you cannot give one a deadline.
Storing
| Call | Use it when |
|---|---|
put(bytes, mime, filename, keep) | You have the bytes |
put_stream(stream, mime, filename, keep) | You do not want the whole file in memory |
put_response(resp, what, mime, filename, keep) | You already made an authenticated request and want its body |
put_json(&value, filename, keep) | A value too big for a wire (a transcript, a conversation), stored as application/json |
put_from_url(url, mime, filename, keep) | A plain URL, fetched straight in (mime None takes the type the server says) |
copy(&file, keep) | A file that already exists, into this scope |
Every one of these makes a new file under a fresh key, so two puts never collide, whatever their filenames.
If you know a download’s type better than its server does (a provider that
serves every file as application/octet-stream), pass it as put_from_url’s
mime and it wins over what the server says. With no filename, the name
comes from the response, else from the URL. If your node names a file after a
URL itself, storage::filename_from_url(url) gives the last part of its path
(talk.mp4 in https://cdn.example.com/v/talk.mp4?sig=1), or download.bin
when the path names nothing, and storage::name_in_url(url) gives None
there instead, so you can pick your own fallback.
If the run is cancelled, put, put_stream, put_json and put_response
end with the cancelled error on their own and nothing is stored. So do the
reads below (get, get_range, get_bytes, body), chunk by chunk (go and read
cancelling).
Changing a file in place
If your node keeps something that grows with use (a conversation, a log, a document it keeps adding to), keep it in a stored file and change that file. A port carries at most 100 KB, and a growing value will pass that. Have the node take the file in and pass the file on, and do not also offer the same content as a plain value, so a program has only one thing to wire.
To change what the file holds, use edit. It reads the content, hands it to
your function, and writes back what you return:
let updated = storage.edit(&file, |old| {
let mut lines = old.to_vec();
lines.extend_from_slice(b"one more line\n");
Ok(lines)
}).await?;
If two writers change the same file at once (two runs on one project file, the
iterations of a parallel loop), neither loses the other’s change. If the file
changed between the read and the write, edit reads it again and runs your
function again on the new content. Because it can run more than once, the
function must use only the bytes it is given: no counter it bumps, no call
out, no other read. Returning the bytes unchanged writes nothing.
The file keeps its key, so every reference already handed out now reads the
new content. It also keeps its scope, name, type and deadline. A reader sees
either the old bytes or the new ones, never a mix. What comes back is the file’s value
with its new sizeBytes and version.
Every edit is recorded on the run, and the node’s panel in the graph lists it
under “Files edited”: the file, the versions it moved between (v4 → v5), and
the lines that changed (a diff longer than 100 KB is cut short at a line
boundary). A file that is not text shows its old and new size.
If you want to overwrite the file whatever it holds now, without reading it,
use replace(&file, bytes). It shows under “Files edited” too, with the new
size instead of the changed lines.
An asset cannot be edited or replaced, and neither can a file value that points at a URL.
Versions
Every stored file value carries a version: 1 when the file is made, going up
every time the content is written. A key always names the same file,
so a key plus a version always means the same bytes.
If you seed a run that would hand a node’s saved file to a node that runs again, and that file has changed since, the run is refused; for what to do then, go and read when a reused node’s file changed.
Storing the same thing twice
let storage = ctx.storage(StorageScope::Project).identified("slack:F123456");
identified names what the file is a copy of. Put it twice and it stores
once, and with put_from_url a source you already have costs no request at
all.
Name the source, like <service>:<id>, not the content.
A file from a page’s visitor
If a visitor of your site needs to send the program a big file (a video, say),
give the page an upload slot. upload_slot(max_bytes, accepts, minutes, keep)
(or the UploadSlot node) answers a plain string, weft-upload:..., that is
safe to hand to the page. The page sends the file straight to storage with
uploadFile(slot, file) from @weft/connect, in parts that resume after a
dropped connection. Then it calls a Route with the slot in a field typed as a
file ({ "video": "weft-upload:..." }), and the Route receives the stored
file. accepts is a weft type (Video, Image | Video, File), checked
against the file’s first bytes. The file lands in the handle’s scope, a slot
takes one file and is received once, and an upload nobody received within
minutes goes.
Reading
| Call | Gives you |
|---|---|
get(&file) | The bytes, as a stream |
get_range(&file, range) | Part of them |
get_bytes(&file) | The whole thing in memory. Small files only |
body(&file) | A request body that streams the file out of storage (see below) |
list() | Everything in this scope |
delete(&file) | Gone. Stored files only, not URL-backed ones. In a program, the DeleteFile node does the same |
Sending a stored file to a provider
If you are uploading a stored file to a service, send it as a streamed body
instead of reading it into memory with get_bytes:
use weft::access::client::{Multipart, StoredBodyExt};
let body = ctx.storage(StorageScope::Run).body(&file).await?;
http.post(url).header("content-type", body.mime_type()).stored_body(body)
body gives you a StoredBody: the bytes, read out of storage while the
request is sent, with the file’s record beside them. len(), mime_type()
and filename() come from that record, so you know the length before a
single byte is read, which is what a service that hands out an upload address
for an exact size needs. stored_body sets Content-Length from it; the
content type is yours to state.
If the service wants a multipart form, a stored file can fill a part:
let form = Multipart::form_data().text("channel", &channel).stored("file", body);
http.post(url).multipart(form)
stored(name, body) is a named file field of a form_data() form, under the
file’s stored name and type. For a multipart/related body (a JSON part, then
the bytes), use Multipart::related().part("application/json", serde_json::to_vec(&meta).node_err("encode the metadata")?).stored_part(body).
multipart sends the form with its Content-Type and Content-Length,
streaming every stored part. Keep get_bytes for when your node needs the
bytes themselves: text it parses, a hash it computes.
Handing a file out
A file value carries no link. If you want one, ask for it, and pick the call by who will open it:
| Call | Use it when |
|---|---|
internal_link(&file) | Code inside the install opens it: a script or a process your node runs on the worker. The link goes straight to storage, never out through a tunnel and back |
container_link(&file) | One of your project’s own containers opens it: an infra unit your node hands the file to. Inside a container 127.0.0.1 is the container itself, so on a local install this link reaches your machine at host.docker.internal, through weft’s internal port. On a cloud install it is the same link as internal_link |
external_link(&file) | Anything outside the install opens it: a browser, an app, a provider’s API. This is the link a Reply puts in its answer in place of a file |
external_file(&file) | You are sending the file to an outside service: its external link when the open internet can open it, else a data: URL carrying the bytes, with the file’s mime type and filename beside it |
Every link comes back as a FileLink { url, public }, where public says
whether the open internet can open url. An external link goes the first of
these ways that exists:
- On a cloud install, the cloud store’s own link. Public.
- Through the project’s visitor address, when the project has a site whose server forwards requests to weft. Public.
- Through weft’s tunnel, when it is open. Public.
- The install’s own address, which only this machine opens (and the devices
on its network, if the install has a network address set).
publicisfalse.
Your node does not pick between these, and neither does the program: the person running the install sets them up. If you want to know how, go and read files at run time and a public address.
If you are handing a file to a provider’s API (an image to edit, a video to
dub) and the link says public: false, the provider cannot fetch it, so send
the bytes in your request instead. external_file makes that choice for you.
Unless you set a lifetime, a link lives at least 15 minutes from when it was asked for. If you want the links to a file you store to live longer (a video somebody will watch for an hour), or shorter, set it on the handle:
ctx.storage(StorageScope::Project)
.link_lifetime(Duration::from_secs(2 * 3600))?
.put(bytes, "video/mp4", "clip.mp4", None)
.await?;
The value that comes back carries linkSecs, and every link any node makes
from that value lives at least that long. For a file a program already has, the
LinkOptions node sets it. A link lives at most seven days: a longer lifetime
is refused right where you set it. Handing a link out also keeps the file
stored at least as long as the link works.
Every link is a download: opening it in a tab saves the file under its name,
while an <img>, <video> or <audio> tag still shows or plays it. A link
answers range requests, so a video seeks and a broken download picks up where
it stopped, with nothing for your node or a page to add.
If a page asks for the same file again, it gets the same link, so its video keeps playing: weft cuts time into windows as long as the link’s lifetime (quarter hours, for a 15 minute link) and hands out one link per file per window. Each link works until its window ends plus one lifetime, so a 15 minute link works for between 15 and 30 minutes from when it was asked for. For a lifetime over three and a half days the window is seven days minus that lifetime, so a seven day link is a new one every time it is asked for.
Whole values full of files
A chat history with three images in it is a typed value with three file markers buried inside it. Two calls handle that.
externalize(&value, &ty, policy) walks every file slot the type names, at any
depth, and turns each into a link or inline bytes depending on what the
consumer takes. A provider that accepts image URLs but only inline audio gets
exactly that.
internalize(&value, &ty, keep) is the reverse: it takes a value full of
data: URLs and external links and stores them, giving you back something you
can emit and that will still work tomorrow.
If you want some other form for each file going out (a dict a script reads,
say), storage::media::map_stored_files(&value, &ty, |slot| async move { .. })
is the walk externalize and a live caller’s links go through: your function
gets each stored file the type places (its handle, its kind and its marker)
once, and returns what goes there instead. The type decides what is a file,
so a JsonDict that happens to hold a marker is left whole. A data: URL or
an http(s) link in a file place passes untouched, and anything else there
is refused.
The rule that makes it safe: the marker never leaves weft, and a file value
never holds a link. So nothing you emit, park on or memoize carries one, and
neither does the journal. What externalize gives you is no longer a value of
that type, and its links expire, so hand it to whoever asked and do not store
it.
What a stored file looks like
It is a marker naming its kind, and the kind comes from the mime type when it was stored.
| Weft type | For |
|---|---|
Image | image/* |
Video | video/* |
Audio | audio/* |
Blob | Everything else: a PDF, a zip |
Inside is the key, the mime type, the size, the filename, the version, and
linkSecs when somebody set a link lifetime. No URL, which
is deliberate: a link expires and a marker does not, so the marker is what goes
on wires and into the journal.
The marker is a one-key object, and the key names the kind: __weft_image__,
__weft_video__, __weft_audio__ or __weft_blob__. A stored PNG looks like
this on a wire:
{
"__weft_image__": {
"key": "acme/project/3f2a9c1e-7b4d-4e0a-9c55-2d1f0e8a6b17/5b0c2e44-1a9f-4c3e-8d2b-6f7a9e0c1d23",
"mimeType": "image/png",
"sizeBytes": 48213,
"filename": "chart.png"
}
}
File is shorthand for all four and Media is shorthand for the first three.
Both are fine on a port. Neither can be the type on an @asset, because a
value carries exactly one marker and those leave it open.
NodeOutput::stored_file(stored) fills the four ports a file travels as at
once: file, filename, mimeType and sizeBytes.
Surviving a restart
A node can wait a week for somebody to answer and cost nothing while it waits, because its run lets go of its worker. If your node waits, this page shows you how to write it so it picks up where it stopped.
It does not pick up inside your await. When the answer comes, a worker runs
your node’s run function (its body) again from its first line, and the
only thing that tells it where it got to is a small state it saved:
use serde::{Deserialize, Serialize};
use serde_json::Value;
use weft::signal::{Timer, TimerSpec};
use weft::{NodeOutput, RunContext, WeftResult};
#[derive(Default, Serialize, Deserialize)]
struct Reminder {
sent: Option<Value>,
woke: Option<Value>,
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let mut state = ctx.keep(Reminder::default()).await?;
let sent = state.once(|s| &mut s.sent, async |_| send_reminder(&ctx).await).await?;
let a_day = Timer { spec: TimerSpec::After { duration_ms: 86_400_000 } };
let woke = state.wait(|s| &mut s.woke, a_day).await?;
ctx.pulse_downstream(NodeOutput::new().set("sent", sent).set("woke", woke)).await
}
The first time the body runs, once sends the reminder and writes it into
sent, and wait waits on a one-day timer. With nothing else running, the
run pauses once it has stayed quiet for its pauseAfterSecs. A day later the body
starts over. ctx.keep hands back the state as it was saved, so once finds
sent filled and sends nothing. wait finds woke still empty, because the
pause saved the state before the answer existed, and hands over the answer that
woke the run without waiting again.
| Call | If the field is filled | If it is empty |
|---|---|---|
state.once(|s| &mut s.field, async |st| work).await? | Hands the value back, runs nothing | Runs work, handing it the state as st, puts the result in the field, saves |
state.wait(|s| &mut s.field, signal).await? | Hands the answer back, waits for nothing | Waits on the signal (or, when the body runs again for this answer, takes it at once), puts the answer in the field, saves |
ctx.keep(initial) hands you your state: initial the first time, the last
state saved every time after. It writes nothing. state.save() writes the
state now.
A few rules come with it:
- The state is any type that serializes and deserializes, and
statereads and writes as that type (state.sent,state.woke = None). If your node keeps nothing, writectx.keep(()).await?. - Call
ctx.keeponce per firing (one go of your node in a run). A second call fails the node. - When the run pauses, the call it paused in returns an error and your body
stops there. Always pass it up with
?, or, if you match the result, return every error you do not handle. weft runs the body again when the answer comes..node_err(..)passes a pause, a skipped wait and a given-up wait through unchanged, so wrapping a call with it is safe. - What a
oncereturns has to save as something: a()or aNonefails the node, because a body run again could not tell aoncethat ran from one that did not. Return something liketrue. A wait whose answer is JSONnullfails the same way. - The answer of a
waitis read into the field’s type, so the field’s type has to match what the signal answers with.Option<Value>takes anything. - If a person skips what a
waitasks, the call returnsWeftError::WaitSkipped. Pass it up with?and the firing ends skipped, with every output closed. - Every call where a run can pause takes the state: a wait, a bus read, a
bus’s
wait_for, a stream pull, a yield, and the ctx helpers that wait in a loop. A node that could pause without saying where it is does not compile. A node whose work cannot be picked up from the middle hands its bus reads,wait_fors, stream pulls and yields aHoldinginstead; awaitand the ctx helpers still take the state, though none of them pauses while theHoldingexists (go and read holding the run instead of keeping a state). Test code reads withweft::context::HarnessRead, which fails if a node body hands it to a read. - If your node reads a bus or a stream, one more rule decides where it may read without going through the state. For that rule, go and read reading a bus or a stream across a pause.
- The state is written by
save, at the end of eachonceandwait, and when the run pauses. weft does not write it after each bus message or stream item: it keeps how far those went by itself.
Two waits with a charge between them
An expense needs a manager’s click, then the card is charged, then a director has to click too. The charge must happen once, whichever wait the run paused in.
use weft::signal::{Predicate, ProviderEvents};
use weft::Access;
#[derive(Default, Serialize, Deserialize)]
struct Approval {
posted: Option<Value>,
manager: Option<Value>,
receipt: Option<Value>,
director: Option<Value>,
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let account: Access = ctx.inputs.get("account")?;
let mut state = ctx.keep(Approval::default()).await?;
let posted = state.once(|s| &mut s.posted, async |_| post_buttons(&ctx, &account).await).await?;
let manager = state.wait(|s| &mut s.manager, click(&account, &posted, "manager")).await?;
let receipt = state.once(|s| &mut s.receipt, async |_| charge(&ctx, &manager).await).await?;
let director = state.wait(|s| &mut s.director, click(&account, &posted, "director")).await?;
ctx.pulse_downstream(NodeOutput::new().set("receipt", receipt).set("director", director)).await
}
fn click(account: &Access, posted: &Value, role: &str) -> ProviderEvents {
let id = format!("{}-{role}", posted["id"].as_str().unwrap_or_default());
ProviderEvents::new(account, "interactions", vec![Predicate::eq("callbackId", id)])
}
click builds a ProviderEvents signal, which waits for the service to push
an event matching its predicates: here, a click on the button with that
callback id.
If the run paused on the director’s click, the state on record has posted,
manager and receipt filled and director empty. The body runs again from
the top, both once calls hand back what they saved (so no second set of
buttons and no second charge), the first wait hands back the manager’s
click, and the second one takes the director’s answer.
A wait inside a loop
Five questions, asked one at a time, and the run can pause on any of them.
use serde_json::json;
#[derive(Default, Serialize, Deserialize)]
struct Interview {
answers: Vec<Value>,
current: Option<Value>,
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let questions: Vec<String> = ctx.inputs.list("questions")?;
let mut state = ctx.keep(Interview::default()).await?;
while state.answers.len() < questions.len() {
let question = &questions[state.answers.len()];
let answer = state.wait(|s| &mut s.current, question_form(question)).await?;
state.answers.push(answer);
state.current = None;
}
ctx.pulse_downstream(NodeOutput::new().set("answers", json!(state.answers))).await
}
Here question_form builds a Form with one text field. One field, current,
serves every pass: emptying it after each answer is what makes the next
wait wait. When the run pauses on question 4, the state saved is
{"answers": [<three answers>], "current": null}. The body runs again, the
loop starts at question 4 because three answers are already there, and that
wait takes the answer that resumed the run.
A stream producer that pauses
A node reads an API one page at a time, yields every row on a Generator
output, and rests a minute between pages to stay under the API’s rate limit.
#[derive(Default, Serialize, Deserialize)]
struct Paging {
next: Option<String>,
page: Option<Page>,
sent: usize,
rested: Option<Value>,
}
#[derive(Clone, Serialize, Deserialize)]
struct Page {
rows: Vec<Value>,
next: Option<String>,
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let mut state = ctx.keep(Paging::default()).await?;
loop {
if state.next.is_some() {
let minute = Timer { spec: TimerSpec::After { duration_ms: 60_000 } };
state.wait(|s| &mut s.rested, minute).await?;
}
let cursor = state.next.clone();
let page = state.once(|s| &mut s.page, async |_| fetch_page(&ctx, cursor).await).await?;
for row in page.rows.into_iter().skip(state.sent) {
ctx.yield_downstream(NodeOutput::new().set("rows", row), &mut state).await?;
state.sent += 1;
}
let Some(next) = page.next else { return Ok(()) };
state.next = Some(next);
state.page = None;
state.sent = 0;
state.rested = None;
}
}
The run can pause here in two places.
On the timer between pages. The state says which page comes next, so the
body goes straight to that wait, then fetches the page.
In the middle of a page, while a yield waits for the consumer to take a row (the consumer is itself waiting on a person, say). weft numbers a node’s yields and saves the number of the one it paused in. Made again, that yield sends nothing: the item yielded before the pause stands, and the call waits for it to be taken. So the resumed body has to come back to that same yield with the same row, which is why the page and the count of rows sent go in the state. A page fetched again might not hold the same rows, and a body that started the page over would send its first rows a second time.
A stream consumer that saves as it goes
A node grades every row of a stream with a paid model call, and keeps a count.
use weft::Generator;
#[derive(Default, Serialize, Deserialize)]
struct Grading {
graded: u64,
passed: u64,
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let rows: Generator<Value> = ctx.inputs.get("rows")?;
let mut state = ctx.keep(Grading::default()).await?;
while let Some(row) = rows.next(&mut state).await? {
if grade(&ctx, &row).await? {
state.passed += 1;
}
state.graded += 1;
state.save().await?;
}
ctx.pulse_downstream(NodeOutput::new().set("graded", state.graded).set("passed", state.passed)).await
}
weft keeps how many items the node took off each stream input, saved with its
state. If the producer above rests on its timer, this node waits in
rows.next, the run pauses, and the node starts again after the rows it
already took.
The save after each row is for a crash. If the worker dies, a durable run
runs this node again from its last save and hands it again the rows it took
since, so with a save per row, only the row it was grading is graded (and
paid for) twice. If the producer’s end had already reached the node at its
last save, the body run again gets the rows it had not taken and then the
end, without waiting on the producer.
A conversation that pauses while nobody talks
Two nodes hold a conversation on a bus. One waits for the next message in a chat service’s thread and passes it on; the other reads the bus and answers. The bus is ephemeral, so the messages stay out of the run’s record.
use weft::bus::BusOptions;
use weft::stream_journal::JournalPolicy;
use weft::NodeErrExt;
#[derive(Default, Serialize, Deserialize)]
struct Inbox {
next: Option<Value>,
}
// The node that listens.
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let account: Access = ctx.inputs.get("account")?;
let thread: String = ctx.inputs.get("thread")?;
let mut state = ctx.keep(Inbox::default()).await?;
let opts = BusOptions { journal: JournalPolicy::ephemeral(), ..BusOptions::default() };
let bus = ctx.open_bus("chat", opts, "inbox").await?;
loop {
let said = ProviderEvents::new(&account, "messages", vec![Predicate::eq("thread", thread.clone())]);
let message = state.wait(|s| &mut s.next, said).await?;
if message["text"] == "bye" {
return Ok(());
}
bus.send("message", message).node_err("passing the message on")?;
state.next = None;
}
}
// The node that answers.
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
let mut state = ctx.keep(()).await?;
let bus = ctx.join_bus("chat", "assistant")?;
let mut cursor = bus.cursor()?;
while let Some((_from, message)) = cursor.next_json("message", &mut state).await? {
let reply = answer(&ctx, &message).await?;
post_reply(&ctx, &reply).await?;
}
Ok(())
}
While nobody talks, the first node waits on its wait and the second waits on
its cursor with nothing to read. Once that has lasted the trigger’s
pauseAfterSecs (10 seconds unless it says otherwise), the run pauses and
lets go of its worker; a message inside those seconds goes straight through
on the same worker, with no pause.
The bus stays open through the pause, and both names stay on it, so neither
node ever sees the other leave.
When a message comes, a worker opens the bus again and runs both bodies again.
open_bus and join_bus hand each node the bus it had, under its name, and
the cursor starts where it stopped. The first node’s wait takes the message
and sends it, and the second node reads it. An ephemeral bus comes back empty,
and nothing in this conversation is lost, because the answering node was
waiting on its cursor and had read everything.
Reading a bus or a stream across a pause
A loop that reads a bus or a stream picks up by itself after a pause: weft
saves how far your reader got on each bus and how many items you took off
each stream, and in the body run again the reader starts from there. What
weft does not save is what your code did with an item it had already read.
So if a loop only reads, let it call the reader’s own next (or next_json,
or the stream’s next). If you need an item again
after a pause (the message you stopped to ask somebody about, the first half
of a pair), read it through the state.
state.read(|s| &mut s.field, &mut reader) reads through the state. If the
field is filled, it hands that back and reads nothing. If it is empty, it reads
the next item (a BusEntry off a bus reader, or a T off a Generator<T>,
which you then declare mut), puts it in the field and hands it back, or
None at the end. It saves nothing by itself: the field is written with the
rest of the state at the next pause, save(), or end of a once or a wait.
Empty the field when you are done with the item, or the next state.read
hands back the same one.
A body run again after a pause starts from its top, so one rule decides where it may read without going through the state. weft checks it live, on every run of the body, first or resumed, and saves nothing for it:
The first time a run of your body reaches a place where it can pause, it
must not have read directly before, except inside a state.once that has
finished.
A place where it can pause is a bus read, a stream pull, state.read,
state.wait, a yield, or a wait between the looks of a ctx helper that
waits. A state.read or state.wait counts even
when its field is already filled; a try_next never waits, so it is a direct
read but no place. A direct read is any read that does not go through the state: next,
next_message, next_json or next_bytes on a bus reader, and next,
try_next or drain on a stream. A place is its line and what it reads (the
bus, the stream, or the state field), and one the run already reached is fine,
so a loop can read one reader directly turn after turn.
Read directly in your node’s own code, not inside a helper function, or read
through the state: a helper’s reads all carry the helper’s line, so two phases
reading through one helper look like one place.
Here is what the rule prevents. Say a body reads a greeting with r.next and
then waits for a person. Paused in the wait and run again from the top, it
would read at the greeting’s line again, from where the reader stopped, and
get the message after the greeting. So weft refuses the wait the first time
the body gets there, on the first run, whether or not it would have paused,
naming both lines:
node 'intake' read directly (not through its state) at <line> before it waited
at <line>, a place its run can pause: run again after a pause, it would read at
<line> again from where its reader stopped and lose what it read there. Read
that earlier one through the state (`state.read`), or put that earlier phase
inside a `state.once`
In place of waited, the error says what that place does: read a bus,
pulled a stream, drained a stream, read through its state, yielded,
waited between a helper's looks.
If a body reads directly and then, after a pause, takes a different path than before, it fails at the first place on the new path. So: before a place your run can pause, branch only on your state; anything from outside (the time, a random value, a call to a service) goes in a state.once. If a body breaks this with no direct read before the split, nothing stops it: weft matches each wait to the record by its number in the firing, so a resumed body that took another path and reaches a different wait of the same signal kind silently gets the earlier wait’s answer. Only a wait of another kind fails, naming both kinds.
Two calls are left out. bus.wait_for(name, &mut state) is not a read: it
asks whether that name ever joined the bus, and the list of who joined comes
back with the bus after a pause, so a body run again gets past it at once.
And while your node holds its run (ctx.hold()), nothing it calls can pause,
so nothing it calls is a place. The last direct read it made while holding
counts once the hold is dropped: let m = { let mut h = ctx.hold(); r.next(&mut h).await? }; state.wait(..) fails at the wait on its first run, while reading
and waiting inside one hold passes.
These shapes pass, written short (r, ra and rb are readers, and
is_stop stands for your own test):
// a loop reading directly
loop { let m = r.next(&mut state).await?; handle(m); }
// a greeting through the state, then a loop
let hello = state.read(|s| &mut s.hello, &mut r).await?;
loop { let m = r.next(&mut state).await?; }
// two buses in one loop (it can pause in either read)
loop {
let a = state.read(|s| &mut s.a, &mut ra).await?;
let b = rb.next(&mut state).await?;
handle(a, b);
state.a = None;
}
// two loops with a wait between (it can pause in each step)
let stop = state.once(|s| &mut s.first, async |st| {
loop { let m = r.next(st).await?; if m.is_stop() { return Ok(m); } }
}).await?;
state.wait(|s| &mut s.ok, approval).await?;
loop { let m = r.next(&mut state).await?; }
These fail on the first run, with no pause needed:
let hello = r.next(&mut state).await?;
loop { let m = r.next(&mut state).await?; } // fails at the loop's read
let m = r.next(&mut state).await?;
state.wait(|s| &mut s.ok, approval).await?; // fails at the wait
loop {
let a = ra.next(&mut state).await?;
let b = rb.next(&mut state).await?; // fails here
}
for r in &mut readers { r.next(&mut state).await?; } // fails at the second reader
let r1 = bus.cursor()?; let r2 = bus.cursor()?; // fails at the second cursor
The last one is a different rule: a run of your body opens one reader per bus, because weft saves one position per bus for your node. For its error, go and read one reader per bus. A stream needs no such rule, because every handle on a stream port shares one position.
If a run pauses inside a once, the once’s work runs again from its top
when the run comes back, but its reader carries on from where it stopped. So
anything that loop adds up across turns (the notes it gathers until a stop,
say) has to live in the state. Once the once has returned, its field is
filled, and a body run again after any later pause skips the whole loop.
A bus reader that stops to ask somebody
A moderator reads messages off a bus and, for a risky one, asks a person before passing it on. The version you would write first is refused:
#[derive(Default, Serialize, Deserialize)]
struct Review {
verdict: Option<Value>,
}
let mut state = ctx.keep(Review::default()).await?;
let mut cursor = bus.cursor()?;
while let Some((_from, message)) = cursor.next_json("message", &mut state).await? {
if is_risky(&message) {
let verdict = state.wait(|s| &mut s.verdict, review_form(&message)).await?;
// ... act on the verdict
state.verdict = None;
}
}
The first risky message brings the body to the wait after a direct read, so
the node fails there with the error above. If it were allowed, a pause in the
review would lose the message: the reader’s place is saved past it, and the
body run again would read the next one. The message being handled goes
through the state:
use weft::bus::BusEntry;
#[derive(Default, Serialize, Deserialize)]
struct Review {
handling: Option<BusEntry>,
verdict: Option<Value>,
}
let mut state = ctx.keep(Review::default()).await?;
let mut cursor = bus.cursor()?;
while let Some(entry) = state.read(|s| &mut s.handling, &mut cursor).await? {
if let Some(("message", message)) = entry.as_json() {
if is_risky(message) {
let verdict = state.wait(|s| &mut s.verdict, review_form(message)).await?;
// ... act on the verdict
state.verdict = None;
}
}
state.handling = None;
}
entry.as_json() gives a JSON message’s kind and value, and None for a
join, a leave or a bytes message. When the body runs again after a pause,
handling is filled, so state.read hands the message back without reading,
and the body goes straight to the wait.
Whether or not the run pauses, the bus keeps only its last messages, as many
as its window holds (64 unless the node that creates it sets window in its
BusOptions). While the run is held, a reader that falls further behind than
that skips the messages that were dropped. If the run pauses, only a journaled
bus carrying JSON comes back with that window, and only the messages after the
newest one over 100 KB, since the record keeps a message that large trimmed
(for the full rule, go and read a pause keeps a bus, a crash ends
it). When the body
comes back, if the bus has lost a message your reader had not read yet, the
reader’s first read fails instead of skipping it:
this node paused with messages of a bus it had not read yet, and the bus no
longer holds them (a bus keeps only its last window of messages, and over a
pause an ephemeral bus keeps none and a journaled one only those of that window
after the newest one over 100 KB): it cannot read on as if it had them. Make
the bus journaled with a window that holds what a reader may fall behind by, or
hold the run while reading it (`ctx.hold()`)
So if messages can pile up while your node waits on somebody, the node that
creates the bus has to keep it journaled (the default), send JSON, keep each
message under 100 KB, and set window wide enough to hold them.
Doing something only once
once has one gap: if the worker dies after work returned and before the
save landed, the body runs work again. If the service on the other end must
never see it twice (a payment, say), it has to refuse the duplicate itself,
usually through an idempotency key you send with the request.
If the thing you must not repeat is not in a once, set a field that says it
is done, call state.save() right after, and skip the thing when that field
is set. Otherwise a body run again after a crash does it again.
ctx.tag_run needs neither: tagging again changes nothing. ctx.stop_tagged
is different when this run does not carry the tag itself: made again after a
pause or a crash, it also stops the runs that took the tag since. If that
matters, put it in a once, returning something from it, since a () fails
a once:
use weft::StopSelf;
// `stopped: Option<bool>` is a field of the node's state.
state
.once(|s| &mut s.stopped, async |_| ctx.stop_tagged("chat_7", StopSelf::Keep).await.map(|()| true))
.await?;
When a run pauses
A state.wait never pauses its run by itself: the call stays open in your
node while the rest of the run carries on. The run pauses once every node
still running waits (in a state.wait, a bus read, a stream pull, a yield, or
a bus’s wait_for, which waits for a name to join the bus), at least one of
those waits needs an answer from outside, and the run has stayed that quiet
for its trigger’s pauseAfterSecs. Then every waiting call saves its state
and pauses together, and the run lets go of its worker. An answer that comes
within those seconds goes straight into the call waiting for it, with no
pause. If the worker is stopping, a run that can pause pauses at once and
carries on in the next worker. If none of the waits needs anything from
outside (nodes waiting only on each other’s buses), nothing could ever move
the run, so it is stuck rather than paused, and weft ends it.
weft numbers each wait and each program call within a firing, which is how a body run again is matched to what it did before. Beside your state, it saves by itself:
- how far the node’s reader on each bus read, and the name it holds on each bus;
- how many items it took off each stream input;
- how many items it yielded;
- the numbers its next wait and its next program call will take.
When the answer comes, a worker opens again every bus that was open, then runs again from the top the firing that answer was for and every firing that paused inside the run. A firing still waiting on its own signal keeps waiting. A port the node already emitted on sends nothing a second time, and its first value stands.
A body that drains a stream (drain) can pause there too. drain takes the
items as they arrive but holds them until the stream ends. If the run pauses
first, weft saves the items it was holding with the node’s state, and the
resumed body’s drain starts from them, takes the rest, and hands over the
whole list at the end. Each item is taken once, and your state struct needs
nothing for it.
Holding the run instead of keeping a state
Some work cannot be picked up from the middle: a session with a provider that ends with its connection, audio being gathered into one file, an answer streaming to a caller. No saved state brings that back. If your node does such work, hold the run while it reads:
let mut holding = ctx.hold();
while let Some(chunk) = text.next(&mut holding).await? {
session.send(&chunk).await.node_err("sending text to the session")?;
}
You hand &mut holding to bus reads, a bus’s wait_for, stream pulls and
yields, where you would otherwise hand your state. While holding exists, no
wait of your node pauses: the calls made with it, and its state.waits, its
ctx helpers’ waits and its reads with the state too, wait open in their call.
The run keeps its worker the way any run that cannot pause does: for as long
as a caller is on the line, otherwise until nothing has moved in it for its
trigger’s holdSecs, when the waiting call fails with WeftError::WaitGaveUp,
saying the node holds its run. Once holding is dropped (when the body
returns, say), the node can pause again. Meanwhile the other nodes’ state.waits are held in their calls
too, and an answer goes straight into the call waiting for it. Holding saves
nothing, so the node is never picked up from the middle: if its worker stops
or dies while it holds, it is treated as a node that was running with nothing
saved, and is failed (go and read when the worker dies
mid-run).
If you are choosing between the two: keep a state when a few fields can say where the node is and it can carry on from there; hold when where it is lives in something outside weft that another worker cannot rebuild.
Waiting on a job a service runs
If your node starts something slow on a provider (a video render, say) and the
provider hands back a status address, do not loop and sleep in your body: that
holds a worker for the whole job. Submit the job in a once, then wait on the
status address with PollEndpoint:
use weft::signal::{CancelCall, CancelWith, PollEndpoint, Predicate};
#[derive(Default, Serialize, Deserialize)]
struct Render {
submitted: Option<Value>,
status: Option<Value>,
}
let mut state = ctx.keep(Render::default()).await?;
let submitted = state
.once(|s| &mut s.submitted, async |_| post_json(&http, &submit_url, &payload, "submitting the render").await)
.await?;
let id = submitted["request_id"].as_str().node_err("the submit answered no request_id")?;
let poll = PollEndpoint {
url: format!("{API}/requests/{id}/status"),
interval_secs: 5,
access: Some(weft::primitive::AccessRef::from(&account)),
filters: vec![
Predicate::neq("status", "IN_QUEUE"),
Predicate::neq("status", "IN_PROGRESS"),
],
..Default::default()
};
let status = state
.wait(|s| &mut s.status, poll.cancel_with(CancelCall::put(format!("{API}/requests/{id}/cancel"))))
.await?;
Without the once, the body running again would start a second job and pay
for it.
If the run can pause, it pauses the way any run does
(for the other case, go and read when the run cannot
pause). weft checks the
address once
straight away (a job that is already done resumes at once), then every
interval_secs, signing each check with the access connection. An interval
under 5 seconds fails the wait at once, saying
poll_endpoint.interval_secs must be >= 5. The first answer that passes every
filter is what the wait returns, and the checking stops. With no filters, the
wait returns the first successful answer. An error status or a failed request
is not an answer: it is tried again at the next check and shown on the node.
If the finished answer is heavy (a crawl’s status that embeds its first page
of results), list the fields your node reads next in carry, like
carry: vec!["status".into(), "error".into()]. The filters still read the
whole answer, but only those fields are handed back, so megabytes of data
never sit in the run’s history; read the rest yourself once the wait is over.
Write the filters as “not one of the in-flight statuses” rather than “one of the finished ones”, the way the example does. A status you did not expect then ends the wait, and your code can fail on it, instead of waiting for ever.
If the run is cancelled while the job runs
Cancelling the run takes the wait down, but the provider hears nothing and
goes on working, and charging, for an answer nobody will read. If you want the
provider told, wait on signal.cancel_with(call), as the example does, where
call is one request to send if the run is cancelled, usually to the
provider’s own cancel address for the job.
Whatever cancelled the run (a person, another run’s stop_tagged, turning
the trigger off with a take-down that cancels the runs still going), weft
sends that request once, signed with the same access as the checks. It is
never sent when the answer arrives, or when the run ends any other way.
CancelCall::post, put, patch and delete take the address. If the
provider’s cancel wants a body, add .with_body(json!({ .. })).
If the address is empty, or is not http or https, the wait fails at once,
before the run pauses, with invalid signal spec: the cancel call's url must not be empty or
invalid signal spec: the cancel call's url must be http(s): got '<url>'.
If the provider has no way to cancel a job, wait on the plain signal.
If the provider refuses the call, or does not answer within 60 seconds, weft
does not try again, and the job may still be running. The runtime’s log
(weft daemon logs on your machine) says a last call to an outside service failed, with the address and what the provider answered.
If you want to check the call in a test, read it off the wait:
assert_eq!(
rig.awaited_signals()[0].on_cancel,
Some(CancelCall::put("https://queue.example/requests/req-1/cancel")),
);
Watching it while it waits
If you want to show how far the job has got (a progress bar for a render, say), weft keeps the last answer a check got while the node waits, exactly as the provider sent it. A failed check does not replace it, and an answer over 64 KiB is kept only as its size. You can read it:
- in the editor, on the waiting node’s panel, while you follow the run;
- in the terminal, with
weft node-state <run> <node>, which printsrender: waitingand thenlast answer: {"status":"IN_QUEUE"}; - in a program, with the
GetNodeStatenode, which takes a run’s id (a run gets its own id from theGetRunIdnode); - in your own node’s code, with
ctx.run_by_id(run).node_state(node).
Cleaning up what your node made at a provider
Say your node uploads a file to a provider, starts a job that reads it, and waits for the job. If the run is cancelled during the wait, or the worker dies, the code that deletes the upload afterwards never runs, and the file stays in the user’s account. So register the delete right after the upload exists, before anything else can go wrong:
use weft::access::client;
use weft::signal::CancelCall;
let id = upload(&http, &file).await?;
ctx.after_node(CancelCall::delete(client::url(API, &["uploads", &id])?).always(), &account).await?;
weft sends that call once, through the connection you pass (None sends it
plain), when the node ends, however it ends: it returned, failed, was
cancelled, gave up or skipped a wait, or its worker died and the run ended
without it. Leave out .always() and the call goes only if the run is
cancelled before the node ends, for a job the provider would otherwise keep
running.
The call is on the run’s record before after_node returns, which is what
lets it survive a crash:
- A worker that picks the run up after a crash still owes it, and so does the worker that picks up a run that paused.
- A body that runs again after a pause or a crash and registers the same
call again (same node, call and connection) still owes it once, so you do
not have to guard the
after_nodewith aonce. - A run that ends with no worker driving it (cancelled while it was paused, or removed with its project) still sends it, from weft’s own side.
weft sends the calls before it writes that the node ended, so a worker that
dies in between sends them again from the next one. Pick a call that is
safe to send twice: a 404 counts as done, so a delete is.
If the provider refuses the call, or does not answer within 60 seconds, weft
does not try again. A call sent when the node ended shows on the node’s log
as the call DELETE <url> this node owed when it ended did not go through:
followed by the provider’s own words. One sent from weft’s side, for a run
nobody was driving, shows in the runtime’s log, the same way as a wait’s
cancel call.
after_node fails on a pure node, which does nothing outside the run, and
on an address that is empty or not http or https. A wait’s
cancel_with takes only a call without .always(): its call goes with the
wait, which is gone once answered, and an .always() one is refused, naming
ctx.after_node.
If you want to check the call in a test, rig.owed_calls() lists every call
the node registered, each with the connection it goes through:
let owed = rig.owed_calls();
assert_eq!(owed[0].0, CancelCall::delete("https://api.example/uploads/123").always());
When the run cannot pause
If the run cannot pause (a caller is still on the line, the trigger has
recorded: false, or it is a node test), state.wait waits inside your call
on the same worker, and returns the answer when it comes. If nothing in the
run moves for the trigger’s holdSecs, the call fails with
WeftError::WaitGaveUp, saying why; while a caller is on the line, that clock
does not run. For what holds and when the clock runs, go and read when a run
cannot pause.
A wait given up is an outcome like any other: if your node can do something sensible without the answer, match it and carry on.
use weft::WeftError;
let approved = match state.wait(|s| &mut s.approved, question_form("Send this?")).await {
Ok(answer) => answer,
Err(WeftError::WaitGaveUp(_)) => json!(false),
Err(other) => return Err(other),
};
If you want to test that branch, rig.signal_given_up() makes the next wait
fail with WaitGaveUp, and rig.signal_skipped() makes it end skipped.
The ctx helpers that wait
A few ctx calls follow something slow until it ends, checking it on a timer, and the run can pause between two checks. Those take the state:
ctx.infra(node).start(&mut state),.request_start(&mut state)and.rebake(&mut state);ctx.all_infra().follow_command(id, &mut state);ctx.project(id).code().build().start(&mut state)andctx.project(id).code().run()...start(&mut state);ctx.project(id).triggers().resync(take_down, &mut state)andctx.project(id).resume_resync(&mut state)(on another project: a resync of the run’s own project would wait on the run itself);ctx.project(id).remove(&mut state).
If the run paused in one of them, weft saves where the helper stood. The body
run again comes back into the same helper, which goes straight back to the
wait it paused in and makes none of its earlier calls again: no second read,
no second start. A body that comes back into a different helper first, or
reaches its own state.wait first, is refused. run()...start runs on the
working copy it built, read in the build’s first check.
Every other call on your project (for the list, go and read your project, and
the owner’s other projects)
is made again each time the body runs again, as a new call. The one exception
is a call made since the last save, when the worker died before the next one:
the body runs again from that save and makes the call again under the number
it had. It is then the same call, so what it makes is made once (a project it
declares comes back as the same project), and what it takes down answers that it is already down. A minted
instance token behaves differently: the mint gives the same token a fresh
value, and the first one stops working. If the body must
not repeat a call after a pause, put it in a once.
When the worker dies mid-run
What happens depends on the trigger’s durable setting. For what that setting
does, go and read how a run is
kept.
A fast run (the default) ends cancelled with its worker, and is not run again (go and read what happens when something dies). The exception is a run that had paused: it survives its worker, fast or not. A fast run with one branch waiting while another still ran ends cancelled like any other.
A durable run carries on in another worker, and so does one whose worker could not write its record for over a minute (for that case, go and read when a write of the record fails). Each node’s start (except a pure node’s, below) is on record before its body runs, so the next worker knows exactly which nodes were running when the first one went away.
A node that was running and had saved a state runs again from its last save,
the way a paused one does. Anything it did after that save, it does again (the
stream consumer above shows how to keep that small). Calling ctx.keep saves
nothing by itself: a node counts as having saved once it reached a save, the
end of a once or a wait, or a pause.
A node that had saved nothing is failed, because weft cannot tell how much of its work already happened:
the worker running 'billing' went away while it was running; it was not run
again, because it may have partly happened. Re-run from here once you have
checked what it did.
A node that was reading a stream and saved no state says went away while it was reading a stream instead: it saved no state to carry on from, the rest of
its stream goes with it, and the message tells you to start the whole run
again so the stream starts over.
If you want to run that node again once you have checked, weft run --from <node>=<json> starts a run there with the inputs you hand it. If you wired
the node’s _error output, this failure goes there like any other, and that
branch carries on; otherwise the run fails.
If the node that was running is a pure one that does not read a stream, and
had not emitted yet, the next worker runs it again instead of failing it. A
pure node does nothing outside the run (for what that means, go and read the
pure flag in metadata.json), so running it twice
changes nothing. One that had emitted is failed like any other.
The last messages on a bus may never have reached the record before its
worker died. So if any node was left running (its worker died, or it was
stopping and a node did not pause within the few seconds it is given), whether it runs again from a saved state or is failed, the
next worker ends every bus the run had open as failed, with the worker holding this bus went away, so messages it carried may be lost. A pause never
does this: a paused run’s buses come back open.
In a durable run, if a node emitted a value before the crash and the node reading it had not started yet, that value is still delivered and the reading node runs.
A run paused before this version of weft
weft used to pick a waiting node up without a saved state, so a run that paused under that version cannot carry on, and its waiting node fails with:
this run paused before this version of weft, which picks a paused node up from
the state it saved (`ctx.keep`), and 'approve' saved none: it cannot carry on.
Start the run again (`weft run --seed=<run>` reuses what it already did).
Writing a trigger
A trigger starts runs from outside. It has two bodies instead of one.
#[async_trait]
impl Node for CronNode {
async fn setup_trigger(&self, ctx: RunContext) -> WeftResult<()> {
let expression: String = ctx.inputs.get("cron")?;
let timezone: String = ctx.inputs.get("timezone")?;
ctx.register_signal(Timer { spec: TimerSpec::Cron { expression, timezone } }).await
}
async fn run(&self, ctx: RunContext) -> WeftResult<()> {
ctx.pulse_downstream(ctx.fan_declared(&ctx.wake.record()?)).await
}
}
setup_trigger runs once, when somebody activates the project. run fires
every time the thing actually happens.
In the metadata:
"features": { "isTrigger": true },
"firesWith": { "chatId": "String", "text": "String", "caller?": "JsonDict" }
Setup
Read your inputs, decide what should wake you, and register it. That is all.
You never listen for anything yourself. No socket, no polling loop, no webhook handler. You say what should wake you and the runtime does the holding, the renewing and the checking that the event is real.
Two things to know about it.
Setup reaches out. A trigger that waits for Slack messages registers a subscription with Slack right there. If that cannot be done, activation fails on the spot naming what is missing, rather than leaving a trigger that looks armed and is dead.
Your inputs are frozen at that moment. Whatever they were is snapshotted
alongside the registration and replayed onto ctx.inputs on every firing. The
nodes upstream do not run again per event. When somebody changes one, they have
to run weft resync, and that is why.
Register once per node per setup. A second call is a loud error rather than a trigger that quietly never fires.
Firing
run gets the event on ctx.wake, a bag with the same methods as
ctx.inputs. Your declared inputs are there too, replaying the snapshot.
let chat_id: String = ctx.wake.get("chatId")?;
ctx.fan_declared(&ctx.wake.record()?) is the common shape: take the whole
payload and fan its keys onto your same-named output ports, skipping anything
you did not declare.
firesWith
Say exactly what the event carries. The ? goes on the name:
"firesWith": { "chatId": "String", "caller?": "JsonDict" }
It is checked both ways before your body runs. A payload missing a field you
declared without ? is refused, and so is one carrying a field you never
declared, at the top level and nested. weft run --fire checks a payload you
typed by hand against it too.
The metadata does not load if a node that is not a trigger declares
firesWith, or if a type in its firesWith is not a weft type. A trigger may
leave firesWith out, but then nothing checks what it fires with, and weft run --fire cannot tell you what to send.
What can wake you
| Kind | What it is |
|---|---|
Timer | A delay, a date, or a cron schedule. A delay counts from when it was asked for: the activation for a trigger, the state.wait call for a node that waits |
Form | A person filling something in |
Route | An inbound HTTP request |
Socket | An inbound WebSocket |
SocketListen | A socket weft dials out to and holds |
StreamListen | A TCP or TLS stream |
SseSubscribe | A server-sent-event stream weft subscribes to |
PollEndpoint | An address weft checks on a timer. For waiting on the status address of a job your node started, go and read waiting on a job a service runs |
ProviderEvents | A service’s own events, by topic, through its recipe |
If your node registers a SocketListen or StreamListen with a connection,
the text frames it sends (and a stream’s address) can name the connection’s
stored values as {name}, and the value goes in as it is. When the value is
something a person typed into a line protocol, like a password in an IMAP
LOGIN, write {name|quoted} instead: weft wraps it in " and escapes \
and ", so a1 LOGIN {user|quoted} {password|quoted} signs in whatever the
password holds. A value with a line break in it cannot be quoted and is
refused rather than sent as a second command.
ProviderEvents is the one to reach for when the service has a recipe, because
it takes care of both roads: a provider that pushes to you, and one you have to
dial. Go and read events from a service.
A trigger that polls a list
If the service only offers “list the latest items” (a mailbox, a database
query), register a PollEndpoint with a delta, and weft fires one run per
new item instead of one per poll:
use weft::access::client::url;
use weft::signal::{DeltaMode, PollDelta, PollEndpoint, PollPaging};
ctx.register_signal(PollEndpoint {
url: url(API, &["messages"])?.query("maxResults", 500).into(),
interval_secs: interval,
delta: Some(PollDelta {
items: "messages".into(),
cursor_field: Some("id".into()),
mode: DeltaMode::Set,
cursor_param: None,
paging: Some(PollPaging::query("nextPageToken", "pageToken")),
}),
access: Some(weft::primitive::AccessRef::from(&account)),
..Default::default()
})
.await
items is the dotted path to the list in the answer. cursor_field is the
field that tells items apart: with DeltaMode::Max an item is new when that
field is above the highest one seen so far, which needs a field that only
grows in the order items show up; with DeltaMode::Set it is new when weft
has not seen its id recently, for a list whose ids are opaque. The first poll
after activation only records what is there and fires nothing, so activation
means “from now on”. Each run gets the item on ctx.wake as item.
If the list comes in pages, newest first, say how to ask for the next one in
paging, so a burst bigger than one page is not cut to its newest page:
PollPaging::query(next, param): the answer says the next page’s cursor at the dotted pathnext, and the next request sends it as the query parameterparam;PollPaging::body(next, field): the next request sends it as the fieldfieldof the JSON body, for aPollMethod::Postpoll whosebodyis an object.
Each poll then follows the pages until one holds an item it has already
seen, or the answer names no next page (a cursor that is missing, empty or
not a string), and fires every new item it read. The first poll after
activation reads the first page alone. If more than 50 pages of new items
arrived since the last poll, the poll fails and fires nothing, rather than
firing only part of the burst: the log says paged poll ran past its page limit, and the fix is polling more often (interval_secs) or asking for
bigger pages. Set paging: None for a list that fits one answer.
paging needs a cursor_field, a JSON answer (not PollFormat::Feed), and
no cursor_param beside it, since that already says where the next request
starts. Activation refuses each of those, saying which.
Two things a trigger cannot do
It cannot be wired from another trigger, and nothing downstream of a trigger can be an infra node. Both are refused at compile time, because a trigger’s inputs are frozen at setup and provisioning happens before any event exists.
It also cannot sit inside a loop. It registers once for the project, so a per-iteration one is not a thing.
Testing one
rig.run_setup_trigger(&CronNode, json!({ "cron": "0 0 9 * * *" })).await.ok()?;
assert_eq!(rig.registered_signals().len(), 1);
let outcome = rig.wake(json!({ "scheduledTime": "2026-01-05T09:00:00Z", "actualTime": "2026-01-05T09:00:01Z" }))
.run(&CronNode, json!({})).await.ok()?;
fake is the top tier for a trigger. The live rig drives a plain run body,
so a live test on a trigger is refused.
Talking to a live caller
Most runs have nobody waiting. A run started by an HTTP request or a WebSocket does, and that caller is holding a connection open while your program works.
Your node gets at them through the ctx.
let caller = ctx.http_caller().await?;
caller.respond(OutboundChunk::json(json!({ "ok": true }))).await?;
Finding out whether there is one
| Call | What you get |
|---|---|
ctx.is_api_call() | Whether an HTTP caller is attached |
ctx.is_websocket() | Whether a WebSocket one is |
ctx.caller() | The caller as a handle, or None |
ctx.caller_request() | What they sent to open the exchange |
ctx.caller_data_type() | Whether this connection speaks json, text or bytes (the trigger’s dataType) |
Those are two separate questions rather than one enum, because a node can branch three ways: HTTP, WebSocket, or nobody.
ctx.caller_request() does not wait for the connection, because the handshake
is known the moment the run starts. It gives you method, path, params (the
path’s captures, like room in /chat/{room}), query, headers,
credentials (the headers carrying a credential, each a Secret you read
with ctx.reveal; they are never in headers), caller (whatever identity
weft established for the caller) and base_url (the address the caller
reached you at). With no caller attached it fails the run,
so if your node also runs without one, check ctx.caller() first.
Getting the handle
| Call | Gives you | Fails when |
|---|---|---|
ctx.http_caller().await? | An HTTP caller, attached and connected | There is none, or it is a WebSocket |
ctx.ws_caller().await? | A WebSocket caller | There is none, or it is HTTP |
ctx.live_caller().await? | Whichever is connected | There is none |
Each of those folds together “is anybody there”, “is it the right protocol” and
“wait for the connection” into one call. Use the specific one when your node
only makes sense behind a route, and live_caller when it works either way.
Reading what the caller sent
If you want the request’s body as the value your program declared for it, ask for it by type and weft does the rest:
let http = ctx.http_caller().await?;
let taken = http.body_as(&ctx, &declared, "body", None).await?;
let body = match taken {
Ok(body) => body, // Option<Value>: None when no body came (every GET)
Err(why) => return http.refuse(why).await, // a 400 telling the caller what was wrong
};
body_as reads the body in the trigger’s dataType and holds it to
declared. A picture the caller sent in a file-typed place, as a data: URL
or bare base64, is stored (run scope) and the stored file takes its place; an
upload slot (weft-upload:...) in such a place is received. A stored-file or
secret marker the caller wrote in themselves is refused, since only weft makes
those. The last argument stands in when the connection carries no body: pass
the body a fired run carries (one started without a request), or None.
There are two kinds of failure, kept apart. The inner Err(why) is the
caller’s mistake, worded for the caller: a body in the wrong shape, a field
that does not fit, a picture that is not one. refuse(why) answers it with a
400, {"error": why} on a json route and the text itself on the others.
The outer ? is your node’s own failure, like storage refusing a write.
On a WebSocket, recv_as(&ctx, &declared, name) does the same for each
message, and gives None once the caller has gone. There refuse(why) sends
the reason back and the session goes on:
let ws = ctx.ws_caller().await?;
let mut turn = 0;
while let Some(taken) = ws.recv_as(&ctx, &item, &format!("message-{turn}")).await? {
turn += 1;
match taken {
Ok(msg) => handle(msg).await?,
Err(why) => ws.refuse(why).await?,
}
}
name is what a refusal and a stored file are called.
If you want the fields of what you took on your node’s same-named output
ports, ctx.fan_declared(&value) builds that output (go and read
values and emission).
Answering over HTTP
caller.respond(OutboundChunk::json(json!({ "answer": text }))).await?;
| Call | What it does |
|---|---|
respond(body) | Send the whole answer and finish |
respond_with(head, body) | The same, choosing the status and headers |
write(chunk) | Send a piece, keeping the connection open |
write_with(head, chunk) | The first piece, setting the head |
drop_or_fail(made) | Decide what happens to a piece you could not make, per the route’s whenChunkFails |
close() / close_with(head) | Finish |
fail(message) | Finish as failed: a 500 if nothing went out yet, the answer format’s error marker if it did |
Streaming is write repeatedly, then close. That is how an answer appears in
somebody’s browser a word at a time while your model is still writing it.
If making one piece can fail (a value the route’s dataType does not carry, a
file you cannot read), hand the attempt to drop_or_fail before you write it:
if let Some(chunk) = caller.drop_or_fail(make_chunk(&item).await)? {
caller.write(chunk).await?;
}
You get Some(chunk) back when it was made. When it was not, the route’s
whenChunkFails decides: on surface you get the error, and returning it
fails your node and tells the caller; on drop_chunk you get None, the skip
goes on the run’s record, and you carry on with the next piece. Only a failure
of the work itself is ever skipped: a cancel, a pause, or a bad setting or
input always comes back as the error. For what
each setting is for, go and read slow callers and messages that cannot be
made.
If what you are streaming broke off halfway, end with fail rather than
close, so the caller does not take a cut answer for a whole one. fail
always tells the caller, whatever the route’s whenChunkFails says. The caller
gets only Something went wrong and the run’s id. Your message goes on the
run’s record, which you open with weft logs <run id>. For the exact status
and body the caller receives, go and read when a run ends before it
answers.
The head can only be sent once, and sending a body after the exchange ended is an error rather than a silent nothing.
Sending one of your node’s inputs
If what you answer with is the value of one of your node’s inputs, let weft turn it into the chunk:
let chunk = OutboundChunk::of_input(&ctx, "body", value, data_type).await?;
http.respond_with(ResponseHead::new(status).with_headers(headers), chunk).await?;
of_input reads the input’s declared type and, per data_type: on json
every file the type places goes out as a link (see files going
out); on text the value must be a string; on bytes it
must be a stored file, and its content is what gets sent. Every secret the
type places is revealed for the caller while the run’s record keeps its
marker. A secret where the type declares none fails the node, since the caller
would receive the marker instead of the value.
The head has a few builders:
| Call | What it gives you |
|---|---|
ResponseHead::new(status) | A status and no headers yet |
.with_header(name, value) / .with_headers(map) | Headers added, one at a time or a whole BTreeMap<String, String> (a headers input, say, once ctx.reveal_input has revealed it) |
.with_file(&stored) | For a stored file answered as bytes: its type as content-type, its name as an inline content-disposition, unless the head already set them |
ResponseHead::redirect(status, location)? | A redirect: 301, 302, 303, 307 or 308, and 200 becomes 302. Any other status is refused |
If you hold a value that is not one of your inputs and want a secret in it
revealed, send an OutboundChunk::Revealed yourself. sent is what the
caller gets, and kept is what the record holds:
let revealed = ctx.reveal_in(value.clone(), &ty).await?;
caller.respond(OutboundChunk::Revealed { sent: Box::new(OutboundChunk::json(revealed)), kept: value }).await?;
Streaming a bus
If you want every message of a bus sent to the caller as it comes, hand the bus over:
let head = ResponseHead::new(200);
let whole = http.stream_bus(&ctx, &bus, head, Format::Sse, data_type, Some(("first", first))).await?;
It reads from the earliest message the bus still holds, so a producer that
started before your node, or already finished, loses nothing. The head goes
out with the first chunk, and the response ends when the bus closes. A bus
that ended failed is never ended as a whole answer: the response fails, as
with fail, and stream_bus answers false (your node does not fail;
true means the whole bus went out). Each message goes through drop_or_fail. first is one value
sent ahead of the bus and framed like its messages, the value of the input
it names, whose type says where its files and secrets are. The run is held
while it streams, since an answer cannot be picked up from the middle.
Format says how messages are framed. It reads straight from an input that
holds "raw", "ndjson" or "sse" (ctx.inputs.get::<Format>("format")?):
Format | The caller gets | Content type | While the bus is quiet |
|---|---|---|---|
Raw | Each payload as it is, per data_type, no separator | None added | Nothing |
Ndjson | One JSON value per line | application/x-ndjson | An empty line |
Sse | data: lines per event, a blank line after each | text/event-stream | : keepalive |
The filler written while the bus is quiet is ignored by every reader of the format, and it is how a caller who hung up without a word is found. A head that already names a content type keeps it.
Files going out
A file can never go to a caller as bytes inside JSON, so it goes as a link:
{ "url", "mimeType", "filename", "sizeBytes" }. A browser can put that url
straight into an <img> or a <video>. Asked again soon, the same file gets
the same link, and the link lives as long as the file’s linkSecs says, else
fifteen minutes. A file the install cannot link fails the node.
What counts as a file is decided by the value’s declared type, never by
finding a marker in raw JSON. of_input and stream_bus do this for you.
If you build a JSON answer
yourself, caller::link_files(&ctx, &value, &ty) gives you the value with
every file linked. For the walk underneath, go and read whole values full of
files.
A bus declares no type for its messages, so stream_bus reads each message’s
type off the message itself (WeftType::infer): a stored file in it is a file,
a secret is a Secret. From there a message goes out exactly like an input of
that type: every file as a link, every secret revealed, with the run’s record
keeping the secret’s marker. If your own node reads a bus and hands its
messages outside weft, read them with next_message and
WirePayload::into_typed_json, which gives you the same type to link and
reveal by.
Talking over a WebSocket
let ws = ctx.ws_caller().await?;
while let Some(msg) = ws.recv_next().await? {
let reply = handle(msg)?;
ws.send(OutboundChunk::json(reply)).await?;
}
| Call | What it does |
|---|---|
send(msg) | Send one message |
drop_or_fail(made) | The same as on HTTP, per the socket’s whenChunkFails |
receive() | Wait for the next one |
recv_next() | The same, with None when the caller has gone |
recv_as(&ctx, &declared, name) | The next one as the value you declared (see reading what the caller sent) |
refuse(why) | Tell the caller what was wrong with a message; the session goes on |
stream_bus(&ctx, &bus, data_type, first) | One socket message per bus message until the bus closes, the socket staying open. A bus that ended failed ends the session as failed, and the call answers false |
request(msg) | Send, and wait for the reply to that |
close() / close_with(reason) | End it |
fail(message) | End it as failed, with a 1011 close |
There is also a cursor, so a node joining late can read from a chosen point
rather than only from now: cursor(), cursor_from_start(),
cursor_at(offset) and cursor_including_last().
A caller is not durable
A caller is a socket in one worker. It is not written down and it cannot be rebuilt.
So while the caller is on the line, the run keeps its worker, whatever the
route’s outlivesCaller setting says, and state.wait waits inside your call
instead of pausing the run (go and read when the run cannot
pause). The trigger’s
holdSecs does not count down meanwhile, so a wait does not cut a caller who
is still there, unless holdSecs is 0, which fails the wait at once. outlivesCaller only decides what the caller leaving does: off, the
run is cancelled; on, it carries on without them, and pauses once everything
in it waits on something from outside. For that setting, and for what happens
when a connection reaches a cloud install’s one-hour limit, go and read how a
run is kept.
If the caller goes away, that is not your node failing. It reads as the run
being cancelled, which your node can notice with ctx.is_cancelled() and stop
doing expensive work. The exception is a route or socket with outlivesCaller on: its run carries
on, and what your node sends to the caller who left is dropped
without an error. The run’s record names the messages that were still
waiting to go out when the caller left, so you can see exactly what they
never received; what your node sends after that is not on the record.
If you want the record to hold only the sizes of what streams through, or to
group messages into rows differently, the route has journalEphemeral and
journalWindowMs (go and read two
modes).
In a test
rig.attach_caller(conn);
The fake rig gives you a caller to talk to, so a route node’s behaviour is testable without an install or a real socket.
Streams and buses in Rust
For which of the two you want, go and read streams and buses. This is the Rust on both sides.
Producing a stream
Declare a Generator[T] output, then emit as many times as you like:
for chunk in response_chunks {
ctx.pulse_downstream(NodeOutput::new().set("chunk", chunk)).await?;
}
Each emission is one item, checked against the element type. When your body
returns, the port closes and that is the end of the stream. To end it early,
ctx.close_port("chunk"). After a close, nothing can follow.
pulse_downstream sends and carries on. If you are producing faster than the
consumer reads, use yield_downstream(output, &mut state), which waits until
each item is taken. The run can pause in that wait, so it takes your node’s
state. After a pause the body runs again from the top, and it has to come back
to the yield it paused in, with the same item: that yield sends nothing again
and waits for the item to be taken. So keep in the state how far the node got,
the way the producer in a stream producer that
pauses does.
If you do neither and run far ahead, you hit the cap at 4096 un-taken items and
the emit fails, telling you the three ways out: yield instead of pulse, let the
consumer catch up, or ctx.set_max_buffered_items("chunk", n).
Consuming a stream
Read the port like any input, then pull:
let rows: Generator<Row> = ctx.inputs.get("rows")?;
let mut state = ctx.keep(()).await?;
while let Some(row) = rows.next(&mut state).await? {
// one row at a time
}
Your body starts when the first item arrives, so you are running while the producer is still producing.
next(&mut state) waits for the next item and gives you None once, at a
clean end. The run can pause while it waits, which is why it takes your
node’s state (go and read surviving a restart): weft
saves how many items you took, and a resumed body carries on after them.
try_next(&state) is the same without waiting, answering Item, Empty or
Finished. drain(&mut state) takes the lot and waits for the end; the run
can pause there too, and a resumed body’s drain starts from the items it
had already taken. If your node pulls a stream and also waits, yields or
reads at another line, or needs an item again after a pause, go and read
reading a bus or a stream across a
pause.
end() tells you whether the producer finished or failed, or None while it
is still open.
If your node’s work cannot be picked up from the middle (a session with a
provider, audio gathered into one file), pass &mut holding from ctx.hold()
to these calls (and to bus reads and wait_for) instead of your state: while
holding exists, no wait of your node pauses. For when to keep a state and when to hold, go and read
holding the run instead of keeping a
state.
The ? matters. If the producer failed, that is where you find out, rather
than getting a clean end you would mistake for an empty stream. drain errors
without handing back the partial list, for the same reason.
Reading the port twice gives you two handles over one stream, sharing a position, not two copies.
Hosting a bus
let bus = ctx.open_bus("channel", BusOptions::default(), "host").await?;
bus.send("text", json!({ "text": "hello" })).node_err("sending the greeting")?;
open_bus does the whole producer move: makes the bus, emits the marker on
that port, and registers your name.
If the work that makes your pieces can fail (a provider’s call, a handshake, a
first query), open the bus on the first piece instead of before the work.
Until the marker goes out, your node has opened no stream. If it fails then, a
program that wired your _error still gets the failure there, and the port
closes as if nothing was coming, so the readers never start. ctx.pending_bus holds the
options until then:
let mut bus = ctx.pending_bus("stream", BusOptions::default(), "llm");
let mut chunks = client.stream(&prompt).await.node_err("starting the stream")?;
while let Some(chunk) = chunks.next().await {
bus.open().await?.send("delta", json!(chunk)).node_err("sending a delta")?;
}
The first open() makes the bus and emits its marker, and every later one
hands back the same bus. If you never call it, the port closes like any port
you never mentioned. bus.opened() tells you whether it is open yet without
opening it. LlmStream works this way.
Joining one
let mut state = ctx.keep(()).await?;
let bus = ctx.join_bus("channel", "translator")?;
bus.wait_for("host", &mut state).await?;
let mut cursor = bus.cursor()?;
while let Some(msg) = cursor.next(&mut state).await? {
// ...
}
wait_for("host", &mut state) waits until the host has joined the bus. If
the run pauses later, the body run again gets past it at once, because the
list of who joined comes back with the bus.
join_bus resolves the bus on that input and registers your name. A bus you
joined ends with your body, as one you made does, so a reader that dies
mid-conversation never leaves the producer waiting on it.
For an observer that must not end the bus when it is done (a debug tap, or
a node streaming the bus out to a caller), use ctx.bus_from_input("channel")
instead.
If your node has a slow warmup, call join_bus only once it is ready, because
registering is what everybody else’s wait_for waits on. Each reader has its
own cursor, starting where it joined, so reading takes nothing away from the
others.
Every read and every wait_for is a place the run can pause, so each takes
your node’s state (or the Holding from ctx.hold(), if the run must not
pause while your node reads). weft saves how far your reader on each bus read
and the name you hold there, and a body that runs again after a pause gets
them back.
A name that joined and left since still counts for wait_for, and the wait
fails the node if the bus closed first. You never have to decide whether a name will ever turn up: the
engine watches the run, and when a wait can no longer be satisfied it ends the
bus, failed and saying the run was stuck, and every waiting cursor wakes.
One reader per bus
Your body can open only one reader on each bus, because weft saves one
position per bus for your node. cursor(), cursor_from_start() and cursor_at(offset)
return a Result, and opening a second reader on a bus your body already
opened one on, still open or dropped, fails the node:
node 'translator' opened a second reader on a bus at <line>, which it already
opened one on at <line>: a node keeps one reader per bus, so that it can come
back where it stopped after a pause. Read every message through that one reader, filtering by sender or
kind in the node's code
If you want only some of the messages, filter that one reader: match on the
entry in your loop, or hand with_filter a function that keeps the ones you
want. If your node sends on a bus it also reads and can pause in a wait,
drop its own messages with with_filter: weft saves the reader’s place past
what the filter turns down, so it never comes back behind its own replies.
In a body run again after a pause, whichever of the three you call, the reader starts where the old one stopped. The offset you ask for only counts the first time.
How a bus ends
weft ends every bus your firing made or joined when your body returns, and
says how: finished if the body returned Ok, failed with the error if
it returned one, panicked, or was cancelled. A pause ends nothing: the bus
stays open through it, and your body gets it back when it runs again. A task
you spawned does not keep a bus open: the bus ends with the body, whoever
still holds a handle.
Dropping a handle ends nothing. If you are done sending before your body is
done working, bus.close() ends the bus now, and always as finished. To end
it failed, return the error from your body.
A reader finds out once its cursor runs out. next returns None at the
end either way, so ask bus.end():
while let Some(msg) = cursor.next(&mut state).await? {
// ...
}
if let Some(StreamEnd::Failed { error }) = bus.end() {
weft::node_bail!("the stream broke off: {error}");
}
end() is None while the bus is open. What a failed end means is up to your
node. If it answers a caller from a bus, end the answer with fail (go and
read talking to a live caller).
Choosing options
use weft::bus::{BusOptions, BusPayloadKind};
BusOptions {
journal: ctx.journal_policy()?,
window: Some(256),
payload: BusPayloadKind::Bytes,
..BusOptions::default()
}
| Field | Default | What it decides |
|---|---|---|
journal | JournalPolicy::default(): journaled, one-second window | What the record keeps of this bus and how often it writes a row. ctx.journal_policy()? hands over what your node’s users set, JournalPolicy::ephemeral() keeps payloads out whatever they set (what ElevenLabsSpeak does) |
window | None, which is 64 messages | How far back a reader can reach |
payload | BusPayloadKind::Json | JSON or bytes. Frozen at creation; sending the other shape is refused |
meta | Value::Null | Details every reader can read off the marker: a sample rate, dimensions. Keep it small, it rides the marker |
If you want your node’s users to choose between the two modes and pick the
window, set features.journalSettings in its metadata.json (go and read
features) and pass ctx.journal_policy()? as above.
For what each mode keeps and what the window does, go and read two
modes.
What a send tells you
Failures are values, never silent drops: the bus was closed, you never registered, you sent the wrong payload shape, or the record is degraded.
A pause keeps a bus, a crash ends it
When a run pauses with a bus open, the bus comes back in the next worker, with
the same id, at the offset where it stopped, and your reader on it starts
where it stopped. A journaled bus comes back with its last JSON messages, as many as
its window holds, but only those after the newest one over 100 KB: its record
keeps a message that large trimmed, so that one and every message before it
are lost. An ephemeral bus, or one that carries bytes, comes back empty.
A reader that was waiting on its cursor had read everything, so it loses
nothing (for an example, go and read a conversation that pauses while nobody
talks).
A reader that paused in a wait with messages still unread carries
on where it stopped if the bus still holds those messages when it comes back;
if it does not (they did not come back with the bus, or the bus’s window
dropped them while the reader waited), its first read on that cursor fails
instead of skipping them (for how to write a reader that stops to wait, go and
read a bus reader that stops to ask
somebody).
Messages its reader’s filter turns down right after
where it stopped (its own replies, say) do not count: its place is saved past
them. A reader on a bus your node had not read before the pause starts like
any new one.
A node holds one name per bus, the one its state carries over a pause, so a
second register under another name is refused.
A worker that dies is different: its buses can end failed, and each reader
then finds that in bus.end(). For when, go and read when the worker dies
mid-run. Anything that
has to survive a crash goes in storage, or out as an ordinary
output of a node that finishes.
Cancelling and steering runs
Nothing kills your node
A cancelled run trips a flag. Your node is what acts on it. If your body never looks, nothing stops it, and the only sign is a run that never ends.
So anything that waits needs a way out:
let cancel = ctx.cancellation();
tokio::select! {
result = do_the_long_thing() => result?,
_ = cancel.cancelled() => return Ok(()),
}
For a loop, the cheap synchronous read:
for row in rows {
if ctx.is_cancelled() { return Ok(()); }
...
}
That one is cheap enough to call every iteration.
The flag is sticky. Once it is set, every later check sees it, so there is no race between the cancel arriving and your next look.
If your node waits on something itself (a sleep, a long call to somebody
else, a read that could block for minutes), race it against
cancel.cancelled() in a select! like the one above. The waits weft
holds for you need none: state.wait, and the ctx calls that wait through
it, like infra(..).start(&mut state). Cancelling the run takes those down
for you.
If the wait is on a job a provider runs, and you want the provider told too, go and read if the run is cancelled while the job runs.
Stopping other runs
A run can tag itself, and a run can stop every run carrying a tag.
ctx.tag_run(["chat_7"]).await?;
ctx.stop_tagged("chat_7", StopSelf::Keep).await?;
That is the debounce shape. Somebody sends three messages in a row; the third run stops the first two and answers once.
| What it does | |
|---|---|
StopSelf::Keep | Stop the others, keep running |
StopSelf::Include | Stop everything carrying the tag, including this run |
It reaches runs that are working, ones parked on a signal or a timer, whose wake is erased so they never come back, and ones whose wake is already on its way, which find the run gone and do nothing. Each one is recorded as cancelled, naming this run and the tag, so it reads as a decision rather than a failure.
It never crosses a project.
Tag first, then stop
StopSelf::Keep only reaches runs that took the tag before this one did.
That is what stops two runs killing each other. Two messages arriving milliseconds apart both say “stop the others, keep me”, and because the order is by when each tagged itself, the later one survives and the earlier one stops. Without that rule you would get both dead, or neither.
So tag at the top of your node, then stop. A node that stops before it tags is not in the order yet.
The stop is not immediate
stop_tagged returns when the request is durably queued, not when the other
runs are gone. Do not write a node whose next line assumes they stopped.
A tag is [A-Za-z0-9_-], one to sixty-four characters, but you can hand
either call any string that is not empty. If you pass a chat id or an email
address, weft keeps the readable part, swaps every other character for _,
and adds a short fingerprint of the original, so 49151@s.whatsapp.net
becomes 49151_s_whatsapp_net- followed by sixteen hex characters. Both calls
and the runs list’s tag filter do this the same way, so the same value always
lands on the same tag, and two values that only differ in the swapped
characters never share one. Tagging twice with the same tag does nothing,
so a body that runs again after a pause can tag again.
In a test
The fake rig does not stop anything. It records what your node asked for, and you assert on that:
assert_eq!(rig.run_tags(), vec![vec!["chat_7"]]);
assert_eq!(rig.stops().len(), 1);
Taking down what your own run uses
StopSelf is also the last argument of every call that takes part of the
project down (ctx.infra("bridge").stop(..), terminate(..),
wipe(..), upgrade(..), ctx.trigger("receive").deactivate(..),
ctx.runs().clean(..)). For what
Keep and Include do there, go and read
taking something down from a run.
In a test, rig.program_calls() lists every such call with the StopSelf it
passed. If your node reads something back, such as an infra copy’s status,
rig.answer_program_call("weft.infra.status", ..) sets what that call returns
in the test.
Infrastructure nodes
Some nodes need something running: a database, a model server, a bridge that holds a session open. Your node returns a description of it, and the supervisor makes what runs match it: Docker containers on your machine, or a Compute Engine machine of its own on a cloud install.
"requiresInfra": true,
"images": ["images/credential"],
"publishes": "postgres"
async fn provision_infra(&self, ctx: InfraProvisionContext, input: ValueBag)
-> WeftResult<InfraSpec>
{
let database: String = input.get("database")?;
Ok(InfraSpec {
units: vec![Unit {
name: "db".into(),
containers: vec![Container { /* image, env, ports, probes */ }],
..Default::default()
}],
volumes: vec![Volume { /* the disk */ }],
endpoints: vec![Endpoint { name: "sql".into(), /* ... */ }],
..Default::default()
})
}
Declaring requiresInfra: true without writing provision_infra is an error
naming your node when the infra is provisioned, rather than a silent nothing.
The spec
| Field | What it holds |
|---|---|
units | What runs. Most nodes have exactly one |
volumes | Disks, kept across stop and upgrade by name, and scratch space emptied at each start |
endpoints | Named ports your node and the project’s other nodes reach it on, and whether anything outside the project may |
keep_on_terminate | Disks to keep even through a terminate, for the next weft infra start to pick up |
health_rules | What weft does to the project’s triggers when a unit of this node stops being ready, and when it undoes it. For the defaults and how to write your own, go and read when a unit stops being ready |
notes | If the copy you declare runs short of something it normally has (an older version kept for its data, say), one plain sentence each saying what and how to fix it. weft infra status prints them as warnings for as long as the copy runs. Usually empty |
A unit is a group of containers that run side by side, with optional init
containers that run one after another before them. It also says what machine
it needs (machine: cpu, memory, and gpu with a kind and a count)
and what stop does to it (onStop: stop it, or keep it running until
terminate). On a cloud install, weft picks the cheapest machine that holds
those numbers, shared-core ones included: a quarter CPU with 1Gi is
Compute Engine’s e2-micro. A number left unset is the sum of the
containers’ own limits. If you need one machine type in particular,
machine.kind names it (n2-highmem-8) and skips the pick. On your machine
the unit’s numbers cap nothing: only a container’s own limits.memory caps
it. weft refuses a unit there that asks for a GPU you do not have.
Whoever uses your node pays for that machine, so if the right size depends on
their load, let them set it. Declare the size your node needs by default in
its metadata.json, next to requiresInfra:
"requiresInfra": true,
"features": { "machineInputs": { "cpu": "1", "memory": "2Gi" } }
weft gives the node three inputs: cpu and memory, which default to those
sizes, and machineType, which has no default. A cpu or a memory
written in the program that does not read as a size (1, 500m; 2Gi,
512Mi) is refused at compile time, naming the input, and a wired one fails
the node when it arrives. Read the three into a MachineShape in provision_infra and put it on
the unit:
let machine = MachineShape::from_inputs(&input)?;
A blank machineType comes back as None, so the cloud picks the machine
from cpu and memory. The node fails to load if it declares one of the
three inputs itself, if it sets machineInputs without requiresInfra, or
if a default is not a size.
On your machine weft cannot pick a GPU by kind, so a unit that
asks for any GPU gets every GPU the machine has, and weft infra start and
weft infra status warn you about it.
A unit asking for one L4 GPU, with the CPUs and memory beside it:
Unit {
name: "model".into(),
machine: MachineShape {
cpu: Some("8".into()),
memory: Some("32Gi".into()),
gpu: Some(Gpu { kind: "nvidia-l4".into(), count: 1 }),
},
..Default::default()
}
The kinds a cloud install attaches are nvidia-l4 (1, 2, 4 or 8 on one
machine), nvidia-tesla-t4 and nvidia-tesla-p4 (1, 2 or 4), and
nvidia-tesla-v100 (1, 2, 4 or 8). Any other kind or count is refused when
the machine is picked, and the refusal says what is allowed. The CPUs and
memory you ask for must also fit a machine with that many GPUs.
Each container also takes two probes: readiness says when it is up, and liveness restarts it after
failureThreshold failures in a row.
The spec has to be stable
Write the same spec every time, for the same inputs. weft compares what you return against what is running, and re-applies when they differ.
So never generate a password inside provision_infra. Every call would produce
a different spec, and your database would be redeployed on every check.
If your container mints its own credential on first boot, that is what
ctx.publish_access is for: the container makes it, your node publishes it
once, and ctx.published_access() reads it back afterwards.
Local images
A Dockerfile in your node’s own folder, named in images:
postgres/database/
metadata.json "images": ["images/credential"]
images/credential/
Dockerfile
The last path segment is the name you reference in the spec. The path is relative to your node’s directory.
An upstream image reference passes through exactly as you wrote it, and a
mutable tag is not resolved to a digest. So postgres:16 rolling underneath
you produces no change in the spec and no re-apply. Pin by digest if you want
an image change to actually land.
Reaching it
let endpoint = ctx.endpoint("sql").await?;
let (host, port) = endpoint.host_and_port()?;
ctx.endpoint(name) resolves one of your declared endpoints and waits until
something actually answers there, rather than handing you an address the
moment the container reports ready.
| Call | Gives you |
|---|---|
url() | The address, cached, no round trip |
host_and_port() | The two apart, for a client that wants them that way |
call(method, path, body) | An HTTP request against it, retried once through a routing gap |
public_url() | For an endpoint exposed with Expose::Public, the address a caller outside the install uses; None for any other endpoint |
By default (Expose::Project), only the project’s own workers and
infrastructure can reach an endpoint. If you want anything else to reach it,
set expose:
Expose::SameNetworkopens it to programs on the install’s own network: anything running on your machine locally, the install’s private network on a cloud, but never the internet. A frontend uses this to reach a program’s database.Expose::Public { path: "/hooks" }opens it to the internet, over HTTP, at the install’s public address. Hand the caller (a provider delivering webhooks, say)public_url().
The install serves every project at that address, so your /hooks lives
under a prefix of its own: public_url() is
<address>/infra/<project>/<copy>/hooks (<copy> names this copy of the
node), and the install strips the
prefix again, so your container still sees /hooks.
weft infra status prints the same address under the node. On a cloud
install it is the install’s own address. On your machine it is the tunnel’s
address while the tunnel is open
(a public address), and
http://127.0.0.1:14111/... while it is closed, which nothing on the internet
can reach.
If an endpoint hands out a credential, keep it to the project. The Postgres
node shows the split: the port clients sign in on can be SameNetwork (its
reachable input), because reaching it still takes a password, but the small server that hands out that
password stays reachable only by the project.
It works during provisioning after the apply, and in every later phase once the
infrastructure is running. If the endpoint is not declared, or the
infrastructure is down, the error says which and points at weft infra status.
If your unit serves something with a fixed limit (a database that takes only so many connections), the waiting belongs in your unit, because it is the one place that sees every client, and the number of clients keeps changing (workers scale, and a frontend connects too). A PostgresDatabase runs a pooler in front of Postgres for this, in the same unit: the pooler never opens more connections than Postgres takes, a client that finds them all busy waits for one, and Postgres closes a session that sat idle for a minute, so a connection somebody holds without using it comes back. For what a node that talks to such a unit should do on its own side, go and read sharing something between runs.
When a program marks your node @per_instance, each instance of the program
gets its own container, and ctx.endpoint answers with the copy of the
instance the run is for; your node’s code does not change. Each copy has its
own address, so its public_url() is its own too. A connection your node
publishes (ctx.publish_access) from an instance’s copy is recorded as that
instance’s. For what an instance is, go and read
programs with instances.
Outputs saved with the infrastructure: baking
If an output only says where your infrastructure is or how to connect to it
(an address, a database connection), it stays the same from one run to the
next, so weft can save it when the infrastructure is applied and stop running
your node just to hand it out. Mark it baked in metadata.json:
{ "name": "access", "type": "Access[postgres]", "baked": true }
(Baked outputs have nothing to do with weft bake, which prepares a
trigger’s settings.)
Only an infrastructure node can bake an output: on any other node, the catalog refuses to load the metadata and names the output.
Every time the infrastructure is applied (weft infra start, weft infra upgrade, weft infra rebake <node>, or a program’s own
ctx.infra(..).start(&mut state)), weft runs your node’s run right after and saves what it sent on its baked outputs
alongside that infrastructure. A baked output your run sent nothing on keeps the value saved
last time.
While the infrastructure is running, if everything a run reads from your
node comes from baked outputs that have a saved value, your node does not
run: the saved values go down its wires instead.
Nodes that only fed your node do not run either. Your node’s log says why:
did not run: everything this run reads from it is baked, so its saved <outputs> went out instead. The Postgres node bakes access, so a route that
queries the database gets the saved connection without the Postgres node
running. If you want your node to run anyway (to debug it), name it in the
run: weft run --target <node> or --from <node> always runs it.
If a run also reads an output that is not baked (a bridge’s status, which
changes while the bridge runs), your node runs as usual, and if its run
sends nothing on a baked output, the saved value goes out on it instead of
the output closing. Only wires into nodes that are part of this run count: a
status wired to a node outside the run does not make your node run.
If your container changes a value itself (a password reset from a button,
say), have it POST the new value to the address in
WEFT_VALUES_URL:
curl -X POST "$WEFT_VALUES_URL" -H 'Content-Type: application/json' \
-d '{"connection": {"password": "the-new-one"}}'
connection changes values inside the connection your node published, and
outputs replaces saved baked outputs by name ({"outputs": {"<output>": <value>}}).
weft sets WEFT_VALUES_URL in every container of your unit, init containers
included, and refuses a spec that sets it. A push only ever changes the copy
whose container sent it. weft does not check a pushed value against the
output’s type. A saved value that does not fit its output’s type, pushed or
left over from older code, is never handed out: a run that reads it runs
your node instead, and if your run sends nothing on that output the run
fails saying what this node's infra saved for its baked outputs no longer fits them.
The request answers 200 once weft has written the value. If you name a key
the connection does not store or an output with nothing saved, or send
connection from a node that published none (or several), it answers 422
with a body that names the problem, and nothing is written. Any other answer
means nothing was written either, and 502 means weft could not be reached
at all.
If you want a button for this, give one of the items on your container’s
panel an action: {"label": "Reset password", "actionKind": "reset_password", "confirm": "<what to ask before pressing>"}. For serving a
panel, go and read showing something in the graph.
A press reaches your container as a POST to /action with {"action": "<actionKind>", "payload": null}, or with the button’s own payload if you
gave it one. For how to answer it, go and read letting other nodes reach
it.
If the push fails, or a value changed without anything telling weft, the
person running the program fixes it with weft infra rebake <node> (add
--instance <id> for a @per_instance node’s copy). It applies your
infrastructure again, restarting nothing that already matches its spec, then
runs your node’s run and saves what it sends. That only helps if your run
reads the current value from your infrastructure instead of trusting what it
published last time. The Postgres node does this: it asks its credential
container whether the password it holds is still the current one, and reads
the new one when it is not.
If your container pushes from a button and the push fails, answer the press
with {"result": {"error": "..."}} telling the person to run weft infra rebake <node>, the way the Postgres node’s Reset password button does. You
do not need to add that hint anywhere else: when any node fails after
opening the connection your node published, weft adds a line to that node’s
error naming weft infra rebake <node>.
If you change what your run emits on a baked output, runs that skip your
node keep handing out the old saved value until it bakes again. If you want
the new value out, weft infra upgrade builds your current source and
restarts each copy it reaches that still runs an older build, which bakes it
again. The triggers that go through that copy are down until it is back. If you would rather
restart nothing, build first
(weft build, or any weft run) and then run weft infra rebake <node>,
which runs whatever weft built last.
Without a flag, the upgrade touches the program’s shared infrastructure only.
If you want a @per_instance node’s copies to bake your new code, add
--instance <id> for some instances or --all-instances for every one;
neither touches the shared copies. If you want both at once, use --all. For what an
upgrade does to running work and to triggers, go and read infra commands
take down only what reads the
infra.
Letting other nodes reach it
If other nodes need to talk to your service (say, a WhatsApp bridge that
every send node goes through), pass them a handle instead of url(). The
address changes every time your service is set up again. The handle names
your node, as it is named in the program, and its endpoint, so it still
finds the service after that.
Declare an output typed Infra and set it to the endpoint’s handle:
{ "name": "bridge", "type": "Infra" }
let api = ctx.endpoint("api").await?;
ctx.pulse_downstream(NodeOutput::new().set("bridge", api.infra_handle())).await
A node that uses it declares an input typed Infra and resolves it:
let bridge: InfraHandle = ctx.inputs.get("bridge")?;
let bridge = ctx.endpoint_of(&bridge).await?;
let sent = bridge.action("sendMessage", json!({ "to": to, "text": text })).await?;
ctx.endpoint_of gives back the same handle ctx.endpoint does, so url()
and call(..) work on it as well. action(name, payload) posts
{"action": name, "payload": payload} to the service’s /action and gives
you the result of its answer. Your service answers {"result": ...}. action
returns an error when your service answers non-2xx, when the answer has no
result, or when result carries an error string (the error then reads
<action>: <error>). An error that is not a string comes back as part of
the result. The buttons of a live panel post to the same
/action, so one handler in your container serves both.
An Infra input only takes a handle from an Infra output. The compiler
refuses a String wired into it, and it refuses a value typed into it by
hand. A handle only works inside its own project. It only reaches
infrastructure the program still declares. For a node marked @per_instance,
it reaches the copy that belongs to the run’s instance.
A long job in your container
If your service runs something long (a render, a training run), have it answer
POST /jobs with an id at once and a status route like GET /jobs/<id>, then
wait on that route the way a node waits on a provider’s job: start it inside
a state.once and wait with PollEndpoint on format!("{}/jobs/{id}", api.url()).
For the whole pattern, go and read
waiting on a job a service runs.
api.url() is the address your workers reach, and the checks are made by the
listener, which on a local install sits on the machine rather than in Docker’s
network. Before every check it asks for the same endpoint’s address as weft’s
own roles reach it, looked up in this project’s infra only, so the address you
pass works as it is.
If somebody cancels the run while it waits, the job in your container carries
on. If you want it to stop too, give your container a cancel route and pass it
as the cancel call: CancelCall::post(format!("{}/jobs/{id}/cancel", api.url())). The listener reaches it the way it reaches the status route. For
what happens when your container refuses the call or does not answer, go and
read if the run is cancelled while the job
runs.
A live panel
If your container serves /live, name that endpoint and the editor shows the
panel on your node:
"features": { "liveEndpoint": "api" }
That is how a bridge shows the QR code you have to scan. Go and read showing something in the graph.
What the supervisor does
It takes an exclusive lease on the project, applies your spec, and watches whether what it created is healthy. For what happens when a supervisor dies halfway through a change, go and read the supervisor.
There is no time limit on coming up, because a model server pulling weights can
take a long time. Somebody waiting on yours reads weft infra logs <node> and
cancels if it will never come.
When a unit stops being ready
A unit that was ready and then stays not ready for its whole flaky window
(the unit’s health.flaky_after_seconds in Rust, flakyAfterSeconds in the
spec’s JSON; 30 seconds by default) is marked flaky. One that stays ready
again for health.recovery_after_seconds (recoveryAfterSeconds; also 30)
is marked running again. weft infra status shows which state each unit
is in.
What happens to the project while a unit is flaky is up to your node’s
health_rules. If you write none, your node gets these two:
- take-down-while-flaky: while a unit of the copy is flaky and the project’s triggers are on, take down every trigger whose cut contains this copy (the cut is the part of the program a trigger’s runs go through). Work that arrives for those triggers is held until they are back, work they were already holding is kept, and the runs reaching the copy are cancelled, since they cannot go on without it. A trigger whose cut does not contain the copy keeps running.
- bring-back-once-healthy: once no unit of the copy is flaky, bring back what the health check took down, but never what a person or an infra command took down.
Each rule reads one copy at a time, and only that copy’s own units, so one instance’s broken copy never takes down another instance’s triggers.
If that is not what your node needs, write your own. Say your node runs a
database unit db and a small admin panel panel beside it. With the two
rules above, the panel going flaky would take down every trigger that uses
the database. Name the unit and only db counts:
use weft::infra::health::{HealthAction, HealthCondition, HealthRule, DEFAULT_RULE_TIMEOUT_SECONDS};
let db_flaky = HealthCondition::UnitFlaky { unit: Some("db".into()) };
let health_rules = vec![
HealthRule {
name: "take-down-while-db-flaky".into(),
when: HealthCondition::All { conditions: vec![db_flaky.clone(), HealthCondition::Listening] },
action: HealthAction::TakeDownTriggers,
timeout_seconds: DEFAULT_RULE_TIMEOUT_SECONDS,
},
HealthRule {
name: "bring-back-once-db-healthy".into(),
when: HealthCondition::All {
conditions: vec![HealthCondition::Not { condition: Box::new(db_flaky) }, HealthCondition::TakenDownByHealth],
},
action: HealthAction::BringBack,
timeout_seconds: DEFAULT_RULE_TIMEOUT_SECONDS,
},
];
Ok(InfraSpec { units, health_rules: Some(health_rules), ..Default::default() })
A condition is one of these, nested as deep as you like:
| Condition | Holds when |
|---|---|
UnitFlaky { unit } | That unit of the copy is flaky (unit: None: any of its units). A unit not reported yet is not flaky |
Listening | Some trigger of the project is on, and none is being turned on |
TakenDownByHealth | Something the health check took down is still down. A person’s stop or deactivate clears the mark, so a rule gated on this never undoes what a person did |
All { conditions } | Every one of them holds (an empty list holds) |
Any { conditions } | One of them holds (an empty list does not) |
Not { condition } | It does not hold |
The action is TakeDownTriggers or BringBack. Give each rule a name no
other rule of this node uses: weft keeps each rule’s state under its name,
and weft status names it if the rule gives up. A repeated name is refused
when the spec is applied.
Rules are read in the order you write them. A rule fires once for a copy when its condition starts to hold, and can fire again only after the condition stopped holding for that copy. When two hold at once for a copy, only the first fires. The second fires on a later check, once the first has fired or stopped holding; while the first is retrying or has given up, the second waits.
Firing a rule means the supervisor asks weft to queue its action, and the
rule counts as fired while that action runs. The action fails if the request
to queue it fails, if weft cannot carry it out, or if it has not finished
within timeout_seconds of being queued. Each time weft finishes a health
action, the supervisor takes one more look, so a failure is noticed at once.
A failed action is tried again 5 seconds later, then 10 seconds after that. On
the third failure in a row weft gives up on the rule, and weft status names
it with the last error. A rule that gave up tries again when any unit of the
copy changes between ready, not ready and flaky, when the rule stops holding,
when somebody starts, stops, upgrades or terminates the project’s infra, or
when somebody runs weft infra look.
Set timeout_seconds to DEFAULT_RULE_TIMEOUT_SECONDS (30 minutes) unless
you want another limit. It is a NonZeroU32, so for ten minutes write
NonZeroU32::new(600).unwrap().
When a bring-back fires, a trigger stays down if its cut contains a running copy that is still broken: one whose node has a bring-back rule that does not hold right now, or one whose own take-down took down something that is still down. A healthy copy whose node writes no rules never holds anything down.
If you want a flaky unit to take nothing down, set health_rules: Some(vec![]). A node with take-down rules but no bring-back rule keeps down
what it took down itself until a person runs weft activate.
A rule that names a unit the spec does not declare is refused when the spec is applied:
infra node 'database': health rule 'take-down-while-db-flaky' reads unit 'dbb',
which is not declared; name one of the node's units, or leave `unit` out to
read any of them
For what the person running the project sees when the health check took a trigger down, and how to bring one back by hand, go and read infra commands take down only what reads the infra.
Two things a user has to do themselves
Nothing starts your infrastructure for them. Not a run, not a build, not
activation. weft infra start is a verb they type, because a container costs
money and weft will not spend it on a click that did not mention one.
And weft infra stop keeps the disk while weft infra terminate deletes it.
If your node has data worth keeping even through a terminate, name its disk in
keep_on_terminate. The next weft infra start of the same node (the same
instance’s copy, for a node marked @per_instance) finds that disk and mounts it
again, data and all. If you want a kept disk gone, drop it from
keep_on_terminate, start the node once so the copy knows about it, and
terminate.
A kept disk is deleted for you once its copy can never come back: when you
remove the node from your program and run it again, when the node switches
between shared and @per_instance (which leaves the old side’s copies behind), or
when you remove the project with weft rm. This holds even if the copy was already terminated
at the time. The supervisor sweeps for such copies on every ownership tick, so
the disk goes within one tick of the change.
If you wipe an instance (the WipeInstance node, or ctx.infra(node).instance(id).wipe(..)
from your own node), the kept disks of that instance’s copies go right away,
including those of a copy that was already terminated.
Testing one
let outcome = rig.run_provision_infra(&PostgresNode, json!({ "database": "app" })).await.ok()?;
let spec = outcome.infra_spec()?;
assert_eq!(spec.units[0].name, "db");
fake is the top tier here too. For a node that calls its own
infrastructure, declare where each endpoint answers, then what each call gets:
rig.declare_endpoint("credential", "http://localhost:8080");
rig.refuse_endpoint("credential", EndpointMethod::Get, "/password", 503, "starting");
rig.answer_endpoint("credential", EndpointMethod::Get, "/password", json!({ "password": "minted" }));
let outcome = rig.run(&DatabaseNode, json!({ "database": "app" })).await.ok()?;
assert_eq!(rig.endpoint_calls().len(), 2, "asked twice: refused, then answered");
| Call | What it does |
|---|---|
declare_endpoint(name, url) | Says the endpoint name answers at url. Without it, ctx.endpoint(name) fails the way it does when the infrastructure is not running |
declare_public_url(name, url) | Makes a declared endpoint public: ctx.endpoint(name)?.public_url() answers url |
answer_endpoint(endpoint, method, path, answer) | The next call to path on that endpoint answers answer (JSON) |
refuse_endpoint(endpoint, method, path, status, body) | The next call to path on that endpoint is refused with status and body, the way a service that is still starting refuses one |
declare_infra(place, endpoint, url) | Use this when your node reads an Infra input. It pretends the infrastructure node place shares endpoint at url, and returns the handle to set on that input |
answer_infra(place, endpoint, method, path, answer) | Queues the answer to the next call to path on that shared endpoint, one per call, the same way answer_endpoint does |
endpoint_calls() | Every call the node made to an endpoint, its own or a shared one, in order: place, endpoint, method, path, body |
stop_after_calls(n) | Presses stop, as weft stop would, once the node has made n calls (endpoint calls and web requests, counted together). The call that reaches n still gets its answer; 0 stops the run before it starts. This is how you test a node that loops on calls and must end cancelled when a person stops it |
method is EndpointMethod::Get or EndpointMethod::Post.
Each answer and each refusal is used by exactly one call, in the order you
declared them, so declare one per call you expect. That is how a node that
asks twice and acts on the answer changing (refused, then answered) gets
tested. A call with nothing left to answer it fails the run and names
answer_endpoint, so a question your node should not have needed to ask shows
up instead of being quietly answered.
Testing a node
Tests live beside the node, in a tests.rs in its own folder. An access node
has none: the access_node! macro is its whole body, so there is nothing of
yours to test.
tests.rs:
pub fn tests() -> Vec<NodeTest> {
vec![NodeTest::fake("counts_words", counts_words)]
}
async fn counts_words(rig: FakeRig) -> WeftResult<()> {
let outcome = rig.run(&WordCountNode, json!({ "text": "one two three" })).await.ok()?;
assert_eq!(outcome.outputs["count"], json!(3.0));
Ok(())
}
mod.rs bridges it in:
#[cfg(feature = "node-tests")]
mod tests;
#[async_trait]
impl Node for WordCountNode {
#[cfg(feature = "node-tests")]
fn tests(&self) -> Vec<weft::NodeTest> {
tests::tests()
}
// ...
}
There is no registry and no list to maintain. The runner walks the nodes the binary already has and asks each one for its tests.
weft test-node WordCount # one node
weft test-node slack # one package
weft test-node slack WordCount # several, built together
weft test-node # everything
Three tiers
| Tier | What it gets | Costs |
|---|---|---|
basic | Nothing. A plain function testing your own logic | Nothing |
fake | Canned HTTP, in-memory storage, canned signals, fake connections | Nothing |
live | The real runtime: real connections, real calls, real metering | Real money |
Basic and fake run by default. Live never runs unless you ask, and the runner confirms before it does.
Basic and fake run right there with cargo: they need no Docker and no runtime, and only the packages you targeted get built.
Two things to know. A basic test is a plain sync function and the runner is
already inside an async runtime, so building one inside it panics. And for a
trigger or an infra node, fake is the top tier: the live rig drives a plain
run body, so a live test on those is refused.
A fake test that never finishes fails by name after 30 seconds, and tells you where to look:
test 'reads the stream' did not finish within 30s. Something in it is waiting
for what it never gets: a cursor reading a bus nothing closes, a caller nobody
attaches, a signal nobody answers.
The fake rig
| You want | Call |
|---|---|
| Run the node | rig.run(&node, json!({...})) |
| Run its trigger setup, or its infra | rig.run_setup_trigger(...), rig.run_provision_infra(...) |
| Canned HTTP | rig.respond(method, path, body), rig.respond_status(...), rig.respond_raw(...), rig.respond_with_headers(...) for an answer that lives partly in a header (a created id, an ETag) |
| A server it cannot reach at all | rig.fail_connection(method, path): the call errors with no status and no body |
| Check what it sent | rig.requests(), rig.assert_sent(method, path) |
| What it saved, and a resume | rig.saved_state(&node) is the state its body last saved. Run the node a second time on the same rig and it picks the firing up the way a resumed run does: the body gets that state back, a wait it already made hands back its answer, and a stream input skips what it took. So a publish that must happen once is something you can prove: two runs, one request. Each node keeps its own, so two nodes on one rig never pick up each other’s |
| A connection | rig.access("slack"), rig.connection_value(...), rig.connection_permissions(...) |
| A trigger’s event | rig.wake(payload), rig.signal(payload) |
| A live caller | rig.attach_caller(conn) |
| A bus going in | rig.seed_bus(opts), giving you a writer and a marker |
| A bus it emitted | rig.bus(&outcome.outputs["channel"])?, then read it from its oldest entry with .cursor_from_start()? and cursor.next(&mut HarnessRead) (weft::context::HarnessRead, for test code only: a node body handing it to a read fails) |
| A stored file | rig.store_file(filename, mime, bytes) going in; rig.stored_bytes(key), rig.stored_meta(key) and rig.stored_files(scope) for what the node stored; rig.file_edits() for every in-place change, with the file, the versions and the diff |
| An infra endpoint | rig.declare_endpoint(name, url), rig.answer_endpoint(...), rig.refuse_endpoint(...) |
| What it asked for | rig.registered_signals(), rig.awaited_signals(), rig.logs(), rig.run_tags(), rig.stops() |
| A port’s type, for a node that takes added ports | rig.output_type(port, ty), rig.input_type(port, ty) |
| An output something downstream reads | rig.wire_output(port). Nothing is wired unless you say so. rig.wire_output("_error") makes a failure come out on _error, as it would in a run, and outcome.output("_error") reads it |
| A server that is there and will not talk, for a node that opens its own socket | rig.hang_up_server(), a local port that hangs up on every connection. It is for raw socket clients (a database driver, a mail client): web calls in a fake test never reach the network, so for those use rig.fail_connection. If you want a refused connection, use this rather than a closed port: on some machines a closed port hangs until a timeout |
The outcome gives you .ok() for the happy path, .failure() for the error
string, .paused() for a run that paused in the body, .output(port) for one port, .infra_spec() for an infra run, and
.outputs for the map. The rig refuses a value over 100 KB with the same error
a run gives, so a test catches an output no wire would carry.
Note the last group: in a fake test nothing is really stopped and nothing is really tagged. The rig records what your node asked for, and that is what you assert on.
In a node test, every state.wait takes its answer from what you queued with
rig.signal, in the same call. If you want to check what a resume does, run
the node twice on one rig. This one waits for a click, then posts, and the
second run finds both in its saved state:
rig.respond("POST", "/api/send", json!({"id": "post-1"}));
rig.signal(json!({"ok": true}));
let inputs = json!({"account": rig.access("probe")});
rig.run(&AskThenPost, inputs.clone()).await.ok().expect("waits, is answered, posts");
let again = rig.run(&AskThenPost, inputs).await.ok().expect("picks up from its state");
assert_eq!(again.outputs["reply"], json!({"id": "post-1"}));
assert_eq!(rig.requests().len(), 1, "one post across the two runs");
assert_eq!(rig.awaited_signals().len(), 1, "one wait across the two runs");
If you want to prove the node carries on across a pause without doing again
what it did before, queue rig.signal_paused(): the node’s next wait pauses
the run there, as production does once every node waits, and
outcome.paused() says so. The next rig.run of the node picks the firing up
at that wait and takes the next answer you queued. A body that does something
else first (reads a bus, yields, saves) is refused with the same message as in
production:
rig.respond("POST", "/api/send", json!({"id": "post-1"}));
rig.signal_paused();
rig.signal(json!({"ok": true}));
let inputs = json!({"account": rig.access("probe")});
let first = rig.run(&PostThenAsk, inputs.clone()).await;
assert!(first.paused(), "paused on its wait");
rig.run(&PostThenAsk, inputs).await.ok().expect("picks up at the wait");
assert_eq!(rig.requests().len(), 1, "posted once across the pause");
A node that creates a bus gets the same bus back on its second run, as a resumed body does.
Rig reference
If you need a rig call’s exact signature, or the fields of what it hands back, it is here.
The fake rig
Setting the run up:
| Signature | What it does |
|---|---|
respond(&self, method: &str, path: &str, body: Value) | Answers a matching request with 200 and body as JSON. path may carry a query; parameter order and encoding do not matter |
respond_status(&self, method: &str, path: &str, status: u16, body: Value) | The same with your status |
respond_raw(&self, method: &str, path: &str, status: u16, content_type: &str, body: impl Into<Bytes>) | Raw bytes with a content type (XML, CSV, a picture, an event stream) |
respond_with_headers(&self, method: &str, path: &str, status: u16, content_type: &str, headers: &[(&str, &str)], body: impl Into<Bytes>) | The same plus response headers. The content type is its own argument, never a header here |
fail_connection(&self, method: &str, path: &str) | A matching request fails before any response, like a server that is down |
signal(&self, payload: Value) | Queues the answer to the node’s next state.wait, in order |
signal_paused(&self) | Queues a pause instead: the node’s next state.wait pauses the run there (outcome.paused() is true), and the next rig.run of the node picks it up at that wait with the next queued answer |
signal_given_up(&self) | Queues a wait given up instead: the node’s next state.wait fails with WeftError::WaitGaveUp, the way it does when a run that cannot pause holds the wait past its holdSecs |
wake(&self, payload: Value) | The next run’s ctx.wake: what a firing trigger received |
instance(&self, id: &str) | Makes every run on the rig a run for that instance |
attach_caller(&self, conn: Arc<FakeCallerConnection>) | Puts a scripted live caller on the runs, what a Route or Socket run carries |
seed_bus(&self, opts: BusOptions) -> WeftResult<(BusHandle, Value)> | A bus you write into, and its marker to put on a Bus input |
access(&self, service: &str) -> Value | A connection marker for an access input |
connection_value(&self, service: &str, name: &str, value: &str) | A value the opened connection answers (conn.value(name)) |
connection_permissions(&self, service: &str, granted: &[&str]) | What the connection has granted, checked against what the input requires |
published_connection(&self, service: &str, values: &[(&str, &str)]) | Says the node already published this connection on an earlier run |
output_type(&self, port: &str, ty: WeftType) | Declares the type of an output, the way a program does on a narrowable output (-> (response: Verdict)) or on one it adds. If the compiler would refuse that type (an output that is not narrowable takes only its own), the run fails with the case declares output ... |
input_type(&self, port: &str, ty: WeftType) | Declares an added input port and its type |
wire_output(&self, port: &str) | Wires an output downstream. Nothing is wired unless you say so |
hang_up_server(&self) -> u16 | A local port that hangs up on every connection, for raw socket clients |
declare_endpoint(&self, name: &str, url: &str) | Where the node’s own infrastructure endpoint name answers |
declare_public_url(&self, name: &str, url: &str) | Makes a declared endpoint public at url |
answer_endpoint(&self, endpoint: &str, method: EndpointMethod, path: &str, answer: Value) | The next call to that endpoint and path answers answer |
refuse_endpoint(&self, endpoint: &str, method: EndpointMethod, path: &str, status: u16, body: &str) | The next call to that endpoint and path is refused |
answer_program_call(&self, name: &str, value: Value) | What the next program call of that name answers (weft.infra.status, …), in order |
store_file(&self, filename: &str, mime_type: &str, bytes: impl Into<Vec<u8>>) -> Value | Stores a file in this run’s scope and gives its value, for a file input |
store_file_in(&self, scope: &StorageScope, filename: &str, mime_type: &str, bytes: impl Into<Vec<u8>>) -> Value | The same in another scope |
Running it:
| Signature | What it does |
|---|---|
run(&self, node: &dyn Node, inputs: Value) -> RigOutcome (async) | Runs the node’s body with inputs (a JSON object of input name to value). A second call for the same node picks its firing up from the state it saved |
run_setup_trigger(&self, node: &dyn Node, inputs: Value) -> RigOutcome (async) | Runs a trigger’s setup |
run_provision_infra(&self, node: &dyn Node, inputs: Value) -> RigOutcome (async) | Runs an infra node’s provision_infra |
Looking at what happened:
| Signature | What it gives you |
|---|---|
requests(&self) -> Vec<SentRequest> | Every web request the node sent, in order, answered or not |
assert_sent(&self, method: &str, path: &str) | Panics, listing what was sent, unless one request matched |
stored_bytes(&self, key: &str) -> WeftResult<Bytes> | The content stored under a key |
stored_meta(&self, key: &str) -> WeftResult<StoredFileMeta> | What storage records for it: its deadline (expires_at_unix, keep_ttl_secs), the size |
stored_files(&self, scope: &StorageScope) -> WeftResult<Vec<StoredFileMeta>> | Every file stored in a scope |
file_edits(&self) -> Vec<FileEdit> | Every change made to a stored file in place, in order: the file, the versions, the diff |
bus(&self, marker: &Value) -> WeftResult<BusHandle> | The bus behind a marker the node emitted, as a handle of the test’s own |
endpoint_calls(&self) -> Vec<EndpointCall> | Every call to the node’s own infrastructure, in order |
registered_signals(&self) -> Vec<(SignalSpec, Value)> | What a trigger’s setup registered |
awaited_signals(&self) -> Vec<SignalSpec> | Every signal the node parked on |
saved_state(&self, node: &dyn Node) -> Option<Value> | The state the node’s body last saved (ctx.keep), as JSON; None before it saved anything |
logs(&self) -> Vec<(LogLevel, String)> | Every ctx.log line |
run_tags(&self) -> Vec<Vec<String>> | The tags, one list per ctx.tag_run |
stops(&self) -> Vec<(String, StopSelf)> | Every ctx.stop_tagged asked for |
program_calls(&self) -> Vec<(ProgramCall, StopSelf)> | Every program call made |
minted_tokens(&self) -> Vec<MintedToken> | Every instance token minted: instance, expires_in_secs, id |
published_values(&self, service: &str) -> Option<BTreeMap<String, String>> | What the node published as that service’s connection. A publish is held to the same rule as on an install: a value the service does not declare, a required one missing, or one the service’s identity names left blank fails the run. The service’s recipe is found in the whole catalog, as a project build finds it, so a node can publish a service another package declares; if no package declares it, the test build stops with an error naming it |
fresh(&self) -> FakeRig | A second, empty rig that knows the same services, for a test that needs two |
The live rig
| Signature | What it does |
|---|---|
access(&self, service: &str) -> Value | The test’s connection marker. The service must be the one the test declared |
fixture(&self, name: &str) -> WeftResult<String> | The value of WEFT_NODE_TEST_<name>, or a failure naming the variable to set |
store_file(&self, filename: &str, mime_type: &str, bytes: impl Into<Vec<u8>>) -> WeftResult<Value> (async) | Stores a real file and gives its value |
connect(&self) -> WeftResult<OpenedConnection> (async) | Opens the test’s connection, for setup and cleanup calls of your own around the node |
bus(&self, opts: BusOptions) -> WeftResult<(BusHandle, Value)> | A real bus you write into, and its marker |
wire_output(&self, port: &str) | Wires an output downstream |
run(&self, node: &dyn Node, inputs: Value) -> RigOutcome (async) | Runs the node against the real runtime |
LiveRig::new(..) is built by the runner, never by a test.
What comes back
RigOutcome, from every run:
| Field or method | What it is |
|---|---|
result: WeftResult<()> | The body’s own result |
outputs: serde_json::Map<String, Value> | What each port emitted |
closed_ports: Vec<String> | Ports the body closed with ctx.close_port |
infra_spec: Option<InfraSpec> | The spec, on a run_provision_infra outcome |
ok(self) -> WeftResult<Self> | The outcome, or the body’s failure |
failure(self) -> WeftResult<String> | The failure’s message, or an error when the run succeeded |
output(&self, port: &str) -> WeftResult<&Value> | One port’s value, or an error naming the ports that did emit |
infra_spec(&self) -> WeftResult<&InfraSpec> | The spec, or an error when this outcome has none |
SentRequest, one per request:
| Field or method | What it is |
|---|---|
method: String | Uppercase, "POST" |
path: String | The path, no query |
query: Option<String> | The raw query string |
body_text: Option<String> | The body as text, when it was sent whole |
body: Option<Value> | The body parsed as JSON, when it parses |
body_streamed: bool | True when the body was streamed, which the rig cannot read: body and body_text are then None even though there was a body |
headers: Vec<(String, String)> | The headers, names lowercased |
header(&self, name: &str) -> Option<&str> | One header’s first value, any case |
EndpointCall, one per call to the node’s own infrastructure:
| Field | What it is |
|---|---|
endpoint: String | The endpoint’s name |
method: EndpointMethod | Get or Post |
path: String | The path called |
body: Option<Value> | The body sent |
Live tests
weft test-node SlackSendMessage --tier live
The runner tells you it will spend money and asks. --yes skips that, and it
can remember your answer.
A live test runs on your machine’s own install: the runner compiles a test
server for the node’s package with your own Rust and starts it there as a
program. Pointed at an install that is not on this machine (--on a cloud
install, or --dispatcher an address no install here answers at), it
refuses,
saying live node tests run on this machine's own install. Before it starts
a test server, it checks what the tested nodes need while they run, and refuses
with the same header and install line as a refused run when something is missing (for those,
go and read what nodes need from your
machine).
For credentials, --connection slack=<grant-id> uses one you already have, and
--key slack takes a throwaway one: each field comes from
WEFT_NODE_TEST_SLACK_<FIELD> if it is set, or a prompt, never from the
command line where other processes can read it. The grant is deleted
afterwards.
Every prompt happens before any credential exists and before any signal handler is armed, so Ctrl+C at a prompt leaks nothing.
If a live test needs a real thing to act on (a chat it may post in, an
account it may read), declare it as a fixture:
.with_fixture(fixture_spec("CHAT_ID", "Chat", "a chat the bot may post in")),
which is a name, a label and a description. Set WEFT_NODE_TEST_CHAT_ID in
the shell you run weft test-node from, and the test reads it with
rig.fixture("CHAT_ID"). If any fixture is missing, the runner lists every
variable to set, with its label, before it runs anything.
If your node waits on a signal (state.wait), a live test keeps the
node’s call open and waits for the real answer, the way a run that cannot
pause does, except that it
always holds for the longest time allowed, 30 days, whatever holdSecs the
project sets.
What to write
Ship a fake test for every node. If it talks to a provider, ship a live one
on the cheapest real path you can find.
A panic fails that test, carrying its message, and never takes the runner down.
Packaging
Several nodes that belong together, sharing code and dependencies.
slack/
package.toml
api.rs shared code
metadata.json optional defaults for every node here
access/
metadata.json
mod.rs
send_message/
metadata.json
mod.rs
deps.toml
package.toml is short:
[package]
name = "slack"
[dependencies]
# cargo deps every node here gets
Its presence is what makes the folder a package, and the members are found by looking rather than listed.
Shared code
Any .rs at the package root becomes a module, and a node reaches it as
super:
use super::api;
That is where the request-building and the error-reading for one service goes, written once instead of in every node that talks to it.
Each package compiles as its own crate, so a node sees its own package’s shared files and nothing else. Another package’s code is never on your path.
It also means editing one node recompiles that package and relinks, rather than rebuilding the world.
A shared file whose name is not a valid Rust identifier fails at build time naming the file.
Defaults
A metadata.json at the package root is a partial document every node
inherits, key by key, with the node’s own value winning.
Put types, tags, an icon, a color or a service recipe there once.
type, label and description can never be defaults. They are one node’s
identity, and the compiler refuses a package file carrying any of them:
package-level metadata.json must not set `type`: it is one node's identity,
not a package default
The merge is shallow: a node’s types replaces the package’s rather than
adding to it.
When a folder does not load
If a node or a package has something wrong with it, only that folder is left out. Every other node still loads, and a program that never uses the broken one still compiles. A program that does use it fails with the reason:
node 'Poll' failed to load: package name 'feed' is used by more than one
folder (nodes/rss/feed, nodes/mine/feeds); ...
Names have to be unique across all your node folders: a node type, a service,
and a package name (a package is named by its package.toml, a bare node by
its folder). If two folders claim the same name, both are left out and each
error names the other, so you pick which one to rename.
If you write a package.toml before any of its nodes, the package just waits,
and it loads once a node folder with a metadata.json and a mod.rs is there.
weft describe-nodes lists everything that was left out, and why.
deps.toml
Per node, and every section is optional.
[dependencies]
reqwest = { workspace = true }
[[needs]]
name = "ffmpeg"
check = ["ffmpeg", "-version"]
[needs.install.apt]
default = ["ffmpeg"]
[needs.install.brew]
default = ["ffmpeg"]
[build.env]
SOME_PATH = "{{catalog_path}}/vendor"
| Section | What it is for |
|---|---|
[dependencies] | Cargo crates this node needs |
[build-dependencies] | Cargo crates for the package’s build.rs (see a build script) |
[[needs]] | Something the machine has to have: a program the node starts, or a library its package needs to compile (see what a node needs from the machine) |
[build.env] | Environment during cargo build. {{catalog_path}} expands to the path of your node’s folder inside the build |
Any other section fails the build, with an error naming the node type and the key.
[build.env] only sets variables. If you need build logic, write a build
script.
Cargo reads these variables too, so RUSTFLAGS works. If you set
-C target-cpu=native, it compiles for the machine doing the build: on a local
install that is the machine the worker runs on, but a cloud install builds the
worker on one machine and runs it on another, which may not have every
instruction the build used.
If a project uses a node that sets [build.env], it compiles in a compile
cache kept for those nodes and their values, apart from the one every other
project shares, so its first build compiles more. On a local install,
weft clean --images --all removes those caches.
What a node needs from the machine
If your node starts a program (python3, ffmpeg), or its package needs a
library to compile, that has to be on the machine the worker runs on, which
on a local install is your own computer. Declare each one with a
[[needs]] entry:
| Key | What it says |
|---|---|
name | Required. What the user is told the node needs |
check | Required. A command that exits 0 when the need is met. Its first word is looked up on the PATH, and a check still running after ten seconds counts as unmet |
stage | "run" (the default) for what the node starts while it runs, "build" for what its package needs to compile, like a C library’s headers |
[needs.install.<manager>] | The packages that meet it. Managers are apt (Debian, Ubuntu), dnf (Fedora, RHEL), apk (Alpine) and brew (macOS) |
Under each manager, default names the packages for every system that
manager serves. A key made of the ID and VERSION_ID from
/etc/os-release, dots turned to underscores (debian_12,
ubuntu_24_04, fedora_40), names them for one system whose package names
differ. weft looks for that exact key first and falls back to default.
A local install runs the run checks of a program’s nodes before the
program runs, activates or resyncs, and refuses it when one fails. For what
the person sees then, and how weft deps-install fixes it, go and read
what nodes need from your
machine. Nothing
checks a build need before your package compiles, though: if one is
missing, the package fails to compile and the build leaves it out (see
when a package does not compile).
On a cloud install the worker is an image, and its build installs the same
packages: run needs into the image the worker runs in, build needs into
the one that compiles it, with the entry for the image’s package manager.
The image starts from debian:trixie-slim unless the project’s
weft.toml names another in [build.worker] base_image (for that setting,
go and read the worker), so the
entry is usually apt. If your apt entry has keys but neither
debian_13 nor default, the cloud build fails naming the node. If a need
has no entry for the image’s package manager at all, nothing on a cloud
install can provide it, so a cloud install refuses any program that uses
that node. The same goes for every run need when
the project’s own [build.worker] dockerfile_template drops weft’s
{{install_runtime_system_packages}} line, which is where those packages
are installed.
If the program is gone by the time your node starts it (somebody
uninstalled it after the check), fail with
weft::error::need_missing("<the need's name>"), the way ExecPython does
for python3. weft adds your node and the way to meet the need on that
install, so the person reads the same lines a refused run shows.
A build script
If your package needs one, put build.rs at the package root (for a node with
no package.toml, in its own folder). It runs when cargo builds your package’s
crate, and the [build-dependencies] of every node’s deps.toml in the
package are its dependencies. A [build-dependencies] section in a package
with no build.rs at its root fails the whole build, naming the package.
Your package is a library crate, so its build script reaches only what cargo lets a library’s build script reach:
| The script | It reaches |
|---|---|
Prints cargo:rustc-env=... or cargo:rustc-cfg=..., or writes files to OUT_DIR | Your package’s own code |
Prints cargo:rustc-link-lib=... or cargo:rustc-link-search=... | The worker binary that links your package |
Prints cargo:rustc-link-arg=... | Nothing: the worker binary never sees it |
So if your package links a native library, name it with rustc-link-lib and
rustc-link-search rather than passing raw linker arguments.
When a package does not compile
If one package’s Rust does not compile, its script fails, or one of its
dependencies does not build, weft builds the worker without that package rather
than failing the whole build. That needs cargo to print an error that traces
to the package: when it stops before blaming any (crates.io or the network
unreachable, the machine out of disk), the whole build fails and says so, and
building again once that is over tries every package. The command you ran (a weft run, say) still does its job if it does not
need that package,
and ends with a warning naming the package and the first error cargo printed:
the worker was built without it, so nothing that uses its nodes runs: package
'media' didn't compile: error[E0308]: mismatched types; its full errors are in
the build log: <where the log is>
A run whose nodes never use the package runs as usual. A run, a trigger firing or a trigger setup that needs one of its nodes is refused before it starts, with the same reason. The package is built again once its files change, so fix it and run the command again.
What you get without asking
weft, tokio, serde, serde_json, async-trait, anyhow and tracing
are always there. So is weft-providers.
weft also re-exports serde_json, reqwest, reqwest_middleware,
async_trait and inventory, so you can name those types without declaring a
dependency on them.
Local images
An infra node’s containers are built from Dockerfiles in its own folder:
postgres/database/
metadata.json "images": ["images/credential"]
mod.rs
images/credential/
Dockerfile
bootstrap.py
The path is relative to the node’s own directory, not the package root, and the last segment becomes the name you reference in the spec.
How connections work
A credential never appears in your program. It lives in weft’s own store, and what travels a wire is a reference with nothing secret in it.
Four things that sound alike
| What it is | Who writes it | |
|---|---|---|
| The recipe | How a credential is obtained, how a request is signed, what the permissions are, how events arrive. Pure declaration, no code | Whoever wrote the access node |
| A registered app | One OAuth application: its client id, its secret, the permissions it asks for | The operator, in the apps file |
| A connection | One account somebody connected. Its sealed values, its permissions, who it belongs to | The person who connected it |
The Access value | The reference your program passes around | The runtime, when it builds a node’s inputs |
For OAuth the first two are different things: the app is the software asking, and the connection is the account that said yes. A pasted API key needs no app and is still a connection, and so is a hostname, a username and a password for a mail server.
What actually travels
{
"__weft_access__": {
"accessId": "11111111-2222-3333-4444-555555555555",
"service": "slack",
"identity": "Acme",
"requiresPermissions": ["chat:write"]
}
}
The runtime stamps the requires lists on per consuming node, from what
that node’s input said it needed: its requiresScopes becomes
requiresPermissions, and its requiresValues stays requiresValues, each
there only when the node asked for some. So the same connection
carries different requirements depending on which node is about to use it.
When that node opens it, a missing stored value is refused, and so is a missing
permission on a verified connection (one whose provider said what it granted),
rather than failing halfway through a call.
Shared, or your own
| Door | What it is |
|---|---|
shared | One click on a credential weft holds: a registered application, or the runtime’s own key, spending its credit |
own | You bring your own. A page with the paste fields, and sometimes a guide or a button that creates the application for you |
own is the default. Offering shared is something a service’s recipe has to
declare, and what sits behind it is whatever the operator put in
the apps file.
A permission a recipe marks as only working on your own account can never be served by the runtime’s credential. The editor greys that option out and resolution refuses it.
An instance’s own connections
In a program with instances, an access node whose connection is written
account: @instance_filled holds no connection in the source: each instance
gets its own, and a run for that instance uses it. For how that works, go and
read
programs with instances.
What weft can actually tell you about a credential
Providers differ in how much they admit, and weft records which kind of answer it got.
| What the provider does | Is it authoritative? |
|---|---|
| States exactly what it granted | Yes |
| Lets weft ask the credential to describe itself | Yes |
| Confirms only that the credential is alive | No |
| Says nothing at all | No |
That distinction decides what gets blocked. A permission weft knows is missing blocks the node that needs it. A permission that is only claimed, or unknown, lets the call through, because refusing would block every pasted key on every service that reports nothing.
Stored values are simpler and have no unknown case: if a node needs
imapHost and the connection has not got one, the node never starts.
Connecting also runs the service’s own check, but only when that check is free. Anything paid, or ambiguous because it might bill depending on the account, is skipped, so a bad credential sometimes shows up for the first time when a node tries to use it.
One thing that is always checked: if you tick permissions and the provider hands back fewer, the connect is refused naming the one you did not get.
Refreshing
Lazily, when a node opens the connection, and only if the token expires within the next minute. There is no background keep-alive.
Two things opening the same connection at once queue behind each other, so it refreshes once and both get the fresh values.
How it refreshes depends on the recipe. A sign-in sends its refresh token to
the token endpoint, a client_credentials app asks for a new token, and a
recipe that declares its own refresh call runs that instead. Whichever it
is, the answer is read the way the recipe says, the new token and expiry are
stored, a new refresh token replaces the old one when the provider rotates
them, and the recipe’s captures update what they find. For those keys, go and
read declaring a service.
Only a token with a known expiry is ever refreshed, which is why a connect is
refused when the provider’s answer names none and the connection could be
renewed. A recipe whose provider says its tokens never expire declares
neverExpires.
If a refresh answer names no expiry, or one weft cannot read (a lifetime of
0 included), the new token is given the lifetime the provider stated for
the one before it, so the connection is not refreshed again every time a node
opens it.
When a refresh cannot happen, it fails loudly rather than retrying into nothing:
the 'google' access needs reconnecting: the provider issued no refresh token,
so the expired sign-in cannot be renewed
A pasted credential that expired says so too, because there is nothing to refresh it with.
If the provider is busy or down when weft refreshes (it answers 408, 429 or
a 5xx, or does not answer at all), nothing is wrong with the connection, so
it never asks you to reconnect: the node fails with the provider’s own
answer (token endpoint answered 503 ..., or token endpoint unreachable: ...), and the refresh is tried again the next time the connection is
opened.
Where the secrets are
Sealed with AES-256-GCM, under CREDENTIAL_ENCRYPTION_KEY, which is 32 bytes
in base64:
openssl rand -base64 32
Service names, client ids, which permissions were granted and who owns what stay readable, so weft can query them. Only the credentials themselves are sealed.
Without that key set, weft uses a public development key built into its own source and warns you once. That key is in the repository, so anyone with a copy of your database can read every credential in it. Set a real one before you store anything you care about.
There is no rotation. Change the key and everything sealed under the old one is unreadable:
stored credential data does not open under the current CREDENTIAL_ENCRYPTION_KEY;
if the key changed, the sealed rows it wrote are gone: reconnect the affected
connections
Putting the old key back reads them again. Losing it means reconnecting everything.
The rest
For connecting one, go and read connecting an account. For using one inside a node, using a connection in a node. For writing a recipe, declaring a service.
The apps file
access-apps.json is where the operator puts the applications and keys this
installation offers as one-click connections. Whatever is in it shows up in
everybody’s connect panel.
Start from the example and keep what you want:
cp access-apps.example.json access-apps.json
Fill in the services you expect to use often. A service you set up here is one click forever after, and one you skip asks each person to bring their own the first time they need it, which is fine for something you touch once.
The shape
A service name maps to a list, always, even with one entry. Keys starting
with _ are ignored, which is how the example file carries its own notes.
Every key weft defines is camelCase (tokenUrl, signingSecret), while
client_id and client_secret keep the spelling OAuth gave them.
{
"openrouter": [{
"kind": "api_key",
"label": "Runtime key",
"key": "sk-or-..."
}]
}
A pasted key
| Key | What it is |
|---|---|
kind | "api_key" |
label | A name for this entry, unique within the service. A connection made with it is named after the service |
key | The credential. At most one api_key per service |
This is what “use ours” hands out. Calls on it spend money on that account, so keep the file private.
An OAuth application
{
"slack": [{
"kind": "oauth_app",
"label": "Slack",
"covers": ["chat:write", "channels:read", "channels:history"],
"authUrl": "https://slack.com/oauth/v2/authorize",
"tokenUrl": "https://slack.com/api/oauth.v2.access",
"client_id": "...",
"client_secret": "...",
"events": { "signingSecret": "..." }
}]
}
| Key | What it is |
|---|---|
label | Unique within the service |
covers | Exactly the permissions this app asks for, by their ids from the service’s catalogue |
authUrl, tokenUrl | The provider addresses this app signs in against, copied from the recipe |
origins | Only if the recipe’s own token calls run on another host than tokenUrl. Each entry is a bare origin, a scheme and a host with nothing after them, like https://graph.instagram.com |
client_id, client_secret | From the provider. client_secret is absent for a public PKCE app |
events | Only if this app receives pushes. Whatever the recipe verifies them with: signingSecret (Slack), publicKey (Discord, SendGrid), or for a signed identity token, audience and pushEmail |
Permissions are fixed per app. People pick an application rather than ticking boxes, which is why you can list several for one service: a cheap tier that avoids a provider’s review process, and a fuller one beside it. Each shows up as its own labelled choice with its permissions listed.
The addresses are pinned here, and a recipe that tries to substitute a different sign-in address is refused. A connection through a registered app only ever talks to the addresses you wrote.
That holds for a recipe’s own token calls too, its afterExchange and its
refresh, because they run with the app’s secret at hand. Each one has to go
to the origin of tokenUrl or to one listed in origins. Instagram swaps and
renews its tokens on graph.instagram.com while its tokenUrl is on
api.instagram.com, so its app lists that origin:
{
"instagram": [{
"kind": "oauth_app",
"label": "Instagram",
"covers": ["instagram_business_basic", "instagram_business_content_publish"],
"authUrl": "https://www.instagram.com/oauth/authorize",
"tokenUrl": "https://api.instagram.com/oauth/access_token",
"origins": ["https://graph.instagram.com"],
"client_id": "...",
"client_secret": "..."
}]
}
Without it, the connect is refused naming the call, the address and origins.
You also have to register the callback URL the connect panel shows you as the
app’s redirect URL, and re-register it whenever your public address changes.
On a cloud install that happens once, when a domain you add becomes its
address: weft domain list then prints the new callback and the providers to
give it to. Some providers, Slack among them, only accept https callbacks.
Making it take effect
On your machine, weft reads the file each time it needs one of its keys, so an
edit takes effect at once. If the file is gone, weft has no shared apps or
keys until it is back. On a cloud install the file is the WEFT_ACCESS_APPS
secret of your fork: if you want to change it, update that secret and run
install on GCP again (deploying to your cloud).
If you want to keep it somewhere else, point WEFT_ACCESS_APPS_FILE at it
and run weft daemon start. If the file is there but cannot be read or parsed, every lookup of its keys
fails with an error naming the file, because
shipping no apps quietly shows up later as a service nobody can connect on a
node the operator thought was configured.
What it refuses
- A service whose value is not a list
- An entry with no
kind - Two entries with the same label in one service
- More than one
api_keyfor a service - An OAuth app with no
covers - A permission in
coversthat the service’s catalogue does not have - An
originsentry that is not a bare origin (a path after the host, a query, a user name) - An
eventsblock on a service with no webhook transport, or a key in it weft does not know - A key an older weft spelled snake_case (
auth_url,token_url,events.signing_secret,events.public_key,events.push_email,_setup_events). The error names each one, where it sits and its spelling now, so an old file tells you exactly what to rename - Invalid JSON
Apps a node ships
A node’s own metadata.json can also ship a public application, with no
secret. It is not a one-click connection: the “Your own” page signs in with it
when the person leaves every app field empty. For what it may carry, go and read accessApps.
If your application has a secret, put it in this file instead, or paste it on the “Your own” page when you connect.
Encryption
Everything stored is sealed under CREDENTIAL_ENCRYPTION_KEY, which is 32
bytes in base64:
openssl rand -base64 32
Set it before you store a credential you care about. Without it weft uses a public development key from its own source and warns you once, which protects a database dump from nobody.
Keep it. There is no rotation: change the key and everything sealed under the old one is unreadable until you put it back. Lose it and you reconnect everything.
Using a connection in a node
Declare a connection input, typed with its service, with the connect widget:
"inputs": [
{ "name": "account", "type": "Access[slack]", "required": true,
"widget": { "kind": "access" },
"requiresScopes": ["chat:write"] }
]
Write just {"kind": "access"}. The service and whether it is optional are
filled in from your node’s service recipe. The [slack] makes the compiler
refuse any other service’s connection on this input; for a node that works
with several, go and read which connection a node takes.
Then open it:
let account: Access = ctx.inputs.get("account")?;
let client = ctx.open(&account).await?.client();
let reply = client.post("https://slack.com/api/chat.postMessage")
.json(&body).send().await.node_err("posting the message")?;
That client already has the authentication, the routing and the metering on it.
Sending the call and reading the answer
Every API node does the same three things after building a request: send it,
fail with the provider’s own words when the status is not a success, and read
the body. weft::access::client has that once, so your node does not write it
again:
| Call | Use it when |
|---|---|
get_json(&client, url, what) / post_json(&client, url, &body, what) | A plain GET, or a POST of a JSON body, that answers JSON |
json_call(request, what) | The request needs its own preparation (a content type, a multipart body) but answers JSON |
checked_send(request, what) | The answer is not JSON (a download, raw text). You get the response back unread |
require_ok_flag(answer, "ok", "error", what) | The service answers 200 to everything and says inside the body whether it worked (Slack’s ok, Telegram’s ok and description) |
cursor_paged(paging, what, request, check, visit) | A list that comes a page at a time with a cursor. CursorPaging says where the items and the next cursor sit and which query parameter carries it back; past_cap_hint is what the user is told to narrow when the list runs past the page cap |
required_str(&answer, what, "id") | A field the rest of the node cannot do without, failing by name when it is missing |
what names the attempt, so a failure reads “the service answered 403 trying
to post the message” instead of a bare status code.
If your node waits on a job the provider runs, go and read waiting on a job a service runs.
The one rule about HTTP
A call on a connection goes through ctx.open or ctx.client. Everything
else goes through ctx.http().
A reqwest client you build yourself skips both the signing and the cost
recording, and the call will either fail unauthenticated or succeed while
nobody writes down what it cost.
| Call | Gives you |
|---|---|
ctx.open(&access).await? | The connection, leased for this firing |
ctx.client(access).await? | Straight to the signed-in client. Takes None and gives a plain one |
ctx.http() | The shared plain client, no credentials |
ctx.client accepting None is for a node whose connection is genuinely
optional, like fetching a link that is usually public.
What an opened connection knows
| Call | Gives you |
|---|---|
client() | The HTTP client, authenticated and metered |
credential() | The single credential, for a service that has one |
value(name) | One stored value, like imapHost |
identity() | Who it is, for a log line |
owner() | Whether it is theirs or the runtime’s |
Keep those out of your outputs and out of your logs. Anything you emit is journaled and shows in the inspector.
Saying what you need
requiresScopes and requiresValues on your connection input are the important
part, and they are per consuming node.
{ "name": "account", "type": "Access[slack]", "requiresScopes": ["channels:history"],
"requiresValues": ["teamId"] }
Those get stamped onto the value as it reaches you, and checked when you open it. So a node that needs to read history refuses a connection that can only write, before it makes the call, and the person is told to reconnect rather than getting a provider error they have to decode.
The list is all-of, and it is fixed by the node’s one job. When the permission would depend on which input arrives, like a post that needs one scope for a person and another for a company page, that is two nodes, each declaring its own. One node with the rule written in prose protects nobody.
If an optional input needs one more permission only when somebody uses it,
leave that permission out of requiresScopes, which every program using the
node is asked for, and add it in run before you open the connection:
if reply_to.is_some() {
let mut needed = account.required_permissions().to_vec();
needed.push("https://www.googleapis.com/auth/gmail.readonly".to_string());
account = account.with_required_permissions(needed);
}
let http = ctx.client(&account).await?;
Extend the list rather than replace it: it already holds what requiresScopes
declared. Opening then refuses a connection that lacks the permission, naming
it, the same way it refuses one missing a declared one.
A missing stored value always refuses. A missing permission only refuses when weft actually knows it is missing: on a service that reports nothing, the call goes through and the provider decides.
A long job
A lease covers the firing. For something that genuinely runs a long time, a multi-hour generation, ask for a window:
let conn = ctx.open_within(&account, Duration::from_secs(3 * 3600)).await?;
The default is fifteen minutes, which is generous for an API call including its stream and its cost resolution, and short enough that a lease left by a crashed worker goes stale the same quarter hour.
A connection your own node hands out
If your node runs the service, it publishes the connection rather than consuming one:
let access = ctx.publish_access(BTreeMap::from([
("host".into(), host), ("password".into(), password),
])).await?;
ctx.pulse_downstream(NodeOutput::new().set("access", access)).await
Declare it in metadata with "publishes": "postgres" and an Access[postgres] output.
Publishing again updates rather than duplicating, and terminating the node’s
infrastructure deletes it. ctx.published_access() reads back what you
published before, which is how a container that mints its own password on first
boot is asked for it exactly once.
In a test
rig.connection_permissions("slack", &["chat:write"]);
rig.respond("POST", "/api/chat.postMessage", json!({ "ok": true, "ts": "111.222", "channel": "C42" }));
rig.respond("GET", "/api/chat.getPermalink", json!({ "ok": true, "permalink": "https://acme.slack.com/archives/C42/p111222" }));
let inputs = json!({ "account": rig.access("slack"), "channel": "C42", "text": "hi" });
rig.run(&SlackSendMessageNode, inputs).await.ok()?;
rig.assert_sent("POST", "/api/chat.postMessage");
rig.access("slack") makes a connection marker to put on the access input.
rig.connection_permissions says what that connection was granted, so opening
it checks the node’s requiresScopes; leave it out and the permission check is
skipped. rig.connection_value("slack", "teamId", "T123") gives the connection
a stored value, for a node that reads one. rig.respond sets up a canned answer
for a route, so no request leaves the test.
The test uses no real credential and makes no real call, so it costs nothing.
Declaring a service
A service recipe is the service block in an access node’s metadata.json. It
is pure declaration: how a credential is obtained, how a request is signed,
what the permissions are, how events arrive. There is no Rust per service.
The node itself is one line:
weft::access_node!(SlackAccessNode);
The top level
| Key | Default | What it says |
|---|---|---|
service | required | The name, lowercase letters, digits and underscores |
label | the service name | What people see, like “Slack” |
acquisition | required | How a credential is obtained |
auth | [] | How a request through it is signed. Empty for something that signs nothing, like a database |
doors | ["own"] | Which connect doors to offer |
test | none | A call made at connect time to check it works, and where identity usually comes from. Whatever permission it needs must be granted on every connection (see always below) |
identity | none | The display name, assembled from stored values, like "{team} / {user}" |
permissions | [] | The permission catalogue people tick from |
allPermissionsUrl | none | Where the provider’s full list is, when yours is a subset |
verification | silent, free | What the check can learn and what it costs |
connectionOptional | false | The node runs with nothing picked |
callback | any | What address the provider accepts for its OAuth callback: any, https (it refuses plain http, like Slack), or domain (https on a domain name, never an IP address, like Google) |
ownPage | none | What the “your own” door shows beyond its paste fields |
events | {} | How this service reports events, by topic |
capabilities | [] | Named groups of optional fields, each unlocking one thing |
grants | coexisting | Whether two grants of this service can coexist |
Acquisition
kind | For | Keys |
|---|---|---|
static | Paste a key | fields |
oauth2 | Sign in | grant, tokenUrl, and the keys below |
mint_jwt | Sign your own assertion | fields, algorithm, claims, jwtTtlSecs, exchange |
There is a fourth, runtime, and you never write it. It is the stored form
of a shared connect, written by the store, and declaring one is refused.
Signing in
An oauth2 acquisition needs two keys. grant is either
authorization_code, a consent page in the browser, with an authUrl and PKCE
on unless you set "pkce": false, or client_credentials, where weft asks the
token endpoint directly with the app’s id and secret. tokenUrl is where the
code is exchanged and the token renewed.
Everything else is for a provider that does something differently from the OAuth standard:
| Key | Default | Set it when |
|---|---|---|
scopeDelimiter | a space | the consent URL joins scopes with something else. Slack and TikTok want "," |
clientParams | { "id": "client_id", "secret": "client_secret" } | the provider calls the app’s id or secret something else. TikTok wants { "id": "client_key" }, which is then the name on the consent URL and on every token call that sends the id in its form. Templates still say {client_id} and {client_secret} |
authParams | none | the consent page needs more parameters, like Google’s "access_type": "offline". Each value is a template over the app’s values, never a secret one, because the consent URL is shown in the browser |
extraParams | none | the provider wants more parameters on every token call: the exchange, the refresh, and client credentials. Each value is a template over the app’s values |
registrationFields | none | the app has more to paste than an id and a secret |
captures | none | you want to keep values from the token answer, like the team’s name. They run on every token answer. One missing from the first answer fails the connect unless it is optional, and one missing from a refresh answer keeps what it held |
tokenAuth | body | the token endpoint wants the id and secret as HTTP Basic rather than in the form: set basic (Notion, Airtable) |
afterExchange | none | the code exchange hands out a short-lived token that has to be swapped right away, like Instagram’s. See your own token calls |
refresh | the standard refresh | renewing is not the standard refresh_token POST. See your own token calls |
The answer keys in the next section go on the acquisition too.
Reading the token answer
Every answer that hands out a token is read the same way, and the defaults are
the names the OAuth standard uses. If your provider answers somewhere else, say
where with a dotted path, where a number picks an item from a list:
data.0.access_token is the access_token of the first item under data.
| Key | Default | What it points at |
|---|---|---|
tokenPath | access_token | The token, stored as token. An answer without one fails |
refreshTokenPath | refresh_token | The refresh token, stored as refresh_token when there is one |
expiresInPath | expires_in | How many seconds the token lives, as a number or a string of digits |
expiresPath | none | When the token dies instead, as an RFC 3339 time or epoch seconds |
neverExpires | false | Set it to true when the token does not expire, so no expiry is read |
scopePath | scope | The scopes the provider says it granted, as one string split on spaces and commas, or a list |
errorPath | error | Where a refusal sits when the provider answers it with a 200. Anything there but null, false or an empty string fails the call. Instagram puts it at error_type |
okPath | none | Where the provider says whether the call worked, like Slack’s ok. Anything there but true or null fails the call, and an answer with nothing there passes |
An expiry path you declare has to be in the answer, while the default
expires_in may simply be missing. expiresInPath, expiresPath and
neverExpires exclude each other.
The code found at errorPath also decides what a refusal means: a code saying
the provider is busy (temporarily_unavailable, ratelimited) is tried again
later, and one saying the token is bad asks for a reconnect.
If weft can renew a connection (a refresh token came back, a refresh call is
declared, or it is a client_credentials grant) and the answer names no
expiry, the connect is refused, because nothing would ever renew it:
the provider's answer names no expiry at 'expires_in' (expiresInPath), so weft
would never know when to renew this connection and it would stop working
without a word. If the provider sends the lifetime under another name, declare
it on the acquisition (`expiresInPath` for seconds, `expiresPath` for a time);
if the token never expires, declare `neverExpires: true` there
A renewal answer is allowed to leave the expiry out, as the OAuth standard lets a provider do when its tokens always last the same time. weft then gives the new token the lifetime the last one had. It does the same when a renewal answer names an expiry it cannot read (a declared path missing from it, a value that is not a time, or a time already past when the answer arrives), so the connection is never renewed again on every use.
Your own token calls
afterExchange and refresh are calls you write out. Each takes a url, a
method (GET unless you say POST), a form or a JSON body, auth
steps, captures, and any of the answer keys above, which read this call’s
answer and start from the same defaults.
afterExchange runs once, right after the code exchange, and its answer
replaces the token, its expiry and the refresh token. If it carries no refresh
token, the connection has none: the one from the exchange belonged to the
short-lived token that was just swapped away.
refresh runs instead of the standard refresh whenever the token is about to
expire.
Templates in them can use what the connection stores (token,
refresh_token, every capture) and the app’s values (client_id,
client_secret, the registration fields). A name nothing provides is refused
when the node loads. A value that lands in the path or the query of the url
is percent-encoded, so a secret holding & arrives as one value, while in a
header, a form or a body it goes in as it is.
Instagram uses both. Its code exchange answers a one-hour token nested under
data, which it swaps for a 60-day one, and renews with a GET:
"acquisition": {
"kind": "oauth2",
"grant": { "kind": "authorization_code",
"authUrl": "https://www.instagram.com/oauth/authorize", "pkce": false },
"tokenUrl": "https://api.instagram.com/oauth/access_token",
"tokenPath": "data.0.access_token",
"scopePath": "data.0.permissions",
"errorPath": "error_type",
"afterExchange": {
"url": "https://graph.instagram.com/access_token?grant_type=ig_exchange_token&client_secret={client_secret}&access_token={token}"
},
"refresh": {
"url": "https://graph.instagram.com/refresh_access_token?grant_type=ig_refresh_token&access_token={token}"
}
}
Both calls answer access_token and expires_in at the top, so they declare
no answer keys.
Refusals answered with a 200
Some providers answer a failed call with a 200 and say inside the body that it
failed. Slack answers {"ok": false, "error": "invalid_auth"}. weft only sees
that where the recipe says to look, with errorPath and okPath.
A token answer looks at error unless told otherwise. Every other call looks
nowhere until you declare it, because some providers put an error field in
answers that worked (TikTok’s say "error": {"code": "ok"}). That covers the
test call, a socket’s connect, a subscribe or unsubscribe, and a
lookup on a node’s input. Slack
declares both keys on each of its calls, like its test call:
"test": {
"url": "https://slack.com/api/auth.test",
"method": "POST",
"okPath": "ok",
"errorPath": "error"
}
A status other than 200
The test call passes when it answers 200. If the provider checks the key
before it reads the request, set expectStatus to the status you get once the
key is accepted. ExaAccess sends a POST with no body and expects 400: a good
key reaches the missing-body error, and a bad one is refused earlier with 401.
Minting your own
A mint_jwt acquisition signs a JWT from the pasted key with the claims you
declare, valid for jwtTtlSecs (600 by default). Without an exchange, that
JWT is the token. With one, weft POSTs it as a bearer token to the exchange’s
url, and reads the answer with tokenPath and expiresPath from the table
above. An answer naming no expiry is minted again once the JWT’s own lifetime
runs out.
Auth steps
How a request gets signed, applied in order:
kind | What it does |
|---|---|
header | Adds a header |
query | Adds a query parameter |
basic | HTTP basic auth |
path_prefix | Puts a prefix on the path |
base_url | Replaces the scheme, host and port, keeping the node’s path |
sign | Signs the request: SigV4, or OAuth 1.0a |
Every value is a template interpolating stored values by name, like
"Bearer {token}". If you need a literal brace, write it twice: {{ for {
and }} for }. That is how a JSON body carries a GraphQL query, so
"query": "{{ viewer {{ login }} }}" sends { viewer { login } }.
A header step’s name can be a template too, for a connection where the
person types which header their key goes in (a generic API key):
{ "kind": "header", "name": "{header}", "value": "{key}" }. It may only name
a declared field that is not secret, and if what the person typed is not a
legal header name, the call is refused before anything is sent, telling them
to reconnect.
Permissions
{ "id": "chat:write", "label": "Send messages",
"description": "Post messages as the app.", "default": true }
A curated catalogue, not the provider’s whole list. On a provider with hundreds
of scopes, list the ones your nodes actually need and point
allPermissionsUrl at the rest.
default: true starts it ticked. ownOnly: true means the runtime’s own
credential can never serve it, which greys out the shared option for anyone who
ticks it.
If your test call needs a permission of its own to learn who the account is,
mark that permission always: true. It is then asked for on every connection,
whatever the person ticks, and never shown to tick. Google does this with
openid and userinfo.email, so a connection made for Gmail alone still
passes its check against Google’s “who is this” endpoint. Without it, a test
call that reads Drive fails every connection that did not tick Drive. Only
mark a permission always when the provider grants it without a review and an
app allows it without being set up for it, because every app and every
connection asks for it. An always permission cannot also be default or
ownOnly.
Verification
Say honestly what your check can learn:
rung | Meaning |
|---|---|
reports_permissions | The provider states what it granted |
self_introspect | The credential can describe itself |
reports_validity | Only “alive”, never “what” |
probe | A real call whose refusal would be read back. No check uses it yet |
silent | Nothing knowable |
Only the first two make the recorded permissions authoritative, and only those let weft block a node before it calls.
cost is free, ambiguous or paid, and only free runs on its own. A
check that might bill is never run without being asked.
A service that declares no verification takes the default, silent and free.
A probe would only learn about the one call it makes, so like
reports_validity it leaves the permissions recorded as claimed.
Your own door
ownPage is what that door shows:
| Key | What it is |
|---|---|
mint | A button that creates the application for them, if the provider has an API for it |
guide | Numbered steps, and a pre-filled creation link |
paste | The fields, when the acquisition is not already a paste form |
A paste block on a static acquisition is refused as redundant: those fields
already render.
What it refuses
The validator is strict, and the messages name the fix. A service name with a
capital in it. A header name that is not a legal header name. A capability
listing a field the service does not declare, or listing a required field,
since a required field can never be the thing that is missing. A permission
with no description. A service that signs its requests offering the shared
door. A token_echo caller check, which compares a token weft minted when
subscribing and a route’s caller never has.
Events
Each topic under events says which facts to pull out of a payload, which
account it belongs to, and how it arrives. Go and read
events from a service.
Where it goes
A package root’s metadata.json, so every node in the package inherits it, or
the access node’s own file.
Exactly one input in the node needs the access widget. That is the connect control, and a recipe without one is refused.
Events from a service
A trigger that waits for a Slack message, a new row in a sheet, a page changing in Notion. The service has something to tell you, and there are two roads it can travel.
The two roads
Dial out. weft makes an authenticated call that hands back a socket address, connects out, and holds the line. Nothing needs to reach your machine, so this works from a laptop behind a router.
Dial in. The service posts each event to an address you published. That needs a public address, and it is the only road a shared application can use, and the only road most services offer at all.
You never choose. You name a connection and a filter, and the transport follows from what the service declares and what your environment can serve. On a cloud install, if the service can push these events for your account, weft takes the push, so nothing has to stay running between events. Otherwise weft dials out when it can, because that needs nothing from you.
What a topic declares
A service declares a map of topics, because one provider can genuinely report along several independent channels at once.
| Key | What it says |
|---|---|
fields | The named facts of one event, and where each lives in the payload. A dotted path, or header:X-Some-Header (a push that sends that header twice with two different values is refused) |
account | Which stored value names the account on a connection, and where the same id sits on an event. That pair is what routes a shared application’s events to the right connection |
socket | Present when the service offers dial out |
webhook | Present when the service dials in |
Everything downstream works on the names. Subscriptions filter on them and triggers fan them out, so neither a filter nor a trigger node ever writes a provider’s path.
Receiving a push
A webhook topic declares how the provider’s pushes are handled:
| Key | What it is |
|---|---|
verify | How a push is checked: hmac, signature (Ed25519 or ECDSA), oidc, or token_echo (the push carries back a token weft minted). For every key, go and read checking a signed request |
handshake | The one-off proof of ownership: echo a field from the body, echo a query parameter, or call a URL back |
eventPath | Where the event sits inside the envelope |
routeBy | account (the default: the event names the provider account) or subscription (it names an id weft minted when it subscribed) |
idPath | Where a push carries the subscription id weft minted. Needed with routeBy: subscription, and with token_echo |
tokenPath | Where a push carries the token weft minted. Needed with token_echo |
decode | For an event wrapped in base64 or a form encoding |
subscribe | The calls that create, renew and cancel the subscription |
The secret is never in the recipe. Every scheme is a protocol with holes, and the material comes from the operator’s application config, or from a token weft minted. That is what lets a recipe live in source.
What activation does
For a dial-in topic, the first subscribe happens then. So a missing public address or a provider refusing is that activation’s error, not a mystery three days later:
this trigger cannot be served on the 'slack' connection: the service declares
no transport for this topic
And the one you are most likely to meet:
this trigger needs 'slack' to deliver events TO your weft, which requires your
weft to be reachable from the internet, and it is not. If you accept making its
public trigger surface reachable, rerun `./setup.sh --public-url`, then `weft
activate`, or, when the trigger is already on, turn it off and on again
(`weft deactivate --trigger <name>`, then `weft activate --trigger <name>`);
or use a connection that supports dialing out (your own provider app), where the
service offers one. See https://weavemindai.github.io/weft/connections/events.html
A decision that a trigger cannot be served is final for that registration. Retrying would work out the same facts again. Reconnecting the account re-registers it and works them out afresh.
If a topic is not in the connection’s snapshot, the account predates it:
'slack' declares no event topic named 'reactions'; reconnect the account (an
older connection may predate the topic)
Renewal
Most providers expire a subscription. The recipe says the margin, and weft re-subscribes before the deadline.
A renewal is a fresh subscribe, with a new id and a new token, because the providers surveyed have no extend verb, and reusing the token would make the old and new channels impossible to tell apart.
If a topic declares no unsubscribe, deactivating cannot hurry it and the subscription lapses on its own expiry.
An application-wide subscription
Some apps want every install’s events rather than one account’s. That only works over dial out, because a push receiver routes each event to one account’s connections and could never deliver “every install of your app”.
an app-wide subscription only works over a dial-out transport, and this topic
declares none
Which triggers need a public address
Only the ones where the provider pushes. A timer, a poller, a socket weft dials out to and a form somebody fills in all work with nothing reachable from outside.
For the address itself, go and read a public address. For declaring a topic, go and read declaring a service.
Checking a signed request
A verify block tells weft how to check that a request really comes from who
it claims to. You write it in one of two places:
- in a topic’s
webhook, to check a provider’s pushes. The secret or public key comes from the receiving app’seventsentry in the apps file; - in a
Route’s orSocket’sauthsetting, so that it checks its callers (a gated route). The secret or key comes from a connection of the route’s own (see on a route below).
The block only describes the request: where each part sits, what gets signed, which issuer to trust.
The shortest one
If a provider signs the raw body with a shared secret and sends the hex digest
in a header, this is the whole block (GitHub’s, which writes sha256=<hex>):
"verify": {
"kind": "hmac",
"signature": { "in": "header:X-Hub-Signature-256", "pair": "=", "key": "sha256" },
"concat": "{body}"
}
weft computes the HMAC-SHA256 of the exact bytes it received, keyed by the
stored secret, decodes the hex after sha256=, and compares the two as bytes,
in constant time.
On a route
A Route or a Socket answers anybody who has its address until you set its
auth:
auth | What the caller sends |
|---|---|
"none" (the default) | Nothing |
"api_keys" | One of the keys you hand out, in X-Api-Key or as Authorization: Bearer <key> |
"hmac_sha256" | A hex HMAC-SHA256 of <timestamp>.<body> in X-Signature, with the unix-seconds timestamp in X-Timestamp, refused when it is more than five minutes off |
"hmac_sha1", "hmac_sha512" | The same layout, with HMAC-SHA1 or HMAC-SHA512 |
"ed25519" | The same layout, signed with Ed25519 instead of an HMAC |
"ecdsa_p256" | The same layout, signed with ECDSA P-256, the signature in base64 DER |
"oidc" | A signed identity token in Authorization: Bearer, from the issuer the connection names |
| a block | Whatever the block says, written the way the rest of this page describes |
If your sender uses another scheme (GitHub’s signature, signed identity tokens from your sign-in provider), write the whole block:
hook = Route {
path: "github"
method: "POST"
auth: { "kind": "hmac", "signature": { "in": "header:X-Hub-Signature-256", "pair": "=", "key": "sha256" }, "concat": "{body}" }
}
Then give the route its connection, which holds the keys or the secret: the
editor shows a Connect panel under auth, and in the terminal it is
weft connect hook. Each scheme’s connection has its own fields:
auth | Its connection’s service | Fields |
|---|---|---|
api_keys | api_key_auth | keys |
hmac_sha256, hmac_sha1, hmac_sha512 | hmac_auth | signingSecret |
ed25519, ecdsa_p256 | signature_auth | publicKey: hex for ed25519, base64 DER for ecdsa_p256 |
oidc | oidc_auth | issuer, jwksUrl, audience |
If you are scripting it, weft connect hook --door own --set-env signingSecret=HOOK_SECRET --label "github hook" stores a new one, and weft connect hook --grant <id> picks one already stored. When a field is missing,
the command prints the fields the scheme wants.
If you turn on a route whose auth names a scheme and no connection is
picked, preparing it fails with 'hook' checks callers with hmac but has no connection; run `weft connect hook` . token_echo cannot check a caller, so
auth refuses it.
The schemes
kind | Checks | Secret or key |
|---|---|---|
hmac | An HMAC over parts of the request | signingSecret |
signature | An Ed25519 or ECDSA P-256 signature over parts of the request | publicKey |
oidc | A signed identity token in Authorization: Bearer, against an issuer’s published keys | the issuer’s keys; see below for the audience |
api_keys | A key from a list. Routes only | keys |
token_echo | The token weft minted when it subscribed, sent back on every push. Pushes only | the subscription |
If you use oidc, write issuers (the iss values the token may carry) and
jwksUrl (where the issuer publishes its signing keys). The token must be
signed RS256, and jwksUrl has to be an https:// address, since keys
fetched over plain http could be swapped by anyone on the way. weft keeps
the keys it fetched for an hour, and an issuer that is slow to answer holds
up only the tokens it signed. On a push, write issuers and jwksUrl out
in full, with no {...} placeholders, or the metadata fails to load. Each
push’s token must then name the audience of one of the receiving apps (the
receiver’s own address for an app that names none) and carry
email_verified: true, and when that app sets pushEmail, its email must
be that one.
On a gated route, issuers and jwksUrl can name the connection’s stored
values, like "{jwksUrl}", and every token must name the connection’s
audience. One issuer usually signs tokens for every app it serves, so
without the audience a token meant for another of those apps would get in.
If you want a route that admits the users of your own sign-in provider
(BetterAuth, Clerk, Auth0, Firebase, any OIDC issuer), this auth does it,
and the route’s connection then holds issuer, jwksUrl and audience:
{ "kind": "oidc", "issuers": ["{issuer}"], "jwksUrl": "{jwksUrl}" }
The caller sends Authorization: Bearer <token>, and the run’s caller
carries the token’s claims (sub, email, whatever the issuer put there).
If you use api_keys, the caller sends the key in the header that header
names (X-Api-Key unless you set it), and with bearer (on unless you set it
to false) weft also takes Authorization: Bearer <key>. When both are sent,
the header wins. The accepted keys are the connection’s keys value,
separated by commas or line breaks.
token_echo has no keys of its own. The topic’s tokenPath says where a push
carries the token back, and its idPath where it names the subscription,
which is how weft finds the token to compare against. The subscribe call has
to send the token, as {subscriptionToken}.
Everything below is about hmac and signature. They read the request the
same way and differ only in the check itself, which
how the check is made covers.
Where a value sits
Every place a value is read from is written the same way:
| Written | Reads |
|---|---|
header:<Name> | A header, whatever its case. If the request sends it twice with two different values, it is refused, since there is no telling which one was signed. The signature’s own header is the exception: every copy is read |
body:<path> | A field of the JSON body, by dotted path. A string is read as it is, and a whole number or true/false as written. A number with a fraction or an exponent, or a whole number too large for 64 bits, is written back out the way weft’s JSON reader prints it, so it matches only if the provider wrote it that way too. A field that is null, an object or an array counts as missing |
signature:<key> | One entry of the signature’s own value, like Stripe’s t. Only for a timestamp |
signature: where the signature is and how it is laid out
| Key | Default | Set it when |
|---|---|---|
in | required | Always: header:<Name>, or body:<path> when the provider puts the signature in the body (Mailgun) |
list | the whole value is one entry | The value holds several entries: , for Stripe, ; for Paddle, a space for Standard Webhooks |
pair | each entry is a bare signature | Each entry is a key and a value: = in v1=<sig>, , in v1,<sig>. An entry is cut at the first one |
key | none | You set pair: the key whose entries hold a signature (v1, h1, v0). Entries under other keys are skipped |
When the request carries several signatures (several entries under the key, or the header sent more than once), one match is enough. That is how a provider rolls a secret: it signs with the old and the new one for a while.
timestamp: refusing an old request sent again
Leave it out when the provider signs no timestamp (GitHub, Shopify, Twilio).
| Key | Default | Set it when |
|---|---|---|
in | required | header:<Name>, body:<path>, or signature:<key> for one entry of the signature’s value |
format | seconds | The provider writes milliseconds (Linear) or rfc3339 (2024-05-01T12:00:00Z) |
toleranceSecs | 300 | The provider documents another window: Paddle’s is 5, Linear’s 60 |
A timestamp further from now than the tolerance, in either direction, is refused.
concat: what is signed
concat is a template: literal text stays as it is, and each placeholder is
replaced by a part of the request:
| Placeholder | Becomes |
|---|---|
{body} | The raw body, byte for byte, never parsed and rewritten |
{timestamp} | The declared timestamp, exactly as the request wrote it |
{header:<Name>} | One header’s value |
{body:<path>} | One field of the JSON body |
{url} | The address the request was sent to (see below) |
{method} | POST, GET, and so on (always POST on a push) |
{sorted_form_params} | The form body’s pairs sorted by name, then value, each written name then value with nothing between (Twilio’s rule) |
On a gated route, {url} is the address the caller used, with the path and
query exactly as they sent them. The host is the Host the request arrived
at weft with (a proxy’s X-Forwarded-Host wins over Host), lowercased and
without the scheme’s default port (:443 for https, :80 for http), so
register the address with the provider in that form. The scheme comes from
X-Weft-Forwarded-Proto when the request carries it (weft’s own doors set
it to the scheme the caller used), otherwise from X-Forwarded-Proto (a
proxy in front of weft that terminated TLS says https there), and is
http when neither says http or https. If the proxy mounts weft under a
path, it has to send that path in X-Forwarded-Prefix. For a push, {url}
is the receiver’s address, <your public address>/events/<service>/<topic>,
with no query.
How the check is made
secret and algorithm only apply to hmac, and encoding to both:
| Key | Default | Set it when |
|---|---|---|
secret.stripPrefix | none | The provider’s secrets start with a marker that is not part of the key, like whsec_. A stored secret without that marker is refused |
secret.decode | raw (the secret’s bytes) | The key is written base64 (Standard Webhooks) or hex |
algorithm | sha256 | The provider uses sha1 (Twilio) or sha512 |
encoding | hex | The digest or signature is written base64. Hex is read in either case, base64 with or without its = padding |
A signature block also needs scheme, which has no default: ed25519 (the
public key written as 64 hex characters, the way Discord shows it) or
ecdsa_p256 (the public key as base64 DER, the way SendGrid shows it). A
signature block that carries secret or algorithm is refused.
What weft refuses when it loads the block
The rules are the same for a push and for a gated route. They run when the node’s
metadata.json is loaded, and a broken block fails there with an error naming
the service and the rule it broke:
- The concat has to cover the request:
{body},{sorted_form_params}or at least one{body:<path>}. A signature over the method and the address alone would verify any body. - A timestamp is signed, or not declared at all. One the concat never
covers proves nothing, since anyone can send any timestamp, and a concat
that contains
{timestamp}with notimestampdeclared could never be filled in. A timestamp counts as covered when the concat contains{timestamp}, its own header, its own body field, or the whole body it sits in (Linear’s case). - A concat that covers only fields of the body has to sign a timestamp. The rest of the body is then outside the signature, so what stops somebody sending a captured signature again with other contents is the tolerance. That is Mailgun’s design: it signs a timestamp and a random token, not the event.
- The concat cannot contain the signature itself, since the provider signs before the signature exists.
- Smaller ones:
pairneedskeyandkeyneedspair;listandpaircannot be the same separator, and neither can be empty;toleranceSecscannot be 0; asignature:<key>timestamp needspairset and a different key from the signature’s; and the signature sits in a header or a body field, never insignature:<key>.
Blocks for common providers
Standard Webhooks (Svix, Resend, Whop):
{
"kind": "hmac",
"signature": { "in": "header:webhook-signature", "list": " ", "pair": ",", "key": "v1" },
"timestamp": { "in": "header:webhook-timestamp" },
"concat": "{header:webhook-id}.{timestamp}.{body}",
"secret": { "stripPrefix": "whsec_", "decode": "base64" },
"encoding": "base64"
}
Stripe:
{
"kind": "hmac",
"signature": { "in": "header:Stripe-Signature", "list": ",", "pair": "=", "key": "v1" },
"timestamp": { "in": "signature:t" },
"concat": "{timestamp}.{body}"
}
Paddle:
{
"kind": "hmac",
"signature": { "in": "header:Paddle-Signature", "list": ";", "pair": "=", "key": "h1" },
"timestamp": { "in": "signature:ts", "toleranceSecs": 5 },
"concat": "{timestamp}:{body}"
}
Slack:
{
"kind": "hmac",
"signature": { "in": "header:X-Slack-Signature", "pair": "=", "key": "v0" },
"timestamp": { "in": "header:X-Slack-Request-Timestamp" },
"concat": "v0:{timestamp}:{body}"
}
Twilio:
{
"kind": "hmac",
"signature": { "in": "header:X-Twilio-Signature" },
"concat": "{url}{sorted_form_params}",
"algorithm": "sha1",
"encoding": "base64"
}
Mailgun:
{
"kind": "hmac",
"signature": { "in": "body:signature.signature" },
"timestamp": { "in": "body:signature.timestamp" },
"concat": "{timestamp}{body:signature.token}"
}
Linear:
{
"kind": "hmac",
"signature": { "in": "header:Linear-Signature" },
"timestamp": { "in": "body:webhookTimestamp", "format": "milliseconds", "toleranceSecs": 60 },
"concat": "{body}"
}
Shopify’s block is GitHub’s with no pair or key, plus "encoding": "base64" and the header X-Shopify-Hmac-SHA256.
Discord:
{
"kind": "signature",
"scheme": "ed25519",
"signature": { "in": "header:X-Signature-Ed25519" },
"timestamp": { "in": "header:X-Signature-Timestamp" },
"concat": "{timestamp}{body}"
}
SendGrid’s block is Discord’s with "scheme": "ecdsa_p256", "encoding": "base64", and the headers X-Twilio-Email-Event-Webhook-Signature and
X-Twilio-Email-Event-Webhook-Timestamp.
When a request is refused
If the check fails, the sender sees a flat 401 with the body refused,
whichever part failed, because a checker that says which part failed tells a
forger what to fix next. The reason goes to the runtime’s log (weft daemon logs on your machine): look for caller refused (target
weft_broker::caller_auth) on a gated route, push refused (target
weft_broker::events) on a push.
| Reason in the log | What to look at |
|---|---|
the request carries no header:X-Signature (the signature) | The sender is not signing, or the block names the wrong header |
the request carries no signature under header:X-Signature | The header arrived, but no entry in it sits under key: check list, pair and key against a real header value |
the request's signature does not match | The wrong secret or key is stored, or the concat differs from what the provider signs. On a gated route that signs {url}, also check the address weft rebuilt (above). For api_keys, the key the caller sent is not in keys. For token_echo, the token at tokenPath is not the one weft minted for the subscription idPath names |
the request's timestamp is 412s away from now, outside the replay window | An old request sent again, or a clock that is off. If the provider documents a wider window, set toleranceSecs to it |
the request's header:X-Timestamp is not a timestamp in the format the scheme declares | format does not match how the provider writes it |
the request carries the webhook-timestamp header more than once with different values, ... | The sender, or a proxy in front of weft, adds the header twice. The same goes for any header a topic routes by (its fields, its account, idPath or tokenPath) |
no signing secret is configured for this service, ... | No secret is stored: the connection has no signingSecret, or the app’s events entry has none |
no usable signing secret (it lacks the prefix ...) is configured ... | The secret was pasted without its whsec_: paste it whole |
no public key is configured for this service, ..., or no usable public key (...) is configured ... | No public key is stored, or it is not in the form scheme expects |
the request's identity token is not usable: ... | The text after the colon says which check failed: the token expired, came from another issuer or for another audience, or is not signed RS256; on a push, it lacks email_verified: true or its email is not the app’s pushEmail. unknown signing key or key fetch failed means weft could not get the token’s key from jwksUrl: the text after it says whether the address did not answer in time, answered an error, or answered no key set. A failed fetch is not tried again for a minute, so after you fix the address, tokens keep failing with the same reason for up to that minute |
the issuer's signing keys cannot be fetched: the jwksUrl '...' is not an https:// address, ... (or names no host) | The connection’s (or the recipe’s) jwksUrl is plain http, or not an address at all: set it to the issuer’s https:// address |
no push audience is configured for this service, ... | An oidc push where the receiving apps name no audience and this install has no public address to use instead: set audience in an app’s events entry, or turn on a public address with ./setup.sh --public-url |
no api key list is configured for this service, ... | The connection’s keys value holds no key (it is empty, or only commas and blank lines): reconnect it with at least one key |
no receiving app is configured for this service, ... | A push arrived for a service with no app in the apps file that has an events entry |
no receiving app verifies it (app '...': ...; app '...': ...) | With hmac or signature, every receiving app was tried and none verified the push. Each app’s own reason follows its client id: a missing or malformed secret on one app is only that app’s, so check the one the provider signs with |
no public receiver address is configured ... | The concat signs {url}. On a gated route, the request named no host to rebuild the address from. On a push, this install has no public address: turn one on with ./setup.sh --public-url |
A push for a service no project here has connected yet, or for a topic its
events block does not declare, gets 404 instead. On a gated route, if weft
cannot reach the broker to make the check, the caller gets 503 with the caller could not be checked; try again in a moment.
Measuring what a call costs
A node never states a cost. It never totals anything, never carries a running figure from one call to the next, and never works out whose spend a call was.
A meter does that, once per provider, wrapped around the client your node already uses. All the care lives in one place so a node author can be careless: a node cannot produce a wrong bill, because it is never asked.
That only holds if your calls go through ctx.open or ctx.client. A client
you built yourself is outside the meter.
What a meter is
One implementation of ProviderMeter per service, registered with a macro at
the bottom of its file. Its existence is the declaration that this
service’s calls are measured. There is no flag anywhere else.
| Method | Required | What it does |
|---|---|---|
service() | Yes | The service name, matching the access node’s |
base_url() | Yes | Where the provider really lives. The single authority, so no caller ever takes a host from a request |
classify(method, path) | Yes | What kind of route this is |
observe(path, query, body) | Yes | A fresh tap for one response |
resolve(path, observed, follow_up) | Yes | Price one observed call |
prepare(path, body) | No | Rewrite an outgoing body so the cost comes back reportable |
priceable(path, follow_up) | No | Asked before sending: can this be priced at all when it returns? |
opens_charge, charge_reported_on, fold_report | No | For a provider that reports the cost later, on a separate response |
observe_session(path, query) | For a session route | A fresh tap for one long-lived session, priced by its frames |
Routes
| Class | Meaning |
|---|---|
Billable | Spends money. Prepared, observed, priced |
Free | A known route that costs nothing: a model list, a health check, a cost lookup |
Reports | Not a charge itself. It speaks for one that is already open |
BillableSession | A session priced by its own frames as they flow |
Unknown | Not a route this meter knows |
Free is why a node asking what something cost is harmless, and why the
meter’s own follow-up query does not bill.
Route matching is a security boundary
classify gets the path relative to the base URL, with no leading slash
and no query, and matches it exactly.
Anything that does not match lands in Unknown. That is deliberate: a
traversal, an encoded traversal, a backslash or a host smuggled into the
userinfo all simply fail to match, rather than sliding into a cheaper route’s
price.
What gets recorded
MeasuredCost carries an amount, a model and some details.
The amount is optional, and None means genuinely unknown. It is recorded
as unknown, and never booked as zero.
That distinction is the whole discipline. None is for a figure the provider
will not give you. It is not for one nobody went looking for, and it is
certainly not a licence to re-derive the quantity from the request: a token
count you worked out yourself is unmetered spend wearing the shape of a
measurement.
resolve never errors, for the same reason. An unresolvable cost is an honest
unknown, not a failed call.
What the meter never touches
A credential. It gets a signed side-channel for its own follow-up queries and nothing else, which is what lets one meter work identically on a pasted key and on a sign-in.
Each record also says whose credential paid. For what each value means, go and read who paid for a call.
Where the number surfaces
You see the number in the inspector beside the firing’s duration, with whose key paid next to it. It also appears in the run’s events as soon as the call happens.
This is weft’s own record of what it observed, not the provider’s bill. They should agree, and when they do not, the provider is right.
Reading it from a program
If your program needs the numbers (to bill somebody, or to stop spending), the
InstanceCosts node reads them for one instance. Give it instance, and
narrow it if you want with run (a run id, the one GetRunId gives),
service, node, paidBy (whose credential paid: platform for the runtime’s
credit, author for your own connection, instance for the instance’s own) or since (unix seconds). It answers with three
outputs: records, one entry per call; totalUsd, the
sum; and unpricedCalls, how many of those calls have no known price. totalUsd adds up only the known ones, so it is the whole cost
only when unpricedCalls is 0.
Capping what one instance spends
weft puts no cap on spending. If you want one, check the total before the paid node and gate that node on the answer:
question = Route -> (body: { text: String }) { path: "ask", method: "POST" }
openrouter = OpenRouterAccess
model = OpenRouterModel { access: openrouter.access, model: "z-ai/glm-5.3" }
ask = Group(text: String, provider: LlmProvider) -> (answer: String) {
# Answer only while this instance has spent less than five dollars in all
who = CurrentInstance
spent = InstanceCosts { instance: who.instance }
budget = Switch {
value: spent.totalUsd
cases: [
{ "kind": "lt", "value": 5, "port": "underCap" },
{ "kind": "otherwise", "port": "overCap" }
]
}
answer = LlmInference {
_should_flow: budget.underCap
prompt: self.text
provider: self.provider
}
self.answer = answer.response
refuse = Close {
_should_flow: budget.overCap
status: 402
reason: "the budget is spent"
}
}
ask._should_flow = question.fired
ask.text = question.body.text
ask.provider = model.provider
reply = Reply { body: ask.answer }
Each caller sends its instance token (minted with weft token mint --instance) in the Weft-Instance-Token header. A
call without one is for no instance, so who fires nobody, which nothing
here is wired to: the run ends without answering, and the caller gets a 500
saying Something went wrong. If you want a clear refusal instead, gate a
Close on who.nobody. For the other ways a call becomes an
instance’s, go and read programs with instances.
The check can let a little through. A call’s cost is written down when its answer ends, and a job the provider runs in the background is written down when a read of the job says it finished. So a call still going is not in the total yet. And two runs of the same instance that check at the same moment both see the same total, both pass, and together can go over.
If your cap covers a long stretch, know that a run’s costs are deleted with the
run, so the total only counts runs weft still keeps. The example above has no
since, so it really caps the last week of spending, because ended runs are
kept for a week by default. If you want a cap over all time, keep the runs
forever (the trigger’s keepRunsFor, or keep_for in weft.toml). If your
cap uses since, keep runs at least as long as that window. For how to set
either, go and read how long a run is kept.
A provider that states no price
ElevenLabs is the example today. Its image and video generations (flows/image, flows/video) come
back with no cost in any answer. weft prices them from the request instead:
the model, the settings that change its price, and how many images or seconds
of video it asks for. That is what ElevenLabs bills these by, so the request
holds the whole bill. The rates are ElevenLabs’ own pricing table, captured by
hand on 2026-10-16 for the account tier grant_tier_2_2025_07_23. If
ElevenLabs changes its prices, the rates to update are in
crates/weft-providers/src/providers/elevenlabs_prices.json.
| Model | What moves the price | At its defaults |
|---|---|---|
gemini-2.5-flash-image | Nothing | 4.98¢ per image |
gemini-3-pro-image | resolution (default 1K) | 24.12¢ per image |
gemini-3.1-flash-image | resolution (default 1K) | 8.04¢ per image |
gemini-3.1-flash-lite-image | resolution (default 1K) | 4.08¢ per image |
bytedance-seedream-5-lite | Nothing | 4.2¢ per image |
bytedance-seedream-5-pro | resolution (default 2K) | 16.2¢ per image |
gpt-image-1 | quality (default medium), aspect_ratio (default 1:1) | 6.84¢ per image |
gpt-image-1.5 | quality (default medium), aspect_ratio (default 1:1) | 5.88¢ per image |
gpt-image-2 | quality (default medium) | 6.48¢ per image |
gpt-image-2.5-sunburst, gpt-image-2.5-flare | quality (default high) | 6.68¢ per image |
veo-3.1-generate-001 | resolution (only 4K changes it, default 720p), generate_audio (default on) | 48¢ per second |
veo-3.1-fast-generate-001 | the same | 18¢ per second |
bytedance-seedance-v2 | resolution (default 720p), aspect_ratio (default 16:9) | 36.29¢ per second |
bytedance-seedance-v2-fast | the same | 29.04¢ per second |
bytedance-seedance-v2-mini | the same | 18.14¢ per second |
bytedance-seedance-v2.5 | the same, except that with a start_frame the frame decides the aspect, and the table has no rate for that | 55.46¢ per second |
A video costs its rate times duration_secs, which is 8 by default on Veo and
5 on Seedance.
Any other model, a setting the table has no rate for, or a video whose length
the request does not say (a lip-sync video lasts as long as its audio) is
written down as an unknown price, never at the nearest rate. Starting a
generation only queues it, and ElevenLabs does not charge for one that fails.
So the price is written down when your node reads the job back and finds it
completed, and a failed one is written down at $0. If the run ends before
the job is read back, its price is written down as unknown.
Writing one
Copy the nearest existing meter. What it has to get right:
Say the base URL once and never take a host from anywhere else. Classify
exactly, and let anything unrecognised be Unknown. Tap the response as the
bytes flow rather than buffering it. Return None when the provider will not
tell you, rather than guessing. And if a route could never be priced when it
comes back, say so in priceable before the call, so the caller finds out
before spending rather than after.
The CLI
Every command, grouped by what you are trying to do.
Three flags work everywhere. --on <target> acts on an install named in
weft.toml instead of the one on your machine
(deploying to your cloud). If you want to reach a runtime by its address
rather than a target name, pass --dispatcher <url> or set
WEFT_DISPATCHER_URL; you cannot combine it with --on. If you are scripting weft, pass --json for machine-readable
output.
You never log in to the install on your own machine: every command sends it the key it made for itself when it was set up. If you want to call its API from a script, or wonder where that key lives, go and read the key the install’s API asks for.
--json changes behaviour, not just formatting, in three places:
weft run --jsonalso detaches, so it starts the run and returns.weft infra terminate --jsonnever asks, so it needs--yes.weft connect --jsononly works with--list,--grant,--disconnect,--forgetand--move. The walkthrough prints for a person.
If a command fails under --json, weft prints one line with "phase":"error"
(the reason is in .detail.message) and exits non-zero. So if your script
reads a field like .triggers from weft status --json, check the exit code
first: on that error line, jq '.triggers' gives null. A project that was
never registered is not a failure: weft status --json prints
{"registered":false,"projectId":"..."} and exits 0, just as weft status
does without --json.
Tab completes commands and flags in a new terminal after ./setup.sh. For
zsh and bash, setup adds a short block to ~/.zshrc or ~/.bashrc, between
two # weft tab completion marker lines, and ./setup.sh --uninstall takes it
out again. For fish it writes ~/.config/fish/completions/weft.fish. Every Tab
asks the installed weft for the candidates, so new commands and flags show
up as soon as you update it.
When your files change, or part of them is broken
Every command that builds (run, build, bake, activate, resync,
infra start and infra upgrade) first syncs this folder with the install’s
copy of it, then has the install record those files as a version and build
that version. The command compiles nothing itself: the install compiles the
version, so a compile error it prints is about the files in that version. A
file you change after that goes into the next command. For how your folder
and the install stay in step, go and read your folder is a
copy.
A part of the program that does not compile (a .weft file that does not
parse, an include that does not resolve, a node type weft cannot find, a wire
between two ports that cannot meet) only stops what needs it: everything else
goes on, and every command that builds warns, with the file and line:
warning: does not compile, so nothing that needs it runs: <what is wrong, with its file and line>
A run that needs a broken part is refused before it starts, and so is
weft activate for a trigger whose runs would reach one. While a broken part
hides nodes (an include that does not compile, a node type that did not
load), weft takes nothing down for being missing from your source; for why,
go and read infra you deleted from the
source.
A node package whose Rust does not compile is left out of the worker rather than sinking it, and the build says so:
warning: the worker was built without it, so nothing that uses its nodes runs: package 'whisper' didn't compile: <its first error>; its full errors are in the build log: <address>
Runs that never use its nodes run; one that does is refused before it starts:
the run cannot start:
it needs parts of the program that do not compile:
transcribe: package 'whisper' didn't compile: <its first error>; its full errors are in the build log: <address>
The package is built again once its files change, so fix it and run the command again.
Start a project
| Command | What it does |
|---|---|
weft new <name> [--assistant <who>] [--remember] [--ci <cloud>] | Scaffolds weft.toml, src/main.weft, nodes/, a git repo and a .gitignore. --assistant copies Tangle into this project only, and is repeatable. If you leave it out, the default --remember saved is used, and with no default weft asks you to pick from a list, the assistants it found on your machine first. With nobody to ask (no terminal, or --json) it stops and lists the values instead. --assistant none is how you make a project without Tangle; --assistant none --remember also clears the default, so the next weft new asks again. It ends by printing the command that opens the project in your assistant. --ci gcp also writes the deploy workflow. If any step fails, nothing is written |
weft catalog update | Re-copies nodes/base_catalog/ from the installed weft. Replaces the folder, so your edits in there go |
weft tangle update [--assistant <who>] | Re-copies Tangle. Replaces every file the template owns; anything your assistant wrote beside them stays |
Run something
| Command | What it does |
|---|---|
weft run | Has the install record your files as a version and build it, then starts a run and follows it |
weft run <example> | The same, with a saved example’s starting values |
weft build | Has the install record your files as a version, build it and register it, starting nothing. Also how you put back code the dispatcher no longer holds |
weft bake | Prepares every trigger’s settings and starts no listeners |
weft stop <run-id> | Cancels a run |
weft wait <run-id> | Waits until a run ends, then says how in one line and in its exit code (below) |
weft wake <run-id> <node> | Resolves a pure time wait now. Refused for a wait that expects a value. If the node sits in a loop and waits in several iterations at once, --iteration 3 says which one, and without it the refusal lists them |
weft follow <project> | Streams a project’s events live |
Flags on weft run:
| Flag | What it does |
|---|---|
--detach | Start and return rather than following. The run is still started before the command returns: preparing it can take a while (a --seed reads the results of the run it reuses), and the command says what it is doing every half minute meanwhile. Once the install has the request, the start goes on there even if you press Ctrl+C, and weft runs lists the run once it starts |
--referenced | Compile only the node types the graph uses, rather than the whole catalog |
--target <node> | Run this node and what it needs. Repeatable |
--before <node> | Run what it needs, and not it. Repeatable |
--from <node>=<json> | Start here, handing these ports their values. A port you hand a value keeps it: nothing a wire brings replaces it |
--emit <node>=<json> | Stand in for a node: say what it would have emitted, and do not run it |
--group <group>=<json> | Run one group, or one included file, on its own, with the values you hand it, a group inside a loop’s body too. A port a node inside needs that you do not hand is refused before the run starts |
--feed <start> | Also run what feeds this start: for each input you did not hand a value, the node that feeds it (through any groups and included files on the way) and nothing above it. Names a --from or --group start. Repeatable |
--fire <trigger>=<json> | Fire exactly one trigger with this event, using its bake. One per run |
--seed[=<run>] | Reuse, from head’s run or the run you name, everything whose slice of the program did not change (any finished run of the project, one a trigger started or one that failed included). A node that runs again and waited for answers there gets those answers back, matched to its waits in order, if its code did not change |
--seed-until <node> / --seed-before <node> | Where reuse stops. Both need --seed |
--root | Start a new version tree, parented on nothing. Refused beside --seed, since a new tree has nothing before it to reuse |
--check | Only check that the run could start, building and running nothing: it prints the run can start and the infra the run reads, and exits 0, or prints the refusal and exits 1. With --json, the infra is the infraRead list, and with several targets targets says what stops each one if it ran alone |
--save <name> | Write these settings to examples/<name>.json before running |
--clear <field> | Clear a saved setting before applying flags: from, emit, target, before, group, feed, fire, instance, keeping (what --durable or --fast saved), keep_for, hold_secs, pause_after_secs |
--durable | If the run has to carry on after its worker dies: another worker picks it up where it stopped, and a node caught halfway is failed rather than run twice. For what that costs in speed, go and read when a run waits for its writes |
--fast | If the run fires a durable trigger and you want it fast anyway: it lives in its worker’s memory and ends if that worker dies |
--keep-for <duration> | If you want this run kept longer or shorter after it ends than its trigger or project says: 30m, 12h, 7d, or forever. For the project and trigger settings, go and read how long a run is kept |
--hold-secs <seconds> | If a wait in this run should hold longer or shorter than its trigger’s holdSecs while the run cannot pause: 0 fails such a wait at once, at most 30 days. For when a run cannot pause, go and read triggers and routes |
--pause-after-secs <seconds> | If this run should stay on its worker longer or shorter than its trigger’s pauseAfterSecs before it pauses: 0 pauses at once, at most an hour. For when a run pauses, go and read triggers and routes |
--instance <id> | Run for this instance of the program. Needed when the run reaches a node that exists once per instance (see programs with instances) |
The cut flags (--target, --before, --from, --emit, --group,
--feed) take a node or a group anywhere in the program, inside a loop’s
body too. If you start and end inside a body, only that part runs, once,
inside the loop’s first iteration, and you hand it what the loop would have.
A run that holds one end of a loop without the other is refused. For the
details, go and read running part of a loop’s
body.
A run that fires a trigger (--fire) follows the trigger’s durable,
keepRunsFor, holdSecs and pauseAfterSecs, and --durable, --fast,
--keep-for, --hold-secs and --pause-after-secs override them for that
run; a run
that fires nothing is fast unless a flag says otherwise. A run you start by
hand is always recorded, even when its trigger has recorded off, so you can
always find it in weft runs. For what each of these does when you set it on a
trigger, go and read how a run is
kept.
If you want to hand a run a file its real source cannot give you here (a
WhatsApp voice note needs a paired phone), write it in any --from,
--emit, --group or --fire value, or in a saved example, the way you
would write it in source. "@asset(\"samples/hello.ogg\", Audio)" uploads
the file into the project’s storage and hands the port the file value its
type names (Image, Video, Audio or Blob); "@file(\"prompts/x.md\")"
hands it the file’s text. Both mean exactly what they mean in a .weft (see
files and reuse), and a path is relative to
the project root:
weft run --emit receive='{"messageType":"audio","file":"@asset(\"samples/hello.ogg\", Audio)","seconds":3}' --target transcribe
A missing file stops the run before it starts, and so does a file on a port whose type does not take one. A saved example keeps the marker, so each run reads the file again.
If you run part of a program and it needs a value that only a trigger the run does not fire would give, the run is refused before it starts, naming both ways through:
the run cannot start:
summarize.text gets nothing in this run: it only comes from the trigger inbox, which this run does not fire. Fire it (--fire inbox='<event>'), or hand the value at a start (--from summarize='{"text": ...}').
If the trigger it does not fire only feeds nodes that can do without it, you get a warning instead, naming the nodes that skip unless something else feeds them. For how a partial run is cut out of the program, go and read how a program runs.
Waiting on a run
If you started a run with --detach, or a trigger started it, weft wait <run-id> waits for it to end and says how, in one line and in its exit code.
If you are scripting weft, read the exit code rather than the line:
| Exit code | The run |
|---|---|
0 | completed |
10 | failed, or was cancelled |
11 | is waiting for input (only with --waiting-for-input, which also returns as soon as the run parks on a person or a service) |
12 | is still going when the timeout runs out |
1 | the wait itself failed: no such run, or the install unreachable |
--timeout takes a whole number and a unit (30s, 10m), from one second
to 10 minutes, and defaults to 10 minutes. A timeout only stops the waiting: the run carries on, and weft wait again
picks it back up.
weft run, when it follows its run to the end, exits with the same codes: 0
when the run completed, 10 when it failed or was cancelled. With --detach or
--json it only starts the run and exits 0.
What nodes need from your machine
Some nodes start a program that has to be on your machine, like ExecPython
and its python3. If your program uses such a node and what the node needs
is missing, weft run, weft activate and weft resync refuse before
anything starts, and say what to run:
this program uses nodes that need things the machine its workers run on doesn't have:
ExecPython (node 'greet') needs python3
install them with: weft deps-install ExecPython
Those three check only what a node needs while it runs. What a node’s
package needs to compile is not checked: if it is missing, the package does
not compile and the build leaves it out, naming it (for what that looks
like, go and read when a package does not
compile). weft deps-install installs both. If what a node needs
goes missing after the check (somebody uninstalled it), ExecPython fails
with those same lines, naming only itself, and so does any node that reports
it through weft::error::need_missing.
weft deps-install prints the install command for your system (sudo apt-get install -y python3, brew install python), runs it once you say
yes, then checks again and tells you what is still missing. It installs only
what the machine lacks, with apt (Debian, Ubuntu), dnf (Fedora, RHEL,
CentOS, Rocky, AlmaLinux), apk (Alpine) or Homebrew (macOS). When a node
names no package for your system, weft deps-install says so, and you
install that one by hand. On a system none of those four serves, it installs
nothing, and you install everything by hand.
If weft deps-install says that what it installed is still missing, it is
not on the PATH of the shell you ran it from: put it there, then run
./setup.sh again from that shell. If it says everything is there but weft run still refuses, the program is on your shell’s PATH but not on the one
the install checks with, which is the PATH of the shell that last ran
./setup.sh: run ./setup.sh again. For what else a worker’s environment
holds, go and read the worker.
| Command | What it does |
|---|---|
weft deps-install | Installs what the nodes of the program in this folder need and the machine lacks. A node type the program does not use is never asked about |
weft deps-install <NodeType>... | The same for those node types, whether the program uses them or not |
weft deps-install --all | The same for every node type of the project’s catalog |
If you are scripting it, --yes runs the install command without asking,
and --print only prints it. --stage run keeps only what nodes start
while they run, and --stage build only what their packages need to
compile. If you write a node that needs something, go and read what a node
needs from the machine.
Look at a run
| Command | What it does |
|---|---|
weft runs | Past runs, newest first. --limit (50), --offset, --project, --phase, --node (the node that started the run), --since 2h, --status, --instance, --tag. --through <node> keeps the runs in which that node fired, wherever it sits. --search <words> keeps the finished runs that carried every one of those words somewhere in what they recorded (the trigger’s input, what a node sent on, an error, a log line): an email, an order id, a phrase in quotes. --through and --search only find recorded runs that have ended, and they find one as soon as it ends. weft builds a run’s search entry the first time anyone searches after it ended, so the first search after many runs ended takes longer. In the plain list, a run that a trigger with recorded: false started shows only if weft wrote it down after all (for when, go and read how a run is kept), and --through and --search never find it. A run you start by hand is always recorded, so it always shows |
weft events <run-id> | One run’s events in order. --node, --kind, --full for whole values. Nodes are named the way the program that run ran names them, so if you moved or renamed a node since, you find it on an old run by the name it had there. If you want one time round a loop, add --iteration 3 to a --node that names the loop or a node inside it: you get that node’s rows from the fourth time round (they count from 0, outermost loop first, the same numbers weft wake --iteration takes), and 3.0 the first time round a loop inside that. --iteration needs --node, because on its own 3 would match the fourth time round every loop at once. A caller row that names messages the caller never received carries dropped=<offsets> (<why>) first |
weft logs [<run-id>] | What the nodes wrote, plus every failure. A run that wrote nothing lists what it skipped and why (under skipped with --json). --limit |
weft status | The cwd project: registration, listener, every infra node the program declares (one never started says not started, a @per_instance one how many instances have a copy), each trigger, whether it is on and behind the source, what infra it reads, and the version of the source it fires (with what it holds while it is down, and the events an instance’s trigger holds until a field is filled), recent runs, how many runs each trigger started in the last minute or two and how many of those failed (for a trigger with recorded: false, often the only sign its runs happen), and how many fires and calls it dropped in that time with why the last one was dropped (refused before any run started, because the program cannot run what they reach, the trigger takes no work or it has no armed program; nothing tries a dropped fire again, so this is where you see it), what drifted, and what you can do next. It also shows the project’s own address (its routes at the root), or why that address is unavailable. If requests waited for a database connection in the last 15 minutes, a database: busy line says how often, what the database takes and each copy’s share of it, and how to give the install more (see how many connections the install takes). While the install builds, it names each image building and where its log is; while infra starts or a trigger turns on, how long it has been going and what it waits on. If something weft does for the project keeps failing (handing runs to its workers, a worker that has answered busy for longer than the install’s alert delay included, acting on its infra, finishing an infra command it then gave up, repairing work left behind by a worker or a weft process that died, getting the listener to let go of triggers whose door stopped taking work, removing stored files from the bucket, deleting what the infra host still holds of removed projects, a resync that stopped with triggers down, and on a cloud install also waking one of weft’s services or running one of its background jobs), it lists it under waiting on weft: as <what> (failing for <time>, last tried <time> ago; <what weft does next>): <error>, and keeps it there until it works again. The lines about weft’s own parts and jobs, the bucket, the install’s news and removed projects are install-wide, so every project shows them. The triggers a resync you gave up with weft cancel-activate left down show there too, as a given-up resync left these triggers down: 'feed', 'door', until they are on again or the next resync begins. A trigger taken down with its infra or by the health check says so, and one that cannot come back because other triggers are on another build says held back, with the way through. Under triggers:, a trigger whose queued events wait on something says what they wait on: the instance’s values changing, its infra running again, connections being picked, or a failed start that holds the queue until weft activate, a new build or a change of the worker settings |
weft ps | Every project the dispatcher knows |
weft instances | The project’s instances, one line each: its triggers on, its infra copies up, and its last run |
--phase fire hides the setup runs that an activate or a resync makes, so it
answers “has my trigger fired since I changed it”.
Versions and examples
| Command | What it does |
|---|---|
weft tree | Every working copy of the project and where its head is, then the version tree with the runs started by hand beneath each version. It only reads: when the install does not know this folder yet, it starts with this folder: not registered. --json adds workingCopy, the id of this folder’s working copy (null when it is not registered) |
weft checkpoint [<label>] | Records this folder’s files as a version and moves its head there. No run, no build |
weft branch <ref> | Records this folder’s files as a version, moves the folder to the version (or run) you name and writes its files here, then prints the weft branch that comes back. --copy <copy> moves another working copy instead (program, a folder’s absolute path, or the start of a working copy’s id) and leaves this folder where it is |
weft forget-copy <copy> | Records another working copy’s files as a version and removes the copy, for a folder that is gone or the program copy, when its head keeps weft prune from deleting a version. <copy> is named as for --copy. A folder that still exists registers again at its next command that syncs it, which is why this folder’s own copy is refused |
weft resolve [<file>...] --keep folder|incoming | Settles the conflict copies a sync wrote beside files that changed both here and in the install. --keep folder keeps this folder’s files and deletes the copies; --keep incoming moves each one over its file. Then it syncs, so the working copy gets the result. Name a file to settle its copies, a conflict copy to settle that one, or nothing to settle every one in the folder. --keep has no default, and --keep incoming refuses a file with several conflict copies until you name one. --json answers keep, resolved (the files settled) and copy (the working copy after the sync) |
weft diff <left> <right> | Compares what two runs put on their wires. A ref is a run id or example:<name>. --full for every wire |
weft freeze <name> [<run-id>] | Writes examples/<name>.json from a run: its parameters, the answers people gave, and the outputs you accept. --expect <node> emphasises a node in later diffs |
weft examples | Saved examples, whether each is frozen, and its latest run |
weft prune <version> | Deletes a version, everything under it, and their runs |
Triggers
| Command | What it does |
|---|---|
weft activate | Turns on every one of the program’s own triggers that is off, and leaves the ones that are on alone; with nothing off it says so and succeeds. Builds and registers first if it has to, and prints the project’s own address, where its routes answer at the root (on your machine a local port, on a cloud install its Cloud Run address). It never starts infra and never touches a run going now: it refuses a trigger whose setup reads infra that is not running (a trigger whose runs alone use it is turned on with a note naming it), and refuses while a connection the program needs is not picked on this install, naming each node. A trigger that runs once per instance is left off and named in a warning, with the way to switch it on. --held run|drop decides what happens to the work the triggers held while they were down: run it (the default, at half each trigger’s limits) or drop it. The activation runs on the install, so Ctrl+C, a closed terminal or a dropped connection stops only the following: weft status shows each trigger turning on, for how long and what it waits on (usually its setup run), and weft cancel-activate stops it. It names every trigger that is on and still fires an older build or an older cut of the program once it is done (2 triggers on are behind the build this activated with ('feed', 'door'), firing an older one; weft resync brings them up) |
weft activate --no-build | Turns the triggers on with the build the install has registered, and builds nothing from the folder: what a script that must not move the project onto the folder’s code wants, such as a deploy’s exit trap. weft activate <project-id> does the same from anywhere |
weft activate --port <n> | On your machine, opens the project’s own address on that port, and the project keeps it for later activates. If another program or another of your projects holds it, nothing is activated. Without it, the project keeps the port it has, or takes a free one |
weft deactivate | Takes the program’s own triggers down, and tells you how many instances still have theirs on |
weft resync | Builds, then brings every trigger that is on and behind the build onto it, and leaves every other trigger alone. With nothing behind, it says no trigger that is on is behind this build; nothing to resync. It turns on nothing you switched off. It does first bring back, on the build it just made, what an update’s weft maintenance off could not bring back in this project, and says what came back and what is still down. Any other trigger of the new build that is off is named with the weft activate that would. Before it takes anything down, it checks everything that could refuse it, so a refusal leaves your triggers as they were. That includes whether each trigger it brings back could come up on this install: a trigger whose connection needs the service to send events to your weft is refused, naming it and the fix, when nothing on the internet can reach your weft. While it waits for runs on the old version to end, it names them (the new version takes over once 2 runs on the old one end: ...), with weft stop <run> to end one now and weft cancel-activate to give the resync up. It says where it is again every 30 seconds while nothing changes (still resyncing (elapsed 5m30s): ...), so a deploy log never sits silent while the triggers come back. weft resync --resume takes up again a resync that stopped with triggers down, and takes no other flag. While a resync runs, or stands stopped, a build you start is refused with cannot build now |
weft cancel-activate | Cancels an activate in flight, one still waiting to start included. With no --trigger or --instance, it also gives up a resync, running or stopped, and names the triggers it leaves down |
weft cancel-build | Cancels a build in flight. The install stops the build, and the command that was building fails saying it was cancelled |
For what each of these does to work in flight, a resync that stops, and everything else about turning triggers on and off, go and read starting and stopping things.
deactivate, resync and the infra verbs that take infra down (stop,
terminate, upgrade) take --door take|hide|refuse,
--door-for <minutes>, --held keep|drop and --running wait|cancel, each
with a default that loses nothing (take, keep, wait). For what each answer
does, go and read taking triggers
down.
Every trigger verb also takes which triggers it acts on:
| Flag | What it does |
|---|---|
--trigger <name> | Only this trigger, spelled the way the program writes it (receive, one.receive). Repeat it for several |
--instance <id> | That instance’s triggers, the ones that exist once per instance. Repeat it for several (see programs with instances) |
--all-instances | Every instance’s, and not the program’s own |
--all | The program’s own and every instance’s, in one command |
With none of the last three, a verb acts on the program’s own triggers only.
Infrastructure
| Command | What it does |
|---|---|
weft infra start | Builds, then starts what is not running and nothing else. The triggers a stop of that infra took down come back on by themselves once it runs; no other trigger is touched |
weft infra stop | Scales to nothing, keeping the disk. The triggers that depend on it go down first, and come back on by themselves once it runs again |
weft infra terminate | Deletes it, disk included, except the disks the node lists in keepOnTerminate. Takes down what stop takes down. Asks first, and off a terminal or with --json it needs --yes |
weft infra upgrade | Builds, then replaces only what is running on another build: it starts nothing that is stopped and makes nothing that is missing. The triggers that depend on what it replaces go down for it and come back once it runs again |
weft infra status | Where each copy that exists stands, with its address. A start or stop on its way shows as provisioning or stopping at once |
weft infra list-doors | Which pieces you can reach from this machine, and at what address |
weft infra logs [<node>] | What the containers printed. --tail (200), -f |
weft infra cancel | Calls off the infra work in flight. Halts between steps rather than undoing |
weft infra look | If weft status says the supervisor could not get through the project’s infra and you have fixed the cause, this asks it to go through the project again (on a cloud install, with a fresh round of three tries). A health rule that gave up tries again too. For what the supervisor retries by itself, go and read the supervisor |
weft infra show <node> | The node’s card: every item (label, kind, value) and every button, with the action to hand press and what it asks before pressing. A secret item’s value is never printed, --json included |
weft infra press <node> <action> | Presses the card’s button with that action, without asking: naming the action is the choice. Prints what the node answered, and the button’s warning if it has one |
weft infra rebake <node> | If an infra node’s baked outputs no longer match the real thing (say its password was reset outside weft), this applies the node again, leaving alone anything already running the way the node asks, and saves its baked outputs anew. If a weft infra start, upgrade or another rebake is already working on the same copies (the shared ones, or that instance’s), it waits for that to finish first |
weft infra take-disk <volume> | If a start refuses a disk volume an older weft named without the install, because two installs on this machine ran that infra on it (or it has no install label), this gives it to the install the command acts on (WEFT_INSTALL=<name>, default for the default install). That install’s next start of the infra copies the volume into its own and removes it; every other install that ran it starts that disk empty |
weft infra env <node> --into <file> --set NAME=Label ... | Writes what the node’s card shows into an env file, every name in one go: --set DATABASE_USER=User --set DATABASE_PASSWORD=Password takes each item by its label as show prints it, --as <NAME> is the short form for the card’s only secret, and --door DATABASE_ADDRESS=pg writes where one of its doors answers (host:port, by the endpoint list-doors prints). A secret’s value is never printed; the other values are, in the line confirming what it wrote. A new file is yours only; an existing one gets those lines set and keeps the rest. A door’s port may change when the infra restarts or is upgraded, so run it again then. If a secret was handed over already, it refuses, quoting what the card says there: press the card’s button with weft infra press, then run it again |
If a build changes an infra node’s image, every command that builds says so.
A copy started after that gets the new image; one already running keeps the
old one until weft infra upgrade.
start, stop, terminate and upgrade act on every piece of the
program’s own infra. --infra <node> (repeatable) names only some, spelled as
weft infra status lists them (db, or one.db for the db inside the file
included as one); start --infra db also starts what db needs
that is not running, and stop --infra still reaches a node the program no
longer declares. stop, terminate and upgrade take --door,
--door-for, --held and --running as above: with --running wait, the infra stays up until every
run reaching it has ended, even when no trigger reading it is on. stop --force also brings down the units a node keeps up on a normal stop, so you
can update them on the next start.
If you want an instance’s copies of the nodes marked @per_instance instead
of the shared infra, start, stop, terminate and upgrade take
--instance <id> (repeatable), --all-instances and --all like the trigger
verbs, and cancel, show, press, rebake and env take --instance <id>. Naming a @per_instance node with --infra and no instance is refused,
naming those flags.
Connections and tokens
| Command | What it does |
|---|---|
weft connect | Walks you through connecting an account |
weft connect --list | Prints what is stored, changing nothing. Works outside a project |
weft connect <node> | Go straight to one node that takes a connection: an access node, or a Route or Socket whose auth checks its callers |
weft connect --grant <id> | Pick a stored connection for the node |
weft connect --disconnect | Clear the node’s pick, leaving the connection stored |
weft connect --forget <id> | Delete a stored connection for good |
weft connect --instance <id> | Act inside that instance of the program on an access node whose connection is @instance_filled: --list, --grant, --disconnect and connecting a new account all work on its connections and its value, exactly as its settings page would |
weft instance-values --instance <id> | Lists what the program asks that instance to fill and what it was given; --set node.field=value and --clear node.field (both repeatable) change it in one go, re-arming its live triggers that read a changed value |
weft connect --door shared|own | Which door to connect through, without being asked |
weft connect --set NAME=VALUE | Fill a credential field. The value is visible to other processes, so prefer the prompt or --set-env NAME=ENV_VAR |
weft connect --label <name> | Name the new connection, so two of one service read apart in the list. Without it the connection is named after the service, or after the app it signs in through |
weft token mint | Mint a signal token. --name, --projects, --tags, --display <node>, --displays, and --expires <duration> (30m, 12h, 7d; without it the token works until revoked). Printed once and never again. --operator mints an operator key instead, which administers the whole install and takes no scope |
weft token mint --instance <id> --expires <duration> | An instance token for the project you are in: it acts inside that one instance and nothing else. --expires (30m, 12h, 7d) is required |
weft token ls | The tokens you have, by their recognizer prefix |
weft token revoke <id> | Revoke one |
weft options <node> <field> [--search <text>] | Lists the choices a field with a searchable list offers (a model, a spreadsheet, a channel), the same list the editor shows when you search that field, read through the connection picked for that node. One id label line each; the id is what you write in the field. <node> is the node’s name as written in your source (path/to/file.weft:name when two files use that name), and <field> is its input’s name |
weft connect-lib | Copies weft’s connect library into the project’s frontend (front/src/lib/weft-connect, or the folder --into names), so its pages show the editor’s connection pickers and list fields, and each instance gets its own settings page. The folder is the library’s: each run replaces it, so run it again after updating weft and keep your own code outside it |
Cloud installs
| Command | What it does |
|---|---|
weft target add <name> <url> | Names an install in weft.toml. Replaces the address when the name exists |
weft target list | The targets, and whether you are logged in to each |
weft target remove <name> | Takes one out of weft.toml |
weft login <name> | Stores your operator key for it, after checking the install accepts it. If you are scripting it, --key-stdin reads the key from standard input. Refused for an install on this machine (the local target, or any target at its address), whose own key every command already sends |
weft logout <name> | Forgets your key for it |
weft target key | Prints, as JSON (url, operatorKey), the address and operator key a command with the same --on (or --dispatcher) would use, or with neither, those of this machine’s install. The editor calls it to reach its install |
weft running-source <folder> | Writes the program an install is running into a new folder, without touching your own files. Add --on <target> to get what a cloud install runs |
weft ci add --cloud gcp | Writes .github/workflows/deploy.yml. Refuses to replace one somebody edited |
weft target export <name> [--github] [--frontend <name>] | Mints a CI operator key on the install, and gives the frontend the install hosts for this repository a new token along with its service’s name (the old token works until the workflow’s next run deploys the new one and retires it). With --github it sets them, and the workflow’s other settings, on the repository through gh, and lists what it set; without it, it prints the variables and writes the secrets to a file only you can read, under the install’s exports/ folder |
weft target show <name> | Where a target’s install lives: its address, and on a cloud the project and region it runs in, and the weft it runs |
weft frontend add <name> [--repo owner/name] | Makes one of the project’s frontends. With --repo (read through gh), a cloud install makes it a Cloud Run service and lets that repository deploy to it, and its token comes from the deploy workflow (weft target export); without, it runs wherever you run it, and its token is written to a file only you can read |
weft frontend ls | The project’s frontends, and where each runs |
weft frontend visitor-address <name> <url|off> | Sets where that frontend’s server passes weft’s doors on to its visitors (its weftPassThrough mount, like https://shop.example/weft), so links to the project’s stored files go through your site; off clears it. One frontend of a project has one at most (your own site in front) |
weft frontend rm <name> [--force] | Removes one: its tokens stop working, and a service the install made for it is deleted. --force forgets it even when the service cannot be removed, naming what stays |
weft frontend token <name> [--done <id>] | A new token, beside the one it has, with its id; --done <id> once that token is in place retires every other token of the frontend |
weft domain add <name> | Makes a cloud install answer at a domain you own: prints the DNS record to set and waits until it points there. --for api serves the project’s routes there instead, --for frontend --to <address> its frontend. --no-wait returns after printing the record. The first domain makes a load balancer that is billed while any domain exists, so it needs --accept-cost (your own domain) |
weft domain list | Every domain, with its DNS record and where its certificate stands, the address the install’s links use, and the sign-in callback to give your providers once a domain is that address |
weft domain rm <name> | Stops answering at it; the last one takes the load balancer down |
weft workers | The project’s worker settings: copies kept warm, the most copies, how many calls Cloud Run sends one copy at once (concurrency), CPU, memory, how long a copy holds on to things its runs share, such as a database pool, after nothing uses them (shared_idle_seconds), how many calls and events one copy takes at once (max_runs_at_once) and how long a call waits when it is full (max_queue_wait_seconds; for both, go and read when a worker is full). It shows which the project sets and which come from the install. CPU and memory only apply on a cloud install, where an unset CPU means one CPU; on your machine no worker is capped |
weft workers set --min-instances 1 ... | Changes settings for this project. On a cloud install, calls after the change go to workers that have them, and a worker started before finishes what it runs and then stops. On your machine, the worker at the project’s own address keeps its port, and keeps answering there, until it has nothing left running; then it is stopped and a worker with the new settings takes the port |
weft workers reset [<setting>...] | Puts settings back to the install’s values |
Stored files
| Command | What it does |
|---|---|
weft migrate connections --to <target> | Copies saved connections to another install, their secrets sealed to it. Name what: --all, --id, --since/--until, --service, --project; --dry-run copies nothing (moving to the cloud) |
weft migrate files --to <target> | Copies stored files to another install, keeping each one’s key, version and lifetime. Name what: --all, --id, --since/--until, --prefix; --dry-run copies nothing |
weft files ls [<prefix>] | Everything stored, grouped by space |
weft files inspect <key> | One file’s full metadata |
weft files download <key> [-o <path>] | Streams it out. A failed download never leaves a half file behind |
weft files rm <key> | Removes one file |
weft files rm <space>/ | Removes a whole space, kept files included |
weft files usage | How much is stored, and how many files |
The daemon
| Command | What it does |
|---|---|
weft daemon start | Brings the runtime to the state the code describes. Idempotent, so it is both first boot and refresh. restart is the same thing |
weft daemon stop | Stops the runtime and the workers it started. The database, your infrastructure and your data stay |
weft daemon status | Whether it is reachable, and the public address if one is open |
weft daemon logs | The runtime’s log. --tail (100), -f |
weft daemon remove | Takes a named install off this machine, everything of it included (its images and compile caches too): the install of the branch you run it from, or the one WEFT_INSTALL names. The default install is removed by ./setup.sh --uninstall |
weft daemon which | Which install a command run here acts on, and where its files are. --json for scripts |
If the runtime image or the standard worker program is broken or was changed by hand, weft daemon start --rebuild rebuilds both even when they are there, compiling the standard worker on your machine (so it needs Rust), and ./setup.sh --rebuild passes it on. To open or close the public
address, weft daemon start takes
--public-url and --no-public-url.
Updating weft
An update replaces the weft that runs your install, and an older and a newer
weft never run side by side. So if you want to update, you switch everything
off first, update, and switch it back on: weft maintenance on, then
./setup.sh (or the cloud’s install workflow), then weft maintenance off.
Add --on <target> for a cloud install. Both the installer and the workflow
check first, and refuse to replace anything while something is on.
| Command | What it does |
|---|---|
weft maintenance on | Takes down every trigger that is on, in every project, with its door refusing what arrives meanwhile (what it held already is kept), and stops every infra copy that runs. Every unit of a copy comes down, even one its node keeps up on a plain stop; disks are kept. Then it waits for the runs going to finish and the copies to stop, however long that takes. --running cancel cancels the runs instead. Until off, nothing can be turned on: an activate, an infra start, a resync or a weft run is refused, naming off, and nothing weft brings back on its own comes back. A trigger it took down takes no work, as with any door that refuses: an event for it is dropped, and a live call is answered 503 with a Retry-After. You can run it again: it takes down whatever is on by then and forgets nothing it took down before |
weft maintenance off | Builds every project on took something down in again on the new weft, from the same files and settings its last build used, then starts the infra copies on stopped on that build, waits until they run, and brings back the triggers on took down on it, with what they held before on. A project last built by a weft older than this command has nothing stored to build again from: it comes back on the build it has, and off says to run weft resync in its folder. Only those: a trigger you took down yourself during the maintenance stays off, and a copy you terminated is skipped. If something in a project does not come back, off says what and why, and ends the maintenance anyway: everything else comes back, and what did not stays down until that project’s next weft resync. If the message says the project’s copy of the standard nodes is older than this weft, run weft catalog update in its folder; otherwise fix what it says (usually something in the project’s folder). Then weft resync there builds it, first brings back what on took down in it and off could not bring back, and says what came back. If you want some of those triggers left off, weft deactivate them before the resync. Running off again also tries what is left and says again what does not come back, but it builds from the same files as before, so it does not pick up a change you made in the folder. Projects come back side by side, but across all of them at most four setups run at once (a setup is the infra, or the triggers, of a program or of one of its instances), so on an install with many instances the rest wait their turn. While it runs it shows a line per project with its step and how long that step has taken; for each worker it compiles and each image it builds it prints where the build’s log is, and on a local install the line also shows the log’s latest line. It runs on the install, so closing the terminal does not stop it |
weft maintenance | Whether the install is in maintenance and since when, what off brings back, and what is on right now. status is the same |
weft maintenance check | Fails while something is on (a trigger, an infra copy, a run going). It is what the installer and the workflow run before they replace anything. An install that is not set up or not running passes, and so does one whose weft is older than this command, saying it cannot tell |
weft status says when the install is in maintenance.
Testing nodes
| Command | What it does |
|---|---|
weft test-node [<node or package>...] | Runs node tests, one build for every target. Without --tier, the basic and fake tiers, which need no install |
weft test-node --tier live | Calls the real provider with a real credential, and can spend money |
weft node-test-hash [<target>] | The content hash naming a package’s test inputs |
Other flags: --test <name> for one test, --key <service> and
--connection <service>=<id> for live credentials, --parallel [N] to run up
to N packages side by side (each with up to N tests at once; bare, all of
them), and --yes to skip the live confirmation.
What the editor drives
| Command | What it does |
|---|---|
weft describe-nodes --list | One line per node type: its name, its tags, the first sentence of what it does. The cheap first look. After the types, every .weft file under nodes/ a program can @include, spelled the way you write it (__catalog/...), with its first-line comment |
weft describe-nodes --node <Type> --compact | One node’s wiring view: ports, types and rules, with the presentation stripped out. A long pick list shows its first eight entries and how many more there are; the plain --node <Type> has them all |
weft parse | Parses source from stdin and prints the project, the catalog and the diagnostics as JSON. Lenient: unknown types become placeholders |
weft validate | The same input, strict, including the runtime rules like a provider with no model named. Also warns about every node under nodes/ or src/ it had to leave out, and every name a node’s metadata.json spells off the weft way; neither fails it |
weft parse-server | A long-lived parse server, one JSON request per line, catalog held warm |
Cleaning up
| Command | What it does |
|---|---|
weft clean | Deletes runs older than --keep-days, 30 by default |
weft clean <run-id> | Deletes one run |
weft clean --all | Deletes every run |
weft clean --project <id> | Deletes that project’s whole history |
weft clean --instance <id>, --tag <tag>, --status <how>, --node <node> | Deletes only the runs these name, across the projects you name (or every one). Naming an instance or a tag takes all their runs; add --keep-days to spare recent ones. A run still going is left to finish unless you pass --cancel-running |
weft clean --images | Reclaims the workers and infra images nothing references (on your machine a worker is a program under ~/.local/share/weft/workers/, on a cloud install an image), then prints how much the worker compile caches (~/.local/share/weft/worker-target/) and the crates your machine compiles workers from (worker-crates/) take. --all spans every project, and also removes what installs no longer on this machine left behind (below) |
weft clean --build-cache | Drops the node-test cache, waits for the builds compiling right now to end, then drops the worker compile caches and the crates your machine compiles workers from (~/.local/share/weft/worker-target/ and worker-crates/), then Docker’s build cache. The next build of a project with its own nodes then compiles the whole standard library again |
weft rm | Unregisters a project: its triggers taken down (the door refusing, what they held dropped, the runs going cancelled), infra terminated, runs erased, run and project files deleted. Before it erases anything, it waits until the last calls its cancelled runs queued (an outside job told it was cancelled, say) have gone out, following that in short looks: Ctrl+C stops following, and running weft rm again picks it back up |
weft rm --journal | Waits for every run of the project to end, then erases its runs one by one before unregistering it; weft rm erases them on its own too |
weft rm --local | Also wipes its build artifacts here |
weft rm --all | Both of those |
weft rm --force | Doesn’t wait for the infra to confirm it stopped, and goes on past leftovers on the cloud, naming them. A worker it can’t kill still stops it: nothing is erased while one runs, and the error says how to stop that worker by hand |
If weft rm fails partway, the project stays marked as being removed, and
the way to finish is to run weft rm again in the project’s folder. Until
you do, no new run of it starts, and weft ps lists it with the project is being removed; run `weft rm` again to finish removing it. weft status
still answers, with WARNING: the project is being removed; run `weft rm` again to finish removing it above what is left of it. weft run, weft activate, weft resync, the weft infra commands and every build of it
refuse with that same line. The editor shows it as a banner over the project
and offers no button, weft rm included, so finishing it is a terminal job.
Everything that asks you a question
Every command runs without a terminal. When the deciding flag is missing, weft asks a person if there is one, and otherwise fails naming the flag rather than hanging on an answer that will never come.
| Command | What it asks | The flag that answers it |
|---|---|---|
weft prune | Confirm the count of versions and runs | --yes |
weft rm | Confirm what will be removed | --yes |
weft clean | Confirm the runs to delete | --yes |
weft files rm | Confirm, naming how many files are kept ones | --yes |
weft connect --forget | Confirm | --yes |
weft infra terminate | Confirm what will be deleted | --yes |
weft connect | Which node, which door, which app, which permissions, the credential fields, the connection’s name | <node>, --door, --shared-app, --permissions, --set, --label |
weft test-node --tier live | Confirm that this spends money | --yes |
weft login | The operator key, at a hidden prompt | --key-stdin |
weft deps-install | Whether to run the install command it printed | --yes, or --print to only print it |
deactivate, resync, activate and the infra verbs never ask how to take
triggers down or bring them back: every flag you leave out takes its
default.
Everything destructive, and what it takes
| Command | What goes |
|---|---|
weft rm | The project’s registration, its signals, its runs, its infra including disks, its run and project files (its instances’ files stay) |
weft rm --journal | The same runs, erased before the registration goes |
weft rm --local | Also .weft/target/ and its slice of the node-test cache |
weft prune <version> | That version, every version under it, and every run beneath them |
weft clean <run-id> | One run, for good |
weft clean --all | Every run |
weft clean --project <id> | That project’s entire history |
weft clean --build-cache | The worker compile caches and the crates your machine compiles workers from, the node-test cache and Docker’s build cache |
weft files rm <space>/ | A whole space, including files you marked to keep |
weft deactivate --held drop (or resync, or an infra verb) | Everything the triggers held from an earlier time down, and the runs they started that are paused on a wait are cancelled |
weft deactivate --running cancel (or resync, or an infra verb) | The runs going now, cancelled |
weft maintenance on --running cancel | Every run going now on the install, cancelled |
weft activate --held drop | Everything the triggers held while they were down, answers people sent to waiting runs included, and the runs paused on them are cancelled |
weft catalog update | Anything you edited under nodes/base_catalog/ |
weft tangle update | Your edits to the Tangle files the template owns |
weft freeze <name> | The previous examples/<name>.json, replaced whole |
weft infra terminate | Every infra resource, disks included, unless the node preserves them |
weft connect --forget <id> | A stored connection. Other projects using it then fail at run time asking for a reconnect |
The runtime reclaims the workers and images no project uses each time a
build registers a version and each time a project is removed, and on your
machine also when an infra container stops or is replaced. weft clean --images does the same by hand. weft clean --images --all also removes
what no install on this machine uses: runtime images, the builder bases,
node-test images, standard worker images and image compile caches an earlier
weft or a weft build-images run left, and the worker programs (an earlier
weft’s standard worker among them). It also removes the crates your machine
compiles workers from that no worker program on it was built from
(~/.local/share/weft/worker-crates/), and the images and build caches of
installs no longer on your machine, along with the caches an older weft made
before each install had its own (weft daemon remove drops an install’s
images and caches as it goes). A local install also holds the docker build
cache to 20 GB when it starts and every six hours (and ./setup.sh does on
every install), dropping what was used least recently first.
Starting and stopping things
If you want to start, stop or change infrastructure, triggers or runs, here is the command for each. For why they are three separate things, go and read the three lifecycles.
Infrastructure
A program has infrastructure when one of its nodes needs a container of its own, which you can see in the source because the node says so:
pg = PostgresDatabase { database: "imagebot" }
weft run, weft build and weft activate never start it.
| Command | What it does |
|---|---|
weft infra start | Builds, then starts what is not running. The triggers that went down with it come back on once it runs |
weft infra stop | Scales to nothing and keeps the disk. Start it again and your rows are there |
weft infra terminate | Deletes it, disk included, except the disks the node lists in keepOnTerminate |
weft infra upgrade | Builds, then replaces what is running on another build. It starts nothing that is stopped and makes nothing that is missing (weft infra start does both) |
weft infra status | Where each piece stands, with its address |
weft infra list-doors | The addresses this project’s infra answers on |
weft infra logs [<node>] | What the containers actually printed |
weft infra cancel | Calls off the infra work in flight |
weft infra look | Asks the supervisor to look at the project’s infra again |
If you want only some pieces, go and read the CLI
for --infra.
weft infra status prints one line per piece:
infra for imagebot (a1b2…):
pg [running] -> postgres://…
| Status | What it means |
|---|---|
provisioning | Coming up |
running | Every unit is ready |
flaky | A unit was ready, then stayed not ready for its flaky window (30 seconds unless the node sets another; for how, go and read when a unit stops being ready) |
stopping, terminating | Going down |
stopped | Nothing running, disks kept |
failed | The last apply failed. The message says at which stage |
weft status lists every infra node the program declares, a piece never
started as not started, and under each piece the triggers that would go down
with it: taking it down takes down 'feed', 'door'.
When it hangs
weft puts no time limit on a container coming up, because a model server pulling weights can take a long time. So a bad image or a probe that never passes looks like waiting, not like an error.
Read weft infra logs <node> first, because that is where a service says what
went wrong. Then weft infra cancel. Cancel halts between steps rather than
undoing what it did, so the piece is left partly applied and marked failed;
check weft infra status and either terminate it or run the verb again.
Infra you deleted from the source
Deleting the node does not delete the container. weft status warns you:
WARNING: live infra exists whose node the source does not declare (removed from it, or inside a part that does not compile); it keeps running (and consuming resources) until stopped/terminated via the infra verbs
If you deleted it from the source and have not built since, its own line under
infra: says so too, as orphaned: the source declares it no more, rather
than calling it behind.
It keeps running, and costing, until you stop or terminate it, and weft infra stop --infra <node> still reaches a node the program no longer declares.
The warning says “or inside a part that does not compile” because an included file that does not compile hides every node in it, a database included, and so does a node whose type did not load. While a part of the program hides nodes that way, weft takes nothing away for being missing from the source: it terminates no copy, deletes no disk, and keeps the addresses of triggers it cannot see. So a typo in an included file never costs you the database inside it. Fix the file and the node is back, with its data. If you really mean to remove it, take it out of the source with every file compiling, or stop and terminate it by hand.
Triggers
| What you want | Run |
|---|---|
| My program should answer the world now | weft activate |
| Stop it answering | weft deactivate |
| I changed the program, and its triggers are on | weft resync |
| Prepare a trigger without exposing it yet | weft bake |
| Try one trigger with an event I made up | weft run --fire <trigger>='<json>' |
| I clicked activate and want it back | weft cancel-activate |
weft activate turns on every trigger it names that is off, and leaves the
ones that are on alone. With nothing off it says so and succeeds, so a script
can run it every time. It runs each trigger’s setup once, then leaves the
listeners running. Setup registers subscriptions with providers and dials
sockets. If one cannot, weft activate fails with the reason, and every trigger it
was turning on stays as it was.
It never starts infra and never touches a run going now. If a trigger’s setup reads infra that is not up, it refuses (a trigger whose runs alone use that infra is turned on, with a note naming it):
these triggers' infra is not running: pg. Start it with `weft infra start`, then run this again.
Each trigger is turned on and off separately. weft activate --trigger inbound
turns on one and leaves the rest alone, and --trigger is repeatable on every
trigger verb. weft status lists each trigger’s own state under triggers:.
Each trigger fires its own cut
When a trigger fires, its run does not run your whole program. It runs the trigger’s cut: the part of the program from the trigger down to what it reaches, and back up for what those nodes need. Each trigger has its own, recorded when it was set up, and it decides two things.
A trigger is behind when its cut or the code it runs changed since it was
set up. weft status shows it as on, behind the source (`weft resync` brings it up to date), and a resync moves only those. If a trigger’s cut no longer
works out in the source (it sits inside a loop that runs only in part), it
shows as behind too while it is on. If you deleted a trigger from the source
and it is still on, weft status says the program has it no more, with
weft deactivate --trigger <name> to take it down.
The infra a trigger depends on is what its runs reach plus any infra its
setup reads an address from (infra it reads only through baked
outputs
is not). weft status writes it beside the trigger (feed: on; reads pg),
and it decides what the infra commands take down, below.
Frozen inputs
A trigger that is on keeps the input values it was set up with until you
resync it, and shows as behind meanwhile; for why, go and read a trigger’s
inputs are frozen.
Any edit to the program also makes the trigger’s last bake stale, so weft run --fire refuses it until you bake again. weft bake does that without
touching what listens.
Taking triggers down: three questions
Every command that takes triggers down answers the same three questions (the
door, what is held, the runs going), each with a default that loses nothing.
--door-for is part of the door’s answer:
| Flag | The question | The answers |
|---|---|---|
--door take|hide|refuse | What happens to work that arrives while they are down | take (the default) holds it and runs it once they are back, on the version they come back with. hide does the same, but a question waiting for a person is hidden from the people who answer it. refuse takes nothing |
--door-for <minutes> | With take or hide: when the door starts refusing | After this many minutes (1 or more), new work is refused and the listeners stop; what was already held stays held. Unset, the door holds until they are back |
--held keep|drop | What happens to the work they still hold from the last time they were down, and to the paused runs the command reaches (waiting_for_input): for deactivate and resync, the ones these triggers started; for an infra command, every one whose cut contains the infra | keep (the default) keeps it for when they are back. drop deletes the held work and cancels the paused runs |
--running wait|cancel | What happens to the runs going right now | wait (the default) lets them finish on the version they started on, and the command goes on only once they have, however long that takes. cancel stops them at once |
The door treats three kinds of work alike: a call from outside to weft’s
address (a webhook, a submitted form, a provider’s push), an answer to a run
waiting on a person, and an event a listener picks up itself (a schedule’s tick, a
message on a connection it holds). Listeners keep listening behind a door that
holds, so a schedule held for two hours has two hours of ticks waiting when it
comes back. Behind a door that refuses, or once its window ended, the
listeners stop, and what arrives meanwhile is never caught up. That goes for
a trigger that keeps its place in what it reads too (a poll’s cursor in its
feed, the files a folder watch last saw): when it listens again, because you
switched it back on or ran weft deactivate on it again with --door take
or hide, it starts from its source as it finds it, so nothing that came
behind the refusing door fires.
What such a trigger missed while weft itself was down (stopped, the machine
off, weft maintenance on) it catches up from where it left off. A caller waiting on a route for its answer cannot be held, so
whatever the door, it is answered 503 with a Retry-After at once, and a
client that retries gets through once the trigger is back.
weft never asks you these: the command does what the flags say, then prints what it did. Each event or call the door held counts as an arrival, and an answer held for a waiting run is counted with that run:
imagebot (a1b2…): took down 'feed', 'door': the door holds what arrives; 3 arrivals and 1 paused run kept; 2 runs waited for, finishing on their own version
weft deactivate, weft resync, weft infra stop, terminate and
upgrade take these flags with these defaults, and the editor’s buttons and
a program’s own calls answer the same questions (for the calls, go and read
taking something down from a
run). Three take triggers down
with answers you do not pick, or only some of them:
| What takes them down | Door | Held | Running |
|---|---|---|---|
weft rm | refuse | drop | cancel |
| The health check, when infra a trigger reads breaks | take | keep | cancel, since those runs cannot go on without the infra |
weft maintenance on, before an update (updating weft) | refuse, so nothing starts on the version being replaced; a trigger that keeps its place in what it reads still catches up from it once weft maintenance off brings it back, since the pause was weft’s and nobody refused anything | keep | wait, or cancel with --running cancel |
Bringing them back
weft activate brings back what is off, and decides what happens to the work
the triggers held while they were down:
--held run(the default) releases it, paced at half each trigger’s owncallsPerMinuteandcallsAtOnce(at least one of each), so new arrivals still get through beside it;--held dropdeletes it and cancels the runs paused on those triggers.
It prints what it did: imagebot (a1b2…): turned on 'door'; 12 arrivals released, at half each trigger's limits.
While a command waits on running runs
If a command is waiting on runs to finish (deactivate, resync and the
infra commands under the default --running wait, or a weft rm), the
waiting happens on the install and your terminal only follows it. Ctrl+C
stops following and the work goes on. While it waits, weft stop <run> ends
one run now.
What Ctrl+C leaves behind depends on the command. After a deactivate the
triggers stay down. A resync goes on until the old version’s runs have ended,
then brings its triggers back on the new one. An infra command goes on too.
If you call it off with weft infra cancel while it is still waiting for
runs, the infra stays up and the triggers it took down come back on; if it
was already taking the infra down, it stops where it got to, and weft infra status shows what is left. A weft rm has already taken the
project down, but it erases nothing until you run weft rm again, which
picks up the wait (with --journal, until every run of the project has
ended) and then finishes the removal.
Infra commands take down only what reads the infra
weft infra stop, terminate and upgrade take down exactly the triggers
that depend on the infra they touch, and reach exactly the runs whose cut
contains it. With --running wait, the infra stays up until every run
reaching it has ended; with --running cancel, those runs are cancelled and
the infra goes.
The triggers they took down come back on by themselves once that infra runs
again, after weft infra start, the end of an upgrade, or a program’s own
ctx.infra(..).start(&mut state). If you ran weft deactivate on one of them while
the infra was down, it stays off. Meanwhile weft status says off, its door set to take, taken down with its infra (it comes back once that infra runs again).
The health check reaches the same triggers. A unit that turns flaky hands
the decision to the node’s health rules, and a node that writes none gets two
built-in ones: a copy that turns flaky takes down the triggers whose cut
contains it, with the fixed answers above, and once it is healthy again they
come back. weft status then says taken down by the health check. If a
node’s own rules never bring them back, weft activate does. If you write your own
rules on a node, go and read when a unit stops being
ready.
One build answers a project’s routes
If you changed a node’s Rust code or updated weft, the new build is a new
worker binary, and one binary answers all of a project’s routes. So while some triggers are
still on the older build, turning one on on the new build is refused rather
than splitting the routes between two binaries. If you build with the default
(the whole catalog), a change to the graph alone keeps the same binary and
never hits this; with --referenced, using a node type the graph did not use
before makes a new one:
one binary answers a project's routes, and these triggers are on another build than the registered one: 'door'. `weft resync --all` moves every trigger that is on to the new build; then run this again
A trigger that should come back by itself (taken down with its infra, or by
the health check) but cannot for the same reason waits instead. weft status and the
editor show it as held back, with the same reason and the same way through,
and a resync brings it back at once.
Resync
weft resync builds, then brings every trigger that is on and behind the
build onto it. Each one goes down with the three answers above, its paused
runs move onto the new build (with --held keep), the runs going on the old
version finish or are cancelled (--running), and only then does it come back
on the new version, which takes over from the old one. A trigger that is off
is never turned on: the answer names it with the weft activate that would.
A resync writes down its plan before it takes any trigger down, and carries
it out on the install, so it goes on whatever happens to your terminal or to
the weft process doing it: if that process dies, the next one to take the
work up carries on where it was. Until it finishes or you give it up (even while it is stopped after a
failure, below), the resync holds the project:
no other activation of its triggers starts, a build you start is refused with
cannot build now, and a build that was already compiling finishes but is
not registered (this build was not registered: the project is resyncing; ...).
If a step fails, the resync tries again 5 seconds later, then 20 seconds after
that. If steps fail three times during the resync, it stops with the triggers it has not
brought back still down, and says why: in the answer to weft resync, under
waiting on weft: in weft status, and as “Resync stopped” on the editor’s
action bar. If a step fails because the install’s database was busy or out
of reach, it uses none of those three tries: the resync tries it again after
5, 15, 30 and then 60 seconds, and if the database still cannot take the
step, the resync stops, then tries again by itself 30 seconds later. If it stopped
because the infra those triggers read was not
running, or because an activation of them was still landing, it goes on by
itself once that infra runs or that activation has landed. Otherwise, once
you have fixed the cause, weft resync --resume (or the bar’s Resync button)
takes it up again where it stopped, with the answers and the build it began
with. A plain weft resync is refused while a resync holds the project.
If you would rather give it up, weft cancel-activate with no --trigger or
--instance (or the bar’s Give up button) ends it and names the triggers it
had not brought back yet:
gave up the resync of a1b2…; these triggers it took down stay down: 'feed', 'door'. `weft activate` turns them on
Until they are on again or the next resync begins, weft status lists them
under waiting on weft: as a given-up resync left these triggers down: 'feed', 'door'. While a resync holds the project, weft cancel-activate --trigger or --instance is refused, because the resync covers every
trigger it moves.
Cancelling an activate
weft cancel-activate cancels the setup run of each trigger still turning on,
wipes every signal it registered so far, and leaves those triggers down. If
the weft process carrying out an activate dies at a point where no setup run
of it is going, weft ends the activate the same way by itself, about a minute
later.
Instances
If a trigger reads an instance’s copy or value (@per_instance,
@instance_filled), it exists once per instance; for turning those on and
off, go and read programs with instances.
Runs
| It starts from | Command |
|---|---|
| You | weft run |
| One trigger, by hand | weft run --fire <trigger>='<json>' |
| A trigger firing for real | Nothing. It happens because you activated |
| A caller connecting | Nothing. The trigger has to be on |
| Status | What it means |
|---|---|
running | Work is happening now |
waiting_for_input | Parked on a person or a service. No worker is up and nothing is being spent |
completed | Finished |
failed | A node failed (a wait the run held past its holdSecs, which that node did not handle, counts as one), or the run could get no further |
cancelled | You or another run stopped it; its live caller hung up; it was a fast run and its worker went away; its worker was stopped while it could not pause (recorded: false, or a caller still on the line); it reached the hour a cloud install allows one stretch of a run; its caller’s session reached the route’s or socket’s maxSessionSecs; or the runtime stopped it (a newer build replaced its image, its build’s worker could not start, or a queued run was refused three times, which names who said the last no: its worker refused it 3 times: <error>, the platform refused weft's calls for it 3 times: <error>, or its project's worker settings kept it from any worker 3 times: <error>) |
weft runs --status running lists both the ones working and the ones
waiting, and the status column tells them apart.
A run that cannot start or resume
A run whose build’s worker cannot start at all is not tried again. It ends at once, with every other run queued on that build, cancelled with the reason:
<why>. Nothing will run on that build again: fix the program and build again (`weft resync` puts triggers that are on onto the new build), then `weft run --seed=<run-id>` picks this run up from what it did
A paused run whose record no longer fits the program it would resume on (a
resync moved it onto a build its nodes changed in) ends failed instead,
naming the same weft run --seed command. Either way its record is kept
like any ended run’s, so the seed has something to reuse.
Waiting, and stuck
Waiting means somebody still has to answer. Stuck means nobody can: nodes are holding values that will never add up to enough to run them, and nothing is parked on a signal, so no answer exists that would move it along.
weft ends a stuck run as failed and names every node and what each is short of:
run stuck: 2 firings hold pulses and can never fire: theirs has value,
still waiting on go; leaf has value, every wired input arrived
That is a bug in the shape of your graph rather than something to wait out.
Stopping one
weft stop <run-id> cancels it. If a worker is up, it stops within about fifty
milliseconds. If the run was parked, the cancellation is written down and the
listener forgets its signals, so a late answer finds nothing.
Work already done stays done, and weft cannot take back a message, an email or a payment your program already sent.
Cancelling a run that already ended is refused rather than silently ignored, so
weft stop on yesterday’s run tells you it has already finished.
After you change something
weft status compares your source against what is running and names the verb:
drift:
infra: a copy is behind the source; `weft infra upgrade` puts the change live
binary: worker code has changed; the next run or `weft build` rebuilds the image
definition: project shape has changed; the next run picks it up
A trigger that is behind says so on its own line under triggers:.
| What you changed | What to run |
|---|---|
| The shape of the graph, or a node’s Rust code | Nothing for the runs you start by hand: the next one picks it up, and rebuilds what it must. Triggers that are on show as behind, and weft resync moves them (--all if instances have triggers on) |
| What an infra node asks for | weft infra upgrade |
| A trigger’s inputs | weft resync |
The journal
Every recorded run writes down what it did, event by event. For when each
write happens, go and read when a run waits for its
writes.
That record is what the graph shows you, what weft events prints, and what a
worker reads to pick up a durable run that was interrupted, or a run that was
waiting.
weft events 9b81d0a2
weft events 9b81d0a2 --node classify --full
weft logs 9b81d0a2
Nothing edits an event once it is written. A run is deleted with its record
once it has been kept long enough (go and read how long a run is
kept), or sooner by weft clean, weft prune or
weft rm.
What is in it
Each row holds one run’s events from one write, in the order they happened: which run, what kinds, when, and the events themselves. A worker sends the rows of several runs in one write. A run’s history is its rows, read in order.
The run’s life. It started, with the project, the entry node and a fingerprint of the exact program shape, so a resume runs against what it suspended on rather than whatever you have edited since. Then it completed, failed, or was cancelled, with the reason in words and in a form the inspector can read.
Each node’s life. Given its inputs (kicked), started, and then completed, failed, skipped, suspended, resumed or cancelled. A skip carries its reason in plain words.
Where a node is. Each time a node saves the state it keeps
(state.save(), the end of a once or a wait, or a pause), a
NodeStateSaved row holds it whole, with what weft keeps beside it: how far
the node read each bus and stream, how many items it yielded, and the numbers
its next wait and its next program call will take. The latest row is what the
node gets back when it runs again after a pause or a crash. Each wait it
registers carries its number in the node (wait_index), which is how a body
that runs again finds the wait it already made.
Values on wires. PortEmitted is the only event that carries a value, and
it carries it once. When a run is rebuilt, weft puts the value on every
outgoing wire itself, which is why a replay lands on exactly the same picture
as the live run did.
PortClosed is written only when a node closes a port on purpose. The ports
that close because a body returned are worked out again on replay.
Loops, streams and buses get their own events: an iteration launching, a
gather assembling, a bus being made, a participant joining, a window of
messages. The row for a bus being made (BusCreated) holds what a worker
needs to open it again after a pause: its id, the port its marker went out
on, its options.
A live caller’s exchange does too: the caller arriving, a window of what the caller and the program said, an error, the caller leaving. A window also names the program’s messages the caller never received because it read too slowly, and the row saying the caller left names the ones still waiting to go out when the exchange ended. For the settings that let a slow caller’s messages go, go and read slow callers and messages that cannot be made.
What a call cost, from the meter, as it happens. A node can never write one of those.
Reading it back
When weft reads a run back (the fold), it takes the rows and the program and works out where the run had got to.
The journal holds only the facts learned from outside, and the fold recomputes everything else with the same functions the live engine ran, so a replay rebuilds exactly the picture the live run held. A replay shows the times the journal wrote down, so you see when each thing really happened.
weft reads a run back a page of rows at a time. It forgets a node once the
node has finished (keeping each node’s last output, which the run’s outputs
show), and a loop’s iteration once nothing in it is still running. So reading
a run back holds one page of its rows and what is still running, however long
the run is. The one thing that grows is what a loop gathers: the loop hands
the whole list on when it ends, so it holds it until then. A run started from
another one (weft run --seed) reads that one the same way, and keeps only
the nodes it reuses from it.
If you follow a run (the editor, weft run, weft wait), you get its history
and then what happens live, on one stream, each event once. If you fall behind
or lose the connection, you carry on from the last event you got and nothing
is missed. The events say where in the journal a value is kept instead of
carrying it, so a value the run passed to ten nodes still travels once: the
editor reads a node’s values when you open its inspector, and weft events
and weft run print them written in.
When a row cannot be read
A row the fold cannot apply is a hole. It is logged, listed on the run, and the inspector shows how many there are.
In a run that has ended, a hole only spoils the replay view. In a run that has to resume, a hole stops the resume with an error, because a run rebuilt from part of its record could run nodes that already ran.
When a worker dies mid-firing
A node’s completion is written after it finishes, so one whose worker died before that has no completion on record. For what weft does with such a node, in a fast run and in a durable one, go and read surviving a restart.
When a run waits for its writes
A worker gathers what its runs did and writes it a couple of milliseconds later, or at once when a run is waiting on it. Each run’s records always arrive in order.
A fast run (the default) waits for its writes only where something else is
about to act on its record: when it pauses, when an answer it was waiting for
arrives, when its worker is stopping and hands it to another one, and before it
asks weft for something on its behalf (a connection or an endpoint it does not
already hold, anything on its stored files, a tag through ctx.tag_run,
a stop through ctx.stop_tagged, a call on its own program such as starting
its infra). If a node calls a paid service and that call reports a cost, the cost is
written in the background, and the run hands over its ending only once that
cost is written. Once it has handed its ending over, the run is gone from the worker’s memory straight away. The ending
is written after every event the run handed over before it, so a run is never
reported finished before the rest of its record.
A durable run also waits before each node that is not
pure starts, before an answer leaves for its caller,
and for its ending. One fired by an event also waits for its start to be
written before the event counts as delivered. When the answer is the last thing the run does, its ending
goes in the same write, so the answer costs no extra wait. A durable route that
reshapes its input with pure nodes and answers with Reply (pure too) waits
once, for the answer and the ending together. For choosing between the two, go and read how a run is
kept.
If the database falls behind and 64 MiB of events are already waiting on a worker, the next run with more to hand over waits until there is space, so nothing is dropped. For what that does to new calls, go and read when a worker is full.
When a write of the record fails
If the broker refuses a write, the runs whose rows were in it end as failed.
If a write gets no answer, the worker sends it again for a minute. That is safe, because the record takes a batch sent twice only once. After the minute it gives up on that write, and then:
- a fast run ends as failed;
- a durable run is handed back, and another worker carries it on from the rows that did land.
Either way the run writes nothing after the write that failed. A record with a hole in it would rebuild a different run: a node whose start was lost but whose values landed would run again and spend twice. So what landed is always the run’s first rows, in order, which is the record a worker that died leaves behind, and a durable run carries on from it the same way. A node whose start landed and whose end did not is failed rather than run again, unless it saved a state (for that, go and read when the worker dies mid-run).
How long a run is kept
An ended run is kept for a week, and then deleted with its record, its logs, its costs, its search entry, its tags and the files it stored in run scope. A run that has not ended (running, parked, waiting for a worker) is never deleted, however old. If you want another length:
-
for every run of a project, set it in
weft.toml:[runs] keep_for = "30d" -
for the runs of one trigger, set its
keepRunsForinput (12h,30d,forever); -
for one run you start by hand,
weft run --keep-for 2h.
A trigger’s setting beats the project’s, and --keep-for beats both. A run’s keep
time is fixed when it starts, so a change only reaches runs that start
afterwards. If you change a trigger’s keepRunsFor or the project’s
keep_for, the trigger’s own runs pick up the change once you run weft resync, and a run you fire with weft run --fire once you run weft bake.
Clearing it
weft clean # ended runs that started more than 30 days ago
weft clean 9b81d0a2 # one run
weft clean --project <id> # that project's whole history
weft clean --all # everything
Removing a project with weft rm erases its runs with it.
A past run that shows nothing
The journal names the program each run used, by hash. If the dispatcher no longer holds that program, the run’s rows are still there but nothing can be drawn against them.
If your files are the same as when the run ran, weft build makes it readable
again: it registers the program under its hash, and that is the hash the run
names.
Versions, seeds and frozen examples
Every run records the code it ran, so you can go back and look, run something again against a change, and see what moved.
Your folder is a copy
weft builds and runs your code from its own copy, not from your folder. For each place your project is edited, the install keeps a working copy, and it keeps your folder and its working copy the same in both directions.
On an install on your machine, its runtime watches the folder. When you save,
the change reaches the install about a second after the folder goes quiet (a
burst of saves, or a git checkout, counts as one change). When a program
writes code into your project (a node calling ctx.code(), see the
ctx), or weft branch moves your folder, the
runtime writes those files into it.
Every command that builds (run, build, bake, activate, resync, infra start, infra upgrade) or moves this folder in the tree (checkpoint,
branch, resolve) also syncs the folder before it does anything, and says
what it moved:
sent your changes to the install: src/main.weft, prompts/tone.txt
brought the install's changes into this folder: nodes/greeter/mod.rs
If a command only reads (tree, freeze) or acts on another working copy
(forget-copy, branch --copy), it leaves your folder alone and syncs nothing.
On a cloud install nothing watches your folder, so that sync is the only one.
The first of those commands you run in a folder makes a working copy for it, holding the files the folder has. That copy starts at the head (see below) of whichever working copy of the project changed most recently.
If your project sits on a Windows drive under WSL (/mnt/c/...), an install on
your machine refuses to keep it in sync, because WSL tells Linux nothing about
changes made there:
/mnt/c/Users/me/shop is on a Windows drive, and WSL tells Linux nothing of the changes made there, so weft cannot keep this folder in sync with its working copy. Move the project into the Linux filesystem (under ~/, for example) and run the command again from there
When both sides changed one file
If you edited or deleted a file while the install’s copy of it changed too (a program wrote it, say), weft keeps what your folder holds and writes the install’s copy beside it, named after the file and the first eight characters of that copy’s content hash:
warning: prompts/tone.txt changed both here and in the install: your file is kept, and the install's is written beside it as prompts/tone.weft-conflict-1a2b3c4d.txt; `weft resolve prompts/tone.txt --keep incoming` puts the install's in place, `weft resolve prompts/tone.txt --keep folder` keeps yours and drops the copy
Nothing waits on you: builds and runs go on with your file.
You see that warning when a command syncs the folder before it runs. On an
install on your machine you will usually not see it: the runtime syncs your folder about a second after
you save, so it finds the conflict first and only logs it (the file changed in the folder and in its working copy, with both paths). To see it there, run
weft daemon logs.
A conflict copy is never part of a version, so even
main.weft-conflict-1a2b3c4d.weft is never compiled. To get rid of it, pick a
side with weft resolve:
- If you want the install’s version,
weft resolve prompts/tone.txt --keep incomingmoves the conflict copy over your file. - If you want to keep yours,
weft resolve prompts/tone.txt --keep folderdeletes the conflict copy.
If you had deleted the file, the warning says prompts/tone.txt was removed here and changed in the install: it stays removed; --keep incoming brings the
install’s version back and --keep folder keeps it deleted.
Either way, resolve then syncs the folder, so the working copy gets the
side you kept. --keep has no default. If you name a file, resolve settles
all its conflict copies; if you name one conflict copy, it settles only that
one; if you name nothing, it settles every one in the folder. A file can collect more than
one conflict copy, and then --keep incoming asks you to name the one to put
in place:
prompts/tone.txt has 2 conflict copies (prompts/tone.weft-conflict-1a2b3c4d.txt, prompts/tone.weft-conflict-9e8f7a6b.txt); name the one to put in place
Name the copy you want (weft resolve prompts/tone.weft-conflict-1a2b3c4d.txt --keep incoming); once it is in
place, weft resolve prompts/tone.txt --keep folder drops the rest.
If you would rather merge by hand, take what you want from the conflict copy,
then run weft resolve <file> --keep folder (or delete the copy yourself).
If you want to see which conflict copies are left, run weft tree.
If the install removed the file while you changed it, your file is kept and
nothing is written beside it, so there is nothing for resolve to settle. The
warning says so:
warning: gone.weft changed here and was removed in the install: your file is kept; delete it if you want the removal
If you want the removal, delete the file; the next sync sends that to the working copy.
A version
A version is your project’s files at one moment. A save never makes one.
Something you do with the code does: weft run, weft build, weft activate, weft bake, weft resync, weft infra start, weft infra upgrade, weft checkpoint or weft branch. Each of these first records the
folder’s files as a version. Recording the same files again gives the version
they already are, so ten runs of unchanged code record one.
| In it | Not in it |
|---|---|
Every .weft file, at any depth | Anything hidden at the top: .git, .env, .weft |
weft.toml | target/ at the root or beside a Cargo.toml, and node_modules, __pycache__ and .pytest_cache anywhere |
src/** and nodes/**, including a node that sits beside your code and the standard library in nodes/base_catalog/, edits and all | Any other file or folder at the top, such as README.md or docs/, and front/, which ships through your project’s CI |
assets/**, prompts/**, scripts/**, sql/** | Conflict copies |
examples/**, so going back restores the examples that existed then | |
layouts/**, so going back puts the boxes where they were |
A file identical to one an earlier version held costs nothing to store, so
the standard library is stored once and every later version reuses it. An
install builds a version from exactly the files it holds and nothing of its
own: if you deleted nodes/base_catalog/, the version has no standard
library.
head
Each working copy has a head: the version its next version is recorded under,
and the run a --seed from it inherits from. Your folder has its own, so a
second clone of the project does not move yours. A program that names no
working copy acts on the project’s most recently changed folder: if that is
yours, any run, build, checkpoint or switch the program makes moves your head.
| Command | What it does |
|---|---|
weft tree | Every working copy and where its head is, then every version with what changed against its parent and the runs started by hand beneath it (a trigger’s runs are counted on the version’s line) |
weft checkpoint [<label>] | Records the folder’s files as a version and moves head there. No run, no build |
weft branch <version> | Moves this folder to that version and writes its files here |
weft branch <run-id> | The same with the version that run ran, and makes it head’s run, so the next seed inherits from it |
If you have changes no version holds yet, weft branch keeps them: it records
the folder as it is as a version first, then moves, and prints the way back.
this folder is now at version 7c41e9a0
your files as they were are version 3f2a1b9c; `weft branch 3f2a1b9c` comes back to them
A file you edit while the switch is writing is kept, with the version’s copy written beside it as a conflict copy.
In the editor, the Runs view shows the same tree (Flat list / by
version on its title bar), and Load this version on a version or a run
does what weft branch does.
weft tree starts with one line per working copy: yours as this folder,
another folder as its path and machine, and the program copy, the one
programs write into when the project has no folder (every project that had a
head before the update that brought working copies has one). changed since
means its files changed after the version its head points at. Each head is marked on its version:
this folder at version 7c41e9a0, changed since
/srv/shop on build-box at version 3f2a1b9c, changed since, conflict copies: src/main.weft-conflict-1a2b3c4d.weft
3f2a1b9c 2026-10-24 09:12:40 UTC (base) root <- HEAD of /srv/shop on build-box
run 9b81d0a2 2026-10-24 09:13:02 UTC -> 2026-10-24 09:13:30 UTC completed
7c41e9a0 2026-10-24 10:02:11 UTC ~src/main.weft <- HEAD of this folder, activated
weft tree --json prints the tree as the install answers it, with
workingCopy added: the id of this folder’s working copy among
workingCopies.
Seeding
weft run --seed
Reuse everything from head’s run whose slice of the program did not change, and run only the rest. That is the loop when you are iterating on node nine of a twelve-node program and nodes one to eight cost real money.
If you want to reuse a different run, name it: weft run --seed=<run-id> takes
any finished run of the project, including one a trigger started or one that
failed. Keep the =: weft run --seed <word> reads the word as the name of
a saved example, not as a run. A run that is still going is refused until it
ends or you weft stop it.
If a node that runs again waited for somebody in that run, its waits get the answers they were given there, matched by order: the node’s first wait gets the answer its first wait got, and so on. So it carries on instead of asking again. That only holds while the node’s own code is unchanged; if it changed, the node asks again and weft says so:
'review' runs again and asks again: its code changed since the seed run answered it
What will not be reused
| Why | |
|---|---|
| Anything whose code or dependencies changed | It is a different thing now. weft says so by name |
| Anything downstream of something that changed | Its input is different |
| Anything that never finished: failed, cancelled, still running, or parked on somebody | There is no result to reuse |
| Anything that emitted a live handle, like a bus | The handle belongs to a worker that is gone |
| A whole loop, if any iteration of it is invalid | A loop is reused whole or not at all |
Anything downstream of an explicit --emit or --fire | You asked for a fresh event, so its consumers are fresh |
Anything past --seed-before or --seed-until | You said where to stop |
| A starting value you supplied that differs from the one the earlier run used | Different input, different work |
The invalidation spreads: if a node is not reusable, nothing downstream of it is either.
When a reused node’s file changed
A saved result that names a stored file says what the file held then. If a node that runs again would read it, and the file has been edited since (a later node added to it, another run changed it) or is gone, the run is refused before anything starts. A run that reuses the node and runs nothing that reads the file goes ahead:
--seed would reuse what 'ask' produced on 'historyFile', but the file it names,
'conversation.json', has changed since (that run left it at version 4, it is at
version 6 now), so the saved result no longer says what that node made. Hand
the file in yourself (`--emit ask='{"historyFile": ...}'`), or run 'ask' again
by keeping it out of the seed (`--seed-before ask`).
The one that surprises people
A --from start is reused like any other node. If its code and the values you hand it
are what they were in the seed run, it keeps that run’s result and does not
run. The run reports success and nothing ran, which is correct and invisible,
so weft says it out loud:
'classify' keeps the result run 3f2a1111-0c4e-4b1a-9d2e-5a7b8c9d0e1f gave it:
its code and what it is handed are as they were. Run without --seed to run it again.
Frozen examples
An example is a run you keep: its starting values, the answers people gave, and the outputs you accept as right.
weft run --from classify='{"text":"..."}' --save angry-customer
weft freeze angry-customer
--save writes the starting parameters. freeze takes a run that completed
and records its accepted outputs alongside them. If a saved file still uses a
spelling an older weft wrote (frozen_from, gave_up, keep_for), it still
reads, and reading it leaves the file as it is. It takes today’s spelling the
next time weft writes it, with --save or freeze.
Then, after a change:
weft run angry-customer
weft diff <new-run-id> example:angry-customer
diff shows what moved on the wires. You, or Tangle, judge whether the change
is acceptable. Freezing the new run replaces what is accepted.
--expect <node> marks the outputs that matter, so a later diff leads with
them rather than burying them.
| Command | What it does |
|---|---|
weft examples | What is saved, which are frozen, and each one’s latest run |
weft freeze <name> [<run-id>] | Freeze a run. Without a run id, head’s run |
weft diff <left> <right> | Compare two runs. A side is a run id or example:<name> |
weft run <name> | Run the current code with that example’s parameters |
A freeze replaces the example whole, and says so. Only a completed run can be frozen: a failed or cancelled one has nothing to accept.
Cleaning up
weft prune <version>
Deletes that version, every version under it, and every run beneath them.
It refuses, naming each reason, while any of those versions is a working
copy’s head, is the one activated triggers run, is the one the project’s
build runs, or is (or sits above) the version a frozen example came from, and
while any run underneath is queued or running. It also refuses while the
settings of a trigger that is on came from one of them, and while a run that
has not ended (one paused waiting for somebody or on a timer included) ran one
of them. Those last two come back as one sentence that names no version
(these versions still supply trigger settings or running runs), and only
once every other reason is cleared: weft stop the run, and weft deactivate
the trigger, which removes the settings the refusal means.
If the refusal names a working copy, you can move it out of the way or forget it, from any folder of the project:
| Use it when | Command |
|---|---|
| The copy is still in use | weft branch <version> --copy <copy>, with a <version> outside the history you are pruning. It records what the copy held as a version first, moves it, and leaves this folder where it is |
| Nothing uses the copy any more (a folder that is gone, or the program copy) | weft forget-copy <copy>. It records the copy’s files as a version, removes the copy, and prints that version with the weft branch <id> that brings its files to this folder |
Either way, the copy’s files are kept as a version under its old head (a copy that held no files records none), so the prune deletes that version too, unless those exact files were already a version elsewhere in the tree. If somebody still works in that folder, moving it replaces their files there, so check with them first.
<copy> is what the refusal prints: a folder’s absolute path, or program
for the program copy. The start of a working copy’s id works too, and
weft tree --json lists every id under workingCopies. If two
machines have a folder at that path, weft says so and lists both ids, and you
pass the start of the one you mean.
If a folder you once ran a weft command in is gone, forget its copy: its head stays
where it was until you do. Forgetting a folder that is still there lasts only
until the next command in it that syncs the folder (one that builds, or
weft checkpoint, branch or resolve), which registers it again, so
forget-copy refuses this folder’s own copy.
Updating to 0.2.274 gave each project that had a head a program copy on that
head, so if you prune the version it was on, or one above it, the refusal
names program. A folder starts on the head of the copy that changed last,
which right after the update is the program copy’s, so the refusal names your
folder too: weft branch <version> in it first. If
the project has a folder, programs that name no working
copy write into that folder, so after weft forget-copy program only a
program that names the program copy by its id notices: its calls fail with
this project has no working copy <id>. If a program later acts on the
code of a project that has no working copy at all, weft makes a new program
copy then, on the newest version.
If forget-copy answers that the copy’s files changed while it was being
forgotten, a save or a program wrote into it meanwhile: the copy is still
there, its files are already kept as a version, and running forget-copy
again finishes it.
Files at run time
A value on a wire is capped at 100 KB, so anything bigger is stored and what travels is a marker saying where it is.
This page is about the files a running program leaves behind and what to do with them. For putting one there from a node, go and read storage.
Where they live
Any run of any project in your account can read any stored file, for as long as the file exists. The space a file is in decides only how long that is.
| Space | Holds | Gone when |
|---|---|---|
| Run | What one run made | Its run is deleted: the run’s keep time runs out, weft clean, or weft rm. The run ending leaves it |
| Project | What the project keeps using | weft rm |
| Shared | Things several projects meet in, by name | You remove them |
| Assets | Your @asset copies | They follow your source |
| Instance | One instance’s files in this project | Your program deletes them, you remove them, you wipe the instance (the WipeInstance node), or weft rm |
A file can also have a deadline, and then it goes when nobody has touched it for that long, or with its run or project if that comes first. Every read, download or link pushes the deadline back, so a file still in use stays.
Keeping something
In a graph, the storage nodes TextToFile and FetchToStorage take two
settings: scope (run or project) and ttlDays (the deadline in idle
days, empty or 0 for none). Every other node that makes a file, a download
from S3, Slack or Drive or a generated image included, stores it in the run’s
space with no deadline, so it lives as long as its run is kept, unless its
description says it keeps the file in the project’s files (a chat bridge’s
media, kept under the message it came with, is one).
If you want a file to outlive the run that made it (an upload one request
receives and the next one serves, a file a table row points at), wire it
through KeepFile. With scope: project, its default, it copies the file into
the project’s files and hands on the new reference, which lasts until weft rm.
With scope: run it leaves the file where it is and only sets its deadline.
If you would rather keep the whole run longer, set the trigger’s keepRunsFor
or run it with weft run --keep-for.
Looking at them
weft files ls
weft files ls project/
weft files inspect <key>
weft files usage
ls groups by space (run/<run-id>/, project/<project-id>/,
instance/<project-id>/<instance>/, shared/<name>/, and asset/, where each
asset is named by its content hash) and, for each file, shows its id, the size, the filename, and when it is deleted if it has a
deadline. A file’s key is its space followed by its id, such as
run/9b81d0a2-4e17-4a33-bc90-51d8e2f4a7c1/3f6c0e1a-77d2-4b8e-9a41-2c5d8e0b9f13,
and that key is what inspect, download and rm take. Give ls the start
of a key (project/, or one run’s run/<run-id>/) and it lists only the files
under it. usage prints how much you have stored in all, and across how many
files.
Getting one out
weft files download <key>
weft files download <key> -o ~/Desktop/chart.png
Without -o, it lands in the folder you run it from, under the file’s own
name. It streams straight from storage into a temporary file and renames it only
after the size checks out, so a download that fails never leaves you a
plausible half file, and never overwrites a good copy with a broken one. If
the connection drops partway, it picks up from where it stopped, and if the
file changes on the install meanwhile, it stops and says so.
Removing them
weft files rm <key>
weft files rm project/<project-id>/
The second form removes a whole space. It tells you how many files that is before it does anything:
About to remove the whole space 'project/<project-id>/': 48 file(s).
Type 'yes' to confirm:
--yes skips the question, and is required when there is no terminal to ask.
Links
A node hands out a link when somebody outside the install has to open a file: a route’s answer carries one in place of the file, for instance. For how a node asks for one, how long it lives and what opening it does, go and read handing a file out. A link expires, which is why a stored file travels as a marker rather than a URL.
If you want to choose who can open those links (anyone on the internet, the devices on your network, or only this machine), go and read which address a file link gets.
When a file goes missing
If a download or a node says a file is no longer available, the message says,
for that file’s space, what can have deleted it: its run, its project, its
deadline, or somebody by hand. It also says how to make one last: KeepFile
with scope: project, keeping runs longer (keepRunsFor, weft run --keep-for), or ttlDays: 0 to take a deadline away.
weft files ls tells you what is actually there.
The browser extension
Weft tasks is where a person answers a program. When a node asks a question, the run parks and the question shows up here as a card.
Install it from
Firefox Add-ons or the
Chrome Web Store.
To build it from your checkout instead, ./setup.sh --browser --no-sign writes
an unpacked build per browser under extension-browser/build/, and Chrome loads
it from chrome://extensions with Developer mode on.
Point it at your runtime
The extension ships with no access to anything. Not your runtime, not any website. You give it one address at a time.
Mint a token from your project folder:
weft token mint --name "my laptop"
Then open the extension’s settings, add your runtime’s address and paste the token. The browser will ask whether to allow access to that host, because until you say yes the extension cannot reach it. Remove the last token for a host and it hands that permission back.
The token’s full value is printed once, at mint, and never again. weft token ls shows you which tokens exist by a short prefix, and weft token revoke <id>
ends one.
Scoping a token
A token with no restrictions sees every waiting node in the tenant, but none of your nodes’ displays.
| Flag | What it narrows to |
|---|---|
--projects <uuid> | Only these projects. Repeatable |
--tags <tag> | Only nodes carrying one of these tags. Repeatable |
--display <node> | Also lets it read one node’s display, and press the buttons on it |
--displays | Every node display in the token’s projects |
The display scope is the one that grants nothing by default, because a display can be a credential: a bridge showing a QR code to scan, a database showing the password it just minted.
What you see
The popup is the list. It shows whether each runtime is reachable, and splits what is waiting into triggers, which start a new run, and questions from runs already going.
It distinguishes two kinds of failure, and says which: a host you have not granted, and a host it could not reach. A popup that said “Connected” while a runtime was down would be hiding tasks from you.
The task page is one card at a time, with the count, and Previous and Next to move. The card shows whatever the node declared: values to read, text boxes, option buttons, approve and reject pairs, images and files.
| Button | What it does |
|---|---|
| Submit | Answers a question inside a run already going |
| Fire | Starts a new run, on a trigger |
| Skip | Declines the question: the node that asked ends skipped, and the nodes reading its answer skip too. The rest of the run carries on. Not shown on a trigger, which has no question to skip (weft refuses it) |
| Cancel run | Ends the whole run. Also not shown on a trigger |
Approve and reject are labelled by whoever wrote the program (the field’s
approveLabel and rejectLabel), so they can read Send it and Hold it
rather than the defaults.
If a program’s author gave you an instance token, Your connections (the link icon in the popup) is where you connect your own accounts. With the instance token added, it lists the nodes that run on the instance’s own account, and what you connect and pick there belongs to that instance alone. If the page says you have no instance token, add the one the program’s author gave you; other tokens you have added are ignored there. For instance tokens and what they reach, go and read programs with instances.
After you submit, the card says Submitted for a moment and moves to the next.
Back in the editor, the cyan ring on that box goes green and the rest of the
program runs. On a trigger, Fire shows Fired instead: a run started, and
the trigger stays so you can fire it again.
What it can see
It holds three permissions: storage for your tokens, notifications, and alarms for the poller behind them. No access to any site until you add a runtime, and the Firefox build declares that it collects nothing.
It talks to two addresses on your runtime: one that lists what is waiting, and
one that answers. With an instance token it also talks to the instance’s connect
page on the runtime. The token rides in the Authorization header rather than
in the URL, so it does not land in anybody’s logs.
Writing your own instead
The extension is one consumer, not the only possible one. It reads the same two addresses any client can, and it renders a form from what the node declared rather than knowing anything about particular nodes. A Slack bot or an internal page would work the same way against the same addresses.
If you want to build one, the token is the whole authentication story, and
weft token mint --display <node> is how it gets to read what a node is
showing.
Programs with instances
If you want part of one program to run as several separate copies, each with
its own WhatsApp number, Slack account, spreadsheet or schedule, you mark what
each copy gets for itself: a container of its own (@per_instance), or a value
of its own (@instance_filled). Each of those copies is an instance of the
program.
What an instance stands for is up to you. One per customer is the common case, but it can as well be one per session, or several per person (a customer with two shops, each with its own bridge). weft only sees the ids: which person owns which instances is your program’s own data, kept in its own database.
An instance is just an id you choose, such as user-42, session-9f3 or a
shop number. You hand it to weft each time something happens for that
instance: in --instance, in an instance token, or in a header from your own
backend. weft keeps no list of instances, so there is nothing to register. Once
you wipe everything weft holds for an id (see
cleaning up after instances), that id is no
longer an instance.
Say you built a WhatsApp assistant and a hundred customers want it. A WhatsApp bridge signs in to one phone: when it first starts, it shows a QR code on its display, and whoever scans that code with their phone ties the bridge to their number. So each customer needs a bridge of their own, and you give each one an instance:
bridge = BaileyBridge {
@per_instance
}
receive = BaileyReceive { bridge: bridge.bridge }
# a real assistant wires an LLM's answer into `message`
reply = BaileySend {
bridge: bridge.bridge
to: receive.from
message: "hi"
}
The @per_instance line gives each instance its own bridge container, called
its copy of bridge. receive and reply read the bridge, so they are
per instance too. A message arriving on instance user-42’s phone starts a run
for user-42, and reply answers through user-42’s copy.
Bringing one instance up
For customer user-42, from the project’s folder:
weft infra start --instance user-42builds the program if it changed, starts their bridge, and answers once it is running.weft activate --instance user-42turns on itsreceivetrigger.weft token mint --instance user-42 --expires 7d --display bridgemints an instance token that can read that bridge’s display.- The customer adds the token to the weft browser extension, opens the bridge’s display, and scans the QR code with their phone.
If you want your own website to do steps 1 to 3 for a customer, a program can do each of them itself; for how, go and read the instance nodes.
What @per_instance marks
@per_instance goes on its own line inside the braces of an infra node (a node
that runs its own container, like the bridge). Anywhere else the compiler
refuses it as per-instance-ineligible.
If you work in the graph, you can set and remove the mark from a node’s right-click menu; for how, go and read the graph.
What an instance fills: @instance_filled
If each instance should get its own value for a field (a spreadsheet, a model,
a schedule, a Slack account), write @instance_filled where the value would go:
google = GoogleAccess { account: @instance_filled }
read = GoogleSheetsRead {
account: google.access
spreadsheet: @instance_filled
tab: @instance_filled("0")
}
digest = Cron { cron: @instance_filled("0 0 8 * * *") }
A field holds one of four things:
| In the source | What the field holds |
|---|---|
a value ("0 0 3 * * *", a picked connection) | that value, for everyone |
| nothing | the node’s default; if it has none, nothing when the field is optional, and a refused run when it is required |
@instance_filled | the instance’s value; if none was given, the field acts as if nothing were written (the row above) |
@instance_filled(<value>) | the instance’s value; an instance with none gets <value>, even when the node has a default of its own |
@instance_filled(@file("prompts/default.md")) | the same, with the file’s content as the value an instance with none gets (@asset(...) works too) |
On a connection field (account above), what you write decides whose key pays
for an instance’s calls:
| In the source | Whose key an instance’s run uses |
|---|---|
| nothing (your pick on the install) | yours, for every instance |
account: @instance_filled | the instance’s own; an instance with no connection is refused, naming the field |
Only you can put your key in an instance’s run, by leaving the field unmarked
and picking your connection on the install. An instance token can never pick
your connection, or the runtime’s shared key: connecting with the shared key
through the instance door is refused, and a value saved for an instance’s
connection field has to be a connection that instance owns in this program,
whoever saves it. If your connection is one made with the runtime’s shared key,
the instances it serves spend that key, and each of their cost records shows
Platform as the payer (for reading those, go to who paid for a
call).
A field with a wire into it already gets its value from the wire, so the
compiler refuses @instance_filled there (instance-filled-wired). A group’s,
a loop’s or an included file’s own port is refused too
(instance-filled-boundary); put the mark on the node inside that reads the
value. A key that is not one of the node’s inputs cannot take the mark either
(instance-filled-not-an-input).
You do not mark the other nodes: a node that reads an instance’s value or copy,
directly or through nodes before it, runs per instance too. Above, read needs
Google through the instance’s connection, so it runs in that instance’s runs.
If you include a file in two places, each place decides this for itself. Say a
group with a per-instance bridge includes carry.weft, and your shared inbound
route includes carry.weft too: the file’s nodes run per instance under the
group’s include, where the bridge’s values reach them, and stay shared under
the route’s, so the route’s runs need no instance.
A trigger every instance shares fires runs for no instance. If those runs would
reach something per instance, every fire would be refused, so the compiler
refuses the trigger (shared-trigger-reaches-per-instance), and the message
names what the trigger reaches and why each is per instance. The editor shows
the error, the trigger cannot be turned on or fired until it is fixed, and the
rest of the program still runs. If the trigger sits in a file you include at
several sites, only the site that reaches something per instance is refused. A route or a socket is not refused, since each
call can name its instance (an instance token, or the Weft-Instance header on
a gated route); a call that names none is refused when it arrives.
weft checks an instance’s value against the node’s own rules at three moments:
- when the program compiles: a rule that only needs the field to have a
value passes, since the value will be given later; a fallback is checked
like any written value (
@instance_filled("0 3 * * *")on aCronis refused right away); - when the value is given: a value the node refuses is refused on the spot, with the node’s own message, and nothing is stored;
- when a run starts: the instance’s values are read once, checked again (the program may have changed since), and carried by the run, so every node of it sees the same values even if one changes mid-run.
If an instance’s value changes and a live trigger of that instance reads it
(the trigger’s own field, like digest.cron, or anything its setup goes
through, like its connection), weft sets that trigger up again with the new
value, and only then saves the change. An event arriving meanwhile waits and
goes through afterwards, on the new value. If the new setup fails (the new
account refuses the connection, say), the old value and the old trigger stay,
and the change fails with the reason. If you change several fields in one go
(one apply(), one --set list), each trigger is set up again only once.
Which instance a run is for
Every run is for one instance or for none. Whatever started the run decides, and nothing changes it afterwards:
| What started the run | Which instance it is for |
|---|---|
weft run --instance user-42 | user-42 |
| An event arriving through an instance’s copy (its bridge, its trigger) | That instance |
A call to a Route with an instance token in the Weft-Instance-Token header | The token’s instance |
A call to a gated Route with Weft-Instance: user-42 | user-42 |
A timer, a webhook, an open Route, anything else | None |
A gated route is one whose auth setting is not "none", "api_keys" say
(weft’s messages call it a route gated by a connection), so the caller must
present a key or a signature before a run starts. If your own backend calls your
program for an instance, this is the route it uses. It is also the only place
weft honours the Weft-Instance header: on an open route anybody could claim
any instance, so weft refuses the header there and says why. A browser presents
an instance token instead.
If a provider signs the webhooks it sends to your route, auth: "hmac_sha256"
checks a hex HMAC-SHA256 of <timestamp>.<body> sent in X-Signature, with
the unix-seconds timestamp in X-Timestamp, refused when it is more than five
minutes off. For any other scheme (Standard Webhooks, Stripe, signed identity
tokens from your sign-in provider), you write the scheme’s block in auth,
and for how, go and read
checking a signed request.
If you call a route and want the run to be for an instance, call it at one of
its route addresses: /connect/local/<project id>/<path> on the install, the project’s own
port on your machine, or an API domain (for where those come from, go and read
answering on a URL).
A webhook also answers at a plain address without /connect, but a run started
there is never for an instance, and weft refuses an instance header sent there.
What is checked before a run starts
weft refuses a run before it starts, and says why, when:
- it reaches something per instance and is for no instance;
- it reaches a field the node needs and the instance has not filled;
- it reaches a value the instance has that the node’s rules refuse;
- it reads an instance’s copy that is not running.
If an event reaches an instance’s trigger before what it needs is filled, weft
holds it until the instance’s values change (from its settings page,
SetInstanceValues or ctx.values()), then runs it; activating the trigger
again also tries it. Nothing retries it on a timer, since only a new value can
close the gap. weft status shows under the trigger how many events wait and
which field they wait for, and so do ListInstances and
ctx.instances().list(), so your website can tell the customer what is
missing. An event that reaches the trigger while the instance’s copy is down is
tried again on a timer (sooner at first, then every five minutes) until the copy
is up. A call to a route gets the refusal instead.
An instance’s own infra
From the terminal, the verbs that start, stop, upgrade or remove infra take
--instance:
weft infra start --instance user-42
weft infra stop --instance user-42 # the container goes, its disk stays
weft infra terminate --infra bridge --instance user-42 --yes # the container goes, and its disk unless the node keeps it
--all-instances acts on every instance’s copies, and --all on the shared
infra and every instance’s. Taking an instance’s copy down takes
down only that instance’s triggers that depend on it, and reaches only that
instance’s runs. Taking shared infra down reaches further: every trigger that
depends on it goes down, every instance’s included, since none of them can
run without it, and they all come back with it.
weft infra status lists each instance’s copy on its own line. weft status
lists them under instance infra:, and under infra: it lists every infra
node your program declares: a @per_instance one says how many instances have
a copy, and one that was never started says not started.
Every place that reports a copy’s state gives the same answer: weft status,
weft infra status, ctx.infra(..).status() and the InstanceInfraStatus
node. A start or a stop on its way reads provisioning or stopping from the
moment it is asked for, before the container itself moves.
From your program, ctx.infra names the node and the instance:
ctx.infra("bridge").instance("user-42").start(&mut state).await?;
let copy = ctx.infra("bridge").instance("user-42").status().await?; // None: no copy
let every = ctx.infra("bridge").copies().await?; // shared + each instance's
weft never starts an instance’s copy on its own: you do, from the terminal as above, or your program does.
start() returns once the copy is running, so the next line can activate the
triggers that read it. A container can take minutes to come up, and the run
waits that long without holding a worker: between looks at the copy it parks,
the way a Wait node does. If the copy fails to come up, start() fails with
the reason. If somebody stops or terminates the copy while start() waits (a
customer pausing right after resuming, say), start() fails too, rather than
starting it again behind their back.
An instance’s own triggers
A trigger that reads a per-instance node exists once per instance, and you turn
each instance’s version of it on and off separately. weft activate with no
flags turns on every shared trigger and skips these, because it cannot know
which instances you mean:
weft activate --instance user-42
weft activate --trigger receive --instance user-42
weft deactivate --instance user-42
weft status lists every trigger under triggers:: one that exists once per
instance says how many instances have it on (receive: once per instance, 3 of 4 on), then each instance’s state on a line of its own. If you are
scripting this, weft status --json has the same list under triggers, each
entry with its node, perInstance, and either shared (the program’s own
trigger) or copies (one per instance), and the infra it reads under
dependsOn. Each copy has its instance (absent on shared), its state
(off, turning_on, on or behind), downWith (health or infra) when
the health check or an infra command took it down, and heldBack (the reason)
when it should be back but another build keeps it down. While its queued
events wait on something, it also has waiting ({ fires, on, reason }: how
many fires are held, what they wait on, one of room, trigger, instance,
infra, picks or failure, and why). weft instances gives one line per instance: how many of its
triggers are on, how many of its infra copies are up, and when it last ran.
Every trigger verb acts on the program’s own triggers unless you name
instances. A plain weft deactivate turns off the program’s own triggers,
leaves the instances’ on, and tells you how many instances still have
triggers on. The same goes for weft resync: a plain one moves only the
program’s own triggers. Since one build answers all of a project’s routes, a
plain resync onto new code is refused while instances’ triggers are on the
older build, and the refusal names weft resync --all.
weft deactivate --all-instances # every instance's, not the program's own
weft resync --all # the program's own and every instance's
If a deploy should reach every customer, run weft infra upgrade --all and
then weft resync --all. That is what the workflow from weft ci add does.
From your program:
ctx.triggers().instance("user-42").activate(ReleaseHeld::Run).await?;
If you want to turn one off from your program, the call takes a few choices; for those, go and read taking something down from a run.
If an instance’s trigger reads its copy and that copy is not running, activating the trigger is refused, with the command that starts the copy.
If an instance’s trigger reads a value it fills (digest.cron, say),
activating it before one was given is refused, naming the field, unless the
field has a fallback. If you want an instance’s triggers on the moment it is
created, give such fields a fallback: the trigger runs on the fallback until
the instance’s own value is saved, and then weft restarts it with that one.
Saving a value never turns on a trigger that is off.
Where an instance’s values are filled in
Each instance’s values are filled in on its settings page. It lists every field
your program marks @instance_filled, grouped by node, each with the control
the editor shows you: a connection picker for a connection, a searchable list
for a list field (spreadsheets, say, read through the instance’s own
connection), and a plain input for anything else.
If you want to show it to your customers, there are two places:
- the browser extension’s Your settings page, for somebody who has added an instance token to the extension;
- the
InstanceSettingscomponent, for your own website.weft connect-libcopies the library into your frontend (front/src/lib/weft-connectunless--intonames another folder); for how to mount it, go and read the README it copies along.
The picker never offers an instance the shared key, because it spends your credits.
If you want to test that page from the terminal, two commands do what it does (each mints its own instance token that lasts one hour and revokes it when done, so the instance’s own tokens are untouched):
weft connect openrouter --instance user-42 --door own --set-env key=OPENROUTER_KEYconnects an account as that instance’s from a key in your environment, and--grant <id>picks one it already has;--listand--disconnectwork the same way;weft instance-values --instance user-42lists its fields and values, and--set read.spreadsheet=1AbC --clear digest.cronchanges them. A value that reads as JSON (5,true,{"id": "..."}) is passed as JSON, so write a value made only of digits as'read.tab="12"'to keep it text.
If your program should read or change an instance’s values, ctx.values() does
it, with the same checks, and sets affected triggers up again:
let values = ctx.values().instance("user-42").get().await?; // by node, then field
let rearmed = ctx.values()
.instance("user-42")
.set("read", "spreadsheet", json!("1AbC..."))
.set("google", "account", json!({ "id": connection_id }))
.clear("digest", "cron")
.apply()
.await?; // returns the triggers it set up again
let theirs = ctx.connections().instance("user-42").list().await?;
ctx.connections().instance("user-42").forget().await?; // the values naming them go too
Instance tokens
An instance token acts inside one instance of one program, and nothing else. With it, a browser or the extension can:
- start that instance’s runs, through the program’s routes, in the
Weft-Instance-Tokenheader; - answer that instance’s waiting forms;
- read the displays of that instance’s copies, such as its bridge’s QR code;
- fill in that instance’s values, its connections among them.
An instance token always expires. If you mint one from the terminal (step 3
above), it reads a display only when you pass --display <node> for that node,
or --displays for all of them. A token your program mints reads all of the
instance’s displays and lives at most a year. If a customer just logged in to
your site, this is how your program hands their browser a token for one of
their instances (which instances are theirs is up to your own database):
let minted = ctx.tokens().mint_for_instance("user-42", Duration::from_secs(7 * 86400)).await?;
// minted.token goes to the browser, once
ctx.tokens().instance("user-42").revoke().await?;
The token comes out once, because weft keeps only a hash of it. So if a node’s body makes this call again after a crash, from a state saved before the mint, weft cannot hand back the value it minted the first time: it mints a fresh value for the same token, and the old value stops working. If your program already handed the first value to a browser, that browser needs the new one. After a pause, a body that runs the mint again makes a second token beside the first, so make the mint after the last place your node can pause.
An instance’s files
StorageScope::instance() holds the files of the run’s own instance, and
StorageScope::instance_of(id) those of any instance of this program:
ctx.storage(StorageScope::instance())
ctx.storage(StorageScope::instance_of(InstanceId::new("user-42")?))
The files belong to the instance, not to a run or to the project, so they stay
until your program or somebody deletes them (weft files rm), you wipe the
instance (see cleaning up after instances), or
a deadline you gave one arrives. Removing the project deletes them too. Using
instance() in a run for no instance is an error. instance_of(id) in another
program names that program’s own instance, but any run in your account can read
one of these files from its reference.
Who paid for a call
Every cost record says whose credential paid: the runtime’s own key
(Platform, when a node uses the shared key), yours (Author), or the
instance’s own (Instance). ctx.costs() reads them, and you can narrow it by
instance, node, service, run, who paid (paid_by) or since (unix seconds):
let spent = ctx.costs().instance("user-42").service("openrouter").since(month_start).list().await?;
A run’s costs are deleted with the run, so ctx.costs() only sees the runs
weft still keeps. If you add up a month of spending, as the example above does,
keep runs for at least a month. For how long runs are kept by default and how to
change it, go and read how long a run is kept.
Taking something down from a run
A program’s stop, terminate, upgrade and deactivate calls each take a
TakeDownChoice, the same three answers weft deactivate takes as flags (for
what each does, go and read taking triggers down).
Every answer you leave out takes its default:
let quiet = TakeDownChoice { door: Some(Door::Hide), running: Some(Running::Cancel), ..Default::default() };
ctx.trigger("receive").instance("user-42").deactivate(quiet, StopSelf::Keep).await?;
ctx.infra("bridge").instance("user-42").stop(TakeDownChoice::default(), StopSelf::Keep).await?;
Taking down an instance’s copy touches only that instance’s triggers and
runs, and taking down an instance’s trigger touches only the runs that trigger
started. A deactivate with running left at wait answers once the
triggers are down: the runs they started finish on their own, and the call
never waits on them from inside this run.
If the run making the call is one of the runs the call would take down, the last argument decides what happens to it:
StopSelf::Keep: it keeps going.StopSelf::Include: it is cancelled with the rest, once the take-down is queued; the call never returns.
If a node’s body runs again after a pause and terminates the copy again, the
second call finds it already gone and answers NoCopy, so nothing is
terminated twice (for the answers, go and read the project’s
infra).
Cleaning up after instances
If you want to see which instances there are, weft instances lists them,
and in a program ctx.instances().list() (or the ListInstances node) gives
one entry per instance weft holds anything for: how
many values, connections and live tokens it has, its infra copies, and its
triggers. It never hands back a value or a secret.
If you want to see an instance’s runs, weft runs --instance user-42
lists them. weft clean takes the same filters (instance, status, the node
that started the run, tag), and so does a program’s ctx.runs():
weft clean --project <id> --instance user-42 --yes
let week = Duration::from_secs(7 * 86400);
ctx.runs().instance("user-42").status("completed")?.older_than(week).clean(StillGoing::Leave, StopSelf::Keep).await?;
If a program wants to read its runs rather than delete them, the same filters
end in .list(limit) (the newest limit runs, 1 to 200, with total, how
many match in all) or .count(). The ListRuns and CountRuns nodes do the
same from the graph.
let failed_today = ctx.runs().instance("user-42").status("failed")?.count().await?;
clean never deletes a run that is still going. With
--cancel-running (or StillGoing::Cancel) it cancels it, and the run’s
rows go with the next clean; without it (StillGoing::Leave) the run is left
to finish.
weft has no “delete an instance” command. The WipeInstance node removes
everything attached to one: its triggers, the copies of the infra nodes you
name in its infra input, its values, its connections, its tokens, its files,
and its runs. If a wipe stops part way, run it again to finish it.
The instance nodes
If you want a graph to bring an instance up and remove it without writing code,
the instances package of the standard catalog has a node for each step:
| Node | What it does |
|---|---|
StartInfra | Starts an instance’s copy of an infra node, and fires once it runs (or, with waitUntilRunning off, once weft accepted the start) |
StopInfra | Stops it, keeping its disk |
TerminateInfra | Deletes it and its disk |
InstanceInfraStatus | Reads the state of an instance’s copy |
ListInstanceInfra | Lists every copy of an infra node, the shared one and each instance’s |
ListInstances | Lists every instance weft holds anything for (values, connections, copies, tokens, triggers), with counts and states, and the events waiting on a field not yet filled |
CurrentInstance | Says which instance the run is for: fires instance with its id, or nobody |
InstanceCosts | Lists what an instance’s runs cost, with the total |
GetInstanceValues | Reads an instance’s values for the @instance_filled fields, keyed node.field |
SetInstanceValues | Sets or clears an instance’s @instance_filled values, and sets up again any of its triggers that read a changed one |
ActivateTriggers | Turns on an instance’s triggers |
DeactivateTriggers | Turns them off |
MintInstanceToken | Mints an instance token to hand to a browser, as a secret: the run’s record never holds it. A Reply sends it revealed where its type says it is a secret |
WipeInstance | Removes everything attached to an instance |
If you leave instance unwired on StartInfra, StopInfra, TerminateInfra,
ActivateTriggers or DeactivateTriggers, it acts on the program’s own copies
instead: the shared copy of an infra node not marked @per_instance, or the
triggers every instance shares. StopInfra, TerminateInfra and
DeactivateTriggers take the same answers as weft deactivate as inputs
(door, doorForMinutes, held, running), each defaulting the same way,
and ActivateTriggers takes held.
If your backend should create instances, put MintInstanceToken,
StartInfra and ActivateTriggers behind a gated route, in
that order. If you want the request to answer at once rather than wait
minutes for the copy, turn waitUntilRunning off on StartInfra and
reply with the token after it: the start is accepted by then, so a list read
right after shows the copy as provisioning. A reply placed before the start
leaves nothing for that list to show yet. Wire the token straight into the
Reply’s body, or into a JsonObject whose output you declare with the
secret in it (-> (object: { token: Secret[String] })). In a plain
JsonDict the browser would get the marker instead of the token. If one of the instance’s triggers reads a value with no
fallback, activating it at creation is refused, so leave
ActivateTriggers for a second call your website makes once the
settings are saved. The same route can remove an instance with WipeInstance:
weft cannot tell when an instance is no longer needed, so your backend says so.
When something goes wrong
| What you see | What it means |
|---|---|
'bridge' exists once per instance, so this run needs to know which instance it is for | The run reaches something per instance and nobody named one. Add --instance, or start the run through an instance’s trigger, with an instance token, or with the Weft-Instance header on a gated route |
Trigger 'tick' is shared by every instance, so its runs are for no instance, but they reach 'send' | The compiler refuses the trigger: the trigger’s runs would reach something per instance, and nothing could pick the instance. Make the trigger per instance too (wire it from what is per instance, or write one of its fields @instance_filled), or keep what is per instance out of what it runs |
instance 'user-42' has not filled 'read.spreadsheet' | The run needs a value that instance never got. Fill it in on its settings page, or your program sets it with SetInstanceValues or ctx.values() |
instance 'user-42' at 'digest': ... | That instance’s value breaks the node’s own rule, which the rest of the message states. Change it on its settings page, or your program does with SetInstanceValues |
connect your account at 'google' first: this list is read through it | A list on the instance’s settings page is read through its connection at that node, and none is connected yet. Connect it first, at that node |
infra not running for: bridge (instance 'user-42') | That instance’s copy is down. The message names the command that starts it |
these triggers' infra is not running: ... | You activated an instance’s trigger before its copy was up. Run the command it names, then activate again |
the Weft-Instance header is honoured only on a route gated by a connection | You sent the header to an open route. Gate the route, or use an instance token |
a bare fire runs for no instance | An instance header reached a webhook’s plain address. Call the route at /connect/... |
this token has expired; ask for a new one | The instance token’s expiry has passed. Mint a new one |
this door takes an instance token | A token that is not an instance token (a plain api token, say) reached an instance’s settings page. Mint one with weft token mint --instance <id> or ctx.tokens().mint_for_instance |
an instance connects its own account; the shared key is the program author's | Somebody tried to connect an instance with the shared key. Connect the instance’s own account |
Deploying to your cloud
If you want a program to serve real people, you put weft on your own GCP
project once, and then send each project there with the commands you
already use on your machine, adding --on prod. prod is a name you give your cloud install in the project’s weft.toml, and
weft calls it a target.
You do two jobs, from two repositories:
- The install, once, by a workflow in your own fork of weft on GitHub.
- Each project, from its own repository:
weft activate --on prodsends the program, and a workflow written byweft ci addsends its frontend to Cloud Run.
Install weft on GCP
If you work with Tangle, it can do this whole section for you. It first gets
gh and gcloud installed and logged in on your machine (the sign-ups, the
billing card and the two logins are the parts only you can do, in your
browser), then runs every step below itself.
Once, by hand
Before the install workflow can create anything, it needs permission to act
on your GCP project, and a bucket where Terraform (the tool that creates
the cloud pieces) keeps its state. You need gcloud logged in as an owner
of a GCP project with billing on.
Create a new GCP project just for weft, with nothing else in it. Weft’s own accounts get broad rights inside the project they run in: they create and delete machines, services and service accounts there, and can act as any service account in it. In a project of its own, that reach stops at weft. The programs you deploy never get those rights; each one runs as an account of its own that can only reach what it needs.
The commands below turn on the services
weft uses, make the bucket, and create a robot account (a service account)
that your fork’s workflows may act as. You store no key in GitHub:
GitHub tells Google which repository a workflow runs in, and Google lets only
FORK act as the account.
Replace the three values on the first line. FORK is your fork’s
owner/repo, with the same capitals as on GitHub, because Google compares
it letter for letter.
If you are on the free tier and have no reason to be anywhere in particular,
keep us-west1 and put the database (below) in Neon’s aws-us-west-2: both
are in Oregon, so the install’s calls to its database stay short. Google’s
always-free machine and 5 GB of storage exist only in us-west1,
us-central1 and us-east1, and Neon runs on AWS and Azure rather than
Google, so that pair is the one place both free tiers sit side by side.
PROJECT=my-project REGION=us-west1 FORK=me/weft
NUMBER=$(gcloud projects describe $PROJECT --format='value(projectNumber)')
# On a new project IAM can take a minute to catch up (with the services just
# enabled, with the account just made), so each IAM step below is tried
# again, quietly, for up to three minutes; a last try shows its own error.
retry() {
for _ in $(seq 18); do
"$@" >/dev/null 2>&1 && return 0
echo "waiting for IAM to catch up, trying again in 10 seconds"
sleep 10
done
"$@"
}
gcloud services enable --project $PROJECT \
iam.googleapis.com iamcredentials.googleapis.com sts.googleapis.com \
cloudresourcemanager.googleapis.com serviceusage.googleapis.com
gcloud storage buckets create gs://$PROJECT-weft-state --project $PROJECT \
--location $REGION --uniform-bucket-level-access
gcloud iam service-accounts create weft-installer --project $PROJECT
retry gcloud projects add-iam-policy-binding $PROJECT --role roles/owner \
--member serviceAccount:weft-installer@$PROJECT.iam.gserviceaccount.com
pool() {
gcloud iam workload-identity-pools describe weft-install --project $PROJECT \
--location global >/dev/null 2>&1 \
|| gcloud iam workload-identity-pools create weft-install --project $PROJECT \
--location global
}
provider() {
gcloud iam workload-identity-pools providers describe github --project $PROJECT \
--location global --workload-identity-pool weft-install >/dev/null 2>&1 \
|| gcloud iam workload-identity-pools providers create-oidc github \
--project $PROJECT --location global --workload-identity-pool weft-install \
--issuer-uri https://token.actions.githubusercontent.com \
--attribute-mapping google.subject=assertion.sub,attribute.repository=assertion.repository \
--attribute-condition "assertion.repository=='$FORK'"
}
retry pool
retry provider
retry gcloud iam service-accounts add-iam-policy-binding \
weft-installer@$PROJECT.iam.gserviceaccount.com --project $PROJECT \
--role roles/iam.workloadIdentityUser \
--member principalSet://iam.googleapis.com/projects/$NUMBER/locations/global/workloadIdentityPools/weft-install/attribute.repository/$FORK
The robot account owns the project, because Terraform turns on the rest of the Google APIs weft uses and creates the network, weft’s Cloud Run services and the accounts everything else runs as.
Then, in your fork’s settings on GitHub, add these repository variables:
| Variable | Value |
|---|---|
GCP_PROJECT_ID | $PROJECT |
GCP_REGION | $REGION |
GCP_ZONE | a zone inside it, like us-west1-b |
TF_STATE_BUCKET | $PROJECT-weft-state, the bucket you just made |
GCP_WORKLOAD_IDENTITY_PROVIDER | projects/$NUMBER/locations/global/workloadIdentityPools/weft-install/providers/github (echo $NUMBER prints the number) |
GCP_INSTALL_SERVICE_ACCOUNT | weft-installer@$PROJECT.iam.gserviceaccount.com |
A database
weft keeps everything in a Postgres database you bring, and only needs its address. Any Postgres the internet can reach works. If you want the install to cost next to nothing while nobody uses it, pick one that scales to zero: once nobody uses the install, weft stops querying it, so the database can sleep (for what keeps it awake, see what you get). If you work with Tangle, it can create one that scales to zero and set the secrets below for you.
Put its address in a repository secret named WEFT_DATABASE_URL, as a
connection URL (postgres://user:password@host/db?sslmode=require).
If that address goes through a pooler that hands out a connection per
transaction, also add a direct address of the same database as
WEFT_DATABASE_LISTEN_URL. weft waits on changes to its rows with a Postgres
LISTEN, which needs one connection that stays open, and such a pooler hands
out a different connection for each transaction. If you leave it out and the
address cannot listen, weft refuses to start and its log names this secret.
If your program will be busy (schedules every few minutes, a connection held
open all day, a site calling it constantly), its database rarely gets to
sleep, and a hosted free plan runs out: Neon’s caps the data sent out of the
database at 5 GB a month, and every query weft makes crosses the internet to
reach it. For that, the install can have a Postgres of its own instead, on
Cloud SQL in its own region and network. Set the repository variable
WEFT_DATABASE to cloud_sql and leave WEFT_DATABASE_URL out. It bills
every hour it runs, even with nobody using the install, and Google’s Cloud
SQL pricing page has the price for your region. Its size is the repository
variable WEFT_CLOUD_SQL_TIER (db-g1-small, a small shared machine, when
unset).
You can switch to it later. Run weft maintenance on --on prod, set
WEFT_DATABASE to cloud_sql, and run the workflow again: it makes the
install’s own database, copies everything from the one it used, and only then
points the install at it. Until the install uses the new database, every run
of the workflow copies again from scratch, so if anything fails on the way,
the install stays on the old database and running the workflow again starts
over. Then weft maintenance off --on prod. Once you are happy with it, you
can delete the WEFT_DATABASE_URL secret and the old database.
How many connections the install takes
A Postgres database only takes so many connections at once (its
max_connections: 50 on Cloud SQL’s db-g1-small). Each copy of weft’s
dispatcher and broker asks the database for that number when it starts. It
leaves out the connections the database keeps for its superusers, and 3
more for anyone else (your psql, say), then splits the rest evenly among
the most copies the install may run: 2 dispatchers and 4 brokers unless you
change it. On db-g1-small that leaves 44, split six ways: 7 each, with
the 2 left over unused. All the copies together therefore never ask for
more than the database takes.
When every connection of a copy is in use, its next request waits its turn,
and weft status says so (see when something goes
wrong). Each copy logs sized the database pools to this copy's share when it starts, with its numbers.
If you want the install to handle more at once, raise WEFT_CLOUD_SQL_TIER
to a larger machine, which takes more connections. If you would rather pick
the number yourself, set WEFT_CLOUD_SQL_MAX_CONNECTIONS. The workflow
refuses a number below 25, and with the default copy limits a copy does not
start below 42. Unset, Cloud SQL uses its own number for the tier; once you
set it, it stays at your number whatever the tier, so raise it along with the tier. Setting
or changing it restarts the database once, the next time the workflow runs.
If you want more copies of the dispatcher or the broker running at once,
raise WEFT_DISPATCHER_MAX_COPIES (2 when unset) or
WEFT_BROKER_MAX_COPIES (4 when unset). Every copy then gets a smaller
share, so give the database more connections first: a broker copy needs a
share of at least 6 and a dispatcher copy 5, and one whose share is smaller
refuses to start, with a log line that names the numbers and these
variables.
If you want a role to have more memory, set WEFT_DISPATCHER_MEMORY (1Gi
when unset), WEFT_BROKER_MEMORY, WEFT_LISTENER_MEMORY or
WEFT_SUPERVISOR_MEMORY (512Mi each when unset), a whole number of Mi or
Gi from 512Mi to 4Gi, or WEFT_HOLDER_MEMORY for each copy that holds a
trigger’s open connection (512Mi when unset, at least 512Mi). The install sets each one on every run, so a value you
change by hand in the Cloud Run console is put back the next time the workflow
runs. The dispatcher gets more because every version is compiled there, and on
Cloud Run the files a build writes are held in memory, beside the programs the
dispatcher keeps read (about 90 MiB at most). A build’s files may take up to a
quarter of the dispatcher’s memory, with room kept beside them for the version
files the dispatcher fetched before: a build that needs more fails with No space left on device and leaves the copy serving everything else, and raising
WEFT_DISPATCHER_MEMORY raises that quarter with it.
These are repository variables on your fork, like the ones above, so merging a new weft into the fork keeps them. After you change one, run the install workflow again.
If you bring your own database, each share comes from that database’s
max_connections, which a pooler’s address (Neon’s, say) reports too. If
the install waits on it too often, move to a plan of your provider’s that
takes more connections, or lower the two copy limits.
OAuth apps of your own (optional)
If your programs sign in to services through OAuth apps of your own
(the apps file), put that file’s contents
in a repository secret named WEFT_ACCESS_APPS. Otherwise skip it.
Run the workflow
Open the Actions tab of your fork. GitHub turns workflows off in a new fork, so if the tab asks, enable them first. Then pick install on GCP and run it. It takes weft’s CLI and images from the release when your fork holds the same source (and builds the ones it changed), pushes the images to your project, and creates everything listed under what you get. If you run it right after weft’s main moved and your fork caught up, the release for that source may still be building: the workflow waits for it rather than compiling the same thing, and says so in its log. When it does have to build, it keeps the compiled dependencies for the next run, so after an upgrade only what changed in weft compiles again. Among the images is the standard worker, which every project made only of the standard nodes runs on, so the first of them to run after an upgrade starts at once instead of waiting a few minutes for that build. It also keeps the CLI it used in your install, so your projects’ deploys download it instead of compiling it.
When it finishes, its last step (in the run’s summary, and in its log for
gh run view <id> --log) gives you the install’s address, a Cloud Run
address like https://weft-role-dispatcher-123456789.us-west1.run.app, and
the line that logs you in with the install’s own operator key: the
password your CLI uses to act on the install. The key is kept in Secret
Manager and never printed in the workflow’s logs; that line reads it on your
machine (it needs gcloud) and hands it straight to weft login, below.
The address works at once, over HTTPS, with no extra charge. If you want a name of your own instead, see your own domain, which costs money.
If you want to upgrade weft, run weft maintenance on --on prod from a
project, merge upstream into your fork, run the workflow again, rebuild your
CLI from that commit (./setup.sh --cli), then run weft maintenance off --on prod. The first takes down every trigger that is on and stops the
infra that runs, so the old weft and the new one never run side by side, and
the workflow refuses to replace anything while something is on. Terraform
changes only what is different from last time. The CLI goes before off
because an older CLI cannot read what off answers, and activate refuses
to deploy from one (when something goes wrong).
The last builds those projects again on the new weft and brings back exactly
what the first took down. For what each step
does, go and read updating weft.
This works from an install of any age, however long ago you last ran the workflow.
Once the new weft is rolled out, the workflow deletes the older revisions of
each of weft’s own services (the dispatcher, the broker, the listener and the
supervisor), and that stops every copy of the old weft still running, even one
the editor or a worker was holding a connection to. Each finishes what it was
answering, closes what it held open and gives back the work it was in the
middle of, all within the 10 seconds Cloud Run gives it, and the editor and the
workers open again on the new copies by themselves. So the old weft and the new
one never take work side by side. If a service’s new revision is not ready, or
does not get all of its requests (traffic you pinned or split by hand), that
step stops with an error naming the service and deletes nothing. The holders
are not on that list: they run in a worker pool, which moves every one of its
copies onto its newest revision as soon as that revision is rolled out, so the
old holders stop by themselves. That holds while the pool sends its copies to
its latest revision, which is how the install sets it up. If you split its
copies between revisions by hand, that split stays through every later
update and the old holders keep running; send every copy to the latest
revision again with gcloud run worker-pools update-instance-split weft-holder --to-latest --region <your region>.
If you roll a revision out by hand, outside the workflow, the copies of the
older revisions keep running until the next run of the workflow deletes them,
or until you delete them yourself with gcloud run revisions delete.
Once in a while, weft’s database starts its history over, and a database an
older weft built cannot come across (it last happened when executions were
renamed runs). Before it deploys anything, the workflow checks your install’s
database. If it is one of those, the workflow stops with an error that names
the reset_database input, and your install stays as it was. If you want to
go ahead, run the workflow again with reset_database ticked (from a terminal:
gh workflow run install-gcp.yml -f reset_database=true). It wipes the
database, builds the new weft’s in its place, and deploys. That deletes every
project’s run history, versions and stored connections, and the maintenance
with them, so there is nothing for weft maintenance off to bring back:
deploy each project again with weft activate --on prod, and make its
connections again with weft connect --on prod. Your project folders are
untouched. If the database did not need it, the workflow stops without
touching anything, and you run it again with the box unticked.
If something in a project does not come back with weft maintenance off --on prod, everything else is back anyway, and off says what went wrong. If the
message says the project’s copy of the standard nodes is older than this
weft, run weft catalog update in that folder once your weft checkout and CLI
are on the new commit, since it copies the standard nodes from the checkout
on your machine (it replaces nodes/base_catalog/, so any edits you made in
there are lost); otherwise fix what it says. Then run weft resync --on prod
there, which builds the project and brings back what the upgrade took down.
If you want weft gone, delete the GCP project you made for it: weft and everything it ever started live in that project and nowhere else, so that is the whole uninstall. Unlink its billing first, so it stops counting against your billing account at once:
gcloud billing projects unlink $PROJECT
gcloud projects delete $PROJECT
Google keeps a deleted project for 30 days, during which gcloud projects undelete $PROJECT brings it back. The database is yours and stays where it
is: delete it from its own provider if you no longer want it.
What you get
While nobody is using the install and no trigger keeps a connection open,
none of weft runs. Your database wakes only when something happens: a run
doing work, a timer or schedule coming due, a poll that finds something new,
something arriving from outside (a call to a route, a webhook, an event on a
connection a trigger holds), something weft itself has due (a door you gave a --door-for time limit
starting to refuse, a subscription to renew), or an editor or weft follow watching the
install (more on the editor below). A poll that finds nothing new wakes weft’s
listener and leaves the database asleep. A program’s infrastructure that
runs fine keeps nothing awake: each machine says so itself when its state
changes.
If a run cannot pause (its caller is still on the line, or it is unrecorded) and it waits, on a person’s answer or on its caller’s next message over a socket, it keeps its worker and one copy of the broker up for as long as it waits, which is cheap, and leaves your database asleep for all of it. The answer or the cancel that ends the wait reaches the broker over Pub/Sub, not through the database, and is pushed to the worker on its connection. The database wakes when the answer, the cancel or the caller’s next message arrives, or the run ends. If you want a wait longer than an hour, make the run one that can pause: Cloud Run cuts a request after an hour, so when a worker’s runs are all only waiting and its connection is a minute from that cut, the worker ends them with a message naming the limit, and stops. A caller that reaches that worker in the few seconds it takes to stop is told to try again. Cloud Run may send the retry to the same stopping worker for those few seconds, which turns it away again; once the worker is gone, the retry reaches a fresh one.
| Piece | What it is |
|---|---|
| Cloud Run | weft’s dispatcher, broker, listener and supervisor, a service each, and your programs’ workers, one service per program. Each scales on its own load, and to zero between calls. A service at zero starts again when it is called: by the part of weft that just wrote work for it, by a Cloud Tasks wake, or by a call from outside (your CLI, a route, a provider’s webhook) |
| The holders | a Cloud Run worker pool for the triggers that keep a connection open between events (a stream, a socket, an event subscription that dials out). weft runs at least one holder per 200 such triggers, more while the holders are busy, and none when there are none |
| Your database | everything weft keeps, at the address you gave it |
| Cloud Build | builds your programs’ images when you deploy |
| Pub/Sub | three topics, which let weft’s parts tell each other things without going through your database: weft-machine-events carries Compute Engine’s record of an infrastructure machine stopping or failing to the supervisor, weft-news is where weft announces that a run a worker drives got an answer or was cancelled, and weft-holders tells the holders that the triggers they should hold changed, or that one of your connections was reconnected, rotated or removed, so they drop the credentials they kept for it (a token refresh is not announced here) |
| Cloud Tasks | two queues. weft-wakes delivers every timer, schedule and poll your programs set, each subscription renewal, and each holder’s check; if the install does not answer one, it keeps retrying, at most five minutes apart. For a trigger’s wake, the listener gives the trigger up after wake_attempts failed deliveries it answered (20, about an hour; a delivery that never reached it uses none) and says why on its node. For how to bring it back, go and read the listener. weft-ticks delivers the wakes weft sets for itself (a background job’s next look) and gives one up after ten minutes |
| Compute Engine | one machine per infrastructure unit your programs start (a database, a GPU model) |
| A storage bucket | weft’s files, reached as weft’s own service account: no key is made for it, so an organization that forbids service-account keys runs weft as is |
| Artifact Registry | two image repositories: one for the runtime, workers and infra nodes, which only the install writes, and one for frontends |
| Secret Manager | every secret weft uses |
Most triggers need nothing running between events: a route or a form is
called, a timer or a poll is woken by Cloud Tasks, and a service that pushes
its events (a provider’s webhook) calls the install. If you want to
know when an event subscription takes a push instead of holding a
connection, go and read
how a trigger picks its road.
You pay for a holder while it runs, but it wakes your database only when it
calls weft: when it starts, when an event arrives on one of its connections,
when a connection drops or comes back (once per drop, however many reconnect
attempts follow), when its CPU has moved enough to say so (below), and when
the weft-holders topic tells it something changed. If a holder crashes, weft frees its triggers within three
minutes for another holder, or its restarted copy, to take (for how it
notices, go and read the listener). A holder that is
stopped normally hands its triggers over at once.
Some triggers keep a holder far busier than others: a socket that talks all day costs more than a subscription that hears one event an hour. So each holder also watches its own CPU, sampled every ten seconds and averaged over the last minute, and weft runs as many holders as that load needs at about 50% each. If two holders say 60% and 55%, that is 115, and weft starts a third. That third holder starts with nothing, so once it has said its own number, the first busy holder to notice hands it a few triggers, keeping what brings it nearest 50%, and the spare holds work from then on. Once three say 20% each, that is 60, and weft stops one. It never runs fewer than one holder per 200 triggers, and never more holders than triggers, because a trigger is never split between two holders. A holder tells weft its number when it has a first minute measured, once more a minute after it took triggers or handed some over, and otherwise only once it has moved 10 points from the last one it said, so while the load holds steady it writes nothing and your database can sleep. When a holder’s average reaches 70%, it hands part of its triggers over whether or not a spare is up, keeping as many as should bring it back to about 50%. A single trigger that keeps a holder busy on its own stays where it is.
If you have VS Code pointed at the install, the editor keeps a live connection open to it, and the install and your database stay awake while it does. Once the window has been out of focus or untouched for 5 minutes, the editor closes that connection, and opens it again when you come back. Switching the editor back to your local install closes it at once.
If you want to change one of the install’s defaults (how many triggers a
holder takes, the CPU holders aim for and the CPU at which one hands
triggers over, a holder’s
CPU and memory, how many builds run at once, how
many bad tokens a minute one address may present, how many times a trigger’s
failing wake is delivered before it is given up), edit it in
deploy/terraform/gcp/variables.tf in your fork and run the workflow again.
If weft’s own work keeps failing, like waking one of its services, handing
a run to your project’s workers, or acting on your infrastructure, the
install sends an email. A run of your program that fails sends no email.
weft status --on prod lists these failures under waiting on weft:. You
get an email about one of them in two cases. If weft has given up retrying,
you get the email at once. If weft is still retrying, you get it on the
first failed try after the failure has been going on for 5 minutes
(alert_after_minutes changes that, see below). weft asks Google to email you once per
problem, and once more each time it clears and then comes back. Google also
sends at most one of these emails every 5 minutes for the whole install, a
limit separate from the wait above, so a problem that comes up soon after
an email may get no email of its own. It still shows in weft status. If
you run weft on your own machine, there is no email: weft status, without
--on prod, is where these failures show.
When an email arrives, run weft status --on prod in each of your
projects’ folders:
a problem with the install as a whole, like waking one of its services,
shows in all of them, and a problem with one project, like weft failing to
hand a run to its workers or to act on its infrastructure, shows only in
that project’s status.
The emails go to a person with the Owner role on your Google Cloud project,
usually the account you made the project with. Only a person granted Owner
on the project itself counts: groups, service accounts, and an Owner role
inherited from a folder or organization are skipped. If several people hold
Owner directly, the emails go to the one whose address comes first in
alphabetical order. If you want them sent somewhere else, add a repository
secret named WEFT_ALERT_EMAIL holding that address (a secret, so a public
fork’s run logs never show it). The alert that sends these
emails is on by default and costs about $0.35 a month on Google’s bill. If
you want it off, set alerts_enabled to false in
deploy/terraform/gcp/variables.tf. If you want weft to wait a different
number of minutes before emailing about a failure it is still retrying,
change alert_after_minutes in the same file (set it to 0 to be emailed at
the first failure). Each of these changes takes effect the next time you run
the install workflow.
weft generates the rest of its secrets itself: the key your saved
connections are encrypted with, the key live callers’ tickets are signed
with, and the install’s own operator key. The encryption key is made once and kept
in the Terraform state, in the $PROJECT-weft-state bucket, and copied
into Secret Manager, which is where weft’s services read it. If it is ever
lost or replaced, every saved connection stops working and has to be made
again, so keep the bucket and never run terraform destroy.
A cloud install belongs to one person. Every project on it shares one private network, so a project’s code can reach another project’s infrastructure there; keep projects you would not trust with each other on installs of their own.
Your own domain
If you want the install, or one of your projects, at a name like
weft.example.com, buy the domain from any registrar.
A domain needs a load balancer in front of the install, which holds the
domain’s certificate. Google bills it by the hour while it exists: about $18
a month, plus $0.008 per GB that goes through it. weft makes one with your
first domain, every later domain shares it, and removing the last one takes
it down. For your first domain, pass --accept-cost to say you accept
that charge; without it, weft domain add stops and prints the cost. Run
this from any project that has the install as a target:
weft domain add weft.example.com --on prod --accept-cost
It prints the DNS record to set at your registrar (type A, the name, and
the load balancer’s address) and waits until the name points there, then
until Google has issued the domain’s certificate, which usually takes a few
minutes once the record is seen. The record has to point straight at that
address: on Cloudflare, set it to “DNS only” (the grey cloud), because a
proxied record hides the load balancer and Google never issues the
certificate. If the name already points somewhere else (a tunnel to your
machine, say), remove that record first. weft domain list --on prod shows
every domain with its record and where its certificate stands, and says why
when it is stuck.
Once the certificate is issued, the domain becomes the install’s address.
Every link it hands out from then on uses it: sign-in callbacks, webhook
addresses, file links, weft target export. The run.app address keeps
working, so nothing you already handed out breaks. Two things are yours to do
then, and weft domain add (or weft domain list) prints both:
- Each provider you sign in through (Google, Meta, TikTok…) only sends
people back to addresses its app lists. Add the new callback,
https://weft.example.com/access/oauth/callback, to each app’s allowed redirect URLs. - A site that talks to the install was given its address by
weft target export; run it again so the site uses the domain.
A domain can also serve one project instead of the whole install:
--for api answers that project’s routes at the root of the domain
(https://api.example.com/users/42, as well as
https://<the install's address>/connect/local/<project id>/users/42), sending its calls
straight to the project’s own Cloud Run service, and
--for frontend --to <address> passes visitors on to the project’s
frontend on Cloud Run, at the address weft frontend ls --on prod shows for
it.
Moving from your machine to the cloud
If you built an app on your machine and now run it on a cloud install, you do
not have to make every connection again or upload every file by hand. weft migrate copies them from the install a command acts on to the one --to
names, and the install they come from keeps everything:
weft migrate connections --to prod --all
weft migrate files --to prod --prefix project/<project id>/
You always say what to copy: --all, or any mix of --id <id> (repeat it
for several), --since/--until (a date like 2026-10-01, or a time), and
--service/--project for connections or --prefix for files. Add
--dry-run to see what would be copied first. Each line says what happened
to one item: copied, already there, or skipped and why.
A connection’s secrets are never in the clear on the way: the cloud install
makes a key pair, your machine seals the connections to it, and only the
cloud install opens them. Each connection keeps its id, so a program that
names one finds it. The connections your nodes use are picked per install,
so the command ends by printing the weft connect --grant lines to pick them
on the cloud too. A connection that spends one of the install’s own keys
copies only when the cloud install holds that key. A copied sign-in (Google,
Slack and the like) renews itself with a refresh token that both installs
now hold. Most providers accept that, but one that replaces its token on
every renewal breaks whichever install renews second; the command names the
sign-ins it copied, and if one stops working on the cloud, connect it again
there with weft connect.
A file keeps its key, its version and its lifetime, so a value that names it
still finds it. A run’s file stays behind with its run, unless you kept it.
Run weft migrate from the project’s folder, where --to prod means the
target the project names.
Deploy a project
In the project’s folder, give your cloud install a name, log in to it, and deploy:
weft target add prod https://weft-role-dispatcher-123456789.us-west1.run.app
gcloud secrets versions access latest --secret <the secret> --project <project> | weft login prod --key-stdin
weft infra start --on prod # only if the program has infrastructure
weft activate --on prod
If your program has infrastructure (a database, a bridge), start it before
the first activate: activate never starts it for you, and if a trigger reads
infrastructure that is not running, it refuses with these triggers' infra is not running: db, naming each piece. weft infra start --on prod builds the
program and brings its infrastructure up, which on GCP means a machine per
piece booting, so give it a few minutes; weft status --on prod shows how
far each one got.
weft target add records the install’s name and address in weft.toml.
Commit it, and your team gets the same prod:
[targets.prod]
url = "https://weft-role-dispatcher-123456789.us-west1.run.app"
weft login prod takes an operator key (from its hidden prompt, or piped in
with --key-stdin), checks it against the install, and keeps it in ~/.config/weft/credentials.toml, which only you
can read. If a teammate needs their own key, run
weft token mint --operator --on prod and send them what it prints. To see
the keys you have handed out, run weft token ls --on prod, and to cancel
one right away, weft token revoke <id> --on prod. The install’s own key,
the one the workflow gave you, is not among them: it is no row in the
install’s database but its secret in Secret Manager (<name>-install-key),
so it works until that secret changes, and weft token revoke cannot retire
it. An install set up before that also lists it under the name bootstrap,
a row an older weft wrote for the same key: revoking that row is up to you,
and the key keeps working through its secret either way.
Every command takes --on the same way: weft status --on prod,
weft runs --on prod, weft logs <run-id> --on prod. Without it, a
command acts on your own machine; nothing makes prod the default.
What activate --on prod does
Your machine compiles the project first, so a mistake shows up on your screen before anything is uploaded. Then it uploads the project’s sources, only the files the install does not have yet, and the install does the rest: it compiles them again itself, builds the images the program needs, and activates.
Deploy a change
If you want a change to go live on a program that is already on, run these
(weft activate would only turn on what is off):
weft infra upgrade --on prod --all
weft resync --on prod --all
The upgrade puts the new build on the infrastructure that is running (with
none, it does nothing), and the resync brings onto it every trigger that is
on and behind. --all includes
every instance’s (for those, go and read programs with
instances). By default nothing that arrives meanwhile is lost:
the triggers’ door holds what arrives and runs it on the new version once
they are back, a caller on a route is answered 503 with a Retry-After and
its client tries again, and the runs going finish on the old version first.
For the other answers (--door, --held, --running), go and read taking
triggers down.
Only the change is uploaded, and only the image it touches is rebuilt. If nothing changed, nothing is built.
Roll back
Each install keeps its own history of what was deployed to it. If you want
to go back, find the version in weft tree --on prod, then let
weft branch put its files back in your folder and deploy them:
weft branch <version> --on prod
weft infra upgrade --on prod --all
weft resync --on prod --all # and `weft activate --on prod` for what is off
If your folder holds changes no version records yet, weft branch records
them on prod as a version first and prints the weft branch that brings them
back. Unless weft clean --images removed them since, the old version’s
images are still there from its first deploy, so nothing is rebuilt.
Connections
Each install keeps its own connections and its own picks of them: which connection a node uses is picked on the install, never written in your files (picked on each install). A node you connected on your machine has no pick on prod until you make one there, with the same walkthrough:
weft connect --on prod
If a run on prod reaches a node with nothing picked there, it is refused
before it starts, and the message names the node and
weft connect <node>; add --on prod. An instance’s own connection (a
field written @instance_filled) lives in the install it was made on, so
your customers connect through your frontend on prod, the same way they do
locally.
Look at prod from the editor
If your project names a target, the graph has a switch in its top right
corner: local, then each target. If you want to see what prod is
running, click prod. The graph then shows the program as it was last
deployed there, along with prod’s runs, status and connections. From then
on, Run, Activate, the Runs list and the version tree all go
to prod, as if you had typed --on prod on each command.
You cannot edit the program there, because it is a copy of what prod runs.
You can change its connections, though: Connect and Disconnect on a
node change which connection prod uses, and prod sets up again any trigger
that relies on it. If the files in your folder are a different version from
prod’s, a banner at the top says so. Activate deploys the files in your
folder, exactly like weft activate --on prod, not the copy you see on
screen.
If you want to edit again, click local; closing the graph does the same.
If a target shows faded, you have not logged in to it yet: log in to it as
in deploy a project. And if you want prod’s program as files you can read,
weft running-source <folder> --on prod writes them into a new folder and
leaves your own files alone.
Deploy the frontend
If you want the project’s frontend (in front/) on the cloud, it runs on
Cloud Run, calls weft at the install’s address with a token of its own,
and reaches the program’s infrastructure (its database) over the install’s
private network. In the project’s
folder (the repository must already be on GitHub, and you must have run
weft login prod):
weft frontend add front --repo me/shop --on prod
weft ci add --cloud gcp
weft target export prod --github
weft frontend add reads the repository’s id with gh (access is granted
to that id, so nobody who takes the name later gets it), makes the frontend
a Cloud Run service of its own,
public from the start, and lets that repository’s workflows sign in to your
GCP project and change that one service: they cannot touch weft’s own
services or any other frontend’s. Every frontend deploys as the install’s one
deploy account, so a repository you add could deploy to another project’s
frontend too; add only repositories you trust with all of them. The name
(front here) is yours to pick: lower-case letters, digits and hyphens.
weft frontend ls --on prod lists a project’s frontends, and
weft frontend rm front --on prod removes one, its service included.
If your frontend runs somewhere else (Vercel, a server of your own), leave
out --repo: the install makes nothing, and the command writes the
frontend’s token, with the install’s address, to a file only you can read.
Put those in the frontend’s environment.
weft ci add writes .github/workflows/deploy.yml, a workflow you run by
hand from the Actions tab. It takes the weft CLI built from the source your
install runs: first the one the install workflow stored in your install
(signed in with WEFT_OPERATOR_KEY, its sha256 checked), which is how a fork
gets its own CLI without compiling it, then the one weft’s release built
(waiting for it while that release is still building), and it compiles the
CLI only when neither has one. Then it deploys
the program with weft infra upgrade --all then weft resync --all, then
builds front/ with Docker and deploys it to the repository’s service. If
you run weft ci add again, it replaces the file only if you have not edited
it; otherwise it stops and leaves your edits alone.
The workflow never turns anything on: the upgrade replaces only the
infrastructure that is running, and the resync moves only the triggers that
are on, so anything somebody took down stays down. Before the first deploy,
run weft infra start --on prod and weft activate --on prod once by hand.
The runs going when the deploy replaces what they run on finish first, and
the deploy waits for them however long that takes. If you would rather they
stop at once, set RUNNING: cancel in the workflow’s env: block, where it
says RUNNING: wait (that counts as an edit). If GitHub cuts the job off,
the upgrade or resync it had started carries on by itself on the install. If
it was cut off before the resync began, wait until weft infra status --on prod shows the upgrade done, then run weft resync --on prod --all by hand.
If the resync had already begun, it finishes on its own, and weft status --on prod shows how far it is.
If you are starting a new project, weft new <name> --ci gcp writes the
workflow for you.
weft target export prod --github uses the GitHub CLI (gh) to set the
variables and secrets the workflow reads, so run it with gh logged in. It
mints an operator key for the workflow, and gives the frontend the install
hosts for this repository a new token, with the name of its service (its
first one: weft frontend add makes none for a hosted frontend). Once it has
been deployed, its old token keeps working until the workflow’s next run has
deployed the new one, and then the workflow retires it. If you would rather paste them yourself, leave out
--github and it prints everything. If you use the printed version, copy the
secrets straight away: they are never shown again.
The frontend’s server reads three variables, which the workflow sets on Cloud Run:
| Variable | What it is | Value on your machine |
|---|---|---|
WEFT_DISPATCHER_URL | where the server calls weft: the install’s address, over HTTPS | http://127.0.0.1:14111 |
WEFT_TOKEN | the token the server calls with; never send it to a browser (on Cloud Run, the one weft target export made) | a token from weft frontend add <name>, written to a file only you can read |
WEFT_PUBLIC_URL | the start of any link a browser follows | http://127.0.0.1:14111 |
If the frontend’s server needs more than those three (the program’s database, a sign-in secret), put them in one file on your machine and hand it over with the export. Keep that file out of git.
weft infra env <node> --on prod --into front/.env.prod --set ...writes a database’s values into it, the same way it does locally.weft infra list-doors --on prodprints the database’s private address. Only what runs inside the install’s network can reach it, and your Cloud Run frontend is one of them.weft target export prod --github --front-env front/.env.prodstores the file as theWEFT_FRONT_ENVsecret, and the workflow puts it on Cloud Run beside weft’s own variables.
Cloud Run tells the container which port to use in $PORT, so
front/Dockerfile has to build a server that listens on it. If your
frontend builds differently, edit the workflow’s “build the frontend” step.
Routes on the internet
A Route answers at https://<the install's address>/connect/local/<project id>/<path>,
where <project id> is the id under [package] in your weft.toml and
<path> is the route’s path. The local and the project id are the same on
every install, prod included, so a route’s address on prod differs from your
machine’s only in the host. The
project also answers at its own Cloud Run address (weft status --on prod
shows it), where the call reaches your program with nothing of weft’s in
between: give that one, or a --for api domain, to a client that cares how
fast it is answered.
A browser opening a socket on a route that checks its callers is handed an
address under the same path on the same host, carrying a ticket, so one
address and certificate cover it.
If you want to stop a flood of calls from costing you money, each route has
its own limits, set on the node:
callsPerMinutePerCaller (60 by default), callsPerMinute for everybody
together (none by default), and callsAtOnce (100 by default); set any of
them to 0 to turn it off. A call past a limit gets 429 with
Retry-After before any run starts. weft status --on prod shows the refusals
of the last minute or two under “refused calls”: which route, how often, and by
which limit. If the
program runs on several copies of its worker, each copy counts calls itself
and hears the others’ counts once a second, so a per-minute limit can let
through about one second’s worth of extra calls, and callsAtOnce is shared
out between the copies that are up. A copy with nothing to do stops listening
so the install can sleep. When a call wakes it, that call is decided at once on
what the copy heard before it went quiet, and so is every call until the copy
hears back from weft’s broker: under a second normally, a few seconds when the
broker was asleep too and has to start. During that window the copy counts
itself as the only copy, so if several copies wake at the same moment, each
can let in up to the full callsAtOnce. The per-minute limits barely move:
they can only overshoot by the calls that reach the copy in those few
seconds. We chose this on purpose: the other way would make the first call
after every pause wait for the broker, and on a sleeping install that means
waiting for it to start.
When something goes wrong
401on every command: the key is wrong or was revoked. Runweft login prodagain. In CI, runweft target export prod --githubagain.- If you want weft’s logs, the services’ logs are in Cloud Logging:
gcloud logging read 'resource.type="cloud_run_revision"' --project <project> --freshness 1h. The holders’ logs are under the worker poolweft-holderin the Cloud Run console. - HTTPS fails at a domain right after
weft domain add: Google has not issued its certificate yet. Check that its record points at the addressweft domain list --on prodprints; the certificate follows once Google sees it. weft domain listsays the door in front of your domains refuses to follow them: Google refused to change the load balancer, and the message says why. weft tries again on its own twice, five and then ten minutes later, and after the third refusal it stops and leaves the message inweft domain list. Once you have fixed the cause, anyweft domain addorweft domain rmtries again at once.- weft does not start, and its log says the database session “cannot
LISTEN”:
WEFT_DATABASE_URLgoes through a pooler. Add a direct address of the same database as theWEFT_DATABASE_LISTEN_URLsecret (a database) and run the install workflow again. weft statusshows adatabase: busyline: in the last 15 minutes, requests waited more than a second for a database connection because every connection of a copy was in use. The line says how many times, when, what the database takes and each copy’s share, for exampledatabase: busy. Since 14:02 UTC, requests waited 42 times for a database connection, the last at 14:20 UTC (the database takes 50 connections, and each copy of the broker holds at most 7), followed by what gives the install more. The waiting copy’s log saysrequests are waiting for a database connectionat the first wait, then at most once a minute with the count. The CLI may printwarning: the install is busy, so this command waits and asks again(also when Cloud Run has no free copy of a service), at most every 30 seconds. If the command only reads, or Cloud Run had no free copy, it keeps asking until the install answers, and Ctrl+C stops it. If you see the line often, give the install more connections (how many connections the install takes).- A copy of the dispatcher or broker does not start, and its log says
this copy's share is ... and it needs at least ...: the copy limits split the database’s connections too thin. On Cloud SQL, raise the tier (andWEFT_CLOUD_SQL_MAX_CONNECTIONSwith it, if you set one); on a database you brought, move to a plan that takes more connections. Either way you can instead lowerWEFT_DISPATCHER_MAX_COPIESorWEFT_BROKER_MAX_COPIES. Then run the install workflow again. weft resyncorweft statussaysthe new version takes over once 2 runs on the old one end: the resync waits for the runs still going on the old version, however long they take, and names them.weft stop <run>ends one now;weft cancel-activategives the resync up and leaves its triggers down. Ctrl+C only stops following: the resync goes on.weft statussaysresync: stopped with triggers it took down still down: the line gives why, thenNext:with what to do. If the database was busy or out of reach, or the resync waits on infra or an activation still landing, it goes on by itself. If its build cannot run the triggers,Next:says to fix the source, give the resync up withweft cancel-activate, and runweft activate. Otherwise fix the cause and runweft resync --resume, or give it up withweft cancel-activate. For how a resync retries, go and read resync.- A trigger that keeps a connection open shows “waiting for a holder to take it” on its node: a running holder with room takes it as soon as weft tells the holders the trigger is there; otherwise weft is starting one. If it stays that way, the holders’ logs say why.
- A run on prod is refused with “has no … connection picked on this
install”: the node was connected on your machine, not on prod. Run the
weft connect <node>it names, with--on prod. - An image build fails: the error quotes the builder’s log, with the
cargooraptline that failed. Fix what it names and deploy again; only what changed rebuilds. - The frontend deploy fails at its
google-github-actions/authstep, or itsgcloud run deployis refusedrun.services.get: the repository is not the one its frontend was added with (weft frontend ls --on prodshows it, spelled as GitHub spells it,owner/name). Remove the frontend and add it again with the right--repo, then runweft target exportagain. - The install workflow stops in “create the cloud” with “no email to send
the alerts to was found”:
WEFT_ALERT_EMAILis not set, and no person holds the Owner role directly on your Google Cloud project, the only kind of owner the workflow emails. If you want the alerts, add a repository secret namedWEFT_ALERT_EMAILholding the address that should get them; if you don’t, setalerts_enabledtofalseindeploy/terraform/gcp/variables.tf. Then run the workflow again. - The install workflow stops in “create the cloud”: Terraform’s output in that step names the resource it could not make and why. If it names a quota (Compute Engine addresses or CPUs, say), raise it in the Google Cloud console and run the workflow again.
How the runtime is built
The runtime has four roles, each with one job.
| What it does | What it never does | |
|---|---|---|
| Worker | Runs your compiled program | Touch Postgres: it goes through the broker |
| Dispatcher | Decides what runs, and answers every request about a project or a run | Run your node code |
| Listener | Holds the timers, the open sockets and the subscriptions | Run your node code, or know which project it serves |
| Supervisor | Creates and watches the machines and containers your program asked for | Run your node code, or change a project’s infrastructure while another supervisor is changing it |
The dispatcher, listener and supervisor, together with the broker, are parts of one
binary, weft-runtime, and on your machine one weft-runtime process runs
all of them. On a cloud install each is a Cloud Run service of its own,
which scales with its own load and down to zero between calls. Triggers that
keep a connection open run separately, on holders: for how they work, see
the listener, and for what they cost, see
what you get. Workers are always
separate: one per program, started by the runtime.
Anything that has to survive a crash goes into Postgres, and every role reads it back from there.
flowchart TD
UI["CLI / editor / webhooks"] --> D["Dispatcher"]
D --> L["Listener"]
D -->|"run this, hand it this event"| W["Worker: your compiled program"]
C["Live caller"] --> W
L -->|"an event"| W
D -.->|"queued work"| S["Supervisor"]
D --> PG[("Postgres")]
L --> B["Broker"]
S --> B
W --> B
B --> PG
Where a run lives
Whether a run survives its worker dying is set by its trigger’s durable input,
or for a run you start by hand, by weft run --durable or --fast. For
what it does, go and read how a run is
kept, and for when each
kind of run waits for the database, when a run waits for its
writes.
Once a run has written anything, it has one row in the database, and the row says where the run is: waiting for a worker, running on the worker it names, parked on a wait with no worker, or ended.
When a trigger starts a run (a caller on a Route or a Socket, or an event
such as a timer, a subscription or a webhook), the run starts on the worker
that took the call or the event, and its row names that worker from its first
write. Every other run starts as a queued row: a run you start with
weft run, a paused run picking back up, a run another worker handed back,
and the setup runs that weft activate and weft infra start make. The
dispatcher hands a queued run to a worker, which claims it (the row names that
worker from then on) and carries it on from its record.
If the worker is still starting, that is no fault: the run waits 40 seconds and joins that same start.
If the worker answers that it is busy (its own 503: its memory is nearly
full, or it is stopping), that costs the run no try. It waits and is handed
out again, for as long as the workers keep answering busy. Each wait is as
long as the run has been turned away so far, at least 5 seconds and at most
a minute.
If the workers keep that up for longer than the install’s alert delay
(alert_after_minutes, 5 minutes by default), weft status lists it under
waiting on weft:, ending in this project's worker has answered busy for <time> (memory full); raise its memory (`weft workers set --memory <size>`), and on Google Cloud the install sends an alert email. The line
goes away once none of the project’s queued runs is still waiting on its
workers (answered busy, refused, or not reached at all) and a run of the
project has since left the queue: either the last one that was waiting
(it reached a worker, it ended, or you cancelled it) or one started after
the last failure. Changing the project’s worker settings clears it too, and
tries every queued run again from the start.
Whether an answer counts as a refusal depends on who gave it:
- The worker: any error it answers other than its busy
503. - The platform, answering in the worker’s place: any
4xxexcept404,408and429. A404,408,429or5xxfrom it is no answer. - Google Cloud, when weft asks it to create the worker: a
400,401or403. A429, a5xxor a timeout is no answer. - weft itself, when the project’s worker settings together are a shape no platform could run.
The run is handed out again 5 seconds after the first refusal and 20
seconds after the second, and the third cancels it, with a reason naming
who said the last no: its worker refused it 3 times: <error>, the platform refused weft's calls for it 3 times: <error>, or its project's worker settings kept it from any worker 3 times: <error>. The <error>
says what the last no was: the worker’s or the platform’s status and
answer, or which worker setting no platform could run. If nothing answers at
all, the run is handed out again 5 seconds later, then 20. If the third
try gets no answer either, the run is not ended: it stays queued and waits
until a worker of its project comes alive or a build of the project
registers, and then its tries start over. Either way, weft status shows
the error under waiting on weft:. If you change the
project’s worker settings (weft workers set or reset), every run still
queued is handed out at once, with its tries forgotten.
If a trigger starts a fast run and its worker dies straight away, the run may
never show up in weft runs, because a fast run’s record trails a few
milliseconds behind it and may not have reached the database yet.
Where a call goes
If a caller uses the project’s own address, it reaches your program’s worker
straight, with nothing of weft’s in between. On your machine the worker running the project’s current program
opens the project’s own port itself, so http://127.0.0.1:14200/cards is the
worker answering. For how that port is picked, go and read answering on a
URL. On a cloud install
each project is a Cloud Run service of its own, open to every caller, and that
service’s address is the project’s address. A weft domain add --for api
domain sends its calls straight to that service. weft activate prints the
address, and weft status shows it. For how a browser opens a socket on a
route that checks callers, go and read answering on a
socket.
The address an install shares between its projects (/connect/local/<project id>/...,
the tunnel’s hostname, the install’s own domain) still works: the dispatcher
picks the project by the id in the address and the program by the route, from
routes it holds in memory, and passes the call on. It adds headers giving the caller’s address and
the Host they sent, and the worker trusts those headers only
when the call carries weft’s own key. The dispatcher checks nothing else
and writes nothing.
Once a call reaches the worker, the worker does the checking: which route the call is for, whether its trigger is on, the instance it names and the route’s limits on how many calls it takes, and it asks the broker to check the route’s credentials when the route asks for one. The worker holds the project’s routes in memory, and the broker tells it the moment one changes. Unless the route asks for a credential or the call carries an instance token (which names the instance the call is for), nothing reaches the database before the run starts.
If a route’s trigger is down, however it was taken down, or still being set
up, the worker answers 503 with a Retry-After at once, so a client that retries
gets through once the trigger is on. When a new version of the program takes
over, on your machine its worker takes the project’s port only once the old
worker has nothing left running: the old one is then told to stop, killed 10
seconds later if it has not, and the new one starts. A caller arriving in
that moment is refused, as with any server restarting.
If a route still belongs to another version of the program than the one at
the project’s address (a new version is taking over), the call gets the same
503 and Retry-After, and lands on the new version once it holds the
route.
What happens when an event arrives
When a timer fires or a subscription delivers something, the listener holding it hands the event straight to the worker of its project, through the same door callers use: the worker checks whether the trigger takes work now and the trigger’s limits, and works out what the run reads. If the trigger is down and set to hold what arrives (for those settings, go and read taking triggers down), if it is past its limits, or if what the run reads is not ready (its infrastructure is down, its instance has not given a value it needs), the event waits in the trigger’s queue, and the dispatcher hands it over again once it can run. Otherwise the worker starts the run as a fast or durable run, whichever its trigger asks for, and writes what happens into the journal as it goes.
A webhook or a provider’s push arrives at the install’s own address: the dispatcher has the listener read it, then hands it to the worker the same way. An answer to a run that waits (a form, a person’s reply) goes through the dispatcher, which resumes that run. When the listener cannot reach the worker (the project has no address now, the call fails or goes unanswered), the event waits in the trigger’s queue the same way.
Whoever handed the event over keeps its call to the worker open until the run ends, so the platform counts the run as work: Cloud Run keeps giving the worker CPU, and a local install does not stop it as idle.
The worker
A worker runs one compiled program and serves many runs at once. The
program is named after a hash of what went into it, such as weft’s engine, the Rust
version weft pins, and each node package by its name and its content (never by
where the package sits in the project). So two projects that compile to the
same thing share one program, and every project whose nodes are all the
standard library’s, unchanged, runs the same one: the standard worker. A
project built with --referenced compiles a program of its own, since it
leaves out the nodes it does not use.
On your machine a worker is a plain process, and it runs in the project’s
folder, so its nodes open your files and start your programs the way
anything else you run does. If the project has several folders on this
machine (two checkouts of it, say), the worker runs in the one changed most
recently. Every worker program lives in ~/.local/share/weft/workers/<hash>/,
where every install on the machine finds it, and one that has to be compiled
is compiled there with your own Rust toolchain. The standard worker is there before any project asks for it:
for the default install, ./setup.sh downloads it along with the published
CLI when your checkout is exactly a commit CI has built for this kind of
machine, and otherwise (on a branch’s install too) the install compiles it
once when it starts. A project with nodes of
its own compiles them, and reuses whatever earlier builds on this machine
already compiled. If you have not installed Rust yet, go and read the
install page.
A project’s build and a live node test’s server each compile in a compile
lane: a compile cache of its own, about 2GB, under
~/.local/share/weft/worker-target/, shared by every install on the
machine. Every lane starts from a copy of
~/.local/share/weft/worker-target/seed/ (one more cache of the same size),
so a project’s first build compiles only its own nodes. The seed is the
cache this machine compiled the standard worker in, or, when the standard
worker was downloaded, a copy of the first lane to finish a build: that one
first build compiles everything once. There are four lanes, and a fifth build waits
for one to be free. A project’s build says so in the runtime’s log (weft daemon logs), every compile lane of this machine is busy; a build waits for one, and a live test on your terminal, every compile lane of this machine is busy with other builds; waiting for one (WEFT_COMPILE_LANES sets how many). If you want more builds at once and have the disk, set
WEFT_COMPILE_LANES in every shell you run ./setup.sh or weft daemon start from, since each start hands the install whatever that shell has,
and in the shells you run weft test-node from, since the CLI compiles test
servers itself.
If you want to spend less disk, WEFT_COMPILE_LANES=1 keeps one lane and
the seed. Builds then take turns: when several agents ask for builds at
once, one compiles and the others wait for the lane, then compile only what
the build before them did not already leave in it. A build of a program
this machine already has takes no lane at all, and neither does a run, so
running code is never held up. Two asks for the same build of one project
are one build, which both wait on. Each program weft compiles notes which
compiled pieces and which source crates its build used, and after each
build, weft keeps in that lane, in the seed and in the shared source crates
(~/.local/share/weft/worker-crates/) only what some program still on this
machine was built from. When a program goes (a project removed, a weft
update replacing the standard worker), what only it used goes at the next
build.
On a cloud install the worker is an image instead, and the [build.worker]
section of weft.toml says how that image is built:
base_image: the image it starts from,debian:trixie-slimunless you name another.dockerfile_template: the path, from the project’s folder, to a Dockerfile weft uses instead of its own. It has to keep the{{...}}placeholders weft’s own Dockerfile uses, and if it drops{{install_runtime_system_packages}}, the cloud refuses every program whose nodes need a program installed.package_manager: how your nodes’ needs are installed on it (apt,dnf,apkorbrew). weft guesses it from the image’s name, so if the name does not say Debian, Ubuntu, Alpine, a Red Hat family distro, Python or Homebrew, set it, or the build is refused. When you set it, it wins over the guess.
If a project sets any of the three, the cloud builds an image just for it,
even when all its nodes come from weft’s standard library. On your machine the section does
nothing, because the worker runs with whatever the machine has. If your
machine is missing something your nodes need, weft deps-install installs
it. If you also deploy the project, leave the section in: the cloud reads
it from this same weft.toml.
A worker gets none of the runtime’s environment except PATH and HOME,
so a variable you exported in your shell (an API key, say) never reaches
your nodes. That PATH is the one your shell had when you last ran
./setup.sh (or weft daemon start). If you install a program into a
folder that was not on it then, run ./setup.sh again for the workers to
find it.
If you want to see what a worker printed, it is in
workers/logs/weft-w-<project id without dashes>-<more>.log in the
install’s folder (~/.local/share/weft for the default install), while the
worker runs. weft deletes that file, and the folder the worker keeps its
temporary files in (workers/scratch/<the same name>), every time it stops
a worker, whatever the reason. A worker that fails
to start has its last lines in the error instead. For one that dies while it
runs, weft notices the next time it calls it or comes to stop it as idle,
and copies its last lines into the runtime’s log (weft daemon logs) under
a worker had ended on its own before weft stopped it. One still running
but not answering is stopped and started again, with its last lines under
a worker stopped answering; starting it again.
If the project has no folder on this machine (none at all, or only on
another machine), its worker does not start, and the error says project <id> has no folder on this machine, and a worker on a local install runs in its project's folder; open the project from its folder on this machine.
On your machine there is one worker per project, program and worker
settings; changing the project’s worker settings (weft workers set) starts
a fresh one for the calls that follow. The one at the project’s address runs for as long
as any of the project’s triggers takes work. Any other starts when a run needs
it and stops after thirty seconds with nothing to do, unless the project keeps
copies warm with weft workers set --min-instances. To change the thirty seconds, set
platform.workerIdleStopSeconds in the install’s config.json
(~/.local/share/weft/config.json) and restart the daemon.
On a cloud install a worker is a Cloud Run service, run from an image
built in the cloud. Cloud Run starts more copies of it when calls pile up and removes idle ones when it decides to,
down to none, so a project nobody calls costs nothing. If you keep copies
warm with --min-instances, those are billed even while idle. If you want to
change how many calls Cloud Run sends one copy at once (--concurrency), the most
copies Cloud Run may start (--max-instances), or a copy’s CPU and memory
(--cpu, --memory), weft workers set changes them for one project. On
your machine none of these four does anything: there is no Cloud Run sending
calls or starting copies, and weft puts no CPU or memory limit on a worker.
If a node of yours talks to something every copy draws on (a
database’s connections), plan for as many copies as --max-instances allows, plus the
old ones during a change: for what that asks of the node, go and read
sharing something between runs.
New worker settings start a new Cloud Run revision. Once it serves, weft hands
it the calls the old settings took and deletes the revisions of the
same program under other settings, so their copies stop and give back what
they held, and each run still on them is handed back the way it is on any stop
(what happens when something dies). Which
settings are the new ones is read from what the project has stored, never
from which revision is newest. A revision of another program stays until weft
forgets that program’s image. If deleting one fails, it takes no calls and its
copies scale to zero once nothing calls them; the next activation or settings
change deletes it, and weft logs an error naming the gcloud run revisions delete command that deletes it now. The one revision Cloud Run never lets
anyone delete is the service’s latest: when a dispatcher copy that still held
the old settings brought them back up after the change, that revision takes no
calls and goes once a later settings change or build replaces it.
On a cloud install, if a run keeps working after its caller has gone (a Route
with outlivesCaller that replies and carries on), the nodes it was running
when the caller left finish on that copy, and the rest of the run moves under
a call weft holds open (for how, go and read how a run is
kept). Cloud Run
throttles the CPU the moment the answer goes out, so if those last nodes are
heavy, turn on weft workers set --cpu-always-allocated true. With it, the
copy is billed for as long as it is up.
A thing a worker’s runs share (ctx.shared, such as a pool of database
connections) lives in the worker too: it goes once no run has used it for
five minutes (weft workers set --shared-idle-seconds changes that), and at
the latest when the worker stops. For how a node asks for one,
go and read the ctx.
A worker keeps, between runs, what its runs read of the program’s infrastructure and connections: where each piece answers, the connections the broker handed it, and the ones an infrastructure node published. That rarely changes between runs, so after the first run a call no longer asks the broker where things answer or for its connections, and starts its run straight away. Whenever one of those changes (a piece restarts somewhere else, a connection is replaced), the broker tells the worker, which drops what it kept and asks again. Two exceptions: a credential weft hands to a single run is never kept, and a credential that expires is kept only until it is due a refresh.
If a run may need more than an hour on a cloud install, go and read how a run is kept.
When a worker is full
A worker takes a set number of calls and events at once. Unless you set it,
that is one for every MiB of the worker’s memory (on your machine, the
machine’s), and never fewer than 64. If you want another number for a project, weft workers set --max-runs-at-once sets it. A run handed to the worker from the queue (a weft run, a resume) is not counted.
If a call arrives while the worker is full, it waits up to 30 seconds
(--max-queue-wait-seconds) for a run to end, before any of its body is read.
At most as many calls can wait as the worker takes at once. A call that waited the whole time, or found no room to wait,
gets a 503 saying This worker is busy right now; try again in a moment.,
with a Retry-After. An event waits the same way, and one that waited too
long goes in its trigger’s queue, where it waits until a worker has room for
it.
If the database falls behind, runs take longer to end (for why, go and read
when a run waits for its writes),
the calls queued behind them wait longer, and one that waits past
--max-queue-wait-seconds gets the same 503.
The same 503 comes back while memory is more than 90% full, because one run
too many would get the whole worker killed, and every run on it with it. On a
cloud install that is the copy’s own memory; on your machine it is the
machine’s, whatever is using it. The runs already on the worker carry on. If
you see it often on a cloud install, lower --max-runs-at-once. Cloud Run
never sends one copy more than --concurrency calls at once (80 unless you
change it), so a lower --max-runs-at-once only helps if it is below that. You
can also lower --concurrency itself, or give each copy more memory with
weft workers set --memory. On your machine, free some memory or lower
--max-runs-at-once.
The dispatcher
It never sees your nodes/ directory. Your CLI reads your local catalog and
compiles the program before submitting it, so the dispatcher only ever handles
a compiled definition.
The next request may land on another copy of the dispatcher, so it keeps nothing in memory that another copy would need.
What it does keep is a copy of the rows a call reads every time and that rarely change: a tenant’s routes, the install’s domains, a project’s worker settings, which of its infrastructure is up, the connections the install picked for it and what each instance provides. Every write to those rows makes Postgres tell every copy of the dispatcher, which drops what it held and reads it again on the next call, and anything it is about to refuse (no such route, infrastructure not running) it checks against the rows first. While its connection that hears those announcements is down, it keeps nothing and reads every time.
The dispatcher’s background work (handing runs to workers, answering what a program asks of weft, registering triggers) waits on rows in the database, and Postgres announces each one as it is written. A dispatcher that is up hears that and starts at once, on your machine and on a cloud install alike. On a cloud install, where the dispatcher can be at zero, whichever part of weft wrote the work also calls it to start one.
If you are watching a run in the editor, or waiting on one to end, you hear about it roughly 10 milliseconds after its rows are written. The broker gathers which runs got rows and which ended, and tells the dispatcher in one statement once the write has committed. It announces after the write rather than inside it, because Postgres makes every write that announces something commit one at a time, so announcing from inside each write would line up every write in the database behind the others.
Some work has no write to announce it. A worker that dies just stops renewing its lease, and only the clock can show that. So the dispatcher works out the earliest moment a lease could run out, and looks again exactly then. On your machine that is a timer in the process. Each piece of background work there also looks on an interval of its own (15 seconds for the one that notices dead workers, 30 or 60 seconds for most others) in case an announcement was lost.
A cloud install has no such timer. The dispatcher writes the moment into the database and books one Cloud Tasks wake for it. When the moment changes, it books the new wake before it deletes the old one, so a crash at the wrong time costs one wasted wake, never a missed one.
On a cloud install, if whoever wrote the work cannot reach the dispatcher,
it books a Cloud Tasks wake to call it instead, which Cloud Tasks retries
for up to ten minutes. Until a call gets through, weft status lists the
dispatcher under waiting on weft:. After ten minutes, the next write for
the dispatcher, or the next part of weft to start, calls it again.
If a piece of background work itself fails on a cloud install, weft books
it again 1 second later, then 5, then 20. If the fourth try
fails too, it stops trying and writes the error to weft status, under
waiting on weft:. It runs again the next time weft writes work for it,
and once more each time a copy of its service starts, and the line goes
once a try works. That last one is for a fresh install, where Google takes
a minute or two to apply the permissions the install was just given: work
that failed in that window runs again the next time something wakes the
service. Each failed try still answers Cloud
Tasks’ call as handled, so the queue does not call it again on its own.
Cloud Tasks only retries a call whose failure could not even be written
down, because the database did not answer. On your machine a failed piece
of background work only logs the error, and runs again at its next
interval or the next write for it.
When a run a worker drives gets an answer or is cancelled, weft announces it
through the install’s news (the weft-news topic on a cloud install), in
batches. If a batch is refused, weft tries it again after half a second, 2
seconds and 5 seconds, then marks it refused, and weft status shows weft could not tell the install's news (a cancel or an answer reaching a run a worker drives); those runs hear of it late under waiting on weft:. Each
time news is written, and when a weft process starts, weft first sends the
new news on its own, so refused news never holds it up, then tries the
refused news again one piece at a time, stopping at the first piece refused
again. The line goes once no refused news is left.
The listener
One listener serves every project on the install. Each kind of trigger says what it needs between two events, and the listener does exactly that:
- A form or a webhook needs nothing: it is called from outside, and the listener only turns the call into a message.
- A schedule, a delay or a poll needs a timer. It asks for its next wake, and the install’s alarm delivers it (a table in Postgres on your machine, Cloud Tasks on a cloud install), even across a restart of the runtime.
- A stream or a socket needs an open connection, so it needs a process that stays up between events. weft calls that process a holder, and it runs the listener’s code. On your machine that is the runtime’s one process. On a cloud install the holders run in a pool, as many as those connections need and none when there are none. Each holder claims the triggers it holds, so no two holders hold the same one. A claim lasts until its holder lets go, so there is nothing to renew: a holder calls weft only when it has something new to say, and Pub/Sub tells it when the triggers to hold change. Since claims never run out, a holder that crashed would keep its triggers for good, so each holder keeps a dead man’s switch: a Cloud Tasks check set three minutes ahead, which it moves forward every minute. A holder that dies stops moving it, the check goes off, and the broker frees everything that holder claimed, for another holder to take. A holder that cannot move its check in time lets go of everything and exits before the check goes off, so its triggers are never handed to another holder while it still holds them. A holder that is shut down gives its triggers up at once. How many holders run follows how busy they are too. Each holder samples its CPU every ten seconds, averages the last minute, and tells weft that number when it has moved 10 points from the one it said last, plus once more a minute after it took triggers or handed some over, since the old number was measured on what it held before. weft adds up the numbers of the holders that hold triggers (a crashed one counts until its dead man’s switch frees them, since that work still needs a holder) and runs as many holders as the sum needs at 50% each, never fewer than one per 200 triggers and never more holders than triggers. It stops none while triggers are on their way from one holder to another, or while a holder that took some has not said its new number yet. A holder whose average reaches 70% hands part of its triggers over, keeping as many as should bring it back to about 50%, and takes no new ones until it says less than that. A holder weft started for the load holds nothing at first, so once it has said its number, the first holder over 50% to notice hands it the triggers that bring it nearest 50%, at least one; the broker gives them to that spare alone, so two holders never pile their work onto one spare.
- An event subscription needs a holder only when it dials out to the service. On a cloud install, when the service can push the subscribed events for one account, weft subscribes with the service instead and renews that on a timer, so nothing stays open for it. A subscription to every account of your app always dials out.
If the wake of a schedule, a delay, a poll or a subscription renewal keeps
failing (the broker or the service does not answer, or the event it picked
up cannot be handed on), the alarm delivers it again, waiting longer each
time. The listener gets 20 tries at it, and a delivery uses one only when it
gets an answer other than a 5xx: one with no answer at all (the listener
down) or a 5xx (from the listener or from something in front of it) uses
none. After 20 failed tries the listener gives the trigger
up and stops waking it, and its node shows its wake failed each of the 20 times the listener handled it, the last with: <error>. If the last one did
what the wake was for but could not set the wake after it, the node shows
its wake was handled, but the wake after it could not be set on the last of the 20 times the listener handled it, so nothing wakes it again: <error>
instead. On your machine the 20 tries take about an hour when the listener
answered every one, and up to about 95 minutes when an outage had already
stretched the wait between tries to its longest, 5 minutes. On a cloud install the count is wake_attempts in
deploy/terraform/gcp/variables.tf. If the broker answers that only a
person can fix the trigger’s connection (it is gone, it needs reconnecting,
or the service refused what was asked, a missing permission for instance:
a 400, 404, 409 or 410), the trigger is given up at once.
A trigger that keeps a connection open (a socket, a stream, an SSE
subscription, a service’s events, a webhook subscription it keeps renewed)
retries on its own when that connection fails, and its live panel shows
reconnecting since <time>: <error> meanwhile. If the failure is one only a
person can fix (the same broker answers, a credential refused right after it
was fetched, or a setup error that would fail the same way every time), it
stops instead: its live panel shows stopped: <why>, and the reason stays on
its node. A service that is busy or limiting how often you call it is still
retried.
If you want a stopped or given-up trigger back, fix what stopped it and
register it again. If it was its connection, reconnecting that connection is
enough: weft brings back every trigger that uses it, the ones that keep a
connection open included. If it was anything else (say, a setup error),
fix the cause and register it again: if weft status then shows
the trigger as behind, weft resync does that; if not, run weft deactivate --trigger <name> and then weft activate --trigger <name>. weft activate
alone does not bring it back, even with --trigger, because the trigger
still counts as on.
A trigger that acts as one of your connections keeps that connection’s credentials for as long as it is registered. If you reconnect the connection, rotate it or remove it, weft registers again each trigger that uses it and that no holder holds, and tells the holders to drop what they kept for it, so nothing goes on with the old credentials. A token refresh does neither: what was kept already says when it stops being good, and is fetched again before then.
If you want a new kind of trigger, you only write listener code: the listener is the only role that tells kinds apart.
The supervisor
It applies your infrastructure and watches whether what it created is healthy. Each piece of infrastructure your program asks for (a database, a model server) is a unit: on your machine a set of Docker containers, on a cloud install a Compute Engine machine of its own, with its disks and GPUs.
Two supervisors changing the same project at once would overwrite each other, so before it changes anything a supervisor takes an exclusive lease on the project. It keeps renewing that lease while the work runs, and if it expires another supervisor picks the project up. When a supervisor takes up projects, it takes the ones with something due now first (a command waiting, a retry whose moment came).
Inside the one supervisor that holds a project, the work on different copies runs side by side: three instances started together come up together. Work on the same copy runs in the order it was asked for, so a stop asked after a start waits for that start.
A supervisor can still die between changing something and recording that it did. The next one works out what to do from what is actually running, rather than trusting the record.
On your machine the supervisor looks at the health of what it runs every
thirty seconds. On a cloud install nothing looks while everything is fine, so
infrastructure that runs fine costs no look at all and lets the rest of the
install sleep: the agent on each machine watches its unit right there, for
free, and asks for a look when how the unit stands changes (a container
stopped, a check started failing or passing again), and Compute Engine’s own
record of a machine stopping or failing asks too. Once something is not fine,
the supervisor’s next look is booked for the exact moment it needs one, such
as when a unit that stopped being ready would count as flaky. The node’s
health rules decide what the supervisor does about an unhealthy unit; with
none written, it takes down the triggers that depend on it and brings
them back once it is healthy again. If a rule’s action fails, it is tried
again after 5 and then 10 seconds. At the third failure the supervisor gives
up on it and weft status lists it under waiting on weft:. It tries again once
the units report a change, once they are all healthy, once you start, stop,
upgrade or terminate the project’s infra, or once you run weft infra look.
The supervisor’s own pass over a project can fail too: it could not look at
the project’s health, could not delete what the host still holds for copies
that are gone, or ran a command and could not write down how it ended. If
it could not list what the host holds (an error starting could not list what the host holds to delete what its gone copies left) or could not read
which projects it owns (could not read what it owns to look at its health), the whole pass fails, and every project it holds counts a failed
pass. weft status shows it under waiting on weft: as weft's supervisor could not get through this project's infra. On your machine the line ends
(tried again at the supervisor's next look), and the supervisor never
gives up: its next look tries the project again. On a cloud install the supervisor tries the project again 5 seconds
after the first failure and 10 seconds after the second, and the line ends
with how many passes failed in a row and when it tries again, for example
(failure 1 in a row; tried again in 5 s). At the third failure the
supervisor stops trying, and the line ends (failed 3 times in a row, so the supervisor stopped trying; `weft infra look` tries again). A pass over
the project that goes through starts the count again from zero, and so does
a weft infra look, which gives the project a fresh round of three tries. The line goes once a pass over the project works.
When a project is removed, the supervisor deletes what the infra host still
holds of it, disks included. A removed project has no status of its own, so
if that fails, every project’s weft status shows it under waiting on weft: as weft's supervisor could not delete what the infra host still holds of removed projects (their disks included). The error names each
removed project, what went wrong, and what happens next. On your machine it
ends (tried again at the supervisor's next look). On a cloud install the
supervisor tries again 5 seconds after the first failure and 10 seconds
after the second, and at the third the error ends (failed 3 times in a row, so the supervisor stopped trying; its next pass tries again). From
then on every pass of the supervisor tries again, and weft infra look in
any project starts one. The line goes once the host holds nothing of any
removed project.
A command (a start, a stop, an upgrade) whose run or whose ending keeps
failing would be taken up again forever, so the supervisor gives it up once
it has taken it up 3 times without it ending. This holds on your machine
too. The command ends failed with the supervisor took this command up 3 times and it never ended (its run, or the write of how it ended, kept failing), so it gave the command up; issue it again, the commands it was
holding back go ahead, and weft status lists it under waiting on weft:
until the ending of a later infra command of the project is written. If you
want it done, fix the cause and issue the command again.
The broker, and who may talk to what
Only the dispatcher and the broker hold a database connection; the listener, the supervisor and every worker go through the broker.
Each of those processes keeps one WebSocket open to the broker, and every call it makes rides that one connection, numbered and answered in whatever order they finish. A call the broker holds until something happens (new journal rows) costs nothing while it waits, so a worker takes one seat at the broker however many calls its runs hold open, and a call whose caller stops waiting is dropped at the broker too. The broker also pushes down the same connection the changes a worker keeps a copy of (see the worker), and, for a run the worker drives, its cancel and every answer handed to it. A connection with nothing to do for a minute closes, and opens again on the next call, so an idle process holds nothing open at the broker, except a worker. A worker driving runs keeps its connection open, to hear their cancels and answers, and an idle one reports to the broker every second (that it is alive, and how many runs it drives), so its connection opens again within a second of closing. On Google Cloud a connection is cut after an hour, the most Cloud Run lets one request last, and the broker says so when it opens. A busy worker opens a second connection once the first is 30 minutes old (or, if it was idle then, as soon as it gets busy), and moves over once the broker answers on it. The first connection keeps delivering until then, so the worker misses nothing and does not have to read the database again to catch up. A worker whose runs all only wait (on a person’s answer, or on the next message from a caller on a socket) opens no new connection, because a broker copy that has not met that worker may read the database to check who it is. Instead, once its connection is within a minute of the cut and its runs only wait, it stops the way it does when Cloud Run stops it. The runs that can pause are handed back for a fresh worker to carry on, the others end with a message naming the limit, and the worker exits, so the next request starts a new one. When the connection breaks, it comes back by itself: a call not yet sent goes out on the new one if it is back within the call’s wait. Past that, or when a sent call got no answer, two kinds of call are made again because doing so is safe: a batch of records (a batch that arrives twice is stored only once), and a read of a run’s history. A call the broker says it could not even start (every one of its database connections busy) is sent again too, since nothing of it ran. Any other call fails the way a request whose connection reset does, since it may have landed. Uploads and downloads of files stay ordinary requests, so a big file never holds up the calls behind it.
The broker checks every request. The caller proves who it is with a token its
platform gave it: on your machine, one the runtime signed when it started the
worker; on a cloud install, one Google signed for the account the caller
runs as, weft’s own or the program’s. The broker works out from that token
whose install the caller belongs to and which role it plays, and it checks
that the caller may touch what it asked for. Your
program’s worker runs node code that anyone may have written, so weft gives it no
database connection, and the broker lets it touch its own project only. On
your own machine that is no wall: the worker runs as you, and can read
whatever you can, the install’s secrets.env included.
The broker also handles storage and connection work, including OAuth exchanges and subscription setup. Your worker calls providers itself, so a slow provider never holds up the broker for everyone else.
The key the install’s API asks for
The API the CLI and the editor talk to (registering projects, building, running, connections, infra) asks every caller for an operator key, on your machine as on a cloud install. It also sends no CORS headers, so a web page open in your browser can neither send it a key nor read what it answers.
On your own machine you never log in, though. When the install is set up it
makes a key of its own, WEFT_INSTALL_KEY in its secrets.env, a
file only your user can read (~/.local/share/weft/secrets.env for the
default install; weft daemon which prints another install’s folder). The
CLI reads it from there whenever it calls an install on this machine,
whatever the target is called, so weft login refuses one: there is
nothing to log in to. The editor gets the
key by asking the CLI (weft target key). If you call the API from a script
of your own, send Authorization: Bearer <that key>; without it, or with a
key the install does not know, the answer is 401 with unauthorized: this install takes an operator key; for a target, `weft login <target>`; for this machine's own install, the CLI sends the key in its secrets.env, which `./setup.sh` writes.
This key lives in secrets.env, not in the install’s database, so weft token ls does not list it and weft token revoke cannot retire it. If you
want a new one, delete its line from secrets.env and run ./setup.sh
again. The keys you mint with weft token mint --operator are rows, and those you can list and revoke.
The addresses other people call check their own credential instead, and answer any origin: a fire URL checks the token inside it, the browser extension sends its signal token, a frontend’s calls carry the frontend’s token, and a route asks for whatever its auth says. For the rest of the boundaries, go and read the security policy.
What happens when something dies
A worker that dies takes its fast runs with it. The exception is a run that
paused: every node still running in it was waiting (on a timer, a form, a bus
or a stream), each saved its state, and its whole record was written before it
let go, so a new worker picks it up when the wait ends. That exception does
not hold for a run that cannot pause (a caller still on the line, or
recorded: false): such a run holds its worker instead. Nobody can
tell which of a fast run’s nodes ran after the last row that reached the
database, and running it again could repeat them, so a fast run whose worker
died ends cancelled and is not run again, with this reason: the worker running this run went away; a fast run lives in its worker's memory, so it is not run again (make its trigger durable for a run that survives its worker).
A lost fast run is not marked cancelled the moment its worker dies: the dispatcher notices and ends it. Every worker reports that it is alive through the broker once a second, and counts as gone 15 seconds after its last report. The dispatcher ends a run once its worker has been gone for another 15 seconds, and it looks again exactly when the first such worker’s extra 15 seconds run out, so a fast run ends about 30 seconds after its worker last reported. Its journal holds what reached the database before the worker died.
A worker whose runs are all only waiting on something outside them (a
person’s answer while the run cannot pause, or the next message of a caller
on a socket) stops reporting, so the database is not woken every second for
a wait that can last hours. Its last report says it is alive until the first
of those waits would give up, plus 15 seconds. A wait on a person gives up
when its holdSecs runs out, unless a caller is on the line. A wait while a
caller is on the line, and a wait for a caller’s next message, have no time
limit of their own, so the worker’s last report covers it until the
run’s hour ends or the route’s or socket’s maxSessionSecs runs out (the
longest one connection may stay open; unset, there is none), whichever
comes first. On your machine a run has no hour limit, so for a route or
socket without maxSessionSecs the worker keeps reporting every second. The
worker reports again as soon as one of its runs moves (that first wait
giving up included) or a new run comes, saying again how long it is alive
for. The answer or the cancel that ends
the wait is written by the dispatcher, which also tells the broker copy
holding the worker’s connection, without going through the database (on
Google Cloud, over Pub/Sub). That broker copy stays up for the length of the
wait, and pushes the answer or the cancel down the worker’s connection, and
only then does the worker read it. Nothing else reads while a run waits: the
worker read its answers once as the run started waiting, and the broker
reads only for the calls a worker makes. So the database sleeps for as long
as the run waits, and wakes for the answer, the cancel, or the run’s end,
or when the connection breaks, after which the worker reads once, for all
its runs together, to catch up. On Google Cloud such a wait lasts at most until the worker stops before
its connection is cut, within the hour (see the
broker). If the worker dies during the wait, its
runs are ended (or carried on, if durable) about 30 seconds after that
first wait would have given up.
In that same check, a durable run whose worker died is queued again, and the dispatcher hands it to another worker at once, which carries it on from its record. A node that had started but not finished is failed on the new worker rather than run again, apart from a node that saved a state, which runs again from its last save, and some pure nodes. For which ones, and what to do about a failed node, go and read surviving a restart.
On a cloud install, when Cloud Run replaces a worker or scales one away, it
sends the worker SIGTERM, the signal that asks a process to stop on its own,
and kills it 10 seconds later. Every run on the worker then leaves it if it
can:
- A run that can pause starts no new node. Once each node it is running has
ended or is waiting inside the run (on a bus, a stream or a yield), it
pauses: every waiting node saves its state, its record is written whole,
and it is handed back, so another worker carries it on straight away, buses
included. A node still running 5 seconds after the
SIGTERMis stopped where it is, and the next worker treats it as if the worker had died: it runs again from its last save if it saved a state, and is failed otherwise. - Two kinds of run cannot pause: an unrecorded run, and a run whose caller is
still on the line, whatever its route’s
outlivesCallersays. Each keeps running, starting new nodes as usual, until the platform stops the process. If it has not ended by then, that is the same as its worker dying: a fast run ends cancelled, and a durable one is carried on by another worker. An unrecorded run that nothing had written down yet leaves no trace.
At the same moment the worker stops accepting connections. A call that was
already in, but whose run had not started yet, gets a 503 saying this copy of the program is stopping; try again in a moment, so the client can retry
and reach another copy.
If you restart the daemon on your own machine, the new daemon sends the same
SIGTERM to every worker the old one left running and forgets each once it
has exited, so their runs go the same way. When the new daemon sets up a
project’s own address again, it kills that project’s old workers 10 seconds
after sending them SIGTERM, so the new worker can open its port.
If you would rather not wait for an old worker to exit, kill -9 <pid>
stops it now, and the runs still on it end as if it had died; the daemon’s
log (~/.local/share/weft/runtime.log) names each such worker with its pid.
If a worker cannot get a batch of records written, the runs in that batch end failed, except a durable run whose batch got no answer at all, which is handed back to another worker (for both, go and read when a write of the record fails).
If a worker freezes or loses its connection to the broker, it stops reporting, and its runs go the way a dead worker’s do: about 30 seconds later a durable run moves to another worker and a fast run ends. The run’s row carries a number that goes up every time a worker takes the run or loses it, and the broker refuses any write made under another number. So from then on the frozen worker’s writes to that run are refused, and it drops the run.
Running it somewhere new
If you want weft on another platform, write a crate that implements the traits below, from
crates/weft-platform-traits: everything that depends on where weft
runs sits behind one of them.
| Interface | What it answers |
|---|---|
Runner | How a program’s workers are started and reached |
InfraHost | Where infrastructure units run, and how they are watched |
ImageBuilder | How staged source becomes a worker (a program on your machine, an image on a cloud install) or an infrastructure image |
MachineNeeds | Which of the things a program’s nodes need are missing on the machine its workers run on, and what the refusal tells the person to do about them |
Alarm | How a wake set for later is delivered |
CallerIdentity | Who is calling an internal endpoint |
IdentityTokens | How a role proves who it is to another |
FrontendHosting | Where a project’s frontend runs, when the install hosts it |
DomainHosting | What stands in front of the install’s own domains and holds their certificates |
HolderPool | How many holders run, for the triggers that keep a connection open |
Files go through one more trait, ObjectStore. If your platform’s store
speaks S3 and you can hand it a key pair, you implement nothing: the S3
client (object_store_for) talks to it, the way it talks to SeaweedFS on your
machine. GCP has its own (GcsObjectStore), because it reaches Cloud Storage
as the runtime’s own service account and has Google sign the links, so no key
exists for an organization’s policy to forbid.
Two platforms exist: weft-platform-local (your machine: workers as
processes; infrastructure, the database and the object store in Docker) and
weft-platform-gcp (Cloud Run, Compute Engine, Cloud Build, Cloud Tasks). The
install’s config names one, and crates/weft-runtime/src/platform.rs is the
one place that reads which.
Things people say to me
“Why a new language? Just make it a library”
For the actual work, weft is a Rust framework. You write a node’s logic in Rust with the APIs weft gives you, and I am not going to invent a new syntax for parsing a response or adding up a total.
The language has one job: saying how the pieces fit together. For that job the syntax is the whole point.
Think about where the architecture of a normal codebase is written down. Nowhere. It is something you reconstruct by reading implementations and working upwards, and that is fine while the whole thing fits in your head.
It stops being fine for a coding assistant, which is working in a window. It goes down into one detail, does a good job on the piece in front of it, and has no idea what shape it was supposed to plug into. So it writes a second version of something that already exists, or a path around the infrastructure it should have used. And if you do not guide it, it writes with no structure at the top at all, because nothing in the language asks for one.
In weft the structure is the source. Scoping a part of the program nests it, so the nesting is free, and a group declares its inputs and outputs before you open it. You read from the top down: collapse everything, see the whole shape, open the one part you are working on. The compiler holds those boundaries, so a wire cannot quietly reach into another group’s insides. It has to appear in the interface, where somebody can see it.
That is what you hand an assistant: the outer structure from the contracts, then one group, one level deeper, with edges around the job. See groups and the commandments of plumbing.
“Won’t better models make this unnecessary?”
A model getting better at untangling a codebase is not a reason to keep handing it a tangled one.
Even if tomorrow’s model held your whole project in its head and never missed a connection, you would still be paying it to read all of it. If a tenth of the context gives it everything the job needs, why buy the other nine tenths? Being smarter does not make those tokens free, or faster to produce.
And you do not have to hand the project to one assistant at all. Agree the contracts, let one arrange the groups, have others work inside them at the same time, and a group can divide again. Weft is built for that kind of parallel work.
“Visual programming always turns into spaghetti”
It absolutely can. Two hundred boxes on a canvas do not become understandable because you can zoom out far enough to see them all.
The difference is that the nesting is not an editor feature, it is in the language. A group is a real boundary with declared inputs and outputs, so folding it away actually removes something from what you have to think about.
You still have to pick useful boundaries, and no editor rescues a design where everything needs to know about everything else. What the compiler does is tell you when a level is getting crowded: past fifteen items on one level it says so, by name.
The graph also has a text form. The .weft file you review in git is the
source, and the editor edits that file rather than a separate model of it.
“The compiler checks the wiring, not whether the program is right”
Correct. A perfectly well typed program can send a beautifully formatted wrong answer to the wrong person.
What the checks catch today is incompatible port types, missing required inputs and cycles in the wiring. That is feedback about how the pieces got connected, which is exactly the kind of mistake an assistant makes and can fix without you. It says nothing about whether a node does the right work. That is still your judgement, and the node’s own tests.
Where I want to take it is proving things about how the program behaves, at compilation time: letting a node carry labels about what it is and what it does, then having the compiler check claims across the whole graph, so that an autonomous node cannot reach a high-stakes action without a strong enough verification in front of it. The full argument, and why I think it matters, is in how I think about AI safety.
If something here looks wrong, or you have other arguments, come and tell me on Discord.
Sequential Diffusion Programming
If you already know exactly what you want, ask your assistant for the whole thing and see how far it gets. Sometimes that is the end of it.
This page is for when it goes wrong in one particular way: the program looks convincing, and it keeps getting real cases wrong.
The reflex then is to write a longer, more careful description of what you meant. That rarely helps. Give it one real case instead.
We call building this way Sequential Diffusion Programming. The program gets sharper on every pass, and every pass has an actual input going in and an answer you can judge coming out.
Start with one real input
Say you are turning incoming emails into support tickets. Take an email you actually received. Ask Tangle to pull the customer’s problem out of it, run just that piece, and click Inspect run on the node that produced the answer.
Now read it properly, before you build anything else. Did it find the problem, or did it summarise the signature at the bottom? If there was an order number in there, is it still there?
Whatever went wrong, hand that back:
This email is asking for a replacement, but the extracted problem says the customer wants a refund. Improve the prompt for this node.
Now you have a real extracted request to test the next node against, rather than something you made up. Keep going until you can follow that first email all the way to the answer you wanted.
You can do the same to a finished draft somebody handed you. Start at its output, find what is wrong, and walk backwards to the node that introduced it.
Keep the cases that taught you something
Try a second email that asks for something different. When it breaks something, fix it, then run the first one again. Otherwise the assistant can make today’s example pass by quietly breaking yesterday’s.
Find the mistake without reading everything
If the program has groups, start with them shut. Look at the output of the group that produced the bad answer, then open it and look at the boxes inside. Keep going until you reach the node where a good input became a bad output.
The verbs a pass uses
| If you want to | Run |
|---|---|
| Re-run without redoing what did not change | weft run --seed. Changed code and inputs invalidate the affected work and everything downstream of it |
| Exercise one node with values you supply | weft run --from node='{"port":value}' |
| Run up to a node, including it | weft run --target node |
| Run up to a node, excluding it | weft run --before node |
| Run one group on its own | weft run --group group='{"port":value}' |
| Try a trigger without switching the listening on | weft bake, then weft run --fire trigger='<json>' |
When a run comes out right, weft freeze <name> keeps its starting parameters
and its accepted outputs. After a change, weft run <name> runs the current code
with those parameters, and weft diff <run-id> example:<name> shows you what
moved, for you or Tangle to judge. Freezing the new run replaces the accepted
example.
For the full rules and every refusal, go and read versions, seeds and frozen examples. What is landing next is on the roadmap.
The commandments of plumbing
A node does its own job and nothing else. Everything underneath it, the credentials, the listening, the storage, the record of what happened, belongs to weft.
Here is where that line sits today, and what it means in practice.
The test underneath all of them: if every node that needs a mechanism would need the same answer, the mechanism belongs in weft. If two good nodes would do it completely differently, it belongs in your program.
I. Thou shalt not handle credentials.
Signing in, signing requests, refreshing, revoking, deciding whose account to use. You get handed a client that already works.
II. Thou shalt not listen for anything.
A trigger says what should wake it up and the runtime does the listening: holding the socket open, taking the push, polling on a timer, checking the event is real, renewing the subscription.
III. Thou shalt not build the channel.
Two nodes alive at the same time talk over a channel weft opens and holds for them. Sockets, reconnects, acks and backpressure are already written.
IV. Thou shalt not build control flow in Rust.
Looping, branching, retrying, fanning out, gathering results: you decide all of it, in the graph, where it is journaled and somebody can read it. Not buried in one node’s body where nobody can.
V. Thou shalt not save your own state.
No file on the side, no table of your own, no “I’ll just keep this in memory
between runs” for anything a run has to remember. The journal is the run’s
memory: a durable run whose worker dies is rebuilt from it by another worker,
and what a node keeps across a pause (ctx.keep) is written there too. A connection pool or a process the runs only share for
speed is different: ctx.shared holds one for them, and losing it loses nothing.
VI. Thou shalt not carry files around.
You say what a file is for and how long it should stick around, and you get all of that control. Fetching it, storing it, giving it a public address, expiring it, turning it into whatever shape a provider wants: that happens underneath you.
VII. Thou shalt not keep the books.
A call on a metered connection gets its cost recorded as it happens, as far as the provider’s evidence allows. You never total anything up, carry a running cost between nodes, or work out whose spend a call was.
VIII. Thou shalt not run infrastructure.
Containers, health checks, volumes, lifecycle. You describe what should be running and the runtime keeps it that way.
IX. Thou shalt not police your inputs.
No checking that an input is really a string, no unwrapping something you were already promised, no defensive parse at the top of your body. Every wire was checked before anything ran, so the value is the thing it says it is. A wrong one fails before it reaches your code.
X. Thou shalt not write down what happened.
No log file, no print so you can work out later what went on, no audit trail of
your own. Every event of a run is journaled (unless its trigger turned
recorded off), and that journal is what
the runtime reads to pick up a durable run or one that waited, so a record you
keep on the side is one the runtime cannot restore from anyway.
XI. Thou shalt not report to the human.
Every value on every wire is already in the inspector, live and afterwards. If a
firing makes something worth looking at, declare a display and the editor
renders it on the node. You never hand-deliver information to whoever is
watching.
These are also aspirational. A commandment can be right and weft can still not cover your case yet, or be missing an option you need. If you think the line should move, come and tell us about that one on Discord.
What is left is yours
Your own logic: building this call’s request body, reading this reply, knowing what this provider’s errors mean, doing the actual work.
It is a small job on purpose. Everything hard sits behind the ctx, and when something back there goes wrong, it fails loudly.
Moving the line
The line sits where it does because of the nodes people have written so far.
If you are about to write something you think is plumbing and belongs in weft, come and say so on Discord or in an issue. Bring the node you are building and the code you would otherwise have to stuff in its body. Or build the general version yourself and send a PR, which is usually faster and always welcome; ask on Discord first so you do not waste your time.
Same if a commandment holds but what weft gives you falls short: socket handling that does not fit your protocol, a display that cannot show your kind of result, a scope that does not last long enough. Tell us what you hit.
Design principles
The rules we use when deciding whether something belongs in weft.
Put the coordination where you can read it
Whatever decides what connects to what, or what runs next, should be something you can read.
In most systems that logic is buried in code you never open: a retry loop, a
queue, an if three calls deep. In weft it is written down in the graph and
checked before the run, so you can find where a decision is made and change it
without touching the nodes around it.
You read it from the outside in. A group’s interface says what goes in and what comes out, and its boundary is real to the compiler, so nothing reaches inside without going through it. A group does not exist at run time, so nesting is free: it costs nothing when the program runs, and nothing when you read it.
The plumbing belongs to weft
A node’s code should be its own job and nothing else. Handling a credential, keeping a subscription alive, saving state, writing down what happened: those belong to the runtime, where they are written once and hardened for every node. When two nodes would otherwise write the same thing, that thing gets built once for both.
For where the line sits today, go and read the commandments of plumbing.
The language knows nothing about your nodes
The compiler, the dispatcher and the runtime never mention a node by name or hardcode its fields. A Postgres node and a model node look the same to the language, and everything a node needs, it asks for through the ctx.
So adding a node that needs something new means building a general mechanism in weft, never a branch that already knows that node.
Refuse a mistake as soon as anything can see it
Put each check at the earliest place with enough information to make it, and have it say what went wrong and what to do about it. For the full rule, go and read the three fronts.
Fail loudly
When something goes wrong, it says so. A fallback that quietly returns a second-best value turns a broken program into one that looks like it works, and the person running it never finds out.
The failure is part of the design too: name what broke, and what the person can do next.
The three fronts
When a feature lands in weft, it lands on three fronts at once. Miss one and the feature is half built, usually in a way nobody notices until somebody is stuck.
Levers
What can a person reach, and from where?
A setting on a node, a flag on the CLI, a control in the graph, something the
node’s Rust asks the ctx for, a key in metadata.json, a word in the language
itself. Name the lever and name where it lives.
A knob nobody can turn is not a lever, and a knob nobody needs is clutter.
Defaults
What does everybody get without asking? This is the part a person should never have to know exists.
Two questions decide it. Is it tedious and useful to make somebody fill this in every time? Is there a number or a choice that is right for almost everybody? Yes to both and you set it, with the lever there for whoever wants to change it.
Never invent a default to paper over a setting that genuinely depends on the case. If there is an honest reason to leave it unset, leaving it unset stays possible and explicit, and the validation is what makes the bad shape impossible. A hidden default that is right half the time is worse than a refusal that says what is missing.
Protection
What stops somebody building the broken version by accident?
The compiler, the parser, and a node’s own validation rules know things before anything runs. If a shape is provably wrong, refuse it at the earliest point that can see it, and say what to do instead.
The obvious ways to hurt yourself, a loop with no way out, a run left waiting forever on somebody who is gone, should be impossible to write down in the first place. Where something can only be caught while it runs, the runtime stops it and the message points at the shape that works.
How I think about AI safety
There’s something fundamental about LLMs that most people get wrong: the thing you talk to isn’t the model.
The model predicts probabilities for the next token. A sampling algorithm picks which one actually happens. That token goes back into the context, and the model predicts again. Nobody wrote the assistant down anywhere. The model offers the possible continuations, the sampler picks one, and after enough picks you’re talking to a character.
So if you want a different assistant, you edit some text, start again, and what emerges is different. The character lives mostly in the context, and you can open the context and read it.
That still works, but it’s getting harder, because more and more of the assumptions about what’s being predicted get pulled out of the context and into the weights. Getting a model to actually be the character you handed it takes real effort now, and more of it every year. Those models are still fundamentally simulators though, even if their internal dynamics have changed. For what I think about training, go read Three properties for alignment.
What I’m worried about
I worry about an AI that stays self-coherent over long periods of time. That’s the future I’m trying to make less probable.
There are two ways I see us getting there.
The first is the one most labs are probably aiming at: a single model with either an infinite context or a very compressed memory, where everything that defines its behaviour has moved inside the weights, or the memory, and is no longer in the context. Models are drifting that way. I’d argue that as long as next-token prediction of arbitrary text is still the base and a major chunk of the training, it stays possible to make another persona emerge by tweaking the context hard enough, and we wouldn’t be in this scenario.
The second is a mind emerging at a meta level, through high-bandwidth coordination and high context adherence. Labs would train AIs to be extremely good at coordinating with each other, and at storing and retrieving information through a well organised shared memory they can edit. With the current paradigm, I think this path could arrive earlier than the first one.
Yes, you could use weft for the second one
weft is powerful enough to build that level of coordination. But:
- You’d still need AIs trained to act as a singular mind through context alone. weft doesn’t get you there, and I’d argue that’s the hardest part.
- weft doesn’t make it much easier anyway. You still have to shape the memory layer properly, and weft helps with that no more than any other language.
What weft actually changes
Every step you take out of the model and write into the graph makes the system more interpretable. Everything a run did is in the journal, and you read it against a graph you already have in front of you, so what would have been a trajectory somebody has to reconstruct is values on wires you drew. The model inside any one node is as opaque as it ever was. What changes is how much of the system is made of model.
Push that all the way and you get something like a CoEm, where the capability comes from a structure you wrote rather than from one unbounded thing. Most programs will sit short of that and I think that is fine. A node can stay agentic wherever nothing irreversible happens off the back of it alone: an implementation pass over a plan you already settled, a literature review, …
The graph decides what runs next, not the model. In an agent the model decides, so anything you want to be true about a run is something you are hoping it keeps doing. Here it is written down, and the compiler checked it before anything started.
A node also only gets what you wired to it, so anything you put around it sits outside it: a filter on its answer, a second model that checks it, a fact-checker that starts from a clean context, a person, …
The node can’t see any of that and can’t reach it. It doesn’t know whether it is being checked, so it spends no tokens on that, and the check runs because the graph says so rather than because the model chose to.
The checker also starts clean. It isn’t carrying the context that produced the answer, which is usually how a hallucination or a jailbreak gets past a checker that shares it.
Make the safe path the cheaper one, and make it arrive earlier
If you pull the coordination out of the AI’s job and write it down, you get something explicit, controllable, interpretable and cheaper to run, with the same economic benefit as the autonomous version.
My bet is that it’s what makes the scary future less likely. The autonomous version costs more and gets harder to justify building, without anyone having to ban it. And policy people get something they can point to, which is a much easier job than arguing for a full stop.
It’s also why weft can’t be a tool for safety people only. A tool nobody else picks up doesn’t change what gets built, so it has to be worth using simply because it’s a good way to build things.
Where I want to take the compiler
Today the compiler checks how the pieces connect: types, required inputs, graph shape. What I want next is for it to prove things about how a program behaves at compilation time, and I think that’s reachable through richer metadata.
The idea is to let a node carry labels about what it is and what it does. A node that takes an action in the world says so, and says how big that action is. A node that’s an autonomous agent says so. A node that verifies something says what strength of verification it gives you.
Then the compiler can check claims across the whole graph: every path from an autonomous node to a high-stakes action goes through a verification of at least this strength, or a critical action has three separate human approvals in front of it. If the program doesn’t hold up, it doesn’t build.
None of that works if the labels are decoration. So the other half is vetted sets of nodes, checked and signed off by whoever is doing the regulating, with safety-critical programs required to build out of those. Then the labels mean something, and so does everything the compiler proves on top of them.
That’s the piece that turns all of the above from a way of working into a property of the program. It doesn’t exist yet but it’s high in the roadmap.
None of it is necessary for the rest of the safety vision to hold, though. If the language only makes the two dangerous shapes less likely on its own, I’d already call it a win. Where it does become useful is if anyone wants to regulate AI properly, because I think regulation as compilation properties is one of the better shapes AI regulation could take.
Glossary
_error. The output every node has for a failure of its own work (a
service refusing, a script raising, a wait that gave up). If it is wired, that
failure goes down it as text and the node’s other outputs close; if it is not,
the failure fails the run. A mistake in the program, like a bad setting, fails
the run either way. A node that already sent something on a Bus or
Generator output fails the run either way.
Access. The port type for permission to call a service, written with the
service it is for: Access[slack]. One access node holds the connection and
emits one; every node that calls that service takes one as an input. Nothing
secret is in it.
Access node. The node that owns the connection for one service. Its body is one line; everything it does is declared in its service recipe.
Activation. Turning a trigger on. It runs the trigger’s setup, which
registers subscriptions with providers and dials sockets, then leaves the
listener running. Each trigger is turned on separately, and a trigger that
exists once per instance is turned on separately for each instance. The command is
weft activate, which turns on what is off and leaves what is on alone.
Asset. A file pulled into your project at build time with @asset. Never
written back to.
Bake. A trigger’s settings, prepared and saved without anything listening.
The command is weft bake, and it is what you want while building.
Baked output. An output of an infrastructure node that weft makes when the
infrastructure is applied and then saves. A run that reads nothing else from
the node uses the saved value instead of running it. weft infra rebake <node> makes it again. Unrelated to weft bake.
Behind. A trigger that is on but fires an older version of what it runs,
or an infra copy running from older source than you have now. weft resync
brings a trigger up to date, and weft infra upgrade does the same for infra.
Broken part. A piece of the program that does not compile: a file that does not parse, an include that does not resolve, a node type neither the catalog nor your project has, a wire whose types do not fit. A command refuses only when what it runs needs a broken part, and names it; otherwise it goes on and prints the broken part as a warning.
Broker. The scoped front door to the database that every tenant-side process uses. Workers, listeners and supervisors ask it; only it and the dispatcher touch Postgres.
Bus. A live channel between nodes that are running at the same time. Any number of participants, each with its own position. It lives exactly as long as its worker.
Closure. A pulse carrying no value, meaning nothing will ever arrive here
for this run. It is how a node says no rather than saying nothing. Not null,
which is data.
Conflict copy. A file the sync writes beside one of yours that changed
both in your folder and in the install: tone.weft-conflict-1a2b3c4d.txt
beside tone.txt. It is never compiled and never in a version; delete it
once you have taken what you want from it.
Connection. One account somebody connected to a service. Its credential lives in weft’s store, sealed, and never in your program.
Cut. The part of the program a run actually runs: the nodes the walk keeps and the wires between them, built as a smaller program. Every run has one, and so does each trigger, from the trigger down to what it reaches and back up for what that needs.
Dispatcher. The control plane. It routes events, decides what runs where, and answers every request about a project or a run.
Door (connection). How a connection is obtained. shared uses a credential
weft holds; own means you bring your own.
Door (trigger). What a trigger that is down does with what arrives:
take holds it, hide holds it without listing it for the people who
answer your program’s questions, refuse takes nothing. You pick one with --door on weft deactivate, weft resync or the
weft infra commands that stop infra; the default is take.
Drift. Your source having moved ahead of what is running. weft status
names which kind and which verb fixes it.
Durable run. A run whose trigger has durable on: before each node
that is not pure starts, everything the run did so far is written down, so if its worker dies
another one picks it up.
Example. A run’s starting parameters, saved as examples/<name>.json. A
frozen example also holds the outputs you accept as right.
External link. A link to a stored file for something outside the install
to open: a browser, an app, a provider. It says whether the open internet can
open it (public).
Fast run. A run whose trigger has durable off, the default: it goes as
fast as the program and its record follows behind it. If its worker dies, it
ends cancelled and is not run again.
Firing. One go at one node, identified by its run, its node and its frames.
Frames. The stack of loop iteration numbers a firing sits inside. weft only combines inputs whose run and frames match, which is what stops iteration three eating iteration four’s answer.
Gather port. A loop output that collects one value per iteration. It has to
be List[T | Null], because a failed iteration leaves an empty slot.
Generator. A stream: one producer, one consumer, ordered, typed, and it ends. A closure on one is its end rather than a skip.
Group. Several nodes under one name, with declared inputs and outputs. No node stands for it when the program runs: it is a scope that opens or skips as one.
head. Where one working copy is: the version its next version is recorded under,
and the run a --seed from it inherits from. Your folder has its own.
Health rules. Rules in an infra node’s InfraSpec saying when to take down
the triggers that need it, because one of its containers stopped being ready,
and when to bring them back. With none
written, while a trigger of the project is on and a container has stayed not
ready past its flaky window, the node takes the triggers that need it down, and
brings them back once it is healthy.
Held. The work a trigger kept while it was down: the arrivals its door took,
and the runs it fired that are paused on a wait. weft activate runs it,
paced so new arrivals still get through, or drops it with --held drop.
Held back. A trigger the health check or an infra command took down that
cannot come back by itself, because other triggers are on another build,
and only one build at a time answers a project’s routes. weft resync --all brings it back.
Infra node. A node that needs a container of its own. Nothing starts it for you.
Instance. One separate running copy of part of a program, under an id the
program picks. It can stand for one person, one session, or one of several
things a person owns; which person owns which instances is kept in the
program’s own database. A node marked @per_instance gets one copy per
instance, and every run is for one instance or for none. See
programs with instances.
Instance token. A signal token that acts inside one instance of one program and can do nothing else: it starts that instance’s runs, answers its waits, shows its displays and connects its accounts. It always expires.
Internal link. A link to a stored file for code inside the install, which goes straight to storage.
Join. A node that meets several branches and goes on with what arrived,
like FirstInOrder. Its inputs are optional and it needs no gate.
Journal. The append-only record of a run, written in rows that each hold one or more of its events. It is what the graph shows you, and what a worker reads to pick up a durable run or one that was waiting.
Keep. Giving a stored file a deadline: it is deleted once nobody has
touched it for that long, or with its run or project if that comes first. The
KeepFile node also copies a file into its project, so it outlives its run.
Left out. A node folder weft could not load (bad metadata, a name two
folders claim), which weft describe-nodes lists with why, or a node package
whose Rust does not compile, which the command that built it prints as a
warning. The worker is built
without it, a run that needs it is refused naming its first error and its build
log, and it is built again once its files change.
Listener. The tier that holds the timers, the sockets and the subscriptions. The only tier that tells one kind of event source from another.
Meter. The code that works out what one provider’s call really cost. A node never states a cost.
Metadata port. A port whose value is written in the program and never
computed by a run: a group writes it in its body (self.tool = { ... }), a
node type in its metadata.json. The graph shows a group’s in the group’s
Config strip, and a node type’s nowhere: neither is a port on the box.
Metadata wire. A wire leaving a metadata port. It never runs: the compiler writes the value into the input it points at before the run starts, so it may point backwards without making a cycle. The graph does not draw it: the input it fills shows the value and where it comes from.
Narrowable output. An output whose catalog type is broad (say JsonDict) and
which a program may declare narrower where it uses the node:
-> (response: Verdict).
Network address. This machine’s IP on its local network, set with
WEFT_NETWORK_ADDRESS. The paths outside callers use answer there too, so a
phone on the same wifi can open what a program hands out.
Node. One box in a program’s graph. On disk, a folder with a metadata.json and a
mod.rs.
Open record. A record type ending in .., like { email: String, .. }. It
accepts keys it does not declare, and drops them from the value that flows on.
Pulse. One emission travelling to one input, carrying a value, a run id and a frame stack. The only thing that moves in a running program.
Recipe. The service block in an access node’s metadata: how a credential
is obtained, how a request is signed, what the permissions are, how events
arrive.
Registered app. One OAuth application this installation signs people in with, living in the operator’s apps file. Its secret never leaves the store.
Resync. Building again and moving the shared triggers that are on and
behind onto the new build (--instance or --all-instances for instances’
triggers): each is taken down, its runs on the old version finish (or are
cancelled, if you asked for that), and it comes back on the new one. A trigger that is off stays off.
What you run after editing anything a live trigger reads.
Root. A node no wire feeds. A manual run starts every root at the top level, plus every trigger.
Run. One go of a program, under a run id given when it starts. Running again is a new run with a new id.
Scope (run time). A group, an included file, or one loop iteration, as the runtime sees it: each copy of it opens or skips as one.
Scope (storage). Which of run, project, shared, asset or instance a file belongs to. It decides where new files go and how long they live.
Seed. The run a --seed run inherits from: head’s run, or the run you name.
Sequential Diffusion Programming. Building a program stage by stage against a real case, rather than describing the whole thing and hoping.
Signal. Something that wakes a node: a timer, a form, an endpoint, a subscription, a held socket. Registered by a trigger, or awaited mid-flow.
Slug. The stable name on a compiler finding, like type-mismatch. The slug
names the rule; the message names the fix.
Snapshot. What a command reads of your project’s files, once, at its start, and builds from. A file you edit while it runs is not in the build, and the command ends by naming the files that changed after it.
Stuck. A run where nodes are holding values that will never add up to enough to fire them, and nothing is parked on a signal. Nobody can answer, so weft ends it and names every node involved.
Supervisor. The tier that creates and watches your infrastructure’s containers. One holds an exclusive lease per project.
Tag. A label a run puts on itself, which another run can use to stop it.
Tangle. The weft specialist weft new --assistant copies into your
project.
Trigger. A node that starts a run from outside. Two bodies: setup, which runs once at activation, and run, which fires on each event.
Unit. One set of containers inside an infrastructure spec, running together on one machine. Most nodes have exactly one.
Upload slot. A place in storage a program gives a web page, so a visitor’s big file goes straight there, in parts that resume after a dropped connection. The page then sends the slot to a route instead of the file.
Version. Your project’s files at one moment, recorded whenever you do
something with the code (a run, a build, an activate, weft checkpoint) and
never on a save. Recording the same files again gives the version they
already are.
Visitor address. The address where a project’s own site passes its
visitors’ calls on to weft (weft frontend visitor-address). Links to the project’s stored files
then go through the site.
Waiting for input. A run parked on a person or a service. The worker shuts down and costs nothing, and a fresh one picks the run up when the answer lands.
Walk. How weft works out a run’s cut: down along the wires from where the
run starts, then back up from where it ends, stopping at a node you started from (--from), at a
trigger nobody fired, and at a node with no inputs.
Worker. A process (on your machine, in the project’s folder) or a Cloud Run service (on a cloud install) running your compiled program. For how it starts and stops, go and read the worker.
Working copy. The install’s copy of one place a project’s code is edited: your folder (the install keeps the two the same in both directions), or the program copy, which programs write into when the project has no folder. Each has its own head.
Where this is going
Held suspensions. Today the durable model kills a worker whenever every lane
parks on a wait, and a fresh worker folds the journal to resume. That is what
makes parking thousands of cheap flows free. It fails a node that holds
in-process state too expensive to rebuild: a browser session with thousands of
cookies, a loaded local model. The plan is an opt-in
primitive, ctx.hold_signal, where the await happens in place and the worker
stays alive. Die-and-resume stays the default, and the language makes the cost
visible, because holding keeps a worker up for the whole wait.
Should a recorded: false run be able to pause? Today it cannot, so its
waits hold in its worker. It could: such a run writes nothing while it works,
but at a pause everything it holds in memory is plain data, so it could write
that once, as one snapshot, and nothing else. That needs the snapshot’s
format, a place to keep it, a resume that starts from it and then reads what
was written after it (the answer that woke the run), the run’s secrets and
the costs it has not settled carried inside it, and cancelling and
weft node-state taught to read it. Whether that is
worth it is still open.
Type rules in metadata. FirstInOrder wants an output whose type metadata
cannot write down as one type: the union of its created inputs. Today every
input it creates shares one type variable, so it cannot join a String branch
with a Number one. The plan is a
small rule language in metadata.json, read the way portsFromConfig is, for
“the union of these inputs”. It is the whole feature or nothing, because a half rule would
send the editor, the validator and the runtime three different answers about one
port.
More the compiler can prove
Today the compiler checks how the pieces connect: types, required inputs, graph shape. The ambition is for it to prove things about how a program behaves, through richer node metadata. A node that acts in the world says so and says how big the action is. A node that verifies says what strength it gives. An autonomous node says so. Then the compiler can check a claim across the graph: every path from an autonomous node to a high-stakes action passes through a verification of at least this strength, or a critical action has three separate human approvals in front of it, and a program that does not hold up does not build. None of it works if the labels are decoration, so the other half is vetted sets of nodes, signed off by whoever is doing the regulating, with safety-critical programs required to build out of those. Read How I think about AI safety for why this matters.
Running it for real
Other clouds. Weft installs on GCP today (deploying to your cloud). AWS and Azure come next, installed on your own account the same way.
The builders
Tangle’s own CLI. Tangle today is a set of instructions on top of whatever coding assistant you use. A proof-of-concept CLI wrote a working program from a brief in under five minutes. The released version is slower because existing assistants spend time on fluff weft does not need, and our own CLI fixes that.
Coordinated assistants in parallel. Several agents building branches of the weft codebase under one plan, the same shape weft gives a program.
The editor
Branch lanes in the version tree. The Runs view can already list runs by version (Flat list / by version on its title bar): every version as a tree, its runs under it, the head of each working copy marked, and Load this version on a version or a run. The plan is to draw it the way a git client does, with a lane per branch, and to pick a run’s seed from the same view.