Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Storage

A running node reads and writes files through ctx.storage. Every write takes a scope, and the scope decides where the file lives and how long it lives. Pick it by how long you need the file to survive.

The scopes

ScopeLives underLifetime
StorageScope::Executionexec/<run>/one run, unless flagged kept
StorageScope::Projectproject/<project_id>/as long as the project
StorageScope::Shared { name }shared/<name>/as long as the owner
StorageScope::Assetthe project’s @asset filesread-only; a worker write to it is refused

Execution is the default, and it is for scratch: intermediate files, temporary downloads, anything nothing cares about once the run ends. It sweeps itself up.

Project is for a project’s own persistent state: a cache, an index, accumulated outputs. It outlives individual runs and is shared across the project’s executions, and it is deleted when the project is deleted.

Shared { name } is tied to the owner rather than any project. It survives runs and project deletion both. Projects naming the same name meet in the same space, and first use auto-grants it.

So if you want a file to survive deleting the project, that is the scope argument and nothing else: Project means no, Shared means yes. Changing that one argument is the whole knob.

Reach for Shared when it is a dataset the owner reuses across projects, a model they paid to build, or anything they would be upset to lose while tidying up.

The verbs

let storage = ctx.storage(StorageScope::Project);

storage.put(bytes, mime, filename, keep).await?;
storage.put_stream(stream, mime, filename, keep).await?;
storage.put_response(resp, what, mime, filename, keep).await?;  // straight from an HTTP response
storage.put_from_url(url, filename, keep).await?;               // the runtime fetches it
storage.identified("whatsapp:m1").put_from_url(url, None, None).await?; // once per identity, see below

storage.get(&handle).await?;
storage.get_bytes(&handle).await?;
storage.get_range(&handle, range).await?;

storage.delete(&handle).await?;
storage.list().await?;

storage.keep(&handle, KeepTtl::Default).await?;
storage.presign(&handle, ttl_secs).await?;      // a temporary link, always fetchable from your body
storage.public_link(&handle, ttl_secs).await?;  // an internet-reachable link, or None

A stored file arriving on one of your inputs already carries a url inside its marker, minted for this firing (an hour): the runtime links every file input before your body runs, so a body that hands the value to something that only fetches URLs needs no call of its own. The link is the internet-reachable one when the install serves one (a public address, or a bucket declared public), so a provider can fetch it too. Otherwise it is signed for the cluster’s own address: your body can fetch it, nothing outside can, and a node that hands a file to something outside asks public_link and inlines the bytes when it answers None. It is stripped from everything you emit, park, or memoize, so the stored form is what travels and the journal never holds a link.

Scope governs writes and lists. Key-addressed verbs act on the key’s own scope, so reading a handle works regardless of which scope you asked for.

If you pull a thing by a stable id (a message, a document at a provider), name it: .identified("<service>:<id>") before the put. The same identity in the same scope is then one file, however many runs ask for it: a put_from_url asks the store first and fetches nothing when the file is there, and two runs fetching at once cannot both land (the second sees a conflict and retries). Pair it with Project scope so the copy outlives the run that first pulled it. The identity is a label, scoped to the scope you put in; choose one that names the source, never the content.

The keep rule

An Execution-scoped file your node emits must be kept.

Every Execution write takes keep: Option<KeepTtl>. None means the file is swept shortly after the run ends.

That is right for scratch and wrong for anything you pulse downstream, because an emitted reference lands in the journal and renders in the editor long after the run, where a swept file shows up as “media expired”. So:

  • A node producing a user-facing artifact (a generated image, synthesized speech, received media) passes Some(KeepTtl::Default). That is 30 days, and every access bumps the clock, so artifacts still in use never expire while abandoned ones age out.
  • A node whose file is cheaply re-fetchable, such as a plain download, may expose a keep boolean config input defaulting to off, and pass keep.then_some(KeepTtl::Default), letting the user decide.

The KeepFile node extends or pins an execution file’s lifetime after the fact. Kept or not, an execution file stays walled to its run; with scope: project the node instead copies it into the project scope and emits the copy, the only form a later run can read.

Files from the graph

A user-supplied file arrives as an ordinary typed input.

"inputs": [
  { "name": "image", "type": "Image", "required": true }
]

The type drives the editor’s file filter and is what gets written into source; a "widget": { "kind": "file_drop", "accept": "image/png" } narrows it further. Your node reads it with ctx.inputs.get::<FileHandle>("image")?.

What lands in source is one clean line:

send = TelegramSendMedia {
  file: @asset("assets/photo.png", Image)
}

The asset sync runs before every build and uploads what the code references. Current source files stay; replaced or removed uploads expire after 30 days without access. Your node never sees any of it: at run time the value is a normal media value, and get and get_bytes read its bytes whichever handle it carries.

Media in typed values

For converting whole typed values at a provider boundary, see Custom types.

The marker stays inside weft

The __weft_image__ / __weft_audio__ / __weft_blob__ wrapper is how a file travels between nodes: it carries the storage key the runtime reads by. Anything you hand to something that is not weft (a provider’s request body, a bridge’s action payload, a form spec a browser renders, a live item) gets the plain thing that consumer reads: a URL string, a data: URL, or a plain { url, mimeType, filename } object. externalize does this for a typed value, public_link and presign for one file. Wrapping a link in a marker and sending it out puts weft’s internal shape in an external contract, and the consumer, which reads value.url, shows nothing. The form image field did exactly that once, and the tasks app rendered “(no image)” over a link that worked. A link you hand out also has a life, so never store one: a form parks the stored file itself, and the person who opens it gets a link minted at that moment through the signal-token files door, however long the form waited.

Reaching files outside the editor

The stored files are addressable independently of the editor:

weft files ls
weft files inspect <key>
weft files download <key>
weft files rm <key>
weft files usage

So data a project wrote in Shared stays reachable after the project is gone.