# Build a personal automation runner (replacing IFTTT) You are building a self-hosted automation runner for one person or a small team. It replaces IFTTT for people who run a handful of "when this happens, do that" applets against services they already have API keys or webhooks for. Build the whole thing end to end. Reliability of the queue and correctness of the scheduler matter more than the number of integrations. ## Stack - One Node 22 app: Next.js (App Router) with TypeScript and Tailwind for the UI and HTTP endpoints, run as one long-lived process. - A second process, `worker/index.ts`, because this tool genuinely needs one: it owns the scheduler, the job queue, and the inbound SMTP receiver. Same repo, shared code in `lib/`. `pnpm dev` starts both. - SQLite: `better-sqlite3` with Drizzle, WAL mode, file at `./data/app.db`. Migrations checked into the repo and run on startup. Keep the schema portable so `DATABASE_URL=postgres://...` works later, but don't build for it now. - Local disk for files: notes under `./data/notes/`, email attachments under `./data/files/events//`, served through an authenticated route with short-lived signed URLs the app mints itself. No bucket. - In-process scheduler: `croner` inside the worker for the trigger tick, retention, and the heartbeat. A row in a `job_locks` table stops two instances from running the same job. - Sessions the app issues itself: on first run the app creates an admin from `ADMIN_EMAIL` and `ADMIN_PASSWORD`, or prompts in the browser if they're unset. Signed, httpOnly cookies. This is a single-user tool, so skip invites. - Realtime from the process: server-sent events fed by an in-memory event bus. No database change feeds, no websocket service. - Email through any SMTP server: `nodemailer` with `SMTP_URL`. Without it, mail is written to `./data/outbox/*.eml` and printed to the log, so every flow works on a laptop with no account. - Inbound email through `smtp-server` running inside the worker, parsed with `mailparser`. Port 25 behind one MX record in production, port 2525 on a laptop. - `isolated-vm` for the filter sandbox. If it won't build on the target platform, fall back to `quickjs-emscripten` behind the same interface in `lib/sandbox.ts`. - `rss-parser` for RSS and Atom, `googleapis` for the optional Sheets action, `zod` for every config schema. - UTC in the database, local time in the UI, `date-fns` and `date-fns-tz` for every conversion. Never do timezone math by hand. - Outside services, all optional: an SMTP relay via `SMTP_URL` (mail sent straight from a VPS lands in spam; the outbox covers dev), Slack and Discord incoming webhooks, ntfy.sh or Pushover (URLs and tokens only), a Google service account for the Sheets action only, and one MX record for inbound mail. Full profile: https://buildyourown.software/deploy.md ## Running it - **Laptop:** `pnpm install && pnpm dev`. That's the whole install. No env vars are required; every setting has a default that works locally. `SESSION_SECRET` and `SECRETS_KEY` are generated on first run and saved under `./data/` when unset. The SMTP receiver listens on 2525. - **Public:** a VPS with 1 CPU and 1 GB RAM, one A record, one MX record, and `docker compose up -d`. The compose file has `app`, `worker`, and `caddy` services plus `backup`. Caddy terminates TLS with a certificate it gets on its own. `./data` is a named volume shared by `app` and `worker`. Publish host port 25 to the worker's 2525 so the process never runs as root. - **Backups:** `pnpm backup` runs `sqlite3 .backup` into `./data/backups/.db` and prunes to the last 14. The `backup` service in compose runs it nightly. Restoring is copying one file back to `./data/app.db`. Notes and attachments live next to it under `./data`, so snapshot the whole volume when you want the files too. - **Updates:** `git pull && docker compose up -d --build`. Migrations run on boot. ## Auth Single admin. On first run, if `ADMIN_EMAIL` and `ADMIN_PASSWORD` are set, create the admin from them. Otherwise the first browser visit shows a setup page that asks for an email and password and creates the admin; the page stops existing once a user row exists. Hash passwords with argon2. A login page sets a signed, httpOnly cookie (HMAC with `SESSION_SECRET`, 30 day expiry, `SameSite=Lax`). Every page and every `/api/*` route requires the session except `/api/hooks/*` (per-applet tokens in the URL), `/api/files/*` (signed URLs), and `/api/health`. No invites; there's one user. ## Data model SQLite types only: `text`, `integer`, `real`. Booleans are integers 0 or 1. Timestamps are ISO 8601 UTC strings (`2026-03-07T13:00:00.000Z`) in `text` columns; they sort and compare correctly as strings because the format is fixed. JSON is `text` with Drizzle's `{ mode: "json" }`. If you'd reach for `timestamptz` or `jsonb` here, don't: there are no JSON indexes and no timezone-aware columns, and every `now` is `new Date().toISOString()`. All tables have `id` (text, uuid), `created_at`, `updated_at`. - `users`: `email` (unique), `password_hash`. One row. - `applets`: `name`, `enabled` (integer, default 1), `trigger_type` (`webhook` | `schedule` | `rss` | `email` | `poll`), `definition` (json, see below), `webhook_token` (32 random bytes base64url, unique), `last_fired_at` (nullable), `run_count` (integer, default 0). - `trigger_state`: `applet_id` (pk, FK), `next_run_at` (nullable), `last_polled_at` (nullable), `seen_keys` (json array, capped at 500, newest last), `last_value` (json, nullable), `etag` (nullable), `last_modified` (nullable), `last_error` (nullable). - `events`: `applet_id`, `source` (same enum as trigger_type plus `manual`), `dedupe_key` (nullable), `payload` (json), `received_at`. Unique index on (`applet_id`, `dedupe_key`) where `dedupe_key` is not null. - `runs`: `applet_id`, `event_id`, `status` (`queued` | `running` | `succeeded` | `failed` | `skipped` | `dead`), `attempt` (integer, default 1), `max_attempts` (integer, default 5), `next_attempt_at`, `locked_by` (nullable), `locked_at` (nullable), `started_at`, `finished_at`, `error` (nullable text), `steps` (json array). Index on (`status`, `next_attempt_at`) and on (`applet_id`, `created_at`). - `secrets`: `name` (unique, `[A-Z0-9_]+`), `value_encrypted` (AES-256-GCM with `SECRETS_KEY`). - `job_locks`: `name` (text, pk), `locked_by` (nullable), `locked_at` (nullable), `expires_at` (nullable), `last_run_at` (nullable). One row per named job: `scheduler_tick`, `retention`, `worker_heartbeat`. Take a lock with one statement, `UPDATE job_locks SET locked_by = ?, locked_at = ?, expires_at = ? WHERE name = ? AND (locked_by IS NULL OR expires_at < ?)`. Zero rows changed means another instance has it; skip this tick. ### Applet definition JSON Store the whole applet in `definition` so it can be exported to a file and imported back. Shape: ```json { "version": 1, "trigger": { "type": "rss", "config": { "url": "https://example.com/feed.xml", "interval_min": 15 } }, "filter": { "code": "if (trigger.title.includes('[draft]')) skip('draft');" }, "transform": { "model": "openai/gpt-4o-mini", "prompt": "Summarize in one sentence: {{trigger.content}}", "output": "summary" }, "actions": [ { "type": "slack", "config": { "webhook_url": "{{secrets.SLACK_WEBHOOK}}", "text": "New post: {{trigger.title}} {{trigger.link}}" } }, { "type": "note", "config": { "folder": "posts", "filename": "{{trigger.published_at | date:'yyyy-MM-dd'}}-{{trigger.title | slug}}.md", "content": "# {{trigger.title}}\n\n{{ai.summary}}" } } ], "retry": { "max_attempts": 5 } } ``` `filter` and `transform` are optional. `actions` has at least one entry. Validate the whole thing with zod on save and on import, with a per-type config schema for every trigger and action. Reject unknown keys. ## Templates (`lib/template.ts`, pure, fully unit-tested) Every string in an action config is a template. Syntax: - `{{path}}` where path is dot and bracket notation into the context: `{{trigger.body.user.name}}`, `{{trigger.items[0].title}}`. - Filters with a pipe: `{{trigger.title | upper}}`. Chain them: `{{trigger.content | strip_html | truncate:140}}`. - Filters to implement: `upper`, `lower`, `trim`, `truncate:N` (adds `…`), `default:'text'`, `json` (JSON.stringify), `strip_html`, `slug` (lowercase, dashes, ASCII only, max 80 chars), `date:'format'` (format an ISO string or epoch ms with date-fns using the applet's `timezone`, default UTC), `urlencode`. - The context has: `trigger` (ingredients from the event payload), `ai` (transform output), `vars` (values set by the filter), `secrets` (resolved lazily), `applet` (`id`, `name`), `run` (`id`, `attempt`), `now` (ISO UTC), `now_local` (ISO in applet timezone). - A missing path renders as an empty string and adds a warning `"unresolved: trigger.foo"` to the step's `warnings` array. It never throws. - A path that resolves to an object or array renders as JSON. - Escaping: `\{{` renders a literal `{{`. - `renderConfig(config, context)` walks an action config recursively, renders every string leaf, and returns `{ rendered, warnings, secretsUsed }`. Secret values used anywhere are replaced with `[redacted]` in whatever gets stored in `runs.steps`. ## Triggers Every trigger produces an event `payload` with the ingredients below. Put each trigger in `lib/triggers/.ts` exporting `configSchema`, `ingredients` (names and descriptions for the UI picker), `sample` (a realistic sample payload for testing), and either `handleRequest` (webhook), `handleMessage` (email), or `poll` (schedule, rss, poll). ### 1. `webhook` - `POST /api/hooks//`. Also accept `GET` with query params so simple services can hit it. - Config: `secret_header` (optional, e.g. `X-Hub-Signature-256`), `hmac_secret` (optional). When both are set, verify an HMAC-SHA256 of the raw body and return 401 on mismatch. - Parse the body by content type: JSON, `application/x-www-form-urlencoded`, `multipart/form-data` (fields only), otherwise raw text in `body.raw`. - Ingredients: `body` (parsed), `headers` (lowercased keys, drop `authorization` and `cookie`), `query`, `method`, `received_at`. - Wrong applet id or token: 404 with an empty body. Applet disabled: 200 with `{ "queued": false, "reason": "disabled" }`. - Success: insert the event and a queued run, publish `run.queued` on the event bus, return 202 `{ "run_id": "...", "event_id": "..." }`. Do no other work in the request. Response time under 50ms. ### 2. `schedule` - Config: `cron` (5-field cron, validated by croner), `timezone` (IANA string, required). - Ingredients: `fired_at` (ISO UTC), `fired_at_local` (ISO with offset in the applet timezone), `date` (`yyyy-MM-dd` local), `time` (`HH:mm` local), `weekday` (`Monday`), `timezone`. - On save or enable, compute `trigger_state.next_run_at` from `now` using croner with `{ timezone }`. - Scheduler tick: select applets with `next_run_at <= now`. For each, insert one event with `dedupe_key = next_run_at` (so a double tick can't double fire), then set `next_run_at` to the next occurrence after `now`. Do not advance from the missed slot. That means downtime yields one catch-up fire and no backlog. - DST: `0 2 * * *` in `America/New_York` on the spring-forward day is skipped or fired at 03:00 per croner's behavior. Document which in the README and write a test that pins it. ### 3. `rss` - Config: `url`, `interval_min` (integer, minimum 5, default 15), `fire_on_first_poll` (bool, default false), `max_items_per_poll` (default 20). - Poll with `If-None-Match` and `If-Modified-Since` from trigger_state. A 304 updates `last_polled_at` and stops. - Item key: `guid`, else `id`, else `link`, else a SHA-256 of `title + published`. Items whose key is in `seen_keys` are ignored. - First poll ever (empty `seen_keys`): store all keys, fire nothing unless `fire_on_first_poll`. - New items fire oldest first, one event each, `dedupe_key = item key`. - Ingredients: `title`, `link`, `content` (full content or description), `summary` (first 300 chars of content with HTML stripped), `author`, `published_at` (ISO, null if the feed has none), `categories` (array), `feed_title`, `feed_url`. - A fetch or parse error sets `trigger_state.last_error`, shows on the applet page, and does not create a run. After 3 consecutive errors, back the interval off to 60 minutes until a success. ### 4. `email` - The worker runs an `smtp-server` receiver on `SMTP_INBOUND_PORT` (default 2525). In production, compose publishes host port 25 to it and you create one MX record for `INBOUND_DOMAIN` pointing at the VPS. On a laptop, deliver test mail to `localhost:2525` with nodemailer or `swaks`. Nothing outside is needed. - Accept `RCPT TO` only for addresses at `INBOUND_DOMAIN` (any local part). Reject every other recipient with 550 so the box is never a relay. Cap messages at 25MB with 552. No auth. STARTTLS is optional; skip it in v1. - Parse with `mailparser` and normalize to: `from` (address), `from_name`, `to` (array of addresses), `cc`, `subject`, `text`, `html`, `date` (ISO UTC), `message_id` (generate one from a hash of the raw message when the header is missing), `attachments` (array of `{ filename, content_type, size, url }`). - Save attachment contents under `./data/files/events//`, up to 5MB each and 20MB per message; larger ones are listed with `url: null`. `url` is a signed link to `GET /api/files/events//?exp=&sig=` (HMAC with `SESSION_SECRET`, valid 24 hours) so an action can pass it along. The retention job deletes the folder with the event. - Applet config: `to_alias` (optional; matches `@` prefix of any `to` address), `from_contains` (optional substring), `subject_regex` (optional). All set rules must match. An applet with no rules matches every inbound email. - One email can match several applets; each gets its own event with `dedupe_key = message_id`. - No match: accept the message with 250, write a line to the worker log, store nothing. Never reject at DATA for a routing miss; the sender would get a bounce. - Ingredients: all the normalized fields above, plus `text_first_line`. ### 5. `poll` - Config: `url`, `method` (`GET` | `POST`, default GET), `headers` (object, values may be templates so secrets work), `body` (optional), `interval_min` (integer, minimum 1, default 5), `path` (dot/bracket path into the JSON response, e.g. `data.price` or `items[0].status`), `condition` (`changed` | `equals` | `not_equals` | `gt` | `lt` | `contains`, default `changed`), `target` (value for the non-`changed` conditions). - Fetch with a 15 second timeout. Non-2xx or non-JSON sets `last_error` and stops. - Extract the value at `path`. Compare with `trigger_state.last_value` using deep equality for `changed`, and numeric comparison for `gt` and `lt` (coerce strings that parse as numbers). - First poll: store `last_value`, fire nothing. - `changed` fires when the value differs from the stored one. `equals` and the others fire when the condition becomes true and was false on the previous poll. Treat it as an edge, so a price that stays under the target fires once and then stays quiet until it goes back over and drops again. - Ingredients: `value`, `previous_value`, `url`, `path`, `changed_at`, `response` (the full JSON body, truncated to 64KB), `status`. ## Actions Each action lives in `lib/actions/.ts` exporting `configSchema`, `fields` (for the UI form: name, label, kind `text | textarea | select | secret | json`, help text), and `run(renderedConfig, ctx) => Promise<{ output, retryable? }>`. Throw `ActionError(message, { retryable })` on failure. - `http`: `url`, `method`, `headers`, `body`, `content_type` (default `application/json`). 20 second timeout. Store `status`, `headers`, `body` (truncated to 64KB) as output. 5xx, 429, and network errors are retryable; other 4xx are not. - `email`: `to`, `subject`, `body_markdown`. Render markdown to HTML and send with nodemailer from `EMAIL_FROM`. With `SMTP_URL` set, use that transport; SMTP 421, 450, 451, 452 and network errors are retryable, other 5xx replies are not. Without `SMTP_URL`, write the message to `./data/outbox/-.eml`, print the recipient and subject to the log, and succeed with the file path as output. Set `Message-ID` from `event_id` plus the action index so a retry through a relay that dedupes doesn't send twice. - `slack`: `webhook_url`, `text`, `blocks` (optional JSON). Post to the incoming webhook. Slack returns `ok` as plain text; treat anything else as failure. - `discord`: `webhook_url`, `content`, `username` (optional). Discord caps `content` at 2000 characters: split on line breaks into chunks and send them in order, 1 second apart. 429 with `retry_after` is retryable. - `sheets`: `spreadsheet_id`, `sheet_name`, `columns` (array of templates, one per cell). Append with `spreadsheets.values.append`, `valueInputOption: USER_ENTERED`, `insertDataOption: INSERT_ROWS`. Auth with a service account from `GOOGLE_SERVICE_ACCOUNT_JSON`. This is the one action that needs a Google account, because Sheets has no other API; without the env var the action fails with a clear "not configured" error and the editor hides the card. The README tells the user to share the sheet with the service account email. Output: the updated range. - `note`: `folder` (relative to `NOTES_ROOT`), `filename`, `content`, `mode` (`create` | `append`, default `create`). Resolve the final path and refuse anything outside `NOTES_ROOT` (check with `path.resolve` and a prefix test; reject `..`, absolute paths, and null bytes). Sanitize the filename to `[a-zA-Z0-9._ -]`. `create` with an existing file appends ` (2)`, ` (3)` before the extension. Create the folder if missing. Output: the absolute path written. - `push`: `provider` (`ntfy` | `pushover`). ntfy: `server` (default `https://ntfy.sh`), `topic`, `title`, `message`, `priority` (1 to 5), `click` (URL), `tags` (comma list), optional `token`. Pushover: `user_key`, `app_token`, `title`, `message`, `priority` (-2 to 2), `url`. Both truncate `message` to the provider limit and record it as a warning. Both are plain HTTPS POSTs with a token; a self-hosted ntfy server works with no account. ## Filter code (`lib/sandbox.ts`) - Run the applet's `filter.code` in an isolate with a 1 second CPU timeout and a 32MB memory limit. No `require`, no `fetch`, no `process`, no timers. - Globals inside: `trigger` (a deep copy of the ingredients), `applet` (`id`, `name`), `vars` (empty object), `skip(reason?)` which throws a sentinel that ends the run as `skipped`, `set(key, value)` which writes into `vars`, `console.log` which appends to the step's `logs` array (capped at 100 lines, 1KB each), and `Date`. - Return `{ vars, logs, skipped, reason }`. `vars` must survive structured clone; drop functions. - Any thrown error, syntax error, or timeout fails the run with `status = failed`, `error` = the message and line number, and no retry. ## AI transform (optional, `lib/transform.ts`) - Config: `model`, `prompt` (a template), `output` (key name, default `output`), `json` (bool, default false). - Call an OpenAI-compatible chat completions endpoint at `OPENROUTER_BASE_URL` (default `https://openrouter.ai/api/v1`) with `OPENROUTER_API_KEY`, 30 second timeout, `temperature: 0`. This is the one step that needs a model; point the base URL at a local Ollama to run it with no account. When `json` is true, ask for JSON in the system prompt and parse it; on parse failure store the raw text under `ai.` and add a warning. - The result goes into `ctx.ai[output]`. Retryable on 429 and 5xx, once, before the run's normal retry policy applies. - Skip the step entirely and warn if `OPENROUTER_API_KEY` is not set. Build this last. ## Worker (`worker/index.ts`) One process. Every periodic job takes its row in `job_locks` first, so running two workers by accident, or restarting one before the old one has exited, never doubles work. Lock TTL is 60 seconds for the tick and 10 minutes for retention; renew while running, release on finish. A lock left behind by a killed process expires and the next tick takes it over. **Scheduler tick**, a `croner` job every 15 seconds under the `scheduler_tick` lock: 1. Select enabled applets with trigger_type in (`schedule`, `rss`, `poll`) and `next_run_at <= now` (or null). 2. For each, run the trigger's `poll`, which may insert events and runs, then set `next_run_at` = now + interval (or the next cron occurrence). 3. Errors in one applet's poll never stop the loop. Log them to `trigger_state.last_error`. **Executor loop**, concurrency `WORKER_CONCURRENCY` (default 4): 1. Claim a run in one statement: update the oldest `queued` run whose `next_attempt_at <= now` and whose applet has no other run in `running`, setting `status = running`, `locked_by = `, `locked_at = now`, `started_at = now`. Returning nothing means idle; sleep 500ms. The row update is the lock; `job_locks` isn't involved per run. 2. Load the applet and event. Build the context. 3. Steps, each appended to `runs.steps` as `{ kind, name, input, output, warnings, logs, started_at, duration_ms, error }`: `filter` (if present), `transform` (if present), then one `action` step per action in order. 4. Action failure: stop. If the error is retryable and `attempt < max_attempts`, set `status = queued`, `attempt += 1`, `next_attempt_at = now + backoff` where backoff is 30s, 2m, 10m, 1h, 6h for attempts 2 through 6. Otherwise `failed` (non-retryable) or `dead` (retries exhausted). 5. Retries re-run the whole step list from the beginning. Earlier successful actions run again; document this and make actions idempotent where the provider allows it (the email `Message-ID` rule above). 6. Stale locks: a run in `running` with `locked_at` older than 10 minutes goes back to `queued` with the same attempt number. 7. Update `applets.run_count` and `last_fired_at` when a run finishes. 8. Retention: delete runs, events, and their attachment folders older than `RUN_RETENTION_DAYS` (default 30) once an hour under the `retention` lock. **Heartbeat:** every 15 seconds write `last_run_at` on the `worker_heartbeat` row. **SMTP receiver:** started at boot on `SMTP_INBOUND_PORT`, stopped on SIGTERM after in-flight messages finish. **Crash safety:** a run is a row before it's anything else, so killing the process mid-step loses nothing. The stale lock rule requeues it with the same attempt, the event's dedupe key stops a trigger from inserting a twin, and `dedupe_key = next_run_at` stops a double fire when two ticks overlap. Test it with `kill -9` on the worker while an `http` action is waiting on a slow URL. ## HTTP endpoints All under `/api`, JSON in and out, session cookie required unless noted. - `POST /setup` (only while no user exists), `POST /login`, `POST /logout`. No session. - `POST /hooks/:appletId/:token` and `GET /hooks/:appletId/:token`: webhook trigger. Token auth, no session. - `GET /files/events/:eventId/:filename?exp=&sig=`: attachment download. Signature auth, no session. 403 on a bad or expired signature. - `GET /applets`, `POST /applets`, `GET /applets/:id`, `PUT /applets/:id`, `DELETE /applets/:id`, `POST /applets/:id/enable`, `POST /applets/:id/disable`. - `POST /applets/:id/test`: body is a payload; creates a `manual` event and a queued run; returns the run id. - `POST /applets/preview`: body is `{ definition, sample }`; returns every action config rendered against the sample with warnings. Does not save or run anything. - `POST /applets/filter-test`: body is `{ code, sample }`; runs the sandbox and returns `{ vars, logs, skipped, reason, error }`. - `GET /runs?applet_id=&status=&from=&to=&cursor=`: paginated, 50 per page, newest first. - `GET /runs/:id`, `POST /runs/:id/retry`, `POST /runs/:id/replay`. - `GET /stream`: server-sent events. The web process keeps an in-memory bus and publishes `run.queued` when a webhook or test event inserts a run. Because the worker is a separate process, a 2 second `croner` job in the web process reads runs with `updated_at` newer than its last check and publishes `run.updated` for each. Events carry `{ run_id, applet_id, status }`. Support `Last-Event-ID` on reconnect. - `GET /secrets` (names and created_at only), `POST /secrets`, `DELETE /secrets/:name`. - `GET /export` (all applets as a JSON array), `POST /import` (body is that array, `?overwrite=true` to replace existing ids). - `GET /health`: returns `{ ok, worker_seen_at }` where `worker_seen_at` is `job_locks.last_run_at` for `worker_heartbeat`. Returns 503 when it's older than 60 seconds so a monitor can tell you the worker died. No session required. Deleting an applet deletes its events, runs, attachments, and trigger state. Disabling keeps everything and stops the scheduler from polling it; queued runs for a disabled applet still finish. ## Environment variables Document every one of these in `.env.example` with a comment. None is required on a laptop. - `ADMIN_EMAIL`, `ADMIN_PASSWORD` (the setup page covers them when unset), `SESSION_SECRET`, `SECRETS_KEY` (32 bytes base64; both generated into `./data/` on first run when unset), `DATA_DIR` (default `./data`), `DATABASE_URL` (default `file:./data/app.db`) - `PUBLIC_URL` (default `http://localhost:3000`; used to build webhook URLs shown in the UI), `INBOUND_DOMAIN` (default `localhost`), `SMTP_INBOUND_PORT` (default 2525) - `SMTP_URL` (outbox when unset), `EMAIL_FROM` (default `automations@localhost`) - `GOOGLE_SERVICE_ACCOUNT_JSON` (the whole JSON, base64 encoded; Sheets action only) - `NOTES_ROOT` (default `./data/notes`) - `NTFY_SERVER`, `NTFY_TOKEN`, `PUSHOVER_APP_TOKEN`, `PUSHOVER_USER_KEY` (defaults the push action can fall back to) - `OPENROUTER_API_KEY`, `OPENROUTER_BASE_URL` - `WORKER_CONCURRENCY` (default 4), `RUN_RETENTION_DAYS` (default 30), `LOG_LEVEL`, `ALLOW_PRIVATE_URLS` ## Limits and edge cases - Cap every stored payload and response body at 64KB. Store `truncated: true` on the event or step when you cut it. - Cap `runs.steps` console logs at 100 lines. Cap `seen_keys` at 500 and drop the oldest. - The webhook route must accept bodies up to 1MB and reject larger ones with 413 before parsing. - A poll or RSS URL must be `http` or `https`. Refuse `file:`, `localhost`, `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, and link-local addresses on save, and again at fetch time after DNS resolution, so a template can't be used to reach the worker's own network. Same rule for the `http` action URL, with an `ALLOW_PRIVATE_URLS=true` env escape hatch for people running everything on a LAN. - Redirects: follow at most 3, re-check the private address rule on each hop. - All timestamps are stored as ISO 8601 UTC strings. Only the `date` template filter and the schedule trigger ever touch a timezone, and both take an IANA name. - Empty template result for a required action field (for example a Slack `text` that renders to an empty string) fails the action with a clear message instead of posting a blank message. - Two events with the same `dedupe_key` for the same applet: the second insert is ignored silently and no run is created. Log it at debug level. - If the worker crashes mid-step, the stale lock rule requeues the run and the retry starts from the first step. Say so in the run detail page when `attempt > 1`. - The SMTP receiver handles at most 10 concurrent connections and closes idle ones after 30 seconds. Attachment filenames go through the same sanitizer as the `note` action. ## Screens Left nav: Applets, Runs, Settings. 0. **Setup (`/setup`) and Login (`/login`)**: setup shows only while no user exists and creates the admin. Login is email and password. Both are plain forms. 1. **Applets (`/`)**: table with name, trigger and action icons, enabled toggle, last run status and time, runs in the last 24h, and a "New applet" button. 2. **New applet (`/applets/new`)**: two steps on one page. - Step 1: pick a trigger from cards (Webhook, Schedule, RSS, Email, Poll), then its config form. Schedule has a cron field with a plain-English preview of the next 3 fire times in the selected timezone. Webhook shows the URL after save. - The cron field accepts presets from a dropdown (every hour, every day at 9:00, weekdays at 9:00, every Monday at 8:00, every 15 minutes) that fill in the expression. - The RSS and poll forms have a "Fetch now" button that runs the trigger's fetch without saving and shows the first three items or the extracted value, so the user can confirm the URL and path before saving. - The email form shows the full inbound address the alias maps to, using `INBOUND_DOMAIN`, and on a laptop a one-line hint with the `localhost:2525` delivery command. - Step 2: pick an action from cards, then its config form. Every text field has an "Insert ingredient" dropdown listing the trigger's ingredients (plus `ai.*` and `vars.*` when those steps exist) and inserts `{{trigger.name}}` at the cursor. A "Preview" button renders every field against the trigger's `sample` payload and shows the result with unresolved paths highlighted. - The `note` action form shows the resolved absolute path preview for the sample. The `sheets` form has an "Add column" button that appends a template field. The `push` form switches its fields when the provider changes. - Below step 2: "Add filter code" (code editor with a "Test filter" button that runs it against the sample and shows logs, vars, and whether it skipped), "Add AI transform", and "Add another action". - Save validates with zod and shows field-level errors. 3. **Applet detail (`/applets/[id]`)**: the definition rendered as forms (editable), the webhook URL with a copy button, `last_error` if any, "Send test event" (a JSON textarea prefilled with the sample that creates a `manual` event and a run), "Export JSON", and the last 50 runs. 4. **Runs (`/runs`)**: global list with filters for applet, status, and date range. Rows update live from `/api/stream`; fall back to a 5 second refresh if the stream drops. 5. **Run detail (`/runs/[id]`)**: the event payload, then each step as a card with rendered input, output, warnings, console logs, duration, and the error. Buttons: "Retry" (re-queue this run with `attempt += 1`) and "Replay" (create a new run from the same event). 6. **Settings (`/settings`)**: secrets (add, rotate, delete; values never shown after save), inbound mail (the address pattern, the MX record to create, and whether the receiver is listening per the heartbeat), outbound mail (whether `SMTP_URL` is set, and the last 20 files in `./data/outbox` when it isn't), Google service account email (parsed from the JSON) and a "test" button that reads the sheet title, default ntfy server and Pushover keys, `NOTES_ROOT` display, and "Export all applets" / "Import applets" (a JSON file; import keeps ids and webhook tokens, and asks before overwriting an existing id). Keep the UI plain. Server components for lists, client components only for the editor and the live runs table. ## Non-goals Do not build OAuth connections to third-party services, a mobile app, location or smart-home triggers, user accounts beyond the single admin, billing, or a marketplace of shared applets. No hosted services beyond an SMTP relay via `SMTP_URL`, Slack and Discord incoming webhooks, ntfy.sh or Pushover, a Google service account for the Sheets action, one MX record for inbound mail, and an optional OpenAI-compatible endpoint for the AI transform. Leave a TODO comment if tempted. ## Acceptance criteria 1. `POST /api/hooks//` with a JSON body returns 202 with a run id in under 50ms, and the run's event payload contains `body`, `headers`, and `query`. A wrong token returns 404 and creates nothing. 2. `renderTemplate` resolves dot and bracket paths, applies chained filters, renders missing paths as empty strings with a warning, and escapes `\{{`. 3. A schedule applet with `0 9 * * *` in `America/New_York` fires at 09:00 local on a day before and a day after the March DST change (13:00 and 14:00 UTC). After 3 hours of simulated downtime spanning one fire time, it fires exactly once on restart. 4. An RSS applet's first poll creates no runs. A second poll with two new items creates two runs, oldest first. A third poll with the same items creates none. A 304 response creates none and updates `last_polled_at`. 5. A poll applet with `condition = changed` fires only when the value at `path` differs from the stored one, and the event has both `value` and `previous_value`. With `condition = lt` and `target = 100`, values 120, 90, 80, 110, 70 fire twice (at 90 and at 70). 6. A message delivered over SMTP to the receiver on port 2525 whose recipient matches an applet's `to_alias` creates a run with `from`, `subject`, and `text` populated. A message matching no applet gets 250 and creates nothing. A recipient outside `INBOUND_DOMAIN` gets 550 at `RCPT TO`. The same `message_id` delivered twice creates one event. 7. Filter code that calls `skip('x')` ends the run as `skipped` with no action step. Filter code that throws marks the run `failed` with the message. `while (true) {}` fails within 2 seconds and the worker keeps processing other runs. `fetch` and `require` are undefined inside the sandbox. 8. An `http` action against a URL returning 500 is retried with the documented backoff and goes `dead` after 5 attempts. A URL returning 400 fails on the first attempt with no retry. 9. Two runs for the same applet execute one after the other; runs for different applets execute concurrently (assert overlap with timestamps). 10. The `sheets` action calls `values.append` with the rendered columns in order and `USER_ENTERED` (mocked client). 11. The `note` action writes the file under `NOTES_ROOT`, appends ` (2)` on a name collision, and rejects `../escape.md` and absolute paths with a non-retryable error. 12. A config containing `{{secrets.SLACK_WEBHOOK}}` resolves the real value at run time, and the stored step input shows `[redacted]` in its place. 13. Export produces a JSON file that, imported into an empty database, recreates every applet with the same ids and webhook tokens. 14. A `discord` action with 4,500 characters of content sends three messages in order, none over 2,000 characters. 15. A fresh clone runs with `pnpm install && pnpm dev` and no env vars, and the first browser visit creates the admin. 16. `docker compose up -d` on a clean Ubuntu VPS with one A record serves the app over HTTPS with no other setup. 17. Killing the process mid-job and restarting it doesn't double-run or lose the job. 18. `pnpm backup` then deleting `./data/app.db` and restoring from the backup brings every record back. 19. Every outbound email in dev shows up in `./data/outbox`. ## Deliverables - The app, the worker, migrations, and a seed script that creates one applet of each trigger type with realistic configs and 20 sample runs in mixed states. - `lib/template.ts`, `lib/sandbox.ts`, and each trigger's `poll` function with unit tests. - `fixtures/` with three sample `.eml` messages (plain text, HTML with an attachment, and one with no `Message-ID` header), a sample RSS feed and Atom feed, and a sample poll response. - `docker-compose.yml` with `app`, `worker`, `caddy`, and `backup`, a `Caddyfile` that serves the host in `PUBLIC_URL`, and `scripts/backup.sh` behind `pnpm backup`. - README covering the laptop run, the VPS run (A record, MX record, port 25, compose), backup and restore, environment variables, the optional outside services (SMTP relay, Slack and Discord webhook URLs, ntfy and Pushover, creating a Google service account and sharing a sheet with it), and the DST behavior of the scheduler. Build in this order: the template renderer and its tests, then the data model and the queue with the `http` action, then the webhook trigger so you can fire runs by hand, then the scheduler with `job_locks` and the polling triggers, then the sandbox, then the SMTP receiver and the remaining actions, then the UI, then compose and backup. Run `pnpm dev` and fire a real webhook after each step.