t1k:web:devops:cloudflare-internal-netbird
| Field | Value |
|---|---|
| Module | devops |
| Version | 1.31.4 |
| Effort | medium |
| Tools | — |
Keywords: -internal host, access app worker destination, cf-access-client-id, cf-access-client-secret, cloudflare internal worker, custom_headers netbird, internal host behind access, netbird service token, non_identity policy, service auth policy, service token access app, worker behind netbird, worker custom domain access
How to invoke
Section titled “How to invoke”/t1k:web:devops:cloudflare-internal-netbird[new|verify|rotate|retire] <name>Worker on <name>-internal behind an Access service token, published on the mesh
Section titled “Worker on <name>-internal behind an Access service token, published on the mesh”An app is internal but runs on the edge. Three pieces, in this order:
peer --NetBird DNS--> <name>.<zone> (the name people use; mesh-only) --TLS/WireGuard--> netbird reverse proxy (stamps X-NetBird-User/Groups) --HTTPS + CF-Access-Client-* headers--> <name>-internal.<zone> (Worker custom domain) --Cloudflare edge--> Access app (service token ONLY) --> the WorkerThe Access app is the only way in; the proxy is the only holder of the token. Same shape as
package-registry staging (upm-staging.the1studio.org → upm-internal-staging.the1studio.org).
Not this skill: a self-hosted container on the mesh with no Worker (/t1k:web:devops:selfhost-netbird),
a public app on Tunnel + Access (/t1k:web:devops:selfhost-cloudflare), platform choice
(/t1k:web:devops:placement).
Executed by
Section titled “Executed by”The script is deterministic and fully specified, so running it is a mechanical step; deciding what to publish, which Access group admits whom, and whether a host may exist at all are not.
| Step | Agent |
|---|---|
Run scripts/internal-netbird.sh <name> --dry-run, then for real once the Worker is deployed, and report its output block verbatim (names, ids, aud, the three verification lines) | t1k-ci-watcher — one command, report what it printed |
After a deploy, poll https://<name>-internal.<zone>/ for the cf-access-aud header to appear and confirm it equals the aud the script printed | t1k-ci-watcher |
Stays on a reasoning tier: choosing the group that should reach the service, deciding the
-internal hostname, approving a token rotation, and every retire/delete (they change who can
reach a host). Never let a cheap tier delete an Access app or a NetBird service.
1. Worker: <name>-internal custom domain
Section titled “1. Worker: <name>-internal custom domain”cloudflare.config.ts, per deployment mode. The -internal host is a Worker custom domain
(DNS + TLS created by the deploy), never a Workers Route:
domains: ["<name>-internal.<zone>"],workersDev: false,previewUrls: false,Deploy with an explicit --mode (a bare cf deploy evaluates as production). Nothing else changes:
no triggers, no separate script per host — one Worker can serve several -internal hosts.
The script must exist before step 2, and the Access app references it by worker id — so the
first deploy necessarily publishes the domain before Access exists. package-registry accepted that
window (its app was attached after the fact). To avoid it, deploy a config with no domains first,
create the app (step 2), then deploy again with domains. [second order not exercised]
2. One command for the rest: scripts/internal-netbird.sh
Section titled “2. One command for the rest: scripts/internal-netbird.sh”cf creates the token, the app and its policy; curl creates the NetBird service. The script does
all four, is idempotent, and verifies the result:
cd .claude/skills/t1k-web-devops-cloudflare-internal-netbird # installed layout./scripts/internal-netbird.sh <name> --zone the1studio.org \ --netbird-token "$NETBIRD_TOKEN" # or NETBIRD_TOKEN in the env| Flag | Default | Meaning |
|---|---|---|
--zone | the1studio.org | zone for both names (the -internal host and the mesh name) |
--netbird-token | $NETBIRD_TOKEN | NetBird user PAT (/api/reverse-proxies) |
--group | All | access group, by id or name; narrow it to the team that should see the service |
--worker | <name> | Worker script name, when it is not the base name |
--dry-run | — | print every name and action, touch nothing |
Names it creates, all derived from <name> (a rerun finds them again):
| Object | Name |
|---|---|
| service token | <name>-netbird |
| Access app | <name>-netbird |
| Access policy (reusable, attached by id) | <name>-netbird |
| NetBird service | <name>.<zone> → <name>-internal.<zone> |
Rerunning is safe, and this is the intended repair path: an existing app, policy and service are
reused (the policy is updated in place, so its id and its attachment never change), and an
existing token is rotated (client_secret_version + 1, previous secret valid 24 h) because a
secret is printed only on create. Every other policy on the app is kept.
What it refuses (each is a real API failure it turns into a message first): a worker name that does
not exist, a worker already claimed by another Access app (409 12132), a NetBird group that does
not exist, a name that is not a legal DNS label, and a name ending in -internal (that is the
Worker, not the base name).
Order matters: the Worker and its -internal domain must already be deployed (step 1). The
script looks the worker up; it never deploys. [unexercised: deploy the Worker with no domains, run this script, then add domains — the verified order publishes the domain first]
The steps it performs, if you are doing them by hand:
# token — a secret comes back ONCE, on createcf zero-trust access service-tokens create --name <name>-netbird --duration forever
# policy: create as a REUSABLE account policy, then attach it BY IDPID=$(cf zero-trust access policies create --name <name>-netbird --decision non_identity \ --include "$(jq -nc --arg t <token-id> '[{service_token:{token_id:$t}}]')" | jq -r .id)
# app + the policy REFERENCE (an inline policy object here would be app-local)cf zero-trust access applications create --body "$(jq -nc --arg w <worker-id> --arg pid "$PID" '{ name: "<name>-netbird", type: "self_hosted", destinations: [{ type: "worker", worker_id: $w }], session_duration: "0s", allowed_idps: [], policies: [{ id: $pid }] }')"
# NetBird service: mesh name -> the -internal host, token in custom_headerscurl -fsS -X POST https://vpn.the1studio.org/api/reverse-proxies/services \ -H "Authorization: Token $NETBIRD_TOKEN" -H 'Content-Type: application/json' \ --data "$(jq -nc --arg host <name> --arg zone <zone> --arg cid <client-id> --arg sec <secret> '{ name: ($host + "." + $zone), domain: ($host + "." + $zone), mode: "http", private: true, enabled: true, access_groups: ["<group-id>"], pass_host_header: false, rewrite_redirects: false, targets: [{ target_id: $zone, target_type: "cluster", path: "/", protocol: "https", host: ($host + "-internal." + $zone), port: 443, enabled: true, options: { direct_upstream: true, custom_headers: { "CF-Access-Client-Id": $cid, "CF-Access-Client-Secret": $sec } } }] }')"The App object, the Decision value and the measurement behind each: references/access-service-token.md.
The three NetBird settings that are not defaults:
| Setting | Value | Why |
|---|---|---|
pass_host_header | false | The Worker routes tenants by Host and answers 404 unknown host for names outside its list. Measured: true → 403, false → 200 for the same backend. |
protocol / port | https / 443 | The target is the public -internal hostname, not a container. |
access_groups | the team, not All | Group ids from GET /api/groups; All admits every peer. |
Status moves pending → active within seconds; a one-label name uses the zone wildcard, so
meta.certificate_issued_at being set immediately is normal.
Verify — the script runs checks 1–3 itself
Section titled “Verify — the script runs checks 1–3 itself”H=<name>-internal.<zone># 1. gated at the edge: 403 and the aud of THIS app (a wrong app answers with its own aud)curl -sS -D- -o /dev/null "https://$H/" | grep -i cf-access-aud
# 2. the token is sufficient, direct (no proxy in the path): 200curl -sS -o /dev/null -w '%{http_code}\n' "https://$H/" \ -H "CF-Access-Client-Id: $CID" -H "CF-Access-Client-Secret: $SEC"
# 3. through the mesh name: the Worker's own 200 + headers — proves custom_headers landedcurl -sS -D- "https://<name>.<zone>/" | sed -n '1p;/x-registry\|x-/p'
# 4. from a host that is NOT on the mesh: must fail (NXDOMAIN/403), never 200Check 3 is the only proof the proxy forwards the token; if 2 passes and 3 answers 403, drop to
GET /api/reverse-proxies/services and read targets[0].options.custom_headers. Check 4 cannot be
automated from a mesh peer — run it from a machine that is not on the VPN.
Rotate, retire
Section titled “Rotate, retire”| Change | Do |
|---|---|
| Rotate the token | Rerun the script: it bumps client_secret_version with a 24 h grace on the old secret, then rewrites custom_headers. By hand: cf zero-trust access service-tokens update <id> --name <n> --client-secret-version <v+1> --previous-client-secret-expires-at <ISO8601> — --name is required, an update without it drops the name off the record (measured) |
| Rename the mesh name | Create the new service, verify, delete the old |
| Retire | Delete the NetBird service, then cf zero-trust access applications delete <app-id> --force, then the token, then drop domains from the Worker config and deploy. Never the reverse: detaching the domain last keeps the host gated until it is gone |
| Add a caller (CI, another proxy) | A new service token + a second policy on the same app (each policy’s include is one OR-list); never a second app on the same worker id (409) |
| Add a second Worker | Add a worker destination to the app (one app then holds several; verified) |
Do not delete the app before the Worker’s domain: that leaves the host public with no gate.
Hard rules
Section titled “Hard rules”worker_id, notworker name.cf workers scripts search --name <w>returns the script uuid. A guessed id creates an app that gates nothing, silently.- A worker id can belong to exactly ONE app. A second app targeting it is a hard 409
(
12132 destination belongs to another application) — e.g. the registry’s staging Worker is already claimed bypackage-registry-netbird. Add a destination to the existing app instead. - A hostname already claimed by another app is the same 409. Read the live apps
(
cf zero-trust access applications list) before creating; a stale app you forgot is the usual claimant.public→workeralso cannot be flipped in place on a claimed hostname, so create the app with the worker destination from the start. - Re-sending a policy object without its
idDETACHES the old one. An app update whosepolicies[]entry has noidcreates a NEW policy; the previous one is silently left behind as an orphaned reusable policy withapp_count: 0— the app keeps working, so nothing looks broken while the account accumulates dead policies (this is exactly what a rename done naively caused). Attach by{id}and mutate the policy throughcf zero-trust access policies update. - Policy attachment is by id; an inline policy object makes an app-local policy (
reusable: false). Use the reusable form (cf zero-trust access policies create, thenpolicies: [{id}]): it can back several apps and it survives a rerun with the same id. - The client secret is readable by anyone holding a NetBird user PAT (
GET /api/reverse-proxies/servicesprintscustom_headers). Treat that PAT as a secret. service-tokens updatewithout--nameerases the token’s name. The record survives withnamegone, so the next lookup by name misses it and a rerun creates a duplicate. Always pass--name, as the script does.
More with symptoms: ledger/gotchas/t1k-web-devops-cloudflare-internal-netbird.md.
Script: scripts/internal-netbird.sh — --dry-run first; its header lists
every name it creates.
References
Section titled “References”- references/access-service-token.md — exact API bodies, the JWT verification contract, verification transcript
/t1k:web:devops:selfhost-netbird— the plain mesh service (no Worker, no Access)/t1k:web:devops:cloudflare-access— creating the app/policy/token correctly, auditing orphans/t1k:cloudflare—cfitself, deploy flags, config shapepackage-registry—cloudflare.config.ts(staging domains + AUD),src/web/guard.ts(service-token-only assertion),src/access/proxy-identity.ts