From 969ac3b2973625ad399078a9288f091d8e80a557 Mon Sep 17 00:00:00 2001 From: Joshua Gilman Date: Tue, 18 Aug 2026 16:21:36 -0700 Subject: [PATCH] docs: document tailscale policy gitops workflow Record the decision to make git the source of truth for the tailnet policy file, the reference facts needed to operate it, and the procedure for changing, verifying, reverting, and emergency-editing the policy. Companion to GilmanLab/networking feat/tailscale-policy-gitops. --- ...002-manage-tailscale-policy-with-gitops.md | 119 +++++++++++++++++ docs/docs/index.md | 9 ++ .../reference/networking/tailscale-policy.md | 102 +++++++++++++++ docs/docs/runbooks/tailscale-policy-change.md | 121 ++++++++++++++++++ docs/mkdocs.yml | 4 + 5 files changed, 355 insertions(+) create mode 100644 docs/docs/decisions/0002-manage-tailscale-policy-with-gitops.md create mode 100644 docs/docs/reference/networking/tailscale-policy.md create mode 100644 docs/docs/runbooks/tailscale-policy-change.md diff --git a/docs/docs/decisions/0002-manage-tailscale-policy-with-gitops.md b/docs/docs/decisions/0002-manage-tailscale-policy-with-gitops.md new file mode 100644 index 0000000..49e5e56 --- /dev/null +++ b/docs/docs/decisions/0002-manage-tailscale-policy-with-gitops.md @@ -0,0 +1,119 @@ +--- +status: accepted +date: 2026-08-18 +--- + +# ADR-0002: Manage the Tailnet Policy File with GitOps + +## Context and Problem Statement + +The tailnet policy file controls tag ownership, auto-approved subnet routes, +network access rules, and Tailscale SSH access. Editing it in the Tailscale +admin console leaves no reviewable history, no validation before a change takes +effect, and no relationship to the repository that owns the network +implementation. Where should the policy file live, and how should it reach the +tailnet? + +## Decision Drivers + +- Keep the policy under review and version control with the rest of the network + implementation. +- Validate a policy change before it reaches the tailnet. +- Avoid storing a long-lived Tailscale credential in the repository or in CI. +- Keep the operational surface small enough for a single-owner lab. + +## Considered Options + +- Track the policy file in `GilmanLab/networking` and sync it with Tailscale's + `gitops-acl-action`. +- Continue editing the policy in the Tailscale admin console. +- Manage the policy with the Tailscale Terraform provider. +- Track the policy in a dedicated private repository. + +## Decision Outcome + +Track the policy file at `tailscale/policy.hujson` in `GilmanLab/networking` and +sync it with the `tailscale/gitops-acl-action` GitHub Action: validate on pull +requests, apply on merge to `master`. Authenticate with a Tailscale workload +identity federation credential holding the `policy_file` scope, so CI exchanges +a GitHub OIDC token for a short-lived API token and no long-lived secret is +stored. + +Git is the source of truth. Admin console editing is disabled through **Prevent +edits in the admin console**, with an external reference pointing at the policy +file. An operator may still use **Edit anyway** for an emergency change, and the +next apply overwrites it. + +The policy file carries no user email addresses, so the repository stays public. +Adding a rule, group, or test that names an individual user requires revisiting +that assumption. + +### Consequences + +- Good, because every policy change is a reviewable diff with history. +- Good, because an invalid policy fails on the pull request instead of on the + tailnet. +- Good, because CI holds no long-lived Tailscale credential. +- Good, because the policy sits beside the network implementation it governs. +- Bad, because applying a change now depends on GitHub Actions availability. +- Bad, because an emergency console edit is silently discarded by the next + apply, so it must be carried back into git. +- Bad, because a policy change and its documentation live in two repositories + and must be merged as companion changes. + +### Confirmation + +The implementation conforms to this decision when: + +- `tailscale/policy.hujson` in `GilmanLab/networking` matches the policy served + by the tailnet. +- `.github/workflows/tailscale-acl.yml` runs the action with `action: test` on + pull requests and `action: apply` on `master`. +- The workflow authenticates with `oauth-client-id` and `audience`, and the + repository holds no Tailscale API key or OAuth secret. +- **Prevent edits in the admin console** is enabled, with an external reference + to the policy file. +- The policy file contains no user email addresses. + +## Pros and Cons of the Options + +### GitOps in `GilmanLab/networking` + +- Good, because the policy is reviewed and versioned with the network + implementation. +- Good, because Tailscale validates the whole policy on every pull request. +- Bad, because it adds a CI dependency to a change that used to be a console + edit. + +### Admin Console Only + +- Good, because it needs no tooling and applies changes immediately. +- Bad, because it has no diff, no review, and no history beyond the console's + own audit trail. +- Bad, because there is no validation step separate from applying the change. + +### Tailscale Terraform Provider + +- Good, because it could manage the policy alongside other Tailscale resources + such as trust credentials and DNS settings. +- Bad, because it requires state storage and a Terraform toolchain for a single + file. +- Bad, because it expresses the policy through provider resources rather than + the policy file Tailscale documents. + +### Dedicated Private Repository + +- Good, because it matches Tailscale's guidance for policies containing user + email addresses. +- Bad, because it separates the policy from the network implementation and its + review flow. +- Bad, because it adds a repository to maintain for one file, while this policy + contains no personal data. + +## More Information + +See the [Tailscale policy reference](../reference/networking/tailscale-policy.md) +for the tailnet's tags, routes, and credential inventory, and the +[Tailscale policy change runbook](../runbooks/tailscale-policy-change.md) for +the change, rollback, and emergency procedures. Tailscale documents the +mechanism in [GitOps for Tailscale with GitHub Actions](https://tailscale.com/docs/integrations/github/gitops). diff --git a/docs/docs/index.md b/docs/docs/index.md index 6e4fb38..35ff96b 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -14,6 +14,8 @@ and runbook lives here. - [ADR-0001: Use VyOS for Layer 3 and Dedicated Switches for Layer 2](decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md) assigns core routing and switching responsibilities. +- [ADR-0002: Manage the Tailnet Policy File with GitOps](decisions/0002-manage-tailscale-policy-with-gitops.md) + makes git the source of truth for the tailnet policy file. ## Designs @@ -31,3 +33,10 @@ and runbook lives here. network devices and their roles. - [Physical connections](reference/networking/physical-connections.md) records every installed cable and its endpoint ports. +- [Tailscale policy](reference/networking/tailscale-policy.md) records the + tailnet identity, tags, auto-approved routes, and sync credentials. + +## Runbooks + +- [Change the Tailscale policy](runbooks/tailscale-policy-change.md) covers + changing, verifying, reverting, and emergency-editing the tailnet policy. diff --git a/docs/docs/reference/networking/tailscale-policy.md b/docs/docs/reference/networking/tailscale-policy.md new file mode 100644 index 0000000..8b86642 --- /dev/null +++ b/docs/docs/reference/networking/tailscale-policy.md @@ -0,0 +1,102 @@ +--- +title: Tailscale policy +description: Tailnet identity, policy file location, tags, and the credentials that sync the policy. +--- + +# Tailscale policy + +The tailnet policy file is the canonical definition of tag ownership, +auto-approved subnet routes, network access rules, and Tailscale SSH access. It +is applied by CI, not by hand; +[ADR-0002](../../decisions/0002-manage-tailscale-policy-with-gitops.md) records +why. To change it, follow the +[policy change runbook](../../runbooks/tailscale-policy-change.md). + +## Tailnet + +| Field | Value | +| --- | --- | +| Tailnet ID | `THZctfF8wr11CNTRL` | +| Admin console | | +| Console editing | Disabled through **Prevent edits in the admin console** | + +## Source of truth + +| Artifact | Location | +| --- | --- | +| Policy file | [`tailscale/policy.hujson`](https://github.com/GilmanLab/networking/blob/master/tailscale/policy.hujson) in `GilmanLab/networking` | +| Sync workflow | [`.github/workflows/tailscale-acl.yml`](https://github.com/GilmanLab/networking/blob/master/.github/workflows/tailscale-acl.yml) | +| Action | [`tailscale/gitops-acl-action`](https://github.com/tailscale/gitops-acl-action), pinned by commit | + +The workflow runs the action with `action: test` on pull requests, which sends +the policy to Tailscale for validation, and with `action: apply` on pushes to +`master` and on manual dispatch, which validates and then writes the policy to +the tailnet. + +The file is [HuJSON](https://github.com/tailscale/hujson): JSON with comments +and trailing commas. Section syntax is documented in Tailscale's +[policy file reference](https://tailscale.com/docs/reference/syntax/policy-file). + +## Tags + +| Tag | Purpose | Owner | +| --- | --- | --- | +| `tag:subnet-router` | Devices that advertise lab and home subnet routes into the tailnet | `autogroup:admin` | + +A tagged device is owned by its tag, not by the user who registered it. Removing +a tag from the policy while a device still carries it leaves that device without +the access the tag granted. + +## Auto-approved routes + +The policy auto-approves the following advertised routes for +`tag:subnet-router`, so a replaced or re-registered subnet router needs no +manual approval: + +| Range | Scope | +| --- | --- | +| `10.10.0.0/16` | Lab | +| `172.16.0.0/16` | Lab | +| `192.168.1.0/24` | Home | +| `192.168.2.0/24` | Home | + +These ranges are authoritative for the tailnet today. The Lab v2 address plan is +not yet written; when it exists, it becomes the canonical source for lab ranges +and these entries must be reconciled with it. + +## Credentials + +CI authenticates with a Tailscale +[workload identity federation](https://tailscale.com/docs/features/workload-identity-federation) +credential. GitHub Actions presents an OIDC token, Tailscale exchanges it for a +short-lived API token, and no long-lived credential is stored. + +| Field | Value | +| --- | --- | +| Credential type | OpenID Connect trust credential | +| Issuer | GitHub Actions | +| Subject | `repo:GilmanLab/networking:*` | +| Scopes | `policy_file`, with `devices:posture_attributes` and `devices:core:read` | +| Audience | `api.tailscale.com/` | + +Trust credentials are managed on the +[Trust credentials](https://login.tailscale.com/admin/settings/trust-credentials) +page. The node-registration credential used elsewhere in the lab is a separate +credential with auth key scopes and must not be reused here. + +The workflow reads three GitHub Actions **variables** in `GilmanLab/networking`. +None is secret; the client ID and audience are published by the admin console. + +| Variable | Value | +| --- | --- | +| `TS_TAILNET` | Tailnet ID | +| `TS_POLICY_CLIENT_ID` | Trust credential client ID | +| `TS_POLICY_AUDIENCE` | `api.tailscale.com/` | + +## Drift + +`gitops-pusher`, which the action runs, detects console edits by comparing the +tailnet's policy checksum against a cache file. CI runners are ephemeral and the +action does not persist that cache, so drift detection reports nothing in this +setup. Console editing is disabled instead, and an apply always overwrites the +tailnet's copy with the file in git. diff --git a/docs/docs/runbooks/tailscale-policy-change.md b/docs/docs/runbooks/tailscale-policy-change.md new file mode 100644 index 0000000..d8b7ed5 --- /dev/null +++ b/docs/docs/runbooks/tailscale-policy-change.md @@ -0,0 +1,121 @@ +--- +title: Change the Tailscale policy +description: Change, verify, roll back, or emergency-edit the tailnet policy file. +--- + +# Change the Tailscale policy + +Use this runbook to change the tailnet policy file: tags, auto-approved routes, +access rules, or Tailscale SSH access. Field-level facts live in the +[Tailscale policy reference](../reference/networking/tailscale-policy.md). + +## Preconditions + +- Write access to `GilmanLab/networking`. +- Owner, Admin, or Network admin on the tailnet, for verification in the admin + console. +- The repository variables `TS_TAILNET`, `TS_POLICY_CLIENT_ID`, and + `TS_POLICY_AUDIENCE` are set, and the trust credential behind them is valid. + +## Safety impact + +An applied policy takes effect on the tailnet immediately and can remove access +from devices and users, including your own Tailscale SSH access. It cannot lock +you out of the admin console, so recovery is always possible. Tailscale +validates the policy before applying it, so a malformed or self-inconsistent +policy fails in CI rather than on the tailnet. + +## Procedure + +1. Create a branch and worktree in `GilmanLab/networking` from the fetched + default branch: + + ```bash + git fetch origin --prune + wt switch --create --base origin/master --no-cd --format=json + ``` + +2. Edit `tailscale/policy.hujson` in the returned worktree path. Keep comments + describing intent next to the rules they explain. + +3. Commit and push, then open a pull request: + + ```bash + git push -u origin HEAD + gh pr create --fill + ``` + +4. Wait for the **Tailscale ACL** check and read its result: + + ```bash + gh pr checks --watch + ``` + + The `Validate policy` step sends the whole file to Tailscale. Failures are + annotated on the offending line of the policy file. Fix and push again until + the check passes. + +5. Squash-merge the pull request: + + ```bash + gh pr merge --squash + ``` + +6. Watch the apply run on `master`: + + ```bash + gh run watch "$(gh run list --workflow=tailscale-acl.yml --branch=master --limit=1 --json databaseId --jq '.[0].databaseId')" + ``` + + The `Apply policy` step logs the tailnet checksum, the local checksum, and + either `no update needed, doing nothing` or a successful write. + +7. Remove the worktree: + + ```bash + wt remove + ``` + +## Verification + +- The apply run concluded successfully. +- The [Access controls](https://login.tailscale.com/admin/acls) page shows the + merged policy. +- The behavior you changed works as intended: for a route change, confirm the + advertised route on the [Machines](https://login.tailscale.com/admin/machines) + page; for an access rule, connect from an affected device. + +## Rollback + +Revert the merged pull request and let the pipeline apply the previous policy: + +```bash +gh pr create --title 'revert: ' --fill # from a revert branch +``` + +Reverting through GitHub keeps git and the tailnet consistent. Do not undo a +change by editing the admin console. + +## Emergency change + +If the pipeline is unavailable and access must be restored now: + +1. Open [Access controls](https://login.tailscale.com/admin/acls) and select + **Edit anyway** to bypass the console lock. +2. Make the smallest change that restores access. +3. Carry the same change into `tailscale/policy.hujson` through a pull request + as soon as possible. + +The next apply overwrites the tailnet's copy with the file in git, so an +emergency edit that is not carried back into git is silently lost. + +## Escalation + +- Validation fails with an error you cannot interpret: check the + [policy file syntax reference](https://tailscale.com/docs/reference/syntax/policy-file). +- The apply step fails to authenticate: inspect the trust credential on the + [Trust credentials](https://login.tailscale.com/admin/settings/trust-credentials) + page, which records the most recent token exchange error, and confirm the + repository variables still match the credential. +- The apply step fails on a checksum mismatch: someone edited the policy in the + console. Reconcile that edit into git, then re-run the workflow. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index e21d559..e1e1362 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -33,6 +33,7 @@ nav: - Home: index.md - Decisions: - "ADR-0001: Use VyOS for Layer 3 and Dedicated Switches for Layer 2": decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md + - "ADR-0002: Manage the Tailnet Policy File with GitOps": decisions/0002-manage-tailscale-policy-with-gitops.md - Designs: - Drafts: - Lab v2 core network: designs/drafts/lab-v2-core-network.md @@ -42,6 +43,9 @@ nav: - Networking: - Hardware: reference/networking/hardware.md - Physical connections: reference/networking/physical-connections.md + - Tailscale policy: reference/networking/tailscale-policy.md + - Runbooks: + - Change the Tailscale policy: runbooks/tailscale-policy-change.md plugins: - search