diff --git a/docs/docs/decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md b/docs/docs/decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md index be56557..6b7d897 100644 --- a/docs/docs/decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md +++ b/docs/docs/decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md @@ -89,5 +89,5 @@ Management addresses on the switches do not violate this decision. ## More Information -See the [Lab v2 core network design](../designs/drafts/lab-v2-core-network.md) +See the [Lab v2 core network design](../designs/lab-v2-core-network.md) and [hardware reference](../reference/networking/hardware.md). diff --git a/docs/docs/designs/drafts/lab-v2-core-network.md b/docs/docs/designs/drafts/lab-v2-core-network.md deleted file mode 100644 index ff1c32b..0000000 --- a/docs/docs/designs/drafts/lab-v2-core-network.md +++ /dev/null @@ -1,166 +0,0 @@ ---- -title: Lab v2 core network -status: draft -authors: - - GilmanLab -created: 2026-08-14 -updated: 2026-08-14 -related-decisions: - - ADR-0001 ---- - -# Lab v2 core network - -## Summary - -The core network uses `gw01`, a Minisforum VP6630 running VyOS, for Layer 3 -routing, firewall policy, and NAT. `sw-core01`, a MikroTik CRS309-1G-8S+IN, -handles core Layer 2 switching and VLAN transport. `sw-mgmt01`, a TRENDnet -TEG-3102WS, connects both non-SFP NICs from each MS-02 node to `gw01` for -management/OOB traffic. `rtr01`, a second MikroTik CRS309-1G-8S+IN, acts as the -home router and connects the lab to the home network and the internet. Device -names are canonical per the [naming registry](../../reference/naming.md). - -This design defines device responsibilities, logical topology, configuration -requirements, failure boundaries, and verification criteria. Address -allocation, VLAN allocation, physical port assignment, and network-service -ownership are outside this document. - -## Goals - -- Keep routing and traffic policy on `gw01`. -- Keep core VLAN transport and physical switching on `sw-core01`. -- Carry MS-02 management/OOB traffic through `sw-mgmt01`. -- Route home-to-lab traffic without source NAT. -- Apply source NAT to lab-to-internet traffic on `gw01`. -- Store network-device configuration in version control. -- Validate behavior before saving a deployed configuration. -- Preserve a recovery path that does not depend on the primary network path. - -## Non-goals - -- Address and VLAN allocation -- Physical port and cable assignment -- DHCP, DNS, or time-service ownership -- Compute-platform network configuration -- Workload network overlays -- Application ingress or service advertisement -- Application DNS records -- Service-to-service traffic policy -- Storage protocol design - -## Logical Topology - -```mermaid -flowchart LR - HOME[Home network] --> RTR[rtr01 home router] - RTR -->|Routed transit| VYOS[gw01 VyOS gateway] - VYOS -->|802.1Q trunk| CRS[sw-core01 core switch] - CRS --> SEGMENTS[Lab network segments] - VYOS -->|Management/OOB uplink| TEG[sw-mgmt01] - TEG -->|Two non-SFP NICs per node| MS02[lab01, lab02, lab03] -``` - -`rtr01` routes traffic between the home network and the `gw01` transit -interface. `gw01` routes lab prefixes, applies firewall policy, and performs -source NAT for internet egress. `sw-core01` carries lab VLANs between `gw01` -and connected lab devices. `sw-mgmt01` connects directly to `gw01` and carries -management/OOB traffic for both non-SFP NICs on each MS-02 node. -The [physical connection map](../../reference/networking/physical-connections.md) is the -authoritative port-to-port cabling record. - -## Device Responsibilities - -| Device | Responsibilities | -| --- | --- | -| `rtr01` (MikroTik CRS309-1G-8S+IN) | Home-network routing, internet access, and the upstream side of the routed lab transit | -| `gw01` (Minisforum VP6630 running VyOS) | Lab gateways, route selection, firewall policy, source NAT, the downstream side of the routed transit, and the management/OOB gateway | -| `sw-core01` (MikroTik CRS309-1G-8S+IN) | Core VLAN transport, access ports, trunks, and physical link aggregation | -| `sw-mgmt01` (TRENDnet TEG-3102WS) | Layer 2 management/OOB connectivity for both non-SFP NICs on each MS-02 and a direct uplink to `gw01` | - -[ADR-0001](../../decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md) -defines the Layer 2 and Layer 3 boundary. - -## Routing and NAT - -The routing design has these invariants: - -- `rtr01` has routes for lab prefixes through the `gw01` transit address. -- `gw01` uses the `rtr01` transit address as its default route. -- `gw01` owns the gateway address for every routed lab segment. -- `sw-core01` and `sw-mgmt01` do not route between lab segments. -- Home-to-lab traffic retains its original source address. -- `gw01` applies source NAT to lab-to-internet traffic. -- Firewall rules distinguish new connections from established reply traffic. - -## Traffic Policy - -`gw01` enforces policy for: - -- Home network to lab segments -- Lab segments to the home network -- Lab segments to the internet -- Traffic between routed lab segments -- Traffic addressed to `gw01` -- Management/OOB traffic through `sw-mgmt01` -- Management traffic addressed to network devices - -Each firewall rule identifies the source, destination, protocol, destination -port, connection direction, and owner. Rules permit required flows explicitly. -Stateful rules permit established reply traffic without permitting a new flow in -the reverse direction. - -## Configuration Requirements - -`gw01`, `sw-core01`, and `sw-mgmt01` each have one version-controlled -configuration source. The deployment process: - -1. Renders the effective configuration. -2. Validates syntax and policy before deployment. -3. Shows the effective change for operator review. -4. Applies the change without saving it as the startup configuration. -5. Verifies required connectivity and policy behavior. -6. Saves the configuration only after verification succeeds. -7. Restores the previous configuration when verification fails. - -Drift detection compares each running configuration with its repository source. - -## Management and Recovery - -Firewall policy limits routine management access to approved source networks. -`sw-mgmt01` carries management/OOB traffic from both non-SFP NICs on each -MS-02 node directly to `gw01`. - -Each network device has a recovery path that remains available when its -production configuration or primary network link fails. Recovery credentials do -not reside in device configuration committed to the repository. - -## Failure Boundaries - -| Failure | Effect | -| --- | --- | -| `rtr01` failure | The lab loses home-network and internet connectivity. Internal lab switching and routing remain available. | -| `gw01` failure | Routed lab segments lose their gateways, inter-segment routing, policy enforcement, management/OOB gateway, and internet egress. | -| `sw-core01` failure | Devices connected through `sw-core01` lose Layer 2 connectivity. | -| `sw-mgmt01` or its `gw01` uplink failure | Both non-SFP NICs on each MS-02 lose management/OOB connectivity. | -| Routed transit failure | Home-to-lab and lab-to-internet traffic stop. Internal lab traffic remains available within its unaffected Layer 2 and Layer 3 paths. | -| `gw01`-to-`sw-core01` trunk failure | VLANs carried by the trunk lose their `gw01` gateways. | -| Invalid configuration | Deployment verification fails and the previous configuration is restored. | - -## Verification - -A deployment is valid when the observed behavior matches these checks: - -- Every connected interface reports the assigned link state and speed. -- Each VLAN is present only on its assigned access ports and trunks. -- A client in each routed segment reaches its `gw01` gateway. -- The `rtr01` and `gw01` route tables contain the required transit and lab routes. -- Home-to-lab traffic retains its home-network source address. -- Lab-to-internet traffic uses the `gw01` source-NAT address. -- Each permitted firewall flow succeeds. -- Each denied firewall flow fails. -- Established reply traffic succeeds without enabling a new reverse flow. -- Both non-SFP NICs on each MS-02 connect through `sw-mgmt01`. -- MS-02 management/OOB traffic reaches `gw01` through the `sw-mgmt01` uplink. -- Management access succeeds only from approved source networks. -- A failed deployment restores the previous configuration. diff --git a/docs/docs/designs/lab-v2-core-network.md b/docs/docs/designs/lab-v2-core-network.md new file mode 100644 index 0000000..53c2651 --- /dev/null +++ b/docs/docs/designs/lab-v2-core-network.md @@ -0,0 +1,245 @@ +--- +title: Lab v2 core network +status: accepted +authors: + - GilmanLab +created: 2026-08-14 +updated: 2026-08-18 +related-decisions: + - ADR-0001 +--- + +# Lab v2 core network + +## Summary + +The core network uses `gw01`, a Protectli VP6630 running VyOS, for Layer 3 +routing, firewall policy, DHCP, DNS forwarding, Tailscale routing, and source +NAT. `sw-core01`, a MikroTik CRS309-1G-8S+IN, carries core VLANs and the +compute-facing SFP+ links. `sw-mgmt01`, a TRENDnet TEG-3102WS, separates the +MS-02 10GbE management interfaces from their 2.5GbE AMT interfaces. + +The [network address and VLAN plan](../reference/networking/address-plan.md) +defines the exact prefixes, gateways, allocations, interface mapping, and +switch port roles. The +[physical connection map](../reference/networking/physical-connections.md) +defines the installed cables. + +## Context and Scope + +`rtr01` connects the home network and internet to `gw01` through a routed +point-to-point transit. `gw01` is the Layer 3 boundary for every lab VLAN. +The switches remain Layer 2 devices under +[ADR-0001](../decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md). + +The design retains the deployed `10.10.0.0/16` lab aggregate and the proven +home-to-lab transit. It removes the former Tinkerbell/PXE network, UM760 +platform-cluster assumptions, BGP peers, PowerDNS authority, IncusOS artifact +server, and `bootstrap-k0s` service. The UM760 becomes `sandbox01`, a general +test and spike host outside the IncusOS cluster. + +## Goals + +- Give every bare-metal management and OOB endpoint a deterministic DHCP + reservation. +- Keep management, OOB, and sandbox/workload traffic in separate VLANs. +- Keep routing, DHCP, DNS forwarding, traffic policy, and NAT on `gw01`. +- Route home-to-lab traffic without source NAT. +- Apply source NAT to lab-to-internet traffic. +- Keep core VLAN transport and physical switching on the dedicated switches. +- Store network-device configuration in version control. +- Validate behavior before saving a deployed configuration. +- Preserve local-console recovery when the routed network is unavailable. + +## Non-goals + +- Compute-facing SFP+ bond and link-aggregation policy +- Kubernetes cluster, pod, service, or load-balancer IPAM +- Application ingress and service advertisement +- Storage protocol and storage-network design +- Application DNS records +- General service-to-service policy +- Time-service ownership +- Dynamic routing without a confirmed consumer + +## Design Overview + +```mermaid +flowchart LR + HOME[Home networks] --> RTR[rtr01] + RTR -->|10.0.0.0/30 transit| GW[gw01] + GW -->|VLAN 10 and 40 trunk| CORE[sw-core01] + CORE -->|SFP+ links, roles assigned later| HOSTS[lab01–03 and nas01] + GW -->|VLAN 10 and 70 trunk| MGMT[sw-mgmt01] + MGMT -->|10GbE RJ45 management| NODES[lab01–03] + MGMT -->|2.5GbE RJ45 AMT| AMT[lab01–03 AMT] + MGMT -->|5GbE RJ45 management| NAS[nas01] + GW -->|VLAN 40 access| SANDBOX[sandbox01] + GW -->|VLAN 70 access| OOB[pikvm01 and kvm01] +``` + +`gw01` owns the gateway address for VLAN 10 management, VLAN 40 +sandbox/workload, and VLAN 70 OOB. The management switch presents VLAN 10 as +untagged access to the upper 10GbE RJ45 port on each MS-02 and VLAN 70 as +untagged access to the lower 2.5GbE vPro/AMT port. Both VLANs use a tagged +uplink to `gw01`. + +The two MS-02 SFP+ interfaces and the `nas01` 10GbE interface remain physically +connected to `sw-core01`. Their instance, cluster, storage, and aggregation +roles are deferred. Initial IncusOS installation and management do not depend +on those links. + +## Device Responsibilities + +| Device | Responsibilities | +| --- | --- | +| `rtr01` | Home routing, internet edge, upstream transit endpoint, and route to `10.10.0.0/16` | +| `gw01` | Lab gateways, static routing, firewall policy, DHCP, DNS forwarding, Tailscale subnet routing, source NAT, and local `glab.lol` mirror | +| `sw-core01` | Layer 2 transport for management and future compute-facing VLANs | +| `sw-mgmt01` | Layer 2 separation of host management and AMT/OOB traffic | +| `sandbox01` | General-purpose tests and spikes on the isolated sandbox/workload VLAN | + +`gw01` does not run PowerDNS, an IncusOS artifact server, `bootstrap-k0s`, or +BGP. Product-side componere tooling builds and burns or serves IncusOS media +locally when needed. + +## Addressing, DHCP, and DNS + +The [network address and VLAN plan](../reference/networking/address-plan.md) +is the single source for address and port values. + +`gw01` provides DHCP on every client VLAN. Named endpoints use reservations +bound to permanent hardware MAC addresses. Unnamed temporary clients use the +documented dynamic pools. IncusOS seeds request DHCP on each node's 10GbE RJ45 +management interface; AMT independently requests DHCP on the 2.5GbE interface. + +Clients use their `gw01` VLAN gateway as the DNS resolver. `gw01` forwards +`glab.lol` to its local mirror of the private Route 53 zone and sends other +queries to configured recursive resolvers. The gateway does not serve the +legacy `lab.gilman.io` zone. + +This cold-start model assumes `gw01` is installed and its version-controlled +network configuration is applied before managed machines boot. + +## Routing and NAT + +- `rtr01` routes `10.10.0.0/16` through `10.0.0.2`. +- `gw01` uses `10.0.0.1` as its default route. +- `gw01` owns the gateway for every routed lab VLAN. +- Neither switch routes between lab segments. +- Home-to-lab and inter-VLAN traffic retain their source addresses. +- `gw01` applies source NAT only to lab traffic leaving for the internet. +- The initial design uses connected and static routes only. + +BGP returns only through a later design with a concrete dynamic-routing or +service-advertisement requirement. + +## Traffic Policy + +`gw01` applies a default-deny policy to routed VLAN boundaries and traffic +addressed to the gateway. Stateful rules permit established and related reply +traffic. + +The baseline policy permits: + +- DHCP and DNS from each client VLAN to `gw01`; +- approved home and Tailscale administration sources to management and OOB; +- required management flows from management to managed lab endpoints; +- internet egress from management and sandbox/workload; +- explicit `glab.lol` mirror traffic. + +The sandbox/workload VLAN cannot initiate connections to management or OOB. +Each additional flow identifies its source, destination, protocol, destination +port, direction, and owner in the version-controlled gateway policy. + +## Configuration and Deployment + +`gw01`, `sw-core01`, and `sw-mgmt01` each have one version-controlled +configuration source. A deployment: + +1. Renders the effective configuration. +2. Validates syntax and policy. +3. Shows the effective change for operator review. +4. Applies the candidate without saving it as startup configuration. +5. Verifies required connectivity and denied flows. +6. Saves only after verification succeeds. +7. Restores the previous configuration when verification fails. + +Drift detection compares each running configuration with its repository source. +Legacy configuration is migration input, not a second source of truth. + +## Management and Recovery + +Routine host management uses VLAN 10. AMT, `pikvm01`, `kvm01`, and +`sw-mgmt01` administration use VLAN 70. Their separation prevents a host +management configuration error from placing AMT directly on the same Layer 2 +segment. + +The PiKVM and TESmart chain provides remote console access to the connected +hosts. A local monitor, keyboard, and mouse attached to `pikvm01` remain the +break-glass path when `gw01` or routed access is unavailable. + +Recovery credentials do not reside in committed device configuration. + +## Failure Boundaries + +| Failure | Effect | +| --- | --- | +| `rtr01` | The lab loses home and internet connectivity; internal VLAN routing remains available. | +| `gw01` | Routed VLANs lose gateways, DHCP, DNS forwarding, policy enforcement, Tailscale routing, and internet egress. | +| `sw-core01` | Compute-facing SFP+ links and its management path fail; copper management and OOB remain available through `sw-mgmt01`. | +| `sw-mgmt01` or its trunk | MS-02 copper management and AMT, `nas01` copper management, and management-switch administration fail. | +| One MS-02 management access port | That node loses routine management; its separate AMT port remains available. | +| One MS-02 AMT access port | That node loses AMT; its separate management port remains available. | +| Invalid candidate configuration | Verification fails and the previous startup configuration remains or is restored. | + +## Delivery + +Migration from the current VyOS configuration removes VLAN 20, Tinkerbell +firewall rules, PowerDNS, IncusOS artifact serving, `bootstrap-k0s`, BGP, and +legacy UM760 bridge behavior. It renames the router to `gw01`, establishes the +management-switch trunk, and applies the canonical DHCP reservations and +firewall boundaries. + +The network may migrate one VLAN at a time. Preserve the routed transit and +current OOB access until replacement paths pass verification. + +## Verification + +A deployment is valid when: + +- every connected interface reports the expected link state and negotiated + speed; +- each VLAN appears only on its assigned access ports and trunks; +- each named endpoint receives its reserved address; +- each client receives `gw01` as its default gateway and DNS resolver; +- `glab.lol` resolves through the local mirror while public DNS still resolves; +- the `rtr01` and `gw01` route tables contain the transit and lab routes; +- home-to-lab traffic retains its home source address; +- lab-to-internet traffic uses the `gw01` source-NAT address; +- each permitted firewall flow succeeds and each denied flow fails; +- the sandbox cannot initiate management or OOB connections; +- each MS-02 retains management when its AMT link is disconnected and retains + AMT when its management link is disconnected; +- no BGP peers or retired gateway services remain; +- a failed candidate deployment leaves or restores the previous startup + configuration. + +## Alternatives Considered + +### Retain the full legacy VyOS configuration + +Rejected because it encodes the abandoned PXE/Tinkerbell path, the former +UM760 platform cluster, permissive inter-VLAN forwarding, BGP peers without a +consumer, and services that no longer belong on the gateway. + +### Use static addresses in IncusOS seeds + +Rejected in favor of DHCP reservations. Reservations keep addresses +deterministic while centralizing changes and MAC-to-address ownership on +`gw01`. + +### Combine management and OOB + +Rejected because independent VLANs preserve a recovery path when routine host +networking is misconfigured and allow narrower firewall policy. diff --git a/docs/docs/index.md b/docs/docs/index.md index 076ebc6..dd339b7 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -22,8 +22,8 @@ and runbook lives here. ## Designs -- [Lab v2 core network design](designs/drafts/lab-v2-core-network.md) defines - the core topology, device boundaries, configuration requirements, and +- [Lab v2 core network design](designs/lab-v2-core-network.md) defines the + accepted topology, device boundaries, network services, migration, and verification criteria. ## Reference @@ -34,6 +34,9 @@ and runbook lives here. device in the lab with its model and specifications. - [Networking hardware](reference/networking/hardware.md) identifies the core network devices and their roles. +- [Network address and VLAN plan](reference/networking/address-plan.md) is the + canonical source for routed prefixes, VLANs, address allocations, gateway + interfaces, and switch port roles. - [Physical connections](reference/networking/physical-connections.md) records every installed cable and its endpoint ports. - [Tailscale policy](reference/networking/tailscale-policy.md) records the diff --git a/docs/docs/reference/hardware-inventory.md b/docs/docs/reference/hardware-inventory.md index 70ddd31..b420cef 100644 --- a/docs/docs/reference/hardware-inventory.md +++ b/docs/docs/reference/hardware-inventory.md @@ -28,6 +28,7 @@ documentation, not this inventory. ### Minisforum UM760 - Quantity: `1` +- Role: general-purpose sandbox host - Hardware details: - `Ryzen 7` - `32GB RAM` @@ -36,6 +37,7 @@ documentation, not this inventory. ### Minisforum MS-02 Ultra - Quantity: `3` +- Product page: - Hardware details: - Model: `MS-02-Ultra` - CPU: `Intel Core Ultra 9 285HX` @@ -44,8 +46,8 @@ documentation, not this inventory. - `Samsung SSD 990 EVO Plus 2TB` - `Patriot M.2 P300 128GB` - `2x 10/25GbE-capable SFP+` ports - - `2x 2.5GbE RJ45` ports - - Intel `vPro/AMT` + - Upper `10GbE RJ45` port + - Lower `2.5GbE RJ45` port with Intel `vPro/AMT` ### MINISFORUM N5 Pro NAS diff --git a/docs/docs/reference/naming.md b/docs/docs/reference/naming.md index dffc86e..664d360 100644 --- a/docs/docs/reference/naming.md +++ b/docs/docs/reference/naming.md @@ -40,12 +40,8 @@ is: | `pikvm01` | PiKVM V4 Plus | KVM-over-IP console | | `kvm01` | TESmart HKS801-EB23 8x1 KVM | Console switch | | `ups01` | APC Smart-UPS SMT1000 | UPS | +| `sandbox01` | Minisforum UM760 | General-purpose test and spike host | -## Unnamed hardware - -| Device | Status | -| --- | --- | -| Minisforum UM760 | Shelf spare; named at commissioning | Models and physical specifications live in the [hardware inventory](hardware-inventory.md). diff --git a/docs/docs/reference/networking/address-plan.md b/docs/docs/reference/networking/address-plan.md new file mode 100644 index 0000000..9c5fcbf --- /dev/null +++ b/docs/docs/reference/networking/address-plan.md @@ -0,0 +1,131 @@ +--- +title: Network Address and VLAN Plan +description: Canonical routed prefixes, VLANs, DHCP allocations, and port roles. +--- + +# Network Address and VLAN Plan + +This document is the canonical source for lab prefixes, VLANs, gateway +interface mapping, DHCP allocations, and logical switch port roles. The +[physical connection map](physical-connections.md) remains authoritative for +installed cables. + +## Routed prefixes + +| Network | Prefix | Gateway | Purpose | +| --- | --- | --- | --- | +| Router transit | `10.0.0.0/30` | `rtr01` `10.0.0.1`; `gw01` `10.0.0.2` | Routed link between the home router and lab gateway | +| Lab aggregate | `10.10.0.0/16` | More-specific VLAN gateways on `gw01` | Route advertised to `rtr01` and Tailscale | +| Home | `192.168.1.0/24` | `rtr01` | Home network routed to the lab without source NAT | +| Home | `192.168.2.0/24` | `rtr01` | Additional home network advertised through Tailscale | + +`gw01` uses `10.0.0.1` as its default route. `rtr01` routes `10.10.0.0/16` +through `10.0.0.2`. `gw01` applies source NAT only when lab traffic exits +toward the internet. Home-to-lab and inter-VLAN traffic retain their source +addresses. + +## VLANs + +| VLAN | Name | Prefix | Gateway | Use | +| --- | --- | --- | --- | --- | +| `10` | Management | `10.10.10.0/24` | `10.10.10.1` | IncusOS management, network-device management, and `nas01` management | +| `40` | Sandbox/workload | `10.10.40.0/24` | `10.10.40.1` | `sandbox01` and future explicitly attached workload endpoints | +| `70` | OOB | `10.10.70.0/24` | `10.10.70.1` | MS-02 AMT, `pikvm01`, `kvm01`, and management-switch administration | + +VLAN 20 and `10.10.20.0/24` are retired. The lab does not retain a PXE or +Tinkerbell provisioning network. + +The management and OOB VLANs remain separate. The sandbox/workload VLAN cannot +initiate connections to management or OOB endpoints. Firewall policy permits +required administration flows explicitly and permits established replies. + +## Address allocations + +### Infrastructure and services + +| Endpoint | Address | Allocation | +| --- | --- | --- | +| `gw01` management gateway | `10.10.10.1` | Interface address | +| `sw-core01` management | `10.10.10.2` | DHCP reservation | +| `gw01` sandbox/workload gateway | `10.10.40.1` | Interface address | +| `gw01` OOB gateway | `10.10.70.1` | Interface address | +| `sw-mgmt01` management | `10.10.70.2` | DHCP reservation | +| `gw01` `glab.lol` mirror | `10.10.10.54` | Local service address | + +### Hosts + +| Device | Management | OOB | Notes | +| --- | --- | --- | --- | +| `lab01` | `10.10.10.11` | `10.10.70.11` | 10GbE RJ45 management; 2.5GbE RJ45 AMT | +| `lab02` | `10.10.10.12` | `10.10.70.12` | 10GbE RJ45 management; 2.5GbE RJ45 AMT | +| `lab03` | `10.10.10.13` | `10.10.70.13` | 10GbE RJ45 management; 2.5GbE RJ45 AMT | +| `nas01` | `10.10.10.14` | — | 5GbE RJ45 management link through `sw-mgmt01` | +| `sandbox01` | `10.10.40.10` | — | Direct untagged sandbox/workload attachment to `gw01` | +| `pikvm01` | — | `10.10.70.20` | Direct untagged attachment to `gw01` | +| `kvm01` | — | `10.10.70.21` | Direct untagged attachment to `gw01` | + +`gw01` supplies DHCP on every client VLAN. Infrastructure and named hosts use +DHCP reservations; the gateway interface and local DNS mirror addresses are +static. Dynamic clients use `.200` through `.250` within each client VLAN. +Reservations use each endpoint's permanent hardware MAC address as recorded in +the version-controlled gateway configuration. + +IncusOS seed data enables DHCP on the 10GbE management interface. The 2.5GbE +vPro interface receives its OOB reservation independently from AMT firmware. + +## Gateway interface mapping + +| `gw01` chassis port | VyOS interface | Mode | Network | +| --- | --- | --- | --- | +| `SFP+ 1` | `eth0` | Routed | Transit `10.0.0.2/30` to `rtr01` | +| `SFP+ 2` | `eth1` | 802.1Q trunk | VLANs 10 and 40 to `sw-core01` | +| `Port 1` | `eth3` | Untagged access | VLAN 40 to `sandbox01` | +| `Port 2` | `eth2` | 802.1Q trunk | VLANs 10 and 70 to `sw-mgmt01` | +| `Port 3` | `eth4` | Untagged access | VLAN 70 to `pikvm01` | +| `Port 4` | `eth5` | Untagged access | VLAN 70 to `kvm01` | + +The non-sequential `eth2` and `eth3` mapping follows the installed cabling and +live link state observed on the VP6630. + +## Switch port roles + +### `sw-mgmt01` + +| Port | Mode | VLAN | Endpoint | +| --- | --- | --- | --- | +| `1` | Trunk | 10, 70 | `gw01` | +| `2` | Access | 10 | `lab01` 10GbE RJ45 management | +| `3` | Access | 70 | `lab01` 2.5GbE RJ45 AMT | +| `4` | Access | 10 | `lab02` 10GbE RJ45 management | +| `5` | Access | 70 | `lab02` 2.5GbE RJ45 AMT | +| `6` | Access | 10 | `lab03` 10GbE RJ45 management | +| `7` | Access | 70 | `lab03` 2.5GbE RJ45 AMT | +| `8` | Access | 10 | `nas01` 5GbE RJ45 management | + +`sw-mgmt01` has 2.5GBASE-T access ports. The MS-02 10GbE management NICs and +the `nas01` 5GbE NIC therefore negotiate no faster than 2.5Gbps on this switch. + +### `sw-core01` + +Port 8 is the 802.1Q trunk to `gw01` and carries VLANs 10 and 40. VLAN 10 +provides the switch management path. Ports 1 through 6 connect the two SFP+ +interfaces from each MS-02, and port 7 connects the `nas01` 10GbE interface. +Those compute-facing links receive instance, cluster, or storage VLAN +membership only after the compute-network design assigns their roles; they are +not required for initial IncusOS management boot. + +## DHCP and DNS ownership + +`gw01` is available before managed hosts boot and owns cold-start DHCP and DNS +forwarding. Clients use their VLAN gateway address as the DNS resolver. + +`gw01` forwards `glab.lol` queries to the local mirror at `10.10.10.54`. The +mirror copies the private Route 53 zone from the AWS subnet-router service. +Other queries use the configured recursive resolvers. `gw01` does not run +PowerDNS and does not serve `lab.gilman.io`. + +## Routing protocols + +The initial core network uses connected and static routes only. It does not +configure BGP. A later design may add BGP when a concrete workload-advertisement +or dynamic-routing requirement exists. diff --git a/docs/docs/reference/networking/hardware.md b/docs/docs/reference/networking/hardware.md index 42df2fe..3eb9ae8 100644 --- a/docs/docs/reference/networking/hardware.md +++ b/docs/docs/reference/networking/hardware.md @@ -25,5 +25,5 @@ Models and physical specifications live in the [hardware inventory](../hardware-inventory.md). The [physical connection map](physical-connections.md) records every installed cable and its endpoint ports. The -[Lab v2 core network design](../../designs/drafts/lab-v2-core-network.md) defines +[Lab v2 core network design](../../designs/lab-v2-core-network.md) defines the logical topology and device boundaries. diff --git a/docs/docs/reference/networking/physical-connections.md b/docs/docs/reference/networking/physical-connections.md index 3064b9d..08a078b 100644 --- a/docs/docs/reference/networking/physical-connections.md +++ b/docs/docs/reference/networking/physical-connections.md @@ -19,12 +19,12 @@ no printed label. | `PHY-003` | `gw01` | `Port 2` | `sw-mgmt01` | `Port 1` | | `PHY-004` | `gw01` | `Port 3` | `pikvm01` | `Unlabeled Ethernet port` | | `PHY-005` | `gw01` | `Port 4` | `kvm01` | `Unlabeled Ethernet port` | -| `PHY-006` | `sw-mgmt01` | `Port 2` | `lab01` | `Top Port` | -| `PHY-007` | `sw-mgmt01` | `Port 3` | `lab01` | `Bottom Port` | -| `PHY-008` | `sw-mgmt01` | `Port 4` | `lab02` | `Top Port` | -| `PHY-009` | `sw-mgmt01` | `Port 5` | `lab02` | `Bottom Port` | -| `PHY-010` | `sw-mgmt01` | `Port 6` | `lab03` | `Top Port` | -| `PHY-011` | `sw-mgmt01` | `Port 7` | `lab03` | `Bottom Port` | +| `PHY-006` | `sw-mgmt01` | `Port 2` | `lab01` | `10GbE RJ45 (upper)` | +| `PHY-007` | `sw-mgmt01` | `Port 3` | `lab01` | `2.5GbE RJ45 (lower, vPro)` | +| `PHY-008` | `sw-mgmt01` | `Port 4` | `lab02` | `10GbE RJ45 (upper)` | +| `PHY-009` | `sw-mgmt01` | `Port 5` | `lab02` | `2.5GbE RJ45 (lower, vPro)` | +| `PHY-010` | `sw-mgmt01` | `Port 6` | `lab03` | `10GbE RJ45 (upper)` | +| `PHY-011` | `sw-mgmt01` | `Port 7` | `lab03` | `2.5GbE RJ45 (lower, vPro)` | | `PHY-012` | `sw-core01` | `Port 1` | `lab01` | `Right SFP 25G` | | `PHY-013` | `sw-core01` | `Port 2` | `lab01` | `Left SFP 25G` | | `PHY-014` | `sw-core01` | `Port 3` | `lab02` | `Right SFP 25G` | @@ -33,14 +33,8 @@ no printed label. | `PHY-017` | `sw-core01` | `Port 6` | `lab03` | `Left SFP 25G` | | `PHY-018` | `sw-core01` | `Port 7` | `nas01` | `10GB Port` | | `PHY-019` | `sw-mgmt01` | `Port 8` | `nas01` | `5GB Port` | +| `PHY-020` | `gw01` | `Port 1` | `sandbox01` | `2.5GbE RJ45` | -## Unconnected ports - -Only ports explicitly identified as unconnected are listed. - -| Device | Port | -| --- | --- | -| `gw01` | `Port 1` | IP addresses, VLANs, bonds, interface names, and link settings belong to the network configuration rather than this physical map. diff --git a/docs/docs/reference/networking/tailscale-policy.md b/docs/docs/reference/networking/tailscale-policy.md index 06d25d9..d041f0f 100644 --- a/docs/docs/reference/networking/tailscale-policy.md +++ b/docs/docs/reference/networking/tailscale-policy.md @@ -60,9 +60,8 @@ manual approval: | `192.168.1.0/24` | Home | | `192.168.2.0/24` | Home | -These ranges are authoritative for the tailnet today. The Lab v2 address plan is -not yet written; when it exists, it becomes the canonical source for lab ranges -and these entries must be reconciled with it. +The [network address and VLAN plan](address-plan.md) is canonical for lab +prefixes. Reconcile this list whenever that plan changes. ## Credentials diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 93e059f..61bb6ac 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -36,12 +36,12 @@ nav: - "ADR-0002: Manage the Tailnet Policy File with GitOps": decisions/0002-manage-tailscale-policy-with-gitops.md - "ADR-0003: Use AWS KMS with PGP Recovery for Secrets": decisions/0003-use-kms-with-pgp-recovery-for-secrets.md - Designs: - - Drafts: - - Lab v2 core network: designs/drafts/lab-v2-core-network.md + - Lab v2 core network: designs/lab-v2-core-network.md - Reference: - Device naming: reference/naming.md - Hardware inventory: reference/hardware-inventory.md - Networking: + - Address and VLAN plan: reference/networking/address-plan.md - Hardware: reference/networking/hardware.md - Physical connections: reference/networking/physical-connections.md - Tailscale policy: reference/networking/tailscale-policy.md