Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
status: accepted
date: 2026-08-14
---

# ADR-0001: Use VyOS for Layer 3 and Dedicated Switches for Layer 2

## Context and Problem Statement

The core network needs explicit ownership for switching, routing, and traffic
policy. The network uses a VyOS gateway, a MikroTik CRS309-1G-8S+IN core switch,
and a TRENDnet TEG-3102WS management/OOB switch. Which devices own each network
function?

## Decision Drivers

- Keep routed gateways and firewall policy on one device.
- Keep switch configuration focused on Layer 2 transport and physical links.
- Make the enforcement point for traffic between routed lab segments explicit.
- Carry MS-02 management/OOB traffic on its dedicated copper switch.
- Use the selected VyOS, MikroTik, and TRENDnet hardware.

## Considered Options

- Use VyOS for Layer 3 and dedicated switches for Layer 2.
- Use MikroTik for Layer 2 and Layer 3, with VyOS at the external edge.
- Use one flat Layer 2 lab network, with VyOS as its external gateway.

## Decision Outcome

Use VyOS for routed lab gateways, route selection, firewall policy, and NAT. Use
the MikroTik CRS309-1G-8S+IN for core Layer 2 switching and VLAN transport. Use
the TRENDnet TEG-3102WS for Layer 2 management/OOB connectivity from both
non-SFP NICs on each MS-02. The TEG-3102WS uplinks directly to the VP6630, which
provides the management/OOB gateway and firewall policy.

DHCP, DNS, and time-service ownership are outside the scope of this decision.

### Consequences

- Good, because routed traffic has one policy-enforcement point.
- Good, because each device has a distinct configuration boundary.
- Good, because the MS-02 management/OOB links use a dedicated physical switch.
- Bad, because traffic between routed lab segments depends on VyOS.
- Bad, because routed segments carried through the CRS309-1G-8S+IN depend on
its trunk to VyOS.
- Bad, because MS-02 management/OOB access depends on the TEG-3102WS and its
uplink to the VP6630.

### Confirmation

The implementation conforms to this decision when:

- VyOS owns the gateway address for each routed lab segment, including the
management/OOB segment.
- VyOS contains the firewall and NAT policy for routed lab traffic.
- Neither switch routes traffic between lab segments.
- CRS309-1G-8S+IN configuration defines core VLAN membership, trunks, access
ports, and physical link aggregation.
- TEG-3102WS configuration and cabling connect both non-SFP NICs from each MS-02
to the VP6630 management/OOB gateway.

Management addresses on the switches do not violate this decision.

## Pros and Cons of the Options

### VyOS Layer 3 and Dedicated Layer 2 Switches

- Good, because routing and firewall policy use the same configuration
boundary.
- Good, because the switches remain independent of higher-level traffic policy.
- Good, because core and management/OOB traffic use separate physical switches.
- Bad, because VyOS is on the forwarding path for all routed lab traffic.
- Bad, because each switch is a failure boundary for its connected links.

### MikroTik Layer 2 and Layer 3

- Good, because the core switch can route traffic without sending it through
the VyOS trunk.
- Bad, because firewall and routing ownership would be split between devices.
- Bad, because the network would need policy coordination between MikroTik and
VyOS.

### Flat Layer 2 Lab Network

- Good, because it requires fewer routed interfaces and policies.
- Bad, because it cannot enforce boundaries between lab network functions.
- Bad, because broadcasts and Layer 2 failures share one domain.

## More Information

See the [Lab v2 core network design](../designs/drafts/lab-v2-core-network.md)
and [hardware reference](../reference/hardware.md).
53 changes: 33 additions & 20 deletions docs/docs/designs/drafts/lab-v2-core-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ related-decisions:
## Summary

The core network uses a Minisforum VP6630 running VyOS for Layer 3 routing,
firewall policy, and NAT. A MikroTik switch handles Layer 2 switching and VLAN
transport. A MikroTik CCR2004 connects the lab to the home network and the
internet.
firewall policy, and NAT. A MikroTik CRS309-1G-8S+IN handles core Layer 2
switching and VLAN transport. A TRENDnet TEG-3102WS connects both non-SFP NICs
from each MS-02 to the VP6630 for management/OOB traffic. A MikroTik CCR2004
connects the lab to the home network and the internet.

This design defines device responsibilities, logical topology, configuration
requirements, failure boundaries, and verification criteria. Address
Expand All @@ -26,7 +27,8 @@ ownership are outside this document.
## Goals

- Keep routing and traffic policy on VyOS.
- Keep VLAN transport and physical switching on MikroTik.
- Keep core VLAN transport and physical switching on the CRS309-1G-8S+IN.
- Carry MS-02 management/OOB traffic through the TEG-3102WS.
- Route home-to-lab traffic without source NAT.
- Apply source NAT to lab-to-internet traffic on VyOS.
- Store network-device configuration in version control.
Expand All @@ -51,24 +53,28 @@ ownership are outside this document.
flowchart LR
HOME[Home network] --> CCR[CCR2004]
CCR -->|Routed transit| VYOS[VP6630 running VyOS]
VYOS -->|802.1Q trunk| SWITCH[MikroTik switch]
SWITCH --> SEGMENTS[Lab network segments]
VYOS -->|802.1Q trunk| CRS[CRS309-1G-8S+IN]
CRS --> SEGMENTS[Lab network segments]
VYOS -->|Management/OOB uplink| TEG[TEG-3102WS]
TEG -->|Two non-SFP NICs per node| MS02[MS-02 nodes]
```

The CCR2004 routes traffic between the home network and the VyOS transit
interface. VyOS routes lab prefixes, applies firewall policy, and performs
source NAT for internet egress. The MikroTik switch carries VLANs between VyOS
and connected lab devices.
source NAT for internet egress. The CRS309-1G-8S+IN carries lab VLANs between
VyOS and connected lab devices. The TEG-3102WS connects directly to the VP6630
and carries management/OOB traffic for both non-SFP NICs on each MS-02.

## Device Responsibilities

| Device | Responsibilities |
| --- | --- |
| MikroTik CCR2004 | Home-network routing, internet access, and the upstream side of the routed lab transit |
| Minisforum VP6630 running VyOS | Lab gateways, route selection, firewall policy, source NAT, and the downstream side of the routed transit |
| MikroTik switch | VLAN transport, access ports, trunks, and physical link aggregation |
| Minisforum VP6630 running VyOS | Lab gateways, route selection, firewall policy, source NAT, the downstream side of the routed transit, and the management/OOB gateway |
| MikroTik CRS309-1G-8S+IN | Core VLAN transport, access ports, trunks, and physical link aggregation |
| TRENDnet TEG-3102WS | Layer 2 management/OOB connectivity for both non-SFP NICs on each MS-02 and a direct uplink to the VP6630 |

[ADR-0001](../../decisions/0001-use-vyos-for-layer-3-and-mikrotik-for-layer-2.md)
[ADR-0001](../../decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md)
defines the Layer 2 and Layer 3 boundary.

## Routing and NAT
Expand All @@ -78,7 +84,7 @@ The routing design has these invariants:
- The CCR2004 has routes for lab prefixes through the VyOS transit address.
- VyOS uses the CCR2004 transit address as its default route.
- VyOS owns the gateway address for every routed lab segment.
- MikroTik does not route between lab segments.
- The CRS309-1G-8S+IN and TEG-3102WS do not route between lab segments.
- Home-to-lab traffic retains its original source address.
- VyOS applies source NAT to lab-to-internet traffic.
- Firewall rules distinguish new connections from established reply traffic.
Expand All @@ -92,6 +98,7 @@ VyOS enforces policy for:
- Lab segments to the internet
- Traffic between routed lab segments
- Traffic addressed to VyOS
- Management/OOB traffic through the TEG-3102WS
- Management traffic addressed to network devices

Each firewall rule identifies the source, destination, protocol, destination
Expand All @@ -101,8 +108,8 @@ the reverse direction.

## Configuration Requirements

VyOS and MikroTik each have one version-controlled configuration source. The
deployment process:
VyOS and both switches each have one version-controlled configuration source.
The deployment process:

1. Renders the effective configuration.
2. Validates syntax and policy before deployment.
Expand All @@ -117,20 +124,23 @@ Drift detection compares each running configuration with its repository source.
## Management and Recovery

Firewall policy limits routine management access to approved source networks.
The TEG-3102WS carries management/OOB traffic from both non-SFP NICs on each
MS-02 directly to the VP6630.

Each device has a recovery path that remains available when its production
configuration or primary network link fails. Recovery credentials do not reside
in device configuration committed to the repository.
Each network device has a recovery path that remains available when its
production configuration or primary network link fails. Recovery credentials do
not reside in device configuration committed to the repository.

## Failure Boundaries

| Failure | Effect |
| --- | --- |
| CCR2004 failure | The lab loses home-network and internet connectivity. Internal lab switching and routing remain available. |
| VP6630 or VyOS failure | Routed lab segments lose their gateways, inter-segment routing, policy enforcement, and internet egress. |
| MikroTik switch failure | Devices connected through the switch lose Layer 2 connectivity. |
| VP6630 or VyOS failure | Routed lab segments lose their gateways, inter-segment routing, policy enforcement, management/OOB gateway, and internet egress. |
| CRS309-1G-8S+IN failure | Devices connected through the core switch lose Layer 2 connectivity. |
| TEG-3102WS or its VP6630 uplink failure | Both non-SFP NICs on each MS-02 lose management/OOB connectivity. |
| Routed transit failure | Home-to-lab and lab-to-internet traffic stop. Internal lab traffic remains available within its unaffected Layer 2 and Layer 3 paths. |
| VyOS-to-MikroTik trunk failure | VLANs carried by the trunk lose their VyOS gateways. |
| VyOS-to-CRS309 trunk failure | VLANs carried by the trunk lose their VyOS gateways. |
| Invalid configuration | Deployment verification fails and the previous configuration is restored. |

## Verification
Expand All @@ -146,5 +156,8 @@ A deployment is valid when the observed behavior matches these checks:
- Each permitted firewall flow succeeds.
- Each denied firewall flow fails.
- Established reply traffic succeeds without enabling a new reverse flow.
- Both non-SFP NICs on each MS-02 connect through the TEG-3102WS.
- MS-02 management/OOB traffic reaches the VP6630 through the TEG-3102WS
uplink.
- Management access succeeds only from approved source networks.
- A failed deployment restores the previous configuration.
9 changes: 5 additions & 4 deletions docs/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,18 @@ description: Architecture and operating documentation for the GilmanLab core net
# GilmanLab Networking

This repository defines the GilmanLab core network. VyOS handles routing,
firewall policy, and NAT. MikroTik handles switching and VLAN transport.
firewall policy, and NAT. A MikroTik CRS309-1G-8S+IN handles core switching and
VLAN transport. A TRENDnet TEG-3102WS handles MS-02 management/OOB switching.

## Documents

- [Lab v2 core network design](designs/drafts/lab-v2-core-network.md) defines
the core topology, device boundaries, configuration requirements, and
verification criteria.
- [ADR-0001: Use VyOS for Layer 3 and MikroTik for Layer 2](decisions/0001-use-vyos-for-layer-3-and-mikrotik-for-layer-2.md)
- [ADR-0001: Use VyOS for Layer 3 and Dedicated Switches for Layer 2](decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md)
assigns routing and switching responsibilities.
- [Hardware](reference/hardware.md) identifies the core network devices and
their roles.
- [Hardware](reference/hardware.md) identifies the core network devices,
connections, and roles.

Compute platforms, workload networking, and application delivery are outside
the core network documentation.
14 changes: 12 additions & 2 deletions docs/docs/reference/hardware.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Hardware
description: Core network devices and their assigned roles.
description: Core network devices, connections, and assigned roles.
---

# Hardware
Expand All @@ -10,13 +10,23 @@ description: Core network devices and their assigned roles.
| Component | Platform | Role |
| --- | --- | --- |
| Lab gateway | Minisforum VP6630 running VyOS | Routes lab networks, enforces firewall policy, and performs source NAT |
| Lab switch | MikroTik switch | Carries VLANs and connects lab devices at Layer 2 |
| Core switch | MikroTik `CRS309-1G-8S+IN` | Carries lab VLANs and connects lab devices at Layer 2 |
| Management/OOB switch | TRENDnet `TEG-3102WS` | Connects both non-SFP NICs from each MS-02 for management/OOB traffic |

## External dependency

| Component | Platform | Role |
| --- | --- | --- |
| Home router | MikroTik CCR2004 | Routes the home network, provides internet access, and terminates the upstream side of the routed lab transit |

## Connections

| Endpoints | Connection |
| --- | --- |
| CCR2004 to VP6630 | Routed lab transit |
| VP6630 to CRS309-1G-8S+IN | 802.1Q trunk for lab VLANs |
| VP6630 to TEG-3102WS | Direct management/OOB uplink |
| TEG-3102WS to each MS-02 | Two non-SFP NICs carrying management/OOB traffic |

See the [Lab v2 core network design](../designs/drafts/lab-v2-core-network.md)
for the topology and device boundaries.
2 changes: 1 addition & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ theme:
nav:
- Home: index.md
- Decisions:
- "ADR-0001: Use VyOS for Layer 3 and MikroTik for Layer 2": decisions/0001-use-vyos-for-layer-3-and-mikrotik-for-layer-2.md
- "ADR-0001: Use VyOS for Layer 3 and Dedicated Switches for Layer 2": decisions/0001-use-vyos-for-layer-3-and-switches-for-layer-2.md
- Designs:
- Drafts:
- Lab v2 core network: designs/drafts/lab-v2-core-network.md
Expand Down