Compare commits

..

No commits in common. "main" and "0.1.0" have entirely different histories.

24 changed files with 578 additions and 1766 deletions

View file

@ -1,8 +1,8 @@
# labctl
Monte un **lab réseau virtuel** à partir d'une topologie YAML : des VM QEMU/KVM reliées par des
segments L2 (des bridges de l'hôte), des routeurs qui font passerelle — en VRRP quand ils sont
plusieurs — et NAT, FRR installé et configuré sur les nœuds qui le demandent, et une provision libre (scripts, fichiers, services à vérifier) pour installer ce
segments L2, des switchs qui font passerelle et NAT, FRR installé et configuré sur les nœuds qui
le demandent, et une provision libre (scripts, fichiers, services à vérifier) pour installer ce
que l'on veut tester. Le plan est **déterministe** : même fichier, mêmes adresses, MAC et ports.
`labctl` tourne sur tout hôte Linux avec KVM (virtualisation imbriquée si l'hôte est lui-même une
@ -32,43 +32,36 @@ labctl status | down | ssh <nœud> [commande] [-run rép]
```
Par défaut : `-run ~/lab-run`, `-cache ~/.cache/labctl`, `-timeout 20m`. `up` télécharge et vérifie
les images (`SHA512SUMS`), crée des disques neufs, les bridges des segments et les taps des nœuds,
démarre les nœuds (routeurs d'abord), attend la fin de cloud-init puis vérifie les services de
chaque nœud ; `down` arrête les nœuds et supprime bridges et taps. `up` et `down` tournent en
root. Un fichier invalide est refusé avec
les images (`SHA512SUMS`), crée des disques neufs, démarre les nœuds (switchs d'abord), attend la
fin de cloud-init puis vérifie les services de chaque nœud. Un fichier invalide est refusé avec
toutes ses erreurs et un code de sortie 1.
## Topologie
Voir `examples/evpn-2hv/` : deux routeurs en VRRP, un route reflector et deux hôtes en EVPN.
Voir `examples/evpn-2hv/` : un switch, un route reflector et deux hôtes en EVPN.
- `images` : `url` et `sums` (HTTPS).
- `segments` : `cidr` IPv4 (/8 à /30), `mtu` (9000 par défaut), `routers` (facultatif, par ordre
de priorité : aucun = L2 pur, un = passerelle sur son interface, plusieurs = passerelle en
VRRP par FRR). Nom ≤ 12 caractères. Chaque segment est un bridge `lb-<segment>` de l'hôte.
- `nodes` : `role` (`router`, `rr` ou `host`), `image`, `cpus`, `memory` (Mio), `segments`,
`addresses` (adresse fixée par segment). Un routeur est un nœud comme les autres, attaché aux
segments dont il est routeur. Champs facultatifs :
- `segments` : `switch`, `cidr` IPv4 (/8 à /30), `mtu` (9000 par défaut). Nom ≤ 12 caractères.
- `nodes` : `role` (`switch`, `rr` ou `host`), `image`, `cpus`, `memory` (Mio), `segments`,
`addresses` (adresse fixée par segment). Champs facultatifs :
- `secondary` — adresses supplémentaires hors du CIDR du segment ;
- `loopback` — adresse sur une interface `dummy` `lo1` ;
- `frr` — un `frr.conf` déposé tel quel ; FRR est installé au premier démarrage (`bgpd`, et
`bfdd` sur `router` et `rr`, `vrrpd` sur les routeurs en VRRP) ;
`bfdd` sur `switch` et `rr`) ;
- `provision` / `post_provision` — scripts exécutés **avant** / **après** FRR ;
- `files` — `{<nom>: <chemin>}`, fichiers **ou répertoires** gravés dans un ISO propre au nœud et
recopiés dans `/etc/lab/files/<nom>` avant les scripts (aucune limite de taille cloud-init) ;
- `services` — unités systemd que `up` vérifie actives.
Les chemins sont relatifs au fichier de topologie. Les scripts reçoivent `LAB_NODE`,
`LAB_UPLINK` (interface qui porte la route par défaut : premier segment qui a une passerelle,
`mgmt0` pour un routeur)
et `LAB_FILES`. Ordre au premier démarrage : service du routeur ou de la loopback, `provision`, FRR,
`LAB_UPLINK` (interface du premier segment, qui porte la route par défaut) et `LAB_FILES`.
Ordre au premier démarrage : script du switch ou de la loopback, `provision`, FRR,
`post_provision` — dans un seul script en `set -eu`, la première étape en échec met cloud-init en
erreur.
Déduit de façon déterministe : passerelle d'un segment = première adresse (s'il a un routeur) ;
adresses suivantes dans l'ordre de déclaration des nœuds ; MAC `02:4c:<nœud>:<nœud>:<segment>:00` ;
tap `lt<i>` par port ; SSH d'administration sur `127.0.0.1:<2200 + rang>`. **Réordonner les
nœuds change les adresses.** Migration depuis 0.1 (rôle `switch`) : voir le guide.
Déduit de façon déterministe : passerelle d'un segment = première adresse (portée par le switch) ;
adresses suivantes dans l'ordre de déclaration des nœuds ; MAC `02:4c:<nœud>:<nœud>:<segment>:<côté>` ;
SSH d'administration sur `127.0.0.1:<2200 + rang>`. **Réordonner les nœuds change les adresses.**
## Hôte loué chez Scaleway : `scripts/lab-host.sh`

View file

@ -26,17 +26,8 @@ const (
var (
execve = syscall.Exec
procDir = "/proc"
geteuid = os.Geteuid
)
func requireRoot(command string, stderr io.Writer) bool {
if geteuid() == 0 {
return true
}
fmt.Fprintf(stderr, "labctl: %s must run as root: it manages the bridges and taps of the lab on this host\n", command)
return false
}
func defaultDir(parts ...string) string {
home, err := os.UserHomeDir()
if err != nil {
@ -90,9 +81,6 @@ func upCmd(args []string, stdout, stderr io.Writer) int {
fs.Usage()
return 2
}
if !requireRoot("up", stderr) {
return 1
}
cache, err := filepath.Abs(*cacheDir)
if err != nil {
fmt.Fprintf(stderr, "labctl: %v\n", err)
@ -166,9 +154,6 @@ func downCmd(args []string, stdout, stderr io.Writer) int {
fs.Usage()
return 2
}
if !requireRoot("down", stderr) {
return 1
}
l, ok := current(*runDir, stdout, stderr)
if !ok {
return 1

View file

@ -17,12 +17,6 @@ func runLab(args ...string) (int, string, string) {
return code, stdout.String(), stderr.String()
}
func asRoot(t *testing.T) {
t.Helper()
geteuid = func() int { return 0 }
t.Cleanup(func() { geteuid = os.Geteuid })
}
func TestRun_UsageOnMissingArguments(t *testing.T) {
for _, args := range [][]string{nil, {"plan"}, {"deploy", "x.yml"}, {"plan", "a.yml", "b.yml"}, {"render", "a.yml", "dir"}, {"render", "-key", "k.pub", "a.yml"}, {"render", "-bogus"}} {
code, _, stderr := runLab(args...)
@ -38,9 +32,9 @@ func TestRun_PlanOfTheShippedExampleTopology(t *testing.T) {
t.Fatalf("code %d, stderr %s", code, stderr)
}
for _, want := range []string{
"lab evpn-2hv: nodes 5, segments 1, ports 5",
"segment underlay: 192.168.14.0/24, mtu 9000, bridge lb-underlay, gateway 192.168.14.1 by vrrp 1 on r1, r2",
"hv2 underlay 192.168.14.12/24 02:4c:00:04:00:00 lt4",
"lab evpn-2hv: nodes 4, segments 1, cables 3",
"gateway 192.168.14.1",
"hv2 underlay 192.168.14.12/24 02:4c:00:03:00:00 20004 <-> sw1 p2",
"rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf",
} {
if !strings.Contains(stdout, want) {
@ -55,10 +49,10 @@ func TestRun_InvalidTopologyExitsWithErrors(t *testing.T) {
images:
deb: { url: https://example.invalid/a, sums: https://example.invalid/b }
segments:
under: { routers: [sw], cidr: 10.0.0.0/31 }
under: { switch: sw, cidr: 10.0.0.0/31 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
rr: { role: spine, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
rr: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
`
if err := os.WriteFile(path, []byte(doc), 0o600); err != nil {
t.Fatal(err)
@ -67,7 +61,7 @@ nodes:
if code != 1 || stdout != "" {
t.Fatalf("code %d, stdout %q", code, stdout)
}
for _, want := range []string{path, `role "spine"`, "prefix length out of range"} {
for _, want := range []string{path, `role "router"`, "prefix length out of range"} {
if !strings.Contains(stderr, want) {
t.Errorf("stderr does not mention %q:\n%s", want, stderr)
}
@ -92,7 +86,7 @@ func TestRun_RenderWritesEveryNodeFiles(t *testing.T) {
if code != 0 {
t.Fatalf("code %d, stderr %s", code, stderr)
}
for _, node := range []string{"r1", "r2", "rr1", "hv1", "hv2"} {
for _, node := range []string{"sw1", "rr1", "hv1", "hv2"} {
if !strings.Contains(stdout, filepath.Join(out, node)) {
t.Errorf("stdout does not list %s:\n%s", node, stdout)
}
@ -111,7 +105,7 @@ func TestRun_RenderWritesEveryNodeFiles(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(args), "\n-netdev\nuser,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2203-:22\n-device\nvirtio-net-pci,netdev=mgmt0,mac=02:4d:00:03:00:00,romfile=\n-netdev\ntap,id=underlay,ifname=lt3,script=no,downscript=no\n") {
if !strings.Contains(string(args), "\n-netdev\nuser,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22\n") {
t.Errorf("qemu.args is not one argument per line:\n%s", args)
}
userData, err := os.ReadFile(filepath.Join(out, "hv1", "user-data"))
@ -156,7 +150,6 @@ func TestRun_LifecycleUsage(t *testing.T) {
}
func TestRun_CommandsWithoutALabInTheRunDir(t *testing.T) {
asRoot(t)
run := t.TempDir()
for _, cmd := range []string{"status", "down", "ssh"} {
args := []string{cmd, "-run", run}
@ -173,14 +166,10 @@ func TestRun_CommandsWithoutALabInTheRunDir(t *testing.T) {
func TestRun_StatusOfAStoppedLab(t *testing.T) {
code, stdout, stderr := runLab("status", "-run", savedLab(t))
want := `node role state pid ssh
r1 router stopped - 127.0.0.1:2200
r2 router stopped - 127.0.0.1:2201
rr1 rr stopped - 127.0.0.1:2202
hv1 host stopped - 127.0.0.1:2203
hv2 host stopped - 127.0.0.1:2204
segment bridge state gateway
underlay lb-underlay absent 192.168.14.1
sw1 switch stopped - 127.0.0.1:2200
rr1 rr stopped - 127.0.0.1:2201
hv1 host stopped - 127.0.0.1:2202
hv2 host stopped - 127.0.0.1:2203
`
if code != 0 || stdout != want {
t.Errorf("code %d, stderr %q, stdout:\n%s", code, stderr, stdout)
@ -188,7 +177,6 @@ underlay lb-underlay absent 192.168.14.1
}
func TestRun_DownOfAStoppedLab(t *testing.T) {
asRoot(t)
code, stdout, stderr := runLab("down", "-run", savedLab(t))
if code != 0 || stdout != "" || stderr != "" {
t.Errorf("code %d, stdout %q, stderr %q", code, stdout, stderr)
@ -220,7 +208,7 @@ func TestRun_SSHExecsSSHWithTheNodePort(t *testing.T) {
"-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=/dev/null",
"-o", "LogLevel=ERROR",
"-p", "2204",
"-p", "2203",
"debian@127.0.0.1",
"ip", "-br", "a",
}
@ -237,7 +225,6 @@ func TestRun_SSHUnknownNode(t *testing.T) {
}
func TestRun_UpRefusesAnInvalidTopologyAndKeepsTheSavedOne(t *testing.T) {
asRoot(t)
run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
bad := filepath.Join(t.TempDir(), "bad.yml")
@ -275,7 +262,6 @@ func TestIsTerminal_DevNullAndPipesAreNotTerminals(t *testing.T) {
}
func TestRun_UpRefusesToReplaceARunningLab(t *testing.T) {
asRoot(t)
run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
proc := t.TempDir()
@ -327,8 +313,7 @@ func TestRun_RenderShipsTheExampleFRRConfigs(t *testing.T) {
t.Fatalf("code %d, stderr %s", code, stderr)
}
for node, wants := range map[string][]string{
"r1": {"router bgp 65100", "interface underlay\n", " vrrp 1 priority 200\n", " vrrp 1 ip 192.168.14.1\n"},
"r2": {"hostname r2", " vrrp 1 priority 190\n", " vrrp 1 ip 192.168.14.1\n"},
"sw1": {"router bgp 65100"},
"rr1": {"bgp listen range 192.168.14.0/24 peer-group fabric"},
"hv1": {"hostname hv1", "neighbor 10.255.255.1 peer-group fabric"},
"hv2": {"hostname hv2", "neighbor 10.255.255.1 peer-group fabric"},
@ -374,9 +359,9 @@ func TestRun_RenderReportsAMissingFRRConfig(t *testing.T) {
images:
deb: { url: https://example.invalid/deb.qcow2, sums: https://example.invalid/SHA512SUMS }
segments:
underlay: { routers: [sw1], cidr: 10.1.0.0/24 }
underlay: { switch: sw1, cidr: 10.1.0.0/24 }
nodes:
sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay], frr: frr/absent.conf }
sw1: { role: switch, image: deb, cpus: 1, memory: 512, frr: frr/absent.conf }
hv1: { role: host, image: deb, cpus: 1, memory: 512, segments: [underlay] }
`
if err := os.WriteFile(topo, []byte(doc), 0o600); err != nil {
@ -387,34 +372,3 @@ nodes:
t.Errorf("code %d, stderr %q", code, stderr)
}
}
func TestRun_UpAndDownRequireRoot(t *testing.T) {
geteuid = func() int { return 1000 }
t.Cleanup(func() { geteuid = os.Geteuid })
run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
other := filepath.Join(t.TempDir(), "other.yml")
if err := os.WriteFile(other, bytes.ReplaceAll(before, []byte("evpn-2hv"), []byte("other")), 0o600); err != nil {
t.Fatal(err)
}
for name, args := range map[string][]string{
"up": {"up", "-run", run, other},
"down": {"down", "-run", run},
} {
code, stdout, stderr := runLab(args...)
if code != 1 || stdout != "" || !strings.Contains(stderr, "labctl: "+name+" must run as root") {
t.Errorf("%s: code %d, stdout %q, stderr %q", name, code, stdout, stderr)
}
}
if after, _ := os.ReadFile(filepath.Join(run, "topology.yml")); string(after) != string(before) {
t.Error("the saved topology was replaced without root")
}
}
func TestRun_StatusAndSSHDoNotRequireRoot(t *testing.T) {
geteuid = func() int { return 1000 }
t.Cleanup(func() { geteuid = os.Geteuid })
if code, _, stderr := runLab("status", "-run", savedLab(t)); code != 0 || strings.Contains(stderr, "root") {
t.Errorf("status: code %d, stderr %q", code, stderr)
}
}

View file

@ -141,37 +141,31 @@ celui d'une release, vérifié) au lieu de compiler.
## Topologie
Un lab est décrit par un fichier YAML : des **nœuds** (les VM) et des **segments** (des réseaux
L2). Exemple livré : [`examples/evpn-2hv/topology.yml`](../examples/evpn-2hv/topology.yml) — deux
routeurs en VRRP, un route reflector et deux hôtes en EVPN.
Un lab est décrit par un fichier YAML : des **nœuds** (les VM) et des **segments** (des réseaux L2
portés par un switch). Exemple livré : [`examples/evpn-2hv/topology.yml`](../examples/evpn-2hv/topology.yml).
Chaque segment est un **bridge Linux sur l'hôte du lab** (`lb-<segment>`), et chaque nœud qui y
est attaché y branche une carte par un tap (`lt<i>`). Le L2 ne dépend donc d'aucune VM : la
perte d'un routeur ne coupe pas le trafic direct entre les autres nœuds du segment. Un
**routeur** est un nœud comme les autres, qui porte en plus la passerelle et le NAT des segments
dont il est routeur.
Chaque nœud non-switch est relié au switch de chacun de ses segments par un câble virtuel QEMU ;
le switch met ces câbles dans un bridge et porte la passerelle du segment.
Ce que le fichier déclare :
**`images`** — `url` de l'image qcow2 et `sums` du fichier de sommes à vérifier, tous deux en
`https://`.
**`segments`** — `cidr` IPv4 entre `/8` et `/30`, `mtu` facultatif — 9000 par défaut, entre 1280
et 9000 —, et `routers`, facultatif : les nœuds de rôle `router` qui portent la passerelle du
segment, **dans l'ordre de priorité**. Sans routeur, le segment est un L2 pur (lien entre VM,
lien inter-cluster) : pas de passerelle, pas de NAT. Nom : 12 caractères au plus, minuscules et
chiffres, parce qu'il devient le nom d'interface dans les VM et, préfixé de `lb-`, celui du bridge
sur l'hôte (15 caractères au plus sous Linux).
**`segments`** — `switch` (un nœud de rôle `switch`), `cidr` IPv4 entre `/8` et `/30`, `mtu`
facultatif — 9000 par défaut, entre 1280 et 9000. Nom : 12 caractères au plus, minuscules et
chiffres, parce qu'il devient le nom d'interface dans les VM et, préfixé de `br-`, celui du
bridge (15 caractères au plus sous Linux).
**`nodes`** — `role` (`router`, `rr` ou `host`), `image`, `cpus`, `memory` en Mio (256 au
**`nodes`** — `role` (`switch`, `rr` ou `host`), `image`, `cpus`, `memory` en Mio (256 au
moins), `segments` auxquels le nœud est relié, et `addresses` pour fixer l'adresse d'un nœud sur
un segment (`addresses: {underlay: 192.168.14.50}`). Un routeur listé dans `segments.<s>.routers`
doit être attaché à `<s>`. Champs facultatifs :
un segment (`addresses: {underlay: 192.168.14.50}`). Un switch ne déclare ni `segments` ni
`addresses` : il porte ceux dont il est le `switch`. Champs facultatifs :
- `secondary` — des adresses supplémentaires par segment, avec leur longueur de préfixe
(`secondary: {underlay: [169.254.0.3/28]}`), posées sur la même interface que l'adresse
principale : même L2, même MAC. Elles doivent être **hors** du CIDR du segment, pour ne
jamais croiser l'attribution automatique ;
jamais croiser l'attribution automatique. Sur un switch, elles vont sur le bridge du segment ;
- `loopback` — une adresse sur une interface `dummy` nommée `lo1` (`loopback: 10.255.255.1/32`) ;
- `frr` — le chemin d'un `frr.conf`, relatif au fichier de topologie : FRR est installé au
démarrage et la configuration déposée **telle quelle** (voir [FRR](#frr)) ;
@ -179,112 +173,67 @@ doit être attaché à `<s>`. Champs facultatifs :
démarrage **avant** FRR ;
- `post_provision` — un second script, exécuté **après** FRR ;
- `files` — des fichiers **ou répertoires** déposés dans `/etc/lab/files/<nom>` avant les
scripts, sous la forme `{<nom>: <chemin>}` (`files: {motd: motd}`). Ils voyagent dans un
`files.iso` propre au nœud (volume `LABFILES`), attaché comme second lecteur et recopié au
premier démarrage : la taille n'est pas limitée par cloud-init, et un binaire de plusieurs
dizaines de Mio passe ;
scripts, sous la forme `{<nom>: <chemin>}` (`files: {motd: motd}`). Ils voyagent
dans un `files.iso` propre au nœud (volume `LABFILES`), attaché comme second lecteur et
recopié au premier démarrage : la taille n'est pas limitée par cloud-init, et un binaire de
plusieurs dizaines de Mio passe ;
- `services` — les unités systemd que `labctl up` vérifie actives sur le nœud.
Les scripts reçoivent `LAB_NODE` (nom du nœud), `LAB_UPLINK` (interface qui porte la route par
défaut : celle du premier segment du nœud qui a une passerelle, `mgmt0` pour un routeur) et
`LAB_FILES`
(`/etc/lab/files`). L'outil ne sait rien de ce qu'ils installent.
Les scripts reçoivent `LAB_NODE` (nom du nœud), `LAB_UPLINK` (interface de son premier segment,
qui porte la route par défaut) et `LAB_FILES` (`/etc/lab/files`). L'outil ne sait rien de ce
qu'ils installent.
**Un seul script de provisionnement par nœud.** cloud-init exécute `runcmd` comme un script `sh`
sans `set -e` : seule la dernière commande compterait, et une provision en échec suivie d'un FRR
installé avec succès passerait pour un démarrage réussi. Chaque nœud reçoit donc
`/usr/local/sbin/lab-provision`, en `set -eu`, qui enchaîne ses étapes — service du routeur ou de
`/usr/local/sbin/lab-provision`, en `set -eu`, qui enchaîne ses étapes — script du switch ou de
la loopback, `provision`, FRR, `post_provision` ; `runcmd` n'appelle que lui, et la première
étape en échec met cloud-init en erreur.
`mgmt0` et `lo1` sont réservés : aucun segment ne peut porter ces noms.
### Passerelle et routeurs
| Routeurs du segment | Passerelle |
|---|---|
| aucun | aucune : les nœuds n'y ont pas de route par défaut |
| un | la première adresse du CIDR, posée sur l'interface du routeur, en plus de la sienne |
| deux ou plus | la première adresse du CIDR, **virtuelle**, portée en VRRP v3 par `vrrpd` de FRR |
Avec plusieurs routeurs, `labctl` prépare tout sur chacun : interface `macvlan` `vr<segment>`
avec la MAC VRRP (`00:00:5e:00:01:<vrid>`) et la passerelle, créée **coupée** (`protodown on`) :
seul `vrrpd` la rétablit, quand il devient maître ; `arp_ignore=1` et `arp_announce=2`
sur l'interface du segment (sans quoi elle répondrait aux ARP de la passerelle avec sa propre
MAC), FRR installé avec `vrrpd`, et la configuration VRRP **ajoutée** à la fin de son `frr.conf`
(ou d'un `frr.conf` minimal s'il n'en déclare pas) :
```text
interface underlay
vrrp 1 version 3
vrrp 1 priority 200
no vrrp 1 preempt
vrrp 1 ip 192.168.14.1
exit
```
Le VRID est le rang du segment plus un (d'où au plus 255 segments à plusieurs routeurs, parmi les
premiers) ; la priorité vaut 200 pour le premier routeur de la liste, puis 10 de moins pour
chacun des suivants. **Pas de préemption** : un routeur qui revient reste en attente, plutôt que
de reprendre la passerelle avant d'avoir ses routes BGP — un prétest l'a montré, la préemption
coupait les sessions qui passaient par lui. Conséquence : le maître est le premier routeur dont
VRRP démarre, la priorité ne départageant que des démarrages simultanés ; un scénario qui arrête
« le maître » doit le demander (`vtysh -c 'show vrrp'`), pas le supposer. Bascule mesurée en
prétest : environ 2,6 s de perte (trois annonces manquées à une seconde d'intervalle).
Chaque routeur masque (NAT nftables) vers `mgmt0` les segments dont il est routeur, et sort par
le NAT de QEMU. Un nœud qui n'est pas routeur prend sa route par défaut et ses DNS (`1.1.1.1`,
`8.8.8.8`) sur le premier de ses segments qui a une passerelle.
### Plan déterministe
Ce que l'outil déduit du fichier — même fichier, même plan :
Ce que l'outil en déduit, de façon déterministe — même fichier, même plan :
| | |
|---|---|
| Passerelle d'un segment | la première adresse du CIDR, si le segment a au moins un routeur |
| Adresse d'un nœud | les suivantes (dès la première sans routeur), **dans l'ordre de déclaration des nœuds** ; une adresse fixée par `addresses` est réservée d'abord et sautée par l'attribution automatique |
| Bridge | `lb-<segment>` sur l'hôte du lab |
| Ports | un par couple (segment, nœud), segments puis nœuds dans l'ordre de déclaration ; le port *i* est le tap `lt<i>` |
| MAC | `02:4c:<nœud>:<nœud>:<segment>:00` — préfixe localement administré, rang du nœud sur deux octets, rang du segment |
| Interfaces | dans la VM, le nom du segment |
| Passerelle d'un segment | la première adresse du CIDR, portée par le switch sur `br-<segment>` |
| Adresse d'un nœud | les suivantes, **dans l'ordre de déclaration des nœuds** ; une adresse fixée par `addresses` est réservée d'abord et sautée par l'attribution automatique |
| Câbles | un par couple (segment, nœud), segments puis nœuds dans l'ordre de déclaration ; le câble *i* utilise les ports UDP `20000 + 2i` (côté nœud) et `20001 + 2i` (côté switch) |
| MAC | `02:4c:<nœud>:<nœud>:<segment>:<côté>` — préfixe localement administré, rang du nœud sur deux octets, rang du segment, `00` côté nœud et `01` côté switch |
| Interfaces | côté nœud, le nom du segment ; côté switch, `p<i>`, du rang du câble |
| SSH d'administration | `127.0.0.1:<2200 + rang du nœud>` sur l'hôte du lab |
> [!WARNING]
> Réordonner les nœuds ou les segments dans le fichier **change les adresses, les MAC, les taps et
> les ports**. C'est assumé pour un lab ; `labctl plan` montre le résultat avant tout lancement.
> Réordonner les nœuds ou les segments dans le fichier **change les adresses, les MAC et les
> ports**. C'est assumé pour un lab ; `labctl plan` montre le résultat avant tout lancement.
Limites : 1000 nœuds, 256 segments.
Limites : 1000 nœuds, 256 segments, et autant de câbles que la plage UDP le permet (22 768).
`labctl plan` valide le fichier et affiche le plan, sans rien lancer :
```text
$ go run ./cmd/labctl plan examples/evpn-2hv/topology.yml
lab evpn-2hv: nodes 5, segments 1, ports 5
lab evpn-2hv: nodes 4, segments 1, cables 3
nodes
name role image cpus memory ssh
r1 router debian12 1 1024 MiB 127.0.0.1:2200
r2 router debian12 1 1024 MiB 127.0.0.1:2201
rr1 rr debian12 1 1024 MiB 127.0.0.1:2202
hv1 host debian12 2 2048 MiB 127.0.0.1:2203
hv2 host debian12 2 2048 MiB 127.0.0.1:2204
sw1 switch debian12 2 1024 MiB 127.0.0.1:2200
rr1 rr debian12 1 1024 MiB 127.0.0.1:2201
hv1 host debian12 2 2048 MiB 127.0.0.1:2202
hv2 host debian12 2 2048 MiB 127.0.0.1:2203
roles
name loopback secondary frr provision post_provision files services
r1 - underlay 169.254.0.1/28 r1.conf - - - -
r2 - underlay 169.254.0.2/28 r2.conf - - - -
sw1 - underlay 169.254.0.1/28 sw1.conf - - - -
rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf - - - -
hv1 - - hv1.conf host.sh - motd -
hv2 - - hv2.conf host.sh - motd -
segment underlay: 192.168.14.0/24, mtu 9000, bridge lb-underlay, gateway 192.168.14.1 by vrrp 1 on r1, r2
node interface address mac tap
r1 underlay 192.168.14.252/24 02:4c:00:00:00:00 lt0
r2 underlay 192.168.14.253/24 02:4c:00:01:00:00 lt1
rr1 underlay 192.168.14.2/24 02:4c:00:02:00:00 lt2
hv1 underlay 192.168.14.11/24 02:4c:00:03:00:00 lt3
hv2 underlay 192.168.14.12/24 02:4c:00:04:00:00 lt4
segment underlay: 192.168.14.0/24, mtu 9000, switch sw1, bridge br-underlay, gateway 192.168.14.1
node interface address mac udp switch port mac udp
rr1 underlay 192.168.14.2/24 02:4c:00:01:00:00 20000 <-> sw1 p0 02:4c:00:01:00:01 20001
hv1 underlay 192.168.14.11/24 02:4c:00:02:00:00 20002 <-> sw1 p1 02:4c:00:02:00:01 20003
hv2 underlay 192.168.14.12/24 02:4c:00:03:00:00 20004 <-> sw1 p2 02:4c:00:03:00:01 20005
```
Un fichier invalide est refusé avec **toutes** ses erreurs à la fois, et un code de sortie 1. Les
@ -293,31 +242,17 @@ champs inconnus et les clés en double sont refusés aussi :
```text
$ labctl plan cassee.yml
labctl: cassee.yml:
segment underlay: rr1 is a rr, not a router
segment underlay: router r9 is not a declared node
segment underlay: rr1 is a rr, not a switch
segment underlay: cidr 10.250.0.0/31 prefix length out of range [/8, /30]
node r1: must be attached to at least one segment
node sw1: switch carries no segment
```
### Migration depuis 0.1
Le rôle `switch` et le champ `segments.<s>.switch` n'existent plus : `labctl` les refuse avec un
message qui dit quoi faire. Pour une topologie 0.1 :
1. `role: switch` devient `role: router` ;
2. le routeur déclare `segments: [<s>]` et, de préférence, une adresse fixée (`addresses`) — en
0.1, le switch n'avait que la passerelle ;
3. `segments.<s>.switch: <nom>` devient `segments.<s>.routers: [<nom>]` ;
4. ses `secondary` restent valables : elles vont désormais sur son interface du segment, et non
plus sur un bridge interne.
## FRR
Au premier démarrage, cloud-init installe FRR (`frr-stable` de `deb.frrouting.org`, sans les
paquets recommandés), active `bgpd` — et `bfdd` sur les routeurs et les route reflectors,
`vrrpd` sur les routeurs d'un segment en VRRP —, puis
paquets recommandés), active `bgpd` — et `bfdd` sur le switch et le route reflector —, puis
dépose le `frr.conf` du nœud et redémarre FRR. La mise à jour des index de paquets est réessayée
pendant cinq minutes : un nœud peut démarrer avant que le routeur, par lequel il sort, n'ait posé
pendant cinq minutes : un nœud peut démarrer avant que le switch, par lequel il sort, n'ait posé
son NAT.
La **clé du dépôt FRR** n'est pas téléchargée au démarrage : elle est enregistrée dans `labctl`
@ -340,7 +275,7 @@ voit pas — une provision qui arme un redémarrage, par exemple : la VM redéma
`runcmd`.
> [!NOTE]
> **Le routeur du lab est un Linux avec FRR, par choix.** Il joue le rôle générique de
> **Le switch du lab est un routeur Linux avec FRR, par choix.** Il joue le rôle générique de
> routeur de cluster : passerelle des nœuds et, dans l'exemple, session eBGP avec BFD vers le
> route reflector, dont il n'accepte que la loopback. L'équipement réel d'une infrastructure
> (MikroTik, Cisco, Juniper, Arista…) n'a besoin que de BGP et d'EVPN ; sa configuration propre
@ -365,17 +300,17 @@ Les chemins de la VM (`disk.qcow2`, `seed.iso`, `console.log`, `qmp.sock`, `qemu
du répertoire du nœud ; `-key` peut être répété, et accepte un fichier `authorized_keys` (lignes
vides et commentaires ignorés). Les fichiers sont créés en `0600`.
Ce que contiennent les arguments QEMU d'un nœud — côté réseau :
Ce que contiennent les arguments QEMU d'un nœud — extrait réel, côté réseau :
```text
-netdev
user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2203-:22
user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22
-device
virtio-net-pci,netdev=mgmt0,mac=02:4d:00:03:00:00,romfile=
virtio-net-pci,netdev=mgmt0,mac=02:4d:00:02:00:00,romfile=
-netdev
tap,id=underlay,ifname=lt3,script=no,downscript=no
dgram,id=underlay,local.type=inet,local.host=127.0.0.1,local.port=20002,remote.type=inet,remote.host=127.0.0.1,remote.port=20003
-device
virtio-net-pci,netdev=underlay,mac=02:4c:00:03:00:00,host_mtu=9000,romfile=
virtio-net-pci,netdev=underlay,mac=02:4c:00:02:00:00,host_mtu=9000,romfile=
```
Les choix qui s'y lisent :
@ -383,11 +318,11 @@ Les choix qui s'y lisent :
- **machine** `q35`, `-accel kvm -cpu host` — le KVM imbriqué des nœuds en dépend ;
`-nodefaults` pour qu'aucun périphérique implicite ne s'ajoute ;
- **administration** (`mgmt0`) : le NAT de QEMU, MAC `02:4d:<nœud>:<nœud>:00:00`, SSH redirigé
sur la boucle locale de l'hôte. `restrict=on` pour tous les nœuds **sauf les routeurs** : un
sur la boucle locale de l'hôte. `restrict=on` pour tous les nœuds **sauf le switch** : un
nœud isolé ne joint ni l'hôte ni l'extérieur par là, seule la redirection SSH passe.
`ipv6=off` partout (voir plus bas) ;
- **segments** : un tap créé par `labctl up` (`script=no` : QEMU ne lance aucun script), branché
sur le bridge du segment ; `host_mtu` annonce le MTU du segment au guest ;
- **câbles** : `dgram` sur `127.0.0.1`, les deux extrémités d'un câble se répondent (port local
de l'une = port distant de l'autre), `host_mtu` annonce le MTU du segment au guest ;
- `romfile=` vide sur toutes les cartes : pas de ROM de démarrage réseau, donc pas de repli sur
un démarrage PXE si le firmware ne trouve pas le disque. Pendant les premiers essais, une VM
restée bloquée sans rien écrire sur sa console, CPU au repos, avait toutes les apparences de
@ -395,22 +330,18 @@ Les choix qui s'y lisent :
Ce que fait cloud-init :
- **toutes les VM** : interfaces nommées d'après leur MAC (`mgmt0`, nom du segment),
`dhcp4: false` partout, `mgmt0` en `10.0.2.15/24` ; connexion SSH par clé seulement,
utilisateur `debian`, `root` désactivé, mot de passe refusé ;
- **un nœud** : adresse et `secondary` sur chaque segment, MTU du segment, route par défaut et
DNS sur le premier de ses segments qui a une passerelle ; `mgmt0` **sans passerelle** ;
- **un routeur** : mêmes adresses, route par défaut par `mgmt0` ; un service `lab-router` active
le routage, pose les passerelles (adresse ou `macvlan` VRRP, voir
[Passerelle et routeurs](#passerelle-et-routeurs)) et le NAT nftables. Le service est rejoué à
chaque démarrage.
- **toutes les VM** : interfaces nommées d'après leur MAC (`mgmt0`, nom du segment, `p<i>`),
`dhcp4: false` partout, `mgmt0` en `10.0.2.15/24` **sans passerelle** ; connexion SSH par
clé seulement, utilisateur `debian`, `root` désactivé, mot de passe refusé ;
- **un nœud** : adresse sur chaque segment, MTU du segment, route par défaut et DNS
(`1.1.1.1`, `8.8.8.8`) sur son **premier** segment — la sortie Internet passe par le switch ;
- **le switch** : route par défaut par `mgmt0` ; un service `lab-switch` crée un bridge
`br-<segment>` par segment (STP désactivé, MTU du segment), y branche ses ports, porte la
passerelle, active le routage et masque (NAT nftables) les segments vers `mgmt0`. Le service
est rejoué à chaque démarrage.
### Vérifié sur de vraies VM
Relevés faits avec le modèle 0.1 — le switch portait alors le bridge du segment, relié aux nœuds
par des câbles UDP ; ce qui touche au L2 et à la passerelle est à revérifier avec les bridges de
l'hôte.
Le switch et le route reflector n'ont pas besoin de KVM imbriqué : `sw1` et `rr1` ont été
démarrés **sur un Mac**, en émulation (TCG), avec Debian 12 `generic` et les fichiers produits
par `labctl render`.
@ -456,45 +387,19 @@ labctl ssh [-run dir] <nœud> [commande…]
3. génère une paire de clés SSH dans le répertoire du lab, si elle n'existe pas encore ;
4. pour chaque nœud : fichiers de `labctl render`, disque **neuf** en overlay qcow2 sur l'image
(`qemu-img create -b`, 20 Gio annoncés), image `cidata` et `files.iso` (`genisoimage`) ;
5. crée les bridges des segments et les taps des nœuds (voir [Réseau de l'hôte](#réseau-de-lhôte)),
et refuse de continuer si l'un d'eux existe déjà ;
6. démarre les QEMU, **routeurs d'abord**, détachés (`-daemonize`) : ils survivent à la session
5. démarre les QEMU, **switchs d'abord**, détachés (`-daemonize`) : ils survivent à la session
SSH qui les a lancés ;
7. attend sur chaque nœud la fin de cloud-init (`cloud-init status --wait` par SSH), jusqu'au
délai `-timeout`, puis vérifie ses services (`frr` compris quand il est installé).
6. attend sur chaque nœud la fin de cloud-init (`cloud-init status --wait` par SSH), jusqu'au
délai `-timeout`, puis vérifie ses services.
La topologie est copiée dans `<run>/topology.yml`. Un échec laisse les nœuds démarrés en place :
`labctl status`, puis `labctl down`.
**`labctl status`** — pour chaque nœud : rôle, état du processus QEMU, PID, port SSH sur la
boucle locale ; pour chaque segment : bridge présent ou absent, passerelle.
boucle locale.
**`labctl down`** — arrête chaque QEMU par `SIGTERM`, puis `SIGKILL` au bout de 30 s, puis
supprime les taps et les bridges du lab. Les disques sont conservés jusqu'au prochain `up`, qui
les recrée.
`up` et `down` doivent tourner **en root** : ils créent et suppriment des interfaces de l'hôte.
### Réseau de l'hôte
Pour chaque segment, `labctl up` exécute :
```text
ip link add name lb-<segment> type bridge stp_state 0 mcast_snooping 0
ip link set dev lb-<segment> addrgenmode none
ip tuntap add dev lt<i> mode tap # pour chaque port du segment
ip link set dev lt<i> addrgenmode none
ip link set dev lt<i> mtu <mtu> master lb-<segment> up
ip link set dev lb-<segment> mtu <mtu> up
```
- **Aucune adresse côté hôte**, IPv4 ni IPv6 (`addrgenmode none` : pas même de lien local) :
l'hôte n'est pas joignable depuis un segment, et ne route rien entre eux.
- **Pas de STP ni de surveillance IGMP** : le bridge se comporte comme un switch simple, et les
annonces VRRP (`224.0.0.18`) circulent sans dépendre d'un querier.
- Si `br_netfilter` est chargé sur l'hôte, `bridge-nf-call-iptables` est mis à 0 : le pare-feu
de l'hôte ne filtre pas le trafic des segments.
- Un lab par hôte : les noms `lb-*` et `lt*` ne sont pas propres à un répertoire `-run`.
**`labctl down`** — arrête chaque QEMU par `SIGTERM`, puis `SIGKILL` au bout de 30 s. Les disques
sont conservés jusqu'au prochain `up`, qui les recrée.
**`labctl ssh`** — ouvre un shell sur un nœud, ou y exécute une commande, avec la clé générée par
`up`. `labctl` cède la place à `ssh`, dont le code de retour est donc celui de la commande. Un
@ -528,7 +433,7 @@ PID réutilisé par un autre programme n'est donc jamais signalé.
### Vérifié sur le serveur de lab
Avec le modèle 0.1 (voir la remarque précédente). Le 2026-10-04, sur un Xeon E5-2640 v3, Debian 12 et QEMU 7.2 sur le serveur, topologie à deux
Le 2026-10-04, sur un Xeon E5-2640 v3, Debian 12 et QEMU 7.2 sur le serveur, topologie à deux
hôtes :
| Vérification | Résultat |
@ -614,8 +519,6 @@ au projet.
- **Clé SSH des VM** : générée par `labctl up` sur le serveur, c'est la seule clé autorisée dans
les VM. Elle ne quitte jamais le serveur, n'ouvre que les VM du lab — qui n'écoutent qu'en
boucle locale — et disparaît avec lui. La clé publique du poste n'est jamais envoyée aux VM.
- **Segments isolés de l'hôte** : bridges et taps sans adresse (voir
[Réseau de l'hôte](#réseau-de-lhôte)) ; seuls les routeurs sortent, par le NAT de QEMU.
- **Clés d'hôte des VM non vérifiées** par `labctl ssh` (`known_hosts` jetable) : elles changent
à chaque `up`. Acceptable uniquement parce que la connexion reste sur la boucle locale d'un
serveur auquel on s'est authentifié.

View file

@ -1,28 +0,0 @@
frr defaults traditional
hostname r2
log syslog informational
!
ip prefix-list RR-LOOPBACK-IN seq 10 permit 10.255.255.1/32
!
route-map RR-IN permit 10
match ip address prefix-list RR-LOOPBACK-IN
exit
!
route-map NO-OUT deny 999
exit
!
router bgp 65100
no bgp default ipv4-unicast
bgp router-id 169.254.0.2
neighbor 169.254.0.3 remote-as 65000
neighbor 169.254.0.3 description rr1
neighbor 169.254.0.3 bfd
!
address-family ipv4 unicast
neighbor 169.254.0.3 activate
neighbor 169.254.0.3 route-map RR-IN in
neighbor 169.254.0.3 route-map NO-OUT out
exit-address-family
!
exit
!

View file

@ -1,5 +1,5 @@
frr defaults traditional
hostname r1
hostname sw1
log syslog informational
!
ip prefix-list RR-LOOPBACK-IN seq 10 permit 10.255.255.1/32

View file

@ -7,19 +7,14 @@ images:
segments:
underlay:
routers: [r1, r2]
switch: sw1
cidr: 192.168.14.0/24
mtu: 9000
nodes:
r1:
{ role: router, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
addresses: { underlay: 192.168.14.252 }, secondary: { underlay: [169.254.0.1/28] },
frr: frr/r1.conf }
r2:
{ role: router, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
addresses: { underlay: 192.168.14.253 }, secondary: { underlay: [169.254.0.2/28] },
frr: frr/r2.conf }
sw1:
{ role: switch, image: debian12, cpus: 2, memory: 1024,
secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf }
rr1:
{ role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
addresses: { underlay: 192.168.14.2 }, secondary: { underlay: [169.254.0.3/28] },

View file

@ -107,10 +107,7 @@ func (l Lab) Up(ctx context.Context, fetcher provision.Fetcher, timeout time.Dur
if err != nil {
return err
}
if err := l.NetworkUp(ctx); err != nil {
return err
}
for _, n := range routersFirst(l.Plan, nodes) {
for _, n := range switchesFirst(l.Plan, nodes) {
if err := os.Remove(l.pidFile(n.Name)); err != nil && !errors.Is(err, os.ErrNotExist) {
return err
}
@ -137,10 +134,10 @@ func (l Lab) Up(ctx context.Context, fetcher provision.Fetcher, timeout time.Dur
return errors.Join(errs...)
}
func routersFirst(p *topology.Plan, nodes []render.Node) []render.Node {
func switchesFirst(p *topology.Plan, nodes []render.Node) []render.Node {
var first, rest []render.Node
for i, n := range nodes {
if p.Nodes[i].Role == topology.RoleRouter {
if p.Nodes[i].Role == topology.RoleSwitch {
first = append(first, n)
} else {
rest = append(rest, n)
@ -170,16 +167,16 @@ func (l Lab) waitReady(ctx context.Context, n topology.NodePlan) error {
}
}
func services(p *topology.Plan, n topology.NodePlan) []string {
func services(n topology.NodePlan) []string {
units := append([]string(nil), n.Services...)
if render.NeedsFRR(p, n) {
if n.FRR != "" {
units = append(units, "frr.service")
}
return units
}
func (l Lab) checkServices(ctx context.Context, n topology.NodePlan) error {
for _, unit := range services(l.Plan, n) {
for _, unit := range services(n) {
if err := l.Runner.Run(ctx, "ssh", l.sshArgs(n, true, []string{"systemctl", "is-active", "--quiet", unit})...); err != nil {
return fmt.Errorf("%s is not active: %w", unit, err)
}
@ -238,10 +235,7 @@ func (l Lab) Status(w io.Writer) error {
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t127.0.0.1:%d\n", n.Name, n.Role, state, shown, n.SSHPort)
}
if err := tw.Flush(); err != nil {
return err
}
return l.networkStatus(w)
return tw.Flush()
}
func (l Lab) Down(ctx context.Context) error {
@ -265,10 +259,7 @@ func (l Lab) Down(ctx context.Context) error {
}
fmt.Fprintf(l.Out, "%s: stopped\n", n.Name)
}
if len(errs) > 0 {
return errors.Join(errs...)
}
return l.NetworkDown(ctx)
return errors.Join(errs...)
}
func (l Lab) stop(ctx context.Context, name string, pid int) error {

View file

@ -38,7 +38,6 @@ type fakeRunner struct {
ssh map[string][]error
always map[string]error
inactive string
links map[string]bool
}
func (f *fakeRunner) Run(_ context.Context, name string, args ...string) error {
@ -55,22 +54,6 @@ func (f *fakeRunner) Run(_ context.Context, name string, args ...string) error {
return err
}
return os.WriteFile(private+".pub", []byte(labKey+"\n"), 0o644)
case "ip":
if f.links == nil {
f.links = map[string]bool{}
}
switch {
case len(args) == 4 && args[0] == "link" && args[1] == "show":
if !f.links[args[3]] {
return exitErr(1)
}
case args[0] == "link" && args[1] == "add" && args[2] == "name":
f.links[args[3]] = true
case args[0] == "tuntap" && args[1] == "add":
f.links[args[3]] = true
case args[0] == "link" && args[1] == "delete":
delete(f.links, args[3])
}
case "ssh":
port := args[indexOf(args, "-p")+1]
if args[len(args)-1] == f.inactive {
@ -98,25 +81,6 @@ func (f *fakeRunner) commands(name string) [][]string {
return out
}
func (f *fakeRunner) ipCalls() []string {
var out []string
for _, c := range f.commands("ip") {
out = append(out, strings.Join(c[1:], " "))
}
return out
}
func (f *fakeRunner) present(links ...string) {
f.mu.Lock()
defer f.mu.Unlock()
if f.links == nil {
f.links = map[string]bool{}
}
for _, l := range links {
f.links[l] = true
}
}
func indexOf(list []string, value string) int {
for i, v := range list {
if v == value {
@ -141,10 +105,7 @@ func mirror(t *testing.T) (*httptest.Server, string) {
return server, server.URL
}
const underlay = ` underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
`
func labPlan(t *testing.T, base, segments, nodes string) *topology.Plan {
func labPlan(t *testing.T, base, nodes string) *topology.Plan {
t.Helper()
doc := `name: evpn-2hv
images:
@ -152,7 +113,8 @@ images:
url: ` + base + `/deb.qcow2
sums: ` + base + `/SHA512SUMS
segments:
` + segments + `nodes:
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 }
nodes:
` + nodes
topo, err := topology.Parse([]byte(doc))
if err != nil {
@ -165,18 +127,9 @@ segments:
return p
}
const routerLast = ` rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
const switchLast = ` rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], services: [agent.service] }
sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
`
const twoSegments = ` red: { routers: [r1, r2], cidr: 10.1.0.0/24 }
blue: { cidr: 10.2.0.0/24, mtu: 1500 }
`
const twoRouters = ` r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 }
`
type fixture struct {
@ -187,11 +140,6 @@ type fixture struct {
}
func newFixture(t *testing.T, nodes string) *fixture {
t.Helper()
return newLab(t, underlay, nodes)
}
func newLab(t *testing.T, segments, nodes string) *fixture {
t.Helper()
server, base := mirror(t)
root := t.TempDir()
@ -199,7 +147,7 @@ func newLab(t *testing.T, segments, nodes string) *fixture {
out := &bytes.Buffer{}
return &fixture{
lab: Lab{
Plan: labPlan(t, base, segments, nodes),
Plan: labPlan(t, base, nodes),
RunDir: filepath.Join(root, "run"),
ProcDir: filepath.Join(root, "proc"),
Runner: runner,
@ -269,7 +217,7 @@ func waitDead(t *testing.T, pid int) {
}
func TestPID_NoPidFileMeansStopped(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil {
t.Errorf("PID = %d, %v", pid, err)
}
@ -277,7 +225,7 @@ func TestPID_NoPidFileMeansStopped(t *testing.T) {
func TestPID_RefusesACorruptedPidFile(t *testing.T) {
for _, content := range []string{"", "abc", "0", "-12"} {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.pidFile(t, "hv1", content)
if _, err := f.lab.PID("hv1"); err == nil || !strings.Contains(err.Error(), "not a pid") {
t.Errorf("pid file %q: error = %v", content, err)
@ -286,7 +234,7 @@ func TestPID_RefusesACorruptedPidFile(t *testing.T) {
}
func TestPID_RecognisesTheNodeProcess(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
want := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if pid, err := f.lab.PID("hv1"); pid != want || err != nil {
t.Errorf("PID = %d, %v, want %d", pid, err, want)
@ -301,7 +249,7 @@ func TestPID_IgnoresAPidReusedByAnotherProcess(t *testing.T) {
}
for name, cmdline := range cases {
t.Run(name, func(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.process(t, "hv1", "sleep 30", cmdline...)
if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil {
t.Errorf("PID = %d, %v, want 0", pid, err)
@ -311,7 +259,7 @@ func TestPID_IgnoresAPidReusedByAnotherProcess(t *testing.T) {
}
func TestPID_IgnoresADeadProcess(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
pid := f.process(t, "hv1", "exit 0", qemu("hv1")...)
waitDead(t, pid)
if got, err := f.lab.PID("hv1"); got != 0 || err != nil {
@ -320,7 +268,7 @@ func TestPID_IgnoresADeadProcess(t *testing.T) {
}
func TestStatus(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
pid := strconv.Itoa(f.process(t, "hv1", "sleep 30", qemu("hv1")...))
var out bytes.Buffer
if err := f.lab.Status(&out); err != nil {
@ -333,16 +281,14 @@ func TestStatus(t *testing.T) {
want := fmt.Sprintf("node role state %-*s ssh\n", w, "pid") +
fmt.Sprintf("rr1 rr stopped %-*s 127.0.0.1:2200\n", w, "-") +
fmt.Sprintf("hv1 host running %-*s 127.0.0.1:2201\n", w, pid) +
fmt.Sprintf("sw1 router stopped %-*s 127.0.0.1:2202\n", w, "-") +
"\nsegment bridge state gateway\n" +
"underlay lb-underlay absent 10.250.0.1\n"
fmt.Sprintf("sw1 switch stopped %-*s 127.0.0.1:2202\n", w, "-")
if out.String() != want {
t.Errorf("status:\n%s\nwant:\n%s", out.String(), want)
}
}
func TestDown_StopsWithSIGTERM(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if err := f.lab.Down(context.Background()); err != nil {
@ -359,7 +305,7 @@ func TestDown_StopsWithSIGTERM(t *testing.T) {
}
func TestDown_FallsBackToSIGKILL(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.lab.Stop = 100 * time.Millisecond
pid := f.process(t, "hv1", `trap "" TERM; while :; do sleep 0.05; done`, qemu("hv1")...)
time.Sleep(50 * time.Millisecond)
@ -376,7 +322,7 @@ func TestDown_FallsBackToSIGKILL(t *testing.T) {
}
func TestDown_NeverSignalsAProcessThatIsNotTheNode(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
pid := f.process(t, "hv1", "sleep 30", "/usr/sbin/sshd", "-D")
if err := f.lab.Down(context.Background()); err != nil {
@ -393,7 +339,7 @@ func TestDown_NeverSignalsAProcessThatIsNotTheNode(t *testing.T) {
}
func TestDown_ReportsACorruptedPidFileAndStopsTheOthers(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.pidFile(t, "rr1", "garbage")
pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
@ -406,7 +352,7 @@ func TestDown_ReportsACorruptedPidFileAndStopsTheOthers(t *testing.T) {
}
func TestSSH_Arguments(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
key := filepath.Join(f.lab.RunDir, "lab_ed25519")
base := []string{
"-i", key,
@ -442,14 +388,14 @@ func TestSSH_Arguments(t *testing.T) {
}
func TestSSH_UnknownNode(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
if _, err := f.lab.SSH("hv9", true, nil); err == nil || !strings.Contains(err.Error(), `node "hv9" is not in lab evpn-2hv`) {
t.Errorf("error = %v", err)
}
}
func TestUp_StartsRoutersFirstAndWaitsForEveryNode(t *testing.T) {
f := newFixture(t, routerLast)
func TestUp_StartsSwitchesFirstAndWaitsForEveryNode(t *testing.T) {
f := newFixture(t, switchLast)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err)
@ -463,7 +409,7 @@ func TestUp_StartsRoutersFirstAndWaitsForEveryNode(t *testing.T) {
started = append(started, c[indexOf(c, "-name")+1])
}
if !reflect.DeepEqual(started, []string{"sw1", "rr1", "hv1"}) {
t.Errorf("start order = %v, want the router first", started)
t.Errorf("start order = %v, want the switch first", started)
}
ssh := f.runner.commands("ssh")
@ -492,7 +438,7 @@ func TestUp_StartsRoutersFirstAndWaitsForEveryNode(t *testing.T) {
}
func TestUp_RetriesWhileSSHIsUnreachable(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.runner.ssh["2201"] = []error{exitErr(255), exitErr(255), nil}
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
@ -514,7 +460,7 @@ func TestUp_ReadinessFailures(t *testing.T) {
}
for name, c := range cases {
t.Run(name, func(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.lab.Poll = 10 * time.Millisecond
f.runner.ssh["2201"] = c.answers
f.runner.always["2201"] = exitErr(255)
@ -532,7 +478,7 @@ func TestUp_ReadinessFailures(t *testing.T) {
}
func TestUp_RecoverableCloudInitErrorsAreReportedNotFatal(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.runner.ssh["2201"] = []error{exitErr(2)}
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
@ -544,7 +490,7 @@ func TestUp_RecoverableCloudInitErrorsAreReportedNotFatal(t *testing.T) {
}
func TestUp_RefusesARunningLab(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.process(t, "hv1", "sleep 30", qemu("hv1")...)
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -558,7 +504,7 @@ func TestUp_RefusesARunningLab(t *testing.T) {
}
func TestUp_RemovesAStalePidFileBeforeStarting(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
pid := f.process(t, "hv1", "exit 0", qemu("hv1")...)
waitDead(t, pid)
@ -571,7 +517,7 @@ func TestUp_RemovesAStalePidFileBeforeStarting(t *testing.T) {
}
func TestUp_StopsAtTheFirstQEMUFailure(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.runner.fail = "qemu-system-x86_64"
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -588,7 +534,7 @@ func TestUp_StopsAtTheFirstQEMUFailure(t *testing.T) {
}
func TestStop_RefusesPidsThatTargetAGroup(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.lab.signal = func(pid int, sig syscall.Signal) error {
t.Fatalf("signal %v sent to pid %d", sig, pid)
return nil
@ -601,7 +547,7 @@ func TestStop_RefusesPidsThatTargetAGroup(t *testing.T) {
}
func TestUp_ChecksTheServicesOfEachRole(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
conf := filepath.Join(t.TempDir(), "sw1.conf")
if err := os.WriteFile(conf, []byte("hostname sw1\n"), 0o600); err != nil {
t.Fatal(err)
@ -628,7 +574,7 @@ func TestUp_ChecksTheServicesOfEachRole(t *testing.T) {
}
func TestUp_ReportsAnInactiveServiceAndWaitsForTheOthers(t *testing.T) {
f := newFixture(t, routerLast)
f := newFixture(t, switchLast)
f.runner.inactive = "agent.service"
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -640,293 +586,3 @@ func TestUp_ReportsAnInactiveServiceAndWaitsForTheOthers(t *testing.T) {
t.Errorf("output = %q", f.out.String())
}
}
func TestStatus_ShowsTheSegments(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present("lb-red")
var out bytes.Buffer
if err := f.lab.Status(&out); err != nil {
t.Fatal(err)
}
want := "\nsegment bridge state gateway\n" +
"red lb-red present 10.1.0.1\n" +
"blue lb-blue absent -\n"
if !strings.HasSuffix(out.String(), want) {
t.Errorf("status:\n%s\ndoes not end with:\n%s", out.String(), want)
}
}
func TestNetworkUp_CreatesBridgesAndTaps(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
want := []string{
"link show dev lb-red",
"link show dev lb-blue",
"link show dev lt0",
"link show dev lt1",
"link show dev lt2",
"link show dev lt3",
"link add name lb-red type bridge stp_state 0 mcast_snooping 0",
"link set dev lb-red addrgenmode none",
"tuntap add dev lt0 mode tap",
"link set dev lt0 addrgenmode none",
"link set dev lt0 mtu 9000 master lb-red up",
"tuntap add dev lt1 mode tap",
"link set dev lt1 addrgenmode none",
"link set dev lt1 mtu 9000 master lb-red up",
"tuntap add dev lt2 mode tap",
"link set dev lt2 addrgenmode none",
"link set dev lt2 mtu 9000 master lb-red up",
"link set dev lb-red mtu 9000 up",
"link add name lb-blue type bridge stp_state 0 mcast_snooping 0",
"link set dev lb-blue addrgenmode none",
"tuntap add dev lt3 mode tap",
"link set dev lt3 addrgenmode none",
"link set dev lt3 mtu 1500 master lb-blue up",
"link set dev lb-blue mtu 1500 up",
}
if got := f.runner.ipCalls(); !reflect.DeepEqual(got, want) {
t.Errorf("ip calls:\n%s\nwant:\n%s", strings.Join(got, "\n"), strings.Join(want, "\n"))
}
if len(f.runner.calls) != len(want) {
t.Errorf("other commands ran: %q", f.runner.calls)
}
}
func TestNetworkUp_RefusesExistingLinks(t *testing.T) {
cases := map[string]struct {
link, want string
}{
"bridge": {"lb-blue", "segment blue: bridge lb-blue already exists on this host: 'labctl down' first, or remove it"},
"tap": {"lt2", "node hv: tap lt2 already exists on this host: 'labctl down' first, or remove it"},
}
for name, c := range cases {
t.Run(name, func(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present(c.link)
dir := filepath.Join(f.lab.ProcDir, "sys", "net", "bridge")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "bridge-nf-call-iptables"), []byte("1\n"), 0o644); err != nil {
t.Fatal(err)
}
err := f.lab.NetworkUp(context.Background())
if err == nil || err.Error() != c.want {
t.Errorf("error = %v, want %q", err, c.want)
}
for _, call := range f.runner.ipCalls() {
if !strings.HasPrefix(call, "link show dev ") {
t.Errorf("%q ran after the refusal", call)
}
}
if data, _ := os.ReadFile(filepath.Join(dir, "bridge-nf-call-iptables")); string(data) != "1\n" {
t.Errorf("bridge filter changed to %q", data)
}
})
}
}
func TestNetworkUp_DisablesTheBridgeFilterWhenPresent(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
dir := filepath.Join(f.lab.ProcDir, "sys", "net", "bridge")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "bridge-nf-call-iptables")
if err := os.WriteFile(path, []byte("1\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
if data, err := os.ReadFile(path); err != nil || string(data) != "0\n" {
t.Errorf("bridge filter = %q, %v, want 0", data, err)
}
}
func TestNetworkUp_LeavesAnAbsentBridgeFilterAlone(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
if _, err := os.Stat(filepath.Join(f.lab.ProcDir, "sys")); !os.IsNotExist(err) {
t.Errorf("bridge filter created under the proc dir: %v", err)
}
}
func TestNetworkUp_ReportsTheFailingSegment(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.fail = "ip"
err := f.lab.NetworkUp(context.Background())
if err == nil || err.Error() != "segment red: ip failed" {
t.Errorf("error = %v", err)
}
}
func TestNetworkDown_DeletesTapsThenBridgesThatExist(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present("lb-red", "lb-blue", "lt0", "lt2", "lt3")
if err := f.lab.NetworkDown(context.Background()); err != nil {
t.Fatalf("NetworkDown: %v", err)
}
var deleted []string
for _, call := range f.runner.ipCalls() {
if link, ok := strings.CutPrefix(call, "link delete dev "); ok {
deleted = append(deleted, link)
}
}
if want := []string{"lt0", "lt2", "lt3", "lb-red", "lb-blue"}; !reflect.DeepEqual(deleted, want) {
t.Errorf("deleted %v, want %v", deleted, want)
}
if len(f.runner.links) != 0 {
t.Errorf("links left: %v", f.runner.links)
}
}
func TestDown_RemovesTheNetworkAfterTheNodes(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay", "lt0", "lt1", "lt2")
f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if err := f.lab.Down(context.Background()); err != nil {
t.Fatalf("Down: %v", err)
}
var deleted []string
for _, call := range f.runner.ipCalls() {
if link, ok := strings.CutPrefix(call, "link delete dev "); ok {
deleted = append(deleted, link)
}
}
if want := []string{"lt0", "lt1", "lt2", "lb-underlay"}; !reflect.DeepEqual(deleted, want) {
t.Errorf("deleted %v, want %v", deleted, want)
}
if f.out.String() != "hv1: stopped\n" {
t.Errorf("output = %q", f.out.String())
}
}
func TestDown_KeepsTheNetworkWhenANodeCannotBeStopped(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay", "lt0", "lt1", "lt2")
f.pidFile(t, "rr1", "garbage")
if err := f.lab.Down(context.Background()); err == nil {
t.Fatal("Down succeeded")
}
if calls := f.runner.ipCalls(); len(calls) != 0 {
t.Errorf("ip calls after a failed stop: %q", calls)
}
}
func TestUp_BuildsTheNetworkBeforeStartingQEMU(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err)
}
lastIP, firstQEMU := -1, -1
var started []string
for i, c := range f.runner.calls {
switch c[0] {
case "ip":
lastIP = i
case "qemu-system-x86_64":
if firstQEMU < 0 {
firstQEMU = i
}
started = append(started, c[indexOf(c, "-name")+1])
}
}
if lastIP < 0 || firstQEMU < 0 || lastIP > firstQEMU {
t.Errorf("last ip call at %d, first qemu start at %d", lastIP, firstQEMU)
}
if !reflect.DeepEqual(started, []string{"r1", "r2", "hv"}) {
t.Errorf("start order = %v", started)
}
for _, link := range []string{"lb-red", "lb-blue", "lt0", "lt1", "lt2", "lt3"} {
if !f.runner.links[link] {
t.Errorf("%s was not created", link)
}
}
}
func TestUp_RefusesAnExistingBridgeBeforeStartingQEMU(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay")
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
if err == nil || !strings.Contains(err.Error(), "bridge lb-underlay already exists") {
t.Errorf("error = %v", err)
}
if n := len(f.runner.commands("qemu-system-x86_64")); n != 0 {
t.Errorf("%d qemu starts", n)
}
}
func TestUp_StopsWhenTheNetworkFails(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.fail = "ip"
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
if err == nil || err.Error() != "segment underlay: ip failed" {
t.Errorf("error = %v", err)
}
if n := len(f.runner.commands("qemu-system-x86_64")); n != 0 {
t.Errorf("%d qemu starts", n)
}
}
func TestUp_ChecksFRROnVRRPRouters(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err)
}
var checks []string
for _, c := range f.runner.commands("ssh") {
if c[len(c)-4] == "systemctl" {
checks = append(checks, c[indexOf(c, "-p")+1]+" "+strings.Join(c[len(c)-4:], " "))
}
}
want := []string{
"2200 systemctl is-active --quiet frr.service",
"2201 systemctl is-active --quiet frr.service",
}
if !reflect.DeepEqual(checks, want) {
t.Errorf("checks = %q, want %q", checks, want)
}
}
func TestServices(t *testing.T) {
f := newLab(t, twoSegments+underlay, twoRouters+` sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay], services: [x.service] }
`)
want := map[string][]string{
"r1": {"frr.service"},
"r2": {"frr.service"},
"hv": nil,
"sw1": {"x.service"},
}
for _, n := range f.lab.Plan.Nodes {
if got := services(f.lab.Plan, n); !reflect.DeepEqual(got, want[n.Name]) {
t.Errorf("%s services = %q, want %q", n.Name, got, want[n.Name])
}
}
}

View file

@ -1,122 +0,0 @@
package machine
import (
"context"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"text/tabwriter"
"git.g3e.fr/syonad/labctl/internal/topology"
)
const bridgeFilter = "sys/net/bridge/bridge-nf-call-iptables"
func (l Lab) exists(ctx context.Context, link string) bool {
return l.Runner.Run(ctx, "ip", "link", "show", "dev", link) == nil
}
func (l Lab) NetworkUp(ctx context.Context) error {
for _, s := range l.Plan.Segments {
if l.exists(ctx, s.Bridge) {
return fmt.Errorf("segment %s: bridge %s already exists on this host: 'labctl down' first, or remove it", s.Name, s.Bridge)
}
}
for _, port := range l.Plan.Ports {
if l.exists(ctx, port.Tap) {
return fmt.Errorf("node %s: tap %s already exists on this host: 'labctl down' first, or remove it", port.Node, port.Tap)
}
}
if err := l.disableBridgeFilter(); err != nil {
return err
}
for _, s := range l.Plan.Segments {
if err := l.ip(ctx,
[]string{"link", "add", "name", s.Bridge, "type", "bridge", "stp_state", "0", "mcast_snooping", "0"},
[]string{"link", "set", "dev", s.Bridge, "addrgenmode", "none"},
); err != nil {
return fmt.Errorf("segment %s: %w", s.Name, err)
}
for _, port := range l.Plan.Ports {
if port.Segment != s.Name {
continue
}
mtu := strconv.Itoa(port.MTU)
if err := l.ip(ctx,
[]string{"tuntap", "add", "dev", port.Tap, "mode", "tap"},
[]string{"link", "set", "dev", port.Tap, "addrgenmode", "none"},
[]string{"link", "set", "dev", port.Tap, "mtu", mtu, "master", s.Bridge, "up"},
); err != nil {
return fmt.Errorf("node %s: %w", port.Node, err)
}
}
if err := l.ip(ctx, []string{"link", "set", "dev", s.Bridge, "mtu", strconv.Itoa(s.MTU), "up"}); err != nil {
return fmt.Errorf("segment %s: %w", s.Name, err)
}
}
return nil
}
func (l Lab) NetworkDown(ctx context.Context) error {
var errs []error
for _, port := range l.Plan.Ports {
if !l.exists(ctx, port.Tap) {
continue
}
if err := l.ip(ctx, []string{"link", "delete", "dev", port.Tap}); err != nil {
errs = append(errs, fmt.Errorf("node %s: %w", port.Node, err))
}
}
for _, s := range l.Plan.Segments {
if !l.exists(ctx, s.Bridge) {
continue
}
if err := l.ip(ctx, []string{"link", "delete", "dev", s.Bridge}); err != nil {
errs = append(errs, fmt.Errorf("segment %s: %w", s.Name, err))
}
}
return errors.Join(errs...)
}
func (l Lab) ip(ctx context.Context, commands ...[]string) error {
for _, args := range commands {
if err := l.Runner.Run(ctx, "ip", args...); err != nil {
return err
}
}
return nil
}
func (l Lab) disableBridgeFilter() error {
path := filepath.Join(l.ProcDir, bridgeFilter)
if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
return nil
}
if err := os.WriteFile(path, []byte("0\n"), 0o644); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
func (l Lab) networkStatus(w io.Writer) error {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintf(tw, "\nsegment\tbridge\tstate\tgateway\n")
for _, s := range l.Plan.Segments {
state := "absent"
if l.exists(context.Background(), s.Bridge) {
state = "present"
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\n", s.Name, s.Bridge, state, gatewayOf(s))
}
return tw.Flush()
}
func gatewayOf(s topology.SegmentPlan) string {
if !s.Gateway.IsValid() {
return "-"
}
return s.Gateway.String()
}

View file

@ -56,12 +56,12 @@ images:
url: ` + img.URL + `
sums: ` + img.Sums + `
segments:
underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 }
nodes:
sw1: { role: router, image: debian12, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
sw1: { role: switch, image: debian12, cpus: 2, memory: 1024 }
rr1: { role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
`
topo, err := topology.Parse([]byte(doc))
if err != nil {

View file

@ -31,7 +31,7 @@ func qemuArgs(p *topology.Plan, n topology.NodePlan, index int, dir string) []st
}
restrict := "on"
if n.Role == topology.RoleRouter {
if n.Role == topology.RoleSwitch {
restrict = "off"
}
args = append(args,
@ -39,11 +39,19 @@ func qemuArgs(p *topology.Plan, n topology.NodePlan, index int, dir string) []st
"-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,romfile=", AdminInterface, adminMAC(index)),
)
for _, port := range nodePorts(p, n.Name) {
args = append(args,
"-netdev", fmt.Sprintf("tap,id=%s,ifname=%s,script=no,downscript=no", port.Interface, port.Tap),
"-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,host_mtu=%d,romfile=", port.Interface, port.MAC, port.MTU),
)
for _, c := range nodeCables(p, n.Name) {
args = append(args, cable(c.NodeInterface, c.NodePort, c.SwitchPort, c.NodeMAC.String(), c.MTU)...)
}
for _, c := range switchCables(p, n.Name) {
args = append(args, cable(c.SwitchInterface, c.SwitchPort, c.NodePort, c.SwitchMAC.String(), c.MTU)...)
}
return args
}
func cable(id string, local, remote int, mac string, mtu int) []string {
return []string{
"-netdev", fmt.Sprintf("dgram,id=%s,local.type=inet,local.host=%s,local.port=%d,remote.type=inet,remote.host=%s,remote.port=%d",
id, loopback, local, loopback, remote),
"-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,host_mtu=%d,romfile=", id, mac, mtu),
}
}

View file

@ -119,46 +119,32 @@ func adminMAC(index int) net.HardwareAddr {
return net.HardwareAddr{0x02, 0x4d, byte(index >> 8), byte(index), 0x00, 0x00}
}
func nodePorts(p *topology.Plan, name string) []topology.Port {
var out []topology.Port
for _, port := range p.Ports {
if port.Node == name {
out = append(out, port)
func nodeCables(p *topology.Plan, name string) []topology.Cable {
var out []topology.Cable
for _, c := range p.Cables {
if c.Node == name {
out = append(out, c)
}
}
return out
}
func routedSegments(p *topology.Plan, name string) []topology.SegmentPlan {
func switchCables(p *topology.Plan, name string) []topology.Cable {
var out []topology.Cable
for _, c := range p.Cables {
if c.Switch == name {
out = append(out, c)
}
}
return out
}
func switchSegments(p *topology.Plan, name string) []topology.SegmentPlan {
var out []topology.SegmentPlan
for _, s := range p.Segments {
for _, r := range s.Routers {
if r == name {
out = append(out, s)
}
if s.Switch == name {
out = append(out, s)
}
}
return out
}
func segmentNamed(p *topology.Plan, name string) topology.SegmentPlan {
for _, s := range p.Segments {
if s.Name == name {
return s
}
}
return topology.SegmentPlan{}
}
func uplink(p *topology.Plan, name string) (topology.Port, bool) {
ports := nodePorts(p, name)
for _, port := range ports {
if segmentNamed(p, port.Segment).Gateway.IsValid() {
return port, true
}
}
if len(ports) > 0 {
return ports[0], false
}
return topology.Port{}, false
}

View file

@ -2,7 +2,6 @@ package render
import (
"reflect"
"strconv"
"strings"
"testing"
@ -19,12 +18,12 @@ images:
url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS
segments:
underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 }
nodes:
sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
`
const twoSegments = `name: two-seg
@ -33,30 +32,13 @@ images:
url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { routers: [sw], cidr: 10.2.0.0/24, mtu: 1500 }
red: { switch: sw, cidr: 10.1.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24, mtu: 1500 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] }
`
const withVRRP = `name: vrrp
images:
deb:
url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS
segments:
red: { routers: [r1, r2], cidr: 10.1.0.0/24 }
blue: { routers: [r1], cidr: 10.2.0.0/24 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, pure] }
r3: { role: router, image: deb, cpus: 1, memory: 512, segments: [pure] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [pure, red] }
l2: { role: host, image: deb, cpus: 1, memory: 512, segments: [pure] }
`
func plan(t *testing.T, doc string) *topology.Plan {
t.Helper()
topo, err := topology.Parse([]byte(doc))
@ -200,7 +182,7 @@ func TestQEMU_HypervisorNetwork(t *testing.T) {
hv := nodeNamed(t, renderAll(t, twoHypervisors), "hv1")
wantNetdevs := []string{
"user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22",
"tap,id=underlay,ifname=lt2,script=no,downscript=no",
"dgram,id=underlay,local.type=inet,local.host=127.0.0.1,local.port=20002,remote.type=inet,remote.host=127.0.0.1,remote.port=20003",
}
wantDevices := []string{
"virtio-net-pci,netdev=mgmt0,mac=02:4d:00:02:00:00,romfile=",
@ -214,31 +196,27 @@ func TestQEMU_HypervisorNetwork(t *testing.T) {
}
}
func TestQEMU_RouterNetwork(t *testing.T) {
sw := nodeNamed(t, renderAll(t, twoSegments), "sw")
func TestQEMU_SwitchNetwork(t *testing.T) {
sw := nodeNamed(t, renderAll(t, twoHypervisors), "sw1")
wantNetdevs := []string{
"user,id=mgmt0,restrict=off,ipv6=off,hostfwd=tcp:127.0.0.1:2200-:22",
"tap,id=red,ifname=lt0,script=no,downscript=no",
"tap,id=blue,ifname=lt2,script=no,downscript=no",
}
wantDevices := []string{
"virtio-net-pci,netdev=mgmt0,mac=02:4d:00:00:00:00,romfile=",
"virtio-net-pci,netdev=red,mac=02:4c:00:00:00:00,host_mtu=9000,romfile=",
"virtio-net-pci,netdev=blue,mac=02:4c:00:00:01:00,host_mtu=1500,romfile=",
"dgram,id=p0,local.type=inet,local.host=127.0.0.1,local.port=20001,remote.type=inet,remote.host=127.0.0.1,remote.port=20000",
"dgram,id=p1,local.type=inet,local.host=127.0.0.1,local.port=20003,remote.type=inet,remote.host=127.0.0.1,remote.port=20002",
"dgram,id=p2,local.type=inet,local.host=127.0.0.1,local.port=20005,remote.type=inet,remote.host=127.0.0.1,remote.port=20004",
}
if got := value(t, sw.QEMU, "-netdev"); !reflect.DeepEqual(got, wantNetdevs) {
t.Errorf("netdevs = %v\nwant %v", got, wantNetdevs)
}
if got := value(t, sw.QEMU, "-device"); !reflect.DeepEqual(got, wantDevices) {
t.Errorf("devices = %v\nwant %v", got, wantDevices)
if !hasValue(sw.QEMU, "-device", "virtio-net-pci,netdev=p2,mac=02:4c:00:03:00:01,host_mtu=9000,romfile=") {
t.Errorf("p2 device missing: %v", sw.QEMU)
}
}
func TestQEMU_OnlyRoutersReachTheOutsideThroughAdministration(t *testing.T) {
for name, n := range renderAll(t, withVRRP) {
func TestQEMU_OnlyTheSwitchReachesTheOutsideThroughAdministration(t *testing.T) {
for name, n := range renderAll(t, twoHypervisors) {
admin := value(t, n.QEMU, "-netdev")[0]
wantRestrict := "restrict=on"
if name == "r1" || name == "r2" || name == "r3" {
if name == "sw1" {
wantRestrict = "restrict=off"
}
if !strings.Contains(admin, ","+wantRestrict+",") {
@ -250,29 +228,34 @@ func TestQEMU_OnlyRoutersReachTheOutsideThroughAdministration(t *testing.T) {
}
}
func TestQEMU_EveryTapOfThePlanIsOpenedOnce(t *testing.T) {
p := plan(t, withVRRP)
opened := map[string]int{}
for _, n := range renderAll(t, withVRRP) {
func TestQEMU_EveryCableEndsMatch(t *testing.T) {
ends := map[string]int{}
for _, n := range renderAll(t, twoHypervisors) {
for _, nd := range value(t, n.QEMU, "-netdev") {
if strings.HasPrefix(nd, "dgram") || strings.HasPrefix(nd, "socket") {
t.Errorf("%s uses a %s netdev", n.Name, nd)
if !strings.HasPrefix(nd, "dgram,") {
continue
}
var local, remote string
for _, kv := range strings.Split(nd, ",") {
if tap, ok := strings.CutPrefix(kv, "ifname="); ok {
opened[tap]++
if v, ok := strings.CutPrefix(kv, "local.port="); ok {
local = v
}
if v, ok := strings.CutPrefix(kv, "remote.port="); ok {
remote = v
}
}
ends[local+">"+remote]++
}
}
if len(opened) != len(p.Ports) {
t.Errorf("%d taps opened, plan has %d ports", len(opened), len(p.Ports))
}
for _, port := range p.Ports {
if opened[port.Tap] != 1 {
t.Errorf("tap %s opened %d times", port.Tap, opened[port.Tap])
for pair, count := range ends {
local, remote, _ := strings.Cut(pair, ">")
if count != 1 || ends[remote+">"+local] != 1 {
t.Errorf("cable end %s has no single matching end", pair)
}
}
if len(ends) != 6 {
t.Errorf("%d cable ends, want 6", len(ends))
}
}
func TestNetworkConfig_Hypervisor(t *testing.T) {
@ -287,7 +270,7 @@ func TestNetworkConfig_Hypervisor(t *testing.T) {
}
under := iface(t, doc, "underlay")
if under.Match.MACAddress != "02:4c:00:02:00:00" || under.SetName != "underlay" || under.MTU != 9000 ||
!reflect.DeepEqual(under.Addresses, []string{"10.250.0.4/24"}) {
!reflect.DeepEqual(under.Addresses, []string{"10.250.0.3/24"}) {
t.Errorf("underlay = %+v", under)
}
if !reflect.DeepEqual(under.Routes, []route{{To: "0.0.0.0/0", Via: "10.250.0.1"}}) {
@ -306,27 +289,17 @@ func TestNetworkConfig_DHCPIsExplicitlyOffEverywhere(t *testing.T) {
}
}
func TestNetworkConfig_Router(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, twoSegments), "sw"))
if len(doc.Ethernets) != 3 {
t.Fatalf("network-config = %+v", doc)
}
func TestNetworkConfig_Switch(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, twoHypervisors), "sw1"))
admin := iface(t, doc, "mgmt0")
if !reflect.DeepEqual(admin.Routes, []route{{To: "0.0.0.0/0", Via: "10.0.2.2"}}) ||
admin.Nameservers == nil || !reflect.DeepEqual(admin.Nameservers.Addresses, []string{"10.0.2.3"}) {
t.Errorf("router mgmt0 = %+v", admin)
t.Errorf("switch mgmt0 = %+v", admin)
}
for name, want := range map[string]struct {
mac, address string
mtu int
}{
"red": {"02:4c:00:00:00:00", "10.1.0.2/24", 9000},
"blue": {"02:4c:00:00:01:00", "10.2.0.2/24", 1500},
} {
e := iface(t, doc, name)
if e.Match.MACAddress != want.mac || e.SetName != name || e.MTU != want.mtu ||
!reflect.DeepEqual(e.Addresses, []string{want.address}) || len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v", name, e)
for port, mac := range map[string]string{"p0": "02:4c:00:01:00:01", "p1": "02:4c:00:02:00:01", "p2": "02:4c:00:03:00:01"} {
e := iface(t, doc, port)
if e.Match.MACAddress != mac || e.SetName != port || e.MTU != 9000 || len(e.Addresses) != 0 || len(e.Routes) != 0 {
t.Errorf("%s = %+v", port, e)
}
}
}
@ -360,71 +333,45 @@ func TestUserData_PasswordLoginIsExplicitlyOff(t *testing.T) {
}
}
func TestNetworkConfig_DefaultRouteOnTheFirstSegmentWithAGateway(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, withVRRP), "hv"))
pure, red := iface(t, doc, "pure"), iface(t, doc, "red")
if len(pure.Routes) != 0 || pure.Nameservers != nil || !reflect.DeepEqual(pure.Addresses, []string{"10.3.0.3/24"}) {
t.Errorf("pure = %+v", pure)
}
if !reflect.DeepEqual(red.Routes, []route{{To: "0.0.0.0/0", Via: "10.1.0.1"}}) || red.Nameservers == nil ||
!reflect.DeepEqual(red.Addresses, []string{"10.1.0.4/24"}) {
t.Errorf("red = %+v", red)
}
}
func TestNetworkConfig_NoDefaultRouteWithoutGateway(t *testing.T) {
for name, e := range network(t, nodeNamed(t, renderAll(t, withVRRP), "l2")).Ethernets {
if len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v, want no route", name, e)
}
}
}
func TestNetworkConfig_RouterHasNoDefaultRouteOnItsSegments(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, withVRRP), "r1"))
for _, name := range []string{"red", "blue"} {
if e := iface(t, doc, name); len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v", name, e)
}
}
if admin := iface(t, doc, "mgmt0"); len(admin.Routes) != 1 {
t.Errorf("mgmt0 = %+v", admin)
}
}
func TestUserData_RouterBuildsGatewayAndNAT(t *testing.T) {
func TestUserData_SwitchBuildsBridgeGatewayAndNAT(t *testing.T) {
cfg := user(t, nodeNamed(t, renderAll(t, twoHypervisors), "sw1"))
if !reflect.DeepEqual(cfg.Packages, []string{"nftables"}) {
t.Errorf("packages = %v", cfg.Packages)
}
script := fileAt(t, cfg, "/usr/local/sbin/lab-router")
script := fileAt(t, cfg, "/usr/local/sbin/lab-switch")
if script.Permissions != "0755" {
t.Errorf("script permissions = %s", script.Permissions)
}
wantScript := `#!/bin/sh
set -eu
sysctl -qw net.ipv4.ip_forward=1
ip addr replace 10.250.0.1/24 dev underlay
nft -f /etc/lab-router.nft
ip link add br-underlay type bridge stp_state 0 2>/dev/null || true
ip link set dev p0 master br-underlay
ip link set dev p0 up
ip link set dev p1 master br-underlay
ip link set dev p1 up
ip link set dev p2 master br-underlay
ip link set dev p2 up
ip link set dev br-underlay mtu 9000
ip addr replace 10.250.0.1/24 dev br-underlay
ip link set dev br-underlay up
nft -f /etc/lab-switch.nft
`
if script.Content != wantScript {
t.Errorf("script:\n%s\nwant:\n%s", script.Content, wantScript)
}
nftFile := fileAt(t, cfg, "/etc/lab-router.nft")
if nftFile.Permissions != "0644" {
t.Errorf("nft permissions = %s", nftFile.Permissions)
}
nft := fileAt(t, cfg, "/etc/lab-switch.nft").Content
for _, want := range []string{
"add table ip lab_nat\ndelete table ip lab_nat\n",
"type nat hook postrouting priority srcnat;",
`ip saddr { 10.250.0.0/24 } oifname "mgmt0" masquerade`,
} {
if !strings.Contains(nftFile.Content, want) {
t.Errorf("nft rules miss %q:\n%s", want, nftFile.Content)
if !strings.Contains(nft, want) {
t.Errorf("nft rules miss %q:\n%s", want, nft)
}
}
unit := fileAt(t, cfg, "/etc/systemd/system/lab-router.service").Content
for _, want := range []string{"Type=oneshot", "RemainAfterExit=yes", "ExecStart=/usr/local/sbin/lab-router", "After=network-online.target", "WantedBy=multi-user.target"} {
unit := fileAt(t, cfg, "/etc/systemd/system/lab-switch.service").Content
for _, want := range []string{"Type=oneshot", "RemainAfterExit=yes", "ExecStart=/usr/local/sbin/lab-switch", "After=network-online.target", "WantedBy=multi-user.target"} {
if !strings.Contains(unit, want) {
t.Errorf("unit misses %q", want)
}
@ -432,82 +379,29 @@ nft -f /etc/lab-router.nft
if !reflect.DeepEqual(cfg.Runcmd, [][]string{{"/usr/local/sbin/lab-provision"}}) {
t.Errorf("runcmd = %v", cfg.Runcmd)
}
if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-router.service\n" {
if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-switch.service\n" {
t.Errorf("lab-provision:\n%s", got)
}
}
func TestUserData_RouterWithTwoSegments(t *testing.T) {
func TestUserData_SwitchWithTwoSegments(t *testing.T) {
cfg := user(t, nodeNamed(t, renderAll(t, twoSegments), "sw"))
script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content
want := "sysctl -qw net.ipv4.ip_forward=1\nip addr replace 10.1.0.1/24 dev red\nip addr replace 10.2.0.1/24 dev blue\nnft -f /etc/lab-router.nft\n"
if !strings.HasSuffix(script, want) {
t.Errorf("script:\n%s\ndoes not end with:\n%s", script, want)
}
if nft := fileAt(t, cfg, "/etc/lab-router.nft").Content; !strings.Contains(nft, "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }") {
t.Errorf("nft:\n%s", nft)
}
}
func TestUserData_RouterScriptWithVRRPAndSingleSegments(t *testing.T) {
script := fileAt(t, user(t, nodeNamed(t, renderAll(t, withVRRP), "r1")), "/usr/local/sbin/lab-router").Content
want := `#!/bin/sh
set -eu
sysctl -qw net.ipv4.ip_forward=1
sysctl -qw net.ipv4.conf.red.arp_ignore=1 net.ipv4.conf.red.arp_announce=2
ip link show dev vrred >/dev/null 2>&1 || { ip link add vrred link red type macvlan mode bridge && ip link set dev vrred protodown on; }
ip link set dev vrred addrgenmode random address 00:00:5e:00:01:01
ip addr replace 10.1.0.1/24 dev vrred
ip link set dev vrred up
ip addr replace 10.2.0.1/24 dev blue
nft -f /etc/lab-router.nft
`
if script != want {
t.Errorf("script:\n%s\nwant:\n%s", script, want)
}
r2 := fileAt(t, user(t, nodeNamed(t, renderAll(t, withVRRP), "r2")), "/usr/local/sbin/lab-router").Content
if !strings.Contains(r2, "ip addr replace 10.1.0.1/24 dev vrred\n") || strings.Contains(r2, "dev pure") {
t.Errorf("r2 script:\n%s", r2)
}
}
func TestUserData_RouterNATCoversTheSegmentsItRoutes(t *testing.T) {
nodes := renderAll(t, withVRRP)
for name, want := range map[string]string{
"r1": "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }",
"r2": "ip saddr { 10.1.0.0/24 }",
script := fileAt(t, cfg, "/usr/local/sbin/lab-switch").Content
for _, want := range []string{
"ip link set dev p0 master br-red\n",
"ip link set dev p1 master br-blue\n",
"ip link set dev br-blue mtu 1500\n",
"ip addr replace 10.2.0.1/24 dev br-blue\n",
} {
if nft := fileAt(t, user(t, nodeNamed(t, nodes, name)), "/etc/lab-router.nft").Content; !strings.Contains(nft, want) {
t.Errorf("%s nft:\n%s\nmisses %s", name, nft, want)
if !strings.Contains(script, want) {
t.Errorf("script misses %q:\n%s", want, script)
}
}
cfg := user(t, nodeNamed(t, nodes, "r3"))
if nft := fileAt(t, cfg, "/etc/lab-router.nft").Content; nft != "add table ip lab_nat\ndelete table ip lab_nat\n" {
t.Errorf("r3 nft:\n%s", nft)
if strings.Contains(script, "p1 master br-red") || strings.Contains(script, "p0 master br-blue") {
t.Errorf("port bridged on the wrong segment:\n%s", script)
}
if script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content; script != "#!/bin/sh\nset -eu\nsysctl -qw net.ipv4.ip_forward=1\nnft -f /etc/lab-router.nft\n" {
t.Errorf("r3 script:\n%s", script)
}
}
func TestVRRPMAC(t *testing.T) {
for vrid, want := range map[int]string{1: "00:00:5e:00:01:01", 16: "00:00:5e:00:01:10", 255: "00:00:5e:00:01:ff"} {
if got := vrrpMAC(vrid); got != want {
t.Errorf("vrrpMAC(%d) = %s, want %s", vrid, got, want)
}
}
}
func TestVRRPPriorityFollowsTheRoutersOrder(t *testing.T) {
var routers []string
for i := 0; i < 22; i++ {
routers = append(routers, "r"+strconv.Itoa(i))
}
s := topology.SegmentPlan{Routers: routers}
for name, want := range map[string]int{"r0": 200, "r1": 190, "r19": 10, "r20": 1, "r21": 1} {
if got := vrrpPriority(s, name); got != want {
t.Errorf("priority of %s = %d, want %d", name, got, want)
}
if nft := fileAt(t, cfg, "/etc/lab-switch.nft").Content; !strings.Contains(nft, "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }") {
t.Errorf("nft:\n%s", nft)
}
}
@ -521,7 +415,7 @@ func TestMetaData(t *testing.T) {
}
}
func TestRender_AdminMacsAreUniqueAndApartFromPortMacs(t *testing.T) {
func TestRender_AdminMacsAreUniqueAndApartFromCableMacs(t *testing.T) {
seen := map[string]string{}
for name, n := range renderAll(t, twoHypervisors) {
for _, d := range value(t, n.QEMU, "-device") {
@ -535,8 +429,8 @@ func TestRender_AdminMacsAreUniqueAndApartFromPortMacs(t *testing.T) {
}
}
}
if len(seen) != 8 {
t.Errorf("%d distinct macs, want 8", len(seen))
if len(seen) != 10 {
t.Errorf("%d distinct macs, want 10", len(seen))
}
}

View file

@ -15,9 +15,9 @@ images:
url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS
segments:
underlay: { routers: [sw1], cidr: 192.168.14.0/24 }
underlay: { switch: sw1, cidr: 192.168.14.0/24 }
nodes:
sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024, secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: rr1.conf }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: hv1.conf, provision: hv.sh, post_provision: hv-post.sh, files: { agent.yml: two.yml }, services: [agent.service] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
@ -79,20 +79,16 @@ sed -i 's/^bgpd=no/bgpd=yes/' /etc/frr/daemons
func TestRoles_SecondaryAddressFollowsThePrimaryOnTheNode(t *testing.T) {
doc := network(t, nodeNamed(t, renderRoles(t), "rr1"))
got := iface(t, doc, "underlay").Addresses
if !reflect.DeepEqual(got, []string{"192.168.14.3/24", "169.254.0.3/28"}) {
if !reflect.DeepEqual(got, []string{"192.168.14.2/24", "169.254.0.3/28"}) {
t.Errorf("rr1 underlay addresses = %v", got)
}
}
func TestRoles_RouterCarriesItsSecondaryOnItsSegmentInterface(t *testing.T) {
sw1 := nodeNamed(t, renderRoles(t), "sw1")
under := iface(t, network(t, sw1), "underlay")
if !reflect.DeepEqual(under.Addresses, []string{"192.168.14.2/24", "169.254.0.1/28"}) || len(under.Routes) != 0 {
t.Errorf("sw1 underlay = %+v", under)
}
script := fileAt(t, user(t, sw1), "/usr/local/sbin/lab-router").Content
if !strings.Contains(script, "ip addr replace 192.168.14.1/24 dev underlay\n") || strings.Contains(script, "169.254.0.1") {
t.Errorf("lab-router:\n%s", script)
func TestRoles_SwitchCarriesItsSecondaryOnTheBridge(t *testing.T) {
script := fileAt(t, user(t, nodeNamed(t, renderRoles(t), "sw1")), "/usr/local/sbin/lab-switch").Content
want := "ip addr replace 192.168.14.1/24 dev br-underlay\nip addr replace 169.254.0.1/28 dev br-underlay\nip link set dev br-underlay up\n"
if !strings.Contains(script, want) {
t.Errorf("lab-switch:\n%s\ndoes not contain:\n%s", script, want)
}
}
@ -160,7 +156,7 @@ func TestRoles_ProvisioningIsOneScriptThatStopsAtTheFirstFailure(t *testing.T) {
env := "LAB_NODE=hv1 LAB_UPLINK=underlay LAB_FILES=/etc/lab/files "
files := "mkdir -p /etc/lab/files /run/lab-files\nmount -o ro LABEL=LABFILES /run/lab-files\ncp -R /run/lab-files/. /etc/lab/files/\numount /run/lab-files\n"
for name, steps := range map[string]string{
"sw1": "systemctl daemon-reload\nsystemctl enable --now lab-router.service\n/usr/local/sbin/lab-frr\n",
"sw1": "systemctl daemon-reload\nsystemctl enable --now lab-switch.service\n/usr/local/sbin/lab-frr\n",
"hv1": files + env + "/usr/local/sbin/lab-node-provision\n/usr/local/sbin/lab-frr\n" + env + "/usr/local/sbin/lab-node-post-provision\n",
} {
cfg := user(t, nodeNamed(t, nodes, name))
@ -242,40 +238,6 @@ func TestRoles_UplinkIsTheInterfaceCarryingTheDefaultRoute(t *testing.T) {
}
}
func TestRoles_UplinkSkipsSegmentsWithoutGateway(t *testing.T) {
p := plan(t, withVRRP)
want := map[string]string{
"hv": "LAB_NODE=hv LAB_UPLINK=red LAB_FILES=/etc/lab/files ",
"l2": "LAB_NODE=l2 LAB_UPLINK=pure LAB_FILES=/etc/lab/files ",
"r1": "LAB_NODE=r1 LAB_UPLINK=mgmt0 LAB_FILES=/etc/lab/files ",
"r3": "LAB_NODE=r3 LAB_UPLINK=mgmt0 LAB_FILES=/etc/lab/files ",
}
for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok {
if got := provisionEnv(p, n); got != w {
t.Errorf("%s provision environment = %q, want %q", n.Name, got, w)
}
}
}
}
func TestRoles_UplinkIsGivenToTheProvisionScript(t *testing.T) {
doc := strings.Replace(withVRRP, "segments: [pure, red] }", "segments: [pure, red], provision: hv.sh }", 1)
nodes, err := Render(plan(t, doc), Options{RunDir: "/srv/lab", AuthorizedKeys: []string{labKey}, Provision: map[string]string{"hv": "#!/bin/sh\n"}})
if err != nil {
t.Fatalf("Render: %v", err)
}
for _, n := range nodes {
if n.Name != "hv" {
continue
}
got := fileAt(t, user(t, n), "/usr/local/sbin/lab-provision").Content
if got != "#!/bin/sh\nset -eu\nLAB_NODE=hv LAB_UPLINK=red LAB_FILES=/etc/lab/files /usr/local/sbin/lab-node-provision\n" {
t.Errorf("lab-provision:\n%s", got)
}
}
}
func TestRoles_NodesWithoutProvisionReceiveNoProvisionFiles(t *testing.T) {
nodes := renderRoles(t)
for _, name := range []string{"sw1", "rr1", "hv2"} {
@ -291,67 +253,8 @@ func TestRoles_FRRDaemons(t *testing.T) {
p := plan(t, withRoles)
want := map[string][]string{"sw1": {"bgpd", "bfdd"}, "rr1": {"bgpd", "bfdd"}, "hv1": {"bgpd"}}
for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(p, n), w) {
t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(p, n), w)
}
}
}
func TestRoles_VRRPAddsVrrpdOnlyToTheRoutersThatRunIt(t *testing.T) {
p := plan(t, withVRRP)
want := map[string][]string{"r1": {"bgpd", "bfdd", "vrrpd"}, "r2": {"bgpd", "bfdd", "vrrpd"}, "r3": {"bgpd", "bfdd"}, "hv": {"bgpd"}}
needs := map[string]bool{"r1": true, "r2": true, "r3": false, "hv": false, "l2": false}
for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(p, n), w) {
t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(p, n), w)
}
if got := NeedsFRR(p, n); got != needs[n.Name] {
t.Errorf("NeedsFRR(%s) = %v, want %v", n.Name, got, needs[n.Name])
}
}
}
func TestRoles_VRRPRouterWithoutFRRFileGetsAGeneratedConfig(t *testing.T) {
nodes := renderAll(t, withVRRP)
cfg := user(t, nodeNamed(t, nodes, "r1"))
want := "frr defaults traditional\nhostname r1\n!\ninterface red\n vrrp 1 version 3\n vrrp 1 priority 200\n no vrrp 1 preempt\n vrrp 1 ip 10.1.0.1\nexit\n!\n"
if f := fileAt(t, cfg, "/etc/lab/frr.conf"); f.Content != want || f.Permissions != "0640" {
t.Errorf("r1 frr.conf (%s):\n%s\nwant:\n%s", f.Permissions, f.Content, want)
}
vrrpd := "sed -i 's/^vrrpd=no/vrrpd=yes/' /etc/frr/daemons\n"
bfd := "sed -i 's/^bfdd=no/bfdd=yes/' /etc/frr/daemons\n"
tail := "install -o frr -g frr -m 0640 /etc/lab/frr.conf /etc/frr/frr.conf\nsystemctl restart frr\n"
if f := fileAt(t, cfg, "/usr/local/sbin/lab-frr"); f.Content != frrScriptHead+bfd+vrrpd+tail {
t.Errorf("r1 lab-frr:\n%s", f.Content)
}
fileAt(t, cfg, "/usr/share/keyrings/frrouting.gpg")
want = "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-router.service\n/usr/local/sbin/lab-frr\n"
if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != want {
t.Errorf("r1 lab-provision:\n%s\nwant:\n%s", got, want)
}
for _, name := range []string{"r3", "hv", "l2"} {
for _, f := range user(t, nodeNamed(t, nodes, name)).WriteFiles {
if strings.Contains(f.Path, "frr") {
t.Errorf("%s receives %s", name, f.Path)
}
}
}
}
func TestRoles_VRRPStanzaFollowsTheUserFRRConfig(t *testing.T) {
doc := strings.Replace(withVRRP, "segments: [red, pure] }", "segments: [red, pure], frr: r2.conf }", 1)
o := Options{RunDir: "/srv/lab", AuthorizedKeys: []string{labKey}, FRR: map[string]string{"r2": "hostname r2\nrouter bgp 65001\n\n\n"}}
nodes, err := Render(plan(t, doc), o)
if err != nil {
t.Fatalf("Render: %v", err)
}
for _, n := range nodes {
if n.Name != "r2" {
continue
}
want := "hostname r2\nrouter bgp 65001\ninterface red\n vrrp 1 version 3\n vrrp 1 priority 190\n no vrrp 1 preempt\n vrrp 1 ip 10.1.0.1\nexit\n!\n"
if got := fileAt(t, user(t, n), "/etc/lab/frr.conf").Content; got != want {
t.Errorf("r2 frr.conf:\n%s\nwant:\n%s", got, want)
if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(n), w) {
t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(n), w)
}
}
}
@ -372,7 +275,7 @@ func TestRoles_RefusesAnUnreadFRRConfig(t *testing.T) {
}
}
func TestRoles_RouterLoopbackIsCreatedByTheRouterScript(t *testing.T) {
func TestRoles_SwitchLoopbackIsCreatedByTheSwitchScript(t *testing.T) {
doc := strings.Replace(withRoles, "secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf", "secondary: { underlay: [169.254.0.1/28] }, loopback: 10.255.254.1/32, frr: sw1.conf", 1)
nodes, err := Render(plan(t, doc), roleOptions())
if err != nil {
@ -384,16 +287,10 @@ func TestRoles_RouterLoopbackIsCreatedByTheRouterScript(t *testing.T) {
sw1 = n
}
}
cfg := user(t, sw1)
script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content
want := "ip addr replace 192.168.14.1/24 dev underlay\nip link add lo1 type dummy 2>/dev/null || true\nip addr replace 10.255.254.1/32 dev lo1\nip link set dev lo1 up\nnft -f /etc/lab-router.nft\n"
script := fileAt(t, user(t, sw1), "/usr/local/sbin/lab-switch").Content
want := "ip link set dev br-underlay up\nip link add lo1 type dummy 2>/dev/null || true\nip addr replace 10.255.254.1/32 dev lo1\nip link set dev lo1 up\nnft -f /etc/lab-switch.nft\n"
if !strings.HasSuffix(script, want) {
t.Errorf("lab-router:\n%s\ndoes not end with:\n%s", script, want)
}
for _, f := range cfg.WriteFiles {
if strings.HasPrefix(f.Path, "/usr/local/sbin/lab-node") || strings.Contains(f.Path, "lab-node.service") {
t.Errorf("sw1 receives %s", f.Path)
}
t.Errorf("lab-switch:\n%s\ndoes not end with:\n%s", script, want)
}
}

View file

@ -12,10 +12,9 @@ import (
)
const (
RouterScript = "/usr/local/sbin/lab-router"
RouterNFT = "/etc/lab-router.nft"
RouterUnit = "/etc/systemd/system/lab-router.service"
VRRPPrefix = "vr"
SwitchScript = "/usr/local/sbin/lab-switch"
SwitchNFT = "/etc/lab-switch.nft"
SwitchUnit = "/etc/systemd/system/lab-switch.service"
NodeScript = "/usr/local/sbin/lab-node"
NodeUnit = "/etc/systemd/system/lab-node.service"
@ -100,14 +99,14 @@ func userData(p *topology.Plan, n topology.NodePlan, o Options) ([]byte, error)
}
var steps []string
switch {
case n.Role == topology.RoleRouter:
case n.Role == topology.RoleSwitch:
cfg.Packages = []string{"nftables"}
cfg.WriteFiles = []writeFile{
{Path: RouterScript, Permissions: "0755", Content: routerScript(p, n)},
{Path: RouterNFT, Permissions: "0644", Content: routerNFT(p, n.Name)},
{Path: RouterUnit, Permissions: "0644", Content: unit("Lab router: gateways and NAT", RouterScript)},
{Path: SwitchScript, Permissions: "0755", Content: switchScript(p, n)},
{Path: SwitchNFT, Permissions: "0644", Content: switchNFT(p, n.Name)},
{Path: SwitchUnit, Permissions: "0644", Content: unit("Lab switch: bridges, gateways and NAT", SwitchScript)},
}
steps = append(steps, "systemctl daemon-reload", "systemctl enable --now lab-router.service")
steps = append(steps, "systemctl daemon-reload", "systemctl enable --now lab-switch.service")
case n.Loopback.IsValid():
cfg.WriteFiles = []writeFile{
{Path: NodeScript, Permissions: "0755", Content: "#!/bin/sh\nset -eu\n" + loopbackLines(n)},
@ -127,11 +126,11 @@ func userData(p *topology.Plan, n topology.NodePlan, o Options) ([]byte, error)
cfg.WriteFiles = append(cfg.WriteFiles, writeFile{Path: NodeProvision, Permissions: "0755", Encoding: "b64", Content: base64.StdEncoding.EncodeToString([]byte(o.Provision[n.Name]))})
steps = append(steps, provisionEnv(p, n)+NodeProvision)
}
if NeedsFRR(p, n) {
if n.FRR != "" {
cfg.WriteFiles = append(cfg.WriteFiles,
writeFile{Path: FRRKey, Permissions: "0644", Encoding: "b64", Content: base64.StdEncoding.EncodeToString(frrKey)},
writeFile{Path: FRRConfig, Permissions: "0640", Content: frrConfig(p, n, o.FRR[n.Name])},
writeFile{Path: FRRScript, Permissions: "0755", Content: frrScript(p, n)},
writeFile{Path: FRRConfig, Permissions: "0640", Content: o.FRR[n.Name]},
writeFile{Path: FRRScript, Permissions: "0755", Content: frrScript(n)},
)
steps = append(steps, FRRScript)
}
@ -157,77 +156,27 @@ func retry(command string) string {
}
func provisionEnv(p *topology.Plan, n topology.NodePlan) string {
port, _ := uplink(p, n.Name)
up := port.Interface
if n.Role == topology.RoleRouter {
up = AdminInterface
uplink := ""
if cables := nodeCables(p, n.Name); len(cables) > 0 {
uplink = cables[0].NodeInterface
}
return fmt.Sprintf("LAB_NODE=%s LAB_UPLINK=%s LAB_FILES=%s ", n.Name, up, FilesDir)
return fmt.Sprintf("LAB_NODE=%s LAB_UPLINK=%s LAB_FILES=%s ", n.Name, uplink, FilesDir)
}
func vrrpSegments(p *topology.Plan, name string) []topology.SegmentPlan {
var out []topology.SegmentPlan
for _, s := range routedSegments(p, name) {
if s.VRID != 0 {
out = append(out, s)
}
func frrDaemons(n topology.NodePlan) []string {
if n.Role == topology.RoleHost {
return []string{"bgpd"}
}
return out
return []string{"bgpd", "bfdd"}
}
func NeedsFRR(p *topology.Plan, n topology.NodePlan) bool {
return n.FRR != "" || len(vrrpSegments(p, n.Name)) > 0
}
func frrDaemons(p *topology.Plan, n topology.NodePlan) []string {
daemons := []string{"bgpd"}
if n.Role != topology.RoleHost {
daemons = append(daemons, "bfdd")
}
if len(vrrpSegments(p, n.Name)) > 0 {
daemons = append(daemons, "vrrpd")
}
return daemons
}
func frrConfig(p *topology.Plan, n topology.NodePlan, given string) string {
segments := vrrpSegments(p, n.Name)
if len(segments) == 0 {
return given
}
var b strings.Builder
if given == "" {
fmt.Fprintf(&b, "frr defaults traditional\nhostname %s\n!\n", n.Name)
} else {
b.WriteString(strings.TrimRight(given, "\n") + "\n")
}
for _, s := range segments {
fmt.Fprintf(&b, "interface %s\n vrrp %d version 3\n vrrp %d priority %d\n no vrrp %d preempt\n vrrp %d ip %s\nexit\n!\n",
s.Name, s.VRID, s.VRID, vrrpPriority(s, n.Name), s.VRID, s.VRID, s.Gateway)
}
return b.String()
}
func vrrpPriority(s topology.SegmentPlan, name string) int {
for i, r := range s.Routers {
if r == name {
return max(200-10*i, 1)
}
}
return 1
}
func vrrpMAC(vrid int) string {
return fmt.Sprintf("00:00:5e:00:01:%02x", vrid)
}
func frrScript(p *topology.Plan, n topology.NodePlan) string {
func frrScript(n topology.NodePlan) string {
var b strings.Builder
b.WriteString("#!/bin/sh\nset -eu\nexport DEBIAN_FRONTEND=noninteractive\n. /etc/os-release\n")
fmt.Fprintf(&b, "echo \"deb [signed-by=%s] %s ${VERSION_CODENAME} %s\" > /etc/apt/sources.list.d/frr.list\n", FRRKey, FRRRepo, FRRSuite)
b.WriteString(retry("apt-get update -qq --error-on=any"))
fmt.Fprintf(&b, "apt-get install -y -qq --no-install-recommends %s\n", FRRPackages)
for _, d := range frrDaemons(p, n) {
for _, d := range frrDaemons(n) {
fmt.Fprintf(&b, "sed -i 's/^%s=no/%s=yes/' /etc/frr/daemons\n", d, d)
}
fmt.Fprintf(&b, "install -o frr -g frr -m 0640 %s /etc/frr/frr.conf\n", FRRConfig)
@ -250,62 +199,77 @@ func networkConfig(p *topology.Plan, n topology.NodePlan, index int) ([]byte, er
SetName: AdminInterface,
Addresses: []string{AdminAddress},
}
router := n.Role == topology.RoleRouter
if router {
if n.Role == topology.RoleSwitch {
admin.Routes = []route{{To: "0.0.0.0/0", Via: AdminGateway}}
admin.Nameservers = &nameservers{Addresses: []string{AdminDNS}}
for _, c := range switchCables(p, n.Name) {
doc.Ethernets[c.SwitchInterface] = ethernet{
Match: match{MACAddress: c.SwitchMAC.String()},
SetName: c.SwitchInterface,
MTU: c.MTU,
}
}
}
doc.Ethernets[AdminInterface] = admin
up, routed := uplink(p, n.Name)
for _, port := range nodePorts(p, n.Name) {
for i, c := range nodeCables(p, n.Name) {
e := ethernet{
Match: match{MACAddress: port.MAC.String()},
SetName: port.Interface,
MTU: port.MTU,
Addresses: []string{port.Address.String()},
Match: match{MACAddress: c.NodeMAC.String()},
SetName: c.NodeInterface,
MTU: c.MTU,
Addresses: []string{c.NodeAddress.String()},
}
for _, prefix := range n.Secondary[port.Segment] {
for _, prefix := range n.Secondary[c.Segment] {
e.Addresses = append(e.Addresses, prefix.String())
}
if !router && routed && port.Segment == up.Segment {
e.Routes = []route{{To: "0.0.0.0/0", Via: segmentNamed(p, port.Segment).Gateway.String()}}
if i == 0 {
e.Routes = []route{{To: "0.0.0.0/0", Via: gatewayOf(p, c.Segment)}}
e.Nameservers = &nameservers{Addresses: Nameservers}
}
doc.Ethernets[port.Interface] = e
doc.Ethernets[c.NodeInterface] = e
}
return yaml.Marshal(doc)
}
func routerScript(p *topology.Plan, n topology.NodePlan) string {
func gatewayOf(p *topology.Plan, segment string) string {
for _, s := range p.Segments {
if s.Name == segment {
return s.Gateway.String()
}
}
return ""
}
func switchScript(p *topology.Plan, n topology.NodePlan) string {
name := n.Name
var b strings.Builder
b.WriteString("#!/bin/sh\nset -eu\nsysctl -qw net.ipv4.ip_forward=1\n")
for _, s := range routedSegments(p, n.Name) {
gateway := fmt.Sprintf("%s/%d", s.Gateway, s.Network.Bits())
if s.VRID == 0 {
fmt.Fprintf(&b, "ip addr replace %s dev %s\n", gateway, s.Name)
continue
for _, s := range switchSegments(p, name) {
fmt.Fprintf(&b, "ip link add %s type bridge stp_state 0 2>/dev/null || true\n", s.Bridge)
for _, c := range switchCables(p, name) {
if c.Segment != s.Name {
continue
}
fmt.Fprintf(&b, "ip link set dev %s master %s\n", c.SwitchInterface, s.Bridge)
fmt.Fprintf(&b, "ip link set dev %s up\n", c.SwitchInterface)
}
vrrp := VRRPPrefix + s.Name
fmt.Fprintf(&b, "sysctl -qw net.ipv4.conf.%s.arp_ignore=1 net.ipv4.conf.%s.arp_announce=2\n", s.Name, s.Name)
fmt.Fprintf(&b, "ip link show dev %s >/dev/null 2>&1 || { ip link add %s link %s type macvlan mode bridge && ip link set dev %s protodown on; }\n", vrrp, vrrp, s.Name, vrrp)
fmt.Fprintf(&b, "ip link set dev %s addrgenmode random address %s\n", vrrp, vrrpMAC(s.VRID))
fmt.Fprintf(&b, "ip addr replace %s dev %s\n", gateway, vrrp)
fmt.Fprintf(&b, "ip link set dev %s up\n", vrrp)
fmt.Fprintf(&b, "ip link set dev %s mtu %d\n", s.Bridge, s.MTU)
fmt.Fprintf(&b, "ip addr replace %s/%d dev %s\n", s.Gateway, s.Network.Bits(), s.Bridge)
for _, prefix := range n.Secondary[s.Name] {
fmt.Fprintf(&b, "ip addr replace %s dev %s\n", prefix, s.Bridge)
}
fmt.Fprintf(&b, "ip link set dev %s up\n", s.Bridge)
}
b.WriteString(loopbackLines(n))
fmt.Fprintf(&b, "nft -f %s\n", RouterNFT)
fmt.Fprintf(&b, "nft -f %s\n", SwitchNFT)
return b.String()
}
func routerNFT(p *topology.Plan, name string) string {
func switchNFT(p *topology.Plan, name string) string {
var networks []string
for _, s := range routedSegments(p, name) {
for _, s := range switchSegments(p, name) {
networks = append(networks, s.Network.String())
}
if len(networks) == 0 {
return "add table ip lab_nat\ndelete table ip lab_nat\n"
}
return fmt.Sprintf(`add table ip lab_nat
delete table ip lab_nat
table ip lab_nat {

View file

@ -12,7 +12,7 @@ import (
func (p *Plan) Write(w io.Writer) error {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintf(tw, "lab %s: nodes %d, segments %d, ports %d\n", p.Name, len(p.Nodes), len(p.Segments), len(p.Ports))
fmt.Fprintf(tw, "lab %s: nodes %d, segments %d, cables %d\n", p.Name, len(p.Nodes), len(p.Segments), len(p.Cables))
fmt.Fprintf(tw, "\nnodes\n")
fmt.Fprintf(tw, " name\trole\timage\tcpus\tmemory\tssh\n")
@ -37,29 +37,21 @@ func (p *Plan) Write(w io.Writer) error {
}
for _, s := range p.Segments {
fmt.Fprintf(tw, "\nsegment %s: %s, mtu %d, bridge %s, %s\n", s.Name, s.Network, s.MTU, s.Bridge, gateway(s))
fmt.Fprintf(tw, " node\tinterface\taddress\tmac\ttap\n")
for _, port := range p.Ports {
if port.Segment != s.Name {
fmt.Fprintf(tw, "\nsegment %s: %s, mtu %d, switch %s, bridge %s, gateway %s\n",
s.Name, s.Network, s.MTU, s.Switch, s.Bridge, s.Gateway)
fmt.Fprintf(tw, " node\tinterface\taddress\tmac\tudp\t\tswitch port\tmac\tudp\n")
for _, c := range p.Cables {
if c.Segment != s.Name {
continue
}
fmt.Fprintf(tw, " %s\t%s\t%s\t%s\t%s\n", port.Node, port.Interface, port.Address, port.MAC, port.Tap)
fmt.Fprintf(tw, " %s\t%s\t%s\t%s\t%d\t<->\t%s %s\t%s\t%d\n",
c.Node, c.NodeInterface, c.NodeAddress, c.NodeMAC, c.NodePort,
c.Switch, c.SwitchInterface, c.SwitchMAC, c.SwitchPort)
}
}
return tw.Flush()
}
func gateway(s SegmentPlan) string {
switch {
case len(s.Routers) == 0:
return "no gateway"
case s.VRID == 0:
return fmt.Sprintf("gateway %s on %s", s.Gateway, s.Routers[0])
default:
return fmt.Sprintf("gateway %s by vrrp %d on %s", s.Gateway, s.VRID, strings.Join(s.Routers, ", "))
}
}
func loopback(n NodePlan) string {
if !n.Loopback.IsValid() {
return ""

View file

@ -9,10 +9,8 @@ import (
)
const (
SSHBasePort = 2200
BridgePrefix = "lb-"
TapPrefix = "lt"
SSHBasePort = 2200
CableBasePort = 20000
maxNodes = 1000
maxSegments = 256
@ -23,16 +21,15 @@ type Plan struct {
Images []Image
Segments []SegmentPlan
Nodes []NodePlan
Ports []Port
Cables []Cable
}
type SegmentPlan struct {
Name string
Switch string
Bridge string
Network netip.Prefix
Gateway netip.Addr
Routers []string
VRID int
MTU int
}
@ -52,14 +49,20 @@ type NodePlan struct {
Services []string
}
type Port struct {
Segment string
MTU int
Node string
Interface string
MAC net.HardwareAddr
Address netip.Prefix
Tap string
type Cable struct {
Segment string
MTU int
Node string
NodeInterface string
NodeMAC net.HardwareAddr
NodeAddress netip.Prefix
NodePort int
Switch string
SwitchInterface string
SwitchMAC net.HardwareAddr
SwitchPort int
}
func Compute(t *Topology) (*Plan, error) {
@ -99,22 +102,17 @@ func Compute(t *Topology) (*Plan, error) {
var errs []error
for segIndex, s := range t.Segments {
network := netip.MustParsePrefix(s.CIDR)
segment := SegmentPlan{
gateway := network.Addr().Next()
p.Segments = append(p.Segments, SegmentPlan{
Name: s.Name,
Bridge: BridgePrefix + s.Name,
Switch: s.Switch,
Bridge: "br-" + s.Name,
Network: network,
Routers: append([]string(nil), s.Routers...),
Gateway: gateway,
MTU: s.MTU,
}
if len(s.Routers) > 0 {
segment.Gateway = network.Addr().Next()
}
if len(s.Routers) > 1 {
segment.VRID = segIndex + 1
}
p.Segments = append(p.Segments, segment)
})
addresses, err := allocate(t, s, network, segment.Gateway)
addresses, err := allocate(t, s, network, gateway)
if err != nil {
errs = append(errs, err)
continue
@ -125,18 +123,26 @@ func Compute(t *Topology) (*Plan, error) {
if !ok {
continue
}
p.Ports = append(p.Ports, Port{
Segment: s.Name,
MTU: s.MTU,
Node: n.Name,
Interface: s.Name,
MAC: mac(nodeIndex, segIndex),
Address: netip.PrefixFrom(address, network.Bits()),
Tap: TapPrefix + strconv.Itoa(len(p.Ports)),
cable := len(p.Cables)
p.Cables = append(p.Cables, Cable{
Segment: s.Name,
MTU: s.MTU,
Node: n.Name,
NodeInterface: s.Name,
NodeMAC: mac(nodeIndex, segIndex, 0),
NodeAddress: netip.PrefixFrom(address, network.Bits()),
NodePort: CableBasePort + 2*cable,
Switch: s.Switch,
SwitchInterface: "p" + strconv.Itoa(cable),
SwitchMAC: mac(nodeIndex, segIndex, 1),
SwitchPort: CableBasePort + 2*cable + 1,
})
}
}
if last := CableBasePort + 2*len(p.Cables) - 1; last > 65535 {
errs = append(errs, fmt.Errorf("cables: %d cables need udp ports up to %d, beyond 65535", len(p.Cables), last))
}
if len(errs) > 0 {
return nil, errors.Join(errs...)
}
@ -145,12 +151,7 @@ func Compute(t *Topology) (*Plan, error) {
func allocate(t *Topology, s Segment, network netip.Prefix, gateway netip.Addr) (map[string]netip.Addr, error) {
broadcast := lastAddr(network)
used := map[netip.Addr]string{}
first := network.Addr().Next()
if gateway.IsValid() {
used[gateway] = "the gateway"
first = gateway.Next()
}
used := map[netip.Addr]string{gateway: s.Switch + " (gateway)"}
result := map[string]netip.Addr{}
var errs []error
@ -178,7 +179,7 @@ func allocate(t *Topology, s Segment, network netip.Prefix, gateway netip.Addr)
}
}
next := first
next := gateway.Next()
for _, name := range auto {
for next != broadcast && used[next] != "" {
next = next.Next()
@ -205,8 +206,8 @@ func lastAddr(network netip.Prefix) netip.Addr {
return netip.AddrFrom4([4]byte{byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)})
}
func mac(node, segment int) net.HardwareAddr {
return net.HardwareAddr{0x02, 0x4c, byte(node >> 8), byte(node), byte(segment), 0x00}
func mac(node, segment, side int) net.HardwareAddr {
return net.HardwareAddr{0x02, 0x4c, byte(node >> 8), byte(node), byte(segment), byte(side)}
}
func contains(list []string, value string) bool {

View file

@ -27,26 +27,15 @@ func computeError(t *testing.T, doc string) string {
return err.Error()
}
func portOf(t *testing.T, p *Plan, node, segment string) Port {
func cableOf(t *testing.T, p *Plan, node, segment string) Cable {
t.Helper()
for _, port := range p.Ports {
if port.Node == node && port.Segment == segment {
return port
for _, c := range p.Cables {
if c.Node == node && c.Segment == segment {
return c
}
}
t.Fatalf("no port for %s on %s", node, segment)
return Port{}
}
func segmentOf(t *testing.T, p *Plan, name string) SegmentPlan {
t.Helper()
for _, s := range p.Segments {
if s.Name == name {
return s
}
}
t.Fatalf("no segment %s", name)
return SegmentPlan{}
t.Fatalf("no cable for %s on %s", node, segment)
return Cable{}
}
func nodeOf(t *testing.T, p *Plan, name string) NodePlan {
@ -62,12 +51,12 @@ func nodeOf(t *testing.T, p *Plan, name string) NodePlan {
const twoHypervisors = header + `
segments:
underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 }
nodes:
sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
`
func TestCompute_TwoHypervisorsPlan(t *testing.T) {
@ -77,34 +66,34 @@ func TestCompute_TwoHypervisorsPlan(t *testing.T) {
t.Fatalf("%d segments, want 1", len(p.Segments))
}
s := p.Segments[0]
if s.Bridge != "lb-underlay" || s.Gateway != netip.MustParseAddr("10.250.0.1") || s.MTU != 9000 || !reflect.DeepEqual(s.Routers, []string{"sw1"}) || s.VRID != 0 {
if s.Bridge != "br-underlay" || s.Gateway != netip.MustParseAddr("10.250.0.1") || s.MTU != 9000 || s.Switch != "sw1" {
t.Errorf("segment = %+v", s)
}
want := []struct {
node, address, mac, tap string
node, address, nodeMAC, switchMAC, switchIface string
nodePort, switchPort int
}{
{"sw1", "10.250.0.2/24", "02:4c:00:00:00:00", "lt0"},
{"rr1", "10.250.0.3/24", "02:4c:00:01:00:00", "lt1"},
{"hv1", "10.250.0.4/24", "02:4c:00:02:00:00", "lt2"},
{"hv2", "10.250.0.5/24", "02:4c:00:03:00:00", "lt3"},
{"rr1", "10.250.0.2/24", "02:4c:00:01:00:00", "02:4c:00:01:00:01", "p0", 20000, 20001},
{"hv1", "10.250.0.3/24", "02:4c:00:02:00:00", "02:4c:00:02:00:01", "p1", 20002, 20003},
{"hv2", "10.250.0.4/24", "02:4c:00:03:00:00", "02:4c:00:03:00:01", "p2", 20004, 20005},
}
if len(p.Ports) != len(want) {
t.Fatalf("%d ports, want %d", len(p.Ports), len(want))
if len(p.Cables) != len(want) {
t.Fatalf("%d cables, want %d", len(p.Cables), len(want))
}
for _, w := range want {
port := portOf(t, p, w.node, "underlay")
if port.Address.String() != w.address {
t.Errorf("%s address = %s, want %s", w.node, port.Address, w.address)
c := cableOf(t, p, w.node, "underlay")
if c.NodeAddress.String() != w.address {
t.Errorf("%s address = %s, want %s", w.node, c.NodeAddress, w.address)
}
if port.MAC.String() != w.mac {
t.Errorf("%s mac = %s, want %s", w.node, port.MAC, w.mac)
if c.NodeMAC.String() != w.nodeMAC || c.SwitchMAC.String() != w.switchMAC {
t.Errorf("%s macs = %s / %s, want %s / %s", w.node, c.NodeMAC, c.SwitchMAC, w.nodeMAC, w.switchMAC)
}
if port.Tap != w.tap {
t.Errorf("%s tap = %s, want %s", w.node, port.Tap, w.tap)
if c.NodePort != w.nodePort || c.SwitchPort != w.switchPort {
t.Errorf("%s ports = %d / %d, want %d / %d", w.node, c.NodePort, c.SwitchPort, w.nodePort, w.switchPort)
}
if port.Interface != "underlay" || port.MTU != 9000 {
t.Errorf("%s port = %+v", w.node, port)
if c.NodeInterface != "underlay" || c.SwitchInterface != w.switchIface || c.Switch != "sw1" || c.MTU != 9000 {
t.Errorf("%s cable = %+v", w.node, c)
}
}
@ -138,206 +127,104 @@ func TestCompute_IsStableAcrossRuns(t *testing.T) {
func TestCompute_AddressesFollowDeclarationOrderNotNames(t *testing.T) {
p := compute(t, header+`
segments:
under: { routers: [sw], cidr: 10.0.0.0/24 }
under: { switch: sw, cidr: 10.0.0.0/24 }
nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
alfa: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
`)
for node, want := range map[string]string{"zeta": "10.0.0.2/24", "sw": "10.0.0.3/24", "alfa": "10.0.0.4/24"} {
if got := portOf(t, p, node, "under").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
if got := cableOf(t, p, "zeta", "under").NodeAddress.String(); got != "10.0.0.2/24" {
t.Errorf("zeta = %s, want 10.0.0.2/24", got)
}
if got := cableOf(t, p, "alfa", "under").NodeAddress.String(); got != "10.0.0.3/24" {
t.Errorf("alfa = %s, want 10.0.0.3/24", got)
}
}
func TestCompute_ExplicitAddressIsKeptAndSkippedByAllocation(t *testing.T) {
p := compute(t, header+`
segments:
under: { routers: [sw], cidr: 10.0.0.0/24 }
under: { switch: sw, cidr: 10.0.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.254} }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.3} }
c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
`)
for node, want := range map[string]string{"sw": "10.0.0.254/24", "a": "10.0.0.2/24", "b": "10.0.0.3/24", "c": "10.0.0.4/24"} {
if got := portOf(t, p, node, "under").Address.String(); got != want {
for node, want := range map[string]string{"a": "10.0.0.2/24", "b": "10.0.0.3/24", "c": "10.0.0.4/24"} {
if got := cableOf(t, p, node, "under").NodeAddress.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithoutRouterHasNoGateway(t *testing.T) {
func TestCompute_TwoSegmentsCableOrderMacsAndPorts(t *testing.T) {
p := compute(t, header+`
segments:
l2: { cidr: 10.3.0.0/24 }
red: { switch: sw, cidr: 10.1.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24, mtu: 1500 }
nodes:
a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
`)
s := segmentOf(t, p, "l2")
if s.Gateway.IsValid() || s.VRID != 0 || len(s.Routers) != 0 || s.Bridge != "lb-l2" {
t.Errorf("segment = %+v, want no gateway", s)
}
for node, want := range map[string]string{"a": "10.3.0.1/24", "b": "10.3.0.2/24"} {
if got := portOf(t, p, node, "l2").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithoutRouterAcceptsTheFirstAddress(t *testing.T) {
p := compute(t, header+`
segments:
l2: { cidr: 10.3.0.0/24 }
nodes:
a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2], addresses: {l2: 10.3.0.1} }
`)
for node, want := range map[string]string{"a": "10.3.0.2/24", "b": "10.3.0.1/24"} {
if got := portOf(t, p, node, "l2").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithOneRouter(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
nodes:
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red] }
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
`)
s := segmentOf(t, p, "red")
if s.Gateway != netip.MustParseAddr("10.1.0.1") || s.VRID != 0 || !reflect.DeepEqual(s.Routers, []string{"r1"}) {
t.Errorf("segment = %+v", s)
}
for node, want := range map[string]string{"hv": "10.1.0.2/24", "r1": "10.1.0.3/24"} {
if got := portOf(t, p, node, "red").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithTwoRoutersUsesVRRP(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
blue: { routers: [r2, r1], cidr: 10.2.0.0/24 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue], addresses: {blue: 10.2.0.250} }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue, pure] }
`)
blue := segmentOf(t, p, "blue")
if blue.VRID != 2 || blue.Gateway != netip.MustParseAddr("10.2.0.1") || !reflect.DeepEqual(blue.Routers, []string{"r2", "r1"}) {
t.Errorf("blue = %+v", blue)
}
if red := segmentOf(t, p, "red"); red.VRID != 0 {
t.Errorf("red vrid = %d, want 0", red.VRID)
}
if pure := segmentOf(t, p, "pure"); pure.VRID != 0 || pure.Gateway.IsValid() {
t.Errorf("pure = %+v", pure)
}
for node, want := range map[string]string{"r1": "10.2.0.2/24", "r2": "10.2.0.250/24", "hv": "10.2.0.3/24"} {
if got := portOf(t, p, node, "blue").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_TwoSegmentsPortOrderMacsAndTaps(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { cidr: 10.2.0.0/24, mtu: 1500 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red] }
`)
got := make([]string, 0, len(p.Ports))
for _, port := range p.Ports {
got = append(got, port.Segment+"/"+port.Node+"/"+port.Tap)
got := make([]string, 0, len(p.Cables))
for _, c := range p.Cables {
got = append(got, c.Segment+"/"+c.Node+"/"+c.SwitchInterface)
}
if strings.Join(got, ",") != "red/sw/lt0,red/hv/lt1,red/rr/lt2,blue/hv/lt3,blue/rr/lt4" {
t.Errorf("port order = %v", got)
if strings.Join(got, ",") != "red/hv/p0,red/rr/p1,blue/hv/p2" {
t.Errorf("cable order = %v", got)
}
blue := portOf(t, p, "hv", "blue")
if blue.MAC.String() != "02:4c:00:01:01:00" || blue.Tap != "lt3" || blue.MTU != 1500 || blue.Interface != "blue" || blue.Address.String() != "10.2.0.1/24" {
blue := cableOf(t, p, "hv", "blue")
if blue.NodeMAC.String() != "02:4c:00:01:01:00" || blue.NodePort != 20004 || blue.MTU != 1500 || blue.NodeAddress.String() != "10.2.0.2/24" {
t.Errorf("hv on blue = %+v", blue)
}
if red := portOf(t, p, "hv", "red"); red.MAC.String() != "02:4c:00:01:00:00" || red.Address.String() != "10.1.0.3/24" {
t.Errorf("hv on red = %+v", red)
}
if rr := portOf(t, p, "rr", "blue"); rr.MAC.String() != "02:4c:00:02:01:00" || rr.Address.String() != "10.2.0.2/24" {
t.Errorf("rr on blue = %+v", rr)
if red := cableOf(t, p, "hv", "red"); red.NodeMAC.String() != "02:4c:00:01:00:00" {
t.Errorf("hv on red mac = %s", red.NodeMAC)
}
}
func TestCompute_MacEncodesNodeAndSegmentRanks(t *testing.T) {
p := compute(t, generated(3, 300))
if got := portOf(t, p, "n299", "s2").MAC.String(); got != "02:4c:01:2b:02:00" {
t.Errorf("n299 on s2 mac = %s, want 02:4c:01:2b:02:00", got)
}
}
func TestCompute_EveryMacTapAndPortIsUnique(t *testing.T) {
func TestCompute_EveryMacAndPortIsUnique(t *testing.T) {
p := compute(t, header+`
segments:
a: { routers: [sw], cidr: 10.1.0.0/24 }
b: { routers: [sw], cidr: 10.2.0.0/24 }
a: { switch: sw, cidr: 10.1.0.0/24 }
b: { switch: sw, cidr: 10.2.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [a, b] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
n1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] }
n2: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] }
n3: { role: host, image: deb, cpus: 1, memory: 512, segments: [b, a] }
`)
macs := map[string]bool{}
taps := map[string]bool{}
ports := map[int]bool{}
for _, n := range p.Nodes {
if ports[n.SSHPort] {
t.Errorf("ssh port %d used twice", n.SSHPort)
}
ports[n.SSHPort] = true
}
for _, port := range p.Ports {
if macs[port.MAC.String()] {
t.Errorf("mac %s used twice", port.MAC)
for _, c := range p.Cables {
for _, m := range []string{c.NodeMAC.String(), c.SwitchMAC.String()} {
if macs[m] {
t.Errorf("mac %s used twice", m)
}
macs[m] = true
}
macs[port.MAC.String()] = true
if taps[port.Tap] {
t.Errorf("tap %s used twice", port.Tap)
for _, port := range []int{c.NodePort, c.SwitchPort} {
if ports[port] {
t.Errorf("port %d used twice", port)
}
ports[port] = true
}
taps[port.Tap] = true
}
if len(macs) != 8 || len(taps) != 8 {
t.Errorf("%d distinct macs and %d distinct taps, want 8", len(macs), len(taps))
if len(macs) != 12 {
t.Errorf("%d distinct macs, want 12", len(macs))
}
}
func TestCompute_MacsAreLocallyAdministeredUnicast(t *testing.T) {
for _, port := range compute(t, twoHypervisors).Ports {
if m := port.MAC.String(); !strings.HasPrefix(m, "02:") {
t.Errorf("mac %s is not locally administered unicast", m)
}
}
}
func TestCompute_InterfaceAndTapNamesFitInIFNAMSIZ(t *testing.T) {
p := compute(t, generated(23, 1000))
for _, port := range p.Ports {
if len(port.Tap) > 15 || len(port.Interface) > 15 {
t.Fatalf("port %+v has a name longer than 15", port)
}
}
for _, s := range p.Segments {
if len(s.Bridge) > 15 {
t.Fatalf("bridge %s is longer than 15", s.Bridge)
for _, c := range compute(t, twoHypervisors).Cables {
for _, m := range []string{c.NodeMAC.String(), c.SwitchMAC.String()} {
if !strings.HasPrefix(m, "02:") {
t.Errorf("mac %s is not locally administered unicast", m)
}
}
}
}
@ -346,9 +233,9 @@ func TestCompute_AddressRejections(t *testing.T) {
base := func(addresses string) string {
return header + `
segments:
under: { routers: [sw], cidr: 10.0.0.0/29 }
under: { switch: sw, cidr: 10.0.0.0/29 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under]` + addresses + ` }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.5} }
`
@ -357,7 +244,7 @@ nodes:
{"outside", ", addresses: {under: 10.0.1.2}", "node a: address 10.0.1.2 is outside segment under"},
{"network", ", addresses: {under: 10.0.0.0}", "node a: address 10.0.0.0 is the network or broadcast"},
{"broadcast", ", addresses: {under: 10.0.0.7}", "node a: address 10.0.0.7 is the network or broadcast"},
{"gateway", ", addresses: {under: 10.0.0.1}", "node a: address 10.0.0.1 on segment under is already taken by the gateway"},
{"gateway", ", addresses: {under: 10.0.0.1}", "node a: address 10.0.0.1 on segment under is already taken by sw (gateway)"},
{"duplicate", ", addresses: {under: 10.0.0.5}", "node b: address 10.0.0.5 on segment under is already taken by a"},
}
for _, c := range cases {
@ -367,58 +254,37 @@ nodes:
}
}
func TestCompute_RouterAddressCannotBeTheGateway(t *testing.T) {
got := computeError(t, header+`
segments:
under: { routers: [sw], cidr: 10.0.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.1} }
`)
requireContains(t, got, "node sw: address 10.0.0.1 on segment under is already taken by the gateway")
}
func TestCompute_ReportsExhaustedSegment(t *testing.T) {
got := computeError(t, header+`
segments:
tiny: { routers: [sw], cidr: 10.0.0.0/30 }
tiny: { switch: sw, cidr: 10.0.0.0/30 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [tiny] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`)
requireContains(t, got, "segment tiny: no address left in 10.0.0.0/30 for node a")
requireContains(t, got, "segment tiny: no address left in 10.0.0.0/30 for node b")
}
func TestCompute_FillsSegmentExactly(t *testing.T) {
p := compute(t, header+`
segments:
tiny: { routers: [sw], cidr: 10.0.0.0/29 }
tiny: { switch: sw, cidr: 10.0.0.0/29 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [tiny] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
e: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`)
if got := portOf(t, p, "d", "tiny").Address.String(); got != "10.0.0.6/29" {
if got := cableOf(t, p, "e", "tiny").NodeAddress.String(); got != "10.0.0.6/29" {
t.Errorf("last node = %s, want 10.0.0.6/29", got)
}
}
func TestCompute_FillsSegmentWithoutRouterExactly(t *testing.T) {
p := compute(t, header+`
segments:
tiny: { cidr: 10.0.0.0/30 }
nodes:
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`)
if got := portOf(t, p, "b", "tiny").Address.String(); got != "10.0.0.2/30" {
t.Errorf("last node = %s, want 10.0.0.2/30", got)
}
}
func TestCompute_RefusesInvalidTopology(t *testing.T) {
requireContains(t, computeError(t, strings.Replace(valid, "role: rr", "role: spine", 1)), `role "spine"`)
requireContains(t, computeError(t, strings.Replace(valid, "role: rr", "role: router", 1)), `role "router"`)
}
func TestLastAddr(t *testing.T) {
@ -439,86 +305,41 @@ func TestWrite_TwoHypervisorsPlan(t *testing.T) {
if err := compute(t, twoHypervisors).Write(&buf); err != nil {
t.Fatalf("Write: %v", err)
}
want := `lab lab-test: nodes 4, segments 1, ports 4
want := `lab lab-test: nodes 4, segments 1, cables 3
nodes
name role image cpus memory ssh
sw1 router deb 2 1024 MiB 127.0.0.1:2200
sw1 switch deb 2 1024 MiB 127.0.0.1:2200
rr1 rr deb 1 1024 MiB 127.0.0.1:2201
hv1 host deb 4 16384 MiB 127.0.0.1:2202
hv2 host deb 4 16384 MiB 127.0.0.1:2203
segment underlay: 10.250.0.0/24, mtu 9000, bridge lb-underlay, gateway 10.250.0.1 on sw1
node interface address mac tap
sw1 underlay 10.250.0.2/24 02:4c:00:00:00:00 lt0
rr1 underlay 10.250.0.3/24 02:4c:00:01:00:00 lt1
hv1 underlay 10.250.0.4/24 02:4c:00:02:00:00 lt2
hv2 underlay 10.250.0.5/24 02:4c:00:03:00:00 lt3
segment underlay: 10.250.0.0/24, mtu 9000, switch sw1, bridge br-underlay, gateway 10.250.0.1
node interface address mac udp switch port mac udp
rr1 underlay 10.250.0.2/24 02:4c:00:01:00:00 20000 <-> sw1 p0 02:4c:00:01:00:01 20001
hv1 underlay 10.250.0.3/24 02:4c:00:02:00:00 20002 <-> sw1 p1 02:4c:00:02:00:01 20003
hv2 underlay 10.250.0.4/24 02:4c:00:03:00:00 20004 <-> sw1 p2 02:4c:00:03:00:01 20005
`
if buf.String() != want {
t.Errorf("output:\n%s\nwant:\n%s", buf.String(), want)
}
}
func TestWrite_GatewayLines(t *testing.T) {
var buf bytes.Buffer
if err := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
blue: { routers: [r2, r1], cidr: 10.2.0.0/24, mtu: 1500 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, pure] }
`).Write(&buf); err != nil {
t.Fatalf("Write: %v", err)
}
for _, want := range []string{
"lab lab-test: nodes 3, segments 3, ports 5\n",
"\nsegment red: 10.1.0.0/24, mtu 9000, bridge lb-red, gateway 10.1.0.1 on r1\n",
"\nsegment blue: 10.2.0.0/24, mtu 1500, bridge lb-blue, gateway 10.2.0.1 by vrrp 2 on r2, r1\n",
"\nsegment pure: 10.3.0.0/24, mtu 9000, bridge lb-pure, no gateway\n",
" hv pure 10.3.0.1/24 02:4c:00:02:02:00 lt4\n",
} {
if !strings.Contains(buf.String(), want) {
t.Errorf("plan:\n%s\ndoes not contain %q", buf.String(), want)
}
}
}
func generated(segments, nodes int) string {
var b strings.Builder
b.WriteString(header + "segments:\n")
names := make([]string, segments)
for i := range names {
names[i] = fmt.Sprintf("s%d", i)
fmt.Fprintf(&b, " %s: { routers: [sw], cidr: 10.%d.0.0/16 }\n", names[i], i%250)
fmt.Fprintf(&b, " %s: { switch: sw, cidr: 10.%d.0.0/16 }\n", names[i], i%250)
}
fmt.Fprintf(&b, "nodes:\n sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", strings.Join(names, ", "))
b.WriteString("nodes:\n sw: { role: switch, image: deb, cpus: 1, memory: 512 }\n")
for i := 1; i < nodes; i++ {
fmt.Fprintf(&b, " n%d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", i, strings.Join(names, ", "))
}
return b.String()
}
func withVRRPAt(segments, rank int) string {
var b strings.Builder
b.WriteString(header + "segments:\n")
names := make([]string, segments)
for i := range names {
names[i] = fmt.Sprintf("s%d", i)
routers := "[r1]"
if i == rank {
routers = "[r1, r2]"
}
fmt.Fprintf(&b, " %s: { routers: %s, cidr: 10.%d.0.0/16 }\n", names[i], routers, i%250)
}
fmt.Fprintf(&b, "nodes:\n r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", strings.Join(names, ", "))
fmt.Fprintf(&b, " r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", names[rank])
return b.String()
}
func TestValidate_RejectsTooManyNodes(t *testing.T) {
requireContains(t, validationError(t, generated(1, 1001)), "nodes: 1001 declared, at most 1000")
}
@ -533,24 +354,6 @@ func TestValidate_RejectsTooManySegments(t *testing.T) {
requireContains(t, validationError(t, generated(257, 2)), "segments: 257 declared, at most 256")
}
func TestValidate_RejectsVRRPBeyondTheVRIDRange(t *testing.T) {
msg := validationError(t, withVRRPAt(256, 255))
requireContains(t, msg, "segment s255: a segment with several routers must be among the first 255")
}
func TestCompute_AcceptsVRRPOnTheLastVRID(t *testing.T) {
p := compute(t, withVRRPAt(256, 254))
if got := segmentOf(t, p, "s254").VRID; got != 255 {
t.Errorf("vrid = %d, want 255", got)
}
}
func TestCompute_ManyPortsNeedNoPortRange(t *testing.T) {
p := compute(t, generated(23, 1000))
if len(p.Ports) != 23000 {
t.Fatalf("%d ports, want 23000", len(p.Ports))
}
if last := p.Ports[len(p.Ports)-1].Tap; last != "lt22999" {
t.Errorf("last tap = %s, want lt22999", last)
}
func TestCompute_RejectsCablesBeyondUDPPortRange(t *testing.T) {
requireContains(t, computeError(t, generated(23, 1000)), "cables: 22977 cables need udp ports up to 65953, beyond 65535")
}

View file

@ -11,9 +11,9 @@ import (
const withRoles = header + `
segments:
underlay: { routers: [sw1], cidr: 192.168.14.0/24 }
underlay: { switch: sw1, cidr: 192.168.14.0/24 }
nodes:
sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024, secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: frr/rr1.conf }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: /etc/lab/hv1.conf, provision: provision/hv.sh, post_provision: /opt/lab/post.sh, files: { agent.yml: agent/two.yml, deploy.sh: /opt/deploy.sh }, services: [agent.service] }
`
@ -95,10 +95,10 @@ func TestValidate_RoleFieldRejections(t *testing.T) {
t.Run(name, func(t *testing.T) {
doc := header + `
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { routers: [sw], cidr: 10.2.0.0/24 }
red: { switch: sw, cidr: 10.1.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
` + c.node + `
`
@ -107,29 +107,29 @@ nodes:
}
}
func TestValidate_RouterSecondaryOnlyOnItsOwnSegments(t *testing.T) {
func TestValidate_SwitchSecondaryOnlyOnItsOwnSegments(t *testing.T) {
doc := header + `
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { routers: [other], cidr: 10.2.0.0/24 }
red: { switch: sw, cidr: 10.1.0.0/24 }
blue: { switch: other, cidr: 10.2.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red], secondary: { red: [169.254.0.1/28], blue: [169.254.1.1/28] } }
other: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
sw: { role: switch, image: deb, cpus: 1, memory: 512, secondary: { red: [169.254.0.1/28], blue: [169.254.1.1/28] } }
other: { role: switch, image: deb, cpus: 1, memory: 512 }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
`
msg := validationError(t, doc)
requireContains(t, msg, "node sw: secondary address given for segment blue it is not attached to")
if bytes.Contains([]byte(msg), []byte("segment red")) {
t.Errorf("the router's own segment was refused:\n%s", msg)
t.Errorf("the switch's own segment was refused:\n%s", msg)
}
}
func TestValidate_LoopbackInterfaceNameIsReserved(t *testing.T) {
doc := header + `
segments:
lo1: { routers: [sw], cidr: 10.1.0.0/24 }
lo1: { switch: sw, cidr: 10.1.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [lo1] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [lo1] }
`
requireContains(t, validationError(t, doc), "segment lo1: name is reserved for the loopback interface")
@ -156,9 +156,9 @@ func TestWrite_NoRolesSectionWithoutRoleFields(t *testing.T) {
var buf bytes.Buffer
doc := header + `
segments:
underlay: { routers: [sw1], cidr: 10.250.0.0/24 }
underlay: { switch: sw1, cidr: 10.250.0.0/24 }
nodes:
sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay] }
sw1: { role: switch, image: deb, cpus: 1, memory: 512 }
rr1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [underlay] }
`
if err := compute(t, doc).Write(&buf); err != nil {
@ -176,7 +176,7 @@ func TestValidate_ProvisionRejections(t *testing.T) {
}{
"the hypervisor role is gone": {
`hv: { role: hypervisor, image: deb, cpus: 1, memory: 512, segments: [red] }`,
`node hv: role "hypervisor" must be one of router, rr, host`,
`node hv: role "hypervisor" must be one of switch, rr, host`,
},
"file name leaving the files directory": {
`hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { ../etc/passwd: a.yml } }`,
@ -203,9 +203,9 @@ func TestValidate_ProvisionRejections(t *testing.T) {
t.Run(name, func(t *testing.T) {
doc := header + `
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
red: { switch: sw, cidr: 10.1.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
` + c.node + `
`
requireContains(t, validationError(t, doc), c.want)
@ -216,9 +216,9 @@ nodes:
func TestValidate_ProvisionIsAllowedOnEveryRole(t *testing.T) {
doc := header + `
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
red: { switch: sw, cidr: 10.1.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red], provision: sw.sh, files: { a.conf: a.conf }, services: [lab-router.service] }
sw: { role: switch, image: deb, cpus: 1, memory: 512, provision: sw.sh, files: { a.conf: a.conf }, services: [lab-switch.service] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red], provision: rr.sh, post_provision: rr-post.sh }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { agent.yml: agent.yml }, services: [agent.service] }
`

View file

@ -11,7 +11,7 @@ import (
)
const (
RoleRouter = "router"
RoleSwitch = "switch"
RoleRR = "rr"
RoleHost = "host"
@ -32,12 +32,10 @@ type Image struct {
}
type Segment struct {
Name string
Routers []string
CIDR string
MTU int
legacySwitch string
Name string
Switch string
CIDR string
MTU int
}
type Node struct {
@ -63,10 +61,9 @@ type fileImage struct {
}
type fileSegment struct {
Routers []string `yaml:"routers"`
CIDR string `yaml:"cidr"`
MTU int `yaml:"mtu"`
Switch string `yaml:"switch"`
Switch string `yaml:"switch"`
CIDR string `yaml:"cidr"`
MTU int `yaml:"mtu"`
}
type fileNode struct {
@ -148,7 +145,7 @@ func Parse(data []byte) (*Topology, error) {
if mtu == 0 {
mtu = DefaultMTU
}
t.Segments = append(t.Segments, Segment{Name: name, Routers: s.Routers, CIDR: s.CIDR, MTU: mtu, legacySwitch: s.Switch})
t.Segments = append(t.Segments, Segment{Name: name, Switch: s.Switch, CIDR: s.CIDR, MTU: mtu})
}
for _, name := range order["nodes"] {
n := f.Nodes[name]

View file

@ -40,10 +40,10 @@ func requireContains(t *testing.T, got, want string) {
func TestParse_KeepsDeclarationOrderOfNodesAndSegments(t *testing.T) {
topo := parse(t, header+`
segments:
zulu: { routers: [sw], cidr: 10.0.1.0/24 }
alpha: { routers: [sw], cidr: 10.0.2.0/24 }
zulu: { switch: sw, cidr: 10.0.1.0/24 }
alpha: { switch: sw, cidr: 10.0.2.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [zulu, alpha] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu, alpha] }
beta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu] }
`)
@ -65,9 +65,9 @@ nodes:
func TestParse_DefaultsMTUTo9000(t *testing.T) {
topo := parse(t, header+`
segments:
under: { cidr: 10.0.0.0/24 }
under: { switch: sw, cidr: 10.0.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
`)
if topo.Segments[0].MTU != 9000 {
t.Errorf("MTU = %d, want 9000", topo.Segments[0].MTU)
@ -77,7 +77,7 @@ nodes:
func TestParse_RejectsUnknownField(t *testing.T) {
_, err := Parse([]byte(header + `
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, ram: 4 }
sw: { role: switch, image: deb, cpus: 1, memory: 512, ram: 4 }
`))
if err == nil || !strings.Contains(err.Error(), "ram") {
t.Fatalf("unknown field not rejected: %v", err)
@ -87,7 +87,7 @@ nodes:
func TestParse_RejectsDuplicateNode(t *testing.T) {
_, err := Parse([]byte(header + `
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512 }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
sw: { role: rr, image: deb, cpus: 1, memory: 512 }
`))
if err == nil || !strings.Contains(err.Error(), "already defined") {
@ -109,9 +109,9 @@ func TestLoad_ReportsPathOnError(t *testing.T) {
const valid = header + `
segments:
under: { routers: [sw], cidr: 10.0.0.0/24 }
under: { switch: sw, cidr: 10.0.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
`
@ -129,11 +129,9 @@ func TestValidate_Rejections(t *testing.T) {
{"image url not https", strings.Replace(valid, "url: https://", "url: http://", 1), "image deb: url"},
{"image sums not https", strings.Replace(valid, "sums: https://", "sums: ftp://", 1), "image deb: sums"},
{"no node", header + "\nsegments: {}\nnodes: {}\n", "at least one node"},
{"legacy segment switch", strings.Replace(valid, "routers: [sw], cidr", "switch: sw, cidr", 1), "segment under: switch was replaced by routers"},
{"legacy role switch", strings.Replace(valid, "role: router", "role: switch", 1), "node sw: role switch was renamed router"},
{"segment router unknown", strings.Replace(valid, "routers: [sw]", "routers: [ghost]", 1), "segment under: router ghost is not a declared node"},
{"segment router not a router", strings.Replace(valid, "routers: [sw]", "routers: [rr]", 1), "segment under: rr is a rr, not a router"},
{"segment router listed twice", strings.Replace(valid, "routers: [sw]", "routers: [sw, sw]", 1), "segment under: router sw listed twice"},
{"segment switch missing", strings.Replace(valid, "switch: sw, cidr", "cidr", 1), "segment under: switch is required"},
{"segment switch unknown", strings.Replace(valid, "switch: sw, cidr", "switch: ghost, cidr", 1), "switch ghost is not a declared node"},
{"segment switch not a switch", strings.Replace(valid, "switch: sw, cidr", "switch: rr, cidr", 1), "rr is a rr, not a switch"},
{"segment name too long", strings.ReplaceAll(valid, "under", "underlayunder"), "segment underlayunder: name must match"},
{"segment named like the admin interface", strings.ReplaceAll(valid, "under", "mgmt0"), "segment mgmt0: name is reserved"},
{"segment name with dash", strings.ReplaceAll(valid, "under", "un-der"), "segment un-der: name must match"},
@ -145,17 +143,19 @@ func TestValidate_Rejections(t *testing.T) {
{"cidr too small", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/31", 1), "prefix length out of range"},
{"cidr too large", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/7", 1), "prefix length out of range"},
{"bad node name", strings.Replace(valid, " rr: {", " RR: {", 1), "node RR: name must match"},
{"bad role", strings.Replace(valid, "role: rr", "role: spine", 1), `role "spine"`},
{"bad role", strings.Replace(valid, "role: rr", "role: router", 1), `role "router"`},
{"unknown image", strings.Replace(valid, "role: rr, image: deb", "role: rr, image: ubuntu", 1), `image "ubuntu" is not declared`},
{"no cpu", strings.Replace(valid, "role: rr, image: deb, cpus: 1", "role: rr, image: deb, cpus: 0", 1), "node rr: cpus must be at least 1"},
{"memory too low", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512", "role: rr, image: deb, cpus: 1, memory: 255", 1), "node rr: memory must be at least 256"},
{"router without segment", valid + " r2: { role: router, image: deb, cpus: 1, memory: 512 }\n", "node r2: must be attached to at least one segment"},
{"memory too low", strings.Replace(valid, "cpus: 1, memory: 512, segments", "cpus: 1, memory: 255, segments", 1), "node rr: memory must be at least 256"},
{"switch with segments", strings.Replace(valid, "role: switch, image: deb, cpus: 1, memory: 512 }", "role: switch, image: deb, cpus: 1, memory: 512, segments: [under] }", 1), "node sw: a switch carries its segments"},
{"switch with addresses", strings.Replace(valid, "role: switch, image: deb, cpus: 1, memory: 512 }", "role: switch, image: deb, cpus: 1, memory: 512, addresses: {under: 10.0.0.9} }", 1), "node sw: a switch carries its segments"},
{"switch without segment", valid + " sw2: { role: switch, image: deb, cpus: 1, memory: 512 }\n", "node sw2: switch carries no segment"},
{"node without segment", valid + " rr2: { role: rr, image: deb, cpus: 1, memory: 512 }\n", "node rr2: must be attached to at least one segment"},
{"node on unknown segment", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [over]", 1), "node rr: segment over is not declared"},
{"node on segment twice", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under, under]", 1), "node rr: segment under listed twice"},
{"segment without node", strings.Replace(valid, " under: { routers: [sw], cidr: 10.0.0.0/24 }", " under: { routers: [sw], cidr: 10.0.0.0/24 }\n empty: { cidr: 10.0.9.0/24 }", 1), "segment empty: no node is attached"},
{"address on foreign segment", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {over: 10.0.0.9}", 1), "node rr: address given for segment over"},
{"address unparsable", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0}", 1), `node rr: address "10.0.0" on under`},
{"node on unknown segment", strings.Replace(valid, "segments: [under]", "segments: [over]", 1), "node rr: segment over is not declared"},
{"node on segment twice", strings.Replace(valid, "segments: [under]", "segments: [under, under]", 1), "node rr: segment under listed twice"},
{"segment without node", strings.Replace(valid, " under: { switch: sw, cidr: 10.0.0.0/24 }", " under: { switch: sw, cidr: 10.0.0.0/24 }\n empty: { switch: sw, cidr: 10.0.9.0/24 }", 1), "segment empty: no node is attached"},
{"address on foreign segment", strings.Replace(valid, "segments: [under]", "segments: [under], addresses: {over: 10.0.0.9}", 1), "address given for segment over"},
{"address unparsable", strings.Replace(valid, "segments: [under]", "segments: [under], addresses: {under: 10.0.0}", 1), `address "10.0.0" on under`},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
@ -164,53 +164,10 @@ func TestValidate_Rejections(t *testing.T) {
}
}
func TestValidate_RejectsRouterNotAttachedToItsSegment(t *testing.T) {
doc := header + `
segments:
red: { routers: [r], cidr: 10.1.0.0/24 }
blue: { routers: [r], cidr: 10.2.0.0/24 }
nodes:
r: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
`
msg := validationError(t, doc)
requireContains(t, msg, "segment red: router r is not attached to it")
if strings.Contains(msg, "segment blue") {
t.Errorf("the router's own segment was refused:\n%s", msg)
}
}
func TestValidate_AcceptsSegmentWithoutRouter(t *testing.T) {
doc := header + `
segments:
l2: { cidr: 10.3.0.0/24 }
nodes:
a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
`
if err := parse(t, doc).Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
}
func TestValidate_AcceptsTwoRoutersOnASegment(t *testing.T) {
doc := header + `
segments:
under: { routers: [r1, r2], cidr: 10.0.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [under] }
`
if err := parse(t, doc).Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
}
func TestValidate_ReportsEveryErrorAtOnce(t *testing.T) {
doc := strings.Replace(valid, "role: rr", "role: spine", 1)
doc := strings.Replace(valid, "role: rr", "role: router", 1)
doc = strings.Replace(doc, "10.0.0.0/24", "10.0.0.0/31", 1)
got := validationError(t, doc)
requireContains(t, got, `role "spine"`)
requireContains(t, got, `role "router"`)
requireContains(t, got, "prefix length out of range")
}

View file

@ -18,8 +18,6 @@ const (
ReservedInterface = "mgmt0"
LoopbackInterface = "lo1"
MaxVRID = 255
)
var (
@ -76,23 +74,14 @@ func (t *Topology) Validate() error {
if s.Name == LoopbackInterface {
add("segment %s: name is reserved for the loopback interface", s.Name)
}
if s.legacySwitch != "" {
add("segment %s: switch was replaced by routers: declare %s with role %s, attach it to %s through its segments, and list it in segments.%s.routers", s.Name, s.legacySwitch, RoleRouter, s.Name, s.Name)
}
seenRouters := map[string]bool{}
for _, name := range s.Routers {
r, ok := nodes[name]
switch {
case seenRouters[name]:
add("segment %s: router %s listed twice", s.Name, name)
case !ok:
add("segment %s: router %s is not a declared node", s.Name, name)
case r.Role != RoleRouter:
add("segment %s: %s is a %s, not a %s", s.Name, name, r.Role, RoleRouter)
case !contains(r.Segments, s.Name):
add("segment %s: router %s is not attached to it", s.Name, name)
}
seenRouters[name] = true
sw, ok := nodes[s.Switch]
switch {
case s.Switch == "":
add("segment %s: switch is required", s.Name)
case !ok:
add("segment %s: switch %s is not a declared node", s.Name, s.Switch)
case sw.Role != RoleSwitch:
add("segment %s: %s is a %s, not a switch", s.Name, s.Switch, sw.Role)
}
if s.MTU < MinMTU || s.MTU > MaxMTU {
add("segment %s: mtu %d out of range [%d, %d]", s.Name, s.MTU, MinMTU, MaxMTU)
@ -110,25 +99,19 @@ func (t *Topology) Validate() error {
}
}
for i, s := range t.Segments {
if len(s.Routers) > 1 {
if i+1 > MaxVRID {
add("segment %s: a segment with several routers must be among the first %d, its rank sets the vrrp id", s.Name, MaxVRID)
}
}
}
attached := map[string]int{}
hosting := map[string]int{}
for _, s := range t.Segments {
hosting[s.Switch]++
}
for _, n := range t.Nodes {
if !namePattern.MatchString(n.Name) {
add("node %s: name must match %s", n.Name, namePattern)
}
switch n.Role {
case RoleRouter, RoleRR, RoleHost:
case "switch":
add("node %s: role switch was renamed %s: attach it to its segments through segments and addresses, and list it in segments.<name>.routers", n.Name, RoleRouter)
case RoleSwitch, RoleRR, RoleHost:
default:
add("node %s: role %q must be one of %s, %s, %s", n.Name, n.Role, RoleRouter, RoleRR, RoleHost)
add("node %s: role %q must be one of %s, %s, %s", n.Name, n.Role, RoleSwitch, RoleRR, RoleHost)
}
if !images[n.Image] {
add("node %s: image %q is not declared", n.Name, n.Image)
@ -140,6 +123,15 @@ func (t *Topology) Validate() error {
add("node %s: memory must be at least %d MiB", n.Name, MinMemory)
}
validateExtras(n, segments, add)
if n.Role == RoleSwitch {
if len(n.Segments) > 0 || len(n.Addresses) > 0 {
add("node %s: a switch carries its segments through segments.<name>.switch, not through segments or addresses", n.Name)
}
if hosting[n.Name] == 0 {
add("node %s: switch carries no segment", n.Name)
}
continue
}
if len(n.Segments) == 0 {
add("node %s: must be attached to at least one segment", n.Name)
}
@ -182,8 +174,16 @@ func (t *Topology) Validate() error {
func validateExtras(n Node, segments map[string]Segment, add func(string, ...any)) {
carried := map[string]bool{}
for _, name := range n.Segments {
carried[name] = true
if n.Role == RoleSwitch {
for name, s := range segments {
if s.Switch == n.Name {
carried[name] = true
}
}
} else {
for _, name := range n.Segments {
carried[name] = true
}
}
names := make([]string, 0, len(n.Secondary))
for name := range n.Secondary {

View file

@ -1,14 +0,0 @@
# Segments sur des bridges de l'hôte, routeurs et passerelle VRRP
**Changement incompatible** : une topologie 0.1 doit être migrée (voir « Migration depuis 0.1 »
dans `docs/guide.md`).
- Chaque segment est un bridge Linux de l'hôte (`lb-<segment>`), chaque interface de VM un tap
(`lt<i>`) ; plus de sockets UDP. `labctl up` et `labctl down` exigent root, `down` retire les
bridges et les taps, `status` affiche les segments.
- Le rôle `switch` devient `router`, et `segments.<s>.switch` devient `segments.<s>.routers`,
une liste ordonnée. Un segment sans routeur est un L2 pur, sans passerelle.
- Un routeur fait du forwarding et masque vers `mgmt0` les segments qu'il route (service
`lab-router`). Avec deux routeurs ou plus, la passerelle est portée en VRRP v3 par `vrrpd` de
FRR, sans préemption ; bascule d'environ 2,6 s à l'arrêt brutal du maître.
- Exemple `examples/evpn-2hv` : deux routeurs `r1` et `r2` en VRRP.