Compare commits

...

2 commits

Author SHA1 Message Date
88354454ef
Merge branch 'feature-2'
All checks were successful
Tests / test (push) Successful in 4m4s
Release Pipeline / set-release-target (push) Successful in 2s
Release Pipeline / upload-assets (lab-host.sh, scripts/lab-host.sh) (push) Successful in 7s
Release Pipeline / build (amd64, linux) (push) Successful in 0s
Release Pipeline / build (arm64, darwin) (push) Successful in 0s
Release Pipeline / checksums (push) Successful in 6s
Release Pipeline / release (push) Successful in 25s
Release Pipeline / publish (push) Successful in 0s
Release Pipeline / build (push) Successful in 1m40s
2026-10-13 09:57:29 +02:00
e9a0c19b72
labctl: segments sur des bridges de l'hôte, routeurs et passerelle VRRP #2
All checks were successful
Tests / test (push) Successful in 3m46s
Signed-off-by: GnomeZworc <nicolas.boufidjeline@g3e.fr>
2026-10-13 09:57:09 +02:00
24 changed files with 1766 additions and 578 deletions

View file

@ -1,8 +1,8 @@
# labctl # labctl
Monte un **lab réseau virtuel** à partir d'une topologie YAML : des VM QEMU/KVM reliées par des Monte un **lab réseau virtuel** à partir d'une topologie YAML : des VM QEMU/KVM reliées par des
segments L2, des switchs qui font passerelle et NAT, FRR installé et configuré sur les nœuds qui segments L2 (des bridges de l'hôte), des routeurs qui font passerelle — en VRRP quand ils sont
le demandent, et une provision libre (scripts, fichiers, services à vérifier) pour installer ce plusieurs — et NAT, FRR installé et configuré sur les nœuds qui le demandent, et une provision libre (scripts, fichiers, services à vérifier) pour installer ce
que l'on veut tester. Le plan est **déterministe** : même fichier, mêmes adresses, MAC et ports. que l'on veut tester. Le plan est **déterministe** : même fichier, mêmes adresses, MAC et ports.
`labctl` tourne sur tout hôte Linux avec KVM (virtualisation imbriquée si l'hôte est lui-même une `labctl` tourne sur tout hôte Linux avec KVM (virtualisation imbriquée si l'hôte est lui-même une
@ -32,36 +32,43 @@ labctl status | down | ssh <nœud> [commande] [-run rép]
``` ```
Par défaut : `-run ~/lab-run`, `-cache ~/.cache/labctl`, `-timeout 20m`. `up` télécharge et vérifie Par défaut : `-run ~/lab-run`, `-cache ~/.cache/labctl`, `-timeout 20m`. `up` télécharge et vérifie
les images (`SHA512SUMS`), crée des disques neufs, démarre les nœuds (switchs d'abord), attend la les images (`SHA512SUMS`), crée des disques neufs, les bridges des segments et les taps des nœuds,
fin de cloud-init puis vérifie les services de chaque nœud. Un fichier invalide est refusé avec démarre les nœuds (routeurs d'abord), attend la fin de cloud-init puis vérifie les services de
chaque nœud ; `down` arrête les nœuds et supprime bridges et taps. `up` et `down` tournent en
root. Un fichier invalide est refusé avec
toutes ses erreurs et un code de sortie 1. toutes ses erreurs et un code de sortie 1.
## Topologie ## Topologie
Voir `examples/evpn-2hv/` : un switch, un route reflector et deux hôtes en EVPN. Voir `examples/evpn-2hv/` : deux routeurs en VRRP, un route reflector et deux hôtes en EVPN.
- `images` : `url` et `sums` (HTTPS). - `images` : `url` et `sums` (HTTPS).
- `segments` : `switch`, `cidr` IPv4 (/8 à /30), `mtu` (9000 par défaut). Nom ≤ 12 caractères. - `segments` : `cidr` IPv4 (/8 à /30), `mtu` (9000 par défaut), `routers` (facultatif, par ordre
- `nodes` : `role` (`switch`, `rr` ou `host`), `image`, `cpus`, `memory` (Mio), `segments`, de priorité : aucun = L2 pur, un = passerelle sur son interface, plusieurs = passerelle en
`addresses` (adresse fixée par segment). Champs facultatifs : VRRP par FRR). Nom ≤ 12 caractères. Chaque segment est un bridge `lb-<segment>` de l'hôte.
- `nodes` : `role` (`router`, `rr` ou `host`), `image`, `cpus`, `memory` (Mio), `segments`,
`addresses` (adresse fixée par segment). Un routeur est un nœud comme les autres, attaché aux
segments dont il est routeur. Champs facultatifs :
- `secondary` — adresses supplémentaires hors du CIDR du segment ; - `secondary` — adresses supplémentaires hors du CIDR du segment ;
- `loopback` — adresse sur une interface `dummy` `lo1` ; - `loopback` — adresse sur une interface `dummy` `lo1` ;
- `frr` — un `frr.conf` déposé tel quel ; FRR est installé au premier démarrage (`bgpd`, et - `frr` — un `frr.conf` déposé tel quel ; FRR est installé au premier démarrage (`bgpd`, et
`bfdd` sur `switch` et `rr`) ; `bfdd` sur `router` et `rr`, `vrrpd` sur les routeurs en VRRP) ;
- `provision` / `post_provision` — scripts exécutés **avant** / **après** FRR ; - `provision` / `post_provision` — scripts exécutés **avant** / **après** FRR ;
- `files` — `{<nom>: <chemin>}`, fichiers **ou répertoires** gravés dans un ISO propre au nœud et - `files` — `{<nom>: <chemin>}`, fichiers **ou répertoires** gravés dans un ISO propre au nœud et
recopiés dans `/etc/lab/files/<nom>` avant les scripts (aucune limite de taille cloud-init) ; recopiés dans `/etc/lab/files/<nom>` avant les scripts (aucune limite de taille cloud-init) ;
- `services` — unités systemd que `up` vérifie actives. - `services` — unités systemd que `up` vérifie actives.
Les chemins sont relatifs au fichier de topologie. Les scripts reçoivent `LAB_NODE`, Les chemins sont relatifs au fichier de topologie. Les scripts reçoivent `LAB_NODE`,
`LAB_UPLINK` (interface du premier segment, qui porte la route par défaut) et `LAB_FILES`. `LAB_UPLINK` (interface qui porte la route par défaut : premier segment qui a une passerelle,
Ordre au premier démarrage : script du switch ou de la loopback, `provision`, FRR, `mgmt0` pour un routeur)
et `LAB_FILES`. Ordre au premier démarrage : service du routeur ou de la loopback, `provision`, FRR,
`post_provision` — dans un seul script en `set -eu`, la première étape en échec met cloud-init en `post_provision` — dans un seul script en `set -eu`, la première étape en échec met cloud-init en
erreur. erreur.
Déduit de façon déterministe : passerelle d'un segment = première adresse (portée par le switch) ; Déduit de façon déterministe : passerelle d'un segment = première adresse (s'il a un routeur) ;
adresses suivantes dans l'ordre de déclaration des nœuds ; MAC `02:4c:<nœud>:<nœud>:<segment>:<côté>` ; adresses suivantes dans l'ordre de déclaration des nœuds ; MAC `02:4c:<nœud>:<nœud>:<segment>:00` ;
SSH d'administration sur `127.0.0.1:<2200 + rang>`. **Réordonner les nœuds change les adresses.** tap `lt<i>` par port ; SSH d'administration sur `127.0.0.1:<2200 + rang>`. **Réordonner les
nœuds change les adresses.** Migration depuis 0.1 (rôle `switch`) : voir le guide.
## Hôte loué chez Scaleway : `scripts/lab-host.sh` ## Hôte loué chez Scaleway : `scripts/lab-host.sh`

View file

@ -26,8 +26,17 @@ const (
var ( var (
execve = syscall.Exec execve = syscall.Exec
procDir = "/proc" procDir = "/proc"
geteuid = os.Geteuid
) )
func requireRoot(command string, stderr io.Writer) bool {
if geteuid() == 0 {
return true
}
fmt.Fprintf(stderr, "labctl: %s must run as root: it manages the bridges and taps of the lab on this host\n", command)
return false
}
func defaultDir(parts ...string) string { func defaultDir(parts ...string) string {
home, err := os.UserHomeDir() home, err := os.UserHomeDir()
if err != nil { if err != nil {
@ -81,6 +90,9 @@ func upCmd(args []string, stdout, stderr io.Writer) int {
fs.Usage() fs.Usage()
return 2 return 2
} }
if !requireRoot("up", stderr) {
return 1
}
cache, err := filepath.Abs(*cacheDir) cache, err := filepath.Abs(*cacheDir)
if err != nil { if err != nil {
fmt.Fprintf(stderr, "labctl: %v\n", err) fmt.Fprintf(stderr, "labctl: %v\n", err)
@ -154,6 +166,9 @@ func downCmd(args []string, stdout, stderr io.Writer) int {
fs.Usage() fs.Usage()
return 2 return 2
} }
if !requireRoot("down", stderr) {
return 1
}
l, ok := current(*runDir, stdout, stderr) l, ok := current(*runDir, stdout, stderr)
if !ok { if !ok {
return 1 return 1

View file

@ -17,6 +17,12 @@ func runLab(args ...string) (int, string, string) {
return code, stdout.String(), stderr.String() return code, stdout.String(), stderr.String()
} }
func asRoot(t *testing.T) {
t.Helper()
geteuid = func() int { return 0 }
t.Cleanup(func() { geteuid = os.Geteuid })
}
func TestRun_UsageOnMissingArguments(t *testing.T) { func TestRun_UsageOnMissingArguments(t *testing.T) {
for _, args := range [][]string{nil, {"plan"}, {"deploy", "x.yml"}, {"plan", "a.yml", "b.yml"}, {"render", "a.yml", "dir"}, {"render", "-key", "k.pub", "a.yml"}, {"render", "-bogus"}} { for _, args := range [][]string{nil, {"plan"}, {"deploy", "x.yml"}, {"plan", "a.yml", "b.yml"}, {"render", "a.yml", "dir"}, {"render", "-key", "k.pub", "a.yml"}, {"render", "-bogus"}} {
code, _, stderr := runLab(args...) code, _, stderr := runLab(args...)
@ -32,9 +38,9 @@ func TestRun_PlanOfTheShippedExampleTopology(t *testing.T) {
t.Fatalf("code %d, stderr %s", code, stderr) t.Fatalf("code %d, stderr %s", code, stderr)
} }
for _, want := range []string{ for _, want := range []string{
"lab evpn-2hv: nodes 4, segments 1, cables 3", "lab evpn-2hv: nodes 5, segments 1, ports 5",
"gateway 192.168.14.1", "segment underlay: 192.168.14.0/24, mtu 9000, bridge lb-underlay, gateway 192.168.14.1 by vrrp 1 on r1, r2",
"hv2 underlay 192.168.14.12/24 02:4c:00:03:00:00 20004 <-> sw1 p2", "hv2 underlay 192.168.14.12/24 02:4c:00:04:00:00 lt4",
"rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf", "rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf",
} { } {
if !strings.Contains(stdout, want) { if !strings.Contains(stdout, want) {
@ -49,10 +55,10 @@ func TestRun_InvalidTopologyExitsWithErrors(t *testing.T) {
images: images:
deb: { url: https://example.invalid/a, sums: https://example.invalid/b } deb: { url: https://example.invalid/a, sums: https://example.invalid/b }
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/31 } under: { routers: [sw], cidr: 10.0.0.0/31 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
rr: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] } rr: { role: spine, image: deb, cpus: 1, memory: 512, segments: [under] }
` `
if err := os.WriteFile(path, []byte(doc), 0o600); err != nil { if err := os.WriteFile(path, []byte(doc), 0o600); err != nil {
t.Fatal(err) t.Fatal(err)
@ -61,7 +67,7 @@ nodes:
if code != 1 || stdout != "" { if code != 1 || stdout != "" {
t.Fatalf("code %d, stdout %q", code, stdout) t.Fatalf("code %d, stdout %q", code, stdout)
} }
for _, want := range []string{path, `role "router"`, "prefix length out of range"} { for _, want := range []string{path, `role "spine"`, "prefix length out of range"} {
if !strings.Contains(stderr, want) { if !strings.Contains(stderr, want) {
t.Errorf("stderr does not mention %q:\n%s", want, stderr) t.Errorf("stderr does not mention %q:\n%s", want, stderr)
} }
@ -86,7 +92,7 @@ func TestRun_RenderWritesEveryNodeFiles(t *testing.T) {
if code != 0 { if code != 0 {
t.Fatalf("code %d, stderr %s", code, stderr) t.Fatalf("code %d, stderr %s", code, stderr)
} }
for _, node := range []string{"sw1", "rr1", "hv1", "hv2"} { for _, node := range []string{"r1", "r2", "rr1", "hv1", "hv2"} {
if !strings.Contains(stdout, filepath.Join(out, node)) { if !strings.Contains(stdout, filepath.Join(out, node)) {
t.Errorf("stdout does not list %s:\n%s", node, stdout) t.Errorf("stdout does not list %s:\n%s", node, stdout)
} }
@ -105,7 +111,7 @@ func TestRun_RenderWritesEveryNodeFiles(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if !strings.Contains(string(args), "\n-netdev\nuser,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22\n") { if !strings.Contains(string(args), "\n-netdev\nuser,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2203-:22\n-device\nvirtio-net-pci,netdev=mgmt0,mac=02:4d:00:03:00:00,romfile=\n-netdev\ntap,id=underlay,ifname=lt3,script=no,downscript=no\n") {
t.Errorf("qemu.args is not one argument per line:\n%s", args) t.Errorf("qemu.args is not one argument per line:\n%s", args)
} }
userData, err := os.ReadFile(filepath.Join(out, "hv1", "user-data")) userData, err := os.ReadFile(filepath.Join(out, "hv1", "user-data"))
@ -150,6 +156,7 @@ func TestRun_LifecycleUsage(t *testing.T) {
} }
func TestRun_CommandsWithoutALabInTheRunDir(t *testing.T) { func TestRun_CommandsWithoutALabInTheRunDir(t *testing.T) {
asRoot(t)
run := t.TempDir() run := t.TempDir()
for _, cmd := range []string{"status", "down", "ssh"} { for _, cmd := range []string{"status", "down", "ssh"} {
args := []string{cmd, "-run", run} args := []string{cmd, "-run", run}
@ -166,10 +173,14 @@ func TestRun_CommandsWithoutALabInTheRunDir(t *testing.T) {
func TestRun_StatusOfAStoppedLab(t *testing.T) { func TestRun_StatusOfAStoppedLab(t *testing.T) {
code, stdout, stderr := runLab("status", "-run", savedLab(t)) code, stdout, stderr := runLab("status", "-run", savedLab(t))
want := `node role state pid ssh want := `node role state pid ssh
sw1 switch stopped - 127.0.0.1:2200 r1 router stopped - 127.0.0.1:2200
rr1 rr stopped - 127.0.0.1:2201 r2 router stopped - 127.0.0.1:2201
hv1 host stopped - 127.0.0.1:2202 rr1 rr stopped - 127.0.0.1:2202
hv2 host stopped - 127.0.0.1:2203 hv1 host stopped - 127.0.0.1:2203
hv2 host stopped - 127.0.0.1:2204
segment bridge state gateway
underlay lb-underlay absent 192.168.14.1
` `
if code != 0 || stdout != want { if code != 0 || stdout != want {
t.Errorf("code %d, stderr %q, stdout:\n%s", code, stderr, stdout) t.Errorf("code %d, stderr %q, stdout:\n%s", code, stderr, stdout)
@ -177,6 +188,7 @@ hv2 host stopped - 127.0.0.1:2203
} }
func TestRun_DownOfAStoppedLab(t *testing.T) { func TestRun_DownOfAStoppedLab(t *testing.T) {
asRoot(t)
code, stdout, stderr := runLab("down", "-run", savedLab(t)) code, stdout, stderr := runLab("down", "-run", savedLab(t))
if code != 0 || stdout != "" || stderr != "" { if code != 0 || stdout != "" || stderr != "" {
t.Errorf("code %d, stdout %q, stderr %q", code, stdout, stderr) t.Errorf("code %d, stdout %q, stderr %q", code, stdout, stderr)
@ -208,7 +220,7 @@ func TestRun_SSHExecsSSHWithTheNodePort(t *testing.T) {
"-o", "StrictHostKeyChecking=no", "-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=/dev/null", "-o", "UserKnownHostsFile=/dev/null",
"-o", "LogLevel=ERROR", "-o", "LogLevel=ERROR",
"-p", "2203", "-p", "2204",
"debian@127.0.0.1", "debian@127.0.0.1",
"ip", "-br", "a", "ip", "-br", "a",
} }
@ -225,6 +237,7 @@ func TestRun_SSHUnknownNode(t *testing.T) {
} }
func TestRun_UpRefusesAnInvalidTopologyAndKeepsTheSavedOne(t *testing.T) { func TestRun_UpRefusesAnInvalidTopologyAndKeepsTheSavedOne(t *testing.T) {
asRoot(t)
run := savedLab(t) run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml")) before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
bad := filepath.Join(t.TempDir(), "bad.yml") bad := filepath.Join(t.TempDir(), "bad.yml")
@ -262,6 +275,7 @@ func TestIsTerminal_DevNullAndPipesAreNotTerminals(t *testing.T) {
} }
func TestRun_UpRefusesToReplaceARunningLab(t *testing.T) { func TestRun_UpRefusesToReplaceARunningLab(t *testing.T) {
asRoot(t)
run := savedLab(t) run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml")) before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
proc := t.TempDir() proc := t.TempDir()
@ -313,7 +327,8 @@ func TestRun_RenderShipsTheExampleFRRConfigs(t *testing.T) {
t.Fatalf("code %d, stderr %s", code, stderr) t.Fatalf("code %d, stderr %s", code, stderr)
} }
for node, wants := range map[string][]string{ for node, wants := range map[string][]string{
"sw1": {"router bgp 65100"}, "r1": {"router bgp 65100", "interface underlay\n", " vrrp 1 priority 200\n", " vrrp 1 ip 192.168.14.1\n"},
"r2": {"hostname r2", " vrrp 1 priority 190\n", " vrrp 1 ip 192.168.14.1\n"},
"rr1": {"bgp listen range 192.168.14.0/24 peer-group fabric"}, "rr1": {"bgp listen range 192.168.14.0/24 peer-group fabric"},
"hv1": {"hostname hv1", "neighbor 10.255.255.1 peer-group fabric"}, "hv1": {"hostname hv1", "neighbor 10.255.255.1 peer-group fabric"},
"hv2": {"hostname hv2", "neighbor 10.255.255.1 peer-group fabric"}, "hv2": {"hostname hv2", "neighbor 10.255.255.1 peer-group fabric"},
@ -359,9 +374,9 @@ func TestRun_RenderReportsAMissingFRRConfig(t *testing.T) {
images: images:
deb: { url: https://example.invalid/deb.qcow2, sums: https://example.invalid/SHA512SUMS } deb: { url: https://example.invalid/deb.qcow2, sums: https://example.invalid/SHA512SUMS }
segments: segments:
underlay: { switch: sw1, cidr: 10.1.0.0/24 } underlay: { routers: [sw1], cidr: 10.1.0.0/24 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 1, memory: 512, frr: frr/absent.conf } sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay], frr: frr/absent.conf }
hv1: { role: host, image: deb, cpus: 1, memory: 512, segments: [underlay] } hv1: { role: host, image: deb, cpus: 1, memory: 512, segments: [underlay] }
` `
if err := os.WriteFile(topo, []byte(doc), 0o600); err != nil { if err := os.WriteFile(topo, []byte(doc), 0o600); err != nil {
@ -372,3 +387,34 @@ nodes:
t.Errorf("code %d, stderr %q", code, stderr) t.Errorf("code %d, stderr %q", code, stderr)
} }
} }
func TestRun_UpAndDownRequireRoot(t *testing.T) {
geteuid = func() int { return 1000 }
t.Cleanup(func() { geteuid = os.Geteuid })
run := savedLab(t)
before, _ := os.ReadFile(filepath.Join(run, "topology.yml"))
other := filepath.Join(t.TempDir(), "other.yml")
if err := os.WriteFile(other, bytes.ReplaceAll(before, []byte("evpn-2hv"), []byte("other")), 0o600); err != nil {
t.Fatal(err)
}
for name, args := range map[string][]string{
"up": {"up", "-run", run, other},
"down": {"down", "-run", run},
} {
code, stdout, stderr := runLab(args...)
if code != 1 || stdout != "" || !strings.Contains(stderr, "labctl: "+name+" must run as root") {
t.Errorf("%s: code %d, stdout %q, stderr %q", name, code, stdout, stderr)
}
}
if after, _ := os.ReadFile(filepath.Join(run, "topology.yml")); string(after) != string(before) {
t.Error("the saved topology was replaced without root")
}
}
func TestRun_StatusAndSSHDoNotRequireRoot(t *testing.T) {
geteuid = func() int { return 1000 }
t.Cleanup(func() { geteuid = os.Geteuid })
if code, _, stderr := runLab("status", "-run", savedLab(t)); code != 0 || strings.Contains(stderr, "root") {
t.Errorf("status: code %d, stderr %q", code, stderr)
}
}

View file

@ -141,31 +141,37 @@ celui d'une release, vérifié) au lieu de compiler.
## Topologie ## Topologie
Un lab est décrit par un fichier YAML : des **nœuds** (les VM) et des **segments** (des réseaux L2 Un lab est décrit par un fichier YAML : des **nœuds** (les VM) et des **segments** (des réseaux
portés par un switch). Exemple livré : [`examples/evpn-2hv/topology.yml`](../examples/evpn-2hv/topology.yml). L2). Exemple livré : [`examples/evpn-2hv/topology.yml`](../examples/evpn-2hv/topology.yml) — deux
routeurs en VRRP, un route reflector et deux hôtes en EVPN.
Chaque nœud non-switch est relié au switch de chacun de ses segments par un câble virtuel QEMU ; Chaque segment est un **bridge Linux sur l'hôte du lab** (`lb-<segment>`), et chaque nœud qui y
le switch met ces câbles dans un bridge et porte la passerelle du segment. est attaché y branche une carte par un tap (`lt<i>`). Le L2 ne dépend donc d'aucune VM : la
perte d'un routeur ne coupe pas le trafic direct entre les autres nœuds du segment. Un
**routeur** est un nœud comme les autres, qui porte en plus la passerelle et le NAT des segments
dont il est routeur.
Ce que le fichier déclare : Ce que le fichier déclare :
**`images`** — `url` de l'image qcow2 et `sums` du fichier de sommes à vérifier, tous deux en **`images`** — `url` de l'image qcow2 et `sums` du fichier de sommes à vérifier, tous deux en
`https://`. `https://`.
**`segments`** — `switch` (un nœud de rôle `switch`), `cidr` IPv4 entre `/8` et `/30`, `mtu` **`segments`** — `cidr` IPv4 entre `/8` et `/30`, `mtu` facultatif — 9000 par défaut, entre 1280
facultatif — 9000 par défaut, entre 1280 et 9000. Nom : 12 caractères au plus, minuscules et et 9000 —, et `routers`, facultatif : les nœuds de rôle `router` qui portent la passerelle du
chiffres, parce qu'il devient le nom d'interface dans les VM et, préfixé de `br-`, celui du segment, **dans l'ordre de priorité**. Sans routeur, le segment est un L2 pur (lien entre VM,
bridge (15 caractères au plus sous Linux). lien inter-cluster) : pas de passerelle, pas de NAT. Nom : 12 caractères au plus, minuscules et
chiffres, parce qu'il devient le nom d'interface dans les VM et, préfixé de `lb-`, celui du bridge
sur l'hôte (15 caractères au plus sous Linux).
**`nodes`** — `role` (`switch`, `rr` ou `host`), `image`, `cpus`, `memory` en Mio (256 au **`nodes`** — `role` (`router`, `rr` ou `host`), `image`, `cpus`, `memory` en Mio (256 au
moins), `segments` auxquels le nœud est relié, et `addresses` pour fixer l'adresse d'un nœud sur moins), `segments` auxquels le nœud est relié, et `addresses` pour fixer l'adresse d'un nœud sur
un segment (`addresses: {underlay: 192.168.14.50}`). Un switch ne déclare ni `segments` ni un segment (`addresses: {underlay: 192.168.14.50}`). Un routeur listé dans `segments.<s>.routers`
`addresses` : il porte ceux dont il est le `switch`. Champs facultatifs : doit être attaché à `<s>`. Champs facultatifs :
- `secondary` — des adresses supplémentaires par segment, avec leur longueur de préfixe - `secondary` — des adresses supplémentaires par segment, avec leur longueur de préfixe
(`secondary: {underlay: [169.254.0.3/28]}`), posées sur la même interface que l'adresse (`secondary: {underlay: [169.254.0.3/28]}`), posées sur la même interface que l'adresse
principale : même L2, même MAC. Elles doivent être **hors** du CIDR du segment, pour ne principale : même L2, même MAC. Elles doivent être **hors** du CIDR du segment, pour ne
jamais croiser l'attribution automatique. Sur un switch, elles vont sur le bridge du segment ; jamais croiser l'attribution automatique ;
- `loopback` — une adresse sur une interface `dummy` nommée `lo1` (`loopback: 10.255.255.1/32`) ; - `loopback` — une adresse sur une interface `dummy` nommée `lo1` (`loopback: 10.255.255.1/32`) ;
- `frr` — le chemin d'un `frr.conf`, relatif au fichier de topologie : FRR est installé au - `frr` — le chemin d'un `frr.conf`, relatif au fichier de topologie : FRR est installé au
démarrage et la configuration déposée **telle quelle** (voir [FRR](#frr)) ; démarrage et la configuration déposée **telle quelle** (voir [FRR](#frr)) ;
@ -173,67 +179,112 @@ un segment (`addresses: {underlay: 192.168.14.50}`). Un switch ne déclare ni `s
démarrage **avant** FRR ; démarrage **avant** FRR ;
- `post_provision` — un second script, exécuté **après** FRR ; - `post_provision` — un second script, exécuté **après** FRR ;
- `files` — des fichiers **ou répertoires** déposés dans `/etc/lab/files/<nom>` avant les - `files` — des fichiers **ou répertoires** déposés dans `/etc/lab/files/<nom>` avant les
scripts, sous la forme `{<nom>: <chemin>}` (`files: {motd: motd}`). Ils voyagent scripts, sous la forme `{<nom>: <chemin>}` (`files: {motd: motd}`). Ils voyagent dans un
dans un `files.iso` propre au nœud (volume `LABFILES`), attaché comme second lecteur et `files.iso` propre au nœud (volume `LABFILES`), attaché comme second lecteur et recopié au
recopié au premier démarrage : la taille n'est pas limitée par cloud-init, et un binaire de premier démarrage : la taille n'est pas limitée par cloud-init, et un binaire de plusieurs
plusieurs dizaines de Mio passe ; dizaines de Mio passe ;
- `services` — les unités systemd que `labctl up` vérifie actives sur le nœud. - `services` — les unités systemd que `labctl up` vérifie actives sur le nœud.
Les scripts reçoivent `LAB_NODE` (nom du nœud), `LAB_UPLINK` (interface de son premier segment, Les scripts reçoivent `LAB_NODE` (nom du nœud), `LAB_UPLINK` (interface qui porte la route par
qui porte la route par défaut) et `LAB_FILES` (`/etc/lab/files`). L'outil ne sait rien de ce défaut : celle du premier segment du nœud qui a une passerelle, `mgmt0` pour un routeur) et
qu'ils installent. `LAB_FILES`
(`/etc/lab/files`). L'outil ne sait rien de ce qu'ils installent.
**Un seul script de provisionnement par nœud.** cloud-init exécute `runcmd` comme un script `sh` **Un seul script de provisionnement par nœud.** cloud-init exécute `runcmd` comme un script `sh`
sans `set -e` : seule la dernière commande compterait, et une provision en échec suivie d'un FRR sans `set -e` : seule la dernière commande compterait, et une provision en échec suivie d'un FRR
installé avec succès passerait pour un démarrage réussi. Chaque nœud reçoit donc installé avec succès passerait pour un démarrage réussi. Chaque nœud reçoit donc
`/usr/local/sbin/lab-provision`, en `set -eu`, qui enchaîne ses étapes — script du switch ou de `/usr/local/sbin/lab-provision`, en `set -eu`, qui enchaîne ses étapes — service du routeur ou de
la loopback, `provision`, FRR, `post_provision` ; `runcmd` n'appelle que lui, et la première la loopback, `provision`, FRR, `post_provision` ; `runcmd` n'appelle que lui, et la première
étape en échec met cloud-init en erreur. étape en échec met cloud-init en erreur.
`mgmt0` et `lo1` sont réservés : aucun segment ne peut porter ces noms. `mgmt0` et `lo1` sont réservés : aucun segment ne peut porter ces noms.
Ce que l'outil en déduit, de façon déterministe — même fichier, même plan : ### Passerelle et routeurs
| Routeurs du segment | Passerelle |
|---|---|
| aucun | aucune : les nœuds n'y ont pas de route par défaut |
| un | la première adresse du CIDR, posée sur l'interface du routeur, en plus de la sienne |
| deux ou plus | la première adresse du CIDR, **virtuelle**, portée en VRRP v3 par `vrrpd` de FRR |
Avec plusieurs routeurs, `labctl` prépare tout sur chacun : interface `macvlan` `vr<segment>`
avec la MAC VRRP (`00:00:5e:00:01:<vrid>`) et la passerelle, créée **coupée** (`protodown on`) :
seul `vrrpd` la rétablit, quand il devient maître ; `arp_ignore=1` et `arp_announce=2`
sur l'interface du segment (sans quoi elle répondrait aux ARP de la passerelle avec sa propre
MAC), FRR installé avec `vrrpd`, et la configuration VRRP **ajoutée** à la fin de son `frr.conf`
(ou d'un `frr.conf` minimal s'il n'en déclare pas) :
```text
interface underlay
vrrp 1 version 3
vrrp 1 priority 200
no vrrp 1 preempt
vrrp 1 ip 192.168.14.1
exit
```
Le VRID est le rang du segment plus un (d'où au plus 255 segments à plusieurs routeurs, parmi les
premiers) ; la priorité vaut 200 pour le premier routeur de la liste, puis 10 de moins pour
chacun des suivants. **Pas de préemption** : un routeur qui revient reste en attente, plutôt que
de reprendre la passerelle avant d'avoir ses routes BGP — un prétest l'a montré, la préemption
coupait les sessions qui passaient par lui. Conséquence : le maître est le premier routeur dont
VRRP démarre, la priorité ne départageant que des démarrages simultanés ; un scénario qui arrête
« le maître » doit le demander (`vtysh -c 'show vrrp'`), pas le supposer. Bascule mesurée en
prétest : environ 2,6 s de perte (trois annonces manquées à une seconde d'intervalle).
Chaque routeur masque (NAT nftables) vers `mgmt0` les segments dont il est routeur, et sort par
le NAT de QEMU. Un nœud qui n'est pas routeur prend sa route par défaut et ses DNS (`1.1.1.1`,
`8.8.8.8`) sur le premier de ses segments qui a une passerelle.
### Plan déterministe
Ce que l'outil déduit du fichier — même fichier, même plan :
| | | | | |
|---|---| |---|---|
| Passerelle d'un segment | la première adresse du CIDR, portée par le switch sur `br-<segment>` | | Passerelle d'un segment | la première adresse du CIDR, si le segment a au moins un routeur |
| Adresse d'un nœud | les suivantes, **dans l'ordre de déclaration des nœuds** ; une adresse fixée par `addresses` est réservée d'abord et sautée par l'attribution automatique | | Adresse d'un nœud | les suivantes (dès la première sans routeur), **dans l'ordre de déclaration des nœuds** ; une adresse fixée par `addresses` est réservée d'abord et sautée par l'attribution automatique |
| Câbles | un par couple (segment, nœud), segments puis nœuds dans l'ordre de déclaration ; le câble *i* utilise les ports UDP `20000 + 2i` (côté nœud) et `20001 + 2i` (côté switch) | | Bridge | `lb-<segment>` sur l'hôte du lab |
| MAC | `02:4c:<nœud>:<nœud>:<segment>:<côté>` — préfixe localement administré, rang du nœud sur deux octets, rang du segment, `00` côté nœud et `01` côté switch | | Ports | un par couple (segment, nœud), segments puis nœuds dans l'ordre de déclaration ; le port *i* est le tap `lt<i>` |
| Interfaces | côté nœud, le nom du segment ; côté switch, `p<i>`, du rang du câble | | MAC | `02:4c:<nœud>:<nœud>:<segment>:00` — préfixe localement administré, rang du nœud sur deux octets, rang du segment |
| Interfaces | dans la VM, le nom du segment |
| SSH d'administration | `127.0.0.1:<2200 + rang du nœud>` sur l'hôte du lab | | SSH d'administration | `127.0.0.1:<2200 + rang du nœud>` sur l'hôte du lab |
> [!WARNING] > [!WARNING]
> Réordonner les nœuds ou les segments dans le fichier **change les adresses, les MAC et les > Réordonner les nœuds ou les segments dans le fichier **change les adresses, les MAC, les taps et
> ports**. C'est assumé pour un lab ; `labctl plan` montre le résultat avant tout lancement. > les ports**. C'est assumé pour un lab ; `labctl plan` montre le résultat avant tout lancement.
Limites : 1000 nœuds, 256 segments, et autant de câbles que la plage UDP le permet (22 768). Limites : 1000 nœuds, 256 segments.
`labctl plan` valide le fichier et affiche le plan, sans rien lancer : `labctl plan` valide le fichier et affiche le plan, sans rien lancer :
```text ```text
$ go run ./cmd/labctl plan examples/evpn-2hv/topology.yml $ go run ./cmd/labctl plan examples/evpn-2hv/topology.yml
lab evpn-2hv: nodes 4, segments 1, cables 3 lab evpn-2hv: nodes 5, segments 1, ports 5
nodes nodes
name role image cpus memory ssh name role image cpus memory ssh
sw1 switch debian12 2 1024 MiB 127.0.0.1:2200 r1 router debian12 1 1024 MiB 127.0.0.1:2200
rr1 rr debian12 1 1024 MiB 127.0.0.1:2201 r2 router debian12 1 1024 MiB 127.0.0.1:2201
hv1 host debian12 2 2048 MiB 127.0.0.1:2202 rr1 rr debian12 1 1024 MiB 127.0.0.1:2202
hv2 host debian12 2 2048 MiB 127.0.0.1:2203 hv1 host debian12 2 2048 MiB 127.0.0.1:2203
hv2 host debian12 2 2048 MiB 127.0.0.1:2204
roles roles
name loopback secondary frr provision post_provision files services name loopback secondary frr provision post_provision files services
sw1 - underlay 169.254.0.1/28 sw1.conf - - - - r1 - underlay 169.254.0.1/28 r1.conf - - - -
r2 - underlay 169.254.0.2/28 r2.conf - - - -
rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf - - - - rr1 lo1 10.255.255.1/32 underlay 169.254.0.3/28 rr1.conf - - - -
hv1 - - hv1.conf host.sh - motd - hv1 - - hv1.conf host.sh - motd -
hv2 - - hv2.conf host.sh - motd - hv2 - - hv2.conf host.sh - motd -
segment underlay: 192.168.14.0/24, mtu 9000, switch sw1, bridge br-underlay, gateway 192.168.14.1 segment underlay: 192.168.14.0/24, mtu 9000, bridge lb-underlay, gateway 192.168.14.1 by vrrp 1 on r1, r2
node interface address mac udp switch port mac udp node interface address mac tap
rr1 underlay 192.168.14.2/24 02:4c:00:01:00:00 20000 <-> sw1 p0 02:4c:00:01:00:01 20001 r1 underlay 192.168.14.252/24 02:4c:00:00:00:00 lt0
hv1 underlay 192.168.14.11/24 02:4c:00:02:00:00 20002 <-> sw1 p1 02:4c:00:02:00:01 20003 r2 underlay 192.168.14.253/24 02:4c:00:01:00:00 lt1
hv2 underlay 192.168.14.12/24 02:4c:00:03:00:00 20004 <-> sw1 p2 02:4c:00:03:00:01 20005 rr1 underlay 192.168.14.2/24 02:4c:00:02:00:00 lt2
hv1 underlay 192.168.14.11/24 02:4c:00:03:00:00 lt3
hv2 underlay 192.168.14.12/24 02:4c:00:04:00:00 lt4
``` ```
Un fichier invalide est refusé avec **toutes** ses erreurs à la fois, et un code de sortie 1. Les Un fichier invalide est refusé avec **toutes** ses erreurs à la fois, et un code de sortie 1. Les
@ -242,17 +293,31 @@ champs inconnus et les clés en double sont refusés aussi :
```text ```text
$ labctl plan cassee.yml $ labctl plan cassee.yml
labctl: cassee.yml: labctl: cassee.yml:
segment underlay: rr1 is a rr, not a switch segment underlay: rr1 is a rr, not a router
segment underlay: router r9 is not a declared node
segment underlay: cidr 10.250.0.0/31 prefix length out of range [/8, /30] segment underlay: cidr 10.250.0.0/31 prefix length out of range [/8, /30]
node sw1: switch carries no segment node r1: must be attached to at least one segment
``` ```
### Migration depuis 0.1
Le rôle `switch` et le champ `segments.<s>.switch` n'existent plus : `labctl` les refuse avec un
message qui dit quoi faire. Pour une topologie 0.1 :
1. `role: switch` devient `role: router` ;
2. le routeur déclare `segments: [<s>]` et, de préférence, une adresse fixée (`addresses`) — en
0.1, le switch n'avait que la passerelle ;
3. `segments.<s>.switch: <nom>` devient `segments.<s>.routers: [<nom>]` ;
4. ses `secondary` restent valables : elles vont désormais sur son interface du segment, et non
plus sur un bridge interne.
## FRR ## FRR
Au premier démarrage, cloud-init installe FRR (`frr-stable` de `deb.frrouting.org`, sans les Au premier démarrage, cloud-init installe FRR (`frr-stable` de `deb.frrouting.org`, sans les
paquets recommandés), active `bgpd` — et `bfdd` sur le switch et le route reflector —, puis paquets recommandés), active `bgpd` — et `bfdd` sur les routeurs et les route reflectors,
`vrrpd` sur les routeurs d'un segment en VRRP —, puis
dépose le `frr.conf` du nœud et redémarre FRR. La mise à jour des index de paquets est réessayée dépose le `frr.conf` du nœud et redémarre FRR. La mise à jour des index de paquets est réessayée
pendant cinq minutes : un nœud peut démarrer avant que le switch, par lequel il sort, n'ait posé pendant cinq minutes : un nœud peut démarrer avant que le routeur, par lequel il sort, n'ait posé
son NAT. son NAT.
La **clé du dépôt FRR** n'est pas téléchargée au démarrage : elle est enregistrée dans `labctl` La **clé du dépôt FRR** n'est pas téléchargée au démarrage : elle est enregistrée dans `labctl`
@ -275,7 +340,7 @@ voit pas — une provision qui arme un redémarrage, par exemple : la VM redéma
`runcmd`. `runcmd`.
> [!NOTE] > [!NOTE]
> **Le switch du lab est un routeur Linux avec FRR, par choix.** Il joue le rôle générique de > **Le routeur du lab est un Linux avec FRR, par choix.** Il joue le rôle générique de
> routeur de cluster : passerelle des nœuds et, dans l'exemple, session eBGP avec BFD vers le > routeur de cluster : passerelle des nœuds et, dans l'exemple, session eBGP avec BFD vers le
> route reflector, dont il n'accepte que la loopback. L'équipement réel d'une infrastructure > route reflector, dont il n'accepte que la loopback. L'équipement réel d'une infrastructure
> (MikroTik, Cisco, Juniper, Arista…) n'a besoin que de BGP et d'EVPN ; sa configuration propre > (MikroTik, Cisco, Juniper, Arista…) n'a besoin que de BGP et d'EVPN ; sa configuration propre
@ -300,17 +365,17 @@ Les chemins de la VM (`disk.qcow2`, `seed.iso`, `console.log`, `qmp.sock`, `qemu
du répertoire du nœud ; `-key` peut être répété, et accepte un fichier `authorized_keys` (lignes du répertoire du nœud ; `-key` peut être répété, et accepte un fichier `authorized_keys` (lignes
vides et commentaires ignorés). Les fichiers sont créés en `0600`. vides et commentaires ignorés). Les fichiers sont créés en `0600`.
Ce que contiennent les arguments QEMU d'un nœud — extrait réel, côté réseau : Ce que contiennent les arguments QEMU d'un nœud — côté réseau :
```text ```text
-netdev -netdev
user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22 user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2203-:22
-device -device
virtio-net-pci,netdev=mgmt0,mac=02:4d:00:02:00:00,romfile= virtio-net-pci,netdev=mgmt0,mac=02:4d:00:03:00:00,romfile=
-netdev -netdev
dgram,id=underlay,local.type=inet,local.host=127.0.0.1,local.port=20002,remote.type=inet,remote.host=127.0.0.1,remote.port=20003 tap,id=underlay,ifname=lt3,script=no,downscript=no
-device -device
virtio-net-pci,netdev=underlay,mac=02:4c:00:02:00:00,host_mtu=9000,romfile= virtio-net-pci,netdev=underlay,mac=02:4c:00:03:00:00,host_mtu=9000,romfile=
``` ```
Les choix qui s'y lisent : Les choix qui s'y lisent :
@ -318,11 +383,11 @@ Les choix qui s'y lisent :
- **machine** `q35`, `-accel kvm -cpu host` — le KVM imbriqué des nœuds en dépend ; - **machine** `q35`, `-accel kvm -cpu host` — le KVM imbriqué des nœuds en dépend ;
`-nodefaults` pour qu'aucun périphérique implicite ne s'ajoute ; `-nodefaults` pour qu'aucun périphérique implicite ne s'ajoute ;
- **administration** (`mgmt0`) : le NAT de QEMU, MAC `02:4d:<nœud>:<nœud>:00:00`, SSH redirigé - **administration** (`mgmt0`) : le NAT de QEMU, MAC `02:4d:<nœud>:<nœud>:00:00`, SSH redirigé
sur la boucle locale de l'hôte. `restrict=on` pour tous les nœuds **sauf le switch** : un sur la boucle locale de l'hôte. `restrict=on` pour tous les nœuds **sauf les routeurs** : un
nœud isolé ne joint ni l'hôte ni l'extérieur par là, seule la redirection SSH passe. nœud isolé ne joint ni l'hôte ni l'extérieur par là, seule la redirection SSH passe.
`ipv6=off` partout (voir plus bas) ; `ipv6=off` partout (voir plus bas) ;
- **câbles** : `dgram` sur `127.0.0.1`, les deux extrémités d'un câble se répondent (port local - **segments** : un tap créé par `labctl up` (`script=no` : QEMU ne lance aucun script), branché
de l'une = port distant de l'autre), `host_mtu` annonce le MTU du segment au guest ; sur le bridge du segment ; `host_mtu` annonce le MTU du segment au guest ;
- `romfile=` vide sur toutes les cartes : pas de ROM de démarrage réseau, donc pas de repli sur - `romfile=` vide sur toutes les cartes : pas de ROM de démarrage réseau, donc pas de repli sur
un démarrage PXE si le firmware ne trouve pas le disque. Pendant les premiers essais, une VM un démarrage PXE si le firmware ne trouve pas le disque. Pendant les premiers essais, une VM
restée bloquée sans rien écrire sur sa console, CPU au repos, avait toutes les apparences de restée bloquée sans rien écrire sur sa console, CPU au repos, avait toutes les apparences de
@ -330,18 +395,22 @@ Les choix qui s'y lisent :
Ce que fait cloud-init : Ce que fait cloud-init :
- **toutes les VM** : interfaces nommées d'après leur MAC (`mgmt0`, nom du segment, `p<i>`), - **toutes les VM** : interfaces nommées d'après leur MAC (`mgmt0`, nom du segment),
`dhcp4: false` partout, `mgmt0` en `10.0.2.15/24` **sans passerelle** ; connexion SSH par `dhcp4: false` partout, `mgmt0` en `10.0.2.15/24` ; connexion SSH par clé seulement,
clé seulement, utilisateur `debian`, `root` désactivé, mot de passe refusé ; utilisateur `debian`, `root` désactivé, mot de passe refusé ;
- **un nœud** : adresse sur chaque segment, MTU du segment, route par défaut et DNS - **un nœud** : adresse et `secondary` sur chaque segment, MTU du segment, route par défaut et
(`1.1.1.1`, `8.8.8.8`) sur son **premier** segment — la sortie Internet passe par le switch ; DNS sur le premier de ses segments qui a une passerelle ; `mgmt0` **sans passerelle** ;
- **le switch** : route par défaut par `mgmt0` ; un service `lab-switch` crée un bridge - **un routeur** : mêmes adresses, route par défaut par `mgmt0` ; un service `lab-router` active
`br-<segment>` par segment (STP désactivé, MTU du segment), y branche ses ports, porte la le routage, pose les passerelles (adresse ou `macvlan` VRRP, voir
passerelle, active le routage et masque (NAT nftables) les segments vers `mgmt0`. Le service [Passerelle et routeurs](#passerelle-et-routeurs)) et le NAT nftables. Le service est rejoué à
est rejoué à chaque démarrage. chaque démarrage.
### Vérifié sur de vraies VM ### Vérifié sur de vraies VM
Relevés faits avec le modèle 0.1 — le switch portait alors le bridge du segment, relié aux nœuds
par des câbles UDP ; ce qui touche au L2 et à la passerelle est à revérifier avec les bridges de
l'hôte.
Le switch et le route reflector n'ont pas besoin de KVM imbriqué : `sw1` et `rr1` ont été Le switch et le route reflector n'ont pas besoin de KVM imbriqué : `sw1` et `rr1` ont été
démarrés **sur un Mac**, en émulation (TCG), avec Debian 12 `generic` et les fichiers produits démarrés **sur un Mac**, en émulation (TCG), avec Debian 12 `generic` et les fichiers produits
par `labctl render`. par `labctl render`.
@ -387,19 +456,45 @@ labctl ssh [-run dir] <nœud> [commande…]
3. génère une paire de clés SSH dans le répertoire du lab, si elle n'existe pas encore ; 3. génère une paire de clés SSH dans le répertoire du lab, si elle n'existe pas encore ;
4. pour chaque nœud : fichiers de `labctl render`, disque **neuf** en overlay qcow2 sur l'image 4. pour chaque nœud : fichiers de `labctl render`, disque **neuf** en overlay qcow2 sur l'image
(`qemu-img create -b`, 20 Gio annoncés), image `cidata` et `files.iso` (`genisoimage`) ; (`qemu-img create -b`, 20 Gio annoncés), image `cidata` et `files.iso` (`genisoimage`) ;
5. démarre les QEMU, **switchs d'abord**, détachés (`-daemonize`) : ils survivent à la session 5. crée les bridges des segments et les taps des nœuds (voir [Réseau de l'hôte](#réseau-de-lhôte)),
et refuse de continuer si l'un d'eux existe déjà ;
6. démarre les QEMU, **routeurs d'abord**, détachés (`-daemonize`) : ils survivent à la session
SSH qui les a lancés ; SSH qui les a lancés ;
6. attend sur chaque nœud la fin de cloud-init (`cloud-init status --wait` par SSH), jusqu'au 7. attend sur chaque nœud la fin de cloud-init (`cloud-init status --wait` par SSH), jusqu'au
délai `-timeout`, puis vérifie ses services. délai `-timeout`, puis vérifie ses services (`frr` compris quand il est installé).
La topologie est copiée dans `<run>/topology.yml`. Un échec laisse les nœuds démarrés en place : La topologie est copiée dans `<run>/topology.yml`. Un échec laisse les nœuds démarrés en place :
`labctl status`, puis `labctl down`. `labctl status`, puis `labctl down`.
**`labctl status`** — pour chaque nœud : rôle, état du processus QEMU, PID, port SSH sur la **`labctl status`** — pour chaque nœud : rôle, état du processus QEMU, PID, port SSH sur la
boucle locale. boucle locale ; pour chaque segment : bridge présent ou absent, passerelle.
**`labctl down`** — arrête chaque QEMU par `SIGTERM`, puis `SIGKILL` au bout de 30 s. Les disques **`labctl down`** — arrête chaque QEMU par `SIGTERM`, puis `SIGKILL` au bout de 30 s, puis
sont conservés jusqu'au prochain `up`, qui les recrée. supprime les taps et les bridges du lab. Les disques sont conservés jusqu'au prochain `up`, qui
les recrée.
`up` et `down` doivent tourner **en root** : ils créent et suppriment des interfaces de l'hôte.
### Réseau de l'hôte
Pour chaque segment, `labctl up` exécute :
```text
ip link add name lb-<segment> type bridge stp_state 0 mcast_snooping 0
ip link set dev lb-<segment> addrgenmode none
ip tuntap add dev lt<i> mode tap # pour chaque port du segment
ip link set dev lt<i> addrgenmode none
ip link set dev lt<i> mtu <mtu> master lb-<segment> up
ip link set dev lb-<segment> mtu <mtu> up
```
- **Aucune adresse côté hôte**, IPv4 ni IPv6 (`addrgenmode none` : pas même de lien local) :
l'hôte n'est pas joignable depuis un segment, et ne route rien entre eux.
- **Pas de STP ni de surveillance IGMP** : le bridge se comporte comme un switch simple, et les
annonces VRRP (`224.0.0.18`) circulent sans dépendre d'un querier.
- Si `br_netfilter` est chargé sur l'hôte, `bridge-nf-call-iptables` est mis à 0 : le pare-feu
de l'hôte ne filtre pas le trafic des segments.
- Un lab par hôte : les noms `lb-*` et `lt*` ne sont pas propres à un répertoire `-run`.
**`labctl ssh`** — ouvre un shell sur un nœud, ou y exécute une commande, avec la clé générée par **`labctl ssh`** — ouvre un shell sur un nœud, ou y exécute une commande, avec la clé générée par
`up`. `labctl` cède la place à `ssh`, dont le code de retour est donc celui de la commande. Un `up`. `labctl` cède la place à `ssh`, dont le code de retour est donc celui de la commande. Un
@ -433,7 +528,7 @@ PID réutilisé par un autre programme n'est donc jamais signalé.
### Vérifié sur le serveur de lab ### Vérifié sur le serveur de lab
Le 2026-10-04, sur un Xeon E5-2640 v3, Debian 12 et QEMU 7.2 sur le serveur, topologie à deux Avec le modèle 0.1 (voir la remarque précédente). Le 2026-10-04, sur un Xeon E5-2640 v3, Debian 12 et QEMU 7.2 sur le serveur, topologie à deux
hôtes : hôtes :
| Vérification | Résultat | | Vérification | Résultat |
@ -519,6 +614,8 @@ au projet.
- **Clé SSH des VM** : générée par `labctl up` sur le serveur, c'est la seule clé autorisée dans - **Clé SSH des VM** : générée par `labctl up` sur le serveur, c'est la seule clé autorisée dans
les VM. Elle ne quitte jamais le serveur, n'ouvre que les VM du lab — qui n'écoutent qu'en les VM. Elle ne quitte jamais le serveur, n'ouvre que les VM du lab — qui n'écoutent qu'en
boucle locale — et disparaît avec lui. La clé publique du poste n'est jamais envoyée aux VM. boucle locale — et disparaît avec lui. La clé publique du poste n'est jamais envoyée aux VM.
- **Segments isolés de l'hôte** : bridges et taps sans adresse (voir
[Réseau de l'hôte](#réseau-de-lhôte)) ; seuls les routeurs sortent, par le NAT de QEMU.
- **Clés d'hôte des VM non vérifiées** par `labctl ssh` (`known_hosts` jetable) : elles changent - **Clés d'hôte des VM non vérifiées** par `labctl ssh` (`known_hosts` jetable) : elles changent
à chaque `up`. Acceptable uniquement parce que la connexion reste sur la boucle locale d'un à chaque `up`. Acceptable uniquement parce que la connexion reste sur la boucle locale d'un
serveur auquel on s'est authentifié. serveur auquel on s'est authentifié.

View file

@ -1,5 +1,5 @@
frr defaults traditional frr defaults traditional
hostname sw1 hostname r1
log syslog informational log syslog informational
! !
ip prefix-list RR-LOOPBACK-IN seq 10 permit 10.255.255.1/32 ip prefix-list RR-LOOPBACK-IN seq 10 permit 10.255.255.1/32

View file

@ -0,0 +1,28 @@
frr defaults traditional
hostname r2
log syslog informational
!
ip prefix-list RR-LOOPBACK-IN seq 10 permit 10.255.255.1/32
!
route-map RR-IN permit 10
match ip address prefix-list RR-LOOPBACK-IN
exit
!
route-map NO-OUT deny 999
exit
!
router bgp 65100
no bgp default ipv4-unicast
bgp router-id 169.254.0.2
neighbor 169.254.0.3 remote-as 65000
neighbor 169.254.0.3 description rr1
neighbor 169.254.0.3 bfd
!
address-family ipv4 unicast
neighbor 169.254.0.3 activate
neighbor 169.254.0.3 route-map RR-IN in
neighbor 169.254.0.3 route-map NO-OUT out
exit-address-family
!
exit
!

View file

@ -7,14 +7,19 @@ images:
segments: segments:
underlay: underlay:
switch: sw1 routers: [r1, r2]
cidr: 192.168.14.0/24 cidr: 192.168.14.0/24
mtu: 9000 mtu: 9000
nodes: nodes:
sw1: r1:
{ role: switch, image: debian12, cpus: 2, memory: 1024, { role: router, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf } addresses: { underlay: 192.168.14.252 }, secondary: { underlay: [169.254.0.1/28] },
frr: frr/r1.conf }
r2:
{ role: router, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
addresses: { underlay: 192.168.14.253 }, secondary: { underlay: [169.254.0.2/28] },
frr: frr/r2.conf }
rr1: rr1:
{ role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay], { role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay],
addresses: { underlay: 192.168.14.2 }, secondary: { underlay: [169.254.0.3/28] }, addresses: { underlay: 192.168.14.2 }, secondary: { underlay: [169.254.0.3/28] },

View file

@ -107,7 +107,10 @@ func (l Lab) Up(ctx context.Context, fetcher provision.Fetcher, timeout time.Dur
if err != nil { if err != nil {
return err return err
} }
for _, n := range switchesFirst(l.Plan, nodes) { if err := l.NetworkUp(ctx); err != nil {
return err
}
for _, n := range routersFirst(l.Plan, nodes) {
if err := os.Remove(l.pidFile(n.Name)); err != nil && !errors.Is(err, os.ErrNotExist) { if err := os.Remove(l.pidFile(n.Name)); err != nil && !errors.Is(err, os.ErrNotExist) {
return err return err
} }
@ -134,10 +137,10 @@ func (l Lab) Up(ctx context.Context, fetcher provision.Fetcher, timeout time.Dur
return errors.Join(errs...) return errors.Join(errs...)
} }
func switchesFirst(p *topology.Plan, nodes []render.Node) []render.Node { func routersFirst(p *topology.Plan, nodes []render.Node) []render.Node {
var first, rest []render.Node var first, rest []render.Node
for i, n := range nodes { for i, n := range nodes {
if p.Nodes[i].Role == topology.RoleSwitch { if p.Nodes[i].Role == topology.RoleRouter {
first = append(first, n) first = append(first, n)
} else { } else {
rest = append(rest, n) rest = append(rest, n)
@ -167,16 +170,16 @@ func (l Lab) waitReady(ctx context.Context, n topology.NodePlan) error {
} }
} }
func services(n topology.NodePlan) []string { func services(p *topology.Plan, n topology.NodePlan) []string {
units := append([]string(nil), n.Services...) units := append([]string(nil), n.Services...)
if n.FRR != "" { if render.NeedsFRR(p, n) {
units = append(units, "frr.service") units = append(units, "frr.service")
} }
return units return units
} }
func (l Lab) checkServices(ctx context.Context, n topology.NodePlan) error { func (l Lab) checkServices(ctx context.Context, n topology.NodePlan) error {
for _, unit := range services(n) { for _, unit := range services(l.Plan, n) {
if err := l.Runner.Run(ctx, "ssh", l.sshArgs(n, true, []string{"systemctl", "is-active", "--quiet", unit})...); err != nil { if err := l.Runner.Run(ctx, "ssh", l.sshArgs(n, true, []string{"systemctl", "is-active", "--quiet", unit})...); err != nil {
return fmt.Errorf("%s is not active: %w", unit, err) return fmt.Errorf("%s is not active: %w", unit, err)
} }
@ -235,7 +238,10 @@ func (l Lab) Status(w io.Writer) error {
} }
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t127.0.0.1:%d\n", n.Name, n.Role, state, shown, n.SSHPort) fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t127.0.0.1:%d\n", n.Name, n.Role, state, shown, n.SSHPort)
} }
return tw.Flush() if err := tw.Flush(); err != nil {
return err
}
return l.networkStatus(w)
} }
func (l Lab) Down(ctx context.Context) error { func (l Lab) Down(ctx context.Context) error {
@ -259,7 +265,10 @@ func (l Lab) Down(ctx context.Context) error {
} }
fmt.Fprintf(l.Out, "%s: stopped\n", n.Name) fmt.Fprintf(l.Out, "%s: stopped\n", n.Name)
} }
return errors.Join(errs...) if len(errs) > 0 {
return errors.Join(errs...)
}
return l.NetworkDown(ctx)
} }
func (l Lab) stop(ctx context.Context, name string, pid int) error { func (l Lab) stop(ctx context.Context, name string, pid int) error {

View file

@ -38,6 +38,7 @@ type fakeRunner struct {
ssh map[string][]error ssh map[string][]error
always map[string]error always map[string]error
inactive string inactive string
links map[string]bool
} }
func (f *fakeRunner) Run(_ context.Context, name string, args ...string) error { func (f *fakeRunner) Run(_ context.Context, name string, args ...string) error {
@ -54,6 +55,22 @@ func (f *fakeRunner) Run(_ context.Context, name string, args ...string) error {
return err return err
} }
return os.WriteFile(private+".pub", []byte(labKey+"\n"), 0o644) return os.WriteFile(private+".pub", []byte(labKey+"\n"), 0o644)
case "ip":
if f.links == nil {
f.links = map[string]bool{}
}
switch {
case len(args) == 4 && args[0] == "link" && args[1] == "show":
if !f.links[args[3]] {
return exitErr(1)
}
case args[0] == "link" && args[1] == "add" && args[2] == "name":
f.links[args[3]] = true
case args[0] == "tuntap" && args[1] == "add":
f.links[args[3]] = true
case args[0] == "link" && args[1] == "delete":
delete(f.links, args[3])
}
case "ssh": case "ssh":
port := args[indexOf(args, "-p")+1] port := args[indexOf(args, "-p")+1]
if args[len(args)-1] == f.inactive { if args[len(args)-1] == f.inactive {
@ -81,6 +98,25 @@ func (f *fakeRunner) commands(name string) [][]string {
return out return out
} }
func (f *fakeRunner) ipCalls() []string {
var out []string
for _, c := range f.commands("ip") {
out = append(out, strings.Join(c[1:], " "))
}
return out
}
func (f *fakeRunner) present(links ...string) {
f.mu.Lock()
defer f.mu.Unlock()
if f.links == nil {
f.links = map[string]bool{}
}
for _, l := range links {
f.links[l] = true
}
}
func indexOf(list []string, value string) int { func indexOf(list []string, value string) int {
for i, v := range list { for i, v := range list {
if v == value { if v == value {
@ -105,7 +141,10 @@ func mirror(t *testing.T) (*httptest.Server, string) {
return server, server.URL return server, server.URL
} }
func labPlan(t *testing.T, base, nodes string) *topology.Plan { const underlay = ` underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
`
func labPlan(t *testing.T, base, segments, nodes string) *topology.Plan {
t.Helper() t.Helper()
doc := `name: evpn-2hv doc := `name: evpn-2hv
images: images:
@ -113,8 +152,7 @@ images:
url: ` + base + `/deb.qcow2 url: ` + base + `/deb.qcow2
sums: ` + base + `/SHA512SUMS sums: ` + base + `/SHA512SUMS
segments: segments:
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 } ` + segments + `nodes:
nodes:
` + nodes ` + nodes
topo, err := topology.Parse([]byte(doc)) topo, err := topology.Parse([]byte(doc))
if err != nil { if err != nil {
@ -127,9 +165,18 @@ nodes:
return p return p
} }
const switchLast = ` rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] } const routerLast = ` rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], services: [agent.service] } hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], services: [agent.service] }
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 } sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
`
const twoSegments = ` red: { routers: [r1, r2], cidr: 10.1.0.0/24 }
blue: { cidr: 10.2.0.0/24, mtu: 1500 }
`
const twoRouters = ` r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
` `
type fixture struct { type fixture struct {
@ -140,6 +187,11 @@ type fixture struct {
} }
func newFixture(t *testing.T, nodes string) *fixture { func newFixture(t *testing.T, nodes string) *fixture {
t.Helper()
return newLab(t, underlay, nodes)
}
func newLab(t *testing.T, segments, nodes string) *fixture {
t.Helper() t.Helper()
server, base := mirror(t) server, base := mirror(t)
root := t.TempDir() root := t.TempDir()
@ -147,7 +199,7 @@ func newFixture(t *testing.T, nodes string) *fixture {
out := &bytes.Buffer{} out := &bytes.Buffer{}
return &fixture{ return &fixture{
lab: Lab{ lab: Lab{
Plan: labPlan(t, base, nodes), Plan: labPlan(t, base, segments, nodes),
RunDir: filepath.Join(root, "run"), RunDir: filepath.Join(root, "run"),
ProcDir: filepath.Join(root, "proc"), ProcDir: filepath.Join(root, "proc"),
Runner: runner, Runner: runner,
@ -217,7 +269,7 @@ func waitDead(t *testing.T, pid int) {
} }
func TestPID_NoPidFileMeansStopped(t *testing.T) { func TestPID_NoPidFileMeansStopped(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil { if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil {
t.Errorf("PID = %d, %v", pid, err) t.Errorf("PID = %d, %v", pid, err)
} }
@ -225,7 +277,7 @@ func TestPID_NoPidFileMeansStopped(t *testing.T) {
func TestPID_RefusesACorruptedPidFile(t *testing.T) { func TestPID_RefusesACorruptedPidFile(t *testing.T) {
for _, content := range []string{"", "abc", "0", "-12"} { for _, content := range []string{"", "abc", "0", "-12"} {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.pidFile(t, "hv1", content) f.pidFile(t, "hv1", content)
if _, err := f.lab.PID("hv1"); err == nil || !strings.Contains(err.Error(), "not a pid") { if _, err := f.lab.PID("hv1"); err == nil || !strings.Contains(err.Error(), "not a pid") {
t.Errorf("pid file %q: error = %v", content, err) t.Errorf("pid file %q: error = %v", content, err)
@ -234,7 +286,7 @@ func TestPID_RefusesACorruptedPidFile(t *testing.T) {
} }
func TestPID_RecognisesTheNodeProcess(t *testing.T) { func TestPID_RecognisesTheNodeProcess(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
want := f.process(t, "hv1", "sleep 30", qemu("hv1")...) want := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if pid, err := f.lab.PID("hv1"); pid != want || err != nil { if pid, err := f.lab.PID("hv1"); pid != want || err != nil {
t.Errorf("PID = %d, %v, want %d", pid, err, want) t.Errorf("PID = %d, %v, want %d", pid, err, want)
@ -249,7 +301,7 @@ func TestPID_IgnoresAPidReusedByAnotherProcess(t *testing.T) {
} }
for name, cmdline := range cases { for name, cmdline := range cases {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.process(t, "hv1", "sleep 30", cmdline...) f.process(t, "hv1", "sleep 30", cmdline...)
if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil { if pid, err := f.lab.PID("hv1"); pid != 0 || err != nil {
t.Errorf("PID = %d, %v, want 0", pid, err) t.Errorf("PID = %d, %v, want 0", pid, err)
@ -259,7 +311,7 @@ func TestPID_IgnoresAPidReusedByAnotherProcess(t *testing.T) {
} }
func TestPID_IgnoresADeadProcess(t *testing.T) { func TestPID_IgnoresADeadProcess(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
pid := f.process(t, "hv1", "exit 0", qemu("hv1")...) pid := f.process(t, "hv1", "exit 0", qemu("hv1")...)
waitDead(t, pid) waitDead(t, pid)
if got, err := f.lab.PID("hv1"); got != 0 || err != nil { if got, err := f.lab.PID("hv1"); got != 0 || err != nil {
@ -268,7 +320,7 @@ func TestPID_IgnoresADeadProcess(t *testing.T) {
} }
func TestStatus(t *testing.T) { func TestStatus(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
pid := strconv.Itoa(f.process(t, "hv1", "sleep 30", qemu("hv1")...)) pid := strconv.Itoa(f.process(t, "hv1", "sleep 30", qemu("hv1")...))
var out bytes.Buffer var out bytes.Buffer
if err := f.lab.Status(&out); err != nil { if err := f.lab.Status(&out); err != nil {
@ -281,14 +333,16 @@ func TestStatus(t *testing.T) {
want := fmt.Sprintf("node role state %-*s ssh\n", w, "pid") + want := fmt.Sprintf("node role state %-*s ssh\n", w, "pid") +
fmt.Sprintf("rr1 rr stopped %-*s 127.0.0.1:2200\n", w, "-") + fmt.Sprintf("rr1 rr stopped %-*s 127.0.0.1:2200\n", w, "-") +
fmt.Sprintf("hv1 host running %-*s 127.0.0.1:2201\n", w, pid) + fmt.Sprintf("hv1 host running %-*s 127.0.0.1:2201\n", w, pid) +
fmt.Sprintf("sw1 switch stopped %-*s 127.0.0.1:2202\n", w, "-") fmt.Sprintf("sw1 router stopped %-*s 127.0.0.1:2202\n", w, "-") +
"\nsegment bridge state gateway\n" +
"underlay lb-underlay absent 10.250.0.1\n"
if out.String() != want { if out.String() != want {
t.Errorf("status:\n%s\nwant:\n%s", out.String(), want) t.Errorf("status:\n%s\nwant:\n%s", out.String(), want)
} }
} }
func TestDown_StopsWithSIGTERM(t *testing.T) { func TestDown_StopsWithSIGTERM(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...) pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if err := f.lab.Down(context.Background()); err != nil { if err := f.lab.Down(context.Background()); err != nil {
@ -305,7 +359,7 @@ func TestDown_StopsWithSIGTERM(t *testing.T) {
} }
func TestDown_FallsBackToSIGKILL(t *testing.T) { func TestDown_FallsBackToSIGKILL(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.lab.Stop = 100 * time.Millisecond f.lab.Stop = 100 * time.Millisecond
pid := f.process(t, "hv1", `trap "" TERM; while :; do sleep 0.05; done`, qemu("hv1")...) pid := f.process(t, "hv1", `trap "" TERM; while :; do sleep 0.05; done`, qemu("hv1")...)
time.Sleep(50 * time.Millisecond) time.Sleep(50 * time.Millisecond)
@ -322,7 +376,7 @@ func TestDown_FallsBackToSIGKILL(t *testing.T) {
} }
func TestDown_NeverSignalsAProcessThatIsNotTheNode(t *testing.T) { func TestDown_NeverSignalsAProcessThatIsNotTheNode(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
pid := f.process(t, "hv1", "sleep 30", "/usr/sbin/sshd", "-D") pid := f.process(t, "hv1", "sleep 30", "/usr/sbin/sshd", "-D")
if err := f.lab.Down(context.Background()); err != nil { if err := f.lab.Down(context.Background()); err != nil {
@ -339,7 +393,7 @@ func TestDown_NeverSignalsAProcessThatIsNotTheNode(t *testing.T) {
} }
func TestDown_ReportsACorruptedPidFileAndStopsTheOthers(t *testing.T) { func TestDown_ReportsACorruptedPidFileAndStopsTheOthers(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.pidFile(t, "rr1", "garbage") f.pidFile(t, "rr1", "garbage")
pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...) pid := f.process(t, "hv1", "sleep 30", qemu("hv1")...)
@ -352,7 +406,7 @@ func TestDown_ReportsACorruptedPidFileAndStopsTheOthers(t *testing.T) {
} }
func TestSSH_Arguments(t *testing.T) { func TestSSH_Arguments(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
key := filepath.Join(f.lab.RunDir, "lab_ed25519") key := filepath.Join(f.lab.RunDir, "lab_ed25519")
base := []string{ base := []string{
"-i", key, "-i", key,
@ -388,14 +442,14 @@ func TestSSH_Arguments(t *testing.T) {
} }
func TestSSH_UnknownNode(t *testing.T) { func TestSSH_UnknownNode(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
if _, err := f.lab.SSH("hv9", true, nil); err == nil || !strings.Contains(err.Error(), `node "hv9" is not in lab evpn-2hv`) { if _, err := f.lab.SSH("hv9", true, nil); err == nil || !strings.Contains(err.Error(), `node "hv9" is not in lab evpn-2hv`) {
t.Errorf("error = %v", err) t.Errorf("error = %v", err)
} }
} }
func TestUp_StartsSwitchesFirstAndWaitsForEveryNode(t *testing.T) { func TestUp_StartsRoutersFirstAndWaitsForEveryNode(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil { if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err) t.Fatalf("Up: %v", err)
@ -409,7 +463,7 @@ func TestUp_StartsSwitchesFirstAndWaitsForEveryNode(t *testing.T) {
started = append(started, c[indexOf(c, "-name")+1]) started = append(started, c[indexOf(c, "-name")+1])
} }
if !reflect.DeepEqual(started, []string{"sw1", "rr1", "hv1"}) { if !reflect.DeepEqual(started, []string{"sw1", "rr1", "hv1"}) {
t.Errorf("start order = %v, want the switch first", started) t.Errorf("start order = %v, want the router first", started)
} }
ssh := f.runner.commands("ssh") ssh := f.runner.commands("ssh")
@ -438,7 +492,7 @@ func TestUp_StartsSwitchesFirstAndWaitsForEveryNode(t *testing.T) {
} }
func TestUp_RetriesWhileSSHIsUnreachable(t *testing.T) { func TestUp_RetriesWhileSSHIsUnreachable(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.runner.ssh["2201"] = []error{exitErr(255), exitErr(255), nil} f.runner.ssh["2201"] = []error{exitErr(255), exitErr(255), nil}
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil { if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
@ -460,7 +514,7 @@ func TestUp_ReadinessFailures(t *testing.T) {
} }
for name, c := range cases { for name, c := range cases {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.lab.Poll = 10 * time.Millisecond f.lab.Poll = 10 * time.Millisecond
f.runner.ssh["2201"] = c.answers f.runner.ssh["2201"] = c.answers
f.runner.always["2201"] = exitErr(255) f.runner.always["2201"] = exitErr(255)
@ -478,7 +532,7 @@ func TestUp_ReadinessFailures(t *testing.T) {
} }
func TestUp_RecoverableCloudInitErrorsAreReportedNotFatal(t *testing.T) { func TestUp_RecoverableCloudInitErrorsAreReportedNotFatal(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.runner.ssh["2201"] = []error{exitErr(2)} f.runner.ssh["2201"] = []error{exitErr(2)}
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil { if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
@ -490,7 +544,7 @@ func TestUp_RecoverableCloudInitErrorsAreReportedNotFatal(t *testing.T) {
} }
func TestUp_RefusesARunningLab(t *testing.T) { func TestUp_RefusesARunningLab(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.process(t, "hv1", "sleep 30", qemu("hv1")...) f.process(t, "hv1", "sleep 30", qemu("hv1")...)
err := f.lab.Up(context.Background(), f.fetcher, time.Second) err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -504,7 +558,7 @@ func TestUp_RefusesARunningLab(t *testing.T) {
} }
func TestUp_RemovesAStalePidFileBeforeStarting(t *testing.T) { func TestUp_RemovesAStalePidFileBeforeStarting(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
pid := f.process(t, "hv1", "exit 0", qemu("hv1")...) pid := f.process(t, "hv1", "exit 0", qemu("hv1")...)
waitDead(t, pid) waitDead(t, pid)
@ -517,7 +571,7 @@ func TestUp_RemovesAStalePidFileBeforeStarting(t *testing.T) {
} }
func TestUp_StopsAtTheFirstQEMUFailure(t *testing.T) { func TestUp_StopsAtTheFirstQEMUFailure(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.runner.fail = "qemu-system-x86_64" f.runner.fail = "qemu-system-x86_64"
err := f.lab.Up(context.Background(), f.fetcher, time.Second) err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -534,7 +588,7 @@ func TestUp_StopsAtTheFirstQEMUFailure(t *testing.T) {
} }
func TestStop_RefusesPidsThatTargetAGroup(t *testing.T) { func TestStop_RefusesPidsThatTargetAGroup(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.lab.signal = func(pid int, sig syscall.Signal) error { f.lab.signal = func(pid int, sig syscall.Signal) error {
t.Fatalf("signal %v sent to pid %d", sig, pid) t.Fatalf("signal %v sent to pid %d", sig, pid)
return nil return nil
@ -547,7 +601,7 @@ func TestStop_RefusesPidsThatTargetAGroup(t *testing.T) {
} }
func TestUp_ChecksTheServicesOfEachRole(t *testing.T) { func TestUp_ChecksTheServicesOfEachRole(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
conf := filepath.Join(t.TempDir(), "sw1.conf") conf := filepath.Join(t.TempDir(), "sw1.conf")
if err := os.WriteFile(conf, []byte("hostname sw1\n"), 0o600); err != nil { if err := os.WriteFile(conf, []byte("hostname sw1\n"), 0o600); err != nil {
t.Fatal(err) t.Fatal(err)
@ -574,7 +628,7 @@ func TestUp_ChecksTheServicesOfEachRole(t *testing.T) {
} }
func TestUp_ReportsAnInactiveServiceAndWaitsForTheOthers(t *testing.T) { func TestUp_ReportsAnInactiveServiceAndWaitsForTheOthers(t *testing.T) {
f := newFixture(t, switchLast) f := newFixture(t, routerLast)
f.runner.inactive = "agent.service" f.runner.inactive = "agent.service"
err := f.lab.Up(context.Background(), f.fetcher, time.Second) err := f.lab.Up(context.Background(), f.fetcher, time.Second)
@ -586,3 +640,293 @@ func TestUp_ReportsAnInactiveServiceAndWaitsForTheOthers(t *testing.T) {
t.Errorf("output = %q", f.out.String()) t.Errorf("output = %q", f.out.String())
} }
} }
func TestStatus_ShowsTheSegments(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present("lb-red")
var out bytes.Buffer
if err := f.lab.Status(&out); err != nil {
t.Fatal(err)
}
want := "\nsegment bridge state gateway\n" +
"red lb-red present 10.1.0.1\n" +
"blue lb-blue absent -\n"
if !strings.HasSuffix(out.String(), want) {
t.Errorf("status:\n%s\ndoes not end with:\n%s", out.String(), want)
}
}
func TestNetworkUp_CreatesBridgesAndTaps(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
want := []string{
"link show dev lb-red",
"link show dev lb-blue",
"link show dev lt0",
"link show dev lt1",
"link show dev lt2",
"link show dev lt3",
"link add name lb-red type bridge stp_state 0 mcast_snooping 0",
"link set dev lb-red addrgenmode none",
"tuntap add dev lt0 mode tap",
"link set dev lt0 addrgenmode none",
"link set dev lt0 mtu 9000 master lb-red up",
"tuntap add dev lt1 mode tap",
"link set dev lt1 addrgenmode none",
"link set dev lt1 mtu 9000 master lb-red up",
"tuntap add dev lt2 mode tap",
"link set dev lt2 addrgenmode none",
"link set dev lt2 mtu 9000 master lb-red up",
"link set dev lb-red mtu 9000 up",
"link add name lb-blue type bridge stp_state 0 mcast_snooping 0",
"link set dev lb-blue addrgenmode none",
"tuntap add dev lt3 mode tap",
"link set dev lt3 addrgenmode none",
"link set dev lt3 mtu 1500 master lb-blue up",
"link set dev lb-blue mtu 1500 up",
}
if got := f.runner.ipCalls(); !reflect.DeepEqual(got, want) {
t.Errorf("ip calls:\n%s\nwant:\n%s", strings.Join(got, "\n"), strings.Join(want, "\n"))
}
if len(f.runner.calls) != len(want) {
t.Errorf("other commands ran: %q", f.runner.calls)
}
}
func TestNetworkUp_RefusesExistingLinks(t *testing.T) {
cases := map[string]struct {
link, want string
}{
"bridge": {"lb-blue", "segment blue: bridge lb-blue already exists on this host: 'labctl down' first, or remove it"},
"tap": {"lt2", "node hv: tap lt2 already exists on this host: 'labctl down' first, or remove it"},
}
for name, c := range cases {
t.Run(name, func(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present(c.link)
dir := filepath.Join(f.lab.ProcDir, "sys", "net", "bridge")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "bridge-nf-call-iptables"), []byte("1\n"), 0o644); err != nil {
t.Fatal(err)
}
err := f.lab.NetworkUp(context.Background())
if err == nil || err.Error() != c.want {
t.Errorf("error = %v, want %q", err, c.want)
}
for _, call := range f.runner.ipCalls() {
if !strings.HasPrefix(call, "link show dev ") {
t.Errorf("%q ran after the refusal", call)
}
}
if data, _ := os.ReadFile(filepath.Join(dir, "bridge-nf-call-iptables")); string(data) != "1\n" {
t.Errorf("bridge filter changed to %q", data)
}
})
}
}
func TestNetworkUp_DisablesTheBridgeFilterWhenPresent(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
dir := filepath.Join(f.lab.ProcDir, "sys", "net", "bridge")
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "bridge-nf-call-iptables")
if err := os.WriteFile(path, []byte("1\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
if data, err := os.ReadFile(path); err != nil || string(data) != "0\n" {
t.Errorf("bridge filter = %q, %v, want 0", data, err)
}
}
func TestNetworkUp_LeavesAnAbsentBridgeFilterAlone(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.NetworkUp(context.Background()); err != nil {
t.Fatalf("NetworkUp: %v", err)
}
if _, err := os.Stat(filepath.Join(f.lab.ProcDir, "sys")); !os.IsNotExist(err) {
t.Errorf("bridge filter created under the proc dir: %v", err)
}
}
func TestNetworkUp_ReportsTheFailingSegment(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.fail = "ip"
err := f.lab.NetworkUp(context.Background())
if err == nil || err.Error() != "segment red: ip failed" {
t.Errorf("error = %v", err)
}
}
func TestNetworkDown_DeletesTapsThenBridgesThatExist(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
f.runner.present("lb-red", "lb-blue", "lt0", "lt2", "lt3")
if err := f.lab.NetworkDown(context.Background()); err != nil {
t.Fatalf("NetworkDown: %v", err)
}
var deleted []string
for _, call := range f.runner.ipCalls() {
if link, ok := strings.CutPrefix(call, "link delete dev "); ok {
deleted = append(deleted, link)
}
}
if want := []string{"lt0", "lt2", "lt3", "lb-red", "lb-blue"}; !reflect.DeepEqual(deleted, want) {
t.Errorf("deleted %v, want %v", deleted, want)
}
if len(f.runner.links) != 0 {
t.Errorf("links left: %v", f.runner.links)
}
}
func TestDown_RemovesTheNetworkAfterTheNodes(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay", "lt0", "lt1", "lt2")
f.process(t, "hv1", "sleep 30", qemu("hv1")...)
if err := f.lab.Down(context.Background()); err != nil {
t.Fatalf("Down: %v", err)
}
var deleted []string
for _, call := range f.runner.ipCalls() {
if link, ok := strings.CutPrefix(call, "link delete dev "); ok {
deleted = append(deleted, link)
}
}
if want := []string{"lt0", "lt1", "lt2", "lb-underlay"}; !reflect.DeepEqual(deleted, want) {
t.Errorf("deleted %v, want %v", deleted, want)
}
if f.out.String() != "hv1: stopped\n" {
t.Errorf("output = %q", f.out.String())
}
}
func TestDown_KeepsTheNetworkWhenANodeCannotBeStopped(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay", "lt0", "lt1", "lt2")
f.pidFile(t, "rr1", "garbage")
if err := f.lab.Down(context.Background()); err == nil {
t.Fatal("Down succeeded")
}
if calls := f.runner.ipCalls(); len(calls) != 0 {
t.Errorf("ip calls after a failed stop: %q", calls)
}
}
func TestUp_BuildsTheNetworkBeforeStartingQEMU(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err)
}
lastIP, firstQEMU := -1, -1
var started []string
for i, c := range f.runner.calls {
switch c[0] {
case "ip":
lastIP = i
case "qemu-system-x86_64":
if firstQEMU < 0 {
firstQEMU = i
}
started = append(started, c[indexOf(c, "-name")+1])
}
}
if lastIP < 0 || firstQEMU < 0 || lastIP > firstQEMU {
t.Errorf("last ip call at %d, first qemu start at %d", lastIP, firstQEMU)
}
if !reflect.DeepEqual(started, []string{"r1", "r2", "hv"}) {
t.Errorf("start order = %v", started)
}
for _, link := range []string{"lb-red", "lb-blue", "lt0", "lt1", "lt2", "lt3"} {
if !f.runner.links[link] {
t.Errorf("%s was not created", link)
}
}
}
func TestUp_RefusesAnExistingBridgeBeforeStartingQEMU(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.present("lb-underlay")
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
if err == nil || !strings.Contains(err.Error(), "bridge lb-underlay already exists") {
t.Errorf("error = %v", err)
}
if n := len(f.runner.commands("qemu-system-x86_64")); n != 0 {
t.Errorf("%d qemu starts", n)
}
}
func TestUp_StopsWhenTheNetworkFails(t *testing.T) {
f := newFixture(t, routerLast)
f.runner.fail = "ip"
err := f.lab.Up(context.Background(), f.fetcher, time.Second)
if err == nil || err.Error() != "segment underlay: ip failed" {
t.Errorf("error = %v", err)
}
if n := len(f.runner.commands("qemu-system-x86_64")); n != 0 {
t.Errorf("%d qemu starts", n)
}
}
func TestUp_ChecksFRROnVRRPRouters(t *testing.T) {
f := newLab(t, twoSegments, twoRouters)
if err := f.lab.Up(context.Background(), f.fetcher, time.Second); err != nil {
t.Fatalf("Up: %v", err)
}
var checks []string
for _, c := range f.runner.commands("ssh") {
if c[len(c)-4] == "systemctl" {
checks = append(checks, c[indexOf(c, "-p")+1]+" "+strings.Join(c[len(c)-4:], " "))
}
}
want := []string{
"2200 systemctl is-active --quiet frr.service",
"2201 systemctl is-active --quiet frr.service",
}
if !reflect.DeepEqual(checks, want) {
t.Errorf("checks = %q, want %q", checks, want)
}
}
func TestServices(t *testing.T) {
f := newLab(t, twoSegments+underlay, twoRouters+` sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay], services: [x.service] }
`)
want := map[string][]string{
"r1": {"frr.service"},
"r2": {"frr.service"},
"hv": nil,
"sw1": {"x.service"},
}
for _, n := range f.lab.Plan.Nodes {
if got := services(f.lab.Plan, n); !reflect.DeepEqual(got, want[n.Name]) {
t.Errorf("%s services = %q, want %q", n.Name, got, want[n.Name])
}
}
}

122
internal/machine/network.go Normal file
View file

@ -0,0 +1,122 @@
package machine
import (
"context"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"text/tabwriter"
"git.g3e.fr/syonad/labctl/internal/topology"
)
const bridgeFilter = "sys/net/bridge/bridge-nf-call-iptables"
func (l Lab) exists(ctx context.Context, link string) bool {
return l.Runner.Run(ctx, "ip", "link", "show", "dev", link) == nil
}
func (l Lab) NetworkUp(ctx context.Context) error {
for _, s := range l.Plan.Segments {
if l.exists(ctx, s.Bridge) {
return fmt.Errorf("segment %s: bridge %s already exists on this host: 'labctl down' first, or remove it", s.Name, s.Bridge)
}
}
for _, port := range l.Plan.Ports {
if l.exists(ctx, port.Tap) {
return fmt.Errorf("node %s: tap %s already exists on this host: 'labctl down' first, or remove it", port.Node, port.Tap)
}
}
if err := l.disableBridgeFilter(); err != nil {
return err
}
for _, s := range l.Plan.Segments {
if err := l.ip(ctx,
[]string{"link", "add", "name", s.Bridge, "type", "bridge", "stp_state", "0", "mcast_snooping", "0"},
[]string{"link", "set", "dev", s.Bridge, "addrgenmode", "none"},
); err != nil {
return fmt.Errorf("segment %s: %w", s.Name, err)
}
for _, port := range l.Plan.Ports {
if port.Segment != s.Name {
continue
}
mtu := strconv.Itoa(port.MTU)
if err := l.ip(ctx,
[]string{"tuntap", "add", "dev", port.Tap, "mode", "tap"},
[]string{"link", "set", "dev", port.Tap, "addrgenmode", "none"},
[]string{"link", "set", "dev", port.Tap, "mtu", mtu, "master", s.Bridge, "up"},
); err != nil {
return fmt.Errorf("node %s: %w", port.Node, err)
}
}
if err := l.ip(ctx, []string{"link", "set", "dev", s.Bridge, "mtu", strconv.Itoa(s.MTU), "up"}); err != nil {
return fmt.Errorf("segment %s: %w", s.Name, err)
}
}
return nil
}
func (l Lab) NetworkDown(ctx context.Context) error {
var errs []error
for _, port := range l.Plan.Ports {
if !l.exists(ctx, port.Tap) {
continue
}
if err := l.ip(ctx, []string{"link", "delete", "dev", port.Tap}); err != nil {
errs = append(errs, fmt.Errorf("node %s: %w", port.Node, err))
}
}
for _, s := range l.Plan.Segments {
if !l.exists(ctx, s.Bridge) {
continue
}
if err := l.ip(ctx, []string{"link", "delete", "dev", s.Bridge}); err != nil {
errs = append(errs, fmt.Errorf("segment %s: %w", s.Name, err))
}
}
return errors.Join(errs...)
}
func (l Lab) ip(ctx context.Context, commands ...[]string) error {
for _, args := range commands {
if err := l.Runner.Run(ctx, "ip", args...); err != nil {
return err
}
}
return nil
}
func (l Lab) disableBridgeFilter() error {
path := filepath.Join(l.ProcDir, bridgeFilter)
if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
return nil
}
if err := os.WriteFile(path, []byte("0\n"), 0o644); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
return nil
}
func (l Lab) networkStatus(w io.Writer) error {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintf(tw, "\nsegment\tbridge\tstate\tgateway\n")
for _, s := range l.Plan.Segments {
state := "absent"
if l.exists(context.Background(), s.Bridge) {
state = "present"
}
fmt.Fprintf(tw, "%s\t%s\t%s\t%s\n", s.Name, s.Bridge, state, gatewayOf(s))
}
return tw.Flush()
}
func gatewayOf(s topology.SegmentPlan) string {
if !s.Gateway.IsValid() {
return "-"
}
return s.Gateway.String()
}

View file

@ -56,12 +56,12 @@ images:
url: ` + img.URL + ` url: ` + img.URL + `
sums: ` + img.Sums + ` sums: ` + img.Sums + `
segments: segments:
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 } underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
nodes: nodes:
sw1: { role: switch, image: debian12, cpus: 2, memory: 1024 } sw1: { role: router, image: debian12, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay] } rr1: { role: rr, image: debian12, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] } hv1: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] } hv2: { role: host, image: debian12, cpus: 4, memory: 16384, segments: [underlay] }
` `
topo, err := topology.Parse([]byte(doc)) topo, err := topology.Parse([]byte(doc))
if err != nil { if err != nil {

View file

@ -31,7 +31,7 @@ func qemuArgs(p *topology.Plan, n topology.NodePlan, index int, dir string) []st
} }
restrict := "on" restrict := "on"
if n.Role == topology.RoleSwitch { if n.Role == topology.RoleRouter {
restrict = "off" restrict = "off"
} }
args = append(args, args = append(args,
@ -39,19 +39,11 @@ func qemuArgs(p *topology.Plan, n topology.NodePlan, index int, dir string) []st
"-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,romfile=", AdminInterface, adminMAC(index)), "-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,romfile=", AdminInterface, adminMAC(index)),
) )
for _, c := range nodeCables(p, n.Name) { for _, port := range nodePorts(p, n.Name) {
args = append(args, cable(c.NodeInterface, c.NodePort, c.SwitchPort, c.NodeMAC.String(), c.MTU)...) args = append(args,
} "-netdev", fmt.Sprintf("tap,id=%s,ifname=%s,script=no,downscript=no", port.Interface, port.Tap),
for _, c := range switchCables(p, n.Name) { "-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,host_mtu=%d,romfile=", port.Interface, port.MAC, port.MTU),
args = append(args, cable(c.SwitchInterface, c.SwitchPort, c.NodePort, c.SwitchMAC.String(), c.MTU)...) )
} }
return args return args
} }
func cable(id string, local, remote int, mac string, mtu int) []string {
return []string{
"-netdev", fmt.Sprintf("dgram,id=%s,local.type=inet,local.host=%s,local.port=%d,remote.type=inet,remote.host=%s,remote.port=%d",
id, loopback, local, loopback, remote),
"-device", fmt.Sprintf("virtio-net-pci,netdev=%s,mac=%s,host_mtu=%d,romfile=", id, mac, mtu),
}
}

View file

@ -119,32 +119,46 @@ func adminMAC(index int) net.HardwareAddr {
return net.HardwareAddr{0x02, 0x4d, byte(index >> 8), byte(index), 0x00, 0x00} return net.HardwareAddr{0x02, 0x4d, byte(index >> 8), byte(index), 0x00, 0x00}
} }
func nodeCables(p *topology.Plan, name string) []topology.Cable { func nodePorts(p *topology.Plan, name string) []topology.Port {
var out []topology.Cable var out []topology.Port
for _, c := range p.Cables { for _, port := range p.Ports {
if c.Node == name { if port.Node == name {
out = append(out, c) out = append(out, port)
} }
} }
return out return out
} }
func switchCables(p *topology.Plan, name string) []topology.Cable { func routedSegments(p *topology.Plan, name string) []topology.SegmentPlan {
var out []topology.Cable
for _, c := range p.Cables {
if c.Switch == name {
out = append(out, c)
}
}
return out
}
func switchSegments(p *topology.Plan, name string) []topology.SegmentPlan {
var out []topology.SegmentPlan var out []topology.SegmentPlan
for _, s := range p.Segments { for _, s := range p.Segments {
if s.Switch == name { for _, r := range s.Routers {
out = append(out, s) if r == name {
out = append(out, s)
}
} }
} }
return out return out
} }
func segmentNamed(p *topology.Plan, name string) topology.SegmentPlan {
for _, s := range p.Segments {
if s.Name == name {
return s
}
}
return topology.SegmentPlan{}
}
func uplink(p *topology.Plan, name string) (topology.Port, bool) {
ports := nodePorts(p, name)
for _, port := range ports {
if segmentNamed(p, port.Segment).Gateway.IsValid() {
return port, true
}
}
if len(ports) > 0 {
return ports[0], false
}
return topology.Port{}, false
}

View file

@ -2,6 +2,7 @@ package render
import ( import (
"reflect" "reflect"
"strconv"
"strings" "strings"
"testing" "testing"
@ -18,12 +19,12 @@ images:
url: https://example.invalid/deb.qcow2 url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS sums: https://example.invalid/SHA512SUMS
segments: segments:
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 } underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 } sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] } rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] } hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] } hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
` `
const twoSegments = `name: two-seg const twoSegments = `name: two-seg
@ -32,13 +33,30 @@ images:
url: https://example.invalid/deb.qcow2 url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS sums: https://example.invalid/SHA512SUMS
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24, mtu: 1500 } blue: { routers: [sw], cidr: 10.2.0.0/24, mtu: 1500 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] } hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] }
` `
const withVRRP = `name: vrrp
images:
deb:
url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS
segments:
red: { routers: [r1, r2], cidr: 10.1.0.0/24 }
blue: { routers: [r1], cidr: 10.2.0.0/24 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, pure] }
r3: { role: router, image: deb, cpus: 1, memory: 512, segments: [pure] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [pure, red] }
l2: { role: host, image: deb, cpus: 1, memory: 512, segments: [pure] }
`
func plan(t *testing.T, doc string) *topology.Plan { func plan(t *testing.T, doc string) *topology.Plan {
t.Helper() t.Helper()
topo, err := topology.Parse([]byte(doc)) topo, err := topology.Parse([]byte(doc))
@ -182,7 +200,7 @@ func TestQEMU_HypervisorNetwork(t *testing.T) {
hv := nodeNamed(t, renderAll(t, twoHypervisors), "hv1") hv := nodeNamed(t, renderAll(t, twoHypervisors), "hv1")
wantNetdevs := []string{ wantNetdevs := []string{
"user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22", "user,id=mgmt0,restrict=on,ipv6=off,hostfwd=tcp:127.0.0.1:2202-:22",
"dgram,id=underlay,local.type=inet,local.host=127.0.0.1,local.port=20002,remote.type=inet,remote.host=127.0.0.1,remote.port=20003", "tap,id=underlay,ifname=lt2,script=no,downscript=no",
} }
wantDevices := []string{ wantDevices := []string{
"virtio-net-pci,netdev=mgmt0,mac=02:4d:00:02:00:00,romfile=", "virtio-net-pci,netdev=mgmt0,mac=02:4d:00:02:00:00,romfile=",
@ -196,27 +214,31 @@ func TestQEMU_HypervisorNetwork(t *testing.T) {
} }
} }
func TestQEMU_SwitchNetwork(t *testing.T) { func TestQEMU_RouterNetwork(t *testing.T) {
sw := nodeNamed(t, renderAll(t, twoHypervisors), "sw1") sw := nodeNamed(t, renderAll(t, twoSegments), "sw")
wantNetdevs := []string{ wantNetdevs := []string{
"user,id=mgmt0,restrict=off,ipv6=off,hostfwd=tcp:127.0.0.1:2200-:22", "user,id=mgmt0,restrict=off,ipv6=off,hostfwd=tcp:127.0.0.1:2200-:22",
"dgram,id=p0,local.type=inet,local.host=127.0.0.1,local.port=20001,remote.type=inet,remote.host=127.0.0.1,remote.port=20000", "tap,id=red,ifname=lt0,script=no,downscript=no",
"dgram,id=p1,local.type=inet,local.host=127.0.0.1,local.port=20003,remote.type=inet,remote.host=127.0.0.1,remote.port=20002", "tap,id=blue,ifname=lt2,script=no,downscript=no",
"dgram,id=p2,local.type=inet,local.host=127.0.0.1,local.port=20005,remote.type=inet,remote.host=127.0.0.1,remote.port=20004", }
wantDevices := []string{
"virtio-net-pci,netdev=mgmt0,mac=02:4d:00:00:00:00,romfile=",
"virtio-net-pci,netdev=red,mac=02:4c:00:00:00:00,host_mtu=9000,romfile=",
"virtio-net-pci,netdev=blue,mac=02:4c:00:00:01:00,host_mtu=1500,romfile=",
} }
if got := value(t, sw.QEMU, "-netdev"); !reflect.DeepEqual(got, wantNetdevs) { if got := value(t, sw.QEMU, "-netdev"); !reflect.DeepEqual(got, wantNetdevs) {
t.Errorf("netdevs = %v\nwant %v", got, wantNetdevs) t.Errorf("netdevs = %v\nwant %v", got, wantNetdevs)
} }
if !hasValue(sw.QEMU, "-device", "virtio-net-pci,netdev=p2,mac=02:4c:00:03:00:01,host_mtu=9000,romfile=") { if got := value(t, sw.QEMU, "-device"); !reflect.DeepEqual(got, wantDevices) {
t.Errorf("p2 device missing: %v", sw.QEMU) t.Errorf("devices = %v\nwant %v", got, wantDevices)
} }
} }
func TestQEMU_OnlyTheSwitchReachesTheOutsideThroughAdministration(t *testing.T) { func TestQEMU_OnlyRoutersReachTheOutsideThroughAdministration(t *testing.T) {
for name, n := range renderAll(t, twoHypervisors) { for name, n := range renderAll(t, withVRRP) {
admin := value(t, n.QEMU, "-netdev")[0] admin := value(t, n.QEMU, "-netdev")[0]
wantRestrict := "restrict=on" wantRestrict := "restrict=on"
if name == "sw1" { if name == "r1" || name == "r2" || name == "r3" {
wantRestrict = "restrict=off" wantRestrict = "restrict=off"
} }
if !strings.Contains(admin, ","+wantRestrict+",") { if !strings.Contains(admin, ","+wantRestrict+",") {
@ -228,33 +250,28 @@ func TestQEMU_OnlyTheSwitchReachesTheOutsideThroughAdministration(t *testing.T)
} }
} }
func TestQEMU_EveryCableEndsMatch(t *testing.T) { func TestQEMU_EveryTapOfThePlanIsOpenedOnce(t *testing.T) {
ends := map[string]int{} p := plan(t, withVRRP)
for _, n := range renderAll(t, twoHypervisors) { opened := map[string]int{}
for _, n := range renderAll(t, withVRRP) {
for _, nd := range value(t, n.QEMU, "-netdev") { for _, nd := range value(t, n.QEMU, "-netdev") {
if !strings.HasPrefix(nd, "dgram,") { if strings.HasPrefix(nd, "dgram") || strings.HasPrefix(nd, "socket") {
continue t.Errorf("%s uses a %s netdev", n.Name, nd)
} }
var local, remote string
for _, kv := range strings.Split(nd, ",") { for _, kv := range strings.Split(nd, ",") {
if v, ok := strings.CutPrefix(kv, "local.port="); ok { if tap, ok := strings.CutPrefix(kv, "ifname="); ok {
local = v opened[tap]++
}
if v, ok := strings.CutPrefix(kv, "remote.port="); ok {
remote = v
} }
} }
ends[local+">"+remote]++
} }
} }
for pair, count := range ends { if len(opened) != len(p.Ports) {
local, remote, _ := strings.Cut(pair, ">") t.Errorf("%d taps opened, plan has %d ports", len(opened), len(p.Ports))
if count != 1 || ends[remote+">"+local] != 1 {
t.Errorf("cable end %s has no single matching end", pair)
}
} }
if len(ends) != 6 { for _, port := range p.Ports {
t.Errorf("%d cable ends, want 6", len(ends)) if opened[port.Tap] != 1 {
t.Errorf("tap %s opened %d times", port.Tap, opened[port.Tap])
}
} }
} }
@ -270,7 +287,7 @@ func TestNetworkConfig_Hypervisor(t *testing.T) {
} }
under := iface(t, doc, "underlay") under := iface(t, doc, "underlay")
if under.Match.MACAddress != "02:4c:00:02:00:00" || under.SetName != "underlay" || under.MTU != 9000 || if under.Match.MACAddress != "02:4c:00:02:00:00" || under.SetName != "underlay" || under.MTU != 9000 ||
!reflect.DeepEqual(under.Addresses, []string{"10.250.0.3/24"}) { !reflect.DeepEqual(under.Addresses, []string{"10.250.0.4/24"}) {
t.Errorf("underlay = %+v", under) t.Errorf("underlay = %+v", under)
} }
if !reflect.DeepEqual(under.Routes, []route{{To: "0.0.0.0/0", Via: "10.250.0.1"}}) { if !reflect.DeepEqual(under.Routes, []route{{To: "0.0.0.0/0", Via: "10.250.0.1"}}) {
@ -289,17 +306,27 @@ func TestNetworkConfig_DHCPIsExplicitlyOffEverywhere(t *testing.T) {
} }
} }
func TestNetworkConfig_Switch(t *testing.T) { func TestNetworkConfig_Router(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, twoHypervisors), "sw1")) doc := network(t, nodeNamed(t, renderAll(t, twoSegments), "sw"))
if len(doc.Ethernets) != 3 {
t.Fatalf("network-config = %+v", doc)
}
admin := iface(t, doc, "mgmt0") admin := iface(t, doc, "mgmt0")
if !reflect.DeepEqual(admin.Routes, []route{{To: "0.0.0.0/0", Via: "10.0.2.2"}}) || if !reflect.DeepEqual(admin.Routes, []route{{To: "0.0.0.0/0", Via: "10.0.2.2"}}) ||
admin.Nameservers == nil || !reflect.DeepEqual(admin.Nameservers.Addresses, []string{"10.0.2.3"}) { admin.Nameservers == nil || !reflect.DeepEqual(admin.Nameservers.Addresses, []string{"10.0.2.3"}) {
t.Errorf("switch mgmt0 = %+v", admin) t.Errorf("router mgmt0 = %+v", admin)
} }
for port, mac := range map[string]string{"p0": "02:4c:00:01:00:01", "p1": "02:4c:00:02:00:01", "p2": "02:4c:00:03:00:01"} { for name, want := range map[string]struct {
e := iface(t, doc, port) mac, address string
if e.Match.MACAddress != mac || e.SetName != port || e.MTU != 9000 || len(e.Addresses) != 0 || len(e.Routes) != 0 { mtu int
t.Errorf("%s = %+v", port, e) }{
"red": {"02:4c:00:00:00:00", "10.1.0.2/24", 9000},
"blue": {"02:4c:00:00:01:00", "10.2.0.2/24", 1500},
} {
e := iface(t, doc, name)
if e.Match.MACAddress != want.mac || e.SetName != name || e.MTU != want.mtu ||
!reflect.DeepEqual(e.Addresses, []string{want.address}) || len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v", name, e)
} }
} }
} }
@ -333,45 +360,71 @@ func TestUserData_PasswordLoginIsExplicitlyOff(t *testing.T) {
} }
} }
func TestUserData_SwitchBuildsBridgeGatewayAndNAT(t *testing.T) { func TestNetworkConfig_DefaultRouteOnTheFirstSegmentWithAGateway(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, withVRRP), "hv"))
pure, red := iface(t, doc, "pure"), iface(t, doc, "red")
if len(pure.Routes) != 0 || pure.Nameservers != nil || !reflect.DeepEqual(pure.Addresses, []string{"10.3.0.3/24"}) {
t.Errorf("pure = %+v", pure)
}
if !reflect.DeepEqual(red.Routes, []route{{To: "0.0.0.0/0", Via: "10.1.0.1"}}) || red.Nameservers == nil ||
!reflect.DeepEqual(red.Addresses, []string{"10.1.0.4/24"}) {
t.Errorf("red = %+v", red)
}
}
func TestNetworkConfig_NoDefaultRouteWithoutGateway(t *testing.T) {
for name, e := range network(t, nodeNamed(t, renderAll(t, withVRRP), "l2")).Ethernets {
if len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v, want no route", name, e)
}
}
}
func TestNetworkConfig_RouterHasNoDefaultRouteOnItsSegments(t *testing.T) {
doc := network(t, nodeNamed(t, renderAll(t, withVRRP), "r1"))
for _, name := range []string{"red", "blue"} {
if e := iface(t, doc, name); len(e.Routes) != 0 || e.Nameservers != nil {
t.Errorf("%s = %+v", name, e)
}
}
if admin := iface(t, doc, "mgmt0"); len(admin.Routes) != 1 {
t.Errorf("mgmt0 = %+v", admin)
}
}
func TestUserData_RouterBuildsGatewayAndNAT(t *testing.T) {
cfg := user(t, nodeNamed(t, renderAll(t, twoHypervisors), "sw1")) cfg := user(t, nodeNamed(t, renderAll(t, twoHypervisors), "sw1"))
if !reflect.DeepEqual(cfg.Packages, []string{"nftables"}) { if !reflect.DeepEqual(cfg.Packages, []string{"nftables"}) {
t.Errorf("packages = %v", cfg.Packages) t.Errorf("packages = %v", cfg.Packages)
} }
script := fileAt(t, cfg, "/usr/local/sbin/lab-switch") script := fileAt(t, cfg, "/usr/local/sbin/lab-router")
if script.Permissions != "0755" { if script.Permissions != "0755" {
t.Errorf("script permissions = %s", script.Permissions) t.Errorf("script permissions = %s", script.Permissions)
} }
wantScript := `#!/bin/sh wantScript := `#!/bin/sh
set -eu set -eu
sysctl -qw net.ipv4.ip_forward=1 sysctl -qw net.ipv4.ip_forward=1
ip link add br-underlay type bridge stp_state 0 2>/dev/null || true ip addr replace 10.250.0.1/24 dev underlay
ip link set dev p0 master br-underlay nft -f /etc/lab-router.nft
ip link set dev p0 up
ip link set dev p1 master br-underlay
ip link set dev p1 up
ip link set dev p2 master br-underlay
ip link set dev p2 up
ip link set dev br-underlay mtu 9000
ip addr replace 10.250.0.1/24 dev br-underlay
ip link set dev br-underlay up
nft -f /etc/lab-switch.nft
` `
if script.Content != wantScript { if script.Content != wantScript {
t.Errorf("script:\n%s\nwant:\n%s", script.Content, wantScript) t.Errorf("script:\n%s\nwant:\n%s", script.Content, wantScript)
} }
nft := fileAt(t, cfg, "/etc/lab-switch.nft").Content nftFile := fileAt(t, cfg, "/etc/lab-router.nft")
if nftFile.Permissions != "0644" {
t.Errorf("nft permissions = %s", nftFile.Permissions)
}
for _, want := range []string{ for _, want := range []string{
"add table ip lab_nat\ndelete table ip lab_nat\n", "add table ip lab_nat\ndelete table ip lab_nat\n",
"type nat hook postrouting priority srcnat;", "type nat hook postrouting priority srcnat;",
`ip saddr { 10.250.0.0/24 } oifname "mgmt0" masquerade`, `ip saddr { 10.250.0.0/24 } oifname "mgmt0" masquerade`,
} { } {
if !strings.Contains(nft, want) { if !strings.Contains(nftFile.Content, want) {
t.Errorf("nft rules miss %q:\n%s", want, nft) t.Errorf("nft rules miss %q:\n%s", want, nftFile.Content)
} }
} }
unit := fileAt(t, cfg, "/etc/systemd/system/lab-switch.service").Content unit := fileAt(t, cfg, "/etc/systemd/system/lab-router.service").Content
for _, want := range []string{"Type=oneshot", "RemainAfterExit=yes", "ExecStart=/usr/local/sbin/lab-switch", "After=network-online.target", "WantedBy=multi-user.target"} { for _, want := range []string{"Type=oneshot", "RemainAfterExit=yes", "ExecStart=/usr/local/sbin/lab-router", "After=network-online.target", "WantedBy=multi-user.target"} {
if !strings.Contains(unit, want) { if !strings.Contains(unit, want) {
t.Errorf("unit misses %q", want) t.Errorf("unit misses %q", want)
} }
@ -379,29 +432,82 @@ nft -f /etc/lab-switch.nft
if !reflect.DeepEqual(cfg.Runcmd, [][]string{{"/usr/local/sbin/lab-provision"}}) { if !reflect.DeepEqual(cfg.Runcmd, [][]string{{"/usr/local/sbin/lab-provision"}}) {
t.Errorf("runcmd = %v", cfg.Runcmd) t.Errorf("runcmd = %v", cfg.Runcmd)
} }
if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-switch.service\n" { if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-router.service\n" {
t.Errorf("lab-provision:\n%s", got) t.Errorf("lab-provision:\n%s", got)
} }
} }
func TestUserData_SwitchWithTwoSegments(t *testing.T) { func TestUserData_RouterWithTwoSegments(t *testing.T) {
cfg := user(t, nodeNamed(t, renderAll(t, twoSegments), "sw")) cfg := user(t, nodeNamed(t, renderAll(t, twoSegments), "sw"))
script := fileAt(t, cfg, "/usr/local/sbin/lab-switch").Content script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content
for _, want := range []string{ want := "sysctl -qw net.ipv4.ip_forward=1\nip addr replace 10.1.0.1/24 dev red\nip addr replace 10.2.0.1/24 dev blue\nnft -f /etc/lab-router.nft\n"
"ip link set dev p0 master br-red\n", if !strings.HasSuffix(script, want) {
"ip link set dev p1 master br-blue\n", t.Errorf("script:\n%s\ndoes not end with:\n%s", script, want)
"ip link set dev br-blue mtu 1500\n", }
"ip addr replace 10.2.0.1/24 dev br-blue\n", if nft := fileAt(t, cfg, "/etc/lab-router.nft").Content; !strings.Contains(nft, "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }") {
t.Errorf("nft:\n%s", nft)
}
}
func TestUserData_RouterScriptWithVRRPAndSingleSegments(t *testing.T) {
script := fileAt(t, user(t, nodeNamed(t, renderAll(t, withVRRP), "r1")), "/usr/local/sbin/lab-router").Content
want := `#!/bin/sh
set -eu
sysctl -qw net.ipv4.ip_forward=1
sysctl -qw net.ipv4.conf.red.arp_ignore=1 net.ipv4.conf.red.arp_announce=2
ip link show dev vrred >/dev/null 2>&1 || { ip link add vrred link red type macvlan mode bridge && ip link set dev vrred protodown on; }
ip link set dev vrred addrgenmode random address 00:00:5e:00:01:01
ip addr replace 10.1.0.1/24 dev vrred
ip link set dev vrred up
ip addr replace 10.2.0.1/24 dev blue
nft -f /etc/lab-router.nft
`
if script != want {
t.Errorf("script:\n%s\nwant:\n%s", script, want)
}
r2 := fileAt(t, user(t, nodeNamed(t, renderAll(t, withVRRP), "r2")), "/usr/local/sbin/lab-router").Content
if !strings.Contains(r2, "ip addr replace 10.1.0.1/24 dev vrred\n") || strings.Contains(r2, "dev pure") {
t.Errorf("r2 script:\n%s", r2)
}
}
func TestUserData_RouterNATCoversTheSegmentsItRoutes(t *testing.T) {
nodes := renderAll(t, withVRRP)
for name, want := range map[string]string{
"r1": "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }",
"r2": "ip saddr { 10.1.0.0/24 }",
} { } {
if !strings.Contains(script, want) { if nft := fileAt(t, user(t, nodeNamed(t, nodes, name)), "/etc/lab-router.nft").Content; !strings.Contains(nft, want) {
t.Errorf("script misses %q:\n%s", want, script) t.Errorf("%s nft:\n%s\nmisses %s", name, nft, want)
} }
} }
if strings.Contains(script, "p1 master br-red") || strings.Contains(script, "p0 master br-blue") { cfg := user(t, nodeNamed(t, nodes, "r3"))
t.Errorf("port bridged on the wrong segment:\n%s", script) if nft := fileAt(t, cfg, "/etc/lab-router.nft").Content; nft != "add table ip lab_nat\ndelete table ip lab_nat\n" {
t.Errorf("r3 nft:\n%s", nft)
} }
if nft := fileAt(t, cfg, "/etc/lab-switch.nft").Content; !strings.Contains(nft, "ip saddr { 10.1.0.0/24, 10.2.0.0/24 }") { if script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content; script != "#!/bin/sh\nset -eu\nsysctl -qw net.ipv4.ip_forward=1\nnft -f /etc/lab-router.nft\n" {
t.Errorf("nft:\n%s", nft) t.Errorf("r3 script:\n%s", script)
}
}
func TestVRRPMAC(t *testing.T) {
for vrid, want := range map[int]string{1: "00:00:5e:00:01:01", 16: "00:00:5e:00:01:10", 255: "00:00:5e:00:01:ff"} {
if got := vrrpMAC(vrid); got != want {
t.Errorf("vrrpMAC(%d) = %s, want %s", vrid, got, want)
}
}
}
func TestVRRPPriorityFollowsTheRoutersOrder(t *testing.T) {
var routers []string
for i := 0; i < 22; i++ {
routers = append(routers, "r"+strconv.Itoa(i))
}
s := topology.SegmentPlan{Routers: routers}
for name, want := range map[string]int{"r0": 200, "r1": 190, "r19": 10, "r20": 1, "r21": 1} {
if got := vrrpPriority(s, name); got != want {
t.Errorf("priority of %s = %d, want %d", name, got, want)
}
} }
} }
@ -415,7 +521,7 @@ func TestMetaData(t *testing.T) {
} }
} }
func TestRender_AdminMacsAreUniqueAndApartFromCableMacs(t *testing.T) { func TestRender_AdminMacsAreUniqueAndApartFromPortMacs(t *testing.T) {
seen := map[string]string{} seen := map[string]string{}
for name, n := range renderAll(t, twoHypervisors) { for name, n := range renderAll(t, twoHypervisors) {
for _, d := range value(t, n.QEMU, "-device") { for _, d := range value(t, n.QEMU, "-device") {
@ -429,8 +535,8 @@ func TestRender_AdminMacsAreUniqueAndApartFromCableMacs(t *testing.T) {
} }
} }
} }
if len(seen) != 10 { if len(seen) != 8 {
t.Errorf("%d distinct macs, want 10", len(seen)) t.Errorf("%d distinct macs, want 8", len(seen))
} }
} }

View file

@ -15,9 +15,9 @@ images:
url: https://example.invalid/deb.qcow2 url: https://example.invalid/deb.qcow2
sums: https://example.invalid/SHA512SUMS sums: https://example.invalid/SHA512SUMS
segments: segments:
underlay: { switch: sw1, cidr: 192.168.14.0/24 } underlay: { routers: [sw1], cidr: 192.168.14.0/24 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 2, memory: 1024, secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf } sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: rr1.conf } rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: rr1.conf }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: hv1.conf, provision: hv.sh, post_provision: hv-post.sh, files: { agent.yml: two.yml }, services: [agent.service] } hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: hv1.conf, provision: hv.sh, post_provision: hv-post.sh, files: { agent.yml: two.yml }, services: [agent.service] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] } hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
@ -79,16 +79,20 @@ sed -i 's/^bgpd=no/bgpd=yes/' /etc/frr/daemons
func TestRoles_SecondaryAddressFollowsThePrimaryOnTheNode(t *testing.T) { func TestRoles_SecondaryAddressFollowsThePrimaryOnTheNode(t *testing.T) {
doc := network(t, nodeNamed(t, renderRoles(t), "rr1")) doc := network(t, nodeNamed(t, renderRoles(t), "rr1"))
got := iface(t, doc, "underlay").Addresses got := iface(t, doc, "underlay").Addresses
if !reflect.DeepEqual(got, []string{"192.168.14.2/24", "169.254.0.3/28"}) { if !reflect.DeepEqual(got, []string{"192.168.14.3/24", "169.254.0.3/28"}) {
t.Errorf("rr1 underlay addresses = %v", got) t.Errorf("rr1 underlay addresses = %v", got)
} }
} }
func TestRoles_SwitchCarriesItsSecondaryOnTheBridge(t *testing.T) { func TestRoles_RouterCarriesItsSecondaryOnItsSegmentInterface(t *testing.T) {
script := fileAt(t, user(t, nodeNamed(t, renderRoles(t), "sw1")), "/usr/local/sbin/lab-switch").Content sw1 := nodeNamed(t, renderRoles(t), "sw1")
want := "ip addr replace 192.168.14.1/24 dev br-underlay\nip addr replace 169.254.0.1/28 dev br-underlay\nip link set dev br-underlay up\n" under := iface(t, network(t, sw1), "underlay")
if !strings.Contains(script, want) { if !reflect.DeepEqual(under.Addresses, []string{"192.168.14.2/24", "169.254.0.1/28"}) || len(under.Routes) != 0 {
t.Errorf("lab-switch:\n%s\ndoes not contain:\n%s", script, want) t.Errorf("sw1 underlay = %+v", under)
}
script := fileAt(t, user(t, sw1), "/usr/local/sbin/lab-router").Content
if !strings.Contains(script, "ip addr replace 192.168.14.1/24 dev underlay\n") || strings.Contains(script, "169.254.0.1") {
t.Errorf("lab-router:\n%s", script)
} }
} }
@ -156,7 +160,7 @@ func TestRoles_ProvisioningIsOneScriptThatStopsAtTheFirstFailure(t *testing.T) {
env := "LAB_NODE=hv1 LAB_UPLINK=underlay LAB_FILES=/etc/lab/files " env := "LAB_NODE=hv1 LAB_UPLINK=underlay LAB_FILES=/etc/lab/files "
files := "mkdir -p /etc/lab/files /run/lab-files\nmount -o ro LABEL=LABFILES /run/lab-files\ncp -R /run/lab-files/. /etc/lab/files/\numount /run/lab-files\n" files := "mkdir -p /etc/lab/files /run/lab-files\nmount -o ro LABEL=LABFILES /run/lab-files\ncp -R /run/lab-files/. /etc/lab/files/\numount /run/lab-files\n"
for name, steps := range map[string]string{ for name, steps := range map[string]string{
"sw1": "systemctl daemon-reload\nsystemctl enable --now lab-switch.service\n/usr/local/sbin/lab-frr\n", "sw1": "systemctl daemon-reload\nsystemctl enable --now lab-router.service\n/usr/local/sbin/lab-frr\n",
"hv1": files + env + "/usr/local/sbin/lab-node-provision\n/usr/local/sbin/lab-frr\n" + env + "/usr/local/sbin/lab-node-post-provision\n", "hv1": files + env + "/usr/local/sbin/lab-node-provision\n/usr/local/sbin/lab-frr\n" + env + "/usr/local/sbin/lab-node-post-provision\n",
} { } {
cfg := user(t, nodeNamed(t, nodes, name)) cfg := user(t, nodeNamed(t, nodes, name))
@ -238,6 +242,40 @@ func TestRoles_UplinkIsTheInterfaceCarryingTheDefaultRoute(t *testing.T) {
} }
} }
func TestRoles_UplinkSkipsSegmentsWithoutGateway(t *testing.T) {
p := plan(t, withVRRP)
want := map[string]string{
"hv": "LAB_NODE=hv LAB_UPLINK=red LAB_FILES=/etc/lab/files ",
"l2": "LAB_NODE=l2 LAB_UPLINK=pure LAB_FILES=/etc/lab/files ",
"r1": "LAB_NODE=r1 LAB_UPLINK=mgmt0 LAB_FILES=/etc/lab/files ",
"r3": "LAB_NODE=r3 LAB_UPLINK=mgmt0 LAB_FILES=/etc/lab/files ",
}
for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok {
if got := provisionEnv(p, n); got != w {
t.Errorf("%s provision environment = %q, want %q", n.Name, got, w)
}
}
}
}
func TestRoles_UplinkIsGivenToTheProvisionScript(t *testing.T) {
doc := strings.Replace(withVRRP, "segments: [pure, red] }", "segments: [pure, red], provision: hv.sh }", 1)
nodes, err := Render(plan(t, doc), Options{RunDir: "/srv/lab", AuthorizedKeys: []string{labKey}, Provision: map[string]string{"hv": "#!/bin/sh\n"}})
if err != nil {
t.Fatalf("Render: %v", err)
}
for _, n := range nodes {
if n.Name != "hv" {
continue
}
got := fileAt(t, user(t, n), "/usr/local/sbin/lab-provision").Content
if got != "#!/bin/sh\nset -eu\nLAB_NODE=hv LAB_UPLINK=red LAB_FILES=/etc/lab/files /usr/local/sbin/lab-node-provision\n" {
t.Errorf("lab-provision:\n%s", got)
}
}
}
func TestRoles_NodesWithoutProvisionReceiveNoProvisionFiles(t *testing.T) { func TestRoles_NodesWithoutProvisionReceiveNoProvisionFiles(t *testing.T) {
nodes := renderRoles(t) nodes := renderRoles(t)
for _, name := range []string{"sw1", "rr1", "hv2"} { for _, name := range []string{"sw1", "rr1", "hv2"} {
@ -253,8 +291,67 @@ func TestRoles_FRRDaemons(t *testing.T) {
p := plan(t, withRoles) p := plan(t, withRoles)
want := map[string][]string{"sw1": {"bgpd", "bfdd"}, "rr1": {"bgpd", "bfdd"}, "hv1": {"bgpd"}} want := map[string][]string{"sw1": {"bgpd", "bfdd"}, "rr1": {"bgpd", "bfdd"}, "hv1": {"bgpd"}}
for _, n := range p.Nodes { for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(n), w) { if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(p, n), w) {
t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(n), w) t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(p, n), w)
}
}
}
func TestRoles_VRRPAddsVrrpdOnlyToTheRoutersThatRunIt(t *testing.T) {
p := plan(t, withVRRP)
want := map[string][]string{"r1": {"bgpd", "bfdd", "vrrpd"}, "r2": {"bgpd", "bfdd", "vrrpd"}, "r3": {"bgpd", "bfdd"}, "hv": {"bgpd"}}
needs := map[string]bool{"r1": true, "r2": true, "r3": false, "hv": false, "l2": false}
for _, n := range p.Nodes {
if w, ok := want[n.Name]; ok && !reflect.DeepEqual(frrDaemons(p, n), w) {
t.Errorf("%s daemons = %v, want %v", n.Name, frrDaemons(p, n), w)
}
if got := NeedsFRR(p, n); got != needs[n.Name] {
t.Errorf("NeedsFRR(%s) = %v, want %v", n.Name, got, needs[n.Name])
}
}
}
func TestRoles_VRRPRouterWithoutFRRFileGetsAGeneratedConfig(t *testing.T) {
nodes := renderAll(t, withVRRP)
cfg := user(t, nodeNamed(t, nodes, "r1"))
want := "frr defaults traditional\nhostname r1\n!\ninterface red\n vrrp 1 version 3\n vrrp 1 priority 200\n no vrrp 1 preempt\n vrrp 1 ip 10.1.0.1\nexit\n!\n"
if f := fileAt(t, cfg, "/etc/lab/frr.conf"); f.Content != want || f.Permissions != "0640" {
t.Errorf("r1 frr.conf (%s):\n%s\nwant:\n%s", f.Permissions, f.Content, want)
}
vrrpd := "sed -i 's/^vrrpd=no/vrrpd=yes/' /etc/frr/daemons\n"
bfd := "sed -i 's/^bfdd=no/bfdd=yes/' /etc/frr/daemons\n"
tail := "install -o frr -g frr -m 0640 /etc/lab/frr.conf /etc/frr/frr.conf\nsystemctl restart frr\n"
if f := fileAt(t, cfg, "/usr/local/sbin/lab-frr"); f.Content != frrScriptHead+bfd+vrrpd+tail {
t.Errorf("r1 lab-frr:\n%s", f.Content)
}
fileAt(t, cfg, "/usr/share/keyrings/frrouting.gpg")
want = "#!/bin/sh\nset -eu\nsystemctl daemon-reload\nsystemctl enable --now lab-router.service\n/usr/local/sbin/lab-frr\n"
if got := fileAt(t, cfg, "/usr/local/sbin/lab-provision").Content; got != want {
t.Errorf("r1 lab-provision:\n%s\nwant:\n%s", got, want)
}
for _, name := range []string{"r3", "hv", "l2"} {
for _, f := range user(t, nodeNamed(t, nodes, name)).WriteFiles {
if strings.Contains(f.Path, "frr") {
t.Errorf("%s receives %s", name, f.Path)
}
}
}
}
func TestRoles_VRRPStanzaFollowsTheUserFRRConfig(t *testing.T) {
doc := strings.Replace(withVRRP, "segments: [red, pure] }", "segments: [red, pure], frr: r2.conf }", 1)
o := Options{RunDir: "/srv/lab", AuthorizedKeys: []string{labKey}, FRR: map[string]string{"r2": "hostname r2\nrouter bgp 65001\n\n\n"}}
nodes, err := Render(plan(t, doc), o)
if err != nil {
t.Fatalf("Render: %v", err)
}
for _, n := range nodes {
if n.Name != "r2" {
continue
}
want := "hostname r2\nrouter bgp 65001\ninterface red\n vrrp 1 version 3\n vrrp 1 priority 190\n no vrrp 1 preempt\n vrrp 1 ip 10.1.0.1\nexit\n!\n"
if got := fileAt(t, user(t, n), "/etc/lab/frr.conf").Content; got != want {
t.Errorf("r2 frr.conf:\n%s\nwant:\n%s", got, want)
} }
} }
} }
@ -275,7 +372,7 @@ func TestRoles_RefusesAnUnreadFRRConfig(t *testing.T) {
} }
} }
func TestRoles_SwitchLoopbackIsCreatedByTheSwitchScript(t *testing.T) { func TestRoles_RouterLoopbackIsCreatedByTheRouterScript(t *testing.T) {
doc := strings.Replace(withRoles, "secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf", "secondary: { underlay: [169.254.0.1/28] }, loopback: 10.255.254.1/32, frr: sw1.conf", 1) doc := strings.Replace(withRoles, "secondary: { underlay: [169.254.0.1/28] }, frr: sw1.conf", "secondary: { underlay: [169.254.0.1/28] }, loopback: 10.255.254.1/32, frr: sw1.conf", 1)
nodes, err := Render(plan(t, doc), roleOptions()) nodes, err := Render(plan(t, doc), roleOptions())
if err != nil { if err != nil {
@ -287,10 +384,16 @@ func TestRoles_SwitchLoopbackIsCreatedByTheSwitchScript(t *testing.T) {
sw1 = n sw1 = n
} }
} }
script := fileAt(t, user(t, sw1), "/usr/local/sbin/lab-switch").Content cfg := user(t, sw1)
want := "ip link set dev br-underlay up\nip link add lo1 type dummy 2>/dev/null || true\nip addr replace 10.255.254.1/32 dev lo1\nip link set dev lo1 up\nnft -f /etc/lab-switch.nft\n" script := fileAt(t, cfg, "/usr/local/sbin/lab-router").Content
want := "ip addr replace 192.168.14.1/24 dev underlay\nip link add lo1 type dummy 2>/dev/null || true\nip addr replace 10.255.254.1/32 dev lo1\nip link set dev lo1 up\nnft -f /etc/lab-router.nft\n"
if !strings.HasSuffix(script, want) { if !strings.HasSuffix(script, want) {
t.Errorf("lab-switch:\n%s\ndoes not end with:\n%s", script, want) t.Errorf("lab-router:\n%s\ndoes not end with:\n%s", script, want)
}
for _, f := range cfg.WriteFiles {
if strings.HasPrefix(f.Path, "/usr/local/sbin/lab-node") || strings.Contains(f.Path, "lab-node.service") {
t.Errorf("sw1 receives %s", f.Path)
}
} }
} }

View file

@ -12,9 +12,10 @@ import (
) )
const ( const (
SwitchScript = "/usr/local/sbin/lab-switch" RouterScript = "/usr/local/sbin/lab-router"
SwitchNFT = "/etc/lab-switch.nft" RouterNFT = "/etc/lab-router.nft"
SwitchUnit = "/etc/systemd/system/lab-switch.service" RouterUnit = "/etc/systemd/system/lab-router.service"
VRRPPrefix = "vr"
NodeScript = "/usr/local/sbin/lab-node" NodeScript = "/usr/local/sbin/lab-node"
NodeUnit = "/etc/systemd/system/lab-node.service" NodeUnit = "/etc/systemd/system/lab-node.service"
@ -99,14 +100,14 @@ func userData(p *topology.Plan, n topology.NodePlan, o Options) ([]byte, error)
} }
var steps []string var steps []string
switch { switch {
case n.Role == topology.RoleSwitch: case n.Role == topology.RoleRouter:
cfg.Packages = []string{"nftables"} cfg.Packages = []string{"nftables"}
cfg.WriteFiles = []writeFile{ cfg.WriteFiles = []writeFile{
{Path: SwitchScript, Permissions: "0755", Content: switchScript(p, n)}, {Path: RouterScript, Permissions: "0755", Content: routerScript(p, n)},
{Path: SwitchNFT, Permissions: "0644", Content: switchNFT(p, n.Name)}, {Path: RouterNFT, Permissions: "0644", Content: routerNFT(p, n.Name)},
{Path: SwitchUnit, Permissions: "0644", Content: unit("Lab switch: bridges, gateways and NAT", SwitchScript)}, {Path: RouterUnit, Permissions: "0644", Content: unit("Lab router: gateways and NAT", RouterScript)},
} }
steps = append(steps, "systemctl daemon-reload", "systemctl enable --now lab-switch.service") steps = append(steps, "systemctl daemon-reload", "systemctl enable --now lab-router.service")
case n.Loopback.IsValid(): case n.Loopback.IsValid():
cfg.WriteFiles = []writeFile{ cfg.WriteFiles = []writeFile{
{Path: NodeScript, Permissions: "0755", Content: "#!/bin/sh\nset -eu\n" + loopbackLines(n)}, {Path: NodeScript, Permissions: "0755", Content: "#!/bin/sh\nset -eu\n" + loopbackLines(n)},
@ -126,11 +127,11 @@ func userData(p *topology.Plan, n topology.NodePlan, o Options) ([]byte, error)
cfg.WriteFiles = append(cfg.WriteFiles, writeFile{Path: NodeProvision, Permissions: "0755", Encoding: "b64", Content: base64.StdEncoding.EncodeToString([]byte(o.Provision[n.Name]))}) cfg.WriteFiles = append(cfg.WriteFiles, writeFile{Path: NodeProvision, Permissions: "0755", Encoding: "b64", Content: base64.StdEncoding.EncodeToString([]byte(o.Provision[n.Name]))})
steps = append(steps, provisionEnv(p, n)+NodeProvision) steps = append(steps, provisionEnv(p, n)+NodeProvision)
} }
if n.FRR != "" { if NeedsFRR(p, n) {
cfg.WriteFiles = append(cfg.WriteFiles, cfg.WriteFiles = append(cfg.WriteFiles,
writeFile{Path: FRRKey, Permissions: "0644", Encoding: "b64", Content: base64.StdEncoding.EncodeToString(frrKey)}, writeFile{Path: FRRKey, Permissions: "0644", Encoding: "b64", Content: base64.StdEncoding.EncodeToString(frrKey)},
writeFile{Path: FRRConfig, Permissions: "0640", Content: o.FRR[n.Name]}, writeFile{Path: FRRConfig, Permissions: "0640", Content: frrConfig(p, n, o.FRR[n.Name])},
writeFile{Path: FRRScript, Permissions: "0755", Content: frrScript(n)}, writeFile{Path: FRRScript, Permissions: "0755", Content: frrScript(p, n)},
) )
steps = append(steps, FRRScript) steps = append(steps, FRRScript)
} }
@ -156,27 +157,77 @@ func retry(command string) string {
} }
func provisionEnv(p *topology.Plan, n topology.NodePlan) string { func provisionEnv(p *topology.Plan, n topology.NodePlan) string {
uplink := "" port, _ := uplink(p, n.Name)
if cables := nodeCables(p, n.Name); len(cables) > 0 { up := port.Interface
uplink = cables[0].NodeInterface if n.Role == topology.RoleRouter {
up = AdminInterface
} }
return fmt.Sprintf("LAB_NODE=%s LAB_UPLINK=%s LAB_FILES=%s ", n.Name, uplink, FilesDir) return fmt.Sprintf("LAB_NODE=%s LAB_UPLINK=%s LAB_FILES=%s ", n.Name, up, FilesDir)
} }
func frrDaemons(n topology.NodePlan) []string { func vrrpSegments(p *topology.Plan, name string) []topology.SegmentPlan {
if n.Role == topology.RoleHost { var out []topology.SegmentPlan
return []string{"bgpd"} for _, s := range routedSegments(p, name) {
if s.VRID != 0 {
out = append(out, s)
}
} }
return []string{"bgpd", "bfdd"} return out
} }
func frrScript(n topology.NodePlan) string { func NeedsFRR(p *topology.Plan, n topology.NodePlan) bool {
return n.FRR != "" || len(vrrpSegments(p, n.Name)) > 0
}
func frrDaemons(p *topology.Plan, n topology.NodePlan) []string {
daemons := []string{"bgpd"}
if n.Role != topology.RoleHost {
daemons = append(daemons, "bfdd")
}
if len(vrrpSegments(p, n.Name)) > 0 {
daemons = append(daemons, "vrrpd")
}
return daemons
}
func frrConfig(p *topology.Plan, n topology.NodePlan, given string) string {
segments := vrrpSegments(p, n.Name)
if len(segments) == 0 {
return given
}
var b strings.Builder
if given == "" {
fmt.Fprintf(&b, "frr defaults traditional\nhostname %s\n!\n", n.Name)
} else {
b.WriteString(strings.TrimRight(given, "\n") + "\n")
}
for _, s := range segments {
fmt.Fprintf(&b, "interface %s\n vrrp %d version 3\n vrrp %d priority %d\n no vrrp %d preempt\n vrrp %d ip %s\nexit\n!\n",
s.Name, s.VRID, s.VRID, vrrpPriority(s, n.Name), s.VRID, s.VRID, s.Gateway)
}
return b.String()
}
func vrrpPriority(s topology.SegmentPlan, name string) int {
for i, r := range s.Routers {
if r == name {
return max(200-10*i, 1)
}
}
return 1
}
func vrrpMAC(vrid int) string {
return fmt.Sprintf("00:00:5e:00:01:%02x", vrid)
}
func frrScript(p *topology.Plan, n topology.NodePlan) string {
var b strings.Builder var b strings.Builder
b.WriteString("#!/bin/sh\nset -eu\nexport DEBIAN_FRONTEND=noninteractive\n. /etc/os-release\n") b.WriteString("#!/bin/sh\nset -eu\nexport DEBIAN_FRONTEND=noninteractive\n. /etc/os-release\n")
fmt.Fprintf(&b, "echo \"deb [signed-by=%s] %s ${VERSION_CODENAME} %s\" > /etc/apt/sources.list.d/frr.list\n", FRRKey, FRRRepo, FRRSuite) fmt.Fprintf(&b, "echo \"deb [signed-by=%s] %s ${VERSION_CODENAME} %s\" > /etc/apt/sources.list.d/frr.list\n", FRRKey, FRRRepo, FRRSuite)
b.WriteString(retry("apt-get update -qq --error-on=any")) b.WriteString(retry("apt-get update -qq --error-on=any"))
fmt.Fprintf(&b, "apt-get install -y -qq --no-install-recommends %s\n", FRRPackages) fmt.Fprintf(&b, "apt-get install -y -qq --no-install-recommends %s\n", FRRPackages)
for _, d := range frrDaemons(n) { for _, d := range frrDaemons(p, n) {
fmt.Fprintf(&b, "sed -i 's/^%s=no/%s=yes/' /etc/frr/daemons\n", d, d) fmt.Fprintf(&b, "sed -i 's/^%s=no/%s=yes/' /etc/frr/daemons\n", d, d)
} }
fmt.Fprintf(&b, "install -o frr -g frr -m 0640 %s /etc/frr/frr.conf\n", FRRConfig) fmt.Fprintf(&b, "install -o frr -g frr -m 0640 %s /etc/frr/frr.conf\n", FRRConfig)
@ -199,77 +250,62 @@ func networkConfig(p *topology.Plan, n topology.NodePlan, index int) ([]byte, er
SetName: AdminInterface, SetName: AdminInterface,
Addresses: []string{AdminAddress}, Addresses: []string{AdminAddress},
} }
if n.Role == topology.RoleSwitch { router := n.Role == topology.RoleRouter
if router {
admin.Routes = []route{{To: "0.0.0.0/0", Via: AdminGateway}} admin.Routes = []route{{To: "0.0.0.0/0", Via: AdminGateway}}
admin.Nameservers = &nameservers{Addresses: []string{AdminDNS}} admin.Nameservers = &nameservers{Addresses: []string{AdminDNS}}
for _, c := range switchCables(p, n.Name) {
doc.Ethernets[c.SwitchInterface] = ethernet{
Match: match{MACAddress: c.SwitchMAC.String()},
SetName: c.SwitchInterface,
MTU: c.MTU,
}
}
} }
doc.Ethernets[AdminInterface] = admin doc.Ethernets[AdminInterface] = admin
for i, c := range nodeCables(p, n.Name) { up, routed := uplink(p, n.Name)
for _, port := range nodePorts(p, n.Name) {
e := ethernet{ e := ethernet{
Match: match{MACAddress: c.NodeMAC.String()}, Match: match{MACAddress: port.MAC.String()},
SetName: c.NodeInterface, SetName: port.Interface,
MTU: c.MTU, MTU: port.MTU,
Addresses: []string{c.NodeAddress.String()}, Addresses: []string{port.Address.String()},
} }
for _, prefix := range n.Secondary[c.Segment] { for _, prefix := range n.Secondary[port.Segment] {
e.Addresses = append(e.Addresses, prefix.String()) e.Addresses = append(e.Addresses, prefix.String())
} }
if i == 0 { if !router && routed && port.Segment == up.Segment {
e.Routes = []route{{To: "0.0.0.0/0", Via: gatewayOf(p, c.Segment)}} e.Routes = []route{{To: "0.0.0.0/0", Via: segmentNamed(p, port.Segment).Gateway.String()}}
e.Nameservers = &nameservers{Addresses: Nameservers} e.Nameservers = &nameservers{Addresses: Nameservers}
} }
doc.Ethernets[c.NodeInterface] = e doc.Ethernets[port.Interface] = e
} }
return yaml.Marshal(doc) return yaml.Marshal(doc)
} }
func gatewayOf(p *topology.Plan, segment string) string { func routerScript(p *topology.Plan, n topology.NodePlan) string {
for _, s := range p.Segments {
if s.Name == segment {
return s.Gateway.String()
}
}
return ""
}
func switchScript(p *topology.Plan, n topology.NodePlan) string {
name := n.Name
var b strings.Builder var b strings.Builder
b.WriteString("#!/bin/sh\nset -eu\nsysctl -qw net.ipv4.ip_forward=1\n") b.WriteString("#!/bin/sh\nset -eu\nsysctl -qw net.ipv4.ip_forward=1\n")
for _, s := range switchSegments(p, name) { for _, s := range routedSegments(p, n.Name) {
fmt.Fprintf(&b, "ip link add %s type bridge stp_state 0 2>/dev/null || true\n", s.Bridge) gateway := fmt.Sprintf("%s/%d", s.Gateway, s.Network.Bits())
for _, c := range switchCables(p, name) { if s.VRID == 0 {
if c.Segment != s.Name { fmt.Fprintf(&b, "ip addr replace %s dev %s\n", gateway, s.Name)
continue continue
}
fmt.Fprintf(&b, "ip link set dev %s master %s\n", c.SwitchInterface, s.Bridge)
fmt.Fprintf(&b, "ip link set dev %s up\n", c.SwitchInterface)
} }
fmt.Fprintf(&b, "ip link set dev %s mtu %d\n", s.Bridge, s.MTU) vrrp := VRRPPrefix + s.Name
fmt.Fprintf(&b, "ip addr replace %s/%d dev %s\n", s.Gateway, s.Network.Bits(), s.Bridge) fmt.Fprintf(&b, "sysctl -qw net.ipv4.conf.%s.arp_ignore=1 net.ipv4.conf.%s.arp_announce=2\n", s.Name, s.Name)
for _, prefix := range n.Secondary[s.Name] { fmt.Fprintf(&b, "ip link show dev %s >/dev/null 2>&1 || { ip link add %s link %s type macvlan mode bridge && ip link set dev %s protodown on; }\n", vrrp, vrrp, s.Name, vrrp)
fmt.Fprintf(&b, "ip addr replace %s dev %s\n", prefix, s.Bridge) fmt.Fprintf(&b, "ip link set dev %s addrgenmode random address %s\n", vrrp, vrrpMAC(s.VRID))
} fmt.Fprintf(&b, "ip addr replace %s dev %s\n", gateway, vrrp)
fmt.Fprintf(&b, "ip link set dev %s up\n", s.Bridge) fmt.Fprintf(&b, "ip link set dev %s up\n", vrrp)
} }
b.WriteString(loopbackLines(n)) b.WriteString(loopbackLines(n))
fmt.Fprintf(&b, "nft -f %s\n", SwitchNFT) fmt.Fprintf(&b, "nft -f %s\n", RouterNFT)
return b.String() return b.String()
} }
func switchNFT(p *topology.Plan, name string) string { func routerNFT(p *topology.Plan, name string) string {
var networks []string var networks []string
for _, s := range switchSegments(p, name) { for _, s := range routedSegments(p, name) {
networks = append(networks, s.Network.String()) networks = append(networks, s.Network.String())
} }
if len(networks) == 0 {
return "add table ip lab_nat\ndelete table ip lab_nat\n"
}
return fmt.Sprintf(`add table ip lab_nat return fmt.Sprintf(`add table ip lab_nat
delete table ip lab_nat delete table ip lab_nat
table ip lab_nat { table ip lab_nat {

View file

@ -12,7 +12,7 @@ import (
func (p *Plan) Write(w io.Writer) error { func (p *Plan) Write(w io.Writer) error {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0) tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintf(tw, "lab %s: nodes %d, segments %d, cables %d\n", p.Name, len(p.Nodes), len(p.Segments), len(p.Cables)) fmt.Fprintf(tw, "lab %s: nodes %d, segments %d, ports %d\n", p.Name, len(p.Nodes), len(p.Segments), len(p.Ports))
fmt.Fprintf(tw, "\nnodes\n") fmt.Fprintf(tw, "\nnodes\n")
fmt.Fprintf(tw, " name\trole\timage\tcpus\tmemory\tssh\n") fmt.Fprintf(tw, " name\trole\timage\tcpus\tmemory\tssh\n")
@ -37,21 +37,29 @@ func (p *Plan) Write(w io.Writer) error {
} }
for _, s := range p.Segments { for _, s := range p.Segments {
fmt.Fprintf(tw, "\nsegment %s: %s, mtu %d, switch %s, bridge %s, gateway %s\n", fmt.Fprintf(tw, "\nsegment %s: %s, mtu %d, bridge %s, %s\n", s.Name, s.Network, s.MTU, s.Bridge, gateway(s))
s.Name, s.Network, s.MTU, s.Switch, s.Bridge, s.Gateway) fmt.Fprintf(tw, " node\tinterface\taddress\tmac\ttap\n")
fmt.Fprintf(tw, " node\tinterface\taddress\tmac\tudp\t\tswitch port\tmac\tudp\n") for _, port := range p.Ports {
for _, c := range p.Cables { if port.Segment != s.Name {
if c.Segment != s.Name {
continue continue
} }
fmt.Fprintf(tw, " %s\t%s\t%s\t%s\t%d\t<->\t%s %s\t%s\t%d\n", fmt.Fprintf(tw, " %s\t%s\t%s\t%s\t%s\n", port.Node, port.Interface, port.Address, port.MAC, port.Tap)
c.Node, c.NodeInterface, c.NodeAddress, c.NodeMAC, c.NodePort,
c.Switch, c.SwitchInterface, c.SwitchMAC, c.SwitchPort)
} }
} }
return tw.Flush() return tw.Flush()
} }
func gateway(s SegmentPlan) string {
switch {
case len(s.Routers) == 0:
return "no gateway"
case s.VRID == 0:
return fmt.Sprintf("gateway %s on %s", s.Gateway, s.Routers[0])
default:
return fmt.Sprintf("gateway %s by vrrp %d on %s", s.Gateway, s.VRID, strings.Join(s.Routers, ", "))
}
}
func loopback(n NodePlan) string { func loopback(n NodePlan) string {
if !n.Loopback.IsValid() { if !n.Loopback.IsValid() {
return "" return ""

View file

@ -9,8 +9,10 @@ import (
) )
const ( const (
SSHBasePort = 2200 SSHBasePort = 2200
CableBasePort = 20000
BridgePrefix = "lb-"
TapPrefix = "lt"
maxNodes = 1000 maxNodes = 1000
maxSegments = 256 maxSegments = 256
@ -21,15 +23,16 @@ type Plan struct {
Images []Image Images []Image
Segments []SegmentPlan Segments []SegmentPlan
Nodes []NodePlan Nodes []NodePlan
Cables []Cable Ports []Port
} }
type SegmentPlan struct { type SegmentPlan struct {
Name string Name string
Switch string
Bridge string Bridge string
Network netip.Prefix Network netip.Prefix
Gateway netip.Addr Gateway netip.Addr
Routers []string
VRID int
MTU int MTU int
} }
@ -49,20 +52,14 @@ type NodePlan struct {
Services []string Services []string
} }
type Cable struct { type Port struct {
Segment string Segment string
MTU int MTU int
Node string
Node string Interface string
NodeInterface string MAC net.HardwareAddr
NodeMAC net.HardwareAddr Address netip.Prefix
NodeAddress netip.Prefix Tap string
NodePort int
Switch string
SwitchInterface string
SwitchMAC net.HardwareAddr
SwitchPort int
} }
func Compute(t *Topology) (*Plan, error) { func Compute(t *Topology) (*Plan, error) {
@ -102,17 +99,22 @@ func Compute(t *Topology) (*Plan, error) {
var errs []error var errs []error
for segIndex, s := range t.Segments { for segIndex, s := range t.Segments {
network := netip.MustParsePrefix(s.CIDR) network := netip.MustParsePrefix(s.CIDR)
gateway := network.Addr().Next() segment := SegmentPlan{
p.Segments = append(p.Segments, SegmentPlan{
Name: s.Name, Name: s.Name,
Switch: s.Switch, Bridge: BridgePrefix + s.Name,
Bridge: "br-" + s.Name,
Network: network, Network: network,
Gateway: gateway, Routers: append([]string(nil), s.Routers...),
MTU: s.MTU, MTU: s.MTU,
}) }
if len(s.Routers) > 0 {
segment.Gateway = network.Addr().Next()
}
if len(s.Routers) > 1 {
segment.VRID = segIndex + 1
}
p.Segments = append(p.Segments, segment)
addresses, err := allocate(t, s, network, gateway) addresses, err := allocate(t, s, network, segment.Gateway)
if err != nil { if err != nil {
errs = append(errs, err) errs = append(errs, err)
continue continue
@ -123,26 +125,18 @@ func Compute(t *Topology) (*Plan, error) {
if !ok { if !ok {
continue continue
} }
cable := len(p.Cables) p.Ports = append(p.Ports, Port{
p.Cables = append(p.Cables, Cable{ Segment: s.Name,
Segment: s.Name, MTU: s.MTU,
MTU: s.MTU, Node: n.Name,
Node: n.Name, Interface: s.Name,
NodeInterface: s.Name, MAC: mac(nodeIndex, segIndex),
NodeMAC: mac(nodeIndex, segIndex, 0), Address: netip.PrefixFrom(address, network.Bits()),
NodeAddress: netip.PrefixFrom(address, network.Bits()), Tap: TapPrefix + strconv.Itoa(len(p.Ports)),
NodePort: CableBasePort + 2*cable,
Switch: s.Switch,
SwitchInterface: "p" + strconv.Itoa(cable),
SwitchMAC: mac(nodeIndex, segIndex, 1),
SwitchPort: CableBasePort + 2*cable + 1,
}) })
} }
} }
if last := CableBasePort + 2*len(p.Cables) - 1; last > 65535 {
errs = append(errs, fmt.Errorf("cables: %d cables need udp ports up to %d, beyond 65535", len(p.Cables), last))
}
if len(errs) > 0 { if len(errs) > 0 {
return nil, errors.Join(errs...) return nil, errors.Join(errs...)
} }
@ -151,7 +145,12 @@ func Compute(t *Topology) (*Plan, error) {
func allocate(t *Topology, s Segment, network netip.Prefix, gateway netip.Addr) (map[string]netip.Addr, error) { func allocate(t *Topology, s Segment, network netip.Prefix, gateway netip.Addr) (map[string]netip.Addr, error) {
broadcast := lastAddr(network) broadcast := lastAddr(network)
used := map[netip.Addr]string{gateway: s.Switch + " (gateway)"} used := map[netip.Addr]string{}
first := network.Addr().Next()
if gateway.IsValid() {
used[gateway] = "the gateway"
first = gateway.Next()
}
result := map[string]netip.Addr{} result := map[string]netip.Addr{}
var errs []error var errs []error
@ -179,7 +178,7 @@ func allocate(t *Topology, s Segment, network netip.Prefix, gateway netip.Addr)
} }
} }
next := gateway.Next() next := first
for _, name := range auto { for _, name := range auto {
for next != broadcast && used[next] != "" { for next != broadcast && used[next] != "" {
next = next.Next() next = next.Next()
@ -206,8 +205,8 @@ func lastAddr(network netip.Prefix) netip.Addr {
return netip.AddrFrom4([4]byte{byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)}) return netip.AddrFrom4([4]byte{byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)})
} }
func mac(node, segment, side int) net.HardwareAddr { func mac(node, segment int) net.HardwareAddr {
return net.HardwareAddr{0x02, 0x4c, byte(node >> 8), byte(node), byte(segment), byte(side)} return net.HardwareAddr{0x02, 0x4c, byte(node >> 8), byte(node), byte(segment), 0x00}
} }
func contains(list []string, value string) bool { func contains(list []string, value string) bool {

View file

@ -27,15 +27,26 @@ func computeError(t *testing.T, doc string) string {
return err.Error() return err.Error()
} }
func cableOf(t *testing.T, p *Plan, node, segment string) Cable { func portOf(t *testing.T, p *Plan, node, segment string) Port {
t.Helper() t.Helper()
for _, c := range p.Cables { for _, port := range p.Ports {
if c.Node == node && c.Segment == segment { if port.Node == node && port.Segment == segment {
return c return port
} }
} }
t.Fatalf("no cable for %s on %s", node, segment) t.Fatalf("no port for %s on %s", node, segment)
return Cable{} return Port{}
}
func segmentOf(t *testing.T, p *Plan, name string) SegmentPlan {
t.Helper()
for _, s := range p.Segments {
if s.Name == name {
return s
}
}
t.Fatalf("no segment %s", name)
return SegmentPlan{}
} }
func nodeOf(t *testing.T, p *Plan, name string) NodePlan { func nodeOf(t *testing.T, p *Plan, name string) NodePlan {
@ -51,12 +62,12 @@ func nodeOf(t *testing.T, p *Plan, name string) NodePlan {
const twoHypervisors = header + ` const twoHypervisors = header + `
segments: segments:
underlay: { switch: sw1, cidr: 10.250.0.0/24, mtu: 9000 } underlay: { routers: [sw1], cidr: 10.250.0.0/24, mtu: 9000 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 2, memory: 1024 } sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay] }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] } rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay] }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] } hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] } hv2: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay] }
` `
func TestCompute_TwoHypervisorsPlan(t *testing.T) { func TestCompute_TwoHypervisorsPlan(t *testing.T) {
@ -66,34 +77,34 @@ func TestCompute_TwoHypervisorsPlan(t *testing.T) {
t.Fatalf("%d segments, want 1", len(p.Segments)) t.Fatalf("%d segments, want 1", len(p.Segments))
} }
s := p.Segments[0] s := p.Segments[0]
if s.Bridge != "br-underlay" || s.Gateway != netip.MustParseAddr("10.250.0.1") || s.MTU != 9000 || s.Switch != "sw1" { if s.Bridge != "lb-underlay" || s.Gateway != netip.MustParseAddr("10.250.0.1") || s.MTU != 9000 || !reflect.DeepEqual(s.Routers, []string{"sw1"}) || s.VRID != 0 {
t.Errorf("segment = %+v", s) t.Errorf("segment = %+v", s)
} }
want := []struct { want := []struct {
node, address, nodeMAC, switchMAC, switchIface string node, address, mac, tap string
nodePort, switchPort int
}{ }{
{"rr1", "10.250.0.2/24", "02:4c:00:01:00:00", "02:4c:00:01:00:01", "p0", 20000, 20001}, {"sw1", "10.250.0.2/24", "02:4c:00:00:00:00", "lt0"},
{"hv1", "10.250.0.3/24", "02:4c:00:02:00:00", "02:4c:00:02:00:01", "p1", 20002, 20003}, {"rr1", "10.250.0.3/24", "02:4c:00:01:00:00", "lt1"},
{"hv2", "10.250.0.4/24", "02:4c:00:03:00:00", "02:4c:00:03:00:01", "p2", 20004, 20005}, {"hv1", "10.250.0.4/24", "02:4c:00:02:00:00", "lt2"},
{"hv2", "10.250.0.5/24", "02:4c:00:03:00:00", "lt3"},
} }
if len(p.Cables) != len(want) { if len(p.Ports) != len(want) {
t.Fatalf("%d cables, want %d", len(p.Cables), len(want)) t.Fatalf("%d ports, want %d", len(p.Ports), len(want))
} }
for _, w := range want { for _, w := range want {
c := cableOf(t, p, w.node, "underlay") port := portOf(t, p, w.node, "underlay")
if c.NodeAddress.String() != w.address { if port.Address.String() != w.address {
t.Errorf("%s address = %s, want %s", w.node, c.NodeAddress, w.address) t.Errorf("%s address = %s, want %s", w.node, port.Address, w.address)
} }
if c.NodeMAC.String() != w.nodeMAC || c.SwitchMAC.String() != w.switchMAC { if port.MAC.String() != w.mac {
t.Errorf("%s macs = %s / %s, want %s / %s", w.node, c.NodeMAC, c.SwitchMAC, w.nodeMAC, w.switchMAC) t.Errorf("%s mac = %s, want %s", w.node, port.MAC, w.mac)
} }
if c.NodePort != w.nodePort || c.SwitchPort != w.switchPort { if port.Tap != w.tap {
t.Errorf("%s ports = %d / %d, want %d / %d", w.node, c.NodePort, c.SwitchPort, w.nodePort, w.switchPort) t.Errorf("%s tap = %s, want %s", w.node, port.Tap, w.tap)
} }
if c.NodeInterface != "underlay" || c.SwitchInterface != w.switchIface || c.Switch != "sw1" || c.MTU != 9000 { if port.Interface != "underlay" || port.MTU != 9000 {
t.Errorf("%s cable = %+v", w.node, c) t.Errorf("%s port = %+v", w.node, port)
} }
} }
@ -127,104 +138,206 @@ func TestCompute_IsStableAcrossRuns(t *testing.T) {
func TestCompute_AddressesFollowDeclarationOrderNotNames(t *testing.T) { func TestCompute_AddressesFollowDeclarationOrderNotNames(t *testing.T) {
p := compute(t, header+` p := compute(t, header+`
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/24 } under: { routers: [sw], cidr: 10.0.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 }
zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] } zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
alfa: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] } alfa: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
`) `)
if got := cableOf(t, p, "zeta", "under").NodeAddress.String(); got != "10.0.0.2/24" { for node, want := range map[string]string{"zeta": "10.0.0.2/24", "sw": "10.0.0.3/24", "alfa": "10.0.0.4/24"} {
t.Errorf("zeta = %s, want 10.0.0.2/24", got) if got := portOf(t, p, node, "under").Address.String(); got != want {
} t.Errorf("%s = %s, want %s", node, got, want)
if got := cableOf(t, p, "alfa", "under").NodeAddress.String(); got != "10.0.0.3/24" { }
t.Errorf("alfa = %s, want 10.0.0.3/24", got)
} }
} }
func TestCompute_ExplicitAddressIsKeptAndSkippedByAllocation(t *testing.T) { func TestCompute_ExplicitAddressIsKeptAndSkippedByAllocation(t *testing.T) {
p := compute(t, header+` p := compute(t, header+`
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/24 } under: { routers: [sw], cidr: 10.0.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.254} }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] } a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.3} } b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.3} }
c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] } c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
`) `)
for node, want := range map[string]string{"a": "10.0.0.2/24", "b": "10.0.0.3/24", "c": "10.0.0.4/24"} { for node, want := range map[string]string{"sw": "10.0.0.254/24", "a": "10.0.0.2/24", "b": "10.0.0.3/24", "c": "10.0.0.4/24"} {
if got := cableOf(t, p, node, "under").NodeAddress.String(); got != want { if got := portOf(t, p, node, "under").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want) t.Errorf("%s = %s, want %s", node, got, want)
} }
} }
} }
func TestCompute_TwoSegmentsCableOrderMacsAndPorts(t *testing.T) { func TestCompute_SegmentWithoutRouterHasNoGateway(t *testing.T) {
p := compute(t, header+` p := compute(t, header+`
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } l2: { cidr: 10.3.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24, mtu: 1500 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] } b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red] }
`) `)
got := make([]string, 0, len(p.Cables)) s := segmentOf(t, p, "l2")
for _, c := range p.Cables { if s.Gateway.IsValid() || s.VRID != 0 || len(s.Routers) != 0 || s.Bridge != "lb-l2" {
got = append(got, c.Segment+"/"+c.Node+"/"+c.SwitchInterface) t.Errorf("segment = %+v, want no gateway", s)
} }
if strings.Join(got, ",") != "red/hv/p0,red/rr/p1,blue/hv/p2" { for node, want := range map[string]string{"a": "10.3.0.1/24", "b": "10.3.0.2/24"} {
t.Errorf("cable order = %v", got) if got := portOf(t, p, node, "l2").Address.String(); got != want {
} t.Errorf("%s = %s, want %s", node, got, want)
blue := cableOf(t, p, "hv", "blue") }
if blue.NodeMAC.String() != "02:4c:00:01:01:00" || blue.NodePort != 20004 || blue.MTU != 1500 || blue.NodeAddress.String() != "10.2.0.2/24" {
t.Errorf("hv on blue = %+v", blue)
}
if red := cableOf(t, p, "hv", "red"); red.NodeMAC.String() != "02:4c:00:01:00:00" {
t.Errorf("hv on red mac = %s", red.NodeMAC)
} }
} }
func TestCompute_EveryMacAndPortIsUnique(t *testing.T) { func TestCompute_SegmentWithoutRouterAcceptsTheFirstAddress(t *testing.T) {
p := compute(t, header+` p := compute(t, header+`
segments: segments:
a: { switch: sw, cidr: 10.1.0.0/24 } l2: { cidr: 10.3.0.0/24 }
b: { switch: sw, cidr: 10.2.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2], addresses: {l2: 10.3.0.1} }
`)
for node, want := range map[string]string{"a": "10.3.0.2/24", "b": "10.3.0.1/24"} {
if got := portOf(t, p, node, "l2").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithOneRouter(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
nodes:
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red] }
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
`)
s := segmentOf(t, p, "red")
if s.Gateway != netip.MustParseAddr("10.1.0.1") || s.VRID != 0 || !reflect.DeepEqual(s.Routers, []string{"r1"}) {
t.Errorf("segment = %+v", s)
}
for node, want := range map[string]string{"hv": "10.1.0.2/24", "r1": "10.1.0.3/24"} {
if got := portOf(t, p, node, "red").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_SegmentWithTwoRoutersUsesVRRP(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
blue: { routers: [r2, r1], cidr: 10.2.0.0/24 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue], addresses: {blue: 10.2.0.250} }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue, pure] }
`)
blue := segmentOf(t, p, "blue")
if blue.VRID != 2 || blue.Gateway != netip.MustParseAddr("10.2.0.1") || !reflect.DeepEqual(blue.Routers, []string{"r2", "r1"}) {
t.Errorf("blue = %+v", blue)
}
if red := segmentOf(t, p, "red"); red.VRID != 0 {
t.Errorf("red vrid = %d, want 0", red.VRID)
}
if pure := segmentOf(t, p, "pure"); pure.VRID != 0 || pure.Gateway.IsValid() {
t.Errorf("pure = %+v", pure)
}
for node, want := range map[string]string{"r1": "10.2.0.2/24", "r2": "10.2.0.250/24", "hv": "10.2.0.3/24"} {
if got := portOf(t, p, node, "blue").Address.String(); got != want {
t.Errorf("%s = %s, want %s", node, got, want)
}
}
}
func TestCompute_TwoSegmentsPortOrderMacsAndTaps(t *testing.T) {
p := compute(t, header+`
segments:
red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { cidr: 10.2.0.0/24, mtu: 1500 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, red] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
`)
got := make([]string, 0, len(p.Ports))
for _, port := range p.Ports {
got = append(got, port.Segment+"/"+port.Node+"/"+port.Tap)
}
if strings.Join(got, ",") != "red/sw/lt0,red/hv/lt1,red/rr/lt2,blue/hv/lt3,blue/rr/lt4" {
t.Errorf("port order = %v", got)
}
blue := portOf(t, p, "hv", "blue")
if blue.MAC.String() != "02:4c:00:01:01:00" || blue.Tap != "lt3" || blue.MTU != 1500 || blue.Interface != "blue" || blue.Address.String() != "10.2.0.1/24" {
t.Errorf("hv on blue = %+v", blue)
}
if red := portOf(t, p, "hv", "red"); red.MAC.String() != "02:4c:00:01:00:00" || red.Address.String() != "10.1.0.3/24" {
t.Errorf("hv on red = %+v", red)
}
if rr := portOf(t, p, "rr", "blue"); rr.MAC.String() != "02:4c:00:02:01:00" || rr.Address.String() != "10.2.0.2/24" {
t.Errorf("rr on blue = %+v", rr)
}
}
func TestCompute_MacEncodesNodeAndSegmentRanks(t *testing.T) {
p := compute(t, generated(3, 300))
if got := portOf(t, p, "n299", "s2").MAC.String(); got != "02:4c:01:2b:02:00" {
t.Errorf("n299 on s2 mac = %s, want 02:4c:01:2b:02:00", got)
}
}
func TestCompute_EveryMacTapAndPortIsUnique(t *testing.T) {
p := compute(t, header+`
segments:
a: { routers: [sw], cidr: 10.1.0.0/24 }
b: { routers: [sw], cidr: 10.2.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [a, b] }
n1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] } n1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] }
n2: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] } n2: { role: rr, image: deb, cpus: 1, memory: 512, segments: [a, b] }
n3: { role: host, image: deb, cpus: 1, memory: 512, segments: [b, a] } n3: { role: host, image: deb, cpus: 1, memory: 512, segments: [b, a] }
`) `)
macs := map[string]bool{} macs := map[string]bool{}
taps := map[string]bool{}
ports := map[int]bool{} ports := map[int]bool{}
for _, n := range p.Nodes { for _, n := range p.Nodes {
if ports[n.SSHPort] {
t.Errorf("ssh port %d used twice", n.SSHPort)
}
ports[n.SSHPort] = true ports[n.SSHPort] = true
} }
for _, c := range p.Cables { for _, port := range p.Ports {
for _, m := range []string{c.NodeMAC.String(), c.SwitchMAC.String()} { if macs[port.MAC.String()] {
if macs[m] { t.Errorf("mac %s used twice", port.MAC)
t.Errorf("mac %s used twice", m)
}
macs[m] = true
} }
for _, port := range []int{c.NodePort, c.SwitchPort} { macs[port.MAC.String()] = true
if ports[port] { if taps[port.Tap] {
t.Errorf("port %d used twice", port) t.Errorf("tap %s used twice", port.Tap)
}
ports[port] = true
} }
taps[port.Tap] = true
} }
if len(macs) != 12 { if len(macs) != 8 || len(taps) != 8 {
t.Errorf("%d distinct macs, want 12", len(macs)) t.Errorf("%d distinct macs and %d distinct taps, want 8", len(macs), len(taps))
} }
} }
func TestCompute_MacsAreLocallyAdministeredUnicast(t *testing.T) { func TestCompute_MacsAreLocallyAdministeredUnicast(t *testing.T) {
for _, c := range compute(t, twoHypervisors).Cables { for _, port := range compute(t, twoHypervisors).Ports {
for _, m := range []string{c.NodeMAC.String(), c.SwitchMAC.String()} { if m := port.MAC.String(); !strings.HasPrefix(m, "02:") {
if !strings.HasPrefix(m, "02:") { t.Errorf("mac %s is not locally administered unicast", m)
t.Errorf("mac %s is not locally administered unicast", m) }
} }
}
func TestCompute_InterfaceAndTapNamesFitInIFNAMSIZ(t *testing.T) {
p := compute(t, generated(23, 1000))
for _, port := range p.Ports {
if len(port.Tap) > 15 || len(port.Interface) > 15 {
t.Fatalf("port %+v has a name longer than 15", port)
}
}
for _, s := range p.Segments {
if len(s.Bridge) > 15 {
t.Fatalf("bridge %s is longer than 15", s.Bridge)
} }
} }
} }
@ -233,9 +346,9 @@ func TestCompute_AddressRejections(t *testing.T) {
base := func(addresses string) string { base := func(addresses string) string {
return header + ` return header + `
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/29 } under: { routers: [sw], cidr: 10.0.0.0/29 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under]` + addresses + ` } a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under]` + addresses + ` }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.5} } b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.5} }
` `
@ -244,7 +357,7 @@ nodes:
{"outside", ", addresses: {under: 10.0.1.2}", "node a: address 10.0.1.2 is outside segment under"}, {"outside", ", addresses: {under: 10.0.1.2}", "node a: address 10.0.1.2 is outside segment under"},
{"network", ", addresses: {under: 10.0.0.0}", "node a: address 10.0.0.0 is the network or broadcast"}, {"network", ", addresses: {under: 10.0.0.0}", "node a: address 10.0.0.0 is the network or broadcast"},
{"broadcast", ", addresses: {under: 10.0.0.7}", "node a: address 10.0.0.7 is the network or broadcast"}, {"broadcast", ", addresses: {under: 10.0.0.7}", "node a: address 10.0.0.7 is the network or broadcast"},
{"gateway", ", addresses: {under: 10.0.0.1}", "node a: address 10.0.0.1 on segment under is already taken by sw (gateway)"}, {"gateway", ", addresses: {under: 10.0.0.1}", "node a: address 10.0.0.1 on segment under is already taken by the gateway"},
{"duplicate", ", addresses: {under: 10.0.0.5}", "node b: address 10.0.0.5 on segment under is already taken by a"}, {"duplicate", ", addresses: {under: 10.0.0.5}", "node b: address 10.0.0.5 on segment under is already taken by a"},
} }
for _, c := range cases { for _, c := range cases {
@ -254,37 +367,58 @@ nodes:
} }
} }
func TestCompute_RouterAddressCannotBeTheGateway(t *testing.T) {
got := computeError(t, header+`
segments:
under: { routers: [sw], cidr: 10.0.0.0/24 }
nodes:
sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0.1} }
`)
requireContains(t, got, "node sw: address 10.0.0.1 on segment under is already taken by the gateway")
}
func TestCompute_ReportsExhaustedSegment(t *testing.T) { func TestCompute_ReportsExhaustedSegment(t *testing.T) {
got := computeError(t, header+` got := computeError(t, header+`
segments: segments:
tiny: { switch: sw, cidr: 10.0.0.0/30 } tiny: { routers: [sw], cidr: 10.0.0.0/30 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [tiny] }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] } a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`) `)
requireContains(t, got, "segment tiny: no address left in 10.0.0.0/30 for node b") requireContains(t, got, "segment tiny: no address left in 10.0.0.0/30 for node a")
} }
func TestCompute_FillsSegmentExactly(t *testing.T) { func TestCompute_FillsSegmentExactly(t *testing.T) {
p := compute(t, header+` p := compute(t, header+`
segments: segments:
tiny: { switch: sw, cidr: 10.0.0.0/29 } tiny: { routers: [sw], cidr: 10.0.0.0/29 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [tiny] }
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] } a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] } b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] } c: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] } d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
e: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`) `)
if got := cableOf(t, p, "e", "tiny").NodeAddress.String(); got != "10.0.0.6/29" { if got := portOf(t, p, "d", "tiny").Address.String(); got != "10.0.0.6/29" {
t.Errorf("last node = %s, want 10.0.0.6/29", got) t.Errorf("last node = %s, want 10.0.0.6/29", got)
} }
} }
func TestCompute_FillsSegmentWithoutRouterExactly(t *testing.T) {
p := compute(t, header+`
segments:
tiny: { cidr: 10.0.0.0/30 }
nodes:
a: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
b: { role: rr, image: deb, cpus: 1, memory: 512, segments: [tiny] }
`)
if got := portOf(t, p, "b", "tiny").Address.String(); got != "10.0.0.2/30" {
t.Errorf("last node = %s, want 10.0.0.2/30", got)
}
}
func TestCompute_RefusesInvalidTopology(t *testing.T) { func TestCompute_RefusesInvalidTopology(t *testing.T) {
requireContains(t, computeError(t, strings.Replace(valid, "role: rr", "role: router", 1)), `role "router"`) requireContains(t, computeError(t, strings.Replace(valid, "role: rr", "role: spine", 1)), `role "spine"`)
} }
func TestLastAddr(t *testing.T) { func TestLastAddr(t *testing.T) {
@ -305,41 +439,86 @@ func TestWrite_TwoHypervisorsPlan(t *testing.T) {
if err := compute(t, twoHypervisors).Write(&buf); err != nil { if err := compute(t, twoHypervisors).Write(&buf); err != nil {
t.Fatalf("Write: %v", err) t.Fatalf("Write: %v", err)
} }
want := `lab lab-test: nodes 4, segments 1, cables 3 want := `lab lab-test: nodes 4, segments 1, ports 4
nodes nodes
name role image cpus memory ssh name role image cpus memory ssh
sw1 switch deb 2 1024 MiB 127.0.0.1:2200 sw1 router deb 2 1024 MiB 127.0.0.1:2200
rr1 rr deb 1 1024 MiB 127.0.0.1:2201 rr1 rr deb 1 1024 MiB 127.0.0.1:2201
hv1 host deb 4 16384 MiB 127.0.0.1:2202 hv1 host deb 4 16384 MiB 127.0.0.1:2202
hv2 host deb 4 16384 MiB 127.0.0.1:2203 hv2 host deb 4 16384 MiB 127.0.0.1:2203
segment underlay: 10.250.0.0/24, mtu 9000, switch sw1, bridge br-underlay, gateway 10.250.0.1 segment underlay: 10.250.0.0/24, mtu 9000, bridge lb-underlay, gateway 10.250.0.1 on sw1
node interface address mac udp switch port mac udp node interface address mac tap
rr1 underlay 10.250.0.2/24 02:4c:00:01:00:00 20000 <-> sw1 p0 02:4c:00:01:00:01 20001 sw1 underlay 10.250.0.2/24 02:4c:00:00:00:00 lt0
hv1 underlay 10.250.0.3/24 02:4c:00:02:00:00 20002 <-> sw1 p1 02:4c:00:02:00:01 20003 rr1 underlay 10.250.0.3/24 02:4c:00:01:00:00 lt1
hv2 underlay 10.250.0.4/24 02:4c:00:03:00:00 20004 <-> sw1 p2 02:4c:00:03:00:01 20005 hv1 underlay 10.250.0.4/24 02:4c:00:02:00:00 lt2
hv2 underlay 10.250.0.5/24 02:4c:00:03:00:00 lt3
` `
if buf.String() != want { if buf.String() != want {
t.Errorf("output:\n%s\nwant:\n%s", buf.String(), want) t.Errorf("output:\n%s\nwant:\n%s", buf.String(), want)
} }
} }
func TestWrite_GatewayLines(t *testing.T) {
var buf bytes.Buffer
if err := compute(t, header+`
segments:
red: { routers: [r1], cidr: 10.1.0.0/24 }
blue: { routers: [r2, r1], cidr: 10.2.0.0/24, mtu: 1500 }
pure: { cidr: 10.3.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [blue, pure] }
`).Write(&buf); err != nil {
t.Fatalf("Write: %v", err)
}
for _, want := range []string{
"lab lab-test: nodes 3, segments 3, ports 5\n",
"\nsegment red: 10.1.0.0/24, mtu 9000, bridge lb-red, gateway 10.1.0.1 on r1\n",
"\nsegment blue: 10.2.0.0/24, mtu 1500, bridge lb-blue, gateway 10.2.0.1 by vrrp 2 on r2, r1\n",
"\nsegment pure: 10.3.0.0/24, mtu 9000, bridge lb-pure, no gateway\n",
" hv pure 10.3.0.1/24 02:4c:00:02:02:00 lt4\n",
} {
if !strings.Contains(buf.String(), want) {
t.Errorf("plan:\n%s\ndoes not contain %q", buf.String(), want)
}
}
}
func generated(segments, nodes int) string { func generated(segments, nodes int) string {
var b strings.Builder var b strings.Builder
b.WriteString(header + "segments:\n") b.WriteString(header + "segments:\n")
names := make([]string, segments) names := make([]string, segments)
for i := range names { for i := range names {
names[i] = fmt.Sprintf("s%d", i) names[i] = fmt.Sprintf("s%d", i)
fmt.Fprintf(&b, " %s: { switch: sw, cidr: 10.%d.0.0/16 }\n", names[i], i%250) fmt.Fprintf(&b, " %s: { routers: [sw], cidr: 10.%d.0.0/16 }\n", names[i], i%250)
} }
b.WriteString("nodes:\n sw: { role: switch, image: deb, cpus: 1, memory: 512 }\n") fmt.Fprintf(&b, "nodes:\n sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", strings.Join(names, ", "))
for i := 1; i < nodes; i++ { for i := 1; i < nodes; i++ {
fmt.Fprintf(&b, " n%d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", i, strings.Join(names, ", ")) fmt.Fprintf(&b, " n%d: { role: rr, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", i, strings.Join(names, ", "))
} }
return b.String() return b.String()
} }
func withVRRPAt(segments, rank int) string {
var b strings.Builder
b.WriteString(header + "segments:\n")
names := make([]string, segments)
for i := range names {
names[i] = fmt.Sprintf("s%d", i)
routers := "[r1]"
if i == rank {
routers = "[r1, r2]"
}
fmt.Fprintf(&b, " %s: { routers: %s, cidr: 10.%d.0.0/16 }\n", names[i], routers, i%250)
}
fmt.Fprintf(&b, "nodes:\n r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", strings.Join(names, ", "))
fmt.Fprintf(&b, " r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [%s] }\n", names[rank])
return b.String()
}
func TestValidate_RejectsTooManyNodes(t *testing.T) { func TestValidate_RejectsTooManyNodes(t *testing.T) {
requireContains(t, validationError(t, generated(1, 1001)), "nodes: 1001 declared, at most 1000") requireContains(t, validationError(t, generated(1, 1001)), "nodes: 1001 declared, at most 1000")
} }
@ -354,6 +533,24 @@ func TestValidate_RejectsTooManySegments(t *testing.T) {
requireContains(t, validationError(t, generated(257, 2)), "segments: 257 declared, at most 256") requireContains(t, validationError(t, generated(257, 2)), "segments: 257 declared, at most 256")
} }
func TestCompute_RejectsCablesBeyondUDPPortRange(t *testing.T) { func TestValidate_RejectsVRRPBeyondTheVRIDRange(t *testing.T) {
requireContains(t, computeError(t, generated(23, 1000)), "cables: 22977 cables need udp ports up to 65953, beyond 65535") msg := validationError(t, withVRRPAt(256, 255))
requireContains(t, msg, "segment s255: a segment with several routers must be among the first 255")
}
func TestCompute_AcceptsVRRPOnTheLastVRID(t *testing.T) {
p := compute(t, withVRRPAt(256, 254))
if got := segmentOf(t, p, "s254").VRID; got != 255 {
t.Errorf("vrid = %d, want 255", got)
}
}
func TestCompute_ManyPortsNeedNoPortRange(t *testing.T) {
p := compute(t, generated(23, 1000))
if len(p.Ports) != 23000 {
t.Fatalf("%d ports, want 23000", len(p.Ports))
}
if last := p.Ports[len(p.Ports)-1].Tap; last != "lt22999" {
t.Errorf("last tap = %s, want lt22999", last)
}
} }

View file

@ -11,9 +11,9 @@ import (
const withRoles = header + ` const withRoles = header + `
segments: segments:
underlay: { switch: sw1, cidr: 192.168.14.0/24 } underlay: { routers: [sw1], cidr: 192.168.14.0/24 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 2, memory: 1024, secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf } sw1: { role: router, image: deb, cpus: 2, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.1/28] }, frr: frr/sw1.conf }
rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: frr/rr1.conf } rr1: { role: rr, image: deb, cpus: 1, memory: 1024, segments: [underlay], secondary: { underlay: [169.254.0.3/28] }, loopback: 10.255.255.1/32, frr: frr/rr1.conf }
hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: /etc/lab/hv1.conf, provision: provision/hv.sh, post_provision: /opt/lab/post.sh, files: { agent.yml: agent/two.yml, deploy.sh: /opt/deploy.sh }, services: [agent.service] } hv1: { role: host, image: deb, cpus: 4, memory: 16384, segments: [underlay], frr: /etc/lab/hv1.conf, provision: provision/hv.sh, post_provision: /opt/lab/post.sh, files: { agent.yml: agent/two.yml, deploy.sh: /opt/deploy.sh }, services: [agent.service] }
` `
@ -95,10 +95,10 @@ func TestValidate_RoleFieldRejections(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
doc := header + ` doc := header + `
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { switch: sw, cidr: 10.2.0.0/24 } blue: { routers: [sw], cidr: 10.2.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] } hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
` + c.node + ` ` + c.node + `
` `
@ -107,29 +107,29 @@ nodes:
} }
} }
func TestValidate_SwitchSecondaryOnlyOnItsOwnSegments(t *testing.T) { func TestValidate_RouterSecondaryOnlyOnItsOwnSegments(t *testing.T) {
doc := header + ` doc := header + `
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } red: { routers: [sw], cidr: 10.1.0.0/24 }
blue: { switch: other, cidr: 10.2.0.0/24 } blue: { routers: [other], cidr: 10.2.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512, secondary: { red: [169.254.0.1/28], blue: [169.254.1.1/28] } } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red], secondary: { red: [169.254.0.1/28], blue: [169.254.1.1/28] } }
other: { role: switch, image: deb, cpus: 1, memory: 512 } other: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] } hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
` `
msg := validationError(t, doc) msg := validationError(t, doc)
requireContains(t, msg, "node sw: secondary address given for segment blue it is not attached to") requireContains(t, msg, "node sw: secondary address given for segment blue it is not attached to")
if bytes.Contains([]byte(msg), []byte("segment red")) { if bytes.Contains([]byte(msg), []byte("segment red")) {
t.Errorf("the switch's own segment was refused:\n%s", msg) t.Errorf("the router's own segment was refused:\n%s", msg)
} }
} }
func TestValidate_LoopbackInterfaceNameIsReserved(t *testing.T) { func TestValidate_LoopbackInterfaceNameIsReserved(t *testing.T) {
doc := header + ` doc := header + `
segments: segments:
lo1: { switch: sw, cidr: 10.1.0.0/24 } lo1: { routers: [sw], cidr: 10.1.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [lo1] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [lo1] } hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [lo1] }
` `
requireContains(t, validationError(t, doc), "segment lo1: name is reserved for the loopback interface") requireContains(t, validationError(t, doc), "segment lo1: name is reserved for the loopback interface")
@ -156,9 +156,9 @@ func TestWrite_NoRolesSectionWithoutRoleFields(t *testing.T) {
var buf bytes.Buffer var buf bytes.Buffer
doc := header + ` doc := header + `
segments: segments:
underlay: { switch: sw1, cidr: 10.250.0.0/24 } underlay: { routers: [sw1], cidr: 10.250.0.0/24 }
nodes: nodes:
sw1: { role: switch, image: deb, cpus: 1, memory: 512 } sw1: { role: router, image: deb, cpus: 1, memory: 512, segments: [underlay] }
rr1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [underlay] } rr1: { role: rr, image: deb, cpus: 1, memory: 512, segments: [underlay] }
` `
if err := compute(t, doc).Write(&buf); err != nil { if err := compute(t, doc).Write(&buf); err != nil {
@ -176,7 +176,7 @@ func TestValidate_ProvisionRejections(t *testing.T) {
}{ }{
"the hypervisor role is gone": { "the hypervisor role is gone": {
`hv: { role: hypervisor, image: deb, cpus: 1, memory: 512, segments: [red] }`, `hv: { role: hypervisor, image: deb, cpus: 1, memory: 512, segments: [red] }`,
`node hv: role "hypervisor" must be one of switch, rr, host`, `node hv: role "hypervisor" must be one of router, rr, host`,
}, },
"file name leaving the files directory": { "file name leaving the files directory": {
`hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { ../etc/passwd: a.yml } }`, `hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { ../etc/passwd: a.yml } }`,
@ -203,9 +203,9 @@ func TestValidate_ProvisionRejections(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
doc := header + ` doc := header + `
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } red: { routers: [sw], cidr: 10.1.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red] }
` + c.node + ` ` + c.node + `
` `
requireContains(t, validationError(t, doc), c.want) requireContains(t, validationError(t, doc), c.want)
@ -216,9 +216,9 @@ nodes:
func TestValidate_ProvisionIsAllowedOnEveryRole(t *testing.T) { func TestValidate_ProvisionIsAllowedOnEveryRole(t *testing.T) {
doc := header + ` doc := header + `
segments: segments:
red: { switch: sw, cidr: 10.1.0.0/24 } red: { routers: [sw], cidr: 10.1.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512, provision: sw.sh, files: { a.conf: a.conf }, services: [lab-switch.service] } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [red], provision: sw.sh, files: { a.conf: a.conf }, services: [lab-router.service] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red], provision: rr.sh, post_provision: rr-post.sh } rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [red], provision: rr.sh, post_provision: rr-post.sh }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { agent.yml: agent.yml }, services: [agent.service] } hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red], files: { agent.yml: agent.yml }, services: [agent.service] }
` `

View file

@ -11,7 +11,7 @@ import (
) )
const ( const (
RoleSwitch = "switch" RoleRouter = "router"
RoleRR = "rr" RoleRR = "rr"
RoleHost = "host" RoleHost = "host"
@ -32,10 +32,12 @@ type Image struct {
} }
type Segment struct { type Segment struct {
Name string Name string
Switch string Routers []string
CIDR string CIDR string
MTU int MTU int
legacySwitch string
} }
type Node struct { type Node struct {
@ -61,9 +63,10 @@ type fileImage struct {
} }
type fileSegment struct { type fileSegment struct {
Switch string `yaml:"switch"` Routers []string `yaml:"routers"`
CIDR string `yaml:"cidr"` CIDR string `yaml:"cidr"`
MTU int `yaml:"mtu"` MTU int `yaml:"mtu"`
Switch string `yaml:"switch"`
} }
type fileNode struct { type fileNode struct {
@ -145,7 +148,7 @@ func Parse(data []byte) (*Topology, error) {
if mtu == 0 { if mtu == 0 {
mtu = DefaultMTU mtu = DefaultMTU
} }
t.Segments = append(t.Segments, Segment{Name: name, Switch: s.Switch, CIDR: s.CIDR, MTU: mtu}) t.Segments = append(t.Segments, Segment{Name: name, Routers: s.Routers, CIDR: s.CIDR, MTU: mtu, legacySwitch: s.Switch})
} }
for _, name := range order["nodes"] { for _, name := range order["nodes"] {
n := f.Nodes[name] n := f.Nodes[name]

View file

@ -40,10 +40,10 @@ func requireContains(t *testing.T, got, want string) {
func TestParse_KeepsDeclarationOrderOfNodesAndSegments(t *testing.T) { func TestParse_KeepsDeclarationOrderOfNodesAndSegments(t *testing.T) {
topo := parse(t, header+` topo := parse(t, header+`
segments: segments:
zulu: { switch: sw, cidr: 10.0.1.0/24 } zulu: { routers: [sw], cidr: 10.0.1.0/24 }
alpha: { switch: sw, cidr: 10.0.2.0/24 } alpha: { routers: [sw], cidr: 10.0.2.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [zulu, alpha] }
zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu, alpha] } zeta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu, alpha] }
beta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu] } beta: { role: rr, image: deb, cpus: 1, memory: 512, segments: [zulu] }
`) `)
@ -65,9 +65,9 @@ nodes:
func TestParse_DefaultsMTUTo9000(t *testing.T) { func TestParse_DefaultsMTUTo9000(t *testing.T) {
topo := parse(t, header+` topo := parse(t, header+`
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/24 } under: { cidr: 10.0.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
`) `)
if topo.Segments[0].MTU != 9000 { if topo.Segments[0].MTU != 9000 {
t.Errorf("MTU = %d, want 9000", topo.Segments[0].MTU) t.Errorf("MTU = %d, want 9000", topo.Segments[0].MTU)
@ -77,7 +77,7 @@ nodes:
func TestParse_RejectsUnknownField(t *testing.T) { func TestParse_RejectsUnknownField(t *testing.T) {
_, err := Parse([]byte(header + ` _, err := Parse([]byte(header + `
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512, ram: 4 } sw: { role: router, image: deb, cpus: 1, memory: 512, ram: 4 }
`)) `))
if err == nil || !strings.Contains(err.Error(), "ram") { if err == nil || !strings.Contains(err.Error(), "ram") {
t.Fatalf("unknown field not rejected: %v", err) t.Fatalf("unknown field not rejected: %v", err)
@ -87,7 +87,7 @@ nodes:
func TestParse_RejectsDuplicateNode(t *testing.T) { func TestParse_RejectsDuplicateNode(t *testing.T) {
_, err := Parse([]byte(header + ` _, err := Parse([]byte(header + `
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512 }
sw: { role: rr, image: deb, cpus: 1, memory: 512 } sw: { role: rr, image: deb, cpus: 1, memory: 512 }
`)) `))
if err == nil || !strings.Contains(err.Error(), "already defined") { if err == nil || !strings.Contains(err.Error(), "already defined") {
@ -109,9 +109,9 @@ func TestLoad_ReportsPathOnError(t *testing.T) {
const valid = header + ` const valid = header + `
segments: segments:
under: { switch: sw, cidr: 10.0.0.0/24 } under: { routers: [sw], cidr: 10.0.0.0/24 }
nodes: nodes:
sw: { role: switch, image: deb, cpus: 1, memory: 512 } sw: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] } rr: { role: rr, image: deb, cpus: 1, memory: 512, segments: [under] }
` `
@ -129,9 +129,11 @@ func TestValidate_Rejections(t *testing.T) {
{"image url not https", strings.Replace(valid, "url: https://", "url: http://", 1), "image deb: url"}, {"image url not https", strings.Replace(valid, "url: https://", "url: http://", 1), "image deb: url"},
{"image sums not https", strings.Replace(valid, "sums: https://", "sums: ftp://", 1), "image deb: sums"}, {"image sums not https", strings.Replace(valid, "sums: https://", "sums: ftp://", 1), "image deb: sums"},
{"no node", header + "\nsegments: {}\nnodes: {}\n", "at least one node"}, {"no node", header + "\nsegments: {}\nnodes: {}\n", "at least one node"},
{"segment switch missing", strings.Replace(valid, "switch: sw, cidr", "cidr", 1), "segment under: switch is required"}, {"legacy segment switch", strings.Replace(valid, "routers: [sw], cidr", "switch: sw, cidr", 1), "segment under: switch was replaced by routers"},
{"segment switch unknown", strings.Replace(valid, "switch: sw, cidr", "switch: ghost, cidr", 1), "switch ghost is not a declared node"}, {"legacy role switch", strings.Replace(valid, "role: router", "role: switch", 1), "node sw: role switch was renamed router"},
{"segment switch not a switch", strings.Replace(valid, "switch: sw, cidr", "switch: rr, cidr", 1), "rr is a rr, not a switch"}, {"segment router unknown", strings.Replace(valid, "routers: [sw]", "routers: [ghost]", 1), "segment under: router ghost is not a declared node"},
{"segment router not a router", strings.Replace(valid, "routers: [sw]", "routers: [rr]", 1), "segment under: rr is a rr, not a router"},
{"segment router listed twice", strings.Replace(valid, "routers: [sw]", "routers: [sw, sw]", 1), "segment under: router sw listed twice"},
{"segment name too long", strings.ReplaceAll(valid, "under", "underlayunder"), "segment underlayunder: name must match"}, {"segment name too long", strings.ReplaceAll(valid, "under", "underlayunder"), "segment underlayunder: name must match"},
{"segment named like the admin interface", strings.ReplaceAll(valid, "under", "mgmt0"), "segment mgmt0: name is reserved"}, {"segment named like the admin interface", strings.ReplaceAll(valid, "under", "mgmt0"), "segment mgmt0: name is reserved"},
{"segment name with dash", strings.ReplaceAll(valid, "under", "un-der"), "segment un-der: name must match"}, {"segment name with dash", strings.ReplaceAll(valid, "under", "un-der"), "segment un-der: name must match"},
@ -143,19 +145,17 @@ func TestValidate_Rejections(t *testing.T) {
{"cidr too small", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/31", 1), "prefix length out of range"}, {"cidr too small", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/31", 1), "prefix length out of range"},
{"cidr too large", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/7", 1), "prefix length out of range"}, {"cidr too large", strings.Replace(valid, "10.0.0.0/24", "10.0.0.0/7", 1), "prefix length out of range"},
{"bad node name", strings.Replace(valid, " rr: {", " RR: {", 1), "node RR: name must match"}, {"bad node name", strings.Replace(valid, " rr: {", " RR: {", 1), "node RR: name must match"},
{"bad role", strings.Replace(valid, "role: rr", "role: router", 1), `role "router"`}, {"bad role", strings.Replace(valid, "role: rr", "role: spine", 1), `role "spine"`},
{"unknown image", strings.Replace(valid, "role: rr, image: deb", "role: rr, image: ubuntu", 1), `image "ubuntu" is not declared`}, {"unknown image", strings.Replace(valid, "role: rr, image: deb", "role: rr, image: ubuntu", 1), `image "ubuntu" is not declared`},
{"no cpu", strings.Replace(valid, "role: rr, image: deb, cpus: 1", "role: rr, image: deb, cpus: 0", 1), "node rr: cpus must be at least 1"}, {"no cpu", strings.Replace(valid, "role: rr, image: deb, cpus: 1", "role: rr, image: deb, cpus: 0", 1), "node rr: cpus must be at least 1"},
{"memory too low", strings.Replace(valid, "cpus: 1, memory: 512, segments", "cpus: 1, memory: 255, segments", 1), "node rr: memory must be at least 256"}, {"memory too low", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512", "role: rr, image: deb, cpus: 1, memory: 255", 1), "node rr: memory must be at least 256"},
{"switch with segments", strings.Replace(valid, "role: switch, image: deb, cpus: 1, memory: 512 }", "role: switch, image: deb, cpus: 1, memory: 512, segments: [under] }", 1), "node sw: a switch carries its segments"}, {"router without segment", valid + " r2: { role: router, image: deb, cpus: 1, memory: 512 }\n", "node r2: must be attached to at least one segment"},
{"switch with addresses", strings.Replace(valid, "role: switch, image: deb, cpus: 1, memory: 512 }", "role: switch, image: deb, cpus: 1, memory: 512, addresses: {under: 10.0.0.9} }", 1), "node sw: a switch carries its segments"},
{"switch without segment", valid + " sw2: { role: switch, image: deb, cpus: 1, memory: 512 }\n", "node sw2: switch carries no segment"},
{"node without segment", valid + " rr2: { role: rr, image: deb, cpus: 1, memory: 512 }\n", "node rr2: must be attached to at least one segment"}, {"node without segment", valid + " rr2: { role: rr, image: deb, cpus: 1, memory: 512 }\n", "node rr2: must be attached to at least one segment"},
{"node on unknown segment", strings.Replace(valid, "segments: [under]", "segments: [over]", 1), "node rr: segment over is not declared"}, {"node on unknown segment", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [over]", 1), "node rr: segment over is not declared"},
{"node on segment twice", strings.Replace(valid, "segments: [under]", "segments: [under, under]", 1), "node rr: segment under listed twice"}, {"node on segment twice", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under, under]", 1), "node rr: segment under listed twice"},
{"segment without node", strings.Replace(valid, " under: { switch: sw, cidr: 10.0.0.0/24 }", " under: { switch: sw, cidr: 10.0.0.0/24 }\n empty: { switch: sw, cidr: 10.0.9.0/24 }", 1), "segment empty: no node is attached"}, {"segment without node", strings.Replace(valid, " under: { routers: [sw], cidr: 10.0.0.0/24 }", " under: { routers: [sw], cidr: 10.0.0.0/24 }\n empty: { cidr: 10.0.9.0/24 }", 1), "segment empty: no node is attached"},
{"address on foreign segment", strings.Replace(valid, "segments: [under]", "segments: [under], addresses: {over: 10.0.0.9}", 1), "address given for segment over"}, {"address on foreign segment", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {over: 10.0.0.9}", 1), "node rr: address given for segment over"},
{"address unparsable", strings.Replace(valid, "segments: [under]", "segments: [under], addresses: {under: 10.0.0}", 1), `address "10.0.0" on under`}, {"address unparsable", strings.Replace(valid, "role: rr, image: deb, cpus: 1, memory: 512, segments: [under]", "role: rr, image: deb, cpus: 1, memory: 512, segments: [under], addresses: {under: 10.0.0}", 1), `node rr: address "10.0.0" on under`},
} }
for _, c := range cases { for _, c := range cases {
t.Run(c.name, func(t *testing.T) { t.Run(c.name, func(t *testing.T) {
@ -164,10 +164,53 @@ func TestValidate_Rejections(t *testing.T) {
} }
} }
func TestValidate_RejectsRouterNotAttachedToItsSegment(t *testing.T) {
doc := header + `
segments:
red: { routers: [r], cidr: 10.1.0.0/24 }
blue: { routers: [r], cidr: 10.2.0.0/24 }
nodes:
r: { role: router, image: deb, cpus: 1, memory: 512, segments: [blue] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [red, blue] }
`
msg := validationError(t, doc)
requireContains(t, msg, "segment red: router r is not attached to it")
if strings.Contains(msg, "segment blue") {
t.Errorf("the router's own segment was refused:\n%s", msg)
}
}
func TestValidate_AcceptsSegmentWithoutRouter(t *testing.T) {
doc := header + `
segments:
l2: { cidr: 10.3.0.0/24 }
nodes:
a: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
b: { role: host, image: deb, cpus: 1, memory: 512, segments: [l2] }
`
if err := parse(t, doc).Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
}
func TestValidate_AcceptsTwoRoutersOnASegment(t *testing.T) {
doc := header + `
segments:
under: { routers: [r1, r2], cidr: 10.0.0.0/24 }
nodes:
r1: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
r2: { role: router, image: deb, cpus: 1, memory: 512, segments: [under] }
hv: { role: host, image: deb, cpus: 1, memory: 512, segments: [under] }
`
if err := parse(t, doc).Validate(); err != nil {
t.Fatalf("Validate: %v", err)
}
}
func TestValidate_ReportsEveryErrorAtOnce(t *testing.T) { func TestValidate_ReportsEveryErrorAtOnce(t *testing.T) {
doc := strings.Replace(valid, "role: rr", "role: router", 1) doc := strings.Replace(valid, "role: rr", "role: spine", 1)
doc = strings.Replace(doc, "10.0.0.0/24", "10.0.0.0/31", 1) doc = strings.Replace(doc, "10.0.0.0/24", "10.0.0.0/31", 1)
got := validationError(t, doc) got := validationError(t, doc)
requireContains(t, got, `role "router"`) requireContains(t, got, `role "spine"`)
requireContains(t, got, "prefix length out of range") requireContains(t, got, "prefix length out of range")
} }

View file

@ -18,6 +18,8 @@ const (
ReservedInterface = "mgmt0" ReservedInterface = "mgmt0"
LoopbackInterface = "lo1" LoopbackInterface = "lo1"
MaxVRID = 255
) )
var ( var (
@ -74,14 +76,23 @@ func (t *Topology) Validate() error {
if s.Name == LoopbackInterface { if s.Name == LoopbackInterface {
add("segment %s: name is reserved for the loopback interface", s.Name) add("segment %s: name is reserved for the loopback interface", s.Name)
} }
sw, ok := nodes[s.Switch] if s.legacySwitch != "" {
switch { add("segment %s: switch was replaced by routers: declare %s with role %s, attach it to %s through its segments, and list it in segments.%s.routers", s.Name, s.legacySwitch, RoleRouter, s.Name, s.Name)
case s.Switch == "": }
add("segment %s: switch is required", s.Name) seenRouters := map[string]bool{}
case !ok: for _, name := range s.Routers {
add("segment %s: switch %s is not a declared node", s.Name, s.Switch) r, ok := nodes[name]
case sw.Role != RoleSwitch: switch {
add("segment %s: %s is a %s, not a switch", s.Name, s.Switch, sw.Role) case seenRouters[name]:
add("segment %s: router %s listed twice", s.Name, name)
case !ok:
add("segment %s: router %s is not a declared node", s.Name, name)
case r.Role != RoleRouter:
add("segment %s: %s is a %s, not a %s", s.Name, name, r.Role, RoleRouter)
case !contains(r.Segments, s.Name):
add("segment %s: router %s is not attached to it", s.Name, name)
}
seenRouters[name] = true
} }
if s.MTU < MinMTU || s.MTU > MaxMTU { if s.MTU < MinMTU || s.MTU > MaxMTU {
add("segment %s: mtu %d out of range [%d, %d]", s.Name, s.MTU, MinMTU, MaxMTU) add("segment %s: mtu %d out of range [%d, %d]", s.Name, s.MTU, MinMTU, MaxMTU)
@ -99,19 +110,25 @@ func (t *Topology) Validate() error {
} }
} }
attached := map[string]int{} for i, s := range t.Segments {
hosting := map[string]int{} if len(s.Routers) > 1 {
for _, s := range t.Segments { if i+1 > MaxVRID {
hosting[s.Switch]++ add("segment %s: a segment with several routers must be among the first %d, its rank sets the vrrp id", s.Name, MaxVRID)
}
}
} }
attached := map[string]int{}
for _, n := range t.Nodes { for _, n := range t.Nodes {
if !namePattern.MatchString(n.Name) { if !namePattern.MatchString(n.Name) {
add("node %s: name must match %s", n.Name, namePattern) add("node %s: name must match %s", n.Name, namePattern)
} }
switch n.Role { switch n.Role {
case RoleSwitch, RoleRR, RoleHost: case RoleRouter, RoleRR, RoleHost:
case "switch":
add("node %s: role switch was renamed %s: attach it to its segments through segments and addresses, and list it in segments.<name>.routers", n.Name, RoleRouter)
default: default:
add("node %s: role %q must be one of %s, %s, %s", n.Name, n.Role, RoleSwitch, RoleRR, RoleHost) add("node %s: role %q must be one of %s, %s, %s", n.Name, n.Role, RoleRouter, RoleRR, RoleHost)
} }
if !images[n.Image] { if !images[n.Image] {
add("node %s: image %q is not declared", n.Name, n.Image) add("node %s: image %q is not declared", n.Name, n.Image)
@ -123,15 +140,6 @@ func (t *Topology) Validate() error {
add("node %s: memory must be at least %d MiB", n.Name, MinMemory) add("node %s: memory must be at least %d MiB", n.Name, MinMemory)
} }
validateExtras(n, segments, add) validateExtras(n, segments, add)
if n.Role == RoleSwitch {
if len(n.Segments) > 0 || len(n.Addresses) > 0 {
add("node %s: a switch carries its segments through segments.<name>.switch, not through segments or addresses", n.Name)
}
if hosting[n.Name] == 0 {
add("node %s: switch carries no segment", n.Name)
}
continue
}
if len(n.Segments) == 0 { if len(n.Segments) == 0 {
add("node %s: must be attached to at least one segment", n.Name) add("node %s: must be attached to at least one segment", n.Name)
} }
@ -174,16 +182,8 @@ func (t *Topology) Validate() error {
func validateExtras(n Node, segments map[string]Segment, add func(string, ...any)) { func validateExtras(n Node, segments map[string]Segment, add func(string, ...any)) {
carried := map[string]bool{} carried := map[string]bool{}
if n.Role == RoleSwitch { for _, name := range n.Segments {
for name, s := range segments { carried[name] = true
if s.Switch == n.Name {
carried[name] = true
}
}
} else {
for _, name := range n.Segments {
carried[name] = true
}
} }
names := make([]string, 0, len(n.Secondary)) names := make([]string, 0, len(n.Secondary))
for name := range n.Secondary { for name := range n.Secondary {

14
release_notes/0.2.0.md Normal file
View file

@ -0,0 +1,14 @@
# Segments sur des bridges de l'hôte, routeurs et passerelle VRRP
**Changement incompatible** : une topologie 0.1 doit être migrée (voir « Migration depuis 0.1 »
dans `docs/guide.md`).
- Chaque segment est un bridge Linux de l'hôte (`lb-<segment>`), chaque interface de VM un tap
(`lt<i>`) ; plus de sockets UDP. `labctl up` et `labctl down` exigent root, `down` retire les
bridges et les taps, `status` affiche les segments.
- Le rôle `switch` devient `router`, et `segments.<s>.switch` devient `segments.<s>.routers`,
une liste ordonnée. Un segment sans routeur est un L2 pur, sans passerelle.
- Un routeur fait du forwarding et masque vers `mgmt0` les segments qu'il route (service
`lab-router`). Avec deux routeurs ou plus, la passerelle est portée en VRRP v3 par `vrrpd` de
FRR, sans préemption ; bascule d'environ 2,6 s à l'arrêt brutal du maître.
- Exemple `examples/evpn-2hv` : deux routeurs `r1` et `r2` en VRRP.