f-76: nft: règles de pont en nftables, ebtables retiré #76
internal/nft pose les règles d'un port en une transaction, dans une table bridge two par netns de VPC. Subnets, gateway anycast et watchdog l'utilisent, même filtrage qu'avec ebtables. bootstrap_kvm.sh et le lab installent nftables ; helpers e2e, doc et notes 0.3.0 suivent. Signed-off-by: GnomeZworc <nicolas.boufidjeline@g3e.fr>
This commit is contained in:
parent
da6295baf1
commit
d98e42e0a8
27 changed files with 823 additions and 341 deletions
|
|
@ -18,7 +18,7 @@ Réseau
|
|||
* **La route vers le CIDR du VPC garde ``interface_ip`` comme next-hop** dans tous les modes sauf
|
||||
``bridge`` : sur un subnet à IP publique, le trafic interne ne doit pas sortir par la gateway
|
||||
publique.
|
||||
* **La MAC anycast d'une gateway ``vxlan`` ne sort jamais du netns** : règles ebtables sur le veth
|
||||
* **La MAC anycast d'une gateway ``vxlan`` ne sort jamais du netns** : règles nftables sur le veth
|
||||
du subnet (source anycast rejetée en entrée et en sortie) **et** entrée FDB locale dans le bridge
|
||||
du netns. Sans l'entrée FDB, que le macvlan ne crée pas, toute trame d'une VM vers sa gateway est
|
||||
inondée vers le fabric et traitée par la gateway de chaque hôte.
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ Paquets
|
|||
- network namespaces : create/enter/delete/call
|
||||
* - ``internal/netif``
|
||||
- netlink : bridge, veth, vxlan, tap, routes, adresses
|
||||
* - ``internal/ebtables``, ``internal/iptables``
|
||||
* - ``internal/nft``, ``internal/iptables``
|
||||
- wrappers dédiés ; ne pas appeler ces binaires ailleurs
|
||||
* - ``internal/qemu``, ``internal/qmp``
|
||||
- lancement de QEMU et client QMP sur socket Unix
|
||||
|
|
|
|||
|
|
@ -96,8 +96,8 @@ La MAC anycast ne doit **jamais** sortir du netns : vue depuis le fabric, elle s
|
|||
EVPN par chaque hôte, détectée en double, et déplacerait l'apprentissage des bridges. Trois
|
||||
mécanismes la confinent :
|
||||
|
||||
* ebtables, sur le veth du subnet : toute trame de **source** anycast est rejetée à l'entrée
|
||||
(``nat PREROUTING``) et à la sortie (``FORWARD`` et ``OUTPUT``) ;
|
||||
* nftables, sur le veth du subnet : toute trame de **source** anycast est rejetée à l'entrée
|
||||
(``prerouting``) et à la sortie (``forward`` et ``output``) ;
|
||||
* une **entrée FDB locale** pour la MAC anycast dans le bridge du netns. Le macvlan ne la crée pas
|
||||
lui-même : sans elle, toute trame d'une VM vers sa gateway est inondée vers le fabric et les
|
||||
autres VM, et traitée par la gateway de chaque hôte ;
|
||||
|
|
@ -106,7 +106,7 @@ mécanismes la confinent :
|
|||
|
||||
IPv6 n'est pas désactivé sur le macvlan : ses trames (DAD, ND) sont arrêtées par le confinement.
|
||||
Le watchdog contrôle le macvlan, sa MAC, les deux adresses et l'entrée FDB ; il ne relit pas les
|
||||
règles ebtables.
|
||||
règles nftables.
|
||||
|
||||
Subnets créés avant la gateway anycast
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -124,19 +124,28 @@ VPC : le trafic sort par le bridge, et la VM est joignable depuis l'host.
|
|||
|
||||
Le même subnet peut être posé sur plusieurs hôtes raccordés au même segment. Chaque hôte porte
|
||||
alors la **même** ``interface_ip`` dans son netns, avec sa propre MAC. Pour qu'une VM ne voie que
|
||||
l'``interface_ip`` de son hôte (gateway, metadata, DHCP), l'agent pose quatre règles ebtables sur
|
||||
l'``interface_ip`` de son hôte (gateway, metadata, DHCP), l'agent pose quatre règles nftables sur
|
||||
le veth du subnet dans le netns (``v-<subnet>-i``), en sortie vers le bridge de l'host :
|
||||
|
||||
* les requêtes ARP des VM **vers** ``interface_ip`` ne quittent pas l'hôte (``FORWARD``) ;
|
||||
* les requêtes ARP des VM **vers** ``interface_ip`` ne quittent pas l'hôte (``forward``) ;
|
||||
* l'ARP émis **depuis** ``interface_ip`` par le netns, requêtes et réponses, ne sort pas
|
||||
(``OUTPUT``) : une VM d'un autre hôte n'associe jamais ``interface_ip`` à cet hôte ;
|
||||
* le DHCP émis depuis ``interface_ip`` ne sort ni en transit (``FORWARD``) ni depuis le serveur
|
||||
local lui-même (``OUTPUT``).
|
||||
(``output``) : une VM d'un autre hôte n'associe jamais ``interface_ip`` à cet hôte ;
|
||||
* le DHCP émis depuis ``interface_ip`` ne sort ni en transit (``forward``) ni depuis le serveur
|
||||
local lui-même (``output``).
|
||||
|
||||
Le trafic entre VM du subnet, sur un même hôte ou sur des hôtes différents, n'est pas concerné.
|
||||
Par conception, une VM ne joint pas le netns d'un autre hôte. Les règles sont retirées à la
|
||||
suppression du subnet ; une règle déjà absente ne fait pas échouer la suppression. Le watchdog
|
||||
signale les règles manquantes.
|
||||
signale les règles manquantes et celles qui ne sont pas de l'agent.
|
||||
|
||||
Règles de pont
|
||||
--------------
|
||||
|
||||
Dans les deux modes, les règles de couche 2 vivent dans une table nftables ``bridge two`` du netns
|
||||
du VPC. Trois chaînes de base (``prerouting``, ``forward``, ``output``) aiguillent chaque trame,
|
||||
selon l'interface d'entrée ou de sortie, vers la chaîne du port concerné (``pre-<port>``,
|
||||
``fwd-<port>``, ``out-<port>``) par les tables ``pre_ports``, ``fwd_ports`` et ``out_ports``.
|
||||
L'agent pose ou retire toutes les règles d'un port en une seule transaction.
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ Prérequis
|
|||
---------
|
||||
|
||||
Un host Linux avec KVM, sur lequel vous avez ``root``. Les opérations réseau (network
|
||||
namespaces, netlink, VXLAN, ebtables, iptables) et QEMU ne fonctionnent que sous Linux.
|
||||
namespaces, netlink, VXLAN, nftables, iptables) et QEMU ne fonctionnent que sous Linux.
|
||||
|
||||
Déploiement
|
||||
-----------
|
||||
|
|
@ -71,7 +71,7 @@ Les options booléennes actives par défaut se **désactivent** par leur forme l
|
|||
Ce que fait ``-i``
|
||||
------------------
|
||||
|
||||
**Paquets** — ``qemu-system-x86``, ``qemu-utils``, ``ovmf``, ``dnsmasq``, ``ebtables``,
|
||||
**Paquets** — ``qemu-system-x86``, ``qemu-utils``, ``ovmf``, ``dnsmasq``, ``nftables``,
|
||||
``iptables``, ``nfs-common``, ``jq``, ``curl``, **sans les paquets recommandés** : ceux de
|
||||
``qemu-system-x86`` (interface graphique, gtk, gstreamer, pilotes de stockage réseau) ne servent
|
||||
pas à un hyperviseur sans écran, et l'installation passe d'environ 270 à 65 Mo — rejouée à chaque
|
||||
|
|
|
|||
|
|
@ -255,9 +255,9 @@ code de sortie vaut 1 si une vérification échoue **ou si aucune n'a été fait
|
|||
**Tests autonomes.** Avant chaque test, le runner supprime les ressources ``tNN`` laissées par un
|
||||
passage précédent, puis relève l'état de chaque hôte de l'en-tête ``# hôtes:``. Après le test, il
|
||||
supprime ses VM, ses subnets puis ses VPC, et vérifie que l'hôte est revenu à l'état relevé
|
||||
(``assert_clean`` : netns, liens, adresses, routes, règles ebtables et iptables, configuration FRR et
|
||||
``frr.conf``, fichiers de ``/run/two``, ``dnsmasq.d`` et des volumes, unités ``two-vm-*``,
|
||||
processus dnsmasq et QEMU). Aucun test ne dépend d'un autre : n'importe lequel se lance seul.
|
||||
(``assert_clean`` : netns, liens, adresses, routes, règles nftables de pont et iptables,
|
||||
configuration FRR et ``frr.conf``, fichiers de ``/run/two``, ``dnsmasq.d`` et des volumes, unités
|
||||
``two-vm-*``, processus dnsmasq et QEMU). Aucun test ne dépend d'un autre : n'importe lequel se lance seul.
|
||||
|
||||
**Plages.** Le test ``NN`` nomme ses ressources ``vp-tNN*``, ``sn-tNN*`` et ``tNN-*``, et utilise
|
||||
``10.NN.0.0/16`` et les VNI ``NN00`` à ``NN99`` : aucune collision entre tests.
|
||||
|
|
@ -286,7 +286,7 @@ Générée par ``test/e2e/run.sh --matrice`` à partir des en-têtes ; à régé
|
|||
- hv1 hv2 hv3
|
||||
- #50, #51, #54, #41
|
||||
* - ``02-prerequis``
|
||||
- hypothèses d'environnement de la suite — firmware UEFI, unités modèles de l'agent, journalctl, format de ebtables -L, br-lan, priorités VRRP lues dans FRR
|
||||
- hypothèses d'environnement de la suite — firmware UEFI, unités modèles de l'agent, journalctl, format de nft list table, br-lan, priorités VRRP lues dans FRR
|
||||
- hv1 hv2 hv3
|
||||
- #70
|
||||
* - ``03-image``
|
||||
|
|
@ -423,7 +423,6 @@ Hypothèses vérifiées en lab, à garder en tête en écrivant un test :
|
|||
* systemd-networkd **ne réécrit pas** un bail renouvelé à l'identique : un renouvellement se
|
||||
vérifie par le DHCPACK dans le journal du serveur (``lease_renewed``), pas par le fichier de bail ;
|
||||
* l'agent n'accepte **qu'un seul disque virtio** (le disque racine, ``vda``) ; les autres sont des ``xvdX`` ;
|
||||
* sur Debian 12, ``ebtables -L`` (nft) garde les zéros de tête des MAC ;
|
||||
* ``interface_ip`` est posée en ``/32`` sur le bridge du netns en mode ``bridge`` ;
|
||||
* depuis le netns d'un hôte, une VM d'un autre hôte ne répond pas : la réponse repart vers la
|
||||
gateway de son propre hôte. Une vérification entre hôtes passe par une VM.
|
||||
|
|
@ -447,7 +446,7 @@ Chaque bloc est précédé des bibliothèques de ``test/e2e/lib/`` :
|
|||
``user_data``, mot de passe), SSH, bail, métadonnées. La MAC de chaque NIC est fournie à l'API :
|
||||
``52:54:00`` suivi des trois derniers octets de l'IP (``mac_for_ip``), unique dans la suite
|
||||
puisque chaque test a sa plage ; ``--nic sn=ip=mac`` en impose une autre ;
|
||||
* ``net.sh`` : VPC, subnets, MAC anycast, trames brutes, règles ebtables, VRRP ;
|
||||
* ``net.sh`` : VPC, subnets, MAC anycast, trames brutes, règles nftables, VRRP ;
|
||||
* ``evpn.sh`` : VRF, BGP, L3VNI, watchdog ;
|
||||
* ``fixture.sh`` : nettoyage et ``assert_clean``, appelés par le runner. Le nettoyage couvre les
|
||||
ressources ``t<NN>`` de l'API, les netns ``t<NN>-*`` et les fichiers ``t<NN>-*`` / ``t<NN>.*`` de
|
||||
|
|
|
|||
|
|
@ -85,9 +85,8 @@ entry``…) disent déjà ce qui manque. À la main, dans le netns du VPC :
|
|||
# route du subnet émise depuis la gateway
|
||||
ip -n <vpc> -4 route show dev br-<subnet>
|
||||
|
||||
# confinement : source anycast rejetée sur le veth du subnet
|
||||
ip netns exec <vpc> ebtables -t nat -L PREROUTING
|
||||
ip netns exec <vpc> ebtables -L FORWARD
|
||||
# confinement : chaînes pre-, fwd- et out-v-<subnet>-i, source anycast rejetée
|
||||
ip netns exec <vpc> nft list table bridge two
|
||||
|
||||
Dans le guest, ``ip neigh show <gateway>`` doit montrer la MAC ``02:01:00:<VNI>``. Une autre MAC —
|
||||
celle d'un hôte, ``02:03:…`` — signale un subnet de l'ancien modèle (sans ``relay_ip``) ou une
|
||||
|
|
|
|||
|
|
@ -1,153 +0,0 @@
|
|||
package ebtables
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func addRule(args ...string) error {
|
||||
return exec.Command("ebtables", append([]string{"-A"}, args...)...).Run()
|
||||
}
|
||||
|
||||
func deleteRule(args ...string) error {
|
||||
return exec.Command("ebtables", append([]string{"-D"}, args...)...).Run()
|
||||
}
|
||||
|
||||
func DropARPToGateway(iface, ip string) error {
|
||||
if err := addRule("FORWARD",
|
||||
"--out-interface", iface,
|
||||
"-p", "arp",
|
||||
"--arp-op", "Request",
|
||||
"--arp-ip-dst", ip,
|
||||
"-j", "DROP"); err != nil {
|
||||
return fmt.Errorf("ebtables arp rule: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func DropDHCP(iface, ip string) error {
|
||||
if err := addRule("FORWARD",
|
||||
"--out-interface", iface,
|
||||
"-p", "IPv4",
|
||||
"--ip-protocol", "udp",
|
||||
"--ip-source-port", "67:68",
|
||||
"--ip-destination-port", "67:68",
|
||||
"--ip-source", ip,
|
||||
"-j", "DROP"); err != nil {
|
||||
return fmt.Errorf("ebtables dhcp rule: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func DeleteARPToGateway(iface, ip string) error {
|
||||
return deleteRule("FORWARD",
|
||||
"--out-interface", iface,
|
||||
"-p", "arp",
|
||||
"--arp-op", "Request",
|
||||
"--arp-ip-dst", ip,
|
||||
"-j", "DROP")
|
||||
}
|
||||
|
||||
func DeleteDHCP(iface, ip string) error {
|
||||
return deleteRule("FORWARD",
|
||||
"--out-interface", iface,
|
||||
"-p", "IPv4",
|
||||
"--ip-protocol", "udp",
|
||||
"--ip-source-port", "67:68",
|
||||
"--ip-destination-port", "67:68",
|
||||
"--ip-source", ip,
|
||||
"-j", "DROP")
|
||||
}
|
||||
|
||||
func confineRules(iface, mac string) [][]string {
|
||||
return [][]string{
|
||||
{"-t", "nat", "PREROUTING", "--in-interface", iface, "-s", mac, "-j", "DROP"},
|
||||
{"-t", "filter", "FORWARD", "--out-interface", iface, "-s", mac, "-j", "DROP"},
|
||||
{"-t", "filter", "OUTPUT", "--out-interface", iface, "-s", mac, "-j", "DROP"},
|
||||
}
|
||||
}
|
||||
|
||||
func withAction(action string, rule []string) []string {
|
||||
return append([]string{rule[0], rule[1], action}, rule[2:]...)
|
||||
}
|
||||
|
||||
func ConfineMAC(iface, mac string) error {
|
||||
for _, rule := range confineRules(iface, mac) {
|
||||
if err := exec.Command("ebtables", withAction("-A", rule)...).Run(); err != nil {
|
||||
return fmt.Errorf("ebtables confine %s on %s (%s %s): %w", mac, iface, rule[1], rule[2], err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func DeleteConfineMAC(iface, mac string) error {
|
||||
var first error
|
||||
for _, rule := range confineRules(iface, mac) {
|
||||
if err := exec.Command("ebtables", withAction("-D", rule)...).Run(); err != nil && first == nil {
|
||||
first = fmt.Errorf("ebtables unconfine %s on %s (%s %s): %w", mac, iface, rule[1], rule[2], err)
|
||||
}
|
||||
}
|
||||
return first
|
||||
}
|
||||
|
||||
func gatewayRules(iface, ip string) [][]string {
|
||||
dhcp := []string{"-p", "IPv4", "--ip-protocol", "udp", "--ip-source-port", "67:68", "--ip-destination-port", "67:68", "--ip-source", ip, "-j", "DROP"}
|
||||
return [][]string{
|
||||
{"-t", "filter", "FORWARD", "--out-interface", iface, "-p", "arp", "--arp-op", "Request", "--arp-ip-dst", ip, "-j", "DROP"},
|
||||
{"-t", "filter", "OUTPUT", "--out-interface", iface, "-p", "arp", "--arp-ip-src", ip, "-j", "DROP"},
|
||||
append([]string{"-t", "filter", "FORWARD", "--out-interface", iface}, dhcp...),
|
||||
append([]string{"-t", "filter", "OUTPUT", "--out-interface", iface}, dhcp...),
|
||||
}
|
||||
}
|
||||
|
||||
func ruleExists(rule []string) (bool, error) {
|
||||
err := exec.Command("ebtables", withAction("--check", rule)...).Run()
|
||||
var exit *exec.ExitError
|
||||
switch {
|
||||
case err == nil:
|
||||
return true, nil
|
||||
case errors.As(err, &exit) && exit.ExitCode() == 1:
|
||||
return false, nil
|
||||
default:
|
||||
return false, fmt.Errorf("ebtables --check %s: %w", strings.Join(rule, " "), err)
|
||||
}
|
||||
}
|
||||
|
||||
func ConfineGateway(iface, ip string) error {
|
||||
for _, rule := range gatewayRules(iface, ip) {
|
||||
if err := exec.Command("ebtables", withAction("-A", rule)...).Run(); err != nil {
|
||||
return fmt.Errorf("ebtables confine gateway %s on %s (%s): %w", ip, iface, strings.Join(rule, " "), err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func DeleteConfineGateway(iface, ip string) error {
|
||||
var first error
|
||||
for _, rule := range gatewayRules(iface, ip) {
|
||||
exists, err := ruleExists(rule)
|
||||
if err == nil && exists {
|
||||
err = exec.Command("ebtables", withAction("-D", rule)...).Run()
|
||||
}
|
||||
if err != nil && first == nil {
|
||||
first = fmt.Errorf("ebtables unconfine gateway %s on %s (%s): %w", ip, iface, strings.Join(rule, " "), err)
|
||||
}
|
||||
}
|
||||
return first
|
||||
}
|
||||
|
||||
func MissingGatewayRules(iface, ip string) ([]string, error) {
|
||||
var missing []string
|
||||
for _, rule := range gatewayRules(iface, ip) {
|
||||
exists, err := ruleExists(rule)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !exists {
|
||||
missing = append(missing, strings.Join(rule, " "))
|
||||
}
|
||||
}
|
||||
return missing, nil
|
||||
}
|
||||
|
|
@ -1,70 +0,0 @@
|
|||
package ebtables
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestConfineRules(t *testing.T) {
|
||||
want := []string{
|
||||
"-t nat -A PREROUTING --in-interface v-s9r1-i -s 02:01:00:00:0b:5f -j DROP",
|
||||
"-t filter -A FORWARD --out-interface v-s9r1-i -s 02:01:00:00:0b:5f -j DROP",
|
||||
"-t filter -A OUTPUT --out-interface v-s9r1-i -s 02:01:00:00:0b:5f -j DROP",
|
||||
}
|
||||
rules := confineRules("v-s9r1-i", "02:01:00:00:0b:5f")
|
||||
if len(rules) != len(want) {
|
||||
t.Fatalf("%d règles, %d attendues", len(rules), len(want))
|
||||
}
|
||||
for i, rule := range rules {
|
||||
if got := strings.Join(withAction("-A", rule), " "); got != want[i] {
|
||||
t.Errorf("règle %d : %q, %q attendu", i, got, want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteUsesTheSameRules(t *testing.T) {
|
||||
for _, rule := range confineRules("v-x-i", "02:01:00:00:00:01") {
|
||||
added := withAction("-A", rule)
|
||||
deleted := withAction("-D", rule)
|
||||
if strings.Replace(strings.Join(added, " "), " -A ", " -D ", 1) != strings.Join(deleted, " ") {
|
||||
t.Errorf("ajout %v et suppression %v divergent", added, deleted)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWithActionDoesNotAlterTheRule(t *testing.T) {
|
||||
rule := confineRules("v-x-i", "02:01:00:00:00:01")[0]
|
||||
before := strings.Join(rule, " ")
|
||||
withAction("-A", rule)
|
||||
withAction("-D", rule)
|
||||
if strings.Join(rule, " ") != before {
|
||||
t.Errorf("la règle a été modifiée : %q devenu %q", before, strings.Join(rule, " "))
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayRules(t *testing.T) {
|
||||
want := []string{
|
||||
"-t filter -A FORWARD --out-interface v-t33a-i -p arp --arp-op Request --arp-ip-dst 10.33.0.1 -j DROP",
|
||||
"-t filter -A OUTPUT --out-interface v-t33a-i -p arp --arp-ip-src 10.33.0.1 -j DROP",
|
||||
"-t filter -A FORWARD --out-interface v-t33a-i -p IPv4 --ip-protocol udp --ip-source-port 67:68 --ip-destination-port 67:68 --ip-source 10.33.0.1 -j DROP",
|
||||
"-t filter -A OUTPUT --out-interface v-t33a-i -p IPv4 --ip-protocol udp --ip-source-port 67:68 --ip-destination-port 67:68 --ip-source 10.33.0.1 -j DROP",
|
||||
}
|
||||
rules := gatewayRules("v-t33a-i", "10.33.0.1")
|
||||
if len(rules) != len(want) {
|
||||
t.Fatalf("%d règles, %d attendues", len(rules), len(want))
|
||||
}
|
||||
for i, rule := range rules {
|
||||
if got := strings.Join(withAction("-A", rule), " "); got != want[i] {
|
||||
t.Errorf("règle %d : %q, %q attendu", i, got, want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestGatewayRulesDoNotShareSlices(t *testing.T) {
|
||||
rules := gatewayRules("v-x-i", "10.0.0.1")
|
||||
before := strings.Join(rules[2], " ")
|
||||
rules[3][len(rules[3])-1] = "ACCEPT"
|
||||
if strings.Join(rules[2], " ") != before {
|
||||
t.Errorf("les deux règles DHCP partagent leur tableau : %q", strings.Join(rules[2], " "))
|
||||
}
|
||||
}
|
||||
353
internal/nft/nft.go
Normal file
353
internal/nft/nft.go
Normal file
|
|
@ -0,0 +1,353 @@
|
|||
package nft
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const Table = "two"
|
||||
|
||||
type hook struct {
|
||||
name string
|
||||
chain string
|
||||
prefix string
|
||||
ports string
|
||||
key string
|
||||
priority string
|
||||
}
|
||||
|
||||
var hooks = []hook{
|
||||
{"prerouting", "prerouting", "pre-", "pre_ports", "iifname", "dstnat"},
|
||||
{"forward", "forward", "fwd-", "fwd_ports", "oifname", "filter"},
|
||||
{"output", "output", "out-", "out_ports", "oifname", "filter"},
|
||||
}
|
||||
|
||||
type Port struct {
|
||||
Name string
|
||||
Prerouting []string
|
||||
Forward []string
|
||||
Output []string
|
||||
}
|
||||
|
||||
func (p Port) rules(h hook) []string {
|
||||
switch h.name {
|
||||
case "prerouting":
|
||||
return p.Prerouting
|
||||
case "forward":
|
||||
return p.Forward
|
||||
default:
|
||||
return p.Output
|
||||
}
|
||||
}
|
||||
|
||||
func DropSourceMAC(mac string) string {
|
||||
return "ether saddr " + mac + " drop"
|
||||
}
|
||||
|
||||
func DropARPRequestTo(ip string) string {
|
||||
return "arp operation request arp daddr ip " + ip + " drop"
|
||||
}
|
||||
|
||||
func DropARPFrom(ip string) string {
|
||||
return "arp saddr ip " + ip + " drop"
|
||||
}
|
||||
|
||||
func DropDHCPFrom(ip string) string {
|
||||
return "ip saddr " + ip + " udp sport 67-68 udp dport 67-68 drop"
|
||||
}
|
||||
|
||||
func run(args []string, script string) ([]byte, error) {
|
||||
cmd := exec.Command("nft", args...)
|
||||
if script != "" {
|
||||
cmd.Stdin = strings.NewReader(script)
|
||||
}
|
||||
var stderr bytes.Buffer
|
||||
cmd.Stderr = &stderr
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("nft %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(stderr.String()))
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func apply(script string) error {
|
||||
_, err := run([]string{"-f", "-"}, script)
|
||||
return err
|
||||
}
|
||||
|
||||
func baseScript() string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "add table bridge %s\n", Table)
|
||||
for _, h := range hooks {
|
||||
fmt.Fprintf(&b, "add map bridge %s %s { type ifname : verdict; }\n", Table, h.ports)
|
||||
}
|
||||
for _, h := range hooks {
|
||||
fmt.Fprintf(&b, "add chain bridge %s %s { type filter hook %s priority %s; policy accept; }\n", Table, h.chain, h.name, h.priority)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func setScript(p Port, base table) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(baseScript())
|
||||
for _, h := range hooks {
|
||||
if !base.hasDispatch(h) {
|
||||
fmt.Fprintf(&b, "add rule bridge %s %s %s vmap @%s\n", Table, h.chain, h.key, h.ports)
|
||||
}
|
||||
}
|
||||
for _, h := range hooks {
|
||||
rules := p.rules(h)
|
||||
chain := h.prefix + p.Name
|
||||
if len(rules) == 0 {
|
||||
if _, ok := base.elements[h.ports][p.Name]; ok {
|
||||
fmt.Fprintf(&b, "delete element bridge %s %s { %q }\n", Table, h.ports, p.Name)
|
||||
}
|
||||
if base.chains[chain] {
|
||||
fmt.Fprintf(&b, "delete chain bridge %s %s\n", Table, chain)
|
||||
}
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, "add chain bridge %s %s\n", Table, chain)
|
||||
fmt.Fprintf(&b, "flush chain bridge %s %s\n", Table, chain)
|
||||
for _, r := range rules {
|
||||
fmt.Fprintf(&b, "add rule bridge %s %s %s\n", Table, chain, r)
|
||||
}
|
||||
target, ok := base.elements[h.ports][p.Name]
|
||||
if ok && target != chain {
|
||||
fmt.Fprintf(&b, "delete element bridge %s %s { %q }\n", Table, h.ports, p.Name)
|
||||
}
|
||||
if target != chain {
|
||||
fmt.Fprintf(&b, "add element bridge %s %s { %q : jump %s }\n", Table, h.ports, p.Name, chain)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func deleteScript(name string, t table) string {
|
||||
var b strings.Builder
|
||||
for _, h := range hooks {
|
||||
if _, ok := t.elements[h.ports][name]; ok {
|
||||
fmt.Fprintf(&b, "delete element bridge %s %s { %q }\n", Table, h.ports, name)
|
||||
}
|
||||
}
|
||||
for _, h := range hooks {
|
||||
if chain := h.prefix + name; t.chains[chain] {
|
||||
fmt.Fprintf(&b, "delete chain bridge %s %s\n", Table, chain)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func SetPort(p Port) error {
|
||||
t, err := readTable()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := apply(setScript(p, t)); err != nil {
|
||||
return fmt.Errorf("set port %s: %w", p.Name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func DeletePort(name string) error {
|
||||
t, err := readTable()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
script := deleteScript(name, t)
|
||||
if script == "" {
|
||||
return nil
|
||||
}
|
||||
if err := apply(script); err != nil {
|
||||
return fmt.Errorf("delete port %s: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func MissingRules(p Port) ([]string, error) {
|
||||
t, err := readTable()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
listings := map[string][]string{}
|
||||
for _, h := range hooks {
|
||||
chain := h.prefix + p.Name
|
||||
if len(p.rules(h)) == 0 || !t.chains[chain] {
|
||||
continue
|
||||
}
|
||||
out, err := run([]string{"list", "chain", "bridge", Table, chain}, "")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
listings[chain] = chainRules(string(out))
|
||||
}
|
||||
return missing(p, t, listings), nil
|
||||
}
|
||||
|
||||
func missing(p Port, t table, listings map[string][]string) []string {
|
||||
var out []string
|
||||
for _, h := range hooks {
|
||||
rules := p.rules(h)
|
||||
if len(rules) == 0 {
|
||||
continue
|
||||
}
|
||||
chain := h.prefix + p.Name
|
||||
if !t.hasDispatch(h) {
|
||||
out = append(out, fmt.Sprintf("%s: %s vmap @%s", h.chain, h.key, h.ports))
|
||||
}
|
||||
if t.elements[h.ports][p.Name] != chain {
|
||||
out = append(out, fmt.Sprintf("%s: %q : jump %s", h.ports, p.Name, chain))
|
||||
}
|
||||
want := map[string]bool{}
|
||||
for _, r := range rules {
|
||||
want[r] = true
|
||||
}
|
||||
have := map[string]bool{}
|
||||
for _, r := range listings[chain] {
|
||||
have[r] = true
|
||||
if !want[r] {
|
||||
out = append(out, chain+": unexpected "+r)
|
||||
}
|
||||
}
|
||||
for _, r := range rules {
|
||||
if !have[r] {
|
||||
out = append(out, chain+": "+r)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func chainRules(listing string) []string {
|
||||
var rules []string
|
||||
depth := 0
|
||||
for _, line := range strings.Split(listing, "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
switch {
|
||||
case strings.HasSuffix(line, "{"):
|
||||
depth++
|
||||
case line == "}":
|
||||
depth--
|
||||
case depth == 2 && line != "":
|
||||
rules = append(rules, line)
|
||||
}
|
||||
}
|
||||
return rules
|
||||
}
|
||||
|
||||
type table struct {
|
||||
exists bool
|
||||
chains map[string]bool
|
||||
elements map[string]map[string]string
|
||||
dispatch map[string]string
|
||||
}
|
||||
|
||||
func (t table) hasDispatch(h hook) bool {
|
||||
return t.dispatch[h.chain] == h.key+" @"+h.ports
|
||||
}
|
||||
|
||||
type listing struct {
|
||||
Nftables []struct {
|
||||
Table *struct {
|
||||
Family string `json:"family"`
|
||||
Name string `json:"name"`
|
||||
} `json:"table"`
|
||||
Chain *struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"chain"`
|
||||
Map *struct {
|
||||
Name string `json:"name"`
|
||||
Elem []json.RawMessage `json:"elem"`
|
||||
} `json:"map"`
|
||||
Rule *struct {
|
||||
Chain string `json:"chain"`
|
||||
Expr []struct {
|
||||
Vmap *struct {
|
||||
Key struct {
|
||||
Meta struct {
|
||||
Key string `json:"key"`
|
||||
} `json:"meta"`
|
||||
} `json:"key"`
|
||||
Data string `json:"data"`
|
||||
} `json:"vmap"`
|
||||
} `json:"expr"`
|
||||
} `json:"rule"`
|
||||
} `json:"nftables"`
|
||||
}
|
||||
|
||||
func readTable() (table, error) {
|
||||
out, err := run([]string{"-j", "list", "tables", "bridge"}, "")
|
||||
if err != nil {
|
||||
return table{}, err
|
||||
}
|
||||
tables, err := parseListing(out)
|
||||
if err != nil || !tables.exists {
|
||||
return tables, err
|
||||
}
|
||||
out, err = run([]string{"-j", "list", "table", "bridge", Table}, "")
|
||||
if err != nil {
|
||||
return table{}, err
|
||||
}
|
||||
return parseListing(out)
|
||||
}
|
||||
|
||||
func parseListing(data []byte) (table, error) {
|
||||
var l listing
|
||||
if err := json.Unmarshal(data, &l); err != nil {
|
||||
return table{}, fmt.Errorf("nft json: %w", err)
|
||||
}
|
||||
t := table{chains: map[string]bool{}, elements: map[string]map[string]string{}, dispatch: map[string]string{}}
|
||||
for _, o := range l.Nftables {
|
||||
switch {
|
||||
case o.Table != nil:
|
||||
if o.Table.Family == "bridge" && o.Table.Name == Table {
|
||||
t.exists = true
|
||||
}
|
||||
case o.Chain != nil:
|
||||
t.chains[o.Chain.Name] = true
|
||||
case o.Map != nil:
|
||||
elems := map[string]string{}
|
||||
for _, raw := range o.Map.Elem {
|
||||
name, target, err := parseElem(raw)
|
||||
if err != nil {
|
||||
return table{}, fmt.Errorf("nft json: map %s: %w", o.Map.Name, err)
|
||||
}
|
||||
elems[name] = target
|
||||
}
|
||||
t.elements[o.Map.Name] = elems
|
||||
case o.Rule != nil:
|
||||
for _, e := range o.Rule.Expr {
|
||||
if e.Vmap != nil {
|
||||
t.dispatch[o.Rule.Chain] = e.Vmap.Key.Meta.Key + " " + e.Vmap.Data
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
|
||||
func parseElem(raw json.RawMessage) (string, string, error) {
|
||||
var pair []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &pair); err != nil || len(pair) != 2 {
|
||||
return "", "", fmt.Errorf("element %s: not a key and verdict pair", raw)
|
||||
}
|
||||
var name string
|
||||
if err := json.Unmarshal(pair[0], &name); err != nil {
|
||||
return "", "", fmt.Errorf("element %s: key: %w", raw, err)
|
||||
}
|
||||
var verdict struct {
|
||||
Jump *struct {
|
||||
Target string `json:"target"`
|
||||
} `json:"jump"`
|
||||
}
|
||||
if err := json.Unmarshal(pair[1], &verdict); err != nil {
|
||||
return "", "", fmt.Errorf("element %s: verdict: %w", raw, err)
|
||||
}
|
||||
if verdict.Jump == nil {
|
||||
return name, "", nil
|
||||
}
|
||||
return name, verdict.Jump.Target, nil
|
||||
}
|
||||
217
internal/nft/nft_test.go
Normal file
217
internal/nft/nft_test.go
Normal file
|
|
@ -0,0 +1,217 @@
|
|||
package nft
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
const listedTable = `{"nftables": [{"metainfo": {"version": "1.0.6", "release_name": "Lester Gooch #5", "json_schema_version": 1}}, {"table": {"family": "bridge", "name": "two", "handle": 1}}, {"map": {"family": "bridge", "name": "pre_ports", "table": "two", "type": "ifname", "handle": 1, "map": "verdict"}}, {"map": {"family": "bridge", "name": "fwd_ports", "table": "two", "type": "ifname", "handle": 2, "map": "verdict", "elem": [["v-t33a-i", {"jump": {"target": "fwd-v-t33a-i"}}], ["v-other-i", {"jump": {"target": "fwd-v-other-i"}}]]}}, {"map": {"family": "bridge", "name": "out_ports", "table": "two", "type": "ifname", "handle": 3, "map": "verdict", "elem": [["v-t33a-i", {"jump": {"target": "out-v-t33a-i"}}]]}}, {"chain": {"family": "bridge", "table": "two", "name": "prerouting", "handle": 4, "type": "filter", "hook": "prerouting", "prio": -300, "policy": "accept"}}, {"chain": {"family": "bridge", "table": "two", "name": "forward", "handle": 5, "type": "filter", "hook": "forward", "prio": -200, "policy": "accept"}}, {"chain": {"family": "bridge", "table": "two", "name": "output", "handle": 6, "type": "filter", "hook": "output", "prio": -200, "policy": "accept"}}, {"chain": {"family": "bridge", "table": "two", "name": "fwd-v-t33a-i", "handle": 10}}, {"chain": {"family": "bridge", "table": "two", "name": "out-v-t33a-i", "handle": 11}}, {"chain": {"family": "bridge", "table": "two", "name": "fwd-v-other-i", "handle": 12}}, {"rule": {"family": "bridge", "table": "two", "chain": "prerouting", "handle": 7, "expr": [{"vmap": {"key": {"meta": {"key": "iifname"}}, "data": "@pre_ports"}}]}}, {"rule": {"family": "bridge", "table": "two", "chain": "forward", "handle": 8, "expr": [{"vmap": {"key": {"meta": {"key": "oifname"}}, "data": "@fwd_ports"}}]}}, {"rule": {"family": "bridge", "table": "two", "chain": "output", "handle": 9, "expr": [{"vmap": {"key": {"meta": {"key": "oifname"}}, "data": "@out_ports"}}]}}]}`
|
||||
|
||||
const noTable = `{"nftables": [{"metainfo": {"version": "1.0.6", "release_name": "Lester Gooch #5", "json_schema_version": 1}}]}`
|
||||
|
||||
const listedChain = `table bridge two {
|
||||
chain fwd-v-t33a-i {
|
||||
arp operation request arp daddr ip 10.33.0.1 drop
|
||||
ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop
|
||||
}
|
||||
}
|
||||
`
|
||||
|
||||
func bridgePort() Port {
|
||||
return Port{
|
||||
Name: "v-t33a-i",
|
||||
Forward: []string{"arp operation request arp daddr ip 10.33.0.1 drop", "ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop"},
|
||||
Output: []string{"arp saddr ip 10.33.0.1 drop", "ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop"},
|
||||
}
|
||||
}
|
||||
|
||||
func mustParse(t *testing.T, data string) table {
|
||||
t.Helper()
|
||||
tb, err := parseListing([]byte(data))
|
||||
if err != nil {
|
||||
t.Fatalf("parseListing: %v", err)
|
||||
}
|
||||
return tb
|
||||
}
|
||||
|
||||
func TestRuleTexts(t *testing.T) {
|
||||
for got, want := range map[string]string{
|
||||
DropSourceMAC("02:01:00:00:0b:5f"): "ether saddr 02:01:00:00:0b:5f drop",
|
||||
DropARPRequestTo("10.33.0.1"): "arp operation request arp daddr ip 10.33.0.1 drop",
|
||||
DropARPFrom("10.33.0.1"): "arp saddr ip 10.33.0.1 drop",
|
||||
DropDHCPFrom("10.33.0.1"): "ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop",
|
||||
} {
|
||||
if got != want {
|
||||
t.Errorf("%q, %q attendu", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetScriptOnAnEmptyNetns(t *testing.T) {
|
||||
want := `add table bridge two
|
||||
add map bridge two pre_ports { type ifname : verdict; }
|
||||
add map bridge two fwd_ports { type ifname : verdict; }
|
||||
add map bridge two out_ports { type ifname : verdict; }
|
||||
add chain bridge two prerouting { type filter hook prerouting priority dstnat; policy accept; }
|
||||
add chain bridge two forward { type filter hook forward priority filter; policy accept; }
|
||||
add chain bridge two output { type filter hook output priority filter; policy accept; }
|
||||
add rule bridge two prerouting iifname vmap @pre_ports
|
||||
add rule bridge two forward oifname vmap @fwd_ports
|
||||
add rule bridge two output oifname vmap @out_ports
|
||||
add chain bridge two fwd-v-t33a-i
|
||||
flush chain bridge two fwd-v-t33a-i
|
||||
add rule bridge two fwd-v-t33a-i arp operation request arp daddr ip 10.33.0.1 drop
|
||||
add rule bridge two fwd-v-t33a-i ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop
|
||||
add element bridge two fwd_ports { "v-t33a-i" : jump fwd-v-t33a-i }
|
||||
add chain bridge two out-v-t33a-i
|
||||
flush chain bridge two out-v-t33a-i
|
||||
add rule bridge two out-v-t33a-i arp saddr ip 10.33.0.1 drop
|
||||
add rule bridge two out-v-t33a-i ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop
|
||||
add element bridge two out_ports { "v-t33a-i" : jump out-v-t33a-i }
|
||||
`
|
||||
if got := setScript(bridgePort(), mustParse(t, noTable)); got != want {
|
||||
t.Errorf("script :\n%s\nattendu :\n%s", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetScriptAgainKeepsDispatchAndElements(t *testing.T) {
|
||||
got := setScript(bridgePort(), mustParse(t, listedTable))
|
||||
for _, absent := range []string{"vmap", "add element", "delete"} {
|
||||
if strings.Contains(got, absent) {
|
||||
t.Errorf("%q présent alors que la table est déjà en place :\n%s", absent, got)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(got, "flush chain bridge two fwd-v-t33a-i\n") {
|
||||
t.Errorf("la chaîne n'est pas vidée avant d'être réécrite :\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetScriptReplacesAnElementPointingElsewhere(t *testing.T) {
|
||||
data := strings.Replace(listedTable, `["v-t33a-i", {"jump": {"target": "fwd-v-t33a-i"}}]`, `["v-t33a-i", {"jump": {"target": "fwd-v-other-i"}}]`, 1)
|
||||
got := setScript(bridgePort(), mustParse(t, data))
|
||||
want := "delete element bridge two fwd_ports { \"v-t33a-i\" }\nadd element bridge two fwd_ports { \"v-t33a-i\" : jump fwd-v-t33a-i }\n"
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("élément non remplacé :\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetScriptRemovesAHookThePortNoLongerUses(t *testing.T) {
|
||||
p := bridgePort()
|
||||
p.Output = nil
|
||||
got := setScript(p, mustParse(t, listedTable))
|
||||
want := "delete element bridge two out_ports { \"v-t33a-i\" }\ndelete chain bridge two out-v-t33a-i\n"
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("crochet output non retiré :\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "pre-v-t33a-i") {
|
||||
t.Errorf("chaîne prerouting touchée alors qu'elle n'existe pas :\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteScriptRemovesElementsBeforeChains(t *testing.T) {
|
||||
want := `delete element bridge two fwd_ports { "v-t33a-i" }
|
||||
delete element bridge two out_ports { "v-t33a-i" }
|
||||
delete chain bridge two fwd-v-t33a-i
|
||||
delete chain bridge two out-v-t33a-i
|
||||
`
|
||||
if got := deleteScript("v-t33a-i", mustParse(t, listedTable)); got != want {
|
||||
t.Errorf("script :\n%s\nattendu :\n%s", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteScriptLeavesOtherPorts(t *testing.T) {
|
||||
got := deleteScript("v-t33a-i", mustParse(t, listedTable))
|
||||
if strings.Contains(got, "v-other-i") {
|
||||
t.Errorf("un autre port est touché :\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteScriptIsEmptyForAnAbsentPort(t *testing.T) {
|
||||
for name, data := range map[string]string{"sans table": noTable, "port inconnu": listedTable} {
|
||||
if got := deleteScript("v-gone-i", mustParse(t, data)); got != "" {
|
||||
t.Errorf("%s : script %q, vide attendu", name, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseListing(t *testing.T) {
|
||||
tb := mustParse(t, listedTable)
|
||||
if !tb.exists {
|
||||
t.Errorf("table two non vue")
|
||||
}
|
||||
if got := tb.elements["fwd_ports"]["v-t33a-i"]; got != "fwd-v-t33a-i" {
|
||||
t.Errorf("fwd_ports[v-t33a-i] = %q", got)
|
||||
}
|
||||
if got := len(tb.elements["pre_ports"]); got != 0 {
|
||||
t.Errorf("pre_ports : %d éléments, 0 attendu", got)
|
||||
}
|
||||
for _, c := range []string{"prerouting", "forward", "output", "fwd-v-t33a-i", "out-v-t33a-i", "fwd-v-other-i"} {
|
||||
if !tb.chains[c] {
|
||||
t.Errorf("chaîne %s non vue", c)
|
||||
}
|
||||
}
|
||||
for _, h := range hooks {
|
||||
if !tb.hasDispatch(h) {
|
||||
t.Errorf("aiguillage de %s non vu : %q", h.chain, tb.dispatch[h.chain])
|
||||
}
|
||||
}
|
||||
if mustParse(t, noTable).exists {
|
||||
t.Errorf("table vue alors qu'absente")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseListingRejectsAMalformedElement(t *testing.T) {
|
||||
data := strings.Replace(listedTable, `["v-other-i", {"jump": {"target": "fwd-v-other-i"}}]`, `"v-other-i"`, 1)
|
||||
if _, err := parseListing([]byte(data)); err == nil || !strings.Contains(err.Error(), "map fwd_ports") {
|
||||
t.Errorf("erreur = %v, erreur sur fwd_ports attendue", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestChainRules(t *testing.T) {
|
||||
got := chainRules(listedChain)
|
||||
want := []string{"arp operation request arp daddr ip 10.33.0.1 drop", "ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop"}
|
||||
if strings.Join(got, "|") != strings.Join(want, "|") {
|
||||
t.Errorf("règles %q, %q attendues", got, want)
|
||||
}
|
||||
if got := chainRules("table bridge two {\n\tchain fwd-b {\n\t}\n}\n"); len(got) != 0 {
|
||||
t.Errorf("chaîne vide : %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func healthyListings() map[string][]string {
|
||||
return map[string][]string{
|
||||
"fwd-v-t33a-i": chainRules(listedChain),
|
||||
"out-v-t33a-i": {"arp saddr ip 10.33.0.1 drop", "ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop"},
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingIsEmptyWhenEverythingIsInPlace(t *testing.T) {
|
||||
if got := missing(bridgePort(), mustParse(t, listedTable), healthyListings()); len(got) != 0 {
|
||||
t.Errorf("manques %q, aucun attendu", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingReportsEachDefect(t *testing.T) {
|
||||
data := strings.Replace(listedTable, `, "elem": [["v-t33a-i", {"jump": {"target": "out-v-t33a-i"}}]]`, "", 1)
|
||||
data = strings.Replace(data, `"data": "@fwd_ports"`, `"data": "@pre_ports"`, 1)
|
||||
listings := healthyListings()
|
||||
listings["fwd-v-t33a-i"] = []string{"arp operation request arp daddr ip 10.33.0.1 drop", "accept"}
|
||||
got := strings.Join(missing(bridgePort(), mustParse(t, data), listings), "\n")
|
||||
for _, want := range []string{
|
||||
"forward: oifname vmap @fwd_ports",
|
||||
`out_ports: "v-t33a-i" : jump out-v-t33a-i`,
|
||||
"fwd-v-t33a-i: ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop",
|
||||
"fwd-v-t33a-i: unexpected accept",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("%q absent de :\n%s", want, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingReportsEverythingWithoutTable(t *testing.T) {
|
||||
got := missing(bridgePort(), mustParse(t, noTable), nil)
|
||||
if len(got) != 8 {
|
||||
t.Errorf("%d manques, 8 attendus (2 aiguillages, 2 éléments, 4 règles) : %q", len(got), got)
|
||||
}
|
||||
}
|
||||
|
|
@ -3,11 +3,10 @@ package subnet
|
|||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
|
||||
"git.g3e.fr/syonad/two/internal/ebtables"
|
||||
"git.g3e.fr/syonad/two/internal/netif"
|
||||
"git.g3e.fr/syonad/two/internal/netns"
|
||||
"git.g3e.fr/syonad/two/internal/nft"
|
||||
"git.g3e.fr/syonad/two/internal/sysctl"
|
||||
)
|
||||
|
||||
|
|
@ -32,7 +31,9 @@ func setupAnycastGateway(d subnetData, vethI string) error {
|
|||
name string
|
||||
fn func() error
|
||||
}{
|
||||
{"confine " + mac.String(), func() error { return ebtables.ConfineMAC(vethI, mac.String()) }},
|
||||
{"filter " + vethI, func() error {
|
||||
return nft.SetPort(AnycastFabricPort(vethI, mac.String(), d.interfaceIP.String(), d.relayIP.String()))
|
||||
}},
|
||||
{"arp sysctls on " + d.bridge, func() error { return setSysctls(arpSysctls(d.bridge)) }},
|
||||
{"relay address on " + d.bridge, func() error { return netif.AddrAdd(d.bridge, d.relayIP) }},
|
||||
{"create " + gw, func() error { return netif.CreateMacvlan(gw, d.bridge, mac) }},
|
||||
|
|
@ -41,10 +42,6 @@ func setupAnycastGateway(d subnetData, vethI string) error {
|
|||
{"gateway address on " + gw, func() error { return netif.AddrAdd(gw, d.interfaceIP) }},
|
||||
{"route " + d.cidr.String(), func() error { return netif.RouteAddFrom(d.bridge, d.cidr, d.interfaceIP) }},
|
||||
{"set up " + gw, func() error { return netif.LinkSetUp(gw) }},
|
||||
{"drop arp to gateway", func() error { return ebtables.DropARPToGateway(vethI, d.interfaceIP.String()) }},
|
||||
{"drop arp to relay", func() error { return ebtables.DropARPToGateway(vethI, d.relayIP.String()) }},
|
||||
{"drop dhcp from gateway", func() error { return ebtables.DropDHCP(vethI, d.interfaceIP.String()) }},
|
||||
{"drop dhcp from relay", func() error { return ebtables.DropDHCP(vethI, d.relayIP.String()) }},
|
||||
}
|
||||
return netns.Call(d.vpc, func() error {
|
||||
for _, step := range steps {
|
||||
|
|
@ -57,21 +54,9 @@ func setupAnycastGateway(d subnetData, vethI string) error {
|
|||
}
|
||||
|
||||
func deleteAnycastGateway(d subnetData, vethI string) error {
|
||||
mac, err := netif.AnycastMAC(d.vxlanID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return netns.Call(d.vpc, func() error {
|
||||
var errs []error
|
||||
for _, ip := range []net.IP{d.interfaceIP, d.relayIP} {
|
||||
if err := ebtables.DeleteARPToGateway(vethI, ip.String()); err != nil {
|
||||
errs = append(errs, fmt.Errorf("delete ebtables arp rule %s: %w", ip, err))
|
||||
}
|
||||
if err := ebtables.DeleteDHCP(vethI, ip.String()); err != nil {
|
||||
errs = append(errs, fmt.Errorf("delete ebtables dhcp rule %s: %w", ip, err))
|
||||
}
|
||||
}
|
||||
if err := ebtables.DeleteConfineMAC(vethI, mac.String()); err != nil {
|
||||
if err := nft.DeletePort(vethI); err != nil {
|
||||
errs = append(errs, err)
|
||||
}
|
||||
if err := netif.DeleteLink(d.gatewayLink()); err != nil {
|
||||
|
|
|
|||
|
|
@ -6,10 +6,10 @@ import (
|
|||
|
||||
configuration "git.g3e.fr/syonad/two/internal/config/agent"
|
||||
"git.g3e.fr/syonad/two/internal/dhcpbackend"
|
||||
"git.g3e.fr/syonad/two/internal/ebtables"
|
||||
"git.g3e.fr/syonad/two/internal/l3vni"
|
||||
"git.g3e.fr/syonad/two/internal/netif"
|
||||
"git.g3e.fr/syonad/two/internal/netns"
|
||||
"git.g3e.fr/syonad/two/internal/nft"
|
||||
"git.g3e.fr/syonad/two/internal/state"
|
||||
|
||||
"github.com/dgraph-io/badger/v4"
|
||||
|
|
@ -118,10 +118,7 @@ func createSubnet(db *badger.DB, subnetName string, d subnetData, backend dhcpba
|
|||
if err := netif.RouteAdd(d.bridge, d.cidr); err != nil {
|
||||
return fmt.Errorf("add route: %w", err)
|
||||
}
|
||||
if err := ebtables.DropARPToGateway(vethI, d.interfaceIP.String()); err != nil {
|
||||
return err
|
||||
}
|
||||
return ebtables.DropDHCP(vethI, d.interfaceIP.String())
|
||||
return nft.SetPort(VxlanFabricPort(vethI, d.interfaceIP.String()))
|
||||
}); err != nil {
|
||||
return fmt.Errorf("configure netns: %w", err)
|
||||
}
|
||||
|
|
@ -133,7 +130,7 @@ func createSubnet(db *badger.DB, subnetName string, d subnetData, backend dhcpba
|
|||
if err := netif.RouteAdd(d.bridge, d.cidr); err != nil {
|
||||
return fmt.Errorf("add route: %w", err)
|
||||
}
|
||||
return ebtables.ConfineGateway(vethI, d.interfaceIP.String())
|
||||
return nft.SetPort(BridgeFabricPort(vethI, d.interfaceIP.String()))
|
||||
}); err != nil {
|
||||
return fmt.Errorf("configure netns: %w", err)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,10 +5,10 @@ import (
|
|||
|
||||
configuration "git.g3e.fr/syonad/two/internal/config/agent"
|
||||
"git.g3e.fr/syonad/two/internal/dhcpbackend"
|
||||
"git.g3e.fr/syonad/two/internal/ebtables"
|
||||
"git.g3e.fr/syonad/two/internal/l3vni"
|
||||
"git.g3e.fr/syonad/two/internal/netif"
|
||||
"git.g3e.fr/syonad/two/internal/netns"
|
||||
"git.g3e.fr/syonad/two/internal/nft"
|
||||
"git.g3e.fr/syonad/two/internal/state"
|
||||
|
||||
"github.com/dgraph-io/badger/v4"
|
||||
|
|
@ -76,11 +76,8 @@ func deleteSubnetVxlan(d subnetData) error {
|
|||
return fmt.Errorf("delete netns resources: %w", err)
|
||||
}
|
||||
} else if err := netns.Call(d.vpc, func() error {
|
||||
if err := ebtables.DeleteARPToGateway(vethI, d.interfaceIP.String()); err != nil {
|
||||
return fmt.Errorf("delete ebtables arp rule: %w", err)
|
||||
}
|
||||
if err := ebtables.DeleteDHCP(vethI, d.interfaceIP.String()); err != nil {
|
||||
return fmt.Errorf("delete ebtables dhcp rule: %w", err)
|
||||
if err := nft.DeletePort(vethI); err != nil {
|
||||
return err
|
||||
}
|
||||
return netif.DeleteLink(d.bridge)
|
||||
}); err != nil {
|
||||
|
|
@ -105,8 +102,8 @@ func deleteSubnetBridge(d subnetData) error {
|
|||
vethI := "v-" + d.subnetID + "-i"
|
||||
|
||||
if err := netns.Call(d.vpc, func() error {
|
||||
if err := ebtables.DeleteConfineGateway(vethI, d.interfaceIP.String()); err != nil {
|
||||
return fmt.Errorf("delete ebtables gateway rules: %w", err)
|
||||
if err := nft.DeletePort(vethI); err != nil {
|
||||
return err
|
||||
}
|
||||
return netif.DeleteLink(d.bridge)
|
||||
}); err != nil {
|
||||
|
|
|
|||
35
internal/subnet/fabric.go
Normal file
35
internal/subnet/fabric.go
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
package subnet
|
||||
|
||||
import (
|
||||
"git.g3e.fr/syonad/two/internal/nft"
|
||||
)
|
||||
|
||||
func VxlanFabricPort(vethI, gatewayIP string) nft.Port {
|
||||
return nft.Port{
|
||||
Name: vethI,
|
||||
Forward: []string{nft.DropARPRequestTo(gatewayIP), nft.DropDHCPFrom(gatewayIP)},
|
||||
}
|
||||
}
|
||||
|
||||
func AnycastFabricPort(vethI, mac, gatewayIP, relayIP string) nft.Port {
|
||||
return nft.Port{
|
||||
Name: vethI,
|
||||
Prerouting: []string{nft.DropSourceMAC(mac)},
|
||||
Forward: []string{
|
||||
nft.DropSourceMAC(mac),
|
||||
nft.DropARPRequestTo(gatewayIP),
|
||||
nft.DropARPRequestTo(relayIP),
|
||||
nft.DropDHCPFrom(gatewayIP),
|
||||
nft.DropDHCPFrom(relayIP),
|
||||
},
|
||||
Output: []string{nft.DropSourceMAC(mac)},
|
||||
}
|
||||
}
|
||||
|
||||
func BridgeFabricPort(vethI, gatewayIP string) nft.Port {
|
||||
return nft.Port{
|
||||
Name: vethI,
|
||||
Forward: []string{nft.DropARPRequestTo(gatewayIP), nft.DropDHCPFrom(gatewayIP)},
|
||||
Output: []string{nft.DropARPFrom(gatewayIP), nft.DropDHCPFrom(gatewayIP)},
|
||||
}
|
||||
}
|
||||
52
internal/subnet/fabric_test.go
Normal file
52
internal/subnet/fabric_test.go
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
package subnet
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func assertRules(t *testing.T, hook string, got, want []string) {
|
||||
t.Helper()
|
||||
if strings.Join(got, "\n") != strings.Join(want, "\n") {
|
||||
t.Errorf("%s : %q, %q attendu", hook, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVxlanFabricPort(t *testing.T) {
|
||||
p := VxlanFabricPort("v-s9r1-i", "10.9.1.254")
|
||||
if p.Name != "v-s9r1-i" {
|
||||
t.Errorf("port %q", p.Name)
|
||||
}
|
||||
assertRules(t, "prerouting", p.Prerouting, nil)
|
||||
assertRules(t, "forward", p.Forward, []string{
|
||||
"arp operation request arp daddr ip 10.9.1.254 drop",
|
||||
"ip saddr 10.9.1.254 udp sport 67-68 udp dport 67-68 drop",
|
||||
})
|
||||
assertRules(t, "output", p.Output, nil)
|
||||
}
|
||||
|
||||
func TestAnycastFabricPort(t *testing.T) {
|
||||
p := AnycastFabricPort("v-s9r1-i", "02:01:00:00:0b:5f", "10.9.1.254", "10.9.1.253")
|
||||
assertRules(t, "prerouting", p.Prerouting, []string{"ether saddr 02:01:00:00:0b:5f drop"})
|
||||
assertRules(t, "forward", p.Forward, []string{
|
||||
"ether saddr 02:01:00:00:0b:5f drop",
|
||||
"arp operation request arp daddr ip 10.9.1.254 drop",
|
||||
"arp operation request arp daddr ip 10.9.1.253 drop",
|
||||
"ip saddr 10.9.1.254 udp sport 67-68 udp dport 67-68 drop",
|
||||
"ip saddr 10.9.1.253 udp sport 67-68 udp dport 67-68 drop",
|
||||
})
|
||||
assertRules(t, "output", p.Output, []string{"ether saddr 02:01:00:00:0b:5f drop"})
|
||||
}
|
||||
|
||||
func TestBridgeFabricPort(t *testing.T) {
|
||||
p := BridgeFabricPort("v-t33a-i", "10.33.0.1")
|
||||
assertRules(t, "prerouting", p.Prerouting, nil)
|
||||
assertRules(t, "forward", p.Forward, []string{
|
||||
"arp operation request arp daddr ip 10.33.0.1 drop",
|
||||
"ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop",
|
||||
})
|
||||
assertRules(t, "output", p.Output, []string{
|
||||
"arp saddr ip 10.33.0.1 drop",
|
||||
"ip saddr 10.33.0.1 udp sport 67-68 udp dport 67-68 drop",
|
||||
})
|
||||
}
|
||||
|
|
@ -3,8 +3,9 @@ package watchdog
|
|||
import (
|
||||
"fmt"
|
||||
|
||||
"git.g3e.fr/syonad/two/internal/ebtables"
|
||||
"git.g3e.fr/syonad/two/internal/netns"
|
||||
"git.g3e.fr/syonad/two/internal/nft"
|
||||
"git.g3e.fr/syonad/two/internal/subnet"
|
||||
"git.g3e.fr/syonad/two/internal/watchdog/notify"
|
||||
"git.g3e.fr/syonad/two/pkg/db/kv"
|
||||
|
||||
|
|
@ -21,13 +22,13 @@ func checkBridgeGateway(db *badger.DB, name, vpc, nsVeth string, n notify.Notifi
|
|||
return
|
||||
}
|
||||
if err := netns.Call(vpc, func() error {
|
||||
missing, err := ebtables.MissingGatewayRules(nsVeth, gatewayIP)
|
||||
missing, err := nft.MissingRules(subnet.BridgeFabricPort(nsVeth, gatewayIP))
|
||||
if err != nil {
|
||||
n.Notify(kindSubnet, name, fmt.Sprintf("gateway ebtables rules unreadable in netns %s: %v", vpc, err))
|
||||
n.Notify(kindSubnet, name, fmt.Sprintf("gateway nftables rules unreadable in netns %s: %v", vpc, err))
|
||||
return nil
|
||||
}
|
||||
for _, rule := range missing {
|
||||
n.Notify(kindSubnet, name, fmt.Sprintf("ebtables rule missing in netns %s: %s", vpc, rule))
|
||||
n.Notify(kindSubnet, name, fmt.Sprintf("nftables rule missing in netns %s: %s", vpc, rule))
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
|
|
|
|||
|
|
@ -40,6 +40,11 @@
|
|||
|
||||
## Mise à jour depuis la 0.2.0
|
||||
|
||||
- **Les règles de pont passent d'ebtables à nftables (#76)** : l'agent appelle `nft` et pose ses
|
||||
règles dans une table `bridge two` du netns de chaque VPC. `bootstrap_kvm.sh` installe
|
||||
`nftables` à la place d'`ebtables`. Les règles ebtables des subnets existants ne sont ni
|
||||
reprises ni retirées : vider l'hyperviseur (VM, subnets, VPC) avant la mise à jour, puis les
|
||||
recréer
|
||||
- **`default_route: true` exige une `gateway` hors mode `bridge`** : un appel qui l'omettait en
|
||||
`vxlan` reçoit désormais un 400. Les subnets existants gardent la route par défaut qu'ils
|
||||
annonçaient jusqu'à leur recréation
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
# L'host est stateless (root sur tmpfs) : rien de ce qui suit ne survit à un
|
||||
# reboot, deploy.sh --bootstrap est rejoué à chaque démarrage.
|
||||
|
||||
KVM_PACKAGES="qemu-system-x86 qemu-utils ovmf dnsmasq ebtables iptables nfs-common jq curl"
|
||||
KVM_PACKAGES="qemu-system-x86 qemu-utils ovmf dnsmasq nftables iptables nfs-common jq curl"
|
||||
|
||||
kvm_packages () {
|
||||
local DRY_RUN="${1}"
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ host_state () {
|
|||
echo "## liens"; ip -o link show | awk -F': ' '{print $2}' | cut -d@ -f1 | sort
|
||||
echo "## adresses"; ip -o -4 addr show | awk '{print $2, $4}' | sort
|
||||
echo "## routes"; ip -4 route show table all | grep -v '^local \|^broadcast ' | sort
|
||||
echo "## ebtables"; ebtables -t filter -L; ebtables -t nat -L
|
||||
echo "## nftables bridge"; nft list ruleset bridge
|
||||
echo "## iptables"; iptables-save | grep -v '^#' | sed 's/ \[[0-9]*:[0-9]*\]$//'
|
||||
echo "## frr"; vtysh -c 'show running-config' | grep -v '^Building\|^Current configuration\|^!'
|
||||
echo "## frr.conf"; sha256sum /etc/frr/frr.conf | cut -d' ' -f1
|
||||
|
|
|
|||
|
|
@ -180,24 +180,37 @@ vm_sniff_start () {
|
|||
sleep 3
|
||||
}
|
||||
|
||||
ebt_mac () {
|
||||
awk -F: 'BEGIN {OFS = ":"} {for (i = 1; i <= NF; i++) sub(/^0/, "", $i); $1 = $1; print tolower($0)}' <<<"${1}"
|
||||
nft_table () {
|
||||
local tables
|
||||
tables=$(ip netns exec "${1}" nft list tables bridge) || return 1
|
||||
grep -qx 'table bridge two' <<<"${tables}" || return 0
|
||||
ip netns exec "${1}" nft list table bridge two
|
||||
}
|
||||
|
||||
ebt_rules () {
|
||||
{ ip netns exec "${1}" ebtables -t nat -L; ip netns exec "${1}" ebtables -t filter -L; } | grep -- "${2}"
|
||||
nft_rules () {
|
||||
local table
|
||||
table=$(nft_table "${1}") || return 1
|
||||
awk -v p="${2}" '$1 == "chain" {c = ($2 == "pre-" p || $2 == "fwd-" p || $2 == "out-" p) ? $2 : ""; next} $1 == "}" {c = ""; next} c != "" && NF {$1 = $1; print c, $0}' <<<"${table}"
|
||||
}
|
||||
|
||||
ebt_confined () {
|
||||
ebt_rules "${1}" "${2}" | grep -ciE -- "-s ($(ebt_mac "${3}")|${3}) .*DROP"
|
||||
nft_count () {
|
||||
nft_rules "${1}" "${2}" | cut -d' ' -f2- | grep -cxF -- "${3}"
|
||||
}
|
||||
|
||||
ebt_arp_drop () {
|
||||
ebt_rules "${1}" "${2}" | grep -i arp | grep -i request | grep -q -- "${3} "
|
||||
nft_confined () {
|
||||
nft_count "${1}" "${2}" "ether saddr $(tr 'A-F' 'a-f' <<<"${3}") drop"
|
||||
}
|
||||
|
||||
ebt_dhcp_drop () {
|
||||
ebt_rules "${1}" "${2}" | grep -i udp | grep -- "${3} " | grep -q "67:68"
|
||||
nft_arp_drop () {
|
||||
[ "$(nft_count "${1}" "${2}" "arp operation request arp daddr ip ${3} drop")" -gt 0 ]
|
||||
}
|
||||
|
||||
nft_dhcp_drop () {
|
||||
[ "$(nft_count "${1}" "${2}" "ip saddr ${3} udp sport 67-68 udp dport 67-68 drop")" -gt 0 ]
|
||||
}
|
||||
|
||||
nft_handle () {
|
||||
ip netns exec "${1}" nft -a list chain bridge two "${2}" | awk -v r="${3}" '{line = $0; sub(/^[ \t]+/, "", line); if (sub(/ # handle [0-9]+$/, "", line) && line == r) print $NF}'
|
||||
}
|
||||
|
||||
vrrp_expected_master () {
|
||||
|
|
|
|||
|
|
@ -552,17 +552,59 @@ test_subnet_create_bridge_sends_mode_iface_type_and_interface_ip () {
|
|||
teardown
|
||||
}
|
||||
|
||||
test_ebt_confined_reads_both_mac_forms () {
|
||||
setup "ebt_confined : MAC avec ou sans zéros de tête (ebtables legacy et nft)"
|
||||
local stub='ebt_rules () { printf "%s\\n" "-s 02:01:00:00:0c:27 -o v-x-i -j DROP" "-s 2:1:0:0:c:27 -i v-x-i -j DROP" "-s 02:01:00:00:0c:28 -o v-x-i -j DROP"; }; '
|
||||
[[ $(node "${stub}ebt_confined ns v-x-i 02:01:00:00:0C:27") == 2 ]] || fail "$(node "${stub}ebt_confined ns v-x-i 02:01:00:00:0C:27")"
|
||||
nft_listing () {
|
||||
printf '%s\n' 'table bridge two {' ' map fwd_ports {' ' type ifname : verdict' ' elements = { "v-x-i" : jump fwd-v-x-i,' ' "v-y-i" : jump fwd-v-y-i }' ' }' '' \
|
||||
' chain forward {' ' type filter hook forward priority filter; policy accept;' ' oifname vmap @fwd_ports' ' }' '' \
|
||||
' chain pre-v-x-i {' ' ether saddr 02:01:00:00:0c:27 drop' ' }' '' \
|
||||
' chain fwd-v-x-i {' ' ether saddr 02:01:00:00:0c:27 drop' ' arp operation request arp daddr ip 10.2.1.254 drop' ' ip saddr 10.2.1.254 udp sport 67-68 udp dport 67-68 drop' ' ip saddr 10.2.1.254 udp dport 53 drop' ' }' '' \
|
||||
' chain fwd-v-x-ix {' ' arp operation request arp daddr ip 10.2.9.254 drop' ' }' '' \
|
||||
' chain fwd-v-y-i {' ' arp operation request arp daddr ip 10.2.2.254 drop' ' }' '}'
|
||||
}
|
||||
|
||||
test_nft_table_reads_the_two_table_only_when_present () {
|
||||
setup "nft_table : table bridge two lue si elle existe, rien sinon, échec si nft échoue"
|
||||
local listing
|
||||
listing=$(mktemp)
|
||||
nft_listing > "${listing}"
|
||||
[[ $(node "ip () { case \"\$*\" in 'netns exec ns nft list tables bridge') echo 'table bridge filter'; echo 'table bridge two' ;; 'netns exec ns nft list table bridge two') cat '${listing}' ;; *) echo \"ip \$*\"; return 1 ;; esac; }; nft_table ns") == "$(cat "${listing}")" ]] || fail "table présente non lue"
|
||||
[[ $(node "ip () { case \"\$*\" in 'netns exec ns nft list tables bridge') echo 'table bridge filter' ;; *) echo \"ip \$*\"; return 1 ;; esac; }; nft_table ns && echo VIDE") == VIDE ]] || fail "table absente : $(node "ip () { echo 'table bridge filter'; }; nft_table ns")"
|
||||
[[ $(node "ip () { return 1; }; nft_table ns || echo ÉCHEC") == ÉCHEC ]] || fail "échec de nft avalé"
|
||||
[[ $(node "nft_table () { return 1; }; nft_rules ns v-x-i || echo ÉCHEC") == ÉCHEC ]] || fail "nft_rules avale l'échec de nft_table"
|
||||
rm -f "${listing}"
|
||||
teardown
|
||||
}
|
||||
|
||||
test_ebt_mac_drops_leading_zeros_like_ebtables () {
|
||||
setup "ebt_mac : octets sans zéro de tête, en minuscules, comme ebtables -L"
|
||||
[[ $(node 'ebt_mac 02:01:00:00:0C:1d') == 2:1:0:0:c:1d ]] || fail "$(node 'ebt_mac 02:01:00:00:0C:1d')"
|
||||
[[ $(node 'ebt_mac aa:10:00:ff:01:00') == aa:10:0:ff:1:0 ]] || fail "$(node 'ebt_mac aa:10:00:ff:01:00')"
|
||||
test_nft_rules_reads_only_the_chains_of_the_port () {
|
||||
setup "nft_rules : règles des chaînes pre-, fwd- et out- du port, préfixées par la chaîne"
|
||||
local stub got want
|
||||
stub="nft_table () { $(declare -f nft_listing | sed 1,2d | sed '$d'); }; "
|
||||
got=$(node "${stub}nft_rules ns v-x-i")
|
||||
want=$(printf '%s\n' 'pre-v-x-i ether saddr 02:01:00:00:0c:27 drop' 'fwd-v-x-i ether saddr 02:01:00:00:0c:27 drop' 'fwd-v-x-i arp operation request arp daddr ip 10.2.1.254 drop' 'fwd-v-x-i ip saddr 10.2.1.254 udp sport 67-68 udp dport 67-68 drop' 'fwd-v-x-i ip saddr 10.2.1.254 udp dport 53 drop')
|
||||
[[ "${got}" == "${want}" ]] || fail "règles lues : ${got}"
|
||||
teardown
|
||||
}
|
||||
|
||||
test_nft_matchers_read_the_rules () {
|
||||
setup "nft_confined / nft_arp_drop / nft_dhcp_drop : règle exacte, MAC en minuscules"
|
||||
local stub
|
||||
stub="nft_table () { $(declare -f nft_listing | sed 1,2d | sed '$d'); }; "
|
||||
[[ $(node "${stub}nft_confined ns v-x-i 02:01:00:00:0C:27") == 2 ]] || fail "confinement : $(node "${stub}nft_confined ns v-x-i 02:01:00:00:0C:27")"
|
||||
[[ $(node "${stub}nft_confined ns v-x-i 02:01:00:00:0c:28") == 0 ]] || fail "autre MAC comptée"
|
||||
[[ $(node "${stub}nft_arp_drop ns v-x-i 10.2.1.254 && echo OUI") == OUI ]] || fail "ARP non lu"
|
||||
[[ $(node "${stub}nft_arp_drop ns v-x-i 10.2.1.25 || echo NON") == NON ]] || fail "10.2.1.25 pris pour 10.2.1.254"
|
||||
[[ $(node "${stub}nft_arp_drop ns v-x-i 10.2.9.254 || echo NON") == NON ]] || fail "chaîne de v-x-ix lue pour v-x-i"
|
||||
[[ $(node "${stub}nft_dhcp_drop ns v-x-i 10.2.1.254 && echo OUI") == OUI ]] || fail "DHCP non lu"
|
||||
[[ $(node "${stub}nft_dhcp_drop ns v-y-i 10.2.1.254 || echo NON") == NON ]] || fail "autre interface acceptée"
|
||||
[[ $(node "nft_rules () { echo 'fwd-v-z-i ip saddr 10.2.1.254 udp dport 53 drop'; }; nft_dhcp_drop ns v-z-i 10.2.1.254 || echo NON") == NON ]] || fail "règle UDP hors DHCP prise pour un blocage DHCP"
|
||||
teardown
|
||||
}
|
||||
|
||||
test_nft_handle_finds_the_exact_rule () {
|
||||
setup "nft_handle : numéro de la règle exacte dans nft -a list chain"
|
||||
local stub
|
||||
stub="ip () { printf '%s\\n' 'table bridge two {' ' chain out-v-x-i { # handle 9' ' arp saddr ip 10.2.1.25 drop # handle 11' ' arp saddr ip 10.2.1.254 drop # handle 12' ' }' '}'; }; "
|
||||
[[ $(node "${stub}nft_handle ns out-v-x-i 'arp saddr ip 10.2.1.254 drop'") == 12 ]] || fail "$(node "${stub}nft_handle ns out-v-x-i 'arp saddr ip 10.2.1.254 drop'")"
|
||||
[[ -z $(node "${stub}nft_handle ns out-v-x-i 'arp saddr ip 10.2.1.2 drop'") ]] || fail "règle absente trouvée"
|
||||
teardown
|
||||
}
|
||||
|
||||
|
|
@ -664,19 +706,6 @@ test_log_gap_measures_the_longest_silence_in_a_window () {
|
|||
teardown
|
||||
}
|
||||
|
||||
test_ebt_matchers_read_the_ebtables_listing () {
|
||||
setup "ebt_confined / ebt_arp_drop / ebt_dhcp_drop : lecture de ebtables -L"
|
||||
local stub='ebt_rules () { printf "%s\n" "-s 2:1:0:0:c:27 -i v-x-i -j DROP" "-s 2:1:0:0:c:27 -o v-x-i -j DROP" "-p ARP -o v-x-i --arp-op Request --arp-ip-dst 10.2.1.254 -j DROP" "-p IPv4 -o v-x-i --ip-src 10.2.1.254 --ip-proto udp --ip-sport 67:68 --ip-dport 67:68 -j DROP" | grep -- "$2"; }; '
|
||||
[[ $(node "${stub}ebt_confined ns v-x-i 02:01:00:00:0c:27") == 2 ]] || fail "confinement : $(node "${stub}ebt_confined ns v-x-i 02:01:00:00:0c:27")"
|
||||
[[ $(node "${stub}ebt_confined ns v-x-i 02:01:00:00:0c:28") == 0 ]] || fail "autre MAC comptée"
|
||||
[[ $(node "${stub}ebt_arp_drop ns v-x-i 10.2.1.254 && echo OUI") == OUI ]] || fail "ARP non lu"
|
||||
[[ $(node "${stub}ebt_arp_drop ns v-x-i 10.2.1.25 || echo NON") == NON ]] || fail "10.2.1.25 pris pour 10.2.1.254"
|
||||
[[ $(node "${stub}ebt_dhcp_drop ns v-x-i 10.2.1.254 && echo OUI") == OUI ]] || fail "DHCP non lu"
|
||||
[[ $(node "${stub}ebt_dhcp_drop ns v-y-i 10.2.1.254 || echo NON") == NON ]] || fail "autre interface acceptée"
|
||||
[[ $(node 'ebt_rules () { echo "-p IPv4 -o v-z-i --ip-src 10.2.1.254 --ip-proto udp --ip-dport 53 -j DROP"; }; ebt_dhcp_drop ns v-z-i 10.2.1.254 || echo NON') == NON ]] || fail "règle UDP hors DHCP prise pour un blocage DHCP"
|
||||
teardown
|
||||
}
|
||||
|
||||
test_wait_bgp_paths_waits_for_the_expected_count () {
|
||||
setup "wait_bgp_paths : nombre de chemins attendu, erreur explicite sinon"
|
||||
[[ $(node 'bgp_paths () { echo 0; }; sleep () { :; }; wait_bgp_paths vp-t53 10.53.1.50/32 0 && echo OUI') == OUI ]] || fail "0 attendu refusé"
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# fonctionnalité: hypothèses d'environnement de la suite — firmware UEFI, unités modèles de l'agent, journalctl, format de ebtables -L, br-lan, priorités VRRP lues dans FRR
|
||||
# fonctionnalité: hypothèses d'environnement de la suite — firmware UEFI, unités modèles de l'agent, journalctl, format de nft list table, br-lan, priorités VRRP lues dans FRR
|
||||
# hôtes: hv1 hv2 hv3
|
||||
# tickets: #70
|
||||
|
||||
|
|
@ -17,20 +17,32 @@ check "${HOST} : journalctl accepte --since @<epoch>" journalctl --since "@$(dat
|
|||
for unit in dhcp@.service dnsmasq@.service metadata@.service; do
|
||||
check "${HOST} : unité modèle ${unit} installée" systemctl cat "${unit}"
|
||||
done
|
||||
NS=t02-ebt
|
||||
NS=t02-nft
|
||||
MAC=02:01:00:00:0c:27
|
||||
ip netns add "${NS}"
|
||||
check "${HOST} : ebtables -L, aucune règle avant (témoin)" test "$(ebt_confined "${NS}" v-t02-i "${MAC}")" -eq 0
|
||||
ip netns exec "${NS}" ebtables -t nat -A PREROUTING --in-interface v-t02-i -s "${MAC}" -j DROP
|
||||
ip netns exec "${NS}" ebtables -t filter -A FORWARD --out-interface v-t02-i -s "${MAC}" -j DROP
|
||||
ip netns exec "${NS}" ebtables -t filter -A OUTPUT --out-interface v-t02-i -s "${MAC}" -j DROP
|
||||
ip netns exec "${NS}" ebtables -A FORWARD --out-interface v-t02-i -p arp --arp-op Request --arp-ip-dst 10.2.1.254 -j DROP
|
||||
ip netns exec "${NS}" ebtables -A FORWARD --out-interface v-t02-i -p IPv4 --ip-protocol udp --ip-source-port 67:68 --ip-destination-port 67:68 --ip-source 10.2.1.254 -j DROP
|
||||
check "${HOST} : ebtables -L, confinement de MAC lu par ebt_confined (3 règles)" test "$(ebt_confined "${NS}" v-t02-i "${MAC}")" -eq 3
|
||||
check "${HOST} : ebtables -L, blocage ARP vers la gateway lu par ebt_arp_drop" ebt_arp_drop "${NS}" v-t02-i 10.2.1.254
|
||||
check "${HOST} : ebtables -L, blocage DHCP lu par ebt_dhcp_drop" ebt_dhcp_drop "${NS}" v-t02-i 10.2.1.254
|
||||
other_ip_ignored () { ! ebt_arp_drop "${NS}" v-t02-i 10.2.1.25; }
|
||||
check "${HOST} : ebt_arp_drop ne confond pas une autre adresse (témoin)" other_ip_ignored
|
||||
check "${HOST} : nft installé" command -v nft
|
||||
check "${HOST} : nft list table, aucune règle avant (témoin)" test "$(nft_confined "${NS}" v-t02-i "${MAC}")" -eq 0
|
||||
ip netns exec "${NS}" nft -f - <<NFT
|
||||
add table bridge two
|
||||
add chain bridge two pre-v-t02-i
|
||||
add rule bridge two pre-v-t02-i ether saddr ${MAC} drop
|
||||
add chain bridge two fwd-v-t02-i
|
||||
add rule bridge two fwd-v-t02-i ether saddr ${MAC} drop
|
||||
add rule bridge two fwd-v-t02-i arp operation request arp daddr ip 10.2.1.254 drop
|
||||
add rule bridge two fwd-v-t02-i ip saddr 10.2.1.254 udp sport 67-68 udp dport 67-68 drop
|
||||
add chain bridge two out-v-t02-i
|
||||
add rule bridge two out-v-t02-i ether saddr ${MAC} drop
|
||||
add chain bridge two fwd-v-t02x-i
|
||||
add rule bridge two fwd-v-t02x-i arp operation request arp daddr ip 10.2.9.254 drop
|
||||
NFT
|
||||
check "${HOST} : nft list table, confinement de MAC lu par nft_confined (3 règles)" test "$(nft_confined "${NS}" v-t02-i "${MAC}")" -eq 3
|
||||
check "${HOST} : nft list table, blocage ARP vers la gateway lu par nft_arp_drop" nft_arp_drop "${NS}" v-t02-i 10.2.1.254
|
||||
check "${HOST} : nft list table, blocage DHCP lu par nft_dhcp_drop" nft_dhcp_drop "${NS}" v-t02-i 10.2.1.254
|
||||
other_ip_ignored () { ! nft_arp_drop "${NS}" v-t02-i 10.2.1.25; }
|
||||
check "${HOST} : nft_arp_drop ne confond pas une autre adresse (témoin)" other_ip_ignored
|
||||
other_port_ignored () { ! nft_arp_drop "${NS}" v-t02-i 10.2.9.254; }
|
||||
check "${HOST} : nft_rules ne lit pas la chaîne d'un autre port (témoin)" other_port_ignored
|
||||
check "${HOST} : nft_handle trouve le numéro d'une règle" test -n "$(nft_handle "${NS}" out-v-t02-i "ether saddr ${MAC} drop")"
|
||||
ip netns del "${NS}"
|
||||
case "${HOST}" in
|
||||
hv1) LAN=10.33.0.11 ;;
|
||||
|
|
|
|||
|
|
@ -18,10 +18,10 @@ check "sn-t31a : route du subnet émise depuis la gateway" sh -c "ip -n vp-t31 -
|
|||
for dev in br-t31a gw-t31a; do
|
||||
check "sn-t31a : ${dev}, arp_ignore=1 et arp_announce=2" sh -c "[ \"\$(ip netns exec vp-t31 sysctl -n net.ipv4.conf.${dev}.arp_ignore) \$(ip netns exec vp-t31 sysctl -n net.ipv4.conf.${dev}.arp_announce)\" = '1 2' ]"
|
||||
done
|
||||
check "sn-t31a : confinement de la MAC anycast sur v-t31a-i (nat PREROUTING, FORWARD, OUTPUT)" test "$(ebt_confined vp-t31 v-t31a-i "${MAC}")" -eq 3
|
||||
check "sn-t31a : confinement de la MAC anycast sur v-t31a-i (prerouting, forward, output)" test "$(nft_confined vp-t31 v-t31a-i "${MAC}")" -eq 3
|
||||
for ip in 10.31.1.254 10.31.1.253; do
|
||||
check "sn-t31a : requêtes ARP vers ${ip} bloquées en sortie de v-t31a-i" ebt_arp_drop vp-t31 v-t31a-i "${ip}"
|
||||
check "sn-t31a : DHCP émis depuis ${ip} bloqué en sortie de v-t31a-i" ebt_dhcp_drop vp-t31 v-t31a-i "${ip}"
|
||||
check "sn-t31a : requêtes ARP vers ${ip} bloquées en sortie de v-t31a-i" nft_arp_drop vp-t31 v-t31a-i "${ip}"
|
||||
check "sn-t31a : DHCP émis depuis ${ip} bloqué en sortie de v-t31a-i" nft_dhcp_drop vp-t31 v-t31a-i "${ip}"
|
||||
done
|
||||
check "VM t31-a" vm_create t31-a sn-t31a 10.31.1.10 "${KEY}"
|
||||
check "VM t31-b" vm_create t31-b sn-t31b 10.31.2.10 "${KEY}"
|
||||
|
|
@ -39,8 +39,8 @@ check "suppression de t31-a" vm_delete t31-a
|
|||
check "suppression de sn-t31a" api_delete subnets/sn-t31a
|
||||
check "sn-t31a supprimé" wait_state subnets/sn-t31a deleted
|
||||
check "sn-t31a : plus de macvlan gw-t31a dans le netns" sh -c "! ip -n vp-t31 link show gw-t31a >/dev/null 2>&1"
|
||||
check "sn-t31a : plus aucune règle ebtables pour v-t31a-i" test -z "$(ebt_rules vp-t31 v-t31a-i)"
|
||||
check "sn-t31b : règles de son propre subnet toujours présentes (témoin)" test -n "$(ebt_rules vp-t31 v-t31b-i)"
|
||||
check "sn-t31a : plus aucune règle nftables pour v-t31a-i" test -z "$(nft_rules vp-t31 v-t31a-i)"
|
||||
check "sn-t31b : règles de son propre subnet toujours présentes (témoin)" test -n "$(nft_rules vp-t31 v-t31b-i)"
|
||||
check "sn-t31a recréé" subnet_create sn-t31a vp-t31 3101 10.31.1.0/24
|
||||
check "VM t31-a2 sur le subnet recréé" vm_create t31-a2 sn-t31a 10.31.1.11 "${KEY}"
|
||||
check "t31-a2 joignable" vm_wait vp-t31 10.31.1.11
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ own_meta () {
|
|||
done
|
||||
[ "${ok}" -eq 5 ] || { echo "${2} : ${ok}/5 réponses avec ses propres métadonnées"; return 1; }
|
||||
}
|
||||
check "hv1 : quatre règles de confinement de 10.33.0.1 sur v-t33a-i" test "$(ebt_rules vp-t33 v-t33a-i | grep -c '10.33.0.1 ')" -eq 4
|
||||
check "hv1 : quatre règles de confinement de 10.33.0.1 sur v-t33a-i" test "$(nft_rules vp-t33 v-t33a-i | grep -c '10.33.0.1 ')" -eq 4
|
||||
RESPONDERS=$(vm_frames vp-t33 10.33.0.21 arp-who ens3 10.33.0.21 10.33.0.1 2)
|
||||
info "MAC qui répondent pour 10.33.0.1 vues de t33-a1 : $(echo ${RESPONDERS})"
|
||||
check "deux hôtes : t33-a1 (hv1) joint t33-b1 (hv2) sur le segment lan" vm_ssh vp-t33 10.33.0.21 'ping -n -c 3 -W 2 10.33.0.23'
|
||||
|
|
@ -74,10 +74,11 @@ check "témoin : l'ARP émis par t33-a1 atteint le segment par v-t33a-e (capture
|
|||
check "aucune trame ARP depuis 10.33.0.1 vers le segment (v-t33a-e)" test "$(cat "${S}/arp")" -eq 0
|
||||
check "aucune trame DHCP depuis 10.33.0.1 vers le segment (v-t33a-e)" test "$(cat "${S}/dhcp")" -eq 0
|
||||
rm -r "${S}"
|
||||
RULE='OUTPUT --out-interface v-t33a-i -p arp --arp-ip-src 10.33.0.1 -j DROP'
|
||||
check "règle ARP OUTPUT retirée à la main" ip netns exec vp-t33 ebtables -D ${RULE}
|
||||
check "watchdog : règle manquante signalée pour sn-t33a" wait_watchdog "ebtables rule missing in netns vp-t33"
|
||||
check "règle ARP OUTPUT reposée" ip netns exec vp-t33 ebtables -A ${RULE}
|
||||
RULE='arp saddr ip 10.33.0.1 drop'
|
||||
HANDLE=$(nft_handle vp-t33 out-v-t33a-i "${RULE}")
|
||||
check "règle ARP output retirée à la main (handle ${HANDLE:-absent})" ip netns exec vp-t33 nft delete rule bridge two out-v-t33a-i handle "${HANDLE}"
|
||||
check "watchdog : règle manquante signalée pour sn-t33a" wait_watchdog "nftables rule missing in netns vp-t33: out-v-t33a-i: ${RULE}"
|
||||
check "règle ARP output reposée" ip netns exec vp-t33 nft add rule bridge two out-v-t33a-i ${RULE}
|
||||
NODE
|
||||
|
||||
on hv2 <<'NODE'
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ sleep 20
|
|||
check "après redémarrage : même processus QEMU pour t61-a (ni arrêtée ni recréée)" test "$(pgrep -f "${VOLUMES}/t61-a.qcow2" | sort | tr '\n' ' ')" = "${QEMU}"
|
||||
check "après redémarrage : même processus QEMU pour t61-d" test "$(pgrep -f "${VOLUMES}/t61-d.qcow2" | sort | tr '\n' ' ')" = "${QEMU_D}"
|
||||
check "après redémarrage : mêmes fichiers sous /run/two, /etc/dnsmasq.d et les volumes (sinon, fichiers en cause)" section_unchanged "${STATE_DIR}/t61.restart" fichiers
|
||||
check "après redémarrage : hôte inchangé (netns, liens, taps, adresses, routes, ebtables, iptables, FRR, frr.conf, fichiers, unités)" same_as "${STATE_DIR}/t61.restart" host_state
|
||||
check "après redémarrage : hôte inchangé (netns, liens, taps, adresses, routes, nftables de pont, iptables, FRR, frr.conf, fichiers, unités)" same_as "${STATE_DIR}/t61.restart" host_state
|
||||
check "après redémarrage : l'API rend les mêmes ressources dans le même état" same_as "${STATE_DIR}/t61.api" api_snapshot
|
||||
for rr in 10.255.255.1 10.255.255.2; do
|
||||
check "après redémarrage : session EVPN vers ${rr} établie, jamais coupée" sh -c "vtysh -c 'show bgp neighbors ${rr}' | grep -q 'BGP state = Established' && [ \"\$(vtysh -c 'show bgp neighbors ${rr}' | grep -o 'dropped [0-9]*')\" = \"\$(cat '${STATE_DIR}/t61.${rr}')\" ]"
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ check "${HOST} : aucune unité two-vm-*" none systemctl list-units --all --plain
|
|||
check "${HOST} : aucune instance DHCP (dhcp@, dnsmasq@) ni de métadonnées (metadata@) active" none sh -c "systemctl list-units --plain --no-legend 'dhcp@*' 'dnsmasq@*' 'metadata@*' | grep -w active"
|
||||
check "${HOST} : aucun socket de contrôle DHCP" none sh -c "ls /run/two/dhcp/*.sock 2>/dev/null"
|
||||
check "${HOST} : aucun processus QEMU" none pgrep -a qemu-system
|
||||
check "${HOST} : aucune règle ebtables qui vise une interface de subnet" none sh -c "{ ebtables -t filter -L; ebtables -t nat -L; } | grep -E -- '-(i|o) v-'"
|
||||
check "${HOST} : aucune règle nftables de pont qui vise une interface de subnet" none sh -c "nft list ruleset bridge | grep -E 'v-[a-z0-9]'"
|
||||
check "${HOST} : aucune règle iptables qui vise une interface de subnet ou de VPC" none sh -c "iptables-save | grep -E -- '(v-|vp-|tap)[a-z0-9]'"
|
||||
check "${HOST} : aucun volume de VM de test" none sh -c "ls '${VOLUMES}'/t[0-9][0-9]-* 2>/dev/null"
|
||||
check "${HOST} : FRR garde la base BGP (témoin)" sh -c "vtysh -c 'show running-config' | grep -q 'peer-group fabric'"
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ for script in deploy.sh bootstrap_kvm.sh; do
|
|||
done
|
||||
chmod 0755 deploy.sh bootstrap_kvm.sh
|
||||
bash ./deploy.sh --noup_script -i -u "${LAB_UPLINK}" -t "${RELEASE}"
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends nftables
|
||||
for binary in "${LAB_FILES}"/bin/*; do
|
||||
[ -f "${binary}" ] || continue
|
||||
name=$(basename "${binary}")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue