Fix the following issues:
1. Endianness Bug: The nftables runner used hardcoded
big-endian byte arrays for firewall mark values (0xff0000, etc.), breaking
bitwise operations on little-endian systems (all x86/x64, ARM). This caused
connmark save/restore rules to silently fail. Fixed by using
binary.NativeEndian to generate correct byte order for the host system.
2. Connmark Restore Conditional Check: The connmark restore
mechanism unconditionally overwrote packet marks, even when Tailscale
hadn't set any mark bits in conntrack. This destroyed mark bits set by
other systems (VPNs, policy routing, vendor flags), breaking coexistence.
Fixed by adding a conditional check to only restore when (ct mark &
0xff0000) != 0, preventing the worst case of wiping all marks to zero.
Changes:
- util/linuxfw/linuxfw.go: Added nativeEndianUint32() helper and updated
all mask functions to use native byte order instead of hardcoded bytes
- util/linuxfw/nftables_runner.go: Added conditional check in
makeConnmarkRestoreExprs() to only restore when ct mark has Tailscale
bits set; added detailed comment about bit preservation limitations
- util/linuxfw/iptables_runner.go: Added conditional check using -m
connmark ! --mark to match nftables behavior
- Tests updated: Fixed byte-level regression tests to expect little-endian
byte sequences and verify the new conditional check
Note: Perfect bit preservation in nftables remains challenging
due to nftables expression VM limitations. The current implementation
prevents the critical case of wiping marks with zero.
Updates #3310Fixes#11803
Related to #8555
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Mike O'Driscoll
48919f708b