wgengine,ipn/ipnlocal: sync wireguard-go peers incrementally on netmap deltas
Previously, any peer added or removed by an incremental netmap delta was only visible to wireguard-go after a full authReconfig: wgcfg's ReconfigDevice re-installed a PeerLookupFunc closing over a freshly built map of every peer's allowed IPs, doing O(n) work per change. Instead, install the wireguard-go device hooks once, backed by live state. Engine.SetPeerConfigFunc installs a single long-lived PeerLookupFunc that queries LocalBackend's per-node RouteManager on demand, and Engine.SyncDevicePeer does O(1) per-peer device sync (remove, or update allowed IPs) as each delta mutation is applied. Full reconfigs keep an O(n peers) device sync for now, but with no lookup closure to reinstall and no removed-peer resurrection race; a later change removes full-config peer syncing entirely. The RouteManager's PeerAllowedIPs accessor backs the new hooks: its sorted output makes unchanged state a no-op update, and its peer filtering mirrors nmcfg.WGCfg, so expired peers and peers predating both DERP and disco contribute no prefixes and thus cannot be lazily created in the device, which matters because wireguard-go validates inbound source IPs against per-peer allowed IPs. The engine's SetPeerByIPPacketFunc callback is now authoritative when installed, since LocalBackend's implementation covers subnet routes and exit-node routes via the RouteManager's outbound table; the engine's own reconfig-time BART table only serves engines running without a LocalBackend. The forced authReconfig on peer add/remove stays for now: the WireGuard device no longer needs it, but OS routes, the quad-100 resolver's MagicDNS hosts map, and tstun's masquerade/jailed peer config are still derived from the full peer set. Making those delta-aware is the next step before gating it. Updates #12542 Change-Id: I3ba8c7c324bca0ad0269279d03f53b1f17fb63a2 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit is contained in:
committed by
Brad Fitzpatrick
parent
ff1c7ef23c
commit
f831469c27
+33
-15
@@ -633,6 +633,7 @@ func NewLocalBackend(logf logger.Logf, logID logid.PublicID, sys *tsd.System, lo
|
|||||||
nb.ready()
|
nb.ready()
|
||||||
|
|
||||||
e.SetPeerByIPPacketFunc(b.lookupPeerByIP)
|
e.SetPeerByIPPacketFunc(b.lookupPeerByIP)
|
||||||
|
e.SetPeerConfigFunc(b.peerAllowedIPs)
|
||||||
e.SetPeerForIPFunc(b.peerForIP)
|
e.SetPeerForIPFunc(b.peerForIP)
|
||||||
e.SetPeerSessionStateFunc(b.onPeerWireGuardState)
|
e.SetPeerSessionStateFunc(b.onPeerWireGuardState)
|
||||||
e.SetNetLogSource(netLogNodeSource{b})
|
e.SetNetLogSource(netLogNodeSource{b})
|
||||||
@@ -2462,7 +2463,8 @@ func (b *LocalBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bo
|
|||||||
// the full-netmap behavior of [tkaFilterNetmapLocked].
|
// the full-netmap behavior of [tkaFilterNetmapLocked].
|
||||||
muts = b.tkaFilterDeltaMutsLocked(muts)
|
muts = b.tkaFilterDeltaMutsLocked(muts)
|
||||||
needsAuthReconfig := netmapDeltaNeedsAuthReconfig(cn, muts)
|
needsAuthReconfig := netmapDeltaNeedsAuthReconfig(cn, muts)
|
||||||
cn.UpdateNetmapDelta(muts)
|
|
||||||
|
changedAllowedIPs, _ := cn.UpdateNetmapDelta(muts)
|
||||||
if buildfeatures.HasDrive {
|
if buildfeatures.HasDrive {
|
||||||
// Drive's lazy remotes-source caches its rebuild keyed by this
|
// Drive's lazy remotes-source caches its rebuild keyed by this
|
||||||
// generation, so any delta — peer add/remove, address change,
|
// generation, so any delta — peer add/remove, address change,
|
||||||
@@ -2504,20 +2506,27 @@ func (b *LocalBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bo
|
|||||||
}
|
}
|
||||||
ms.UpdateNetmapDelta(muts)
|
ms.UpdateNetmapDelta(muts)
|
||||||
|
|
||||||
|
// Sync the WireGuard device for exactly the peers whose allowed
|
||||||
|
// source prefixes changed, as computed by the route manager when
|
||||||
|
// the delta was applied above. Removed peers appear with a nil
|
||||||
|
// value and are removed from the device; SyncDevicePeer reads
|
||||||
|
// the live per-peer config, so only the keys matter here.
|
||||||
|
for k := range changedAllowedIPs {
|
||||||
|
b.e.SyncDevicePeer(k)
|
||||||
|
}
|
||||||
|
|
||||||
// Force a full authReconfig + SetSelfNode on any peer add or
|
// Force a full authReconfig + SetSelfNode on any peer add or
|
||||||
// remove. netmapDeltaNeedsAuthReconfig only considered
|
// remove. The WireGuard device itself no longer depends on this:
|
||||||
// NodeMutationUpsert of already-known NodeIDs whose
|
// the SyncDevicePeer calls above keep its peer set delta-correct,
|
||||||
// peerRouteConfigChanged, so brand-new peers and removes left
|
// and its lazy peer creation reads live state via
|
||||||
// e.lastCfgFull and wgdev's PeerLookupFunc closure stale, and
|
// [wgengine.Engine.SetPeerConfigFunc]. What still rides
|
||||||
// outbound wgdev encryption missed those peers. authReconfig
|
// authReconfig is everything else derived from the full peer set:
|
||||||
// fixes the wireguard side; SetSelfNode refreshes the engine's
|
// OS routes (router.Config), the quad-100 resolver's MagicDNS
|
||||||
// cached self node. PeerForIP / lookupPeerByIP staleness was
|
// hosts map (dnsConfigForNetmap), and tstun's per-peer config
|
||||||
// addressed separately in d4f2917c1b, which routes those lookups
|
// (masquerade addresses and jailed peers, via SetWGConfig). Once
|
||||||
// through nodeBackend's live data, and as part of the broader
|
// those become delta-aware too, this can be gated on the route
|
||||||
// netmap.NetworkMap removal effort the engine no longer caches
|
// manager's OS-routes changes and the tstun-relevant fields
|
||||||
// the netmap at all (see tailscale/corp#43394). As of 2026-06-24
|
// instead of firing on every peer change.
|
||||||
// the only remaining staleness this guards against is
|
|
||||||
// e.lastCfgFull and the wgdev peer set.
|
|
||||||
needsAuthReconfig = needsAuthReconfig || peersUpsertedOrRemoved
|
needsAuthReconfig = needsAuthReconfig || peersUpsertedOrRemoved
|
||||||
if needsAuthReconfig {
|
if needsAuthReconfig {
|
||||||
if peersUpsertedOrRemoved {
|
if peersUpsertedOrRemoved {
|
||||||
@@ -6126,12 +6135,21 @@ func (b *LocalBackend) authReconfigLocked() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
oneCGNATRoute := shouldUseOneCGNATRoute(b.logf, b.sys.NetMon.Get(), b.sys.ControlKnobs(), version.OS())
|
oneCGNATRoute := shouldUseOneCGNATRoute(b.logf, b.sys.NetMon.Get(), b.sys.ControlKnobs(), version.OS())
|
||||||
cn.updateRouteManagerPrefs(routePrefs{
|
// Sync the WireGuard device for any peers whose allowed source
|
||||||
|
// prefixes changed with the new prefs, such as the old and new
|
||||||
|
// exit node when the selection changes. The Reconfig below still
|
||||||
|
// converges every peer via its full device sync, but this
|
||||||
|
// incremental sync is what will remain once the full reconfig is
|
||||||
|
// gated on actual router/DNS changes.
|
||||||
|
changedAllowedIPs := cn.updateRouteManagerPrefs(routePrefs{
|
||||||
ExitNodeID: prefs.ExitNodeID(),
|
ExitNodeID: prefs.ExitNodeID(),
|
||||||
ExitNodeSelected: prefs.ExitNodeID() != "" || prefs.ExitNodeIP().IsValid(),
|
ExitNodeSelected: prefs.ExitNodeID() != "" || prefs.ExitNodeIP().IsValid(),
|
||||||
RouteAll: flags&netmap.AllowSubnetRoutes != 0,
|
RouteAll: flags&netmap.AllowSubnetRoutes != 0,
|
||||||
OneCGNAT: oneCGNATRoute,
|
OneCGNAT: oneCGNATRoute,
|
||||||
})
|
})
|
||||||
|
for k := range changedAllowedIPs {
|
||||||
|
b.e.SyncDevicePeer(k)
|
||||||
|
}
|
||||||
rcfg := b.routerConfigLocked(cfg, prefs, nm, oneCGNATRoute)
|
rcfg := b.routerConfigLocked(cfg, prefs, nm, oneCGNATRoute)
|
||||||
|
|
||||||
// Add these extra Allowed IPs after router configuration, because the expected
|
// Add these extra Allowed IPs after router configuration, because the expected
|
||||||
|
|||||||
@@ -2751,7 +2751,7 @@ func TestNotifyForSessionUserProfilesDedupResetsOnSelfChange(t *testing.T) {
|
|||||||
// tests LocalBackend.updateNetmapDeltaLocked
|
// tests LocalBackend.updateNetmapDeltaLocked
|
||||||
func TestUpdateNetmapDelta(t *testing.T) {
|
func TestUpdateNetmapDelta(t *testing.T) {
|
||||||
b := newTestLocalBackend(t)
|
b := newTestLocalBackend(t)
|
||||||
if b.currentNode().UpdateNetmapDelta(nil) {
|
if _, handled := b.currentNode().UpdateNetmapDelta(nil); handled {
|
||||||
t.Errorf("updateNetmapDeltaLocked() = true, want false with nil netmap")
|
t.Errorf("updateNetmapDeltaLocked() = true, want false with nil netmap")
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2790,7 +2790,7 @@ func TestUpdateNetmapDelta(t *testing.T) {
|
|||||||
t.Fatal("netmap.MutationsFromMapResponse failed")
|
t.Fatal("netmap.MutationsFromMapResponse failed")
|
||||||
}
|
}
|
||||||
|
|
||||||
if !b.currentNode().UpdateNetmapDelta(muts) {
|
if _, handled := b.currentNode().UpdateNetmapDelta(muts); !handled {
|
||||||
t.Fatalf("updateNetmapDeltaLocked() = false, want true with new netmap")
|
t.Fatalf("updateNetmapDeltaLocked() = false, want true with new netmap")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -267,6 +267,22 @@ func (nb *nodeBackend) NodeByKey(k key.NodePublic) (_ tailcfg.NodeID, ok bool) {
|
|||||||
return nid, ok
|
return nid, ok
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// PeerAllowedIPs returns the prefixes from which the peer with the
|
||||||
|
// given public key is currently allowed to originate traffic, or
|
||||||
|
// ok=false if the key is unknown or the peer currently contributes no
|
||||||
|
// prefixes.
|
||||||
|
func (nb *nodeBackend) PeerAllowedIPs(k key.NodePublic) ([]netip.Prefix, bool) {
|
||||||
|
nb.mu.Lock()
|
||||||
|
defer nb.mu.Unlock()
|
||||||
|
id, ok := nb.nodeByKey[k]
|
||||||
|
if !ok {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
// Holding nb.mu satisfies routeMgr's serialization requirement:
|
||||||
|
// all routeMgr mutations also run under nb.mu.
|
||||||
|
return nb.routeMgr.PeerAllowedIPs(id)
|
||||||
|
}
|
||||||
|
|
||||||
// NodeByWireGuardString returns the node ID of the peer whose
|
// NodeByWireGuardString returns the node ID of the peer whose
|
||||||
// [key.NodePublic.WireGuardGoString] form is s (e.g. "peer(IMTB…r7lM)").
|
// [key.NodePublic.WireGuardGoString] form is s (e.g. "peer(IMTB…r7lM)").
|
||||||
// ok is false if no current peer matches.
|
// ok is false if no current peer matches.
|
||||||
@@ -793,11 +809,6 @@ func (nb *nodeBackend) updatePeersLocked() {
|
|||||||
rt.Commit()
|
rt.Commit()
|
||||||
}
|
}
|
||||||
|
|
||||||
// updateRouteManagerPrefs pushes the routing-relevant prefs and the
|
|
||||||
// OneCGNAT decision into the route manager. The exit node arrives as
|
|
||||||
// a stable ID from prefs and is resolved to the current numeric node
|
|
||||||
// ID here; it resolves to zero (no exit node) if that peer is not in
|
|
||||||
// the netmap yet, in which case a later netmap update re-runs this.
|
|
||||||
// routePrefs is the subset of routing-relevant prefs (and derived
|
// routePrefs is the subset of routing-relevant prefs (and derived
|
||||||
// state) that [nodeBackend.updateRouteManagerPrefs] pushes into the
|
// state) that [nodeBackend.updateRouteManagerPrefs] pushes into the
|
||||||
// RouteManager.
|
// RouteManager.
|
||||||
@@ -821,7 +832,13 @@ type routePrefs struct {
|
|||||||
OneCGNAT bool
|
OneCGNAT bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func (nb *nodeBackend) updateRouteManagerPrefs(p routePrefs) {
|
// updateRouteManagerPrefs pushes p into the route manager.
|
||||||
|
//
|
||||||
|
// It returns the peers whose allowed source prefixes changed as a
|
||||||
|
// result (for example the old and new exit node when the selection
|
||||||
|
// changes), as described by [routemanager.Result.AllowedIPs].
|
||||||
|
// In particular, the value for a key will be nil when that peer was removed.
|
||||||
|
func (nb *nodeBackend) updateRouteManagerPrefs(p routePrefs) (changedAllowedIPs map[key.NodePublic][]netip.Prefix) {
|
||||||
nb.mu.Lock()
|
nb.mu.Lock()
|
||||||
defer nb.mu.Unlock()
|
defer nb.mu.Unlock()
|
||||||
var exitID tailcfg.NodeID
|
var exitID tailcfg.NodeID
|
||||||
@@ -840,7 +857,8 @@ func (nb *nodeBackend) updateRouteManagerPrefs(p routePrefs) {
|
|||||||
RouteAll: p.RouteAll,
|
RouteAll: p.RouteAll,
|
||||||
})
|
})
|
||||||
rt.SetTailnetConfig(routemanager.TailnetConfig{OneCGNAT: p.OneCGNAT})
|
rt.SetTailnetConfig(routemanager.TailnetConfig{OneCGNAT: p.OneCGNAT})
|
||||||
rt.Commit()
|
res := rt.Commit()
|
||||||
|
return res.AllowedIPs
|
||||||
}
|
}
|
||||||
|
|
||||||
// setPacketFilter stores the live packet filter rules and parsed
|
// setPacketFilter stores the live packet filter rules and parsed
|
||||||
@@ -872,11 +890,16 @@ func (nb *nodeBackend) mergeUserProfiles(profiles map[tailcfg.UserID]tailcfg.Use
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (nb *nodeBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bool) {
|
// UpdateNetmapDelta applies the given netmap mutations to the live
|
||||||
|
// peer state. It returns the peers whose allowed source prefixes
|
||||||
|
// changed (as described by [routemanager.Result.AllowedIPs]) so the
|
||||||
|
// caller can sync those peers to the WireGuard device, and reports
|
||||||
|
// whether it handled all of the mutations.
|
||||||
|
func (nb *nodeBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (changedAllowedIPs map[key.NodePublic][]netip.Prefix, handled bool) {
|
||||||
nb.mu.Lock()
|
nb.mu.Lock()
|
||||||
defer nb.mu.Unlock()
|
defer nb.mu.Unlock()
|
||||||
if nb.netMap == nil {
|
if nb.netMap == nil {
|
||||||
return false
|
return nil, false
|
||||||
}
|
}
|
||||||
|
|
||||||
// Locally cloned mutable nodes, to avoid calling AsStruct (clone)
|
// Locally cloned mutable nodes, to avoid calling AsStruct (clone)
|
||||||
@@ -890,7 +913,10 @@ func (nb *nodeBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bo
|
|||||||
// in place as the loop runs. And if we return false, we'll just
|
// in place as the loop runs. And if we return false, we'll just
|
||||||
// get a full netmap soon and reset all our state anyway.
|
// get a full netmap soon and reset all our state anyway.
|
||||||
rt := nb.routeMgr.Begin()
|
rt := nb.routeMgr.Begin()
|
||||||
defer rt.Commit()
|
defer func() {
|
||||||
|
res := rt.Commit()
|
||||||
|
changedAllowedIPs = res.AllowedIPs
|
||||||
|
}()
|
||||||
|
|
||||||
for _, m := range muts {
|
for _, m := range muts {
|
||||||
switch m := m.(type) {
|
switch m := m.(type) {
|
||||||
@@ -932,7 +958,7 @@ func (nb *nodeBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bo
|
|||||||
nv, ok := nb.peers[nid]
|
nv, ok := nb.peers[nid]
|
||||||
if !ok {
|
if !ok {
|
||||||
// TODO(bradfitz): unexpected metric?
|
// TODO(bradfitz): unexpected metric?
|
||||||
return false
|
return nil, false
|
||||||
}
|
}
|
||||||
n = nv.AsStruct()
|
n = nv.AsStruct()
|
||||||
mak.Set(&mutableNodes, nv.ID(), n)
|
mak.Set(&mutableNodes, nv.ID(), n)
|
||||||
@@ -943,7 +969,7 @@ func (nb *nodeBackend) UpdateNetmapDelta(muts []netmap.NodeMutation) (handled bo
|
|||||||
nb.peers[nid] = n.View()
|
nb.peers[nid] = n.View()
|
||||||
}
|
}
|
||||||
nb.signalKeyWaitersForTestLocked()
|
nb.signalKeyWaitersForTestLocked()
|
||||||
return true
|
return nil, true
|
||||||
}
|
}
|
||||||
|
|
||||||
// unlockedNodesPermitted reports whether any peer with theUnsignedPeerAPIOnly bool set true has any of its allowed IPs
|
// unlockedNodesPermitted reports whether any peer with theUnsignedPeerAPIOnly bool set true has any of its allowed IPs
|
||||||
|
|||||||
@@ -317,15 +317,21 @@ func TestNodeBackendRouteManager(t *testing.T) {
|
|||||||
wantPeerFor("8.8.8.8", tailcfg.NodeView{}) // exit node not selected
|
wantPeerFor("8.8.8.8", tailcfg.NodeView{}) // exit node not selected
|
||||||
|
|
||||||
// Selecting peer 2 as the exit node resolves its stable ID and
|
// Selecting peer 2 as the exit node resolves its stable ID and
|
||||||
// installs its /0 routes.
|
// installs its /0 routes. The commit reports peer 2's allowed
|
||||||
nb.updateRouteManagerPrefs(routePrefs{ExitNodeID: "stable2", ExitNodeSelected: true})
|
// prefixes as changed.
|
||||||
|
if changed := nb.updateRouteManagerPrefs(routePrefs{ExitNodeID: "stable2", ExitNodeSelected: true}); len(changed) != 1 || changed[p2.Key()] == nil {
|
||||||
|
t.Errorf("updateRouteManagerPrefs(exit=stable2) changed = %v; want just %v", changed, p2.Key())
|
||||||
|
}
|
||||||
wantPeerFor("8.8.8.8", p2)
|
wantPeerFor("8.8.8.8", p2)
|
||||||
|
|
||||||
// A selected exit node that resolves to no current peer must
|
// A selected exit node that resolves to no current peer must
|
||||||
// blackhole internet traffic, not fall back to "no exit node":
|
// blackhole internet traffic, not fall back to "no exit node":
|
||||||
// the default routes stay in the OS route set with no outbound
|
// the default routes stay in the OS route set with no outbound
|
||||||
// peer to carry them.
|
// peer to carry them. Peer 2's allowed prefixes lose the /0s,
|
||||||
nb.updateRouteManagerPrefs(routePrefs{ExitNodeID: "no-such-node", ExitNodeSelected: true})
|
// which the commit reports.
|
||||||
|
if changed := nb.updateRouteManagerPrefs(routePrefs{ExitNodeID: "no-such-node", ExitNodeSelected: true}); len(changed) != 1 || changed[p2.Key()] == nil {
|
||||||
|
t.Errorf("updateRouteManagerPrefs(exit=unresolved) changed = %v; want just %v", changed, p2.Key())
|
||||||
|
}
|
||||||
wantPeerFor("8.8.8.8", tailcfg.NodeView{})
|
wantPeerFor("8.8.8.8", tailcfg.NodeView{})
|
||||||
if !nb.routeMgr.OSRoutes().Get(netip.MustParsePrefix("0.0.0.0/0")) {
|
if !nb.routeMgr.OSRoutes().Get(netip.MustParsePrefix("0.0.0.0/0")) {
|
||||||
t.Error("unresolved exit node: OSRoutes missing 0.0.0.0/0 blackhole route")
|
t.Error("unresolved exit node: OSRoutes missing 0.0.0.0/0 blackhole route")
|
||||||
@@ -339,12 +345,19 @@ func TestNodeBackendRouteManager(t *testing.T) {
|
|||||||
|
|
||||||
// Incremental deltas: add peer 3, remove peer 1.
|
// Incremental deltas: add peer 3, remove peer 1.
|
||||||
p3 := mkPeer(3, "stable3", "100.64.0.3/32")
|
p3 := mkPeer(3, "stable3", "100.64.0.3/32")
|
||||||
if !nb.UpdateNetmapDelta([]netmap.NodeMutation{
|
changed, handled := nb.UpdateNetmapDelta([]netmap.NodeMutation{
|
||||||
netmap.NodeMutationUpsert{Node: p3},
|
netmap.NodeMutationUpsert{Node: p3},
|
||||||
netmap.MakeNodeMutationRemove(1),
|
netmap.MakeNodeMutationRemove(1),
|
||||||
}) {
|
})
|
||||||
|
if !handled {
|
||||||
t.Fatal("UpdateNetmapDelta not handled")
|
t.Fatal("UpdateNetmapDelta not handled")
|
||||||
}
|
}
|
||||||
|
if len(changed) != 2 || changed[p3.Key()] == nil {
|
||||||
|
t.Errorf("UpdateNetmapDelta changed = %v; want entries for %v and %v", changed, p3.Key(), p1.Key())
|
||||||
|
}
|
||||||
|
if v, ok := changed[p1.Key()]; !ok || v != nil {
|
||||||
|
t.Errorf("UpdateNetmapDelta changed[%v] = %v, %v; want nil, true for removed peer", p1.Key(), v, ok)
|
||||||
|
}
|
||||||
wantPeerFor("100.64.0.3", p3)
|
wantPeerFor("100.64.0.3", p3)
|
||||||
wantPeerFor("100.64.0.1", tailcfg.NodeView{})
|
wantPeerFor("100.64.0.1", tailcfg.NodeView{})
|
||||||
|
|
||||||
|
|||||||
+27
-14
@@ -13,25 +13,38 @@ import (
|
|||||||
"tailscale.com/wgengine"
|
"tailscale.com/wgengine"
|
||||||
)
|
)
|
||||||
|
|
||||||
// lookupPeerByIP returns the node public key for the peer that owns the
|
// lookupPeerByIP returns the node public key for the peer that should
|
||||||
// given IP address. It is the fast path for [Engine.SetPeerByIPPacketFunc],
|
// handle traffic to the given IP address. It is installed as the
|
||||||
// handling exact-IP matches against node addresses; subnet routes and exit
|
// [wgengine.Engine.SetPeerByIPPacketFunc] callback: exact node
|
||||||
// nodes are handled by a BART-based fallback in userspaceEngine that uses
|
// addresses hit the nodeByAddr fast path, and subnet routes and
|
||||||
// the wireguard-filtered peer list (see lastCfgFull).
|
// exit-node default routes fall back to the RouteManager's outbound
|
||||||
|
// table, so it stays correct under incremental netmap deltas.
|
||||||
//
|
//
|
||||||
// It is called by wireguard-go on every outbound packet (not cached), so
|
// It is called by wireguard-go on every outbound packet (not cached),
|
||||||
// it must be fast.
|
// so it must be fast.
|
||||||
func (b *LocalBackend) lookupPeerByIP(ip netip.Addr) (key.NodePublic, bool) {
|
func (b *LocalBackend) lookupPeerByIP(ip netip.Addr) (key.NodePublic, bool) {
|
||||||
nb := b.currentNode()
|
nb := b.currentNode()
|
||||||
nid, ok := nb.NodeByAddr(ip)
|
if nid, ok := nb.NodeByAddr(ip); ok {
|
||||||
if !ok {
|
peer, ok := nb.NodeByID(nid)
|
||||||
return key.NodePublic{}, false
|
if !ok {
|
||||||
|
return key.NodePublic{}, false
|
||||||
|
}
|
||||||
|
return peer.Key(), true
|
||||||
}
|
}
|
||||||
peer, ok := nb.NodeByID(nid)
|
if pr, ok := nb.routeMgr.Outbound().Lookup(ip); ok {
|
||||||
if !ok {
|
return pr.Key, true
|
||||||
return key.NodePublic{}, false
|
|
||||||
}
|
}
|
||||||
return peer.Key(), true
|
return key.NodePublic{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// peerAllowedIPs returns the prefixes from which the peer with the
|
||||||
|
// given public key is currently allowed to originate traffic, or
|
||||||
|
// ok=false if the peer is unknown (or currently routable via no
|
||||||
|
// prefix at all). It is installed as the
|
||||||
|
// [wgengine.Engine.SetPeerConfigFunc] callback, backing wireguard-go's
|
||||||
|
// lazy peer creation and per-delta peer sync.
|
||||||
|
func (b *LocalBackend) peerAllowedIPs(k key.NodePublic) (_ []netip.Prefix, ok bool) {
|
||||||
|
return b.currentNode().PeerAllowedIPs(k)
|
||||||
}
|
}
|
||||||
|
|
||||||
// resolveMagicDNS resolves a MagicDNS hostname to the owning node's IP
|
// resolveMagicDNS resolves a MagicDNS hostname to the owning node's IP
|
||||||
|
|||||||
@@ -2016,6 +2016,9 @@ func (e *mockEngine) InstallCaptureHook(packet.CaptureCallback) {}
|
|||||||
|
|
||||||
func (e *mockEngine) SetPeerByIPPacketFunc(func(netip.Addr) (_ key.NodePublic, ok bool)) {}
|
func (e *mockEngine) SetPeerByIPPacketFunc(func(netip.Addr) (_ key.NodePublic, ok bool)) {}
|
||||||
func (e *mockEngine) SetPeerForIPFunc(func(netip.Addr) (_ wgengine.PeerForIP, ok bool)) {}
|
func (e *mockEngine) SetPeerForIPFunc(func(netip.Addr) (_ wgengine.PeerForIP, ok bool)) {}
|
||||||
|
func (e *mockEngine) SetPeerConfigFunc(func(key.NodePublic) (allowedIPs []netip.Prefix, ok bool)) {
|
||||||
|
}
|
||||||
|
func (e *mockEngine) SyncDevicePeer(key.NodePublic) {}
|
||||||
func (e *mockEngine) PeerKeyForIP(netip.Addr) (_ key.NodePublic, _ netip.Prefix, ok bool) {
|
func (e *mockEngine) PeerKeyForIP(netip.Addr) (_ key.NodePublic, _ netip.Prefix, ok bool) {
|
||||||
return key.NodePublic{}, netip.Prefix{}, false
|
return key.NodePublic{}, netip.Prefix{}, false
|
||||||
}
|
}
|
||||||
|
|||||||
+76
-5
@@ -122,6 +122,12 @@ type userspaceEngine struct {
|
|||||||
// for the cold-path control lookups (Ping, TSMP, pendopen, etc).
|
// for the cold-path control lookups (Ping, TSMP, pendopen, etc).
|
||||||
peerForIP atomic.Pointer[func(netip.Addr) (_ PeerForIP, ok bool)]
|
peerForIP atomic.Pointer[func(netip.Addr) (_ PeerForIP, ok bool)]
|
||||||
|
|
||||||
|
// peerConfigFn, if non-nil, is the live per-peer allowed-IPs
|
||||||
|
// source installed via [userspaceEngine.SetPeerConfigFunc]. When
|
||||||
|
// set, wgdev's PeerLookupFunc queries it directly, so reconfigs
|
||||||
|
// no longer install per-config lookup closures.
|
||||||
|
peerConfigFn atomic.Pointer[func(key.NodePublic) (allowedIPs []netip.Prefix, ok bool)]
|
||||||
|
|
||||||
lastCfgFull wgcfg.Config
|
lastCfgFull wgcfg.Config
|
||||||
lastRouter *router.Config
|
lastRouter *router.Config
|
||||||
lastDNSConfig dns.ConfigView // or invalid if none
|
lastDNSConfig dns.ConfigView // or invalid if none
|
||||||
@@ -726,20 +732,84 @@ func (e *userspaceEngine) maybeReconfigWireguardLocked() error {
|
|||||||
e.peerByIPRoute.Store(rt)
|
e.peerByIPRoute.Store(rt)
|
||||||
|
|
||||||
e.logf("wgengine: Reconfig: configuring userspace WireGuard config (with %d peers)", len(full.Peers))
|
e.logf("wgengine: Reconfig: configuring userspace WireGuard config (with %d peers)", len(full.Peers))
|
||||||
if err := wgcfg.ReconfigDevice(e.wgdev, &full, e.logf); err != nil {
|
if e.peerConfigFn.Load() != nil {
|
||||||
|
// The device has a long-lived PeerLookupFunc backed by the
|
||||||
|
// live config source, so only the peer set needs syncing;
|
||||||
|
// there is no per-config lookup closure to (re)install, and
|
||||||
|
// no removed peer can be resurrected with stale state.
|
||||||
|
//
|
||||||
|
// TODO(bradfitz): remove this O(n peers) sync. It's redundant
|
||||||
|
// with the incremental SyncDevicePeer calls that LocalBackend
|
||||||
|
// makes for exactly the peers whose allowed IPs changed. It
|
||||||
|
// only remains because peer changes still force a full
|
||||||
|
// Reconfig; once that's gated on actual router/DNS changes,
|
||||||
|
// this sync (and full-config peer syncing generally) can go.
|
||||||
|
peers := make(map[device.NoisePublicKey][]netip.Prefix, len(full.Peers))
|
||||||
|
for _, p := range full.Peers {
|
||||||
|
peers[p.PublicKey.Raw32()] = p.AllowedIPs
|
||||||
|
}
|
||||||
|
e.wgdev.RemoveMatchingPeers(func(pk device.NoisePublicKey) bool {
|
||||||
|
_, exists := peers[pk]
|
||||||
|
return !exists
|
||||||
|
})
|
||||||
|
// Update AllowedIPs on any already-active peers whose config
|
||||||
|
// may have changed. Peers that don't exist yet will get the
|
||||||
|
// correct AllowedIPs from the device's PeerLookupFunc when
|
||||||
|
// they are lazily created.
|
||||||
|
for pk, allowedIPs := range peers {
|
||||||
|
if peer, ok := e.wgdev.LookupActivePeer(pk); ok {
|
||||||
|
peer.SetAllowedIPs(allowedIPs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if err := wgcfg.ReconfigDevice(e.wgdev, &full, e.logf); err != nil {
|
||||||
e.logf("wgdev.Reconfig: %v", err)
|
e.logf("wgdev.Reconfig: %v", err)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetPeerConfigFunc implements [Engine.SetPeerConfigFunc]. It stores
|
||||||
|
// fn and installs a single wgdev PeerLookupFunc wrapping it, so
|
||||||
|
// lazily-created peers always get current allowed IPs and the lookup
|
||||||
|
// func never needs to be reinstalled as the peer set changes.
|
||||||
|
func (e *userspaceEngine) SetPeerConfigFunc(fn func(key.NodePublic) (allowedIPs []netip.Prefix, ok bool)) {
|
||||||
|
if fn == nil {
|
||||||
|
panic("SetPeerConfigFunc: nil fn")
|
||||||
|
}
|
||||||
|
e.peerConfigFn.Store(&fn)
|
||||||
|
e.wgdev.SetPeerLookupFunc(wgcfg.NewPeerLookupFunc(e.wgdev.Bind(), e.logf, func(pubk device.NoisePublicKey) ([]netip.Prefix, bool) {
|
||||||
|
return fn(key.NodePublicFromRaw32(mem.B(pubk[:])))
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncDevicePeer implements [Engine.SyncDevicePeer].
|
||||||
|
func (e *userspaceEngine) SyncDevicePeer(k key.NodePublic) {
|
||||||
|
fn := e.peerConfigFn.Load()
|
||||||
|
if fn == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
e.wgLock.Lock()
|
||||||
|
defer e.wgLock.Unlock()
|
||||||
|
allowedIPs, ok := (*fn)(k)
|
||||||
|
if !ok {
|
||||||
|
e.wgdev.RemovePeer(k.Raw32())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if peer, ok := e.wgdev.LookupActivePeer(k.Raw32()); ok {
|
||||||
|
peer.SetAllowedIPs(allowedIPs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// SetPeerByIPPacketFunc installs a callback used by wireguard-go to look up
|
// SetPeerByIPPacketFunc installs a callback used by wireguard-go to look up
|
||||||
// which peer should handle an outbound packet by destination IP.
|
// which peer should handle an outbound packet by destination IP.
|
||||||
//
|
//
|
||||||
// fn is an optional fast path for exact node-address matches (e.g. dst is a
|
// If fn is non-nil it is authoritative: LocalBackend's implementation
|
||||||
// Tailscale IP). On miss (or if fn is nil), the engine's own BART table
|
// consults both the exact node-address fast path and the RouteManager's
|
||||||
// ([userspaceEngine.peerByIPRoute], built from the wireguard-filtered peer
|
// outbound table (covering subnet routes and exit-node default routes),
|
||||||
// list) is consulted to handle subnet routes and exit-node default routes.
|
// and stays correct under incremental netmap deltas. The engine's own
|
||||||
|
// BART table ([userspaceEngine.peerByIPRoute], rebuilt only on full
|
||||||
|
// reconfigs) is used only when no fn is installed (e.g. engines running
|
||||||
|
// without a LocalBackend).
|
||||||
//
|
//
|
||||||
// [NewUserspaceEngine] installs a BART-only default at engine creation time,
|
// [NewUserspaceEngine] installs a BART-only default at engine creation time,
|
||||||
// so callers that don't call SetPeerByIPPacketFunc (e.g. those not running
|
// so callers that don't call SetPeerByIPPacketFunc (e.g. those not running
|
||||||
@@ -750,6 +820,7 @@ func (e *userspaceEngine) SetPeerByIPPacketFunc(fn func(netip.Addr) (_ key.NodeP
|
|||||||
if pk, ok := fn(dst); ok {
|
if pk, ok := fn(dst); ok {
|
||||||
return pk.Raw32(), true
|
return pk.Raw32(), true
|
||||||
}
|
}
|
||||||
|
return device.NoisePublicKey{}, false
|
||||||
}
|
}
|
||||||
if rt := e.peerByIPRoute.Load(); rt != nil {
|
if rt := e.peerByIPRoute.Load(); rt != nil {
|
||||||
if pk, ok := rt.Lookup(dst); ok {
|
if pk, ok := rt.Lookup(dst); ok {
|
||||||
|
|||||||
@@ -18,6 +18,27 @@ func NewDevice(tunDev tun.Device, bind conn.Bind, logger *device.Logger) *device
|
|||||||
return device.NewDevice(tunDev, bind, logger)
|
return device.NewDevice(tunDev, bind, logger)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// NewPeerLookupFunc returns a [device.PeerLookupFunc] that lazily
|
||||||
|
// creates peers using allowedIPs as the source of each peer's allowed
|
||||||
|
// IPs. The peer's endpoint is derived from its public key via bind.
|
||||||
|
func NewPeerLookupFunc(bind conn.Bind, logf logger.Logf, allowedIPs func(device.NoisePublicKey) ([]netip.Prefix, bool)) device.PeerLookupFunc {
|
||||||
|
return func(pubk device.NoisePublicKey) (_ *device.NewPeerConfig, ok bool) {
|
||||||
|
ips, ok := allowedIPs(pubk)
|
||||||
|
if !ok {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
ep, err := bind.ParseEndpoint(fmt.Sprintf("%x", pubk[:]))
|
||||||
|
if err != nil {
|
||||||
|
logf("wgcfg: failed to parse endpoint for peer %x: %v", pubk[:8], err)
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
return &device.NewPeerConfig{
|
||||||
|
AllowedIPs: ips,
|
||||||
|
Endpoint: ep,
|
||||||
|
}, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ReconfigDevice replaces the existing device configuration with cfg.
|
// ReconfigDevice replaces the existing device configuration with cfg.
|
||||||
//
|
//
|
||||||
// Instead of using the UAPI text protocol, it uses the wireguard-go direct API
|
// Instead of using the UAPI text protocol, it uses the wireguard-go direct API
|
||||||
|
|||||||
@@ -197,6 +197,33 @@ type Engine interface {
|
|||||||
// look up which peer should handle an outbound packet by destination IP.
|
// look up which peer should handle an outbound packet by destination IP.
|
||||||
SetPeerByIPPacketFunc(func(netip.Addr) (_ key.NodePublic, ok bool))
|
SetPeerByIPPacketFunc(func(netip.Addr) (_ key.NodePublic, ok bool))
|
||||||
|
|
||||||
|
// SetPeerConfigFunc installs the live source of per-peer WireGuard
|
||||||
|
// configuration: given a peer's public key, fn returns the prefixes
|
||||||
|
// the peer is currently allowed to originate traffic from, or
|
||||||
|
// ok=false if the peer is unknown (in which case it must not exist
|
||||||
|
// in the WireGuard device). The engine installs a single
|
||||||
|
// [device.PeerLookupFunc] wrapping fn, so lazily-created peers
|
||||||
|
// always see current state and the lookup func never needs to be
|
||||||
|
// reinstalled as peers come and go.
|
||||||
|
//
|
||||||
|
// It is expected to be called once during LocalBackend construction,
|
||||||
|
// before the first [Engine.Reconfig]. fn is called rarely (when
|
||||||
|
// wireguard-go first hears from a peer it doesn't have) and may
|
||||||
|
// acquire locks.
|
||||||
|
SetPeerConfigFunc(fn func(key.NodePublic) (allowedIPs []netip.Prefix, ok bool))
|
||||||
|
|
||||||
|
// SyncDevicePeer synchronizes the WireGuard device's state for a
|
||||||
|
// single peer with the config source installed via
|
||||||
|
// [Engine.SetPeerConfigFunc]: if the source no longer knows the
|
||||||
|
// peer, it is removed from the device; if the peer is active in the
|
||||||
|
// device, its allowed IPs are updated. It does O(1) work (plus the
|
||||||
|
// config source lookup) and is intended to be called for each peer
|
||||||
|
// added, updated, or removed by an incremental netmap delta,
|
||||||
|
// avoiding a full [Engine.Reconfig].
|
||||||
|
//
|
||||||
|
// It is a no-op if no config source is installed.
|
||||||
|
SyncDevicePeer(key.NodePublic)
|
||||||
|
|
||||||
// SetNetLogSource installs the [NetLogSource] consulted by the
|
// SetNetLogSource installs the [NetLogSource] consulted by the
|
||||||
// engine's network flow logger for node lookups and the current
|
// engine's network flow logger for node lookups and the current
|
||||||
// audit logging identity.
|
// audit logging identity.
|
||||||
|
|||||||
Reference in New Issue
Block a user