AZiT

Exceptions: the PS5 and guests

Two networks leave directly, without weakening the kill switch for the rest.

Why Exceptions by policy routing, on the tour

On this page 4 sections

A game console plays better on a direct path, and guests have no reason to use your VPN. Their internet rules move their packets into the WAN's routing table and tag them; the WAN rules let only tagged packets through, so every other network keeps the full kill switch.

1. What changes in pf.conf#

Four lines. First, tagged packets are translated to the WAN's address:

from /etc/pf.conf
match out on $wan inet tagged DIRECT nat-to ($wan)

Second, the kill switch lets tagged packets through, and only those:

from /etc/pf.conf
pass out quick on $wan inet tagged DIRECT label "allow_direct_out"

And the two networks' internet rules route through the WAN's table and tag their packets:

from /etc/pf.conf
pass in log quick on $ps5 inet from $ps5:network to ! <internal_nets> rtable 1 tag DIRECT label "allow_ps5_internet_direct"
pass in log quick on $guest inet from $guest:network to ! <internal_nets> rtable 1 tag DIRECT label "allow_guest_internet_direct"

pf's state tracking carries the replies back across the two routing domains by itself.

2. Their DNS goes direct too#

Every other network asks the firewall's unbound, which forwards through the tunnel. If the PS5 and guests did too, they would lose name resolution whenever the tunnel is down, even though their route out still works: in practice, no internet at all. So DHCP gives these two networks a public resolver instead, and pf lets their lookups out directly, tagged like the rest of their traffic. The rule sits above block_external_dns, which would otherwise catch it.

from /etc/pf.conf
table <public_dns> const { 9.9.9.9 149.112.112.112 }
pass in quick on { $ps5 $guest } inet proto { tcp udp } to <public_dns> port { domain 853 } \
    rtable 1 tag DIRECT label "allow_direct_dns"

In dhcpd.conf, the PS5 and guest subnets already hand out Quad9 instead of the firewall: that line has been in the file since chapter 4. Until this chapter their lookups simply took the normal route.

3. The complete file#

/etc/pf.confDownload
Show the whole file, 117 lines
# /etc/pf.conf - fw.internal
# Default deny. Blocked traffic is logged, and so is every new connection
# allowed by a pass rule marked "log". Every rule has a label; pflog2syslog
# adds it to each log line as label=<name>, which Vector turns into rule.name.
# --- Interfaces ----
wan = "igc0"
admin = "igc1"
apmgmt = "igc2"
ps5 = "igc3"
media = "igc4"
servers = "igc5"
trusted = "vlan10"
iot = "vlan20"
guest = "vlan30"
vpn = "wg0"
internal = "{ igc1 igc2 igc3 igc4 igc5 vlan10 vlan20 vlan30 }"
# --- Tables ----
table <admin_hosts> const { 192.168.1.10 192.168.10.10 }
table <k8s_nodes> const { 192.168.5.10 192.168.5.20 192.168.5.30 192.168.5.40 192.168.5.50 }
table <k8s_api> const { 192.168.5.100 }
table <k8s_services> const { 192.168.5.200/29 192.168.5.208/29 192.168.5.216/30 192.168.5.220 }
table <unifi_host> const { 192.168.5.60 }
table <unifi_ap> const { 192.168.2.2 }
table <home_assistant> const { 192.168.5.140 }
table <kodi> const { 192.168.4.10 }
table <public_dns> const { 9.9.9.9 149.112.112.112 }
table <vpn_server> const { @VPN_SERVER_ADDRESS@ }
table <internal_nets> const { 192.168.1.0/24 192.168.2.0/24 192.168.3.0/24 192.168.4.0/24 \
    192.168.5.0/24 192.168.10.0/24 192.168.20.0/24 192.168.30.0/24 }
table <martians> const { 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16 \
    172.16.0.0/12 192.0.0.0/24 192.0.2.0/24 192.168.0.0/16 \
    198.18.0.0/15 198.51.100.0/24 203.0.113.0/24 224.0.0.0/3 }
# --- Options ----
set block-policy drop
set skip on lo
set loginterface $wan
# --- Normalisation and NAT ----
match in all scrub (no-df random-id)
match out on $vpn inet from <internal_nets> to any nat-to ($vpn)
match on $vpn scrub (max-mss 1380)
match out on $wan inet tagged DIRECT nat-to ($wan)
# --- Default: block and log everything, no IPv6 ----
block log all label "drop_all"
block quick inet6 all label "drop_ipv6"
# --- DHCP (before the spoofing checks; not logged, dhcpd logs it) ----
pass in quick on $wan inet proto udp from any port bootps to any port bootpc label "allow_wan_dhcp"
pass in quick on $internal inet proto udp from any port bootpc to any port bootps label "allow_dhcp"
# --- Spoofing ----
block in log quick from urpf-failed label "block_spoofed_source"
block in log quick on $wan from <martians> label "block_wan_martians"
# --- Kill switch, layer 2: on WAN only the tunnel, DHCP and DIRECT traffic may leave ----
pass out quick on $wan inet proto udp to <vpn_server> port 51820 label "allow_wan_wireguard"
pass out quick on $wan inet proto udp from any port bootpc to any port bootps label "allow_wan_dhcp_out"
pass out quick on $wan inet tagged DIRECT label "allow_direct_out"
block out log quick on $wan label "block_wan_leak"
# --- Everything the firewall lets in may also leave (logged inbound) ----
pass out quick inet label "allow_out"
# --- Services on the firewall (not logged: very chatty) ----
pass in quick on $internal inet proto { tcp udp } to (self) port domain label "allow_dns_to_fw"
pass in quick on $internal inet proto udp to (self) port ntp label "allow_ntp_to_fw"
pass in quick on $internal inet proto icmp to (self) icmp-type echoreq label "allow_ping_to_fw"
# The PS5 and guests bypass the tunnel, so their DNS does too: a public resolver, straight out
pass in quick on { $ps5 $guest } inet proto { tcp udp } to <public_dns> port { domain 853 } rtable 1 tag DIRECT label "allow_direct_dns"
block in log quick on $internal inet proto { tcp udp } to ! (self) port { domain 853 } label "block_external_dns"
# --- Admin access (logged) ----
pass in log quick on { $admin $trusted } inet proto tcp from <admin_hosts> to (self) port ssh label "allow_admin_ssh_fw"
pass in log quick on { $admin $trusted } inet proto tcp from <admin_hosts> to { <k8s_nodes> <unifi_host> <unifi_ap> <kodi> } port ssh label "allow_admin_ssh_hosts"
pass in log quick on { $admin $trusted } inet proto tcp from <admin_hosts> to <k8s_api> port 6443 label "allow_admin_k8s_api"
pass in log quick on { $admin $trusted } inet proto tcp from <admin_hosts> to <k8s_services> port https label "allow_admin_ingress"
pass in log quick on { $admin $trusted } inet proto tcp from <admin_hosts> to <unifi_host> port 11443 label "allow_admin_unifi_ui"
# WebRTC from the UniFi web UI: direct path from the admin laptop to the controller
pass in log quick on { $admin $trusted } inet proto { tcp udp } from <admin_hosts> to <unifi_host> label "allow_admin_unifi_webrtc"
pass in log quick on { $admin $trusted } inet proto icmp from <admin_hosts> to <internal_nets> icmp-type echoreq label "allow_admin_ping"
# --- Known noise: dropped without logging ----
# Broadcast and multicast discovery (NetBIOS, SSDP, ...)
block in quick on $internal inet to { 255.255.255.255 224.0.0.0/4 192.168.1.255 192.168.2.255 192.168.3.255 192.168.4.255 192.168.5.255 192.168.10.255 192.168.20.255 192.168.30.255 } label "drop_broadcast_multicast"
# Late TCP packets (FIN, ACK, RST+ACK) for connections pf no longer tracks.
# New connections (bare SYN) and tracked connections never reach this rule.
block in quick on $internal inet proto tcp from <internal_nets> flags A/A label "drop_late_tcp"
# WebRTC (UniFi UI) also tries the firewall's public address as a detour.
# Reject it immediately, so it gives up at once.
block return in quick on { $admin $trusted $servers } inet proto udp from { <admin_hosts> <unifi_host> } to ($wan) label "reject_webrtc_wan_detour"
# --- Nothing else may reach any of the firewall's own addresses ----
block in log quick on $internal to (self) label "block_to_fw"
# --- Between networks (logged) ----
pass in log quick on $trusted inet proto tcp from $trusted:network to <home_assistant> port 8123 label "allow_trusted_homeassistant"
pass in log quick on $trusted inet proto tcp from $trusted:network to <kodi> port { 18080 9090 } label "allow_trusted_kodi"
pass in log quick on $servers inet from <home_assistant> to $iot:network label "allow_homeassistant_iot"
pass in quick on $apmgmt inet proto tcp from <unifi_ap> to <unifi_host> port 8080 label "allow_ap_unifi_inform"
pass in log quick on $apmgmt inet proto udp from <unifi_ap> to <unifi_host> port 3478 label "allow_ap_unifi_stun"
pass in log quick on $apmgmt inet proto udp from <unifi_ap> to <unifi_host> port 5514 label "allow_ap_unifi_logs"
# The AC Lite checks its uplink by pinging 1.1.1.1 (not configurable in UniFi 10.x); not logged
pass in quick on $apmgmt inet proto icmp from <unifi_ap> to 1.1.1.1 icmp-type echoreq label "allow_ap_uplink_ping"
pass in log quick on $servers inet proto tcp from <unifi_host> to <unifi_ap> port ssh label "allow_unifi_ap_ssh"
# --- Internet access (logged; APMGMT gets none) ----
pass in log quick on $admin inet from $admin:network to ! <internal_nets> label "allow_admin_internet"
pass in log quick on $trusted inet from $trusted:network to ! <internal_nets> label "allow_trusted_internet"
pass in log quick on $ps5 inet from $ps5:network to ! <internal_nets> rtable 1 tag DIRECT label "allow_ps5_internet_direct"
pass in log quick on $media inet from $media:network to ! <internal_nets> label "allow_media_internet"
pass in log quick on $servers inet from $servers:network to ! <internal_nets> label "allow_servers_internet"
pass in log quick on $iot inet from $iot:network to ! <internal_nets> label "allow_iot_internet"
pass in log quick on $guest inet from $guest:network to ! <internal_nets> rtable 1 tag DIRECT label "allow_guest_internet_direct"
  • This is the firewall's policy as it runs today.
laptop
scp pf.conf fw:/tmp/pf.conf
firewall
doas sh -c 'sleep 300 && pfctl -d' &
doas pf-load

As before: a new SSH session from the laptop, then doas pkill -f 'sleep 300'.

Check

firewall
doas pfctl -s labels | grep -E 'direct'

The four labels allow_ps5_internet_direct, allow_guest_internet_direct, allow_direct_dns and allow_direct_out. The first number after a label counts how often pf evaluated the rule, the second the packets it matched: that one rises as the console and guests use the internet.

Check

A phone on the guest Wi-Fi sees your internet provider's address on a "what is my IP" page; a phone on the trusted Wi-Fi sees the VPN's. The guest Wi-Fi needs the access point: if it is not set up yet, come back to this check after chapter 1 of the Access point guide.

If a phone on the guest Wi-Fi shows a 169.254.… address, it got no answer from DHCP, perhaps because it tried before the rules were loaded, and gave up. Turn its Wi-Fi off and on. doas tail -f /var/log/daemon on the firewall shows each request and the network it came in on: DHCPREQUEST … via vlan30.

4. Prove they do not need the tunnel#

A phone on the guest Wi-Fi should see that it uses Quad9 (https://on.quad9.net says so) and your provider's address. Then take the tunnel down:

firewall
doas ifconfig wg0 down

Check

On the guest phone, a site you have not visited today still opens: a new name, so the answer cannot come from a cache. The laptop, on the admin network, has no internet at all.

firewall
doas ifconfig wg0 up

The price: guests' lookups are not private from your internet provider, and get none of the VPN resolver's blocking. For guests and a console that is the right way round.