So far, I suspect it's a PF issue.
- It's happened on two different machines with different hardware (xl(4) versus rl(4))
- It usually manifests more frequently the longer the machine has been running
- It only happens with PF under moderate (DSL) network load
- It happens regardless of what secondary rules (if any) are loaded
- It only affects packets that are not part of an established session (ICMP, UDP, TCP SYN)
- It manifests as a huge delay, and when it stops happening there's a burst of activity
64 bytes from 66.219.31.21: icmp_seq=40050 ttl=55 time=42.371 ms 64 bytes from 66.219.31.21: icmp_seq=40051 ttl=55 time=56.848 ms 64 bytes from 66.219.31.21: icmp_seq=40052 ttl=55 time=57.587 ms 64 bytes from 66.219.31.21: icmp_seq=40053 ttl=55 time=43.788 ms 64 bytes from 66.219.31.21: icmp_seq=40054 ttl=55 time=75.965 ms 64 bytes from 66.219.31.21: icmp_seq=40055 ttl=55 time=67.637 ms ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available ping: sendto: No buffer space available 64 bytes from 66.219.31.21: icmp_seq=40056 ttl=55 time=22934.168 ms 64 bytes from 66.219.31.21: icmp_seq=40057 ttl=55 time=21959.228 ms 64 bytes from 66.219.31.21: icmp_seq=40079 ttl=55 time=45.744 ms 64 bytes from 66.219.31.21: icmp_seq=40080 ttl=55 time=48.900 ms 64 bytes from 66.219.31.21: icmp_seq=40081 ttl=55 time=33.343 ms 64 bytes from 66.219.31.21: icmp_seq=40082 ttl=55 time=35.842 ms 64 bytes from 66.219.31.21: icmp_seq=40083 ttl=55 time=32.758 ms
- Bouncing the interface (ifconfig down / ifconfig up) does not help
Bouncing PF (pfctl -d / pfctl -e) does not work
- When the firewall is disabled, packets flow fine, but without NAT
- When the firewall is reenabled, symptoms return immediately
- Reloading the ruleset and queue descriptors (pfctl -f /etc/pf/base.pf) instantly resolves the issue