The development of pf within OpenBSD has been a ground-breaking technology for the open source community, both in performance and feature set, and the use of multi-path routing is a popular method to aggregate bandwidth from cheap, consumer connections to provide faster and more robust connectivity.

Leaving aside any discussion on the wisdom of this, we outline a method for achieving it; including two features not mentioned within the OpenBSD or pf FAQs[[0|nor found through cursory searches in mailing list archives]].

  1. Weighted and mixed policy, multi-path routing
  2. Fault redundancy

The following setup (using OpenBSD 5.0) will allow for simple, scalable, dynamic routing across multiple links with automatic fail-over and the abiltity to integrate ALTQ traffic shaping[[1| experienced sys admins will, of course, realise that this simply means we are going to use the inbuilt features of OpenBSD and pf]].

Whilst the same or modified configurations of this may be possible, they have not been considered and are left to the reader.

Routing configuration

The irony of this configuration is that kernel level multipath routing is, in fact, of little value to us. Firstly, it is not weighted and thus won't meet the stated objectives of the article and secondly because, as is pointed out in the FAQ

... if an interface used by a multipath route goes down (i.e., loses carrier), the kernel will still try to forward packets using the route that points to that interface

which fails regarding redunancy. The FAQ goes on to recommend the use of ifstated to deal with this but suggested methods[1] for achieving this are weak.

More importantly, however, multipath routing is valueless on most commodity internet links as the kernel does not intelligently route traffic and thus a connection on one link will likely have packets routed on the other and, as a consequence, fail. Of course, this is not a failure of multipath routing but we again touch upon the wisdom of using it for our stated objective; and once again dodge the issue by saying that we do not. Instead we utilise pf and ifstated.

What this means in practice is that we do not configure a default route for our host. Whilst this is generally simple (i.e. do nothing), transient links like DHCP and PPP must be configured to capture, but ignore, the default route. We'll return to this subject after dealing with the primary principles of the setup.

Basic setup

Since we have no 'default' route, we instead setup pf using it's route-to to push traffic out to available gateways. This setup has the advantage of being stateful by default and thus communication channels remain implicitly associated with a single link and traffic (which is commonly NAT'd) flows as expected.

We then combine this setup with ifstated and two more pf features, 'tables' and 'anchors' which allow us to simply and dynamically alter the firewall configuration in pre-defined ways.

pf configuration

inbound routing choices

must be made inbound

nat must be done outbound

notes about stateless connections * we're not dealing with that here * stateless protocols will be subject to normal expiration

Re-routing outbound traffic to utilise least-state NAT policy

ifstated configuration

haven't thought about this properly and need to figure out the details of the pf configuration (i.e. the anchors and tables) as well as how we wish to monitor for connectivity (barring the obvious link status -- which we wish to use too).

Configuration of transient links


Don't do this. Use proper external services to relay redundantly to commodity links.

References and supplementary links

[[mail list searches - "inet failover solution" - "best way" failover link ]] pf faq openbsd faq