UFW all traffic through tunnel
To use UFW to direct all traffic though a certain interface (like an OpenVPN tunnel) you can use UFW. It comes pre-installed with Ubuntu but not enabled. The only issue with it is that you have to disable the firewall, connect to your tunnel then re-enable the firewall to make it work. I find that's acceptable.
First, deny all outgoing and incoming
sudo ufw default deny outgoing && sudo ufw default deny incoming
Now, allow all traffic on your tunnel (in my case, it was tun0 - look at ifconfig. It may say something like tun0-00 but just use tun0)
sudo ufw allow out on tun0 from any to any && sudo ufw allow in on tun0 from any to any
Now, connect to your tunnel and enable the firewall
sudo ufw enable. Check that you can connect to the internet. Then, disconnect from your tunnel and try to access the internet (or just do a ping to 126.96.36.199).
You should not be able to access the internet.
To allow traffic to a certain IP address in your local network, you can do
sudo ufw allow out to 10.0.0.5 or to allow access to a range
sudo ufw allow out to 10.0.0.0/8 (obviously replace 10.0.0.x with your internal IP address/range). The more restrictive the better in my opinion.