How to route host traffic over lxc?

Question

Some info updated:

Last login: Wed Aug 31 18:10:24 2022
root@pve:~# ip route
default via 192.168.132.1 dev vmbr0 proto kernel onlink
192.168.132.0/24 dev vmbr0 proto kernel scope link src 192.168.132.4
root@pve:~#
root@pve:~# ping 10.0.0.26
PING 10.0.0.26 (10.0.0.26) 56(84) bytes of data.
64 bytes from 10.0.0.26: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 10.0.0.26: icmp_seq=2 ttl=64 time=0.051 ms

10.0.0.26 was the ip that wan interface of openwrt lxc got from the upstream router, it replied the request from the pve host, my guess is through the loopback because ping to 10.0.0.1 or any 10.0.x.x won't get replied.

--

I got the situation that I'm unable to migrate my openwrt vm to lxc .. I used the vm to route and manage the host's network via its virtual bridges but when I used the same configuration on lxc, it doesn't work.

/etc/hosts

127.0.0.1 localhost.localdomain localhost
192.168.132.4 pve

# The following lines are desirable for IPv6 capable hosts

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

/etc/network/interfaces

iface enp3s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address 192.168.132.4/24
        gateway 192.168.132.1
        bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
        bridge-ports enp3s0
        bridge-stp off
        bridge-fd 0

210.conf

cores: 1
memory: 128
net0: name=eth0,bridge=vmbr1,hwaddr=CA:2B:9D:E6:52:08,type=veth
net1: name=eth1,bridge=vmbr0,hwaddr=FA:24:4E:32:4B:9B,type=veth
ostype: unmanaged
rootfs: datastore1:210/vm-210-disk-0.raw,size=204M
swap: 512

/etc/config/network of the OpenWrt container

config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config interface 'wan'
    option ifname 'eth0'
    option proto 'dhcp'

config interface 'wan6'
    option proto 'dhcpv6'
    option ifname '@wan'
    option reqaddress 'try'
    option reqprefix 'auto'

config interface 'lan'
    option proto 'static'
    option ifname 'eth1'
    option type 'bridge'
    option netmask '255.255.255.0'
    option ipaddr '192.168.132.1'

My upstream router ip was 10.0.0.1

Ping from the container (192.168.132.1) to anywhere(including the host, upstream LAN and public) => works

Ping from the host (192.168.132.4) to 192.168.132.1 => works

Ping from the host (192.168.132.4) to external 10.0.0.1 => doesn't work

It was used to work on the vm, but seems not the same case on lxc.

Please shed some light on how to resolve this ..

The same question I asked in the forum

Answer 1

What you need is so called NAT hairpinning (aka NAT loopback, NAT reflection):

NAT loopback, [...] is a feature in many consumer routers which permits the access of a service via the public IP address from inside the local network.

Why is it not working? The IPTables DNAT rule you provided in your question specifies an incoming interface this rule should apply to: -i eth0. Your traffic isn’t coming from eth0 however, but some virtual network interface or whatever. Just removing this restriction to eth0 may be sufficient to make it work.

Alternatively, a separate rule could be added to lxcbr0:

iptables -t nat -A PREROUTING -i lxcbr0 -p tcp --dport 443
--destination 80.x.x.x -j DNAT --to 10.0.3.100:443

Answer 2

Well I just solved it by simply restarting the firewall service until the default policies applied. I add the following statements to my startup script:

until $(iptables -t nat -L PREROUTING |grep -q 'zone_lan_prerouting')
do
    /etc/init.d/firewall restart
    sleep 1
done

So /etc/rc.local would look like this:

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

until $(iptables -t nat -L PREROUTING |grep -q 'zone_lan_prerouting')
do
    /etc/init.d/firewall restart
    sleep 1
done

exit 0

It seems the network changes are somehow not detected when it's a lxc and restarting of the service is therefore not triggered, no additional policies than the defaults are needed.