I was curious if the experts here could give some feedback on preferred /etc/network/interfaces config for Proxmox VE 8 with 4 available NICs. I plan to use Clustering with Corosync and Ceph.
I have separate VLANs for management, corosync, ceph, and other virtualized services.
This is the config I was thinking about, but not sure if it is even necessary to do all the breaking apart of the vlans, etc. Plus with this setup I can ping across VLAN 2,3, and 4 on the machine itself, but can't touch that machine from other computers so there is certainly something wrong in that regard. I thought they would all share the gateway I have setup on vmbr0.2 but it doesn't seem to be the case.
auto lo
iface lo inet loopback
iface enp87s0 inet manual
iface enp88s0 inet manual
iface enp2s0f0 inet manual
iface enp2s0f1 inet manual
iface wlp89s0 inet manual
auto vmbr0
iface vmbr0 inet manual
bridge-ports enp87s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2
auto vmbr0.2
iface vmbr0.2 inet static
address 192.168.2.20/24
gateway 192.168.2.1
#Management
auto vmbr1
iface vmbr1 inet manual
bridge-ports enp88s0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 3
auto vmbr1.3
iface vmbr1.3 inet static
address 192.168.3.20/24
#CoroSync
auto vmbr2
iface vmbr2 inet manual
bridge-ports enp2s0f0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 4
auto vmbr2.4
iface vmbr2.4 inet static
address 192.168.4.20/24
#Ceph
auto vmbr3
iface vmbr3 inet manual
bridge-ports enp2s0f1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 5-998
#Services
source /etc/network/interfaces.d/*
ip link show
to confirm your diagram. If you use PVE firewall, it would complicate the structure quite substantially; bear with it or not use it. I wouldn't be able to help without such a diagram.