Proxmox 3.4-6 IPV6 Networking - ARP Issues

We have just deploy PROXMOX and trying to use IPv6 Networking on VMs.

1.- In brand new Host (Proxmox) we add IPv6 addressing:

root@server:~# more /etc/network/interfaces

By default vmbr0 interface is edited (as it is being used as bridge to VMs)

auto vmbr0
iface vmbr0 inet static
 address  A.B.C.D        
 netmask  255.255.255.192        
 gateway  A.B.C.1        
 bridge_ports eth1        
 bridge_stp off        
 bridge_fd 0

iface vmbr0 inet6 static 

        address 2187:f0d2:3034:de::137

        netmask 64 

        gateway 2187:f0d2:3034:de::1

2.- Verify configuration update after a networking restart

root@server:~# ifconfig vmbr0 

vmbr0     Link encap:Ethernet  HWaddr 00:25:90:57:db:47           

inet addr:A.B.C.D  Bcast:A.B.C.191  Mask:255.255.255.192          

inet6 addr: 2187:f0d2:3034:de::137/64 Scope:Global          

inet6 addr: fe80::225:90ff:fe57:db47/64 Scope:Link          

UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1          

RX packets:3239 errors:0 dropped:0 overruns:0 frame:0          

TX packets:1722 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0 

RX bytes:803279 (784.4 KiB)   TX bytes:1530868 (1.4 MiB)

root@server:~# route -n6 

Kernel IPv6 routing tableDestination                    Next Hop                   Flag Met Ref Use If2187:f0d2:3034:de::/64         ::                         U    256 0     1 vmbr0 

fe80::1/128                    ::                         U    256 0     0 venet0 

fe80::/64                      ::                         U    256 0     0 vmbr0 

fe80::/64                      ::                         U    256 0     0 eth0 

fe80::/64                      ::                         U    256 0     0 eth1 

fe80::/64                      ::                         U    256 0     0 venet0 

::/0                           2187:f0d2:3034:de::1       UG   1024 0    18 vmbr0 

::/0                           ::                         !n   -1  1    26 lo 

::1/128                        ::                         Un   0   1     4 lo 

2187:f0d2:3034:de::137/128     ::                         Un   0   1    39 lo 

fe80::1/128                    ::                         Un   0   1     0 lo 

fe80::225:90ff:fe57:db46/128   ::                         Un   0   1     0 lo 

fe80::225:90ff:fe57:db47/128   ::                         Un   0   1     0 lo 

fe80::225:90ff:fe57:db47/128   ::                         Un   0   1     0 lo 

ff00::/8                       ::                         U    256 0     0 vmbr0 

ff00::/8                       ::                         U    256 0     0 eth0 

ff00::/8                       ::                         U    256 0     0 eth1 

ff00::/8                       ::                         U    256 0     0 venet0 

::/0                           ::                         !n   -1  1    26 lo

3.- Try some ping connectivity to GW. Of course maybe you will receive no response.

root@server:~# ping6  2187:f0d2:3034:de::1 

PING 2187:f0d2:3034:de::1(2607:f0d0:3001:de::1) 56 data bytes 

From 2187:f0d2:3034:de::137 icmp_seq=1 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=3 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=4 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=5 Destination unreachable: Address unreachable 

^C 

--- 2187:f0d2:3034:de::1 ping statistics ---6 packets transmitted, 0 received, +4 errors, 100% packet loss, time 5000ms

root@server:~# ping6 ipv6.google.com 

PING ipv6.google.com(iad23s43-in-x0e.1e100.net) 56 data bytes 

From 2187:f0d2:3034:de::137 icmp_seq=1 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=3 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=4 Destination unreachable: Address unreachable 

From 2187:f0d2:3034:de::137 icmp_seq=5 Destination unreachable: Address unreachable 

^C--- 

ipv6.google.com ping statistics ---7 packets transmitted, 0 received, +3 errors, 100% packet loss, time 6064ms

4.- Adding Loopback interface like this:

iface lo inet6 loopback

5.- Troubleshooting

Verify ARP table (ip -6 neigh)

root@server:~# ip neigh ls 

2187:f0d2:3034:de::1 dev vmbr0  FAILED 

A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE

Showing that ARP resolution failed. So You have to verify properly configuration of IPv6 address on the other side (the router gateway).

I am discarding some physical connectivity issue as my router was resolved properly against its IPv4 address. (assigned - dual stack for GW)

You could also verify Table:

root@server:~# arp -na 

? (A.B.C.1) at 00:08:e3:ff:fd:90 [ether] on vmbr0

Again: Resolution for your GW over IPv6 failed.

6.- Force your server (temporal testing only)

If your NOC or Networking specialist confirmed you that IPv6 address is already assigned to GW try to force adding that entry into ARP table:

ip -6 neigh replace 2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90

Then you could ping GW and ping6 everywhere.

If that worked for you... then you could add that permanent rule inside /etc/network/interface file

iface vmbr0 inet6 static 

    address 2187:f0d2:3034:de::137 

    netmask 64 

    gateway 2187:f0d2:3034:de::1 

    post-up sysctl -p 

    post-up ip -6 neigh replace 2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90

PD: If you want to remove ARP entry you could use:
Review:

root@server:~# ip neigh ls 

2187:f0d2:3034:de::1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 PERMANENT 

A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE 

 Delete

root@server:~# ip -6 neigh del 2187:f0d2:3034:de::1 dev vmbr0 

Verify:

root@server:~# ip neigh ls 

2187:f0d2:3034:de::1 dev vmbr0  FAILED 

A.B.C.1 dev vmbr0 lladdr 00:08:e3:ff:fd:90 REACHABLE

UPDATE: Depending on your Proxmox Kernel, you will need to update it to discard ICMPv6 discovery issues.
More info is posted here: