As everyone mentions, vCloud Network Isolation (VCNI) is the most complicated type of network pool in VMware vCloud Director. It is a proprietary technique (apparently by VMware) that uses MAC-in-MAC encapsulation to distinguish between different private networks in a single physical VLAN.
Among all, VCNI has a big advantage for cloud administrators: It mitigates their need to deal with physical network administrators, because multiple VLANs can be created inside a single carrier VLAN; while in other types of network pools, a VLAN should exist or be created in physical network. Also, since it uses a proprietary technique to create virtual VLANs! (I know, it’s like Virtual Virtual LAN!) the number of VLANs is not limited (to 4096). Of course it’s not infinite, but it’s a very big number: 4 Millions. See here for more details.
However, implementing this type of network pool has a trick! Again, because it encapsulates networking packets, it has its own overhead which is 24 bytes. So, assuming that you create a vCloud Network Isolation network pool (as shown above), you are not done yet. You need to change the value of MTU to 1524 (to be safe, 1600 is recommended) in 3 levels:
- vCloud Director – It’s a secret to me why VMware doesn’t assign 1524 by default while it knows VCNI needs it! You can do this by right-clicking over this network pool and clicking ‘Properties’, then go to: ‘Network Pool MTU’ and change it to 1600.
- vCenter: Go to Home, Networking, choose the distributed switch between hosts; right-click and Edit Settings, select Advanced; change the value of Maximum MTU to 1600.
- Physical switch – Depends on your equipment, but should be done.
Now that I encountered the steps required to have an operational VCNI and also mentioned advantages, keep in mind that there are some disadvantages for this type of network pool that you can find them in this great link explaining more details:
vCloud Director Networking – Part 2 in VMware Technologies Blog
p.s – If MTU is not changed, VCNI will still work but with poor performance because of fragmentation.