Nexus 1000v – DMZ Private VLAN config example


Here is something I am often asked about, but I have not found many examples on the internet.  The actual configuration of 1000v port-profiles when used in a DMZ environment.

In my example environment, I am referencing a single ESXi host, attached to  upstream DMZ switches for external/internet bound traffic.  These DMZ switches are already in use for standalone DMZ hosts, and the VLANs are already in place.  The host is also attached to internal switches/firewall for internal and VMWare Kernel/Mgmt network traffic, including 1000v system VLANs.  The host has the 1000v / VSM’s installed already. If you do not have enough free NICs on your host, consider combining internal and VM Mgmt.

Config:

On Upstream DMZ switches:  A simple trunk port of the DMZ VLANs

(this is just one example)
interface GigabitEthernet5/5
 description Trunk To 1000v host external NICs
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 199,200
 switchport mode trunk
 logging event link-status
 spanning-tree portfast trunk

Nexus Config Translation
 interface e101/1/1
  description Trunk to 1000v host external NICs
  switchport mode trunk
  switchport trunk allow vlan 199-200
  spanning-tree port type edge trunk

On the Upstream Internal switches – another set of trunk ports

interface GigabitEthernet6/2
 description Trunk To 1000v host Internal NICs
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 55
 switchport mode trunk
 logging event link-status
 spanning-tree portfast trunk

interface GigabitEthernet7/2
 description Trunk To 1000v host VM Mgmt NICs
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 3000-3004
 switchport mode trunk
 logging event link-status
 spanning-tree portfast trunk

On the 1000v:

First the VLANs:

vlan 199
  name DMZ-PVLAN-Primary
  private-vlan primary
  private-vlan association 200
vlan 200
  name DMZ-PVLAN-Isolated
  private-vlan isolated
vlan 55
  name Internal Traffic
vlan 3000
  name Vmotion
vlan 3001
  name VM Service Console
vlan 3002
  name 1000v Packet
vlan 3003
  name 1000v Control
vlan 3004
  name 1000v Mgmt

Now the Uplink Ethernet Port-Profiles

port-profile type ethernet External
  vmware port-group
  switchport mode private-vlan trunk promiscuous
  switchport private-vlan mapping trunk 199 200
  switchport private-vlan trunk allowed vlan 199,200
  channel-group auto mode on mac-pinning
  no shutdown
  state enabled
port-profile type ethernet Internal
  vmware port-group
  swichport mode trunk
  switchport trunk allowed vlan 55
  channel-group auto mode on mac-pinning
  no shutdown
  state enabled
port-profile type ethernet VM-Mgmt
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 3000-3004
  channel-group auto mode on mac-pinning
  no shutdown
  system vlan 3002-3003
  state enabled

Lastly, the VM facing vEthernet Port-Profiles

port-profile type vethernet DMZ External Traffic
  vmware port-group
  switchport mode private-vlan host
  switchport private-vlan host-association 199 200
  no shutdown
  state enabled
port-profile type vethernet 1000v-Control
  vmware port-group
  switchport mode access
  switchport access vlan 3002
  system vlan 3002
  no shutdown
  state enabled
port-profile type vethernet 1000v-Packet
  vmware port-group
  switchport mode access
  switchport access vlan 3003
  system vlan 3003
  no shutdown
  state enabled
port-profile type vethernet 1000v-Mgmt
  vmware port-group
  switchport mode access
  switchport access vlan 3004
  no shutdown
  state enabled
port-profile type vethernet VMotion
  vmware port-group
  switchport mode access
  switchport access vlan 3000
  no shutdown
  state enabled
port-profile type vethernet VM-Srv-Console
  vmware port-group
  switchport mode access
  switchport access vlan 3001
  no shutdown
  state enabled
port-profile type vethernet Internal Traffic
  vmware port-group
  switchport mode access
  switchport access vlan 55
  no shutdown
  state enabled

What is the net result?  VM’s in the same port-profile cannot communicate. Traffic goes to the correct uplink port-profile, as long as switchport trunk statements are correct.  You may also consider a VMWare vDS for the vMotion, service console and 1000v mgmt traffic.

This document provides more information on the 1000v in DMZ environments

DMZ Virtualization Using VMware vSphere 4 and the Cisco
Nexus 1000V Virtual Switch

Here is another post talking about 1000v and DMZs

Two vSwitches are better than 1, right?

12 thoughts on “Nexus 1000v – DMZ Private VLAN config example”

  1. Pingback: Trunking Administrative Modes | Cisco Skills

  2. Great site! Lot’s of good info.

    Issue:

    Let’s say you have only 2x1Gb Ethernet ports on your ESX host. From what I have tested on the 1000v it appears that you cannot configure a PVLAN promiscuous trunk in a port-profile that has a system VLAN assigned. So In this case is it correct that you would need one dedicated link for mgmt/system/etc and the other to configure your PVLAN related information? What if you wanted to configure LACP for these two links to you upstream physical switch? Any thoughts on this?

    Reply

    • Hi,

      Thanks for your questions/comments.

      First I suggest that you should try to find a host with at least 4 connections if you are going to implement some private VLANs and avoid the problem.

      If you cannot implement “system vlan” on the ethernet port-profile, then yes you will need another ethernet port-profile for mgmt. This means at least you would need to split up the physical uplinks. You will not be able to do LACP upstream because you would need different configs(trunking different VLANs) on each port.

      1000v Port-Channel Best Practices:

      Click to access n1k-port-channels.pdf

      Reply

  3. Another interesting limitation is that if you wanted to configure multiple primary private VLANs, with associated Secondary’s it appears that you can only configure one primary for the promiscuous trunk mappings to your physical switch. I guess inherently the one PVLAN will take care of this, but if you had the need to configure multiple primary’s for regulatory compliance or just plain old “optics” you would not be able to. Unless there is workaround for this.

    Thanks a lot for your time!

    Reply

    • Hi,

      I believe you can have multiple sets of private VLANs with primary/secondary associations.

      On the upstream switch, its just a standard trunk port, with all the VLANs needed trunked. On the 1000v ethernet (uplink) port-profile, you have

      …
      switchport mode private-vlan trunk promiscuous
      switchport private-vlan mapping trunk 100 101
      switchport private-vlan mapping trunk 200 201
      switchport private-vlan trunk allowed vlan 100,101,200,201
      …

      Then its up to you to setup the primary/secondary mappings in the VLAN configs and the vEthernet port-profiles

      Hope this helped!

      DCH

      Reply

      • Apparently it only excepts one “switchport private-vlan mapping trunk” statement. If you try to add a second it overwrites the initial. Version 4.2(1)SV1(4a)…

  5. How would you configure the interfaces if you are using converged networking (CNA Adapters) and don’t have seprate physical interfaces for VM-mgmt, Internal and External?

    Reply

    • As a previous commenter mentioned, you cannot add “system vlan x,x” command to a promiscuous Ethernet port-profile, so you will not be able two get away with only 2 uplinks. System Vlan is critical for 1000v Control and Packet, and is often recommended for Service Console/VMkernel. You could use only 1 uplink each and create two separate port-profiles, but you are severely compromising your redundancy.

      My suggestion would be to get another ethernet adapter in the host. Perhaps add a quad port 1Gb adapter? If you could only get 2 more 10Gb ports, then combine VM-Mgmt with Internal, and keep External seperate.

      -DCH

      Reply

  6. Good Post, in principle it works, but I have the problem that customer wants to use community-vlans where some VMs are in and also some Hosts on the physical Switch.
    so far so good will work with the config you posted, but between the N1KV and the physical Switch there we have UCS FI in Endhostmode in between (On this all VLANs are configured as regular VLANs)

    So configuring the N1KV uplink as pvlan trunk promiscous forces the N1KV to tag all packet which come from the community-vlan with the tag from the primary vlan.
    So UCS learns MAC Address of VM in primary vlan. The Packet reaches the Host in a community-port of the physical switch. this host answers but its packets are tagged with VLAN-ID of community-VLAN , reaches UCS, and UCS drops packet because it has never seen MAC-Address of VM in the community-VLAN only in primary VLAN.

    Any idea how to go arround ?

    Reply

    • Could you post your 1) VLAN config on 1000v 2) VLAN config on DMZ Switch 3) Uplink Port-profile from 1000v 4) Port-config from upstream switch that UCS FI’s are connected to ?

      Also – is the UCS connected to 2 separate L2 domains?

      thanks – DCH

      Reply

      • Catalyst

        vlan 240
        private-vlan primary
        private-vlan association 250

        vlan 250
        private-vlan community
        !
        interface GigabitEthernet1/0/21
        description agritsch pvlan Test
        switchport private-vlan host-association 240 250
        switchport mode private-vlan host
        !
        interface GigabitEthernet1/0/25
        description Trunk_to_UCS_FI-B
        switchport trunk encapsulation dot1q
        switchport mode trunk
        spanning-tree portfast trunk
        !
        interface GigabitEthernet1/0/26
        description Trunk_to_UCS_FI-A
        switchport trunk encapsulation dot1q
        switchport mode trunk
        spanning-tree portfast trunk

        N1KV:
        Scenario 1 (communication between VM and phys Host in community VLAN works, but communication between SVI and VM does not work)

        vlan 240
        private-vlan primary
        private-vlan association 250
        vlan 250
        private-vlan community

        port-profile type ethernet pvlan-uplink
        vmware port-group
        switchport mode trunk
        switchport trunk allowed vlan 240-260
        channel-group auto mode on mac-pinning
        no shutdown
        description PVLAN-UPLINK
        state enabled

        port-profile type vethernet pvlan240
        vmware port-group
        switchport mode private-vlan host
        switchport private-vlan host-association 240 250
        no shutdown
        state enabled

        Scenario 2 (communication between VM and phys Host in community VLAN does not work, but communication between SVI and VM works)

        vlan 240
        private-vlan primary
        private-vlan association 250
        vlan 250
        private-vlan community

        port-profile type ethernet pvlan-uplink
        vmware port-group
        switchport mode private-vlan trunk promiscuous
        switchport private-vlan trunk allowed vlan 240-260
        switchport private-vlan mapping trunk 240 250
        channel-group auto mode on mac-pinning
        no shutdown
        description PVLAN-UPLINK
        state enabled

        port-profile type vethernet pvlan240
        vmware port-group
        switchport mode private-vlan host
        switchport private-vlan host-association 240 250
        no shutdown
        state enabled

        The VLANs on UCS are configured as “normal” not shared VLANs (UCS does not support community-VLANs, just isolated 😦

Leave a reply to OffTheGrid Cancel reply