Skip to content
Define The Cloud

The Intersection of Technology and Reality

Define The Cloud

The Intersection of Technology and Reality

VXLAN Deep Dive

Joe Onisick (@JoeOnisick), November 6, 2012May 18, 2020

I’ve been spending my free time digging into network virtualization and network overlays.  This is part 1 of a 2 part series, part 2 can be found here: http://www.definethecloud.net/vxlan-deep-divepart-2.  By far the most popular virtualization technique in the data center is VXLAN.  This has as much to do with Cisco and VMware backing the technology as the tech itself.  That being said VXLAN is targeted specifically at the data center and is one of many similar solutions such as: NVGRE and STT.)  VXLAN’s goal is allowing dynamic large scale isolated virtual L2 networks to be created for virtualized and multi-tenant environments.  It does this by encapsulating frames in VXLAN packets.  The standard for VXLAN is under the scope of the IETF NVO3 working group.

 

VxLAN Frame

The VXLAN encapsulation method is IP based and provides for a virtual L2 network.  With VXLAN the full Ethernet Frame (with the exception of the Frame Check Sequence: FCS) is carried as the payload of a UDP packet.  VXLAN utilizes a 24-bit VXLAN header, shown in the diagram, to identify virtual networks.  This header provides for up to 16 million virtual L2 networks.

Frame encapsulation is done by an entity known as a VXLAN Tunnel Endpoint (VTEP.)  A VTEP has two logical interfaces: an uplink and a downlink.  The uplink is responsible for receiving VXLAN frames and acts as a tunnel endpoint with an IP address used for routing VXLAN encapsulated frames.  These IP addresses are infrastructure addresses and are separate from the tenant IP addressing for the nodes using the VXLAN fabric.  VTEP functionality can be implemented in software such as a virtual switch or in the form a physical switch.

VXLAN frames are sent to the IP address assigned to the destination VTEP; this IP is placed in the Outer IP DA.  The IP of the VTEP sending the frame resides in the Outer IP SA.  Packets received on the uplink are mapped from the VXLAN ID to a VLAN and the Ethernet frame payload is sent as an 802.1Q Ethernet frame on the downlink.  During this process the inner MAC SA and VXLAN ID is learned in a local table.  Packets received on the downlink are mapped to a VXLAN ID using the VLAN of the frame.  A lookup is then performed within the VTEP L2 table using the VXLAN ID and destination MAC; this lookup provides the IP address of the destination VTEP.  The frame is then encapsulated and sent out the uplink interface.

image

Using the diagram above for reference a frame entering the downlink on VLAN 100 with a destination MAC of 11:11:11:11:11:11 will be encapsulated in a VXLAN packet with an outer destination address of 10.1.1.1.  The outer source address will be the IP of this VTEP (not shown) and the VXLAN ID will be 1001.

In a traditional L2 switch a behavior known as flood and learn is used for unknown destinations (i.e. a MAC not stored in the MAC table.  This means that if there is a miss when looking up the MAC the frame is flooded out all ports except the one on which it was received.  When a response is sent the MAC is then learned and written to the table.  The next frame for the same MAC will not incur a miss because the table will reflect the port it exists on.  VXLAN preserves this behavior over an IP network using IP multicast groups.

Each VXLAN ID has an assigned IP multicast group to use for traffic flooding (the same multicast group can be shared across VXLAN IDs.)  When a frame is received on the downlink bound for an unknown destination it is encapsulated using the IP of the assigned multicast group as the Outer DA; it’s then sent out the uplink.  Any VTEP with nodes on that VXLAN ID will have joined the multicast group and therefore receive the frame.  This maintains the traditional Ethernet flood and learn behavior.

VTEPs are designed to be implemented as a logical device on an L2 switch.  The L2 switch connects to the VTEP via a logical 802.1Q VLAN trunk.  This trunk contains an VXLAN infrastructure VLAN in addition to the production VLANs.  The infrastructure VLAN is used to carry VXLAN encapsulated traffic to the VXLAN fabric.  The only member interfaces of this VLAN will be VTEP’s logical connection to the bridge itself and the uplink to the VXLAN fabric.  This interface is the ‘uplink’ described above, while the logical 802.1Q trunk is the downlink.

image

Summary

VXLAN is a network overlay technology design for data center networks.  It provides massively increased scalability over VLAN IDs alone while allowing for L2 adjacency over L3 networks.  The VXLAN VTEP can be implemented in both virtual and physical switches allowing the virtual network to map to physical resources and network services.  VXLAN currently has both wide support and hardware adoption in switching ASICS and hardware NICs, as well as virtualization software.

Share this:

  • Facebook
  • X

Related posts:

  1. VXLAN Deep Dive – Part II
  2. FCoE initialization Protocol (FIP) Deep Dive
  3. NVGRE
  4. Stateless Transport Tunneling (STT)
  5. Network Abstraction and Virtualization: Where to Start?
Technical Deep Dive Data CenterNetwork virtualizationnetworkingPrivate CloudVirtualizationVXLAN

Post navigation

Previous post
Next post

Related Posts

Access Layer Network Virtualization: VN-Tag and VEPA

September 12, 2010

One of the highlights of my trip to lovely San Francisco for VMworld was getting to join Scott Lowe and Brad Hedlund for an off the cuff whiteboard session.  I use the term join loosely because I contributed nothing other than a set of ears.  We discussed a few things,…

Share this:

  • Facebook
  • X
Read More
Technical Deep Dive

Why Cisco UCS is my ‘A-Game’ Server Architecture

May 23, 2010December 28, 2018

A-Game: When I discuss my A-Game it’s my go to hardware vendor for a specific data center component.  For example I have an A-Game platform for: Storage SAN LAN (access Layer LAN specifically, you don’t want me near your aggregation, core or WAN) Servers and Blades (traditionally this has been…

Share this:

  • Facebook
  • X
Read More

Server Networking With gen 2 UCS Hardware

October 22, 2011May 18, 2020

** this post has been slightly edited thanks to feedback from Sean McGee** In previous posts I’ve outlined: How UCS server failover occurs from a network perspective: http://www.definethecloud.net/ucs-server-failover How Inter-fabric traffic is handled in End-Host mode: http://www.definethecloud.net/inter-fabric-traffic-in-ucs How inter-fabric traffic is handled in switch mode: http://www.definethecloud.net/inter-fabric-traffic-in-ucspart-ii If you’re not familiar…

Share this:

  • Facebook
  • X
Read More

Comments (72)

Comments navigation

Older comments
  1. dyson hot and cool fan says:
    May 15, 2016 at 5:31 pm

    Hi to all, because I am really eager of reading this webpage’s
    post to be updated on a regular basis. It consists of pleasant data.

  2. jacks blowjob lessons pdf says:
    May 22, 2016 at 12:28 pm

    When some onne searches for his vital thing, so he/she desires to bbe
    available that in detail, so that thing is maintained
    over here.

  3. jacks blow job lessons says:
    May 24, 2016 at 11:37 am

    Tattoos were covered up, Rosacea, birth marks – it seemed
    to be able to do a fantastic job at covering without actually looking like
    makeup. Now gently rub it on the scar in circular motion before rinsing with
    cold water. Anti-oxidant chemicals are valued for their wide healing actions.

  4. highperformancehandbookpdf.blogspot.com says:
    June 1, 2016 at 7:16 pm

    I don’t even understand how I finished up here, but I assumed this
    publish was once good. I don’t understand who you’re however definitely you are going to a well-known blogger in the event you aren’t already.
    Cheers!

  5. menjual blazer pria harga paling murah says:
    June 5, 2016 at 9:57 am

    What i doo not realize iis if truth be told how you’re
    not really a lot more smartly-favored than you may bbe right
    now. You are very intelligent. You already know therefore signifiantly in relation to this subject, produced me for my part imagine it
    from a loot oof numerous angles. Its like women and
    men are not involved until it’s one thing to ddo
    with Lady gaga! Your personal stuffs great. Always
    dea with itt up!

  6. Imran Hussain VAT Fraud says:
    August 8, 2016 at 7:16 pm

    A Top Crimebuster Exposed How a Crack Squad were Sent to
    Dubai – An Exclusive News
    Recently a top crimebuster has revealed the news related to crack squad, which was sent to Dubai.
    The main aim was to hunt down the VAT criminals, who were threatening to bankrupt Britain.
    According to David Odd, the criminals involved in this carousel fraud took £3.5 billion from
    taxpayers. The worst part is that it took them only one year to accomplish their fraudulent goals.

    Assistant Director of HM Revenue and Customs Investigation, Scotland said that his officials were
    helping the authorities of other European countries to deal with carousel fraud
    and it was in fact this cooperation that lead to unearthing of massive scam in Scotland back in 1990.

    The accused claimed VAT on goods, as according to them they imported the goods and then exported them
    abroad. The amazing part was that those goods
    never existed. No one had an idea that the total scam would
    cost £3.5 billion in 2006.
    David Odd believes if these carousel frauds are not checked on time, it would lead to Treasury losses that
    would remain unrecoverable at large and lead the whole country towards an unavoidable financial crises.

    The infamous Imran Hussain, known as Immy from Glasgow,
    34 years old man, was also found in connection with the fraudsters.
    He is allegedly responsible for a VAT scam of £300
    million.
    Later on HMRC introduced a verification program to control VAT repayments claims.
    A lot of efforts are being made for the efficient
    control and is expected to reduce the impact of VAT fraud due to these

  7. free chips says:
    August 15, 2016 at 3:53 am

    Chickens are, after all, farmyard animals so there will be a
    certain amount of odors coming form the coop. With many people vying
    for the same thing, prices are pushed upwards. Pick
    your login name through the boot screen and you will start into Windows, where you are able
    to run System improve from the Start selection as typical.

  8. Pingback: VXLAN چیست؟ - شبکه نرم افزار محور
  9. Happy Luke says:
    December 28, 2016 at 3:40 pm

    My coder is trying to convince me to move to .net from PHP.
    I have always disliked the idea because of the costs.

    But he’s tryiong none the less. I’ve been using Movable-type on a
    number of websites for about a year and am concerned about switching to
    another platform. I have heard great things about blogengine.net.

    Is there a way I can transfer all my wordpress posts into
    it? Any kind of help would be greatly appreciated!

  10. iifymstarterguide.blogspot.com says:
    March 30, 2017 at 2:19 am

    Magnificent itwms from you, man. I have bear in mind your stuff
    previous to and you are simply extremely wonderful.
    I ctually like what you have bought here, really like what you are saying
    and the way whjerein youu assert it. You make it entertaining and you still care
    for to keep it wise. I can’t wait to read much more from
    you. This is really a great web site.

  11. TerrellceTq says:
    April 1, 2017 at 8:15 am

    I see your site needs some unique articles. Writing manually is time consuming,
    but there is solution for this hard task. Just search for;
    Miftolo’s tools rewriter

  12. Keep Fit says:
    April 30, 2017 at 11:00 pm

    You’re quite welcome.

  13. fort bend county homes says:
    May 26, 2017 at 5:28 am

    Hi there, I want to subscribe for this weblog to obtain most recent updates,
    thus where can i do it please help.

  14. Pabrikbrcjakarta.blogspot.com says:
    September 12, 2019 at 3:49 am

    I am in fact thankful to the owner of this web site who
    has shared this great post at here.

  15. href="https://fud-cheat.com/pubg-mobile-cheat-undetected">PUBG Mobile cheat says:
    November 24, 2019 at 6:56 am

    I love Disney Magic Kingdom, it’s such a great game to play! It lets you play with your favourite Disney characters and actually see the magic of the park! I need the gems because I’ve only just started the game in the last week or so, and the extra gems will give me a boost to catch up to everyone who has been playing for so long already!

  16. Pingback: VXLAN-A BRIEF INTRODUCTION – Tech.Thirst
  17. Pingback: NVGRE – Define The Cloud

Comments navigation

Older comments

Comments are closed.

Creative Commons License
This work by Joe Onisick and Define the Cloud, LLC is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License

Disclaimer

All brand and company names are used for identification purposes only. These pages are not sponsored or sanctioned by any of the companies mentioned; they are the sole work and property of the authors. While the author(s) may have professional connections to some of the companies mentioned, all opinions are that of the individuals and may differ from official positions of those companies. This is a personal blog of the author, and does not necessarily represent the opinions and positions of his employer or their partners.
©2025 Define The Cloud | WordPress Theme by SuperbThemes