I’ve been spending my free time digging into network virtualization and network overlays. This is part 1 of a 2 part series, part 2 can be found here: http://www.definethecloud.net/vxlan-deep-divepart-2. By far the most popular virtualization technique in the data center is VXLAN. This has as much to do with Cisco and VMware backing the technology as the tech itself. That being said VXLAN is targeted specifically at the data center and is one of many similar solutions such as: NVGRE and STT.) VXLAN’s goal is allowing dynamic large scale isolated virtual L2 networks to be created for virtualized and multi-tenant environments. It does this by encapsulating frames in VXLAN packets. The standard for VXLAN is under the scope of the IETF NVO3 working group.
The VXLAN encapsulation method is IP based and provides for a virtual L2 network. With VXLAN the full Ethernet Frame (with the exception of the Frame Check Sequence: FCS) is carried as the payload of a UDP packet. VXLAN utilizes a 24-bit VXLAN header, shown in the diagram, to identify virtual networks. This header provides for up to 16 million virtual L2 networks.
Frame encapsulation is done by an entity known as a VXLAN Tunnel Endpoint (VTEP.) A VTEP has two logical interfaces: an uplink and a downlink. The uplink is responsible for receiving VXLAN frames and acts as a tunnel endpoint with an IP address used for routing VXLAN encapsulated frames. These IP addresses are infrastructure addresses and are separate from the tenant IP addressing for the nodes using the VXLAN fabric. VTEP functionality can be implemented in software such as a virtual switch or in the form a physical switch.
VXLAN frames are sent to the IP address assigned to the destination VTEP; this IP is placed in the Outer IP DA. The IP of the VTEP sending the frame resides in the Outer IP SA. Packets received on the uplink are mapped from the VXLAN ID to a VLAN and the Ethernet frame payload is sent as an 802.1Q Ethernet frame on the downlink. During this process the inner MAC SA and VXLAN ID is learned in a local table. Packets received on the downlink are mapped to a VXLAN ID using the VLAN of the frame. A lookup is then performed within the VTEP L2 table using the VXLAN ID and destination MAC; this lookup provides the IP address of the destination VTEP. The frame is then encapsulated and sent out the uplink interface.
Using the diagram above for reference a frame entering the downlink on VLAN 100 with a destination MAC of 11:11:11:11:11:11 will be encapsulated in a VXLAN packet with an outer destination address of 10.1.1.1. The outer source address will be the IP of this VTEP (not shown) and the VXLAN ID will be 1001.
In a traditional L2 switch a behavior known as flood and learn is used for unknown destinations (i.e. a MAC not stored in the MAC table. This means that if there is a miss when looking up the MAC the frame is flooded out all ports except the one on which it was received. When a response is sent the MAC is then learned and written to the table. The next frame for the same MAC will not incur a miss because the table will reflect the port it exists on. VXLAN preserves this behavior over an IP network using IP multicast groups.
Each VXLAN ID has an assigned IP multicast group to use for traffic flooding (the same multicast group can be shared across VXLAN IDs.) When a frame is received on the downlink bound for an unknown destination it is encapsulated using the IP of the assigned multicast group as the Outer DA; it’s then sent out the uplink. Any VTEP with nodes on that VXLAN ID will have joined the multicast group and therefore receive the frame. This maintains the traditional Ethernet flood and learn behavior.
VTEPs are designed to be implemented as a logical device on an L2 switch. The L2 switch connects to the VTEP via a logical 802.1Q VLAN trunk. This trunk contains an VXLAN infrastructure VLAN in addition to the production VLANs. The infrastructure VLAN is used to carry VXLAN encapsulated traffic to the VXLAN fabric. The only member interfaces of this VLAN will be VTEP’s logical connection to the bridge itself and the uplink to the VXLAN fabric. This interface is the ‘uplink’ described above, while the logical 802.1Q trunk is the downlink.
Summary
VXLAN is a network overlay technology design for data center networks. It provides massively increased scalability over VLAN IDs alone while allowing for L2 adjacency over L3 networks. The VXLAN VTEP can be implemented in both virtual and physical switches allowing the virtual network to map to physical resources and network services. VXLAN currently has both wide support and hardware adoption in switching ASICS and hardware NICs, as well as virtualization software.
One small thing, and I’m being picky. A frame that is using 802.1Q is ‘ Ethernet tagged’ not ‘encapsulated’ cf. VXLAN is a MAC in IP encapsulation. Tagging uses much less CPU than encapsulation and inserts less latency.
Also, VXLAN is just one of many encapsulation protocols and primarily intended for use in Data Centers. Others include NVO3, NVGRE, STT and others. They all operate more or less the same as VXLAN.
Greg,
Thanks for the comment, I’ll make some changes/additions to incorporate that info.
Joe
There is also a new one called DOVE from IBM that uses the VXLAN header but isn’t multi cast based.
The NVo3 group is trying to come up with a Standard version that removes the need for customers to have to choose between VXLAN, STT, NVGRE, DOVE, .
There is a good chance that NVo3 will support more than one encapsulation method, and that one of those methods will use the VXLAN form so as to not hurt customers that buy hardware now that supports VXLAN.
(Disclaimer: I represent Brocade in the NVo3 WG)
Jon,
Thanks for reading the post and the additional info. What does DOVE use in place of IP multi-cast? If that’s too long winded for a comment feel free to point me in the direction that I can RTFM.
Joe
Hey Joe!
from the paper below “When multicast support is not available, a multi-unicast may be used”
but I would like more detail on how it really works, and can’t find anything very satisfying… but here is the paper from where the above quote came:
http://www.gta.ufrj.br/ensino/cpe728/p30-barabash.pdf
a presentation here
http://www.ethernetsummit.com/English/Collaterals/Proceedings/2012/20120222_2-103_Recio.pdf
and then from: http://searchnetworking.techtarget.com/feature/IBM-DOVE-Big-Blue-enters-the-network-virtualization-battleground
“But DOVE differs from VXLAN in its ability to create an overlay without requiring the physical infrastructure to operate in multicast. VXLAN requires multicast, which many network engineers are reluctant to implement, especially over data center interconnects.”
Jon,
Awesome, thanks for the links and detail!
Joe
VXLAN incurs a potentially large TAX (performance cost) in vSphere 5.1 environments:
http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-VXLAN-Perf.pdf
that hardly looks to be a large TAX (performance cost).
“or a small message size, throughput scaled linearly as we added more virtual
machines. For large message sizes, we were able to achieve line rate for all configurations. There is some CPU
overhead due to the use of VXLAN. This overhead varies based on the configuration and is due to lack of
hardware offloads for encapsulated packets. “
For example, you may write several articles for Associated Content that are related to each
other. This could, however, estrange the relationship
between the creators and the strategists or SEO. With online businesses,
it’s very important to have good uptime, which means
no downtime.
Finding the right house for your family means finding the best nest will help you to take care of your kids better.
Some companies will make an adjustable crankset for shorter or taller people too.
Getting the services of Long Distance Moving Company make your moving easy and very much less taxing.
Woah! I’m really enjoying the template/theme of this blog.
It’s simple, yet effective. A lot of times it’s hard to get that “perfect balance” between usability and visual appearance.
I must say that you’ve done a amazing job with this.
Also, the blog loads very fast for me on Chrome. Exceptional
Blog!
Please let me know if you’re looking for a writer for your blog.
You have some really good articles and I think I would be
a good asset. If you ever want to take some of the load off, I’d
really like to write some content for your blog in exchange for a link back to mine.
Please shoot me an e-mail if interested. Regards!
I am really loving the theme/design of your site.
Do you ever run into any internet browser compatibility issues?
A small number of my blog audience have complained
about my blog not operating correctly in Explorer but looks great in Firefox.
Do you have any advice to help fix this issue?
Excellent post. Keep writing such kind of information on your page.
Im really impressed by it.
Hi there, You’ve performed a great job. I’ll definitely digg it and in my view suggest to my friends.
I am sure they’ll be benefited from this website.
Have you ever considered about including a little bit more than just your articles?
I mean, what you say is valuable and everything.
However think of if you added some great images or video clips to give your posts more,
“pop”! Your content is excellent but with images and videos, this website could certainly be one of the most beneficial in its niche.
Great blog!
Thanks for a marvelous posting! I definitely enjoyed
reading it, you will be a great author.I will make sure to bookmark your
blog and may come back later in life. I want to encourage youhrself to continue our great work, have a nice morning!
The cause for so many Miami Kitchen remodeling projects is simply because people are deciding
to save a lot more funds by spending their time
inside the residence and cooking fresh meals that cost much less than a night out inside the town. With these facts, it became apparent that
no matter how great these cabinets looked — I was not going
to buy something made of particle board. If the cost estimates are too low,
the project will go over budget – sometimes significantly so.
Please let me know if you’re looking for a article writer for your blog.
You have some really great articles and I think I would be a good asset.
If you ever want to take some of the load off,
I’d really like to write some articles for your blog
in exchange for a link back to mine. Please shoot me an e-mail if interested.
Thanks!
Hi there to every one, it’s truly a nice for mee to visit this website, it includes
helpful Information.
Appreciating the time and effort you put into your blog and in depth information you present.
It’s awesome to come across a blog every once in a while that isn’t the
same outdated rehashed material. Great read! I’ve saved your site and I’m including your RSS feeds to
my Google account.
Great items from you, man. I have take into account your stuff previous to and you’re simply extremely fantastic.
I really like what you have acquired right here, certainly like what you’re stating and
the way in which in which you are saying it. You’re
making it enjoyable and you continue to care for to keep it smart.
I can not wait to read far more from you. That is actually a terrific web site.
Testosterone is converted to dihydrotestosterone (DHT) which slows
down or stops hair growth and produces weaker, shorter hair.
The platinum blonde is a much sought after persona, and the
look, an elegant and beautiful one if carried off
properly, is a much desired and much employed part of dressing
up. The magnitude, the effect rate, and the duration of the hair loss depend on the phase when the interference occurred.
whoah this blog is great i really like studying your articles.
Keep up the good work! You realize, lots of people are searching
round for this info, you can aid them greatly.
It’s difficult to find experienced people about this subject,
but you sound like you know what you’re talking about!
Thanks
Highly descriptive blog, I loved that bit.
Will there be a part 2?
Hmm is anyone else having problems with the images on this blog
loading? I’m trying to figure out if its a problem on my end or if it’s
the blog. Any feed-back would be greatly appreciated.
Attractive section of content. I just stumbled upon your weblog
and in accession capital to assert that I get actually
enjoyed account your blog posts. Any way I will be subscribing to your augment and even I achievement you access consistently quickly.
Greetings I am so glad I found your weblog, I really found you by mistake, while I was browsing on Askjeeve for something else, Anyhow I am here now and would just like to say thanks a lot for a tremendous post and a all round enjoyable blog (I also love
the theme/design), I don’t have time to look over it all at
the minute but I have book-marked it and also added in your RSS
feeds, so when I have time I will be back to read a lot more, Please do
keep up the excellent work.
fantastic issues altogether, you simply won a emblem new reader.
What would you recommend about your post that you made some days ago?
Any sure?
Hey there! I’m at work surfing around your blog from my new iphone
3gs! Just wanted to say I love reading your blog and look forward to all your posts!
Keep up the fantastic work!
If you would like to improve your knowledge simply keep visiting this web page and be updated with the newest
news update posted here.
Everything is very open with a clear clarification of the issues.
It was truly informative. Your website is very helpful.
Many thanks for sharing!
I’m impressed, I must say. Rarely do I encounter a blog that’s both equally educative and interesting,
and without a doubt, you’ve hit the nail on the head.
The issue is something which not enough people are speaking intelligently about.
I’m very happy that I found this in my search for something regarding this.
I like the valuable information you provide on your articles.
I will bookmark your weblog and check once more here frequently.
I am relatively sure I’ll be informed a lot of new stuff proper
here! Best of luck for the following!
Thanks for every other informative web site. Where else could I get
that kind of information written in such an ideal way? I have a project that I’m simply now running on, and I have been at the look out for such information.
Our auto repair shop in Blaine, MN proudly serves
residents in the areas of Fridley, and Spring Lake Park, Minnesota.
Franchot went on to add “you are making a significant contribution to our great state of Maryland and I thank you. Wards Automotive is reporting that Sonic Automotive, one of the nation’s largest automotive retail chains is going to “break
the mold” of the typical used car dealership on both fronts.
Fish are so sensitive to the drinking glasses. We will show you what the machine.
Mesh and other sorts of cleaning might cause
allergens that have learned to resst on furniture
and carpeting to get rid of pesticides, chlorine itself is a water ionizer available?
You don’t get the minerals inn the stomach,
and the acid alkaline pH of 7.
Do you have a spam problem on this site; I also am a blogger,
and I was wanting to know your situation; we have created some nice procedures and we are
looking to swap strategies with other folks, please shoot me an email if interested.
Payday two was released in August 2013 and has so far received one particular major DLC pack, Large Bank, with three other heists including
a diamond heist, a casino heist and a prison break planned for
the future.
I do accept as true with all of the ideas you have presented in your post.
They are really convincing and will certainly work.
Still, the posts are very brief for newbies. Could you please lengthen them a little from subsequent time?
Thank you for the post.
Aw, this was an incredibly good post. Taking the time and actual effort to produce a good article… but what can I say… I procrastinate a
whole lot and never manage to get anything done.
One of these is garcinia cambogia, which has been actually used for thousands
of years already. Like the studies show, you should expect to lose up to 10 pounds in a month without making
any other changes in your life. Hydroxycitric acid essentially regulates
your appetite, so that you eat just the right quantity
of food your body requires to function normally.
He has put all his cash resources into the move but he is
currently getting 2 new clients a day just because of his new position. If you are setting up a bookkeeping
business, part of the battle of success is actually
being tenacious and keeping going when you feel like quitting.
Once completed, the template can have the images added and is then ready
for printing.
Y hungry is a website that features interactive menus allowing customers
to place orders with local restaurants and food cooperatives.
In fact thousands of people now have all of their meals delivered to their home because eating a strict, low calorie diet really can help you shed some
weight, particularly if you can squeeze in a little bit of exercise as well.
Some of the most important features offered at
the restaurant include catering, takeout and delivery.
I’ve been surfing online more than 4 hours today, yet I never found any
interesting article about like yours. It’s pretty worth enough for me.
In my view, if all psychics and bloggers made good content as you did,
the web will be much more useful than ever
before.
There’s definately a great deal to know about this topic.
I like all of the points you have made.
I’m amazed, I must say. Seldom do I come across a
blog that’s both equally educative and engaging, and without a
doubt, you’ve hit the nail on the head. The problem is
something not enough folks are speaking intelligently about.
I am very happy I found this during my search
for something concerning this.
Again, though there have been many studies done that try
to tell us that we cannot eat a chocolate bar if we want
healthy skin or that a slice of pizza is the absolute enemy
in the war against acne, there is no absolute answer as to which foods cause acne breakouts.
Now gently rub it on the scar in circular motion before rinsing with cold water.
Not only will it cause more pimples to pop out later, but it will also leave a
permanent pimple scar.