A Few Good Apps

Developer: Network team, did you order the Code upgrade?!

Operations Manager: You don’t have to answer that question!

Network Engineer: I’ll answer the question. You want answers?

Developer: I think I’m entitled!

Network Engineer: You want answers?!

Developer: I want the truth!

Network Engineer: You can’t handle the truth! Son, we live in a world that has VLANs, and those VLANs have to be plumbed by people with CLIs. Who’s gonna do it? You? You, Database Admin? I have a greater responsibility than you can possibly fathom. You weep for app agility and you curse the network. You have that luxury. You have the luxury of not knowing what I know, that network plumbing, while tragically complex, delivers apps. And my existence, while grotesque and incomprehensible to you, delivers apps! You don’t want the truth, because deep down in places you don’t talk about at parties, you want me on that CLI. You need me on that CLI. We use words like “routing”, “subnets”, “L4 Ports”. We use these words as the backbone of a life spent building networks. You use them as a punch line. I have neither the time nor the inclination to explain myself to a man who rises and sleeps under the blanket of infrastructure that I provide, and then questions the manner in which I provide it! I would rather you just said “thank you”, and went on your way. Otherwise, I suggest you pick up a putty session, and configure a switch. Either way, I don’t give a damn what you think you are entitled to!

Developer: Did you order the Code upgrade?

Network Engineer: I did the job that—-

Developer: Did you order the Code upgrade?!!

Network Engineer: YOU’RE GODDAMN RIGHT I DID!!


In many IT environments today there is a distinct line between the application developers/owners and the infrastructure teams that are responsible for deploying those applications.  These organizational silos lead to tension, lack of agility and other issues.  Much of this is caused by the translation between these teams.  Application teams speak in terms like: objects, attributes, provider, consumer, etc.  Infrastructure teams speak in memory, CPU, VLAN, subnets, ports.  This is exacerbated when delivering apps over the network, which requires connectivity, security, load-balancing etc.  On today’s network devices (virtual or physical) the application must be identified based on Layer 3 addressing and L4 information.  This means the app team must be able to describe components or tiers of an app in those terms (which are foreign to them.)  This slows down the deployment of applications and induces problems with tight controls, security, etc.  I’ve tried to describe this in the graphic below (for people who don’t read good and want to learn to do networking things good too.)


As shown in the graphic, the definition of an application and its actual instantiation onto networking devices (virtual and physical) is very different.  This causes a great deal of the slowed application adoption and the complexity of networking.  Today’s networks don’t have an application centric methodology for describing applications and their requirements.  The same can be said for emerging SDN solutions.  The two most common examples of SDN today are OpenFlow and Network Virtualization.  OpenFlow simply attempts to centralize a control plane that was designed to be distributed for scale and flexibility.  In doing so it  uses 5-tuple matches of IP and TCP/UDP headers to attempt to identify applications as network flows.  This is no different from the model in use today.  Network virtualization faithfully replicates today’s network constructs into a hypervisor, shifting management and adding software layers without solving any of the underlying problem.

What’s needed is a common language for the infrastructure teams and development teams to use.  that common language can be used to describe application connectivity and policy requirements in a way that makes sense to separate parts of the organization and business.  Cisco Application Centric Infrastructure (ACI) uses policy as this common language, and deploys the logical definition of policy onto the network automatically.

Cisco ACI bases network provisioning on the application and the two things required for application delivery: connectivity and policy.  By connectivity we’re describing what group of objects is allowed to connect to other groups of objects.  We are not defining forwarding, as forwarding is handled separately using proven methods, in this case ISIS with a distributed control plane.  When we describe connectivity we simply mean allowing the connection.  Policy is a broader term, and very important to the discussion.  Policy is all of the requirements for an application: SLAs, QoS, Security, L4-7 services etc.  Policy within ACI is designed using reusable ‘contracts.’  This way policy can be designed in advance by the experts and architects with that skill set and then reused whenever required for a new application roll-out.

Applications are deployed on the ACI fabric using an Application Network Profile. An application network profile is simply a logical template for the design and deployment of an applications end-to-end connectivity and policy requirements.  If you’re familiar with Cisco UCS it’s a very similar concept to the UCS Service Profile.  One of the biggest benefits of an Application Network profile is its portability.  They can be built through the API, or GUI, downloaded from Cisco Developer Network (CDN) or the ACI Github community, or provided by the application vendor itself.  They’re simply an XML or JSON representation of the end-to-end requirements for delivering an application.  The graphic below shows an application network profile.


This model provides that common language that can be used by developer teams and operations/infrastructure teams.  To tie this back to the tongue-in-cheek start to this post based on dialogue from “A Few Good Men”, we don’t want to replace the network engineer, but we do want to get them off of the CLI.  Rather than hacking away at repeatable tasks on the command line, we want them using the policy model to define the policy ‘contracts’ for use when deploying applications.  At the same time we want to give them better visibility into what the application requires and what it’s doing on the network.  Rather than troubleshooting devices and flows, why not look at application health?  Rather than manually configuring QoS based on devices, why not set it per application or tier?  Rather than focusing on VLANs and subnets as policy boundaries why not abstract that and group things based on those policy requirements?  Think about it, why should every aspect of a servers policy change because you changed the IP?  That’s what happens on today’s networks.

Call it a DevOps tool, call it automation, call it what you will, ACI looks to use the language of applications to provision the network dynamically and automatically.  Rather than simply providing better management tools for 15 year old concepts that have been overloaded we focus on a new model: application connectivity and policy.

**Disclaimer: I work as a Technical Marketing Engineer for the Cisco BU responsible for Nexus 9000 and ACI.  Feel free to disregard this post as my biased opinion.**

GD Star Rating

True Software Defined Networking (SDN)

The world is, and has been, buzzing about software defined networking. It’s going to revolutionize the entire industry, commoditize hardware, and disrupt all the major players. It’s going to do all that… some day. To date it hasn’t done much but be a great conversation, and more importantly identify the need for change in networking.

In its first generation SDN is a lot of sizzle with no flash. The IT world is trying to truly define it, much like we were with ‘Cloud’ years ago. What’s beginning to emerge is that SDN is more of a methodology then an implementation, and like cloud there are several implementations: OpenFlow, Network Virtualization and Programmable Network Infrastructure.




Open Flow focuses on a separation of control plane and data plane. This provides a centralized method to route traffic based on a 5-tuple match of packet header information. One area OpenFlow falls short is in its dependence on the independent advancement of the protocol itself and the hardware support below. Hardware in the world of switching and routing is Application Specific Integrated Circuits (ASIC) based, and those ASICs typically take three years to refresh. This means that the OpenFlow protocol itself must advance, and then once stabilized silicon vendors can begin building new ASICs to be available three years later.

Network Virtualization

Network virtualization is a faithful reproduction of networking functionality into the hypervisor. This method is intended to provide advanced automation and speed application deployment. The problem here arises in the new tools required to manage and monitor the network, the additional management layer, and the replication of the same underlying complexity.

Programmable Network Infrastructure

Programmable network infrastructure takes the configuration of devices from human to machine CLI/GUI interfaces to APIs and programming agents. This allows for faster, more powerful and less error prone device configuration from automation, orchestration and cloud operating system tools. These advance the configuration of multiple disparate systems but are still designed based on network operating system constructs intended for human use, and the same underlying network complexities such as artificial ties between addressing and policy.

All of these generation 1 SDN solutions simply move the management of the underlying complexity around. They are software designed to operate in the same model, trying to configure existing hardware. They’re simply adding another protocol, or protocols, to the pile of existing complexity.


Truly software defined networks

To truly define the network via software you have to look at the entire solution, not just a single piece. Simply adding a software or hardware layer doesn’t fix the problem, you must look at them in tandem starting with the requirements for today’s networks: automation, application agility, visibility (virtual/physical) security, scale and L4-7 services (virtual/physical.)

If you start with those requirements and think in terms of a blank slate you now have the ability to build things correctly for today and tomorrow’s applications while ensuring backwards compatibility. The place to start is in the software itself, or the logical model. Begin with questions:

1. What’s the purpose of the network?

2. What’s most relevant to the business?

3. What dictates the requirements?

The answer to all three is the application, so that’s the natural starting point. Next you ask who owns, deploys and handles day two operations for an application? The answer is the development team. So you start with a view of applications in a format they would understand.


That format is simple provider/consumer relationships between tiers or components of an application. Each tier may provide and consume services from the next to create the application which is a group of tiers or components, not a single physical server or VM.

You take that idea a step further and understand that the provider/consumer relationships are truly just policy. Policy can describe many things, but here it would be focused on permit/deny, redirect, SLAs, QoS, logging and L4-7 service chaining for security and user experience.


Now you’ve designed a policy model that focuses on the application connectivity and any requirements for those connections, including L4-7 services. With this concept you can instantiate that policy in a reusable format so that policy definition can be repeated for like connections, such as users connecting to a web tier. Additionally the application connectivity definition as a whole could be instantiated as a template or profile for reuse.

You’ve now defined a logical model, based on policy, for how applications should be deployed. With this model in place you can work your way down. Next you’ll need network equipment that can support your new model. Before thinking about the hardware, remember there is an operating system (OS) that will have to interface with your policy model.

Traditional network operating systems are not designed for this type of object oriented policy model. Even highly programmable or Linux based operating systems have not been designed for object programmability that would fully support this model.  You’ll need an OS that’s capable of representing tiers or components of an application as objects, with configurable attributes. Additionally it must be bale to represent physical resources like ports as objects abstracted from the applications that will run on them.  An OS that can be provisioned in terms of policy constructs rather than configuration lines such as switch ports, QoS and ACLs. You’ll need to rewrite the OS.

As you’re writing your OS you’ll need to rethink the switching and routing hardware that will deliver all of those packets and frames. Of course you’ll need: density, bandwidth, low-latency, etc. More importantly you’ll need hardware that can define, interpret and enforce policy based on your new logical model. You’ll need to build hardware tailored to the way you define applications, connectivity and policy.  Hardware that can enforce policy based on logical groupings free of VLAN and subnet based policy instantiation.

If you build these out together, starting with the logical model then defining the OS and hardware to support it, you’ll have built a solution that surpasses the software shims of generation 1 SDN. You’ll have built a solution that focuses on removing the complexity first, then automating, then applying rapid deployment through tools usable by development and operations, better yet DevOps.

If you do that you’ll have truly defined networking based on software. You’ll have defined it from the top all the way down to the ASICs. If you do all that and get it right, you’ll have built Cisco’s Application Centric Infrastructure (ACI.)

For more information on the next generation of data center networking check out www.cisco.com/go/aci.


Disclaimer: ACI is where I’ve been focused for the last year or so, and where my paycheck comes from.  You can feel free to assume I’m biased and this article has no value due to that.  I won’t hate you for it.

GD Star Rating

Engineers Unplugged Episode 14: Application Affinity

I had the pleasure of speaking with Nils Swart (@nlnils) of Plexxi about applications and the network.  You can watch the quick Engineer’s Unplugged below.

GD Star Rating

What Network Virtualization Isn’t

Brad Hedlund recently posted an excellent blog on Network Virtualization.  Network Virtualization is the label used by Brad’s employer VMware/Nicira for their implementation of SDN.  Brad’s article does a great job of outlining the need for changes in networking in order to support current and evolving application deployment models.  He also correctly points out that networking has lagged behind the rest of the data center as technical and operational advancements have been made. 

Network configuration today is laughably archaic when compared to storage, compute and even facilities.  It is still the domain of CLI wizards hacking away on keyboards to configure individual devices.  VMware brought advancements like resource utilization based automatic workload migration to the compute environment.  In order to support this behavior on the network an admin must ensure the appropriate configuration is manually defined on each port that workload may access and every port connecting the two.  This is time consuming, costly and error prone. Brad is right, this is broken.

Brad also correctly points out that network speeds, feeds and packet delivery are adequately evolving and that the friction lies in configuration, policy and service delivery.  These essential network components are still far too static to keep pace with application deployments.  The network needs to evolve, and rapidly, in order to catch up with the rest of the data center.

Brad and I do not differ on the problem(s), or better stated: we do not differ on the drivers for change.  We do however differ on the solution.  Let me preface in advance that Brad and I both work for HW/SW vendors with differing solutions to the problem and differing visions of the future.  Feel free to write the rest of this off as mindless dribble or vendor Kool Aid, I ain’t gonna hate you for it.

Brad makes the case that Network Virtualization is equivalent to server virtualization, and from this simple assumption he poses it as the solution to current network problems.

Let’s start at the top: don’t be fooled by emphatic statements such as Brad’s stating that network virtualization is analogous to server virtualization.  It is not, this is an apples and oranges discussion.  Network virtualization being the orange where you must peel the rind to get to the fruit.  Don’t take my word for it, one of Brad’s colleagues, Scott Lowe, a man much smarter then I says it best:


The issue is that these two concepts are implemented in a very different fashion.  Where server virtualization provides full visibility and partitioning of the underlying hardware, network virtualization simply provides a packet encapsulation technique for frames on the wire.  The diagram below better illustrates our two fruits: apples and oranges.


As the diagram illustrates we are not working with equivalent approaches.  Network virtualization would require partitioning of switch CPU, TCAM, ASIC forwarding, bandwidth etc. to be a true apples-to-apples comparison.  Instead it provides a simple wrapper to encapsulate traffic on the underlying Layer 3 infrastructure.  These are two very different virtualization approaches.

Brad makes his next giant leap in the “What is the Network section.”  Here he makes the assumption that the network consists of only virtual workloads “The “network” we want to virtualize is the complete L2-L7 services viewed by the virtual machines” and the rest of his blog focuses there.  This is fine for those data center environments that are 100% virtualized including servers, services and WAN connectivity and use server virtualization for all of those purposes.  Those environments must also lack PaaS and SaaS systems that aren’t built on virtual servers as those are also non-applicable to the remaining discussion.  So anyone in those environments described will benefit from the discussion, anyone <crickets>.

So Brad and, presumably VMware/Nicira (since network virtualization is their term), define the goal as taking “all of the network services, features, and configuration necessary to provision the application’s virtual network (VLANs, VRFs, Firewall rules, Load Balancer pools & VIPs, IPAM, Routing, isolation, multi-tenancy, etc.) – take all of those features, decouple it from the physical network, and move it into a virtualization software layer for the express purpose of automation.”  So if your looking to build 100% virtualized server environments with no plans to advance up the stack into PaaS, etc. it seems you have found your Huckleberry.

What we really need is not a virtualized network overlay running on top of an L3 infrastructure with no communication or correlation between the two.  What we really need is something another guy much smarter than me (Greg Ferro) described:


Abstraction, independence and isolation, that’s the key to moving the network forward.  This is not provided by network virtualization.  Network virtualization is a coat of paint on the existing building.  Further more that coat of paint is applied without stripping, priming, or removing that floral wall paper your grandmother loved.  The diagram below is how I think of it.

Network Virtualization

With a network virtualization solution you’re placing your applications on a house of cards built on a non-isolated infrastructure of legacy design and thinking.  Without modifying the underlying infrastructure, network virtualization solutions are only as good as the original foundation.  Of course you could replace the data center network with a non-blocking fabric and apply QoS consistently across that underlying fabric (most likely manually) as Brad Hedlund suggests below.


If this is the route you take, to rebuild the foundation before applying network virtualization paint, is network virtualization still the color you want?  If a refresh and reconfigure is required anyway, is this the best method for doing so? 

The network has become complex and unmanageable due to things far older than VMware and server virtualization.  We’ve clung to device centric CLI configuration and the realm of keyboard wizards.  Furthermore we’ve bastardized originally abstracted network constructs such as VLAN, VRF, addressing, routing, and security tying them together and creating a Frankenstein of a data center network.  Are we surprised the villagers are coming with torches and pitch forks?

So overall I agree with Brad, the network needs to be fixed.  We just differ on the solution, I’d like to see more than a coat of paint.  Put lipstick on a pig and all you get is a pretty pig.

lipstick pig

GD Star Rating

Taking a Good Hard Look at SDN

SDN is sitting at the peak of it’s hype cycle (at least I hope it’s the peak.)  Every vendor has a definition and a plan.  Most of those definitions and plans focus around protecting their existing offerings and morphing those into some type of SDN vision.  Products and entire companies have changed their branding from whatever they were to SDN and the markets flooded with SDN solutions that solve very different problems.  This post will take a deep dive into the concepts around SDN and the considerations of a complete solution.  As always with my posts this is focused on the data center network, because I can barely spell WAN, have never spent time on a campus and have no idea what magic it is that service providers do.

The first question anyone considering SDN solutions needs to ask is: What problem(s) am I trying to solve.  Start with the business drivers for the decision.  There are many that SDN solutions look to solve, a few examples are:

  • Faster response to business demands for new tenants, services and applications.
  • More intelligent configuration of network services such as load balancers, firewalls etc.  The ability to dynamically map application tiers to required services.
  • Reductions in cost i.e. CapEx via enabling purchase of lower cost infrastructure and OpEx via reducing administrative overhead of device centric configuration.
  • Ability to create new revenue streams via more intelligent network service offerings.
  • Reduction in lock-in from proprietary systems.
  • Better network integration with cloud management systems and orchestration tools.
  • Better network efficiency through closer match of network resources to application demands.

That leaves a lot of areas with room for improvement in order to accomplish those tasks.  That’s one of the reasons the definition is so loose and applied to such disparate technologies.  In order to keep the definition generic enough to encompass a complete solution there are three major characteristics I prefer for defining an SDN architecture:

  • Flow Management – The ability to define flows across the network based on characteristics of the flow in a centralized fashion.
  • Dynamic Scalability – Providing a network that can scale beyond the capabilities of traditional tools and do so in a fluid fashion.
  • Programmability – The ability for the functionality provided by the network to be configured programmatically typically via APIs.

The Complete Picture:

In looking for a complete solution for Software Defined data center network it’s important to assess all aspects required to deliver cohesive network services and packet delivery:

  • Packet delivery – routing/switching as required.  Considerations such as requirements for bridging semantics (flooding, broadcast), bandwidth, multi-pathing etc.
  • L4-L7 service integration – The ability to map application tiers to required network services such as load-balancers and firewalls.
  • Virtual network integration – Virtual switching support for your chosen hypervisor(s).  This will be more complex in multi-hypervisor environments.
  • Physical network integration – Integration with bare-metal servers, standalone appliances, network storage and existing infrastructure.
  • Physical management – The management of the physical network nodes, required configuration of ports, VLANs, routes, etc.
  • Scalability – Ability to scale application or customer tenancy beyond the 4000 VLAN limit.
  • Flow management – The ability to program network policy from a global perspective.

Depending on your overall goals you may not have requirements in each of these areas but you’ll want to analyze that carefully based on growth expectations.  Don’t run your data center like congress kicking the can (problem) down the road.  The graphic below shows the various layers to be considered when looking at SDN solutions.


Current Options:

The current options for SDN typically provide solutions for one or more of these issues but not all.  The chart below takes a look at some popular options.



VLAN Scale


Bare Metal Support

Physical Network Node MGMT







Nicira/VMware X 3rd Party *   X * X   3rd Party X
Overlays X       X X X X    
OpenFlow     X   X X X X X X
Midokura X X     X   X   X  

X = Support

* = Future Support
This chart is not intended to be all encompassing or to compare all features of equal products (obviously an overlay doesn’t compete with a Nicira or Midokura solution, and each of those rely on overlays of some type.)  Instead it’s intended to show that the various solutions lumped into SDN provide solutions for different areas of the data center network.  One or more tools may be necessary to deploy a full SDN architecture and even then there may be gaps in areas like bare metal support, integration of standalone network appliances and provisioning/monitoring/troubleshooting of physical switch nodes (yes that all still matters.)
API Model:
Another model lumped into SDN is northbound APIs for network devices.  Several networking vendors are in various stages of support for this model.  This model does provide programmability but I would argue against it’s scale.  Using this model requires top down management systems that understand each device, its capabilities and its API.  To scale this type of management system and program network flows this way is not easy and will be error prone.  Additionally this model does not provide any additional functionality, visibility or holistic programmability, simply a better way to configure individual devices. That being said managing via APIs is light years ahead of screen scrapes and CLI scripting.
Hardware Matters:
Let me preface with what I’m not saying: I’m not saying that hardware will/won’t be commoditized, and I’m not saying that custom silicon or merchant silicon is better or worse.
I am saying that the network hardware you choose will matter.  Table sizes, buffer space, TCAM size will all factor in, and depending on your deployment model will be a major factor.  The hardware will also need to provide maximum available bandwidth and efficient ECMP load-balancing for network throughput.  This load-balancing can be greatly affected by the overlay method chosen based on available header information for hashing algorithms.  Additionally your hardware must support the options of the SDN model you choose.  For example in a Nicira/VMware deployment you’ll have future support for management of switches running OVS, you may want these to tie in physical servers, etc.  The same would apply if you choose OpenFlow.  You’ll need switch hardware that provides OpenFlow support, additionally it will need to support your deployment model hybrid or pure OpenFlow.
The hardware also matters in configuration, management, and troubleshooting.  While there is a lot of talk of “We just need any IP connectivity” that IP network still has to be configured and managed.  Layer 2/3 constructs must be put in place, ports must be configured.  This hardware will also have to be monitored, and troubleshot when things fail.  This will be more difficult in cases where the overlay is unknown to the L3 infrastructure at which point two separate independent networks will be involved: physical and logical.
Management Model:

There are several management models to choose from and two examples in the choices I compared above.  OpenFlow uses a centralized top down approach with the controller pushing flows to all network elements and handling policy for new flows forwarded from those devices.  The Nicira/VMware solution uses the same model as OpenFlow.  Midokura on the other hand takes a play from distributed systems and pushes intelligence to the edges in that fashion.  Each model offers various pros/cons and will play a major role in the scale and resiliency of your SDN deployment.

Northbound API:

The Northbound API is different than the device APIs mentioned below.  This API opens the management of your SDN solution as whole up to higher level systems.  Chances are you’re planning to plug your infrastructure into an automation/orchestration solution or cloud platform.  In order to do this you’ll want a robust northbound API for your infrastructure components, in this case your SDN architecture.  If you have these systems in place, or have already picked your horse you’ll want to ensure compatibility with the SDN architectures you consider.  Not all APIs are created equal, and they are far from standardized so you’ll want to know exactly what you’re getting from a functionality perspective and ensure the claims match your upper layer systems needs.

Additional Considerations:

There are several other considerations which will effect both the options chosen and the architecture used some of those:

  • How are flows distributed?
  • How are unknown flows handled?
  • How are new end points discovered?
  • How are required behaviors of bridging handled?
  • How are bad behaviors of bridging minimized (BUM traffic)?
  • What happens during controller failure scenarios?
  • What is the max theoretical/practical scalability?
    • Does that scale apply globally, i.e. physical and virtual switches etc.?
  • What new security concerns (if any) may be introduced?
  • What are the requirements of the IP network (multicast, etc.)
  • How is multi-tenancy handled?
  • What is the feature disparity between virtualized and physical implementation?
  • How does it integrate with existing systems/services?
  • How is traffic load balanced?
  • How is QoS provided?
  • How are software/firmware upgrades handled?
  • What is the disparity between the software implementation and the hardware capabilities, for example OpenFlow on physical switches?
  • Etc.


SDN should be putting the application back in focus and providing tools for more robust and rapid application deployment/change.  In order to effectively do this an SDN architecture should provide functionality for the full life of the packet on the data center network.  The architecture should also provide tools for the scale you forecast as you grow.  Because of the nature of the ecosystem you may find more robust deployment options the more standardized your environment is (I’ve written about standardization several times in the past for example:http://www.networkcomputing.com/private-cloud-tech-center/private-cloud-success-factor-standardiza/231500532 .)  You can see examples of this in the hypervisor support shown in the chart above.

While solutions exist for specific business use cases the market is far from mature.  Products will evolve and as lessons are learned and roadmaps executed we’ll see more robust solutions emerge.  In the interim choose technologies that meet your specific business drivers and deploy them in environments with the largest chance of success, low hanging fruit.  It’s prudent to move into network virtualization in the same fashion you moved into server virtualization, with a staged approach.

GD Star Rating

The App on the Crap (An SDN Story)

I’m feeling Seussish again and looking to tackle SDN this time.  If you missed my first go it was on Hadoop: Horton Hears Hadoop.  Here’s another run:


The app could not flow

Net was too slow to change.

It sat on the server

Waiting on admin for change.


It sat there quite idly

Customers did too

The dev thought, “How I wish

They’d let my app through!”


Too slow to adapt

Too rigid and strict.

The business can’t move.

And that’s my verdict.


So all they could do was to





The dev did not like it.

Not one little bit.


And then

Someone spoke UP!

How that speech gave us PUMP!


We listened!

And we heard it move into the hype!

We listened!

A network of SDN type!

The message quite clear,

“You’ve got no need to gripe.”


“I know it is slow

and the network is messy.

There is a fix

With software that’s dressy!”


“I know some good tricks we can use,”

SDN gal said.

“A header or two,”

Said the gal with the plan.

“Controllers as well.

I will show them to you.

Your CTO

Will not mind if I do.”


Then app and dev

Did not know what to say.

The CTO was out playing golf

For the day.


But the net admin said, “No!

Make that gal go away!

"Tell the SDN gal

You do NOT want to play.

She should not be here.

She should not be about.

She should not be here

When the CTO is out!”


“Now! Now! Have no fear.

Have no fear!” Said the gal.

“My tricks are not bad,”

Said the SDN gal.

“Why you’ll have

So many options from me,

With some tricks that I call

Virtualization you see!”


“Stop this nonsense!” admin said.

“We don’t need to scale!

Stop this nonsense!” Admin said.

“The net cannot fail!”


“Have no fear!” said the gal.

“I will not let net fail.

I will make it dynamic

And people will hail.

Its changes are quick!

It grows very fast!

But there is much more it can do!”


“Look at it!

Look at it now said the gal.”

“With a new overlay

And control from a pal!

It can adapt very fast!

It’s managed quite nicely!

The scale is much greater!

And admin less dicey!

And look!

You can change flows from here!

But there is more dear!

Oh, no.

There is more dear…


“Look at it!

Look at it!

Look at it now!

It’s better you see

But you have to know how.

How it can adapt

And respond to new apps!

How it grows to scale!

And helps those dev chaps!

Can grow past those VLANs

And direct traffic, see!

We wrap Layer two

In Layer three IP!

And we route the IP!

As we grow big from small!

But that is not all.

Oh, no.

That is not all….”


That’s what the gal said…

Then the net went dead!

The apps all went down

From out at the NOC.

The developers,

Watched with eyes open in shock!


And the admin cried out.

With a loud angry shot!

He said, “Do I like this?

Oh no! I do not.

This is not a good trick,”

Said the admin with grit.

“no I don’t like it,

Not one little bit!”


“Now look what you did!”

Said admin to gal.

“Now look at this net!

Look at this mess now pal!

You brought down the apps,

Crashed services too

You cost us some sales

And caused lost revenue.

You SHOULD NOT be here

When the CTOs not.

Get out of the data center!”

Admin said from his spot.


“But I like to be here.

Oh, I like it a lot”

Said the SDN girl

To the admin she shot.

“I will not go away.

I do not wish to go!

And so,” said the SDN girl,




I will show you

Another good trick that I know!”


And then she ran out.

And, then fast as a fox,

The SDN gal

Came back with a box.


A big green wood box.

It was shut with a hook.

“Now look at this trick,”

Said the gal.

“Take a look!”


Then she got up on top

And with no rationale.

“I call this game SDN-IN-A-BOX,”

Said the gal.

“In this box are four things

I will show to you now.

You will like these four things.”

Said the gal with a bow.


“I will pick up the hook.

You will see something new.

Four things. And I call them

The SDN glue.

These things will not harm you.

They want to move frames.”

Then, out of the box

Came her SDN claims!

And they came out quite fast.

They said, “Are you ready?

Now should we get started

Let’s get going already!”


The devs and the apps

Did not know what to do.

So they sat and they watched

Watched the SDN glue.

They stood in their shock

But the admin said “No!

Those things should not be

On this net! Make them go!”


“They should not be here

When the CTOs not!

Put them out! Put them out!”

Admin yelled with a shot.


“Have no fear, Mr. admin,”

Said the SDN gal.

“These things are good things

And good for morale.”

“They’re great.  Oh so great!

They have come to fix things.

They will give back control

To the network today.”


“The first is an overlay,

Number two a vSwitch

But that’s only halfway.”

Was the gals latest pitch.


“We’ll next need control

For the flows as they go.

Something to manage

Those flows as they flow.

But there’s still one more piece

Of this SDN madness.

Device management system

To avoid admin sadness.”


Then the SDN gal

Said with conviction

“We aren’t quite done yet

There’s one more restriction.

We must tie these together

In a cohesive fashion,

If we do not

It’s all stormy weather.

We will organize things

With apps at the center

And let those developers

For once spread their wings.”


“You see in the past,”

Said the SDN gal.

“The net was restrictive

the apps were in hell.

Now we change things around

Put the apps back in focus.

Using these tricks,

And some good hocus pocus.

With a sprinkle of tears

From the unicorn clan,

And a dash of fine dust

A pixie put in this can.

We’ll accomplish the task.”

SDN gal said as she drank from her flask.


And lo and behold,

The network sprang back.

The packets were flowing,

TCP sent it’s ACK.

The admin stood shocked,

As he used the controller.

With this type of thing,

He would be the high roller!

He gaped in amazement

At the tenancy scale.

No longer 4000,

It was net holy grail.


The apps back online,

As CTO entered.

A disaster avoided, he was left with no sign.

Of the mess that had happened,

While he was out and about.

But the faint sound of snoring

SDN girl drunk and passed out.

GD Star Rating

Network Overlays: An Introduction

While network overlays are not a new concept, they have come back into the limelight, thanks to drivers brought on by large-scale virtualization. Several standards have been proposed to enable virtual networks to be layered over a physical network infrastructure: VXLAN, NVGRE, and SST. While each proposed standard uses different encapsulation techniques to solve current network limitations, they share some similarities. Let’s look at how network overlays work in general…

To see the full article visit: http://www.networkcomputing.com/next-gen-network-tech-center/network-overlays-an-introduction/240144228

GD Star Rating

Data Center Overlays 101

I’ve been playing around with Show Me (www.showme.com) as a tool to add some white boarding to the blog.  Here’s my first crack at it covering Data Center Network overlays.

GD Star Rating


The most viable competitor to VXLAN is NVGRE which was proposed by Microsoft, Intel, HP and Dell.  It is another encapsulation technique intended to allow virtual network overlays across the physical network.  Both techniques also remove the scalability issues with VLANs which are bound at a max of 4096.  NVGRE uses Generic Routing Encapsulation (GRE) as the encapsulation method.  It uses the lower 24 bits of the GRE header to represent the Tenant Network Identifier (TNI.)  Like VXLAN this 24 bit space allows for 16 million virtual networks. 


While NVGRE provides optional support for broadcast via IP multi-cast, it does not rely on it for address learning as VXLAN does.  It instead leaves that up to an as of yet undefined control plane protocol.  This control plane protocol will handle the mappings between the “provider” address used in the outer header to designate the remote NVGRE end-point and the “customer” address of the destination.  The lack of reliance of flood and learn behavior replicated over IP multicast potentially makes NVGRE a more scalable solution.  This will be dependent on implementation and underlying hardware.

Another difference between VXLAN and NVGRE will be within its multi-pathing capabilities.  In its current format NVGRE will provides little ability to be properly load-balanced by ECMP.  In order to enhance load-balancing the draft suggests the use of multiple IP addresses per NVGRE host, which will allow for more flows.  This is a common issue with tunneling mechanisms and is solved in VXLAN by using a hash of the inner frame as the UDP source port.  This provides for efficient load balancing by devices capable of 5-tuple balancing decisions.  There are other possible solutions proposed for NVGRE load-balancing, we’ll have to wait and see how they pan out. 

The last major difference between the two protocols is the use of jumbo frames.  VXLAN is intended to stay within a data center where jumbo frame support is nearly ubiquitous, therefore it assumes that support is present and utilizes it.  NVGRE is intended to be able to be used inter-data-enter and therefore allows for provisions to avoid fragmentation.


While NVGRE still needs much clarification it is backed by some of the biggest companies in IT and has some potential benefits.  With the VXLAN capable hardware world expanding quickly you can expect to see more support for NVGRE.  Layer 3 encapsulation techniques as a whole solve the issues of scalability inherent with bridging.  Additionally due to their routed nature they also provide for loop free multi-pathed environments without the need for techniques such as TRILL and technologies based on it.  In order to reach the scale and performance required by tomorrows data centers our networks need change, overlays such as these are one tool towards that goal.

GD Star Rating

Stateless Transport Tunneling (STT)

STT is another tunneling protocol along the lines of the VXLAN and NVGRE proposals.  As with both of those the intent of STT is to provide a network overlay, or virtual network running on top of a physical network.  STT was proposed by Nicira and is therefore not surprisingly written from a software centric view rather than other proposals written from a network centric view.  The main advantage of the STT proposal is it’s ability to be implemented in a software switch while still benefitting from NIC hardware acceleration.  The other advantage of STT is its use of a 64 bit network ID rather than the 32 bit IDs used by NVGRE and VXLAN.

The hardware offload STT grants relieves the server CPU of a significant workload in high bandwidth systems (10G+.)  This separates it from it’s peers that use an IP encapsulation in the soft switch which negate the NIC’s LSO and LRO functions.   The way STT goes about this is by having the software switch inserts header information into the packet to make it look like a TCP packet, as well as the required network virtualization features.  This allows the guest OS to send frames up to 64k to the hypervisor which are encapsulated and sent to the NIC for segmentation.  While this does allow for the HW offload to be utilized it causes several network issues due to it’s use of valid TCP headers it causes issues for many network appliances or “middle boxes.” 

STT is not expected to be ratified and is considered by some to have been proposed for informational purposes, rather than with the end goal of a ratified standard.  With its misuse of a valid TCP header it would be hard pressed for ratification.  STT does bring up the interesting issue of hardware offload.  The IP tunneling protocols mentioned above create extra overhead on host CPUs due to their inability to benefit from NIC acceleration techniques.  VXLAN and NVGRE are intended to be implemented in hardware to solve this problem.  Both VXLAN and NVGRE use a 32 bit network ID because they are intended to be implemented in hardware, this space provides for 16 million tenants.  Hardware implementation is coming quickly in the case of VXLAN with vendors announcing VXLAN capable switches and NICs. 

GD Star Rating