Engineers Unplugged Episode 14: Application Affinity

I had the pleasure of speaking with Nils Swart (@nlnils) of Plexxi about applications and the network.  You can watch the quick Engineer’s Unplugged below.

GD Star Rating

Software Defined networking: The Role of SDN on Compute Infrastructure Administration

#vBrownBag Follow-up Software Defined Networking SDN with Joe Onisick (@jonisick) from ProfessionalVMware on Vimeo.

GD Star Rating

Focus on the Ball: The Application

With the industry talking about Software Defined Networking (SDN) at full hype levels, there is one thing missing from many discussions: the application. SDN promises to reign in the complexity of network infrastructure and provide better tools for deploying services at scale. What often seems to be forgotten are the applications, which are the reason those networks exist. While application focus in itself is not a new concept it seems lost in the noise around SDN as a whole, with a few exceptions such as Plexxi being which focuses on Application Affinity.

Current SDN approaches provide tools to solve issues in one portion or the other of network infrastructure. Flow control mechanisms look to centralize the distribution and configuration of routing and forwarding. Overlays look to build virtual networks on existing IP infrastructure. Virtualized L4-7 services provide solutions to configure, stitch-in and control network services more closely to virtual machines themselves. None of these current approaches looks to tackle the whole picture from an application centric point of view. These solutions also take a myopic view that the VM is the network, this is far from the case.  The closest models fall into dev-ops categories or orchestration but these require a deep understanding of the details and intricacies of the network.

In traditional networking environments there is a disconnect in communication between application and network teams. The languages and concepts are disparate enough that they don’t translate, there is no logical continuation from application developer or owner to network designer. Application teams speak in OS instances, application tiers and components, tooling, language, end-user demands, etc. while network teams speak in switch-ports, VLANs, QoS, IP addressing and Access Control Lists (ACLs). The lack of common understanding and vocabulary causes architectures and implementations to suffer. The graphic below illustrates this relationship:


Building the flexible, scalable, manageable and programmable networks of the future requires a change in focus. The application needs to take center stage; it’s the apps that solve business problems. From this focus, logical and physical topology become secondary and are only designed once application requirements have been mapped out. Application centric policies must be designed first. Policies such as: security, load-balancing, QoS can all be designed based on application requirements, rather than network restrictions. Application developers define these requirements without the need to speak a network language.

Traditional networks begin with a physical topology that is layered with L2 and L3 logical topologies and assumed application mobility and service domains such as a services tier in the aggregation level. Once these topologies are architected and implemented applications are built and deployed on them. This method limits the capabilities available to the application and the services deployed on them.

Application security is an excellent example of a system that suffers from traditional architectures. Network security constructs are implemented in the form of ACLs on switches, routers and firewalls. These entries suffer from two major drawbacks: complexity of design/implementation and scale of the TCAM that stores the entries. This means that application policies must be communicated effectively to network engineers who must translate those requirements into implementable ACLs across multiple devices in the network. This is then defined manually device-by-device. This is a system ripe for PEBKAC errors (Problem Exists Between Keyboard and Chair.)

The complexity and room for error in this system increases exponentially as networks scale, applications move and new services are needed. Additionally this leads to bad practice based on design limitations. Far too often outdated policy entries are left in place due to the complexity and risk of removing entries. This leads to residual entries in place consuming space long after an application is gone. Just as often policies are written more loosely than would be optimal in order to reduce required entries, and optimize space, through wild card summarization.

To break this cycle networking systems need to take an application centric approach which models actual application requirements onto the network in a top down fashion. Systems need to take into account the structure of the application, its components, and how those components interact then provide tools for designing logical policy maps of these relationships. From there these policy maps can be programmatically applied to the networking infrastructure.

An application is not a single software instance running on a server. Applications are made up of the end-points required in a given tier, the tiers required for the service delivered and the policies that define how those tiers communicate, and their unique requirements. The application as a whole must be taken into account in order to provide robust, scalable service delivery.

The illustration below shows this relationship in contrast to the diagram above:


In this model network and application teams develop the systems of policies that define application behavior and push them to the network. Taking the application as a whole into focus instead of the myopic view of VMs, switch ports or IP addresses allows cohesive deployment and manageability at scale. The application is the purpose of having a network; therefore the application should define the network.

This definition of the network by the application should be done in a language that the developers understand, and the network can interpret and implement. For example an app owner labels application traffic as ‘video’ and the network implements policies for bandwidth, QoS, etc. that video requires. These policies are predefined by the network engineers.

An application is more than an IP address and a set of rules; it is an ecosystem of interconnected devices and the policies that define their relationship. Traditional networking techniques anchor application deployment by defining applications in networking terms. In order to accelerate the application deployment (and re-deployment throughout its lifecycle) networks need to provide an application centric view and deployment model.

GD Star Rating

Network Management Needs New Ideas

As networks have grown, the industry has sought better ways in which to manage them at scale. Traditional network management systems are typically device-centric, particularly for network infrastructure. These systems take a top-down management approach and use a central server to push configuration into devices and to manage device state. With few exceptions, this approach provides no additional abstraction or functionally and fundamentally becomes a GUI representation of CLI configuration…

To see the full post visit:

GD Star Rating

What Network Virtualization Isn’t

Brad Hedlund recently posted an excellent blog on Network Virtualization.  Network Virtualization is the label used by Brad’s employer VMware/Nicira for their implementation of SDN.  Brad’s article does a great job of outlining the need for changes in networking in order to support current and evolving application deployment models.  He also correctly points out that networking has lagged behind the rest of the data center as technical and operational advancements have been made. 

Network configuration today is laughably archaic when compared to storage, compute and even facilities.  It is still the domain of CLI wizards hacking away on keyboards to configure individual devices.  VMware brought advancements like resource utilization based automatic workload migration to the compute environment.  In order to support this behavior on the network an admin must ensure the appropriate configuration is manually defined on each port that workload may access and every port connecting the two.  This is time consuming, costly and error prone. Brad is right, this is broken.

Brad also correctly points out that network speeds, feeds and packet delivery are adequately evolving and that the friction lies in configuration, policy and service delivery.  These essential network components are still far too static to keep pace with application deployments.  The network needs to evolve, and rapidly, in order to catch up with the rest of the data center.

Brad and I do not differ on the problem(s), or better stated: we do not differ on the drivers for change.  We do however differ on the solution.  Let me preface in advance that Brad and I both work for HW/SW vendors with differing solutions to the problem and differing visions of the future.  Feel free to write the rest of this off as mindless dribble or vendor Kool Aid, I ain’t gonna hate you for it.

Brad makes the case that Network Virtualization is equivalent to server virtualization, and from this simple assumption he poses it as the solution to current network problems.

Let’s start at the top: don’t be fooled by emphatic statements such as Brad’s stating that network virtualization is analogous to server virtualization.  It is not, this is an apples and oranges discussion.  Network virtualization being the orange where you must peel the rind to get to the fruit.  Don’t take my word for it, one of Brad’s colleagues, Scott Lowe, a man much smarter then I says it best:


The issue is that these two concepts are implemented in a very different fashion.  Where server virtualization provides full visibility and partitioning of the underlying hardware, network virtualization simply provides a packet encapsulation technique for frames on the wire.  The diagram below better illustrates our two fruits: apples and oranges.


As the diagram illustrates we are not working with equivalent approaches.  Network virtualization would require partitioning of switch CPU, TCAM, ASIC forwarding, bandwidth etc. to be a true apples-to-apples comparison.  Instead it provides a simple wrapper to encapsulate traffic on the underlying Layer 3 infrastructure.  These are two very different virtualization approaches.

Brad makes his next giant leap in the “What is the Network section.”  Here he makes the assumption that the network consists of only virtual workloads “The “network” we want to virtualize is the complete L2-L7 services viewed by the virtual machines” and the rest of his blog focuses there.  This is fine for those data center environments that are 100% virtualized including servers, services and WAN connectivity and use server virtualization for all of those purposes.  Those environments must also lack PaaS and SaaS systems that aren’t built on virtual servers as those are also non-applicable to the remaining discussion.  So anyone in those environments described will benefit from the discussion, anyone <crickets>.

So Brad and, presumably VMware/Nicira (since network virtualization is their term), define the goal as taking “all of the network services, features, and configuration necessary to provision the application’s virtual network (VLANs, VRFs, Firewall rules, Load Balancer pools & VIPs, IPAM, Routing, isolation, multi-tenancy, etc.) – take all of those features, decouple it from the physical network, and move it into a virtualization software layer for the express purpose of automation.”  So if your looking to build 100% virtualized server environments with no plans to advance up the stack into PaaS, etc. it seems you have found your Huckleberry.

What we really need is not a virtualized network overlay running on top of an L3 infrastructure with no communication or correlation between the two.  What we really need is something another guy much smarter than me (Greg Ferro) described:


Abstraction, independence and isolation, that’s the key to moving the network forward.  This is not provided by network virtualization.  Network virtualization is a coat of paint on the existing building.  Further more that coat of paint is applied without stripping, priming, or removing that floral wall paper your grandmother loved.  The diagram below is how I think of it.

Network Virtualization

With a network virtualization solution you’re placing your applications on a house of cards built on a non-isolated infrastructure of legacy design and thinking.  Without modifying the underlying infrastructure, network virtualization solutions are only as good as the original foundation.  Of course you could replace the data center network with a non-blocking fabric and apply QoS consistently across that underlying fabric (most likely manually) as Brad Hedlund suggests below.


If this is the route you take, to rebuild the foundation before applying network virtualization paint, is network virtualization still the color you want?  If a refresh and reconfigure is required anyway, is this the best method for doing so? 

The network has become complex and unmanageable due to things far older than VMware and server virtualization.  We’ve clung to device centric CLI configuration and the realm of keyboard wizards.  Furthermore we’ve bastardized originally abstracted network constructs such as VLAN, VRF, addressing, routing, and security tying them together and creating a Frankenstein of a data center network.  Are we surprised the villagers are coming with torches and pitch forks?

So overall I agree with Brad, the network needs to be fixed.  We just differ on the solution, I’d like to see more than a coat of paint.  Put lipstick on a pig and all you get is a pretty pig.

lipstick pig

GD Star Rating

CloudStack Graduates to Top-Level Apache Project

The Apache Software Foundation announced in late March that CloudStack is now a top-level project. This is a promotion from CloudStack’s incubator status, where it had lived after being released as open source by Citrix.

This promotion provides additional encouragement to companies and developers looking to contribute to the project, because it validates the CloudStack community and demonstrates ongoing support under the Apache Software Foundation. To read more visit the full article.

GD Star Rating

OpenStack Video Cage Match With Colin McNamara

This post is a little late, mainly because I’m both lazy and distracted.  That being said I hope you’ll enjoy this video of Colin McNamara (@colinmcnamara) and I debating the merits of OpenStack.  For more Engineer’s unplugged goodness from Amy Lewis (@commsninja) visit:

GD Star Rating

The App on the Crap (An SDN Story)

I’m feeling Seussish again and looking to tackle SDN this time.  If you missed my first go it was on Hadoop: Horton Hears Hadoop.  Here’s another run:


The app could not flow

Net was too slow to change.

It sat on the server

Waiting on admin for change.


It sat there quite idly

Customers did too

The dev thought, “How I wish

They’d let my app through!”


Too slow to adapt

Too rigid and strict.

The business can’t move.

And that’s my verdict.


So all they could do was to





The dev did not like it.

Not one little bit.


And then

Someone spoke UP!

How that speech gave us PUMP!


We listened!

And we heard it move into the hype!

We listened!

A network of SDN type!

The message quite clear,

“You’ve got no need to gripe.”


“I know it is slow

and the network is messy.

There is a fix

With software that’s dressy!”


“I know some good tricks we can use,”

SDN gal said.

“A header or two,”

Said the gal with the plan.

“Controllers as well.

I will show them to you.

Your CTO

Will not mind if I do.”


Then app and dev

Did not know what to say.

The CTO was out playing golf

For the day.


But the net admin said, “No!

Make that gal go away!

"Tell the SDN gal

You do NOT want to play.

She should not be here.

She should not be about.

She should not be here

When the CTO is out!”


“Now! Now! Have no fear.

Have no fear!” Said the gal.

“My tricks are not bad,”

Said the SDN gal.

“Why you’ll have

So many options from me,

With some tricks that I call

Virtualization you see!”


“Stop this nonsense!” admin said.

“We don’t need to scale!

Stop this nonsense!” Admin said.

“The net cannot fail!”


“Have no fear!” said the gal.

“I will not let net fail.

I will make it dynamic

And people will hail.

Its changes are quick!

It grows very fast!

But there is much more it can do!”


“Look at it!

Look at it now said the gal.”

“With a new overlay

And control from a pal!

It can adapt very fast!

It’s managed quite nicely!

The scale is much greater!

And admin less dicey!

And look!

You can change flows from here!

But there is more dear!

Oh, no.

There is more dear…


“Look at it!

Look at it!

Look at it now!

It’s better you see

But you have to know how.

How it can adapt

And respond to new apps!

How it grows to scale!

And helps those dev chaps!

Can grow past those VLANs

And direct traffic, see!

We wrap Layer two

In Layer three IP!

And we route the IP!

As we grow big from small!

But that is not all.

Oh, no.

That is not all….”


That’s what the gal said…

Then the net went dead!

The apps all went down

From out at the NOC.

The developers,

Watched with eyes open in shock!


And the admin cried out.

With a loud angry shot!

He said, “Do I like this?

Oh no! I do not.

This is not a good trick,”

Said the admin with grit.

“no I don’t like it,

Not one little bit!”


“Now look what you did!”

Said admin to gal.

“Now look at this net!

Look at this mess now pal!

You brought down the apps,

Crashed services too

You cost us some sales

And caused lost revenue.

You SHOULD NOT be here

When the CTOs not.

Get out of the data center!”

Admin said from his spot.


“But I like to be here.

Oh, I like it a lot”

Said the SDN girl

To the admin she shot.

“I will not go away.

I do not wish to go!

And so,” said the SDN girl,




I will show you

Another good trick that I know!”


And then she ran out.

And, then fast as a fox,

The SDN gal

Came back with a box.


A big green wood box.

It was shut with a hook.

“Now look at this trick,”

Said the gal.

“Take a look!”


Then she got up on top

And with no rationale.

“I call this game SDN-IN-A-BOX,”

Said the gal.

“In this box are four things

I will show to you now.

You will like these four things.”

Said the gal with a bow.


“I will pick up the hook.

You will see something new.

Four things. And I call them

The SDN glue.

These things will not harm you.

They want to move frames.”

Then, out of the box

Came her SDN claims!

And they came out quite fast.

They said, “Are you ready?

Now should we get started

Let’s get going already!”


The devs and the apps

Did not know what to do.

So they sat and they watched

Watched the SDN glue.

They stood in their shock

But the admin said “No!

Those things should not be

On this net! Make them go!”


“They should not be here

When the CTOs not!

Put them out! Put them out!”

Admin yelled with a shot.


“Have no fear, Mr. admin,”

Said the SDN gal.

“These things are good things

And good for morale.”

“They’re great.  Oh so great!

They have come to fix things.

They will give back control

To the network today.”


“The first is an overlay,

Number two a vSwitch

But that’s only halfway.”

Was the gals latest pitch.


“We’ll next need control

For the flows as they go.

Something to manage

Those flows as they flow.

But there’s still one more piece

Of this SDN madness.

Device management system

To avoid admin sadness.”


Then the SDN gal

Said with conviction

“We aren’t quite done yet

There’s one more restriction.

We must tie these together

In a cohesive fashion,

If we do not

It’s all stormy weather.

We will organize things

With apps at the center

And let those developers

For once spread their wings.”


“You see in the past,”

Said the SDN gal.

“The net was restrictive

the apps were in hell.

Now we change things around

Put the apps back in focus.

Using these tricks,

And some good hocus pocus.

With a sprinkle of tears

From the unicorn clan,

And a dash of fine dust

A pixie put in this can.

We’ll accomplish the task.”

SDN gal said as she drank from her flask.


And lo and behold,

The network sprang back.

The packets were flowing,

TCP sent it’s ACK.

The admin stood shocked,

As he used the controller.

With this type of thing,

He would be the high roller!

He gaped in amazement

At the tenancy scale.

No longer 4000,

It was net holy grail.


The apps back online,

As CTO entered.

A disaster avoided, he was left with no sign.

Of the mess that had happened,

While he was out and about.

But the faint sound of snoring

SDN girl drunk and passed out.

GD Star Rating

Network Overlays: An Introduction

While network overlays are not a new concept, they have come back into the limelight, thanks to drivers brought on by large-scale virtualization. Several standards have been proposed to enable virtual networks to be layered over a physical network infrastructure: VXLAN, NVGRE, and SST. While each proposed standard uses different encapsulation techniques to solve current network limitations, they share some similarities. Let’s look at how network overlays work in general…

To see the full article visit:

GD Star Rating

Why We Need Network Abstraction

The move to highly virtualized data centers and cloud models is straining the network. While traditional data center networks were not designed to support the dynamic nature of today’s workloads, the fact is, the emergence of highly virtualized environments is merely exposing issues that have always existed within network constructs. VLANs, VRFs, subnets, routing, security, etc. have been stretched well beyond their original intent. The way these constructs are currently used limits scale, application expansion, contraction and mobility.  To read the full article visit:

GD Star Rating