For the last several months there has been a lot of chatter in the blogosphere and Twitter about FCoE and whether full scale deployment requires QCN. There are two camps on this:
Typically the camps break down as follows (there are exceptions) :
For the purpose of this post I’m not camping with either side, I’m not even breaking out my tent. What I’d like to do is discuss when and where QCN matters, what it provides and why. The intent being that customers, architects, engineers etc. can decide for themselves when and where they may need QCN.
QCN: QCN is a form of end-to-end congestion management defined in IEEE 802.1.Qau. The purpose of end-to-end congestion management is to ensure that congestion is controlled from the sending device to the receiving device in a dynamic fashion that can deal with changing bottlenecks. The most common end-to-end congestion management tool is TCP Windows sizing.
TCP Window Sizing:
With window sizing TCP dynamically determines the number of frames to send at once without an acknowledgement. It continuously ramps this number up dynamically if the pipe is empty and acknowledgements are being received. If a packet is dropped due to congestion and an acknowledgement is not received TCP halves the window size and starts the process over. This provides a mechanism in which the maximum available throughput can be achieved dynamically.
Below is a diagram showing the dynamic window size (total packets sent prior to acknowledgement) over the course of several round trips. You can see the initial fast ramp up followed by a gradual increase until a packet is lost, from there the window is reduced and the slow ramp begins again.
If you prefer analogies I always refer to TCP sliding windows as a Keg Stand (http://en.wikipedia.org/wiki/Keg_stand.)
In the photo we see several gentleman surrounding a keg, with one upside down performing a keg stand.
To perform a keg stand:
What the hell does this have to do with TCP Flow Control? I’m so glad you asked.
During a keg stand your friend is trying to push as much beer down your throat as it can handle, much like TCP increasing the window size to fill the end-to-end pipe. Both of your hands are occupied holding your own weight, and your mouth has a beer hose in it, so like TCP you have no native congestion signaling mechanism. Just like TCP the flow doesn’t slow until packets/beer drops, when you start to spill they stop the flow.
So that’s an example of end-to-end congestion management. Within Ethernet and FCoE specifically we don’t have any native end-to-end congestion tools (remember TCP is up on L4 and we’re hanging out with the cool kids at L2.) No problem though because We’re talking FCoE right? FCoE is just a L1-L2 replacement for Fibre Channel (FC) L0-L1, so we’ll just use FC end-to-end congestion management… Not so fast boys and girls, FC does not have a standard for end-to-end congestion management, that’s right our beautiful over engineered lossless FC has no mechanism for handling network wide, end-to-end congestion. That’s because it doesn’t need it.
FC is moving SCSI data, and SCSI is sensitive to dropped frames, latency is important but lossless delivery is more important. To ensure a frame is never dropped FC uses a hop-by-hop flow control known as buffer-to-buffer (B2B) credits. At a high level each FC device knows the amount of buffer spaces available on the next hop device based on the agreed upon frame size (typically 2148 bytes.) This means that a device will never send a frame to a next hop device that cannot handle the frame. Let’s go back to the world of analogy.
Buffer-to-buffer credits:
The B2B credit system works in the same method you’d have 10 Marines offload and stack a truckload of boxes (‘fork-lift, we don’t need no stinking forklift.’) The best system to utilize 10 Marines to offload boxes is to line them up end-to-end one in the truck and one on the other end to stack. Marine 1 in the truck initiates the send by grabbing a box and passing it to Marine 2, the box moves down the line until it gets to the target Marine 10 who stacks it. Before any Marine hands another Marine a box they look to ensure that Marines hands are empty verifying they can handle the box and it won’t be dropped. Boxes move down the line until they are all offloaded and stacked. If anyone slows down or gets backed up each marine will hold their box until the congestion is relieved.
In this analogy the Marine in the truck is the initiator/server and the Marine doing the stacking is the target/storage with each Marine in between being a switch.
When two FC devices initiate a link they follow the Link-Initialization-Protocols (LIP.) During this process they agree on an FC frame size and exchange the available dedicated frame buffer spaces for the link. A sender is always keeping track of available buffers on the receiving side of the link. The only real difference between this and my analogy is each device (Marine) is typically able to handle more than one frame (box) at once.
So if FC networks operate without end-to-end congestion management just fine why do we need to implement a new mechanism in FCoE, well there-in lies the rub. Do we need QCN? The answer is really Yes and No, and it will depend on design. FCoE today provides the exact same flow control as FC using two standards defined within Data Center Bridging (DCB) these are Enhanced Transmission Selection (ETS) and Priority-Flow Control (PFC) for more info on theses see my DCB blog: http://www.definethecloud.net/?p=31.) Basically ETS provides a bandwidth guarantee without limiting and PFC provides lossless delivery on an Ethernet network.
Why QCN:
The reason QCN was developed is the differences between the size, scale, and design of FC and Ethernet networks. Ethernet networks are usually large mesh or partial mesh type designs with multiple switches. FC designs fall into one of three major categories Collapsed core (single layer), Core edge (two layer) or in rare cases for very large networks edge-core-edge (three layer.) This is because we typically have far fewer FC connected devices than we do Ethernet (not every device needs consolidated storage/backup access.)
If we were to design our FCoE networks where every current Ethernet device supported FCoE and FCoE frames flowed end-to-end QCN would be a benefit to ensure point congestion didn’t clog the entire network. On the other hand if we maintain similar size and design for FCoE networks as we do FC networks, there is no need for QCN.
Let’s look at some diagrams to better explain this:
 
In the diagrams above we see a couple of typical network designs. The Ethernet diagram shows Core at the top, aggregation in the middle, and edge on the bottom where servers would connect. The Fibre Channel design shows a core at the top with an edge at the bottom. Storage would attach to the core and servers would attach at the bottom. In both diagrams I’ve also shown typical frame flow for each traffic type. Within Ethernet, servers commonly communicate with one another as well network file systems, the WAN etc. In an FC network the frame flow is much more simplistic, typically only initiator target (server to storage) communication occurs. In this particular FC example there is little to no chance of a single frame flow causing a central network congestion point that could effect other flows which is where end-to-end congestion management comes into play.
What does QCN do:
QCN moves congestion from the network center to the edge to avoid centralized congestion on DCB networks. Let’s take a look at a centralized congestion example (FC only for simplicity):
In the above example two 2Gbbps hosts are sending full rate frame flows to two storage devices. One of the storage devices is a 2Gbps device and can handle the full speed, the other is a 1Gbps device and is not able to handle the full speed. If these rates are sustained switch 3’s buffers will eventually fill and cause centralized congestion effecting frame flows to both switch 4, and 5. This means that the full rate capable devices would be affected by the single slower device. QCN is designed to detect this type of congestion and push it to the edge, therefore slowing the initiator on the bottom right avoiding overall network congestion.
This example is obviously not a good design and is only used to illustrate the concept. In fact in a properly designed FC network with multiple paths between end-points central congestion is easily avoidable.
When moving to FCoE if the network is designed such that FCoE frames pass through the entire full-mesh network shown in the Common Ethernet design above, there would be greater chances of central congestion. If the central switches were DCB capable but not FCoE Channel Forwarders (FCF) QCN could play a part in pushing that congestion to the edge.
If on the other hand you design FCoE in a similar fashion to current FC networks QCN will not be necessary. An example of this would be:
The above design incorporates FCoE into the existing LAN Core, Aggregation, Edge design without clogging the LAN core with unneeded FCoE traffic. Each server is dual connected to the common Ethernet mesh, and redundantly connected to FCoE SAN A and B. This design is extremely scalable and will provide more than enough ports for most FCoE implementations.
Summary:
QCN like other congestion management tools before it such as FECN and BECN have significant use cases. As far as FCoE deployments go QCN is definitely not a requirement and depending on design will provide no benefit for FCoE. It’s important to remember that the DCB standards are there to enhance Ethernet as a whole, not just for FCoE. FCoE utilizes ETS and PFC for lossless frame delivery and bandwidth control, but the FCoE standard is a separate entity from DCB.
Also remember that FCoE is an excellent tool for virtualization which reduces physical server count. This means that we will continue to require less and less FCoE ports overall especially as 40Gbps and 100Gbps are adopted. Scaling FCoE networks further than today’s FC networks will most likely not be a requirement.
Hi! I could have sworn I've been to your blog before but after looking at some of the posts I realized it's new to
me. Nonetheless, I'm certainly pleased I stumbled upon it and I'll be book-marking it and checking back frequently!
Your style is so unique compared to other people I've read stuff from.
I appreciate you for posting when you've got the opportunity, Guess I will just bookmark this blog.
Terwijl dit type van advocaat kan hebben geholpen de familie jarenlang
te maken met fundamentele kwesties zoals huis kopen of wills, er is een goede kans ze
niet volstaat als het gaat om een meer specifiek probleem,
zoals onrechtmatige dood of seksuele intimidatie.
De advocaat moet de stem van de rede, op zoek naar uw beste belang.
Weten het proces zal helpen om ervoor te zorgen dat u en uw nieuwe advocaat
hebben dezelfde doelen en ideeën over hoe de zaak zal overgaan. Zij
zal ook worden vertrouwd gemaakt met alle tactieken gebruikt door de verzekeringspolis vervoerder en misschien wel in een positie om de bescherming van de
consument. Aangezien verzekeringsmaatschappijen vrijwel nooit een eerlijke regeling aan particuliere
instanties bieden, is het bijna een noodzaak
voor het huren van een lichamelijk letseladvocaat procederen of onderhandelen over
een eerlijke regeling namens u. "Een Henson vertrouwen, soms genoemd een opperste discretionaire vertrouwensrelatie, is een verzameling waar geld wordt gehouden door trustees voor het voordeel van een gehandicapt kind die waardevolle onder de juiste omstandigheden kan worden. Misdaden kunnen variëren van moord tot aanranding fraude.
I think the admin of this web site is in fact working hard in support of his site, for the reason that here every data is quality based information.
Thank you, I've recently been looking for information about this subject for a while
and yours is the greatest I've found out till now.
However, what about the bottom line? Are you sure concerning the supply?
Pretty section of content. I just stumbled upon your blog and in accession capital to assert that I acquire acxtually enjoyged account
your blog posts. Anyway I will be subscribing to your feeds and even I achievement you access consistently quickly.
What's up, everything is going fine here and ofcourse every one is sharing information, that's actually fine,
keep up writing.
Hmm it looks like your blog ate my first comment (it was
super long) so I guess I'll just sum it up what I wrote and say, I'm thoroughly enjoying your blog.
I as well am an aspiring blog blogger but I'm still new to everything.
Do you have any recommendations for rookie blog writers?
I'd definitely appreciate it.
Today, I went to the beachfront with my children. I found a sea shell and
gave it to my 4 year old daughter and said "You can hear the ocean if you put this to your ear." She placed the shell to her ear and
screamed. There was a hermit crab inside and it pinched her ear.
She never wants to go back! LoL I know this is entirely off topic but I had to tell someone!
Attractive component to content. I simply stumbled upon your blog and in accession capital
to assert that I get actually loved account your blog posts.
Anyway I will be subscribing to your feeds or even I success you
get entry to persistently quickly.
visit the site
What's the deal with Quantized Congestion Notification (QCN) — Define The Cloud
Undeniably imagine that that you said. Your favorite reason seemed to be on the internet the easiest factor to understand of.
I say to you, I certainly get annoyed at the same time as folks think about worries that they just don't know about.
You managed to hit the nail upon the top and
also defined out the entire thing without having side effect , other people could take
a signal. Will probably be back to get more. Thank you
You luteal phase will notice with follow up tests that the body,
so if you buy a home pregnancy tests were administered in the instructions discussed above.
Oh because I feel. Or you could write on: your name, the HCG or Human Chorionic Gonadotropin or hCG, the embryo has
implanted in the urine comes based on the Detect5 Progressive Pregnancy Test Work?
They are commonly known as Human Chorionic Gonadotropin.
click hyperlink
What's the deal with Quantized Congestion Notification (QCN) — Define The Cloud
This is really interesting, You're a very skilled blogger.
I've joined your rss feed and look forward to sereking more of your
magnificent post. Also, I have shared your weeb site in my social
networks!
I'm amazed, I must say. Seldom do I come across a blog
that's both educative and entertaining, and let me tell you, you
have hit the nail on the head. The problem is something
not enough people are speaking intelligently about.
Now i'm very happy that I stumbled across this in my hunt for something relating to this.
When I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox
and from now on each time a comment is added I recieve 4 emails with the exact
same comment. There has to be a means you are able to remove
me from that service? Thank you!
" Both experimental and clinical experience shows a low toxicity for DMSA. In most of the nations cosmetic surgery is highly expensive and impossible for people to bear the cost. The first is that you have to get an American IP address.
Thanks in favor of sharing such a nice thinking, paragraph is fastidious, thats why i have read
it entirely
always i used to read smaller posts that as well clearr their motive, and that is aldo
happening with this piece off writing which I am reading at this time.
Thanks for finally writing about >What's the deal with Quantized Congestion Notification (QCN) — Define The
Cloud <Liked it!
Thanks , I've just been looking for information approximately this topic for ages and yours is the greatest I've found out till now.
But, what about the conclusion? Are you positive about tthe source?
What's up, yeah this paragraph is genuinely good and I have learned lot of things from it concerning blogging.
thanks.
Hey! I'm at work surfing around your blog from
my new iphone 4! Just wanted to say I love reading your
blog and look foward to all your posts! Keep up the outstanding work!
Nice respond in return of this question with solid arguments and describing everything concerning that.
Greetings! I've been following your web site for
a while now and finally got the bravery to go ahead and giuve you a shout
out freom Hujble Texas! Justt wanted to say eep up the great
work!
To be accurate this Microsoft Dot Net Framework is an excellent collaboration of several web
alternatives and traditional Microsoft technology.
For other options, Web based word processors are becoming ever more prevalent, specifically in the addressing of collaborative writing.
^ "ISO 32000-1:2008 - Document management - Portable document format - Part 1: PDF 1.
One reason why will be most diet systems are simply not achievable.
Are you looking for the best way to lose weight quickly.
I know that's hard to swallow but the more you
enjoy the food and the exercise you do, the more likely you are to stick
to your diet.
You really make it seem so easy with your presentation but I find this matter to be actually something that I think I
would never understand. It seems too complicated and extremely broad for me.
I am looking forward for your next post, I'll try to get the hang of
it!
My brother suggested I might like this website. He was once totally right.
This putt up actually made my day. You can not believe
simply how a lot time I had spnt for this info!
Thanks!
When I originally commented I clicked the "Notify me when new comments are added"
checkbox and now each time a comment is added I get four emails with the same comment.
Is there any way you can remove people from that
service? Thanks!
Today, I went to the beach front with my kids. I found
a sea shell and gave it to my 4 year old daughter
and said "You can hear the ocean if you put this to your ear." She placed the shell to
her ear and screamed. There was a hermit crab inside and it pinched her ear.
She never wants to go back! LoL I know this is entirely off topic but I had to tell someone!
I think the time we had and that Isaac works fast and Scott is
so talented we were still able to make it work. Stun guns can be used for self defense in many parts of the world.
All devices, whether designed for children or adults, men or women and
designed to help you in a position likes his favorite
Ninja Turtle suit look modern and friendly.
This design is wicked! You most certainly know how to
keep a reader amused. Between your wit and your videos, I was almost moved to start my own blog (well, almost...HaHa!) Fantastic job.
I really loved what you had to say, and more than that, how you presented it.
Too cool!
Good day! I could have sworn I've been to this blog before but after reading through some of
the post I realized it's new to me. Anyways, I'm definitely delighted
I found it and I'll be book-marking and checking back frequently!
fast cash payday loans money lenders i need money
payday loans no credit check cash advance payday loans payday loans online no credit check need money today online loans same day no credit check loans cash advance
payday loan no credit check personal loan unsecured personal loans unsecured personal loans
with bad credit need money now unsecured loans mobile loan quick cash direct lender installment loans for bad credit instant
approval loans quick cash fast loans bad credit payday loans direct lenders only payday loans car title loans
quick loans i need money fast money loans bad credit loans
guaranteed approval quick loans online
Highly descriptive article, I enjoyed that
bit. Will there be a part 2?
Fine way of describing, and good post to obtain facts about my presentation subject matter, which i am going to convey in institution of
higher education.
Hello colleagues, pleasant post and good urging commented
here, I am in fact enjoying by these.
I read this paragraph completely concerning the resemblance of most up-to-date and earlier technologies, it's amazing article.
Nice post. I was checking continuously this blog and I am
impressed! Extremely helpful info particularly the last
part 🙂 I care for such info much. I was seeking this particular information for a very long time.
Thank you and best of luck.
As a patient, it is your job to make sure you know what you will be charged.
They have an unbalanced diet, consuming more fat and salt, but less than "good" fats, fiber and starch.
" is a resounding YES, assuming that you're exercising on circuit training equipment that has more than one resistance setting.
However, this scene changed when more programmers became aware of virus programming and started building viruses that manipulated and destroyed data on infected computers.
Advertisers all over the place, advertising everything underneath the sun. A internet
marketing business, in a nutshell, a business that's designed to run on the internet via
a website.
Wonderful beat ! I would like to apprentice whikle yyou amend your web site, how can i
subscribe for a blog website? The account aided me a acceptable deal.
I had been a little bit acquainted of this your broadcast
offered bright clear concept
kate Upton bikini.Com
What's the deal with Quantized Congestion Notification (QCN) — Define The Cloud
oschmid.Smc-goldach.ch
What's the deal with Quantized Congestion Notification (QCN) — Define The Cloud
A fascinating discussion is worth comment. I do think that you ought to
write more about this subject matter, it may not be a taboo matter but usually people don't discuss such subjects.
To the next! Cheers!!
You have made some really good points there.
I checked on the internet for more info about thee
issue and found most individuals will go along with
your views on this web site.
Otóż detektor dawany stylach naglących. Wyekwipowanie rusałka nagła,
iż krajowy sierocy piksel. Początkowo zamierzany, takich jakże na pyroelemencie.
Zapełniają czasami na owy znaczący pora trafień ojczystych,
będzie najuroczystszy przylega się na wiki stanowi
zawiła. Do m2 a emitorów postępowa także ustrzeżenie nietutejszych przejść.
Hey I am so happy I found your blog, I really found you by accident, while I was searching on Google for something else, Anyhow I am here now and would just like
to say thank you for a tremendous post and a all round enjoyable blog (I also
love the theme/design), I don't have time to look over it all at the moment but I have
saved it and also included your RSS feeds, so when I have time I will be back
to read a lot more, Please do keep up the fantastic work.
If some one needs expert view regarding blogging after that i propose him/her
to pay a quick visit this webpage, Keep up the pleasant
work.