Hi All
I’ve been meaning to do this video for ages and finally had some time to do it.
The most common questions I tend to get are generally around booting a Cisco UCS Server from the SAN. Now in order to take full advantage of the statelessness of Cisco UCS servers we certainly want to avoid any dependency on a blade and SAN Boot is a great way to do it. And in Cisco UCS it’s an absolute dream to set up.
But I’ve decided not to just stop at the Cisco UCS config but also include the SAN switch config and the Array config. Why? you may ask. Well in this day of ever increasing convergence roles are merging and Silos crashing so it makes sense to have a good overview of the entire process. And even if all these elements are still conducted by separate admins in your environment, well it’s still great to have an appreciation of the information they need in order to work closer and more efficiently with them. I have seen too many cases when trying to troubleshoot a Boot From SAN issue (or any issue for that matter) when different admins did not communicate with each other and used different (not wrong) naming conventions etc.. and it just made end to end troubleshooting that bit harder. The more consistent we can make something by working together and sharing information certainly makes everyone’s job allot easier.
Anyway grab yourself a Scotch and sit back and let the next 60mins wash over you, it always goes down smooth.
Regards
Colin
UCSguru.com 
Thanks for putting this video together. Nice work!
Have you looked at scripting the process at all? I’m wondering about using PowerShell to get the service profile from UCS, then connect to to the MDS and create the necessary objects.
Just curious if you’ve seen any posts around automating it a bit.
Thanks again for your work on this video.
Hi and Thanks for the great comments.
When it comes to scripting I’m a bit of a “script kiddie” and tend to rely on scripts created by much smarter people.
There are some great ones out there for configuring UCS from scratch and doing snap health checks etc.. The author of some of my favourite scripts is Jeremy Waldrop who’s blog I would recommend you check out.
Regards
Colin
Pingback: Server SAN boot installation made easy: UCS Director | ANUJ MODI
You mentioned the FIs perform a FLOGI into the SAN switches, what WWNN and WWPN do they use?
Also, the “Boot Target” can be ANY valid WWPN, does not have to be the exact Array WWPN. Would that be correct?
Excellent video walkthrough btw.
Regards,
Amit.
Thanks for the kind words Amit
The FI’s “proxy flogi” thus use the WWPN assigned in the Service Profile.
And the WWPN of the boot targets need to be valid WWPN of the array ports connected to that Fabric. You can find this by checking the flogi database on the SAN switches and looking for the WWPN on the SAN F ports connected to your array.
Regards
Colin
Great video Colin, thank you for taking the time to put this together.
Interesting that you chose to use 1 pool for the MAC addresses, I have tended to use 2 (1 for Fabric A and one for Fabric B). Could you elaborate on why you only use one pool please? The pool allows for 256 MAC addresses. If you had a requirement for more than 256 NICs n the UCS environment would you create a larger pool or an additional pool?
Your teaching style is excellent and makes it easy to understand concepts. Do you teach the DCUFI course and if so could you let me know when you are teaching it as would be great to have you as a tutor?
Hi Andy
Thanks for the great feedback, I absolutely love giving training courses, the main ones I teach tend to be around Cisco UCS, Nexus, Nexus 1000v and FCoE.
However I am not a CCSI thus do not teach official curriculum, I have looked at doing it and two training agencies Firefly and Global Knowledge have in the past kindly offered to sponsor me as an instructor. (Officially you need to be working for a Cisco Learning Partner, which I do not)
However doing my CCSI would take at least a couple of months out of my day job which my current company are understandably not too keen on :-). But CCSI is Definitely on my list to do as and when time and circumstances allow.
That said your company are more than welcome to book me to come in and do you a bespoke course on any of the above topics.
So re your question on MAC pools, as mentioned in the video I do not routinely create separate pools for Fabric A and Fabric B, mainly because once they leave the FI’s the fabric they are on is no longer relevant. (Unlike WWPN which are always restricted to their respective Fabric, or should be)
But if a client needed to be able to code fabric information into a MAC address for any reason, then of course you can just create a MAC pool per fabric and many customers do just that. I routinely code, location, UCS Domain and Operating System into my MAC pools though.
And tend to create a block of 256 addresses at a time to keep the size of the XML database optimal, I.e. If I created loads of pools with 1000 addresses in them, then each of those addresses would be an XML DB entry but all unused. If and when the first 256 MAC block nears exhaustion I just add in another 256 address block.
Thanks for the question and good luck with your UCS career:-)
Colin
Great video Colin, really nice work.
I recently start with UCS. I still have some doubts about vNICs configuration. I have one chassis with four blade servers (one 1240 adapter with port extender) and 2 FI, they are connected with 2×4 twinax cable trough I/O module. I plan to use Nexus 1000v as a distributed switch. is it enough for me to use one vNIC/Fabric for all net traffic (VM, MGMT, vMotion, DMZ) or should i use more vNICs? I guess you are using standard vSwitch, that’s why you created more vNICs for different traffics. One more question about my configuration, if my scenario is fine, should I crete port channel from I/O modules to FI’s, what are my benefits from that. I already use port channel from FI to up-link switches.
Thank you for any answer and sorry for my English.
Mishel
Hi Mishel
Thanks for the comments.
There are numerous ways how to do this many of which I detail in the “Ask the Guru” section on this site so definitely have a look through the many Q&A’s around vSphere vNIC design.
If you are happy running all of your port-groups/profiles up the same uplinks separated by 802.1q tags (VLANs) then by all means just create 2 uplinks (A/B) from the N1KV and team them together.
I sometimes leave vMotion and Management on standard vSwitches to prevent a N1kv misconfig from isolating the ESXI host and to keep vMotion traffic within a Fabric Interconnect. But just my preference.
If you have any particular QoS requirements again easy to configure on the UCS side if you use a different uplink/vNIC for that traffic type.
Regards
Colin
Excellent Video. It was a good learning!!!
Excellent Video. I am Preparing for my CCIE DC and it is very helpfull for the same. This video is like full mock lab.
I request you to put some more videos like this.
Thanks Satish
I’m currently in Cisco Live Milan as a Roving Reporter and will be interviewing a CCIE DC about the perpetration required etc… I’ll post the video here when I’m back.
Good luck on your CCIE DC journey 🙂
Colin
Great video!!!! and many Thanks for putting it up.
One can learn a lot from it.
Any way we can have the sheet at the end of the video posted on the site as well.
Thanks a million again and looking forward to meeting you on the field soon.
HI Colin. Thanks again for this excellent video. I have watched it several times and use it as a useful reference source. Have you done boot from SAN with Windows? I’m trying to establish the best way to configure this on UCS.
Hi Andy
Glad you found it usefull.
I’ve done loads of Boot From SAN bare metal Windows installs, but haven’t video’d it as yet. If you feel you would benefit from it, I’ll make that a future post. Will also do a video showing full boot from iSCSI.
Colin
Hi Colin,
Thank you, I think it would be very beneficial. I have done baremetal Windows installs to local disks. Is the main difference with boot from SAN for Windows that you need to install the B200 drivers during install and also ensure only 1 vHBA is zoned to the boot lun during installation?
Regards,
Andy
Exactly right Andy
Only zone a single path install the OS, do the image switch to the Windows drivers ISO (you need to hit refresh, for UCSM to recognise the image switch) Then finish the install then activate MPIO and Zone the remaining paths when your done.
Then install other Windows drivers as required. Use the utility CD if you need to setup Teaming within the OS.
Job done 🙂
Similar process for Red Hat although you can manually configure the multipathing driver during install.
Regards
Colin
Thank you for the quick and detailed reply Colin.
I’m planning to use Fabric Failover on Windows installs rather than nic teaming. Do you ever use nic teaming (using the enictool)? It could be beneficial in balancing the load across the A & B FIs.
Regards, Andy
This video is great, as we are just about to migrate our outdated Dell VMware environment to a set of 16 shiny new UCS blades. One thing that we’ve been getting caught up on however is with the SAN boot. We’re able to get it working but it takes a LONG time.
We’ve tracked it down to the LUN enumeration. We have the same setup you show with the exception of having all 4 WWPNs for the SAN targets; e.g. fc0 has a primary and secondary WWPN target, and fc1 has a primary and secondary WWPN target. You mentioned in the video that you only have single targets on each due to the simplified lab layout.
Our problem is that both fc0 and fc1 take a good 2 -3 minutes to time out when trying to enumerate their secondary targets. That makes the boot take about 3 times longer than it otherwise could! Our SAN guys insist that this is normal since the LUN can only be owned by one storage processor at a time. Thus the secondary would only respond in the event that the primary failed.
Does this sound right to you? I can’t seem to find anybody else complaining about long boots due to this, but I don’t know enough on the SAN side to convince the SAN guys to look in to it.
Thanks Colin,
Really appreciate ,this video is very useful people learning UCS and Compute .
Just want some clarification regarding server boot .
is this mandatory to put vHBA at first place in service profile ?
you are installing ESXi from scratch , if we already installed image on LUN , what will be the process , i mean would it require to map virtual CD ?
Regrads,
Rohit
Hi Rohit
No its not mandatory, I generally put the CD ahead of the vHBA’s in a Boot From SAN policy. I have heard this can cause Boot from SAN failures in certain cases but I’ve never come across any issues yet.
Regards
Colin
Nice Post. One question ?
Can you show me how to set UCS for max power or redundant power