Equallogic PS Series Design Considerations
VMware vSphere introduces support for multipathing for iSCSI. Equallogic released a recommended configuration for using MPIO with iSCSI. I have a few observations after working with MPIO and iSCSI. The main lesson is know the capabilities of the storage before you go trying to see how man paths you can have with active IO.
- EqualLogic defines a host connection as 1 iSCSI path to a volume. At VMware Partner Exchange 2010 I was told by a Dell guy, “Yeah, gotta read those release notes!”
- EqualLogic limits the number of hosts in the to 128 per pool or 256 per group connections in the 4000 series (see table 1 for full breakdown) and to 512/2048 per pool/group connections in the 6000 series arrays.
- The EqualLogic MPIO recommendation mentioned above can consume many connections with just a few vSphere hosts.
I was under the false impression that by “hosts” we were talking about physical connections to the array. Especially since the datasheet says “Hosts Accessing PS series Group”. It actually means iSCSI connections to a volume. Therefore if you have 1 host with 128 volumes singly connected via 1 iSCSI path each, you are already at your limit (on the PS4000).
An example of how fast vSphere iSCSI MPIO (Round Robin) can consume available connections can be seen this this scenario. Five vSphere hosts with 2 network cards each on the iSCSI network. If we follow the whitepaper above we will create 4 vmkernel ports per host. Each vmkernel creates an additional connection per volume. Therefore if we have 10 300 GB volumes for datastores we already have 200 iSCSI connections to our Equallogic array. Really no problem for the 6000 series but the 4000 will start to drop connections. I have not even added the connections created by the vStorage API/VCB capable backup server. So here is a formula*:
N – number of hosts
V – number of vmkernel ports
T – number of targeted volumes
B – number of connections from the backup server
C – number of connections
(N * V * T) + B = C
| Equallogic PS Series Array | Connections (pool/group) |
| 4000E | 128/256 |
| 4000X | 128/256 |
| 4000XV | 128/256 |
| 6000E | 512/2048 |
| 6000S | 512/2048 |
| 6000X | 512/2048 |
| 6000XV | 512/2048 |
| 6010,6500,6510 Series | 512/2048 |
Use multiple pools within the group in order to avoid dropped iSCSI connections and provide scalability. This reduces the number of spindles you are hitting with your IO. Using care to know the capacity of the array will help avoid big problems down the road.
*I have seen the connections actually be higher and I can only figure this is because the way EqualLogic does iSCSI redirection.
Jon, recheck line three in your table.
As you’ve noticed multipathing can consume iSCSI session very quickly, so careful planning is important.
Fixed. I must have crossed that up somehow. I didn’t look right to me. Thanks for the catch.
Yes. The planning part is super important. Now that I know it is not a big deal. Thanks again.
Hi Jon,
Nice post! I talked to a couple of the guru’s internally about the content. I’d like to point out a couple of things: I think you might be referencing the old version of the “CONFIGURING VMWARE VSPHERE SOFTWARE ISCSI WITH DELL EQUALLOGIC PS SERIES STORAGE” paper( 1.0) with a 2:1 or even 3:1 example. Version 1.1 (found here: http://www.equallogic.com/resourcecenter/assetview.aspx?id=8453) of the paper has better information about what is actually going on and recommends a 1:1 session connection. FYI – This is something we are looking into.
–Take It easy!
Jeff Sullivan
Delltechcenter.com
@sanpenguin
Yes. I had the first whitepaper hot off the presses. Thanks for the link to the new version. Internally we started doing a 1:1 vmkernel to nic ratio whenever a 4000 series is involved. 6000 and the 6500 don’t seem to sweat when I throw 4 vmkernel ports at it.
I recently used the “CONFIGURING VMWARE VSPHERE SOFTWARE ISCSI WITH DELL EQUALLOGIC PS SERIES STORAGE” to configure a new ESXi 4 server.
The other day I noticed this posting in the VMware section of the Equallogic KnowledgeBase.
“ESX 4, ESXi 4 (including Update1) having issues with dropped “MPIO connections” using native software initiator” – https://na6.salesforce.com/_ui/selfservice/pkb/PublicKnowledgeSolution/d?orgId=00D300000000Ar6&id=50180000000qnpM&retURL=%2Fsol%2Fpublic%2Fsolutionbrowser.jsp%3Fcid%3D02n300000000b3Q%26orgId%3D00D300000000Ar6&ps=1
From what I’ve read on several websites, even the workaround suggested by Equallogic still has the same problem. The other rumor I’ve read is that it will not get fixed until ESX(i) Update 5!!!