I was playing with the REST API and Powershell in order to provision vSphere Datastores. I started to think what else could we do with all the cool information we get from the Pure Storage REST API?
I remembered some really cool people here and here had used the open HTTP Post adapter. So I started to work on how to pull data out of the Flash Array and into vCOPS.
Pure Dashboard
We already get some pretty awesome stats in the Pure web GUI. What we don’t get is the trends and analysis. Also I don’t see how my data reduction increases and decreases over time. Also I don’t get stats from multiple arrays.
First Dashboard with Array Stats, Heat Map, and Health based in vCops Baseline
Array Level Stats
First each of these scripts require Powershell 4.0.
1. Enter the Flash Array Names in the variable for $FlashArrayName. You can see I have 4 arrays in the Pure SE Lab.
2. I create a file with the credential to vCOPS. Since we are going to schedule this script to run every few minutes you need to create this file. More information on creating that credential here http://blogs.technet.com/b/robcost/archive/2008/05/01/powershell-tip-storing-and-using-password-credentials.aspx
You MUST read and do that to create the cred.txt file in c:\temp that I reference in the script.
3. Change the $url variable to be the IP or name of your vCOPS UI server.
4. Don’t forget to modify the Pure Flash Array and Password in each script.
Find it on GitHub https://github.com/2vcps/purevcops-array
[code]
cls
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$FlashArrayName = @(‘pure1′,’pure2′,’pure3′,’pure4’)
$AuthAction = @{
password = "pass"
username = "user"
}
# will ignore SSL or TLS warnings when connecting to the site
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$pass = cat C:\temp\cred.txt | ConvertTo-SecureString
$mycred = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist "admin",$pass
# function to perform the HTTP Post web request
function post-vcops ($custval,$custval2,$custval3)
{
# url for the vCOps UI VM. Should be the IP, NETBIOS name or FQDN
$url = "<vcops ip>"
#write-host "Enter in the admin account for vCenter Operations"
# prompts for admin credentials for vCOps. If running as scheduled task replace with static credentials
$cred = $mycred
# sets resource name
$resname = $custval3
# sets adapter kind
$adaptkind = "Http Post"
$reskind = "Pure FlashArray"
# sets resource description
$resdesc = "<flasharraydesc>"
# sets the metric name
$metname = $custval2
# sets the alarm level
$alrmlev = "0"
# sets the alarm message
$alrmmsg = "alarm message"
# sets the time in epoch and in milliseconds
#This is setting us 7 hours behind
$epoch = [decimal]::Round((New-TimeSpan -Start (get-date -date "01/01/1970") -End (get-date)).TotalMilliseconds)
# takes the above values and combines them to set the body for the Http Post request
# these are comma separated and because they are positional, extra commas exist as place holders for
# parameters we didn’t specify
$body = "$resname,$adaptkind,$reskind,,$resdesc`n$metname,$alrmlev,$alrmmsg,$epoch,$custval"
# executes the Http Post Request
Invoke-WebRequest -Uri "https://$url/HttpPostAdapter/OpenAPIServlet" -Credential $cred -Method Post -Body $body
#write-host $resname
#write-host $custval2 "=" $custval "on" $custval3
}
ForEach($element in $FlashArrayName)
{
$faName = $element.ToString()
$ApiToken = Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/apitoken" -Body $AuthAction
$SessionAction = @{
api_token = $ApiToken.api_token
}
Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/session" -Body $SessionAction -SessionVariable Session
$PureStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?action=monitor" -WebSession $Session
$PureArray = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?space=true" -WebSession $Session
ForEach($FlashArray in $PureStats) {
$wIOs = $FlashArray.writes_per_sec
$rIOs = $FlashArray.reads_per_sec
$rLatency = $FlashArray.usec_per_read_op
$wLatency = $FlashArray.usec_per_write_op
$queueDepth = $FlashArray.queue_depth
$bwInbound = $FlashArray.input_per_sec
$bwOutbound = $FlashArray.output_per_sec
}
ForEach($FlashArray in $PureArray) {
$arrayCap =($FlashArray.capacity)
$arrayDR =($FlashArray.data_reduction)
$arraySS =($FlashArray.shared_space)
$arraySnap =($FlashArray.snapshots)
$arraySys =($FlashArray.system)
$arrayTP =($FlashArray.thin_provisioning)
$arrayTot =($FlashArray.total)
$arrayTR =($FlashArray.total_reduction)
$arrayVol =($FlashArray.volumes)
}
post-vcops($wIOs)("Write IO")($faName)
post-vcops($rIOs)("Read IO")($faName)
post-vcops($rLatency)("Read Latency")($faName)
post-vcops($wLatency)("Write Latency")($faName)
post-vcops($queueDepth)("Queue Depth")($faName)
post-vcops($bwInbound)("Input per Sec")($faName)
post-vcops($bwOutbound)("Output per Sec")($faName)
post-vcops($FlashArray.capacity)("Capacity")($faName)
post-vcops($FlashArray.data_reduction)("Real Data Reduction")($faName)
post-vcops($FlashArray.shared_space)("Shared Space")($faName)
post-vcops($FlashArray.snapshots)("Snapshot Space")($faName)
post-vcops($FlashArray.system)("System Space")($faName)
post-vcops($FlashArray.thin_provisioning)("TP Space")($faName)
post-vcops($FlashArray.total)("Total Space")($faName)
post-vcops($FlashArray.total_reduction)("Faker Total Reduction")($faName)
post-vcops($FlashArray.volumes)("Volumes")($faName)
}
[/code]
For Volumes
Find it on GitHub https://github.com/2vcps/purevcops-volumes
[code]
cls
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$FlashArrayName = @(‘pure1′,’pure2′,’pure3′,’pure4’)
$AuthAction = @{
password = "pass"
username = "user"
}
# will ignore SSL or TLS warnings when connecting to the site
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$pass = cat C:\temp\cred.txt | ConvertTo-SecureString
$mycred = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist "admin",$pass
# function to perform the HTTP Post web request
function post-vcops ($custval,$custval2,$custval3,$custval4)
{
# url for the vCOps UI VM. Should be the IP, NETBIOS name or FQDN
$url = "<vcops ip or name>"
#write-host "Enter in the admin account for vCenter Operations"
# prompts for admin credentials for vCOps. If running as scheduled task replace with static credentials
$cred = $mycred
# sets resource name
$resname = $custval
# sets adapter kind
$adaptkind = "Http Post"
$reskind = "Flash Volumes"
# sets resource description
$resdesc = $custval4
# sets the metric name
$metname = $custval2
# sets the alarm level
$alrmlev = "0"
# sets the alarm message
$alrmmsg = "alarm message"
# sets the time in epoch and in milliseconds
#This is setting us 7 hours behind
$epoch = [decimal]::Round((New-TimeSpan -Start (get-date -date "01/01/1970") -End (get-date)).TotalMilliseconds)
# takes the above values and combines them to set the body for the Http Post request
# these are comma separated and because they are positional, extra commas exist as place holders for
# parameters we didn’t specify
$body = "$resname,$adaptkind,$reskind,,$resdesc`n$metname,$alrmlev,$alrmmsg,$epoch,$custval3"
# executes the Http Post Request
Invoke-WebRequest -Uri "https://$url/HttpPostAdapter/OpenAPIServlet" -Credential $cred -Method Post -Body $body
write-host $custval,$custval2,$custval3
}
ForEach($element in $FlashArrayName)
{
$faName = $element.ToString()
$ApiToken = Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/apitoken" -Body $AuthAction
$SessionAction = @{
api_token = $ApiToken.api_token
}
Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/session" -Body $SessionAction -SessionVariable Session
$PureStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?action=monitor" -WebSession $Session
$PureVolStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/volume?space=true" -WebSession $Session
ForEach($Volume in $PureVolStats) {
#$Volume.data_reduction
#$Volume.name
#$Volume.volumes
#$Volume.shared_space
#$Volume.system
#$Volume.total
#$Volume.total_reduction
#$Volume.snapshots
$adjVolumeSize = ($Volume.Size /1024)/1024/1024
#$Volume.thin_provisioning
post-vcops($Volume.Name)("Volume Size")($adjVolumeSize)($faName)
post-vcops($Volume.Name)("Volume Data Reduction")($Volume.data_reduction)($faName)
post-vcops($Volume.Name)("Volumes")($Volume.volumes)($faName)
post-vcops($Volume.Name)("Shared Space")($Volume.shared_space)($faName)
post-vcops($Volume.Name)("System")($Volume.system)($faName)
post-vcops($Volume.Name)("Total")($Volume.total)($faName)
post-vcops($Volume.Name)("Total Reduction")($Volume.total_reduction)($faName)
post-vcops($Volume.Name)("Thin Provisioning")($Volume.thin_provisioning)($faName)
post-vcops($Volume.Name)("Snapshots")($Volume.snapshots)($faName)
}
}
[/code]
Once each of the scripts is working schedule them as a task on a windows server. I do one for volumes and one for arrays and run them every 5 minutes indefintely. This will start to dump the data into vCOPS.
Now you can make Dashboards.
Creating Dashboards
Login to the UI for vCOPS. You must by in the custom UI, the standar UI hides all of the cool non-vSphere customization you can do.
Go to Environment –> Environment Overview
Expand Resource Kinds
This lets you know that data is being accepted to the array. Other than the Powershell script bombing out and failing this is the only way you know it is working. Now for a new Dashboard.
Click Dashboards -> Add
Drag Resources, Metric Selector, Metric Graph and Heat Map to the Right
Name it and Click OK
Adjust the Layout
I like a nice Column for information and a bigger display area for graphs and heat maps. Adjust to your preference.
Edit the Resources Widget
Edit the Name and filters to tag
Now we just see the Flash Arrays
Select your Resource Provider I named mine Lab Flash Arrays as the Providing Widget for the Metric Selector. Also Select the Lab Flash Arrays and Metric Selector as the Providing Widgets for the Metric Graph.
Edit the Metric Graph Widget by clicking the gear icon
I change the Res. Interaction Mode to SampleCustomViews.xml. This way when I select a Flash Array the Graph does show up until I double click the Metric in the Metric Selector. You are of course free to do it as you like.
The Heat Map
Edit the heat map and you will find tons of options.
Create a Configuration
Name the New Configuration
Group by and Resource Kinds
Group by the Resource Kind and then select Pure Flash Array in the drop down.
Select the Metric to Size the Heatmap and Color the Heatmap
Adjust the colors if you think Read and Green are boring
Save the Config!
Look! A cool new heatmap
Do this for all the metrics you want to have as a drop down in teh dashboard.
Obviously there are a lot more things you can do with the Dashboards and widgets. Hopefully this is enough to get you kicked off.