Sunday, February 17, 2019

Accessing the Cohesity REST API Using PowerShell

I'm a long time scripter, so when I started working for Cohesity, one of the first things I did was start learning how to manage Cohesity through its REST API. Cohesity is a hyperconverged scale-out storage platform that serves as an integrated backup appliance and scale-out NAS platform designed to consolidate secondary storage workloads (backup, file, object, test/dev, analytics) onto a single, massively scalable data fabric that spans data centers and clouds.

Cohesity takes an API-first approach to software design, meaning that all functions exposed in the web and cli-based user interfaces are serviced by REST API calls in the background. That means that anything that can be done in the UI can be done from any programming language that can send HTTP messages.



Why Use The Cohesity REST API?


Over the past year or so, I've been writing scripts for customers to help automate Cohesity workflows for various reasons. For example, during initial setup, I've been asked to provide a way to register protection sources en masse, such as hundreds of NAS shares and physical servers. That's a great use case for a script. Other common use cases include:
  • Infrastructure Orchestration: Customers have automated self-service workflows for provisioning new VMs and need a way to automatically add the new VM to a protection job and role-based access lists
  • DevOps Orchestration: Customers want to automate the provisioning of database clones for test/dev environments
  • Web Portal Integration: Customers want to execute Cohesity workflows from other service catalog portals like ServiceNow or vRealize Automation

Whatever the use case, the good news is that everything that can be done in the Cohesity UI can be done via the REST API.

Why PowerShell?


PowerShell is the ubiquitous language in the Windows scripting world these days. So many of my example scripts are written in PowerShell. I also get a lot of call for scripts that run on linux platforms, so I also write examples in Python. PowerShell is available for linux, but I haven't met many linux admins who are keen on installing PowerShell on their systems. Python on the other hand is often already installed and ready to run.

PowerShell is an awesome language both for scripting but also interactive sessions which are great for demos as well as for interactive inspection of the objects returned by the REST API. The density and speed of code generation in PowerShell is I think unmatched by any other scripting language that I've come across.

Connecting to Cohesity (the hard way)


If the following looks a bit complicated, don't worry. I'll show you the raw steps to access the API just to form a basic understanding of what's going on. Later, we can wrap up these steps into easy function calls. Read on.

The first step in accessing the API is to authenticate with a Cohesity cluster and get an access token for use with subsequent API calls. As with any REST API call, we will need a header, body, and URL to construct our API call. In PowerShell, these will look like this:


$HEADER = @{'accept' = 'application/json'; 
            'content-type' = 'application/json'}

$BODY = ConvertTo-Json @{'domain' = 'local'; 
                         'username' = 'admin'; 
                         'password' = 'mypassword'}

$URL = 'https://mycluster/irisservices/api/v1/public/accessTokens'


You can see that the header specifies that we'll be sending and receiving JSON formatted data, and the body contains our credentials. We can then make our REST API call to get our access token:


$auth = Invoke-RestMethod -Method Post -Uri $URL -Header $HEADER -Body $BODY -SkipCertificateCheck

$auth | fl


accessToken : eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJkb21haW4iOiJMT0NBTCIsImV4cGlyYXRpb24tdGltZSI6IjE1NTA1MjkzNjYiLCJzaWRzL
              Whhc2giOiJ1VWdYNWJGa0NlY1F1UE15aVF0MS1mb0IwQndRcWJFRktQYzk3RHFPOGxBIiwidGVuYW50LWlkIjoiIiwidXNlci1zaWQiOiJTLTE
              tMTAwLTIxLTMzMDkwMTE0LTU3NDA5MzUtMSIsInVzZXJuYW1lIjoiYWRtaW4ifQ.WcdmFkyqYYWm2EhFe4LBT95ycMjRs2wyKeyIGrxuAnKN9C
              NbZAR0u3F3gsyjMo1u_iPxEsoR_Zl_JOLALXDrp400kek-rSCQcpnsmgutLQ8B5nVKKtAceDd1gFcSIeuE_X1ENppfYfw2_GlTW3vcBEe8DWjq
              ua1WY6JAZjJUJ-A
tokenType   : Bearer
privileges  : {ANALYTICS_MODIFY, ANALYTICS_VIEW, CLONE_MODIFY, CLONE_VIEW...}


As you can see, we received back an accessToken, tokenType, and a list of role-based access privileges granted to us by our login. We can then insert the tokenType and accessToken into our header, so that we can make authenticated API calls.


$HEADER = @{'accept' = 'application/json'; 
            'content-type' = 'application/json'; 
            'authorization' = $auth.tokenType + ' ' + $auth.accessToken}


Finally, we can set a new URL for our next API call, and get some real data from the Cohesity cluster. For example, we can get a list of protectionJobs:


$URL = 'https://mycluster/irisservices/api/v1/public/protectionJobs'

$jobs = Invoke-RestMethod -Method Get -Uri $URL -Header $HEADER -SkipCertificateCheck

$jobs | Format-Table

   id name                 environment    policyId                            viewBoxId parentSourceId sourceIds
   -- ----                 -----------    --------                            --------- -------------- ---------            
    7 VM Backup            kVMware        770535285385794:1544976774290:5             5              1 {29, 30}             
   12 Infrastructure       kVMware        770535285385794:1544976774290:5             5              1 {121, 36, 39, 33...} 
   35 Oracle               kOracle        770535285385794:1544976774290:377           5             64 {61}                 
  841 Generic NAS          kGenericNas    770535285385794:1544976774290:25            5             85 {86}                 
 6222 File-Based Backup    kPhysicalFiles 770535285385794:1544976774290:5             5             60 {98}                 
 8028 SQL Backup           kSQL           770535285385794:1544976774290:377           5             46 {31}                 
12886 Scripts              kView          770535285385794:1544976774290:25            5            102 {116}      
          

Let's take a closer look at the first job in the list.


$jobs[0] | fl


id                               : 7
name                             : VM Backup
environment                      : kVMware
policyId                         : 770535285385794:1544976774290:5
viewBoxId                        : 5
parentSourceId                   : 1
sourceIds                        : {29, 30}
excludeSourceIds                 : {19, 87, 24, 26...}
startTime                        : @{hour=23; minute=30}
timezone                         : America/New_York
incrementalProtectionSlaTimeMins : 60
fullProtectionSlaTimeMins        : 120
priority                         : kMedium
alertingPolicy                   : {kFailure}
indexingPolicy                   : @{disableIndexing=False; allowPrefixes=System.Object[]; denyPrefixes=System.Object[]}
qosType                          : kBackupHDD
environmentParameters            : @{vmwareParameters=}
uid                              : @{id=7; clusterId=770535285385794; clusterIncarnationId=1544976774290}
policyAppliedTimeMsecs           : 1548971018230
modificationTimeUsecs            : 1548971018230150
modifiedByUser                   : admin
creationTimeUsecs                : 1544977491464965


As you can see, we can extract all the details of the job.


DateTime Formats


You can see in the output above, that dates and times returned by the API are expressed in Unix Time (also known as POSIX time or UNIX Epoch time), usually in microseconds (since 00:00:00 Jan 1, 1970). In the example above, we can convert the creationTimeUsecs into a human-readable date in PowerShell like so:



$unixTime = ($jobs[0].creationTimeUsecs)/1000000

[datetime]$origin = '1970-01-01 00:00:00'

$origin.AddSeconds($unixTime).ToLocalTime()

Sunday, December 16, 2018 11:24:51 AM


and of course we can perform the conversion in the other direction:



[datetime] $dateString = 'Sunday, December 16, 2018 11:24:51 AM'

([Math]::Floor([decimal](Get-Date($datestring).ToUniversalTime()-uformat "%s")))*1000000

1544977491000000


OK, this is looking really complicated. I really don't want to be using a command like that in my day-to-day scripting. It's time to provide some functions to hide this kind of complexity.


Using a Function Library


Once I learned to do the basics as I've shown above, I developed a function library to make all these steps much easier. I built a number of functions and placed them in a file called cohesity-api.ps1 (see listing at the bottom of this post).

To use the functions, we can source the library, then use its functions like so:


. ./cohesity-api.ps1
apiauth mycluster admin
Connected!
$jobs = api get protectionJobs
usecsToDate $jobs[0].creationTimeUsecs

Sunday, December 16, 2018 11:24:51 AM


As you can see, we've eliminated all of the complicated steps of managing headers, URLs, and date conversions by wrapping all that code into functions in the library. Now we can stop wasting time fussing with the mechanics of API calls and spend more of our time doing what we came here to do: learn to exercise the Cohesity API itself.


Where To Go From Here


Now that we've got a simple way to access the Cohesity REST API, what can you do with it? Well there are several documentation resources at your disposal. First, there is the on-board Swagger page that's hosted on every Cohesity cluster. You can access page by clicking the REST API link at the bottom of the Cohesity UI. This link takes you to: https://<mycluster>/docs/restApiDocs/browse/



The swagger interface shows you that each API call has its own URL and which HTTP verb to use. If you click on one of the items, it will expand to show you the various parameters and even has a 'Try it out' button to make API calls right in the interface.

If you need more comprehensive documentation, every cluster hosts an API doc page:

https://<mycluster>/docs/restApiDocs/bootprint/

and you can also find the API documentation on the Cohesity. com web site at:


Examples


I've written quite a few example scripts already. You can find them here:


I'll explain how I learned how to write these scripts in my next few articles. Coming soon! 

The Unofficial Cohesity REST API Function Library


cohesity-api.ps1