Fetching raw statistics data via the API

As well as a mechanism to return aggregated reports, an API call can also be used to retrieve raw data. This gives a flat report in CSV format you can use in analysis tools such as Tableau, Power BI and others.

Additional information includes:

• date and time of each authentication and transfer

• The reasons why an authentication failed (e.g. bad password)

• Whether or not a transfer to a resource was blocked by restrictive mode

Prerequisites

• The optional reports API functionality to be enabled for your organisation - your account manager will be happy to discuss pricing with you

• You will need to know your domain’s API name. This is usually, but not always, the same as your federation scope.

• An API key from each organisation and sub-organisation you will be querying

  • The key will need either the reports API permission (recommended) or the full access permission.

Important differences

• Due to the size of the reports, raw reports are only available for the last 30 days.

Request

Begin with a GET request to the following URL: https://reports.openathens.net/api/v1/example.org/rawreport?date=<date>

Where example.org is your OpenAthens API name and date is the requested date (format yyyy-MM-dd between yesterday and 30 days ago).

date is the only parameter that can be passed in the request.

GET https://reports.openathens.net/api/v1/example.org/rawreport?date=2023-05-22 
Authorization: OAApiKey <api-key>

(There’s an example available using Postman)

Response payload

The response will be an application/vnd.eduserv.iam.reports.task-v1+json object.

If the job completes in under 5 seconds, it will have a status of FINISHED and a href to the report - see downloading below.

If the job is still running after 5 seconds, it will return a status of RUNNING and a href back to the task status.

Your application will need to allow for both types of response. 20s is a reasonable wait time to code if you get a running response.

Example - still running

GET https://reports.openathens.net/api/v1/example.org/rawreport?date=2023-05-22 
Authorization: OAApiKey <api-key>

Response: HTTP/1/1 200 OK
Content-Type: application/vnd.eduserv.iam.reports.task-v1+json
  {
    "percentComplete": 40, 
    "status": "RUNNING", 
    "links": [ 
      { 
        "href": "https://reports.openathens.net/api/v1/example.org/rawtask/a11b5fff-6170-38e3-b4bb-560bee3e6985",
        "rel": "SELF",
        "type": "application/vnd.eduserv.iam.reports.task-v1+json",
        "method": "get" 
      } 
    ], 
    "id": "a11b5fff-6170-38e3-b4bb-560bee3e6985" 
  }

Example - finished

GET https://reports.openathens.net/api/v1/example.org/rawtask/a11b5fff-6170-38e3-b4bb-560bee3e6985 
Authorization: OAApiKey <api-key> 
  
Response: HTTP/1/1 200 OK 
Content-Type: application/vnd.eduserv.iam.reports.task-v1+json
  {
    "percentComplete": 100,
    "status": "FINISHED",
    "links": [
     {
       "href": "https://reports.openathens.net/api/v1/example.org/rawreport/a11b5fff-6170-38e3-b4bb-560bee3e6985",
       "rel": "DOWN",
       "type": "application/vnd.eduserv.iam.reports.report-v1+json",
       "method": "get"
      }
    ],
    "id": "a11b5fff-6170-38e3-b4bb-560bee3e6985"
  }

Downloading the linked report

Depending on your method you may optionally add a filename parameter to the download URL, e.g. https://reports.openathens.net/api/v1/example.org/rawreport/a11b5fff-6170-38e3-b4bb-560bee3e6985?filename=rawstats_2023-06-20.

The filename parameter is method dependant - e.g. a cURL command will use what you specify using --output or >.

Report format

The reports can be very large, so they’re downloaded as a compressed CSV file. Not all fields will have data for all actions:

Actions

How often should I run the reports?

Unlike the aggregated reports accessed by the other reporting API, these reports only cover one day at a time, and only go back 30 days. If scheduling them we would suggest a daily or weekly cycle. This will give you time to spot any days that don’t run for any reason and address them.

Anything to watch out for?

Changes to the reportable state of attributes in your schema will only affect items recorded after the change is saved.

The report will only cover the organisation where the API key sits, so if you have sub-organisations you will need an API key from each and will need to make additional requests with those keys.

All times and dates are UTC so your timezone can affect when yesterday’s data becomes available.

The filename parameter, if used, goes on the download link and not the initial request.

If you’re using Windows and the zip file won’t open, change the extension to .gz.

(There’s an example available using Postman)

Location data

This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com