Menu
Note that because the new policy under R5 is one of continuous maintenance for incremental changes, this document may mention planned changes.
COUNTER 5 SUSHI is a standard for software developers to use to allow software to automatically retrieve all possible reports for all supported possible time ranges for a given library or consortium. Such a standard is called an “API” and allows the developers to write their programs in whatever language they like.
COUNTER 5 SUSHI uses the “client-server” model where the vendor provides a SUSHI-compliant server, and the user runs “client” software to harvest the reports from the server. The client may be anything from a locally installed desktop program to a cloud-based commercial harvesting service that the end customer interacts with via a web environment.
COUNTER_SUSHI is a RESTful interface returning JSON-formatted reports. This allows retrieval of full reports, or of snippets of usage. It also allows usage data to be embedded in other applications.
A SUSHI URL has four basic sections: the base URL of the vendor’s SUSHI server, the particular report being requested, the authentication credentials identifying the particular customer’s account, and report-specific settings such as date range and other report options.
“REST” basically means a URL can be created that includes all of the information needed to make a successful report request.
JSON is a format for encoding text-based (including Unicode) structured data, more similar to XML than to spreadsheet formats like csv. There are some examples in the “Content of Reports” section below.
Taken together, these mean that a properly authenticated user could even retrieve their data without a special client at all, just using a normal web browser, although that is primarily useful for troubleshooting, as the JSON response will not be user-friendly to read.
Note that including more attributes to show will not just add more “columns” but often results in more “rows” per database/platform/title/item. In particular, the YOP (year of publication) for title-level journal-containing Master Reports can get very large as each title/YOP combination is given its own row.
It is not valid to try to request optional attributes and/or data types for the standard views, as by definition those are meant to be “canned” combinations of those settings.
A SUSHI server implementation has two major components: authentication of the client, based on the credentials included in the request, and creating the report content, that is the response from the SUSHI server to that request.
Authentication settings control which clients/end customers can retrieve which institutions’/consortial reports.
The credentials are embedded in plain text in the SUSHI URL. The customer_id is mandatory; the others are optional at the discretion of the vendor.
The customer_id, requestor_id, and api_key are embedded in the SUSHI URL itself (“parameters”), so it is recommended (but not required) that vendors assign such codes using characters that do not need to be URI-encoded (e.g., convert space characters into %20).
Other authentication options/restrictions that are not visible in the SUSHI URL itself are permitted but are problematic because they represent vendor-specific layers which are typically hidden from end customers and their software developers. These are:
The Project COUNTER team are considering including information about the use of these constraints in future versions of the “/status” API response in a future update.
Vendors are not permitted to use any other constraints to access beyond those enumerated in the Code of Practice or Swaggerhub API documents.
There is no rule in the SUSHI standard for when vendors may “expire” any credentials. However, best practice would be not to do this within the life cycle of the COP5 and that customer’s account as it causes confusion and disruption for individual libraries, and much more work for consortial managers.
Platform parameter
The SUSHI API provides a mechanism for a vendor who needs to distinguish the COUNTER report responses by platform, using a simple “platform=” code as part of the URL. Ideally, the vendor should provide all of their COUNTER data on a single base URL with a single set of authentication credentials per library/consortium and use this code to distinguish the report sets. However, some vendors provide different base URLs for their different platforms and may also provide different authentication credentials for each, for example for their journal content and ebook content. This is allowed, but makes for much more work for the end customers and consortial managers and should be avoided in favor of using the platform code with a single SUSHI server whenever possible.
If you are a consortium administrator, to access usage for your members you will usually have a fully-fledged account and thus will have your own SUSHI credentials: customer_id and, optionally, requestor_id and api_key to get the consortial level reports.
Reports have header information and then “rows” of information about each content item (platform, database, title, or item) being included.
In both the header and content item information, capitalization of the official SUSHI labels must be exactly as provided in the standard, e.g. “Report_Attributes” not “Report_attributes” and “Data_Type” not “data_type”.
All report content must comply with “UTF-8” encoding.
Example of a report header, for a Database Master Report (DR), in its raw JSON. Aside from the date range, filters and attributes for Master Reports should only be included in the header if explicitly selected by the API call – the defaults should not be listed. For standard views whose filters and attributes are pre-defined, those default values should be listed.
{
"Report_Header": {
"Created": "2020-06-29T14:20:43Z",
"Created_By": "Vendor-Name",
"Customer_ID": "12345",
"Report_ID": "DR",
"Release": "5",
"Report_Name": "Database Master Report",
"Institution_Name": "UniversityX",
"Institution_ID": [
{
"Type": "Proprietary",
"Value": "PlatformX:customer_id"
}
],
"Report_Filters": [
{
"Name": "Begin_Date",
"Value": "2020-01-01"
},
{
"Name": "End_Date",
"Value": "2020-07-31"
}
],
"Report_Attributes": [
{
"Name": "Attributes_To_Show",
"Value": "Data_Type|Access_Method"
}
]
}
}
Institution_ID: the “Type” should be one of: ISNI, ISIL, OCLC, Proprietary. These are known as “namespaces”.
Example of a single report item, for a Database Master Report (DR), in its raw JSON. Note that although the requested date range was January to July 2020, only two months, February and April, had usage for this database on this platform. Also note that the dates are not in order; as previously mentioned, JSON generally and the SUSHI standard in particular do not expect the data to be “ordered” by date, or the items to be alphabetical or in any other particular order.
This is a single entry within the “Report_Items” array of the JSON response.
{
"Platform": "Platformname",
"Item_ID": [
{
"Type": "Proprietary",
"Value": "PlatformX:databaseY"
}
],
"Database": "DatabaseName",
"Publisher": "DatabasePublisher",
"Data_Type": "Database",
"Access_Method": "Regular",
"Performance": [
{
"Period": {
"Begin_Date": "2020-04-01",
"End_Date": "2020-04-30"
},
"Instance": [
{
"Metric_Type": "Searches_Automated",
"Count": 2
}
]
},
{
"Period": {
"Begin_Date": "2020-02-01",
"End_Date": "2020-02-29"
},
"Instance": [
{
"Metric_Type": "Searches_Automated",
"Count": 4
}
]
}
]
}
There are some differences between JSON report content and the tabular reports that end customers request manually from their vendors (typically using some kind of customer-accessed report “dashboard” or “account admin” service).
Typically these arise out of the assumption that the tabular reports are meant to be immediately human-readable (e.g., they will open cleanly in Excel), whereas the JSON-format report responses are meant to be further processed by software for human reading.
When there is a discrepancy between the COUNTER website’s description of the requirements of a given report type, and the Swaggerhub API documentation, consider the COP documentation to provide the correct specifications.
Differences between JSON report responses and tabular reports
Additionally there is no way to request an IR via SUSHI that looks like
an IR_A1 because there is no parameter for the Parent_Data_Type filter, although an IR with “include_parent_details=True” requested can be used to generate an IR_A1.
This may be fixed in a future update to the SUSHI API.
Because the point of SUSHI is for software to be able to handle all reports of a given type from all vendors without custom coding per vendor, the specifications for the report header and content must be followed precisely. Examples where there may be problems include formatting of month-year column headers and dates in the report header.
Section 4 of the Code of Practice details which attributes are optional and which mandatory for each kind of report for the tabular reports. Note that “optional” here means for the user to choose to include; providers must implement all attributes.
To maximize the attribute data breakdowns available for each of the Master Reports (ignoring custom data added by individual vendors), use the following fields:
The rule is: list all attributes marked with ‘O’ in tables 4.b, 4.f, 4.k
and 4.p except the Parent and Component elements in attributes_to_show,
separated by ‘|’. For including the Parent and Component elements in IR
add include_parent_details=True and include_component_details=True. This
is explained in section 3.3.8.
There is no rule under 5.0.1 for default inclusion/exclusion, but when incremental release 5.0.2 is official, custom values (created by specific vendors, allowed for in section 11 of the COP5) in enumerated elements like Metric_Type should be excluded by default
unless you specify a filter that includes them.
This is not specific to SUSHI but is a point of confusion worth clarifying. Note that YOP should not be confused with the range of years of the usage report itself, i.e. the years during which the usage took place.
A major change from COUNTER COP4 to COP5 was the breakdown of the rows of usage data by individual YOP when requested for Master Reports (or automatically for TR_J4).
Master Reports only: An important distinction needs to be made between filtering the results to a range of years, versus selecting YOP as an attribute to break down the report (“attributes to show”). If a range of specific years is requested but YOP is NOT selected as an attribute to show, the data will be aggregated (added up) for just those years. But if the YOP attribute is added to the request, each individual year will be a separate row (or rows if other attributes are also added). Because zero-usage rows are not provided in COP5, this can often mean that some YOP years will seem to be “missing”.
Specific to SUSHI, the standard requires vendors to support a parameter to filter to a user-selected range of YOP years.
Standard views are very specifically defined, so there are never any options that can be requested via SUSHI aside from the date range.
Vendors are required to support date ranges from one month to within the most recent 24 months.
The standard requires that vendors provide an option to request all of the report types aggregated across all of their members, in combination with all of the other options for specifying the contents of reports. Vendors implement this through the use of consortium-specific authentication credentials, as the SUSHI standard does not allow vendors to create their own proprietary parameters for the API calls.
SUSHI calls necessarily involve receiving an HTTP header and response code, which should not be confused with the SUSHI report header and SUSHI-defined error codes, or “exceptions”. Appendix F of the COP5 documents the exception codes.
The most basic HTTP response code, 200, simply means that the server is able to provide a proper SUSHI JSON response. Codes 500 and 503 may also be accompanied by a JSON report with a single exception noted in the header. Any other HTTP code (most commonly, 404) generally means the server is not able to provide a JSON response at all.
For instance, if there is a problem with the authentication credentials, that should be reported in the header using the appropriate “2000” level exception, with an HTTP 401 (for 2000 or 2020 exceptions) or 403 (for 2010 exception) error.
Note that the 5.0.2 revision of Appendix F currently underway changes/clarifies some of this. While 404 errors may currently also be combined with a JSON response, if the server is able to do this, under 5.0.2 there should not be JSON with a 404. When these changes are made to Appendix F, you will see a new “HTTP Code” column added.
Project COUNTER provides a free web-based tool for anyone (vendor or consortial or library customer) to use to check the basic responses of a SUSHI server and the content of each kind of report. You will need the base URL and authentication credentials.
As of early August 2020, the Validation Tool is still under development, so it works for most basic validation needs but does not yet check if the JSON report matches the SUSHI request. Also it does not yet have the ability to compare a SUSHI and tabular report to see if they are providing the same data.
You will need to “register” for a free account, which enables the system to keep track of your tests and provide repeat access to SUSHI results for up to 7 days.
The tool does not validate the non-usage reports (server status, supported reports list, and members list (consortial)) but just gives you the JSON.
For the usage reports, the result is a simple summary delivered as an Excel spreadsheet file of whatever problems have been found. The validation report itself does not include the actual JSON response, but you can download that (and re-retrieve the validation reports by download or email) by going to the “Report History” tab at the top.
For the Master Reports, the tool allows you to select whatever mix of filtered attributes you want.
You can also use it for testing tabular reports (which you need to upload the file for).
