mobile-menu mobile-menu-arrow Menu
 

Appendix E

Download Appendix E

Audit Requirements and Tests

Table of Contents

  1. General Audit Requirements
  1. The Required Audit Outputs
  2. The Required Audit Tests

 

1. General Auditing Requirements

 .

Audit Philosophy

The COUNTER audit procedures and tests set out in this Appendix seek to ensure that the usage reports provided by content providers are in line with the COUNTER Code of Practice and follow uniform agreed procedures. To this end, the COUNTER audit seeks to mirror the activity of an institution (a customer) carrying out usage on the content provider’s platform.

Third Party Hosts and Vendors

Two broad categories must be taken into account for usage statistics reporting, and each has additional audit requirements. These categories are:

 

No two third-party hosts/vendors are exactly alike. Prior to the audit each must discuss with COUNTER how they provide usage statistics reporting so that COUNTER can advise which of the two categories above applies to them.

Auditing and Test-Scripts

There are three stages in the COUNTER audit:

 

COUNTER defines specific audit test-scripts for each of the COUNTER-required usage reports. Because content providers may work with different auditors, the test-scripts help to ensure that each auditor follows a common auditing procedure.

 

The COUNTER auditor cannot express an opinion as to usage reported in respect of any other accounts or institutions, or as to aspects of the COUNTER Code of Practice, not specifically tested. Release 5-compliant content providers are reminded, however, that their signed Declaration of COUNTER compliance also covers these aspects of the COUNTER Code of Practice.

  1. Frequency of the Audit

To maintain COUNTER-compliant status an independent audit is required within 6 months of a content provider being listed in the Register of COUNTER Compliant Content Providers and annually thereafter. (Excepted are content providers that are members of COUNTER in the Smaller Publisher category, which may be audited biennially, with permission from COUNTER). Failure to meet these audit requirements will result in a content provider losing its COUNTER-compliant status.

 

If COUNTER does not receive a satisfactory auditor’s report within the specified time, the following control procedures apply:

 

New content providers having signed the Declaration of Compliance:

 

6 months after signing A reminder from COUNTER that the first auditor’s report is required
8 months after signing A final reminder from COUNTER that the first auditor’s report is required
9 months after signing The content provider is removed from the registry and is notified by COUNTER that they are non-compliant and must not make reference to COUNTER or use the COUNTER logo.

 

Content providers previously audited:

 

3 months following the due audit date A reminder from COUNTER that an auditor’s report is required
2 months following the due audit date A further reminder from COUNTER that an auditor’s report is required
5 months following the due audit date A final reminder from the Chair of the COUNTER Executive Committee that an auditor’s report is required
6 months following the due audit date The content provider is removed from the registry and is notified by COUNTER that they are non-compliant and must not make reference to COUNTER or use the COUNTER logo.

2. COUNTER Usage Reports for which an Independent Audit is Required

Independent audits are required for COUNTER reports according to host type(s). See Table 1 (below).

Table 1: COUNTER Reports Requiring Audit

 

Category Report ID (for SUSHI) R5 Report Name Master Report / Standard View Host Type
Platform PR Platform Master Report Master All
Platform PR_P1 Platform Usage Standard View All
Database DR Database Master Report Master – Aggregated Full Content

– A&I Database

– Discovery Service

– eBook Collections

– Multimedia Collection

Database DR_D1 Database Searches and Item Usage Standard View – Aggregated Full Content

– A&I Database

– Discovery Service

– eBook Collections

– Multimedia Collection

Database DR_D2 Database Access Denied Standard View – Aggregated Full Content

– A&I Database

– Discovery Service

– eBook Collections

– Multimedia Collection

Title TR Title Master Report Master – Aggregated Full Content

– eBooks

– eBook Collections

– eJournals

Title TR_B1 Book Requests (excluding OA_Gold) Standard View – Aggregated Full Content

– eBooks

– eBook Collections

Title TR_B2 Book Access Denied Standard View – eBooks

– eBook Collections

Title TR_B3 Book Usage by Access Type Standard View – Aggregated Full Content

– eBooks

– eBook Collections

Title TR_J1 Journal Requests (excluding OA_Gold) Standard View – Aggregated Full Content

– eJournals

Title TR_J2 Journal Access Denied Standard View – eJournals
Title TR_J3 Journal Usage by Access Type Standard View – Aggregated Full Content

– eJournals

Title TR_J4 Journal Requests by YOP (excluding OA_Gold) Standard View – Aggregated Full Content

– eJournals

Item IR Item Master Report Master – Data Repository

– Multimedia Collection

– Repository

– Scholarly Collaboration Network

Item IR_A1 Journal Article Requests Standard View – Repository
Item IR_M1 Multimedia Item Requests Standard View – Multimedia Collection


3. General Conditions for Carrying out an Audit Test

COUNTER defines a reporting period as a calendar month. A report run for any given month MUST reflect all activity of a customer for the entire month in question.

 

This applies also to auditing activity. An auditor should always finalize the audit tests within one and the same calendar month. During the audit period, all activity on the audit accounts not instigated by the auditor should be prevented, as this will make the test reports unreliable and may result in further audit tests that may incur additional costs.

 

To prevent any collision of reported data, an auditor should be allowed to set up and maintain separate accounts for each of the audit tests. All accounts should be set up in such a way that only the auditor carrying out a test can access the content provider’s platform.

 

Prior to the audit, the content provider must supply to the auditor:

  1. Account details for at least 4 separate accounts with access to all areas required to be tested (or specific restrictions for turn-away testing).
  2. Links to download usage reports in all required formats. COUNTER usage reports must be provided as tabular versions, which can be easily imported into Microsoft Excel pivot tables.
  3. SUSHI credentials for the test accounts to enable verification of SUSHI harvesting and formatting of the harvested reports.
  4. A declaration that federated and automated searches have been disaggregated from any searches reported. See the COUNTER Code of Practice for further information on the protocols that apply to federated and automated searches.
  5. If server-side caching is implemented, information on cache settings used should be provided. Note: Server-side caching can cause a discrepancy between the usage recorded in the audit tests and the usage reported by the content provider. Information on cache settings enables the auditor to take them into account when evaluating the results of the report tests. If the content provider does not provide this information, the auditor is likely to require further audit tests that may incur additional costs.

2. The Required Audit Outputs

If the auditor identifies one or more issues, the content provider MUST resolve them and pass the audit within 3 months to maintain COUNTER-compliant status. Please see section 9.2 in the COUNTER Code of Practice.

 

The auditor will provide to the COUNTER Executive Committee a summary report including, as a minimum, the following information:

  1. The name of the content provider
  2. The audit period and date
  3. The usage report(s) tested
  4. For each usage report tested, the test results, indicated as a % of the reported figures over the expected
  5. A summary of any material issues noted with the format/structure, data integrity, and/or delivery of the content provider’s reports. If there are no issues, a PASS should be noted.
  6. A clear indication of the outcome of the audit: PASS, QUALIFIED PASS, or FAIL.
  7. Any other comments that relate to the audit and are worthy of consideration by the COUNTER Executive Committee.

 

Sample Audit Report:

 

Content Provider <name>
Audit Period <mmm/yyyy> Date <mmm/yyyy>
 

Report

 

Usage Activity Result

Report Format  

Data Integrity

Delivery  

Opinion

 

Comments

Tabular SUSHI Reports Interface SUSHI

Server

TR_J1 100% PASS PASS PASS PASS PASS PASS
TR_B1 112% PASS REPORT TOTALS included PASS PASS PASS FAIL SUSHI versions of reports must not have totals.

 

A content provider may need to submit multiple audit reports, some of which may PASS and some of which may FAIL. The results of each report’s tests should be submitted on a separate line. For a content provider to maintain COUNTER-compliant status, each audited report must PASS.

 

3. The Required Audit Tests

 

Stage 1. Report Format: Checking the report layout and file-format against the COUNTER Code of Practice

The auditor will confirm that each of the audit reports complies with the COUNTER Code of Practice.

The following items will be checked:

 

Stage 2. Data Integrity: Checking the usage numbers as reported

The audit-test must be conducted in such a way that the auditor’s activities during the audit-test can be isolated from other activities on the content provider’s site. Depending on the site being tested, the auditor must conduct the audit-test from a computer with a unique IP address and/or using a unique account number.

The auditor must accept user/machine and session cookies when prompted.

Platform Reports

Master Report: PR

The Platform Master Report will be COUNTER-compliant if the following Standard View passes the COUNTER Audit and the figures reported within it are matching what is reported in the Master Report.

Standard View: PR_P1

Platform Usage: A Standard View of the Platform Master Report offering platform-level usage summarized by “Metric_Type”.

 

An audit of this Standard View requires the following:

  1. The auditor must have access to all databases as made available on the platform of the content provider.

 

  1. Audit-test P1-1: Searches_Platform

 

  1. Audit-test P1-2: Total_Item_Requests, Unique_Item_Requests and Unique_Title_Requests

 

 

  1. Audit-test P1-3: Total_Item_Requests and Unique_Item_Requests30-second filters

The auditor must carry out a total of 30 tests on the platform, and each test will consist of 2 requests. There are 2 types of tests that must be carried out:

The auditor must carry out 15 inside tests.
This must result in 15 Total_Item_Requests and 15 Unique_Item_Requests in the PR_P1 Standard View.
This may not be the case if the content provider operates a cache server.
The audit must carry out 15 outside tests.
This must result in 30 Total_Item_Requests and 15 Unique_Item_Requests in the PR_P1 Standard View.
This may not be the case if the content provider operates a cache server.

Audit tests P1-1, P1-2 and P1-3 must take place in separate accounts so that each audit-test can be separately reported.

Database Reports

Master Report: DR

The Database Master Report will be COUNTER-compliant if the following Standard Views pass the COUNTER audits and the figures reported within them match what is reported in the Master Report.

Any Standard View that is not applicable to the content provider does not require auditing. This must be agreed prior to the audit by COUNTER.

Standard View: DR_D1

Databases Searches and Item Usage: Reports on key search and request metrics needed to evaluate a database.

 

An audit of this Standard View requires the following:

  1. The auditor must have access to all databases available on the platform of the content provider.

 

  1. Audit-test D1-1: Searches_Regular and Searches_Automated

Option 2: The content provider has multiple databases, and it is possible to search over all databases or a single database.

The auditor must run 100 searches, including 50 against only 1 selected database and 50 against all databases (without actively choosing).

Each of these searches on a single database must report 1 Searches_Regular in the DR_D1 Standard View.

Each of these searches over all databases must report 1 Searches_Automated against each of the databases offered by the content provider in the DR_D1 Standard View.

Option 3: The content provider has a single database.

The auditor must run 50 searches against the 1 database. Each of these searches must report 1 Searches_Regular in the DR_D1 Standard View.

 

  1. Audit-test D1-2: Total_Item_Requests

 

  1. Audit-test D1-3: Total_Item_Requests 30-second filters

 

  1. Audit-test D1-4: Total_Item_Investigations
    IMPORTANT NOTE: This test does not need to be carried out where the content provider does not offer Investigations that are not also Requests. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

 

  1. Audit-test D1-5: Total_Item_Investigations 30-second filters
    IMPORTANT NOTE: This test does not need to be carried out where the Content provider does not offer Investigations that are not also Requests. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.
  1. If the content provider does not offer investigations that are not also requests, the following figure being reported as a result of the D3-1 and D3-2 audit tests must match in the DR_D1 Standard View: Total_Item_Requests must match Total_Item_Investigations

Audit tests D1-1, D1-2 and D1-3, D1-4 and DB1-5 must take place in separate accounts so that each audit test can be separately reported.

Standard View: DR_D2

Databases Access Denied: Reports on access-denied activity for databases where users were denied access because simultaneous-user licenses were exceeded, or their institution did not have a license for the database.

 

An audit of this Standard View requires the following:

  1. Audit-test D2-1: Limit_Exceeded
    IMPORTANT NOTE: This test cannot be carried out if the content provider does not offer a concurrent/simultaneous user limit. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

The auditor will log into the site. This means that the user limit is at maximum active users.

The auditor will then attempt to log into the site using a different computer. The auditor should then be refused access because of exceeding the concurrent/simultaneous-user limit. Each time access is refused, the auditor will record this as Limit_Exceeded.

The auditor must force 50 Limit_Exceeded turnaways during testing.

Each of these concurrent/simultaneous turnaways must report 1 Limit_Exceeded in the DR_D2 Standard View.

Option 2: The content provider turns the user away when the concurrent/simultaneous user limit is exceeded upon searching or accessing a database.

The auditor will log into the site. This means that the user limit is at maximum active users. The user will then select and make a search on a database (or browse to a database).

The auditor will then log into the site using a different computer. The auditor will then repeat the action made on the previous computer (select and make a search on a database or browse to a database). After the search has been made (or database browsed to) the user should then be refused access because of exceeding the concurrent/simultaneous-user limit. Each time access is refused, the auditor will record this as Limit_Exceeded.

The auditor must force 50 Limit_Exceeded turnaways during testing.

Each of these concurrent/simultaneous turnaways must report 1 Limit_Exceeded in the DR_D2 Standard View.

Option 3: The content provider turns the user away when the concurrent/simultaneous-user limit is exceeded upon accessing an Item within a database.

The auditor will log into the site. This means that the user limit is at maximum active users. The user will then navigate to and request an Item.

The auditor will then log into the site using a different computer. The auditor will then repeat the action made on the previous computer (navigate to and request an Item). After the Item has been requested the user should then be refused access because of exceeding the concurrent/simultaneous-user limit. Each time access is refused, the auditor will record this as Limit_Exceeded.

The auditor must force 50 Limit_Exceeded turnaways during testing.

Each of these concurrent/simultaneous turnaways must report 1 Limit_Exceeded in the DR_D2 Standard View.

 

 

 

 

  1. Audit-test D2-2: No_License

IMPORTANT NOTE: This test cannot be carried out if the content provider does not restrict site content or if restricted content is not displayed. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

 

Audit tests D2-1 and D2-2 must take place in separate accounts so that each audit test can be separately reported.

 

Title Reports

Master Report: TR

The Title Master Report will be COUNTER-compliant if the following Standard Views pass the COUNTER audits and the figures reported within them match what is reported in the Master Report.

Any Standard View that is not applicable to the content provider does not require auditing, this must be agreed prior to the audit by COUNTER.

Standard View: TR_B1

Book Requests (excluding OA_Gold): Reports on full-text activity for non-Gold open access books as Total_Item_Requests and Unique_Title_Requests. The Unique_Title_Requests view provides comparable usage across book platforms. The Total_Item_Requests view shows overall activity; however, numbers between sites will vary significantly based on how the content is delivered (e.g. delivered as a complete book or by chapter.)

An audit of this Standard View requires the following:

  1. The auditor must have access to all book content available by the content provider.
  1. The “Access_Type” for all requests must be Controlled and not OA_Gold.
  1. Audit-test B1-1: Total_Item_Requests and Unique_Title_Requests
  1. Audit-test B1-2: Total_Item_Requests and Unique_Title_Requests 30-second filters

The auditor must carry out 16 inside tests.

Where possible, each title must have 2 Item tests within it (reporting 1 Total_Item_Requests and 1 Unique_Title_Requests).

This must result in 16 Total_Item_Requests and 8 Unique_Title_Requests in the TR_B1 Standard View.

This may not be the case if the content provider operates a cache server.

The audit must carry out 16 outside tests.

Where possible, each title must have 2 Items requested within it (reporting 2 Total_Item_Requests and 1 Unique_Title_Requests).

This must result in 30 Total_Item_Requests and 8 Unique_Title_Requests in the TR_B1 Standard View.

This may not be the case if the content provider operates a cache server.

Audit tests B1-1 and B1-2 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_B2

Book Access Denied: Reports on access denied activity for books where users were denied access because simultaneous-user licenses were exceeded, or their institution did not have a license for the book.

An audit of this Standard View requires the following:

  1. Audit-test B2-1: Limit_Exceeded

IMPORTANT NOTE: This test cannot be carried out if the content provider does not offer a concurrent/simultaneous user limit. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

The auditor will log into the site and access a book item, this means that the user limit is at maximum active users.

The auditor will then log into the site using a different computer. The auditor will then repeat the action made on the previous computer (access a book item). After the item has been requested the user should then be refused access because of exceeding the concurrent/simultaneous user limit. Each time access is refused, the auditor will record this as Limit_Exceeded.

The auditor must force 50 Limit_Exceeded turnaways during testing.

Each of these concurrent/simultaneous turnaways must report 1 Limit_Exceeded in the TR_B2 Standard View.

 

  1. Audit-test B2-2: No_License
    IMPORTANT NOTE: This test cannot be carried out if the content provider does not restrict site content or where restricted content is not displayed. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

The auditor must force 50 No_License during testing.

Each of these Book content not licensed turnaways must report 1No_License in the TR_B2 Standard View.

Audit tests B2-1 and B2-2 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_B3

Book Usage by Access Type: Reports on book usage showing all applicable metric types broken down by “Access_Type”

An audit of this Standard View requires the following:

  1. The auditor must have access to all book content available by the content provider.

 

  1. Audit-test B3-1: Total_Item_Requests, Unique_Item_Requests and Unique_Title_Requests

The auditor must make a total of 100 requests on a subset of unique Items within book titles (50 requests to book Items where the “Access_Type” is Controlled, and 50 requests to book items where the “Access_Type” is OA_Gold).

Each title must have 5 items requested within it (reporting 5 Total_Item_Requests, 5 Unique_Item_Requests and 1 Unique_Title_Requests).

This must result in 50 OA_Gold Total_Item_Requests and 50 Controlled Total_Item_Requests being reported in the TR_B3 Standard View.

This must result in 50 OA_Gold Unique_Item_Requests and 50 Controlled Unique_Item_Requests being reported in the TR_B3 Standard View.

This must result in 10 OA_Gold Unique_Title_Requests and 10 Controlled Unique_Title_Requests being reported in the TR_B3 Standard View.

Option 2: Content provider does not offer OA_Gold items.

The auditor must make a total of 100 requests on a subset of unique Items within book titles.

Where possible, each title must have 5 items requested within it (reporting 5 Total_Item_Requests, 5 Unique_Item_Requests, and 1 Unique_Title_Requests).

This must result in 100 Controlled Total_Item_Requests being reported in the TR_B3 Standard View.

This must result in 100 Controlled Unique_Item_Requests being reported in the TR_B3 Standard View.

This must result in 20 Controlled Unique_Title_Requests being reported in the TR_B3 Standard View.

 

 

 

 

 

The auditor must carry out 16 inside tests (8 tests to book items where the Access_Type is Controlled and 8 tests to book items where the Access_Type is OA_Gold).

Where possible, each title must have 2 book item tests within it (reporting 2 Total_Item_Requests, 2 Unique_Item_Requests, and 1 Unique_Title_Requests).

This must result in 8 Controlled Total_Item_Requests and 8 OA_Gold Total_Item_Requests in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Item_Requests and 8 OA_Gold Unique_Item_Requests in the TR_B3 Standard View.

This must result in 4 Controlled Unique_Title_Requests and 4 OA_Gold Unique_Title_Requests in the TR_B3 Standard View.

(This may not be the case if the content provider operates a cache server.)

The auditor must carry out 16 outside tests (8 tests to book items where the Access_Type is Controlled and 8 tests to book items where the Access_Type is OA_Gold).

Where possible, each title must have 2 book item tests within it (reporting 4 Total_Item_Requests, 2 Unique_Item_Requests, and 1 Unique_Title_Requests).

This must result in 16 Controlled Total_Item_Requests and 16 OA_Gold Total_Item_Requests in the TR_B3 Standard View.

This must result in 8 Controlled Total_Item_Requests 8 OA_Gold Unique_Item_Requests in the TR_B3 Standard View.

This must result in 4 Controlled Unique_Title_Requests and 4 OA_Gold Unique_Title_Requests in the TR_B3 Standard View.

(This may not be the case if the content provider operates a cache server.)

Option 2: Content provider does not offer OA_Gold Items.

The auditor must carry out a total of 32 tests, and each test will consist of 2 requests. There are 2 types of tests that must be carried out:

The auditor must carry out 16 inside tests.

Where possible, each title must have 2 book item tests within it (reporting 2 Total_Item_Requests and 2 Unique_Item_Requests and 1 Unique_Title_Requests).

This must result in 16 Controlled Total_Item_Requests in the TR_B3 Standard View.

This must result in 16 Controlled Unique_Item_Requests in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Title_Requests in the TR_B3 Standard View.

(This may not be the case if the Content provider operates a cache server.)

The auditor must carry out 16 outside tests.

Each title must have 2 book item tests within it (reporting 4 Total_Item_Requests, 2 Unique_Item_Requests, and 1 Unique_Title_Requests).

This must result in 32 Controlled Total_Item_Requests in the TR_B3 Standard View.

This must result in 16 Controlled Unique_Item_Requests in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Title_Requests in the TR_B3 Standard View.

(This may not be the case if the content provider operates a cache server.)

  1. Audit-test B3-3: Total_Item_Investigations, Unique_Item_Investigations, and Unique_Title_Investigations
    IMPORTANT NOTE: This test does not need to be carried out if the content provider does not offer investigations that are not also requests. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

The auditor must make a total of 50 item investigations within a subset of book titles (25 Investigations of items within a book where the Access_Type is Controlled, and 25 investigations of items within a book where the Access_Type is OA_Gold).

Each title must have 5 investigations to unique Items within it (reporting 5 Total_Item_Investigations, 5 Unique_Item_Investigations, and 1 Unique_Title_Investigations).

This must result in 25 OA_Gold Total_Item_Investigations and 25 Controlled Total_Item_Investigations being reported in the TR_B3 Standard View.

This must result in 25 OA_Gold Unique_Item_Investigations and 25 Controlled Unique_Item_Investigations being reported in the TR_B3 Standard View.

This must result in 5 OA_Gold Unique_Title_Investigations and 5 Controlled Unique_Title_Investigations being reported in the TR_B3 Standard View.

Option 2: Content provider does not offer OA_Gold Items.

The auditor must make a total of 50 Investigations within a subset of book titles.

Each title must have 5 investigations to unique items within it (reporting 5 Total_Item_Investigations, 5 Unique_Item_Investigations, and 1 “Unique_Title_Investigations”).

This must result in 50 Controlled Total_Item_Investigations being reported in the TR_B3 Standard View.

This must result in 50 Controlled Unique_Item_Investigations being reported in the TR_B3 Standard View.

This must result in 10 Controlled Unique_Title_Investigations being reported in the TR_B3 Standard View.

 

  1. Audit test B3-4: Total_Item_Investigations, Unique_Item_Investigations, and Unique_Title_Investigations 30-second filters
    IMPORTANT NOTE: This test does not need to be carried out if the content provider does not offer investigations that are not also requests. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

 

 

The auditor must carry out 16 inside tests (8 Investigations to book items where the “Access_Type” is Controlled and 8 investigations to book items where the “Access_Type” is OA_Gold).

Each title must have 2 book item tests within it (reporting 2 Total_Item_Investigations, 2 Unique_Item_Investigations, and 1 Unique_Item_Investigations).

This must result in 8 Controlled Total_Item_Investigations and 8 OA_Gold Total_Item_Investigations in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Item_Investigations and 8 OA_Gold Unique_Item_Investigations in the TR_B3 Standard View.

This must result in 4 Controlled Unique_Title_Investigations and 4 OA_Gold Unique_Title_Investigations in the TR_B3 Standard View.

This may not be the case if the content provider operates a cache server.

The auditor must carry out 16 outside tests (8 tests to book items where the “Access_Type” is Controlled and 8 tests to book items where the “Access_Type” is OA_Gold).

Each title must have 2 book item tests within it (reporting 4 Total_Item_Investigations, 2 Unique_Item_Investigations, and 1 Unique_Title_Investigations).

This must result in 16 Controlled Total_Item_Investigations and 16 OA_Gold Total_Item_Investigations in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Item_Investigations and 8 OA_Gold Unique_Item_Investigations in the TR_B3 Standard View.

This must result in 4 Controlled Unique_Title_Investigations and 4 OA_Gold Unique_Title_Investigations in the TR_B3 Standard View.

This may not be the case if the content provider operates a cache server.

Option 2: Content provider does not offer OA_Gold items.

The auditor must carry out a total of 32 tests, and each test will consist of 2 item investigations. There are 2 types of tests that must be carried out:

The auditor must carry out 16 inside tests.

Each title must have 2 book item tests within it (reporting 2 Total_Item_Investigations, 2 Unique_Item_Investigations, and 1 Unique_Title_Investigations).

This must result in 16 Controlled Total_Item_Investigations in the TR_B3 Standard View.

This must result in 16 Controlled Unique_Item_Investigations in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Title_Investigations in the TR_B3 Standard View.

This may not be the case if the content provider operates a cache server.

The auditor must carry out 16 outside tests.

Each title must have 2 book item tests within it (reporting 4 Total_Item_Investigations, 2 Unique_Item_Investigations, and 1 Unique_Title_Investigations).

This must result in 32 Controlled Total_Item_Investigations in the TR_B3 Standard View.

This must result in 16 Controlled Unique_Item_Investigations in the TR_B3 Standard View.

This must result in 8 Controlled Unique_Title_Investigations in the TR_B3 Standard View.

This may not be the case if the content provider operates a cache server.

 

 

 

  1. If the content provider does not offer Investigations that are not also requests, the following figure being reported as a result of the B3-1 and B3-2 audit tests must match in the TR_B3 Standard View:
    Total_Item_Requests must match Total_Item_Investigations
    Unique_Item_Requests must match Unique_Item_Investigations
    Unique_Title_Requests must match Unique_Title_Investigations

Audit tests B3-1, B3-2, B3-3, and B3-4 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_J1

Journal Requests (excluding OA_Gold): Reports on usage of non-Gold open access journal content as Total_Item_Requests and Unique_Item_Requests. The Unique_Item_Requests provides comparable usage across journal platform by reducing the inflationary effect that occurs when an HTML full text automatically displays and the user then accesses the PDF version. The Total_Item_Requests shows overall activity.

An audit of this Standard View requires the following:

  1. The auditor must have access to all journal content available by the content provider.

 

  1. The “Access_Type” for all requests must be Controlled and not OA_Gold.

 

  1. Audit-test J1-1: Total_Item_Requests and Unique_Item_Requests
  1. Audit-test J1-2: Total_Item_Requests and Unique_Item_Requests30-second filters

The auditor must carry out 15 inside tests.
This must result in 15 Total_Item_Requests and 15 Unique_Item_Requests in the TR_J1 Standard View.
This may not be the case if the content provider operates a cache server.
The audit must carry out 15 outside tests.
This must result in 30 Total_Item_Requests and 15 Unique_Item_Requests in the TR_J1 Standard View.
This may not be the case if the content provider operates a cache server.

Audit tests J1-1 and J1-2 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_J2

Journal Accessed Denied: Reports on Access Denied activity for journal content where users were denied access because simultaneous-user licenses were exceeded, or their institution did not have a license for the title.

An audit of this Standard View requires the following:

  1. Audit-test J2-1: Limit_Exceeded

IMPORTANT NOTE: This test cannot be carried out where the content provider does not offer a concurrent/simultaneous-user limit. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

 

  1. Audit-test J2-2: No_License

IMPORTANT NOTE: This test cannot be carried out if the content provider does not restrict site content or where restricted content is not displayed. This must be declared to the auditor and the COUNTER Executive Committee prior to testing.

Audit tests J2-1 and J2-2 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_J3

Journal Usage by Access Type: Reports on usage of journal content for all metric types broken down by access type.

An audit of this Standard View requires the following:

  1. The auditor must have access to all journal content available by the content provider.
  1. Audit-test J3-1: Total_Item_Requests and Unique_Item_Requests
  1. Audit-test J3-2: Total_Item_Requests and Unique_Item_Requests30-second filters

The auditor must carry out 15 inside tests (8 tests to journal items where the “Access_Type” is Controlled and 7 tests to journal items where the “Access_Type” is OA_Gold).
This must result in 8 Controlled Total_Item_Requests and 7 OA_Gold Total_Item_Requests in the TR_J3 Standard View.
This must result in 8 Controlled Total_Item_Requests 7 OA_Gold Unique_Item_Requests in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
The auditor must carry out 15 outside tests (8 tests to journal items where the “Access_Type” is Controlled and  7 tests to journal items where the “Access_Type” is OA_Gold).
This must result in 16 Controlled Total_Item_Requests and 14 OA_Gold Total_Item_Requests in the TR_J3 Standard View.
This must result in 8 Controlled Total_Item_Requests 7 OA_Gold Unique_Item_Requests in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
Option 2:Content provider does not offer OA_Gold Items.
The auditor must carry out a total of 30 tests, and each test will consist of 2 requests. There are 2 types of tests that must be carried out:

The auditor must carry out 15 inside tests.
This must result in 15 Controlled Total_Item_Requests in the TR_J3 Standard View.
This must result in 15 Controlled Unique_Item_Requests  in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
The auditor must carry out 15 outside tests.
This must result in 30 Controlled Total_Item_Requests in the TR_J3 Standard View.
This must result in 15 Controlled Unique_Item_Requests in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.

 

  1. Audit-test J3-3: Total_Item_Investigations and Unique_Item_Investigations

A content provider will pass this audit test when the sum of the Total_Item_Investigations and Unique_Item_Investigations reported by the content provider in TR_J3 Standard View for the auditor’s test account is within a -8% and +3% reliability window of the sum of the Total_Item_Investigations and Unique_Item_Investigations on the auditor’s report

  1. Audit test J3-4: Total_Item_Investigations and Unique_Item_Investigations 30-second filters

The auditor must carry out 15 inside tests (8 tests to journal items where the “Access_Type” is Controlled and  7 tests to journal items where the “Access_Type” is OA_Gold).
This must result in 8 Controlled Total_Item_Investigations  and 7 OA_Gold Total_Item_Investigations in the TR_J3 Standard View.
This must result in 8 Controlled Unique_Item_Investigations  and 7 OA_Gold Unique_Item_Investigations in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
The auditor must carry out 15 outside tests (8 tests to journal items where the “Access_Type” is Controlled and  7 tests to journal items where the “Access_Type” is OA_Gold).
This must result in 16 Controlled Total_Item_Investigations  and 14 OA_Gold Total_Item_Investigations in the TR_J3 Standard View.
This must result in 8 Controlled Unique_Item_Investigations  and 7 OA_Gold Unique_Item_Investigations in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
Option 2:Content provider does not offer OA_Gold Items.
The auditor must carry out a total of 30 tests, and each test will consist of 2 Investigations. There are 2 types of tests that must be carried out:

The auditor must carry out 15 inside tests.
This must result in 15 Controlled Total_Item_Investigations  in the TR_J3 Standard View.
This must result in 15 Controlled Unique_Item_Investigations  in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.
The auditor must carry out 15 outside tests.
This must result in 30 Controlled Total_Item_Investigations in the TR_J3 Standard View.
This must result in 15 Controlled Unique_Item_Investigations in the TR_J3 Standard View.
This may not be the case if the content provider operates a cache server.

Audit tests J3-1, J3-2, J3-3, and J3-4 must take place in separate accounts so that each audit test can be separately reported.

Standard View: TR_J4

Journal Requests by “YOP” (excluding OA_Gold): Breaks down the usage of non-Gold pen Access journal content by year of publication (“YOP”) providing counts for the metric types Total_Item_Requests and Unique_Item_Requests. Provides the details necessary to analyze usage of content in backfiles or covered by perpetual access agreement. Note: COUNTER reports do not provide access model or perpetual access rights details.

An audit of this Standard View requires the following:

  1. The auditor must have access to all journal content available by the content provider.
  1. The “Access_Type” for all requests must be Controlled and not OA_Gold.
  1. The auditor must record the Year of Publication (“YOP”) of every item accessed during audit testing.
  1. The auditor must ensure that some full-text articles from different years of the same journal are requested during the J4-1 and J4-2 tests. Hence, the auditor should know the numbers expected to appear against each Year of Publication (“YOP”) in the TR_J4 report.
  1. Audit-test J4-1: Total_Item_Requests and Unique_Item_Requests

The auditor must confirm the Year of Publication (“YOP”) of articles covered in J4-1 with appropriate and proportionate spot checks, unless the article is “YOP unknown”.

  1. Audit-test J4-2: Total_Item_Requests and Unique_Item_Requests30-second filters

The auditor must carry out 15 inside tests.
This must result in 15 Total_Item_Requests and 15 Unique_Item_Requests in the TR_J4 Standard View.
This may not be the case if the content provider operates a cache server.
The audit must carry out 15 outside tests.
This must result in 30 Total_Item_Requests and 15 Unique_Item_Requests in the TR_J4 Standard View.
This may not be the case if the content provider operates a cache server.

The auditor must confirm the Year of Publication (YOP) of articles covered in J4-2 with appropriate and proportionate spot checks, unless the article is “YOP unknown”.
Audit tests J4-1 and J4-2 must take place in separate accounts so that each audit test can be separately reported.

Item Reports

Master Report: IR

The Item Master Report will be COUNTER compliant if the following Standard Views pass the COUNTER audits and the figures reported within them match what is reported in the Master Report.

Any Standard View that is not applicable to the content provider does not require auditing. This must be agreed prior to the audit by COUNTER.

Standard View: IR_A1

Reports on journal article requests at the article level. This report is limited to content with a “Data_Type” of Journal, “Section_Type” of article, and metric types of Total_Item_Requests.

 

An audit of this Standard View requires the following:

The auditor must have access to all journal article content available by the content provider.

 

  1. Audit-test A1-1: Total_Item_Requests
  1. Audit-test A1-2: Total_Item_Requests 30-second filters

The auditor must carry out 15 inside tests.
This must result in 15 Total_Item_Requests in the IR_A1 Standard View.
This may not be the case if the content provider operates a cache server.
The audit must carry out 15 outside tests.
This must result in 30 Total_Item_Requests in the IR_A1 Standard View.
This may not be the case if the content provider operates a cache server.

Audit tests A1-1 and A1-2 must take place in separate accounts so that each audit test can be separately reported.

Standard View: IR_M1

Reports on multimedia requests at the item level.

An audit of this Standard View requires the following:

  1. The auditor must have access to all multimedia content available by the content provider.

 

  1. Audit-test M1-1: Total_Item_Requests
  1. Audit-test M1-2: Total_Item_Requests 30-second filters

The auditor must carry out 15 inside tests.
This must result in 15 Total_Item_Requests in the IR_M1 Standard View.
This may not be the case if the content provider operates a cache server.
The audit must carry out 15 outside tests.
This must result in 30 Total_Item_Requests in the IR_M1 Standard View.
This may not be the case if the content provider operates a cache server.

Audit tests M1-1 and M1-2 must take place in separate accounts so that each audit test can be separately reported.

Stage 3. Report Delivery: Checking delivery of the reports

In addition to verifying the delivery of reports in a tabular format, the auditor will check that the COUNTER reports are downloadable using the SUSHI protocol. This may be tested using the COUNTER Report Validation Tool, an open-source tool that provides a series of web-forms and guidance to take users through the steps and parameters needed to connect successfully to SUSHI servers and download content provider reports. The COUNTER Report Validation Tool may be found at: http://validate.projectcounter.org.

A content provider will only pass an audit test if the JSON-formatted report produced via SUSHI matches the total of the relevant usage counted on the equivalent tabular report offered by the content provider. In other words, a report should produce the same results irrespective of the format in which it is delivered.

 

 

 

 
Release 5 Queries COP Register Members Guides Members

Gold and Silver Sponsors