The COUNTER Code of Practice for Release 4

Data Processing

Usage data collected by vendors/intermediaries for the usage reports to be sent to customers should meet the basic requirement that only intended usage is recorded and that all requests that are not intended by the user A person who accesses the online resourceare removed.

Because the way usage records are generated can differ across platforms, it is impractical to describe all the possible filters Limits or restrictions placed on the usage to be included in a COUNTER report usually expressed as a name-value pair, i.e. “Access_Type=Controlled”used to clean up the data. This Code of Practice, therefore, specifies only the requirements to be met by the data to be used for building the usage reports.

Usage data can be generated in a number of ways and COUNTER does not prescribe which approach should be taken. The two most common approaches are: logfile analysis, which reads the logfles containing the web server records all of its transactions; and page tagging, which uses JavaScript on each page to notify a third party server when a page is rendered by a web browser. Each of these two approaches has advantages and disadvantages, summarised below:

Advantages of logfile analysis

The main advantages of logfile analysis Logfile analysis is a method of collecting usage data in which the web server records all of its transactionsover page tagging “Page Tagging” is a method of collecting usage data which uses code, for example, JavaScript on each page to notify a third-party server when a page is rendered by a web-browser.are:

• The web server normally already produces logfiles, so the raw data are already available. No changes to the website are required.
• The data is on the organization’s own servers and is in a standard, rather than a proprietory, format. This makes it easy for an organization to switch programmes later, use several different programmes, and anlyse historical data with a new programme
• Logfiles contain information on visits from search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engine spiders. Although these should not be reported as part of user A person who accesses the online resourceactivity, it is useful information for search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engine optimization.
• Logfiles require no additional DNS Lookups. Thus there are no external server calls which can slow page load speeds, or result in uncounted page views.
• The web server reliably records every transaction A usage event.it makes, including, e.g., serving PDF Portable Document Format, file formatted for the Adobe Acrobat reader. Items such as full-text articles or journals published in PDF format tend to replicate the printed page in appearancedocuments and content generated by scripts, and does not rely on the visitor’s browser co-operating

Advantages of page tagging

The main advantages of page tagging “Page Tagging” is a method of collecting usage data which uses code, for example, JavaScript on each page to notify a third-party server when a page is rendered by a web-browser.over logfile analysis Logfile analysis is a method of collecting usage data in which the web server records all of its transactionsare:

• Counting is activated by opening the page, not requesting it from the server. If a page is cached it will not be counted by the server. Cached pages can account for a significant proprtion of pageviews
• Data is gathered via a compenent ( ‘tag’) in the page, usually written in JavaScript; though Java can be used and increasingly Flash is used. JQuery and AJAX can also be used in conjunction with a server-side scripting language ( such as PHP) to manipulate and store it in a database, allowing complete control over how the data is represented
• The script may have access to additional information on the web client or on the user, not sent in the query
• Page tagging can report on events that do not involve a request A category of COUNTER “Metric Type” that represent a user A person who accesses the online resourceaccessing content (i.e. full text of an “Article”)to the web server
• Page tagging is available to companies who do not have access to their own web servers
• The page tagging “Page Tagging” is a method of collecting usage data which uses code, for example, JavaScript on each page to notify a third-party server when a page is rendered by a web-browser.service manages the process of assigning cookies to visitors; with logfile anlaysis the server has to be configured to do this
• Recently page tagging “Page Tagging” is a method of collecting usage data which uses code, for example, JavaScript on each page to notify a third-party server when a page is rendered by a web-browser.has become a standard in web analytics
• Logfile analysis is almost always performed in-house. Page tagging can be done in house, but is more often provided as a third-party service. The cost differences between these two models can also be a consideration.

Return codes and time filters

1. Only succesful and valid requests should be counted. For web server logs sucessful requests are those with specific NCSA return codes. (200 and 304). The standards for return codes are defined and maintained by NCSA. In case key events are used their definition should match the NCSA standards.(For more information see Appendix D: Guidelines for Implementation.)
2. Records generated by the server together with the requested page (e.g. images, gif’s, style
   sheets (.css)) should be ignored.
3. All users’ double-clicks on an http-link should be counted as only 1 request. The time
   window for occurrence of a double-click Repeated click on the same link by the same user A person who accesses the online resourcewithin a period of 30 seconds. COUNTER requires that that double-clicks that occur in an interval of 30 seconds or less must be removed.should be set at 10 seconds between the first and
   the second mouse-click.

There are a number of options to make sure that a double click comes from one and the same user:

1.  Where only the IP address Internet Protocol (IP) address of the computer on which the session is conducted and may be used by content providers as a means of authentication and authorization and for identifying the institution a user is affiliated with. The identifying network address (typically four 8-bit numbers, aaa.bbb.cc.dd) of the user's computer or proxy.of a user A person who accesses the online resourceis logged that IP should be taken as the field to trace double-clicks
2. When a session-cookie is implemented and logged, the session-cookie should be used to trace the double-clicks.
3. When user-cookies are available and logged, the user-cookie should be used to trace double-clicks
4. When the username of a registered user A person who accesses the online resourceis logged, this username should be used to trace double-clicks.

The options 1 to 4 above have an increasing level of reliability for filtering out double-clicks: option 1 has the lowest level of precision (and may lead to under reporting from the vendor See also “Content Provider”. A publisher An organization whose function is to commission, create, collect, validate, host, distribute and trade information online and/or in printed formor other online information provider who delivers licensed content to the customer An individual or organization that can access a specified range of the “Content provider’s” services and/or content and is subject to terms and conditions agreed with the “Content provider”and with whom the customer An individual or organization that can access a specified range of the “Content provider’s” services and/or content and is subject to terms and conditions agreed with the “Content provider”has a contractual relationship.perspective) while with option 4 the result will be optimal.

The downloading and rendering of a PDF, image, video clip or audio clip may take longer than the rendering of an HTML Article formatted in HTML so as to be readable by a web browser Hypertext Markup Language. A form of text markup readable by web browserspage. Therefore requests by one and the same IP/username/session- or user A person who accesses the online resourcecookie for one and the same PDF, image, video clip or audio clip should be counted as a single request A category of COUNTER “Metric Type” that represent a user A person who accesses the online resourceaccessing content (i.e. full text of an “Article”)if these multiple requests occur within a 30 seconds time window. These multiple requests may also be triggered by pressing a refresh or back button on the desktop by the user.

When two requests are made for one and the same article An item of original written work published in a “Journal”, other serial publication, or in a “Book”. An “Article” is complete, but usually cites other relevant published works in its list of “References”, if it has one. A COUNTER “Data_Type” used to identify and filter usage of content defined as “Articles”.within the above time limits (10
seconds for HTML, 30 seconds for PDF), the first request A category of COUNTER “Metric Type” that represent a user A person who accesses the online resourceaccessing content (i.e. full text of an “Article”)should be removed and the
second retained. Any additional requests for the same article An item of original written work published in a “Journal”, other serial publication, or in a “Book”. An “Article” is complete, but usually cites other relevant published works in its list of “References”, if it has one. A COUNTER “Data_Type” used to identify and filter usage of content defined as “Articles”.within these time limits should
be treated identically: always remove the first and retain the second. (For further
information on the implementation of this protocol, see Appendix D: Guidelines for
Implementation)

Correcting for the effects of federated searches and internet robots on usage statistics

The growing use of federated searches and the spread of internet robots have the potential to inflate enormously the usage statistics reported in the COUNTER reports. Without some control these activities could result in significant over-counting.

COUNTER protocols have been developed to mitigate the inflationary effects of federated searches, internet robots and search-engine prefetching on the reported usage statistics. COUNTER-compliant Vendors are required to implement these protocols, itemised below.

Protocol for federated searches and automated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.agents
Search activity generated by federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engines and automated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.agents should be categorized separately from regular A COUNTER “Access_Method”. Indicates that usage was generated by a human user A person who accesses the online resourcebrowsing/searching a website, rather than by “Text and Data Mining” (“TDM”) processes.searches. Any searches generated from such systems should be included in separate “Searches-federated and automated” counts within Database Report 1 and Platform An interface from a “Content Provider” that delivers the content to the user A person who accesses the online resourceand that counts and provides the COUNTER usage reports.Report 1, and are not to be included in the “Regular Searches” counts in these reports. (See example Database Report 1 and Platform An interface from a “Content Provider” that delivers the content to the user A person who accesses the online resourceand that counts and provides the COUNTER usage reports.Report 1 in Section The first level of subdivision of a “Book” or “Reference Work”.4.1.2 above).
‘Federated Searches’ and ‘Automated Searches’ covered by this protocol are defined in Appendix A.
Federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engines may utilize a variety of techniques to conduct a search, including Z39.50; standard or proprietary XML gateways or APIs; or, by screen-scraping the standard HTML Article formatted in HTML so as to be readable by a web browser Hypertext Markup Language. A form of text markup readable by web browsersinterface. Federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.activity must be recognized regardless of the method of search.  Following are some examples of how search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.activity can be recognized – the content provider An organization whose function is to commission, create, collect, validate, host, distribute and trade information in electronic form and in doing so provides usage data. This includes organisations sometimes referred to as “Publishers”, “Vendors”, “Platforms” and/or “Intermediaries”.may wish to employ one or more of these techniques.

• The Federated Search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engine may be using its own IP address. This IP can be identified and used for segregation of activity.
• If the standard HTML Article formatted in HTML so as to be readable by a web browser Hypertext Markup Language. A form of text markup readable by web browsersinterface is being used, the browser ID within the web logs can be used to identify the activity as coming from a federated search.
• For Z39.50 An international standard protocol created by “NISO” for search.  A Z39.50 client can search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.any Z39.50 compatible online service.  Often used by Federated Search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.services to facilitate searching content at other sites.activity, access is generally achieved through username/password. Create a unique username/password that just the federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engine will use.
• If an API or XML gateway An intermediary online service which does not 4 ECO typically host the items requested by the user. The gateway will either refer the user to another site or service to download the item, or will request the item from another site or service and delivers it to the user within its own gateway environment. Items may be cachedis available, set up an instance of the gateway An intermediary online service which does not 4 ECO typically host the items requested by the user. The gateway will either refer the user to another site or service to download the item, or will request the item from another site or service and delivers it to the user within its own gateway environment. Items may be cachedthat is for the exclusive use of such search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.tools.
• If an API or XML gateway An intermediary online service which does not 4 ECO typically host the items requested by the user. The gateway will either refer the user to another site or service to download the item, or will request the item from another site or service and delivers it to the user within its own gateway environment. Items may be cachedis available, require the federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.to include an identifying parameter when making requests to the gateway.

A list of federated search A user-driven intellectual query, typically equated to submitting the “Search” form of the online service A branded group of online information products from one or more vendors that can be subscribed to/licensed and searched as a complete service, or at a lower level (e.g. a collection)to the server.engines covered by the above protocol is included in Appendix I. This list, which will be updated from time-to-time, should be regarded as the minimum requirement for COUNTER compliant vendors.

Protocol for internet robots and crawlers

Activity generated by internet robots and crawlers must be excluded from all COUNTER usage reports. A list of internet robots that must be excluded is provided in Appendix J. This list, which will be updated from time-to-time, should be regarded as the minimum requirement for COUNTER compliant vendors.

Protocol for tools that enable the bulk downloading of full-text articles and other content items

Only genuine, user-driven usage should be reported. Usage of full-text articles that is initiated by automatic or semi-automatic bulk download A single event where multiple content items are downloaded to the user’s computer.tools, such as Quosa or Pubget) should only be recorded when the user A person who accesses the online resourcehas clicked on the downloaded full-text article The complete text, including all references, figures and tables, of an article, plus links to any supplementary material published with itin order to open it.

Retrospective reporting of errors in usage data

Where vendors discover (or the independent audit reveals) errors in the usage statistics they have been providing in the COUNTER reports, such errors must be corrected within 3 months of their discovery and customers informed of the corrections.

Reporting of usage statistics when journal titles change

When the title of a journal is modified or changed, usage statistics for that journal prior to the title change should be reported against the new title, provided the Journal DOI/ISSN is unchanged, with the original title being dropped from the list. Where a new Journal DOI/ISSN is allocated to the new title, the usage statistics should be reported separately (separate rows), and those for the original title should continue to be reported against the original Journal DOI/ISSN.

Identifying abnormal spikes in usage

What is regarded as an abnormal spike in usage can vary from one institution The organization for which usage is being reported.to another; there are many occasions in which exceptionally high usage in a particular month is genuine. For these reasons COUNTER does not provide a strict protocol for dealing with spikes in usage that must be applied in all situations. Instead COUNTER suggests approaches that have been well-tested and which should prove useful in flagging abnormal spikes in usage that may warrant further investigation. These approaches are described in Appendix D: Guidelines for Implementation. COUNTER does not prescribe a course of action once abnormal spikes in usage have been identified; this is left to the discretion of customer An individual or organization that can access a specified range of the “Content provider’s” services and/or content and is subject to terms and conditions agreed with the “Content provider”and vendor.