RunID: Run Identifiers

Purpose

A "run identifier", or RunID, provides a mechanism for tracking a sequence of service calls through distributed services across the VO. It is passed from one service to another and recorded into logs. An application that collates logs from many services in the VO and generates usage statistics can better estimate the number of people useing the VO by distinguishing between calls from user clients and calls from other services.

Summary of the RunID mechanism

A RunID is envisioned to be a unique string that is passed as an optional argument to a service. The receiving service does two things with the identifier:

• writes the RunID into a log that records when the service is called.
• passes the same RunID to any other services it in turn calls.

A RunID is first created by any service that has been called without a RunID as part of the input and which will eventually call other services.

Requirements

1. it should be easy for a service to pass a RunID to another service with minimal impact on the latter's interface
2. the RunID should be easily captured and recorded by a service into a log file with minimal impact on the implementation
3. it must be possible to use the RunID to group calls to multiple services that ultimately originated from a single user request. As a correllary, it should be possible to distinguish between diferent chains of service calls resulting from separate user requests.
4. it should be possible to use the RunID determine the service (to some level of specificity) that received the original user request that resulted in a chain of service-to-service calls.
5. any others?

Use Case

The Creation of a RunID

1. A service is called without a RunID being passed to it, indicating that the request is coming from a user client rather than another VO service.
2. The service determines that it must call another VO service and, therefore, must create a new RunID
3. The service creates a RunID string combining an identifier for itself and a random string and passes it to the other VO services it calls.

Passing a RunID to an HTTP/GET-based service

An HTTP/GET-based service is a traditional sort of service in which the inputs are given as arguments encoded in the access URL (i.e., after the '?'), usually as keyword-value pairs.

1. The calling service adds as an (optional) argument "RunID=run-id" to the URL.
2. The HTTP server hosting the called service automatically records the the URL used to access the service--including the RunID argument--into the server's access log.
3. The called HTTP/GET service implementation can choose to ignore the RunID argument if it doesn't need to call another service.
4. If the called service must in turn call a 3rd service, it passes the RunID it received on to that service.

Passing a RunID to a SOAP-based service

1. The calling service adds the RunID as an item in the header of the input SOAP message sent to the called service.
2. The Web Service implementation extracts the RunID (if it exists) from the SOAP header and records it to a log along with the date and the host address of the caller.
3. If the called service must in turn call a 3rd service, it passes the RunID it received on to that service.

Using RunIDs to generate usage statistics

1. Each VO service provider emits their access log via a standard logging interface. (Access to this logging service may be restricted.)
2. A usage statistics generation application harvests access logs from many VO service providers .
3. The applicaiton groups together calls from different service providers that have the same associated RunID.
4. For each service it has access data for, the application publishes the total number of calls to the service, the portion of those calls that orignated from a actual user request, and the portion that came from another service. For the latter portion, the application lists services that received the original user request and the number of requests associated with that originating service.

Recommendation for RunID syntax

This document recommends the following syntax for a RunID:

RunID = ivo-id "?" *unreserved

unreserved

any alphanumeric character, "-", "_", or "."

ivo-id

a legal IVOA Identifier

Example: ivo://adil.ncsa/targeted/service?aisd11dc82lsd8bdsg

The embedded IVOA identifier should be that of a resource that is registered with an IVOA compliant registry. In particular, it is recommended that the identifier point to one of the following resource types, in order of preference:

1. The service that created the RunID
2. The data collection or archive being accessed by the service that created the RunID
3. The organization that published and maintains the service that created the RunID.

Example Use with HTTP/GET serivce

Use With Traditional HTTP/GET Services

When a service calls an HTTP/GET service (where the service inputs are given as arguments to a URL), the RunID is passed as an additional argument of the following form:

argument = "RunID=" RunID

Example: http://adil.ncsa.edu/cgi-bin/voquery.pl?RA=180&DEC=30&SR=1&RunID=ivo://adil.ncsa/targeted/service?aisd11dc

Comments

• The IVOA identifier allows the sequence of service calls to be traced back to a person/organization responsible.

• The ? was chosen as the delimiter because the IVOA Identifier spec defines ? and # specifically as "stop characters"--characters that indicate the end of the legal IVOA identifier. # will not work as a delimiter; my experiments with apache showed that the access log chopped off everything past the # before it was recorded to the log.

• The above is fairly lenient on certain issues; note where "should" is used. This could be strengthened.

• The preference list for the IVOA identifer component allows some flexibility for services that are not registered (yet, e.g., because they are experimental).

-- RayPlante - 22 Dec 2004

Recommend text for Support Services Spec.

See section 2 in this variation on v0.24:

• VOSupportInterfaces-RunID.doc

-- RayPlante - 3 Feb 2006