How are WebMarshal browsing times calculated?

This article applies to:

• WebMarshal 6.X and 7.X

Question:

• How are WebMarshal session and domain browsing times calculated?
• Why do WebMarshal time calculations not add up?
• Reported total session time is much less or much more than the total of domain view times

Information:

Calculation of browsing time is complex.

For each web site visited the browser will issue many page requests to obtain the necessary style sheets, graphics and text. Often these requests are not to the original domain. A good example is pages which contain advertising, or links to a third party web analytics tool such as Google Analytics.

WebMarshal only sees the requests for pages and cannot determine the actual time spent reviewing a page once it is downloaded. To calculate browsing time for the Active Sessions view and reports, WebMarshal uses a heuristic estimate.

• For users actively browsing to many domains with image content, using multiple tabs or windows, the total of domain or page view times is likely to be much higher than the total session time.
• For users who leave a webmail client or other application open in the background with automatic refreshing, the total of domain or page view times is likely to be much lower than the total session time.

Definitions:

• An individual browsing request is called a page view.
• A set of browsing requests within a defined time is regarded as a continuous period of browsing, called a session.
• A set of requests for items from a single domain is called a visit.

Notes:

• WebMarshal records a separate domain visit for each protocol used. For instance, if a user visits a domain by HTTP and HTTPS, WebMarshal records two separate visits.
• In WebMarshal 6.5.5 and above, using an application protocol (such as YouTube Video) counts as an additional domain visit.

WebMarshal  allows you to set two values:

• The session timeout defaults to 5 minutes (minimum 2 minutes)
• The page view padding defaults to 20 seconds (maximum 2 minutes).
  • Page View padding is also known as added browsing time.

You can set these values in the WebMarshal Console, Tools > Server and Array Properties > General.

Note: Changing the session timeout can have other effects:

• Warning notice policies can be configured to display once per session
• Users authenticating with Basic Authentication may have to re-enter credentials when a new session starts

To allow useful reporting on sessions and visits, Trustwave recommends that you keep the session timeout reasonably short.

Calculations:

WebMarshal calculates browsing times using the following assumptions:

1. If no new page requests are seen before the session timeout, the user has stopped browsing.
   • This timeout also affects domain visits. This is a change from earlier versions of WebMarshal.
2. If a new page request for a domain is seen within 2 minutes, the user has been viewing pages continuously. WebMarshal calculates user browsing time as a continuous period beginning with the earliest request.
   • Remember that each protocol is treated separately for purposes of calculating domain visits.
3. If no new page requests for a domain are seen within 2 minutes, the user has stopped viewing pages. WebMarshal calculates user browsing time by determining the times of the earliest and last requests, and then adding the page view padding or added browsing time.

Example

Let’s assume that a user browses to three separate pages on the www.marshal.com web site. The first and second page requests are within 5 minutes of each other, while the gap between the second and third pages is greater than 5 minutes.

Each page browsed will download the html for the page, as well as numerous graphics and support files, including data from google-analytics.com.

WebMarshal records this activity as follows:

Interpretation

1. The red arrows represent the 20 second allowance to read the page.
2. The first and second requests to www.marshal.com are combined into one continuous block.
3. The user actually only visited one site, but time is logged against all the domains that data was requested from as a consequence of that one initial request.
4. The sum of the individual domain browsing times exceeds the session time.
   • As shown above this is reasonable, since it is obvious that the requests overlapped.
   • However in the reports it is not apparent that requests may overlap.
5. When a session times out (no activity for over 5 minutes), the domain browsing time (length of the visit) is calculated from the time of the first request until the last request plus 20 seconds of padding.
   • A single request will account as 20 seconds' browsing.
   • Two requests one minute apart is 80 seconds' browsing.
   • Two requests 3 minutes apart is treated as two single requests and accounts as a total of 40 seconds' browsing.
   • Two requests 6 minutes apart is treated as two separate visits within two separate sessions, and accounts as a total of 40 seconds' browsing.
6. The time taken to download a file is NOT important. Each file downloaded adds at most 20 seconds to domain or session time, regardless of file size. A 500MB download is counted the same as a 1KB graphic.

Notes:

For WebMarshal 3.X calculations, see Trustwave Knowledge Base article Q11649.