Trustwave Rapid Response: CrowdStrike Falcon Outage Update. Learn More

Trustwave Rapid Response: CrowdStrike Falcon Outage Update. Learn More

Services
Capture
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

twi-managed-portal-color
Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

twi-briefcase-color-svg
Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

tw-laptop-data
Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

twi-database-color-svg
Database Security

Prevent unauthorized access and exceed compliance requirements.

twi-email-color-svg
Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

tw-officer
Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

tw-network
Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Offensive Security
Solutions to maximize your security ROI
Microsoft Exchange Server Attacks
Stay protected against emerging threats
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats
Loading...
Loading...

FAQ: How does WebMarshal Proxy Caching work?

Expand / Collapse


This article applies to:

  • WebMarshal 6.5 and above

Question:

  • How does Proxy Caching work in WebMarshal?

Reply:

Most HTTP proxies work in the same basic manner. The RFC document for HTTP (RFC-2616) includes a set of rules that indicate which content can and cannot be cached. The RFC also sets out a group of HTTP headers that allow web servers to indicate the cacheability of files and how long each file may be cached.

The WebMarshal proxy cache provides a caching implementation within the bounds of the RFC.

WebMarshal does not currently cache FTP or HTTPS traffic. FTP traffic is not covered by the HTTP RFC, and the RFC for FTP does not specify any mechanisms for defining caching information. HTTPS traffic is specifically excluded because it is supposed to be secure and there is a risk of information disclosure if pages are cached incorrectly. 

When a request comes through for an HTTP resource, WebMarshal checks the cache directory on disk to see if there is a cached response for the requested URL. If a response has been cached, WebMarshal then checks to see if the response is "fresh" enough to be returned to the client. If the response is considered "fresh", WebMarshal will pass the cached content to the engine for policy processing. If the response is "stale" or no response has been cached, WebMarshal will contact the web server and request the item again. If the web server indicates that WebMarshal's cached copy is still valid, WebMarshal will use the cached data. Otherwise the response from the web server will be downloaded. 

The proxy cache does not affect how the WebMarshal policy is applied. When a response is retrieved from the cache, it is still written as a file to the proxy's temp directory then sent to the engine for processing. There is no caching of the rule processing result, because changes to rules or updates to virus scanner data files would immediately invalidate any cached result. 

WebMarshal's cache implementation also has some features that can make life easier for system administrators. 

Cache Exclusions

While most web sites operate within the bounds of the RFC, there are always some that are incorrectly configured or coded. This can lead to situations where content is cached when it shouldn't be. WebMarshal allows sites to be excluded from caching altogether in order to work around such problems. (You can configure this list in the WebMarshal Console, Properties > Proxy Cache.)

Cache Pruning

When the allocated cache space is nearly full, WebMarshal automatically manages the content. The least recently used items are removed to make room for new content. Note that the original download date of the file is not relevant to pruning. Older files will be retained in the cache if they are requested often and have not changed.

Cache Directory Organization

The cache directory is organized in a logical manner that makes it easy to get to files for a given domain. Directories are organised by top-level domain, then the first 3 characters of the domain name, then the full domain name. For example, all pages for www.m86security.com are stored within com\m86\www.m86security.com. If cache information for a domain is ever corrupted, the system administrator simply needs to locate the correct domain and delete it. The cache will carry on as if those files never existed. 

  • The administrator can also verify the cache contents and perform other advanced tasks using the Cache Tool. See Trustwave Knowledgebase article Q12724.

Cache URL Rewriting

YouTube presents a problem for caching proxies. Even though its video files are marked as cacheable for an hour, the URL changes every time the video is played. Because the URL changes, a proxy cannot know if the content will be the same, so it has to download the video again. The WebMarshal cache contains URL rewriting rules for YouTube videos that strip out the changing part of the URL. This allows WebMarshal to serve the same video to multiple users without having to contact the server again.

Notes:

For some best practice technical recommendations about caching, see Trustwave Knowledgebase article Q12720.


To contact Trustwave about this article or to request support:


Rate this Article:
     

Add Your Comments


Comment submission is disabled for anonymous users.
Please send feedback to Trustwave Technical Support or the Webmaster
.