WebMarshal console takes a long time to respond

This article applies to:

• WebMarshal 3.7.X
• WebMarshal 6.X and 7.X

Symptoms:

• The WebMarshal console takes a long time to respond when you commit the configuration
• Configuration backups fail
• Error message: Failed to create backup: Exception of type 'System.OutOfMemoryException' was thrown.

Causes:

• One or more URL categories with over 5000 entries
• A common reason for very large URL categories is URL harvesting with the "Add URL to category" rule action.

Resolution:

In WebMarshal 3.7 and above, you can use FileFilter (text based URL filtering lists) to maintain large URL categories outside the WebMarshal user interface. FileFilter categories are reloaded on a daily schedule.

• FileFilter supports URL stemming but does not support wildcards. See further details in the Notes section below.

Follow the steps below to convert a URL category to a FileFilter category:

1. Log on to the WebMarshal Array Manager computer.
2. Open the WebMarshal Console, and navigate to the URL category you want to convert.
3. Click Export to File at the top of the category window.
4. Save the file as a plain text (.txt) file in the Array Manager FileFilter folder. By default this folder is located at: C:\Program Files\Marshal\WebMarshal\ArrayManager\Policy\FilteringLists\FileFilter
5. Using Notepad or another text editor, open the file you have just saved. At the top of the file, add text within brackets. The text must include an integer and a name for the category, as seen in the example image below.
   
   
   
   
   
• To make the file easy to find, the file name and category name should match.
• Each FileFilter category must have a unique integer identifier. Trustwave recommends you choose a number between 1 and 1000.
• To check which category numbers are already in use, in the Console expand the item URL Filtering Lists. Right-click FileFilter and select Properties, then view the Categories tab. (In 3.7, see the Filtering Lists tab of Server and Array Properties).
6. Remove or adjust wildcard entries. See the Notes section below.
7. Save the file.
8. The new FileFilter category will be visible immediately in the Console under FileFilter.
9. You can add the FileFilter category to any URL category by selecting the URL category and clicking Insert Categories.
10. To retain identical function, add the new FileFilter category to the large URL category that you saved it from, and then delete the individual URLs from the URL category. 
    • Before deleting any wildcard entries see the Notes section below.
• Note that when you delete the URLs, the Console may take some time to respond.

Notes:

• WebMarshal supports up to 256 FileFilter categories.
• To ensure that WebMarshal has loaded the FileFilter URLs, use the filtering list Update Now button. This button is located on the Filtering List Schedule page (or Filtering Lists tab) of Global Settings (Server and Array Properties).

FileFilter and Wildcards:

• FileFilter does not support wildcard entries using *
• FileFilter does support URL stemming. For instance, the entry http://example.com/ also matches http://www.example.com/ and http://anything.example.com/path/
• When moving entries from a URL category to FileFilter:
  • Retain in the URL category any entries with ENDING wildcards (for example, http://example.co*/)
  • Retain any entries with beginning wildcards that are not followed by a . (such as http://*ample.com/)
  • In the FileFilter file, change http://*.example.com/ to http://example.com/
  • However if you only want to match http://*.example.com/ and not the root domain http://example.com/ then retain the entry in the URL category.
  • Remove any entries containing * from the FileFilter file.