Slow configuration loading due to multiple SpamCensor rules

This article applies to:

• Trustwave MailMarshal (SEG)
• SpamCensor
• SpamBotCensor
• Rules using the condition "Where message is categorized as"
• Rules using the condition "Detected as spam by SpamEngine"

Symptoms:

• Configuration reload is very slow (more than 5 minutes to load configuration)
• High memory usage on processing nodes
• Processing may be slow
• Configuration includes a number of rules that use a SpamCensor or Category rule condition (significantly more of these conditions than the default rules)  
• Error message: Failed to create a new Classifier Configuration
  • Note that this is a generic message and could have other causes such as malformed XML

Causes:

• For each Category Script or SpamCensor condition used in a rule, a copy of the script is loaded in memory
• Complex scripts (such as SpamCensor) are large
• Loading multiple scripts takes time and uses significant amounts of memory
• Running the scripts can also take more time

Resolution:

To resolve this issue, optimize your rules so that SpamCensor conditions are called as few times as possible.

Do not create multiple rules that run the same SpamCensor condition in combination with various other conditions (such as sender, recipient, or content conditions).

If you want to apply policy based on a combination of SpamCensor and other conditions, use the "pass message" rule action as in the recommended example below.

Examples:

Problem example

Do not create many separate rules that use a category and another condition, like this:

RULE 1
Where message is incoming
Where addressed to 'Group1'
Where message is categorized as 'Spam'
Move the message to 'Spam - Confirmed'

RULE 2
Where message is incoming
Where addressed to 'Group1'
Where message is categorized as 'Spam-Suspected'
Move the message to 'Spam - Suspected'

RULE 3
Where message is incoming
Where addressed to 'Group2'
Where message is categorized as 'Spam'
Move the message to 'Spam - Confirmed'

RULE 4
Where message is incoming
Where addressed to 'Group2'
Where message is categorized as 'Spam-Suspected'
Move the message to 'Spam - Suspected'

Recommended example

Do use a decision tree with the SpamCensor rule at the top. Based on one or two uses of SpamCensor, pass the messages to policy groups that include the additional conditions, similar to the below.

• Note: When using the "pass" action with several policy groups, make sure the last rule in each group is a catch-all rule that takes a definite action on all messages. This is important so that processing does not fall through to the next group.

(Wording and structure is simplified in this example):

1. Policy group Spam Check
   

   RULE: Check as Spam - Confirmed
   Where message is incoming
   Where message is categorized as 'Spam'
   Pass the message on to 'Quarantine Spam - Confirmed:Qurantine Group 1 confirmed'
   
   RULE: Check as Spam - Suspected
   Where message is incoming
   Where message is categorized as 'Spam-Suspected'
   Pass the message on to 'Quarantine Spam - Suspected:Quarantine Group 1 suspected' 
   
   RULE: Not Spam
   Where message is incoming
   Pass the message on to 'Content Checks:First Content Rule' 

2. Policy group Quarantine Spam - Confirmed
   
   RULE: Quarantine Group 1 confirmed
   Where message is incoming
   Where addressed to 'Group1'
   Move the message to 'Group 1 Spam - Confirmed'
   
   RULE: Quarantine other groups confirmed
   Where message is incoming
   Move the message to 'Other Groups Spam - Confirmed'
   
   
3. Policy group Quarantine Spam - Suspected
   
   RULE: Quarantine Group 1 suspected
   Where message is incoming
   Where addressed to 'Group1'
   Move the message to 'Group 1 Spam - Suspected'
   
   RULE: Quarantine other groups suspected
   Where message is incoming
   Move the message to 'Other Groups Spam - Suspected'
   
   
4. Policy Group Content Checks
   (rules that apply to non-spam messages)