Changing the Nested PDF unpacking limit


This article applies to:

  • Trustwave MailMarshal (SEG) 6.5 and above
  • Trustwave ECM/MailMarshal Exchange 7.X
  • PDF documents

Question:

  • How do I change the limit of recursive unpacking of nested items in PDF documents?
  • Error message: PDF Error - Document too deeply nested

Information:

PDF documents can contain nested items. Some automatically generated documents use thousands of levels of nesting. These documents can cause MailMarshal unpacking to fail.

As of version 6.5, MailMarshal introduced a default limit of 500 levels of recursive unpacking of nested PDF elements. If a document exceeds this level of nesting, MailMarshal will move the message to the Deadletter - Unpacking folder and log the following message: PDF Error - Document too deeply nested.

You can change the allowed level of nesting. However, note the following recommendations:

  • If you set the value above approximately 1000 to 1200 levels, the PDF unpacking is likely to start failing due to memory problems.
  • If you set the value below 200 levels, MailMarshal may not successfully process moderately complex PDF documents, resulting in the '"PDF Error - Document too deeply nested" exception.

Procedure:

To change the allowed level of recursive unpacking of nested PDF elements, set an entry in the Registry on the MailMarshal Array Manager computer.

  1. On the Array Manager, edit the Registry (10.X: use Advanced Settings in the Management Console)
  2. Navigate to the SEG Engine key:
    • In version 8.X: HKEY_LOCAL_MACHINE\SOFTWARE\Trustwave\Secure Email Gateway\Default\Engine
    • 10.X: value names have the prefix Engine. (Engine dot).
    • For full details of the location for each product version, see article Q10832.
  3. If necessary, create the DWORD value MaxPDFNesting
  4. Set the value (decimal) to the desired level of recursion. If the value is absent the default is 500 decimal.
  5. To apply the change, commit configuration and then restart the MailMarshal Engine service on processing servers.

As always, take due care when editing the Registry.


Last Modified 4/1/2020.
https://support.trustwave.com/kb/KnowledgebaseArticle12243.aspx