updated doc for v0.40

Philippe Lagadec
1 parent f1cefbd4
Showing 4 changed files with 113 additions and 38 deletions
oletools/doc/Home.html
oletools/doc/Home.md
oletools/doc/olevba.html
oletools/doc/olevba.md
-<p>python-oletools v0.12 documentation</p>
+<p>python-oletools v0.40 documentation</p>
 <p>===================================</p>
 <p>This is the home page of the documentation for python-oletools. The latest version can be found</p>
 <p><a href="https://bitbucket.org/decalage/oletools/wiki">online</a>, otherwise a copy is provided in the doc subfolder of the package.</p>
-python-oletools v0.12 documentation
+python-oletools v0.40 documentation
 ===================================
 This is the home page of the documentation for python-oletools. The latest version can be found 
@@ -31,7 +31,7 @@
 <li><p>Detect and decodes strings obfuscated with Hex/Base64/StrReverse/Dridex</p></li>
 <li><p>Deobfuscates VBA expressions with any combination of Chr, Asc, Val, StrReverse, Environ, +, &amp;, using a VBA parser built with</p></li>
 </ul>
-<p><a href="http://pyparsing.wikispaces.com">pyparsing</a></p>
+<p><a href="http://pyparsing.wikispaces.com">pyparsing</a>, including custom Hex and Base64 encodings</p>
 <ul>
 <li><p>Extract IOCs/patterns of interest such as IP addresses, URLs, e-mail addresses and executable file names</p></li>
 <li><p>Scan multiple files and sample collections (wildcards, recursive)</p></li>
@@ -68,7 +68,7 @@ Options:
   -z ZIP_PASSWORD, --zip=ZIP_PASSWORD
-                        if the file is a zip archive, open first file from it,
+                        if the file is a zip archive, open all files from it,
                         using the provided password (requires Python 2.6+)
@@ -106,9 +106,7 @@ Options:
   --attr                display the attribute lines at the beginning of VBA
-                        source code
-
-  --each                analyze each VBA module separately</code></pre>
+                        source code</code></pre>
 <h3 id="examples">Examples</h3>
 <p>Scan a single file:</p>
 <pre><code>olevba.py file.doc</code></pre>
@@ -292,24 +290,26 @@ OLE:MA----- \MalwareZoo\VBA\samples\Word within Word macro auto.doc&lt;/code&gt;&lt;/pre&gt;
 <p>IMPORTANT: olevba is currently under active development, therefore this API is likely to change.</p>
 <h3 id="import-olevba">Import olevba</h3>
 <p>First, import the <strong>oletools.olevba</strong> package, using at least the VBA_Parser and VBA_Scanner classes:</p>
-<pre><code>from oletools.olevba import VBA_Parser, VBA_Scanner</code></pre>
+<pre><code>from oletools.olevba import VBA_Parser, TYPE_OLE, TYPE_OpenXML, TYPE_Word2003_XML, TYPE_MHTML </code></pre>
 <h3 id="parse-a-ms-office-file">Parse a MS Office file</h3>
 <p>To parse a file on disk, create an instance of the <strong>VBA_Parser</strong> class, providing the name of the file to open as parameter.</p>
 <p>For example:</p>
-<pre><code>vba = VBA_Parser(&#39;my_file_with_macros.doc&#39;)</code></pre>
+<pre><code>vbaparser = VBA_Parser(&#39;my_file_with_macros.doc&#39;)</code></pre>
 <p>The file may also be provided as a bytes string containing its data. In that case, the actual</p>
 <p>filename must be provided for reference, and the file content with the data parameter. For example:</p>
 <pre><code>myfile = &#39;my_file_with_macros.doc&#39;
 filedata = open(myfile, &#39;rb&#39;).read()
-vba = VBA_Parser(myfile, data=filedata)</code></pre>
-<p>VBA_Parser will raise an exception if the file is not a supported format, either OLE (MS Office 97-2003) or OpenXML</p>
-<p>(MS Office 2007+).</p>
+vbaparser = VBA_Parser(myfile, data=filedata)</code></pre>
+<p>VBA_Parser will raise an exception if the file is not a supported format, such as OLE (MS Office 97-2003), OpenXML</p>
+<p>(MS Office 2007+), MHTML or Word 2003 XML.</p>
+<p>After parsing the file, the attribute <strong>VBA_Parser.type</strong> is a string indicating the file type.</p>
+<p>It can be either TYPE_OLE, TYPE_OpenXML, TYPE_Word2003_XML or TYPE_MHTML. (constants defined in the olevba module)</p>
 <h3 id="detect-vba-macros">Detect VBA macros</h3>
 <p>The method <strong>detect_vba_macros</strong> of a VBA_Parser object returns True if VBA macros have been found in the file,</p>
 <p>False otherwise.</p>
-<pre><code>if vba.detect_vba_macros():
+<pre><code>if vbaparser.detect_vba_macros():
     print &#39;VBA Macros found&#39;
@@ -334,7 +334,7 @@ else:
 <li><p>vba_code: string containing the VBA source code in clear text</p></li>
 </ul>
 <p>Example:</p>
-<pre><code>for (filename, stream_path, vba_filename, vba_code) in vba.extract_macros():
+<pre><code>for (filename, stream_path, vba_filename, vba_code) in vbaparser.extract_macros():
     print &#39;-&#39;*79
@@ -347,7 +347,44 @@ else:
     print &#39;- &#39;*39
     print vba_code</code></pre>
+<p>Alternatively, the VBA_Parser method <strong>extract_all_macros</strong> returns the same results as a list of tuples.</p>
 <h3 id="analyze-vba-source-code">Analyze VBA Source Code</h3>
+<p>Since version 0.40, the VBA_Parser class provides simpler methods than VBA_Scanner to analyze all macros contained</p>
+<p>in a file:</p>
+<p>The methods <strong>scan</strong> or <strong>scan_summary</strong> from the class <strong>VBA_Parser</strong> can be used to scan the source code of all</p>
+<p>VBA modules to find obfuscated strings, suspicious keywords, IOCs, auto-executable macros, etc.</p>
+<p>scan() takes an optional argument include_decoded_strings: if set to True, the results will contain all the encoded</p>
+<p>strings found in the code (Hex, Base64, Dridex) with their decoded value.</p>
+<p>By default, it will include the strings which contain printable characters only.</p>
+<p><strong>VBA_Parser.scan()</strong> returns a list of tuples (type, keyword, description), one for each item in the results.</p>
+<ul>
+<li>type may be either 'AutoExec', 'Suspicious', 'IOC', 'Hex String', 'Base64 String', 'Dridex String' or</li>
+</ul>
+<p>'VBA obfuscated Strings'.</p>
+<ul>
+<li>keyword is the string found for auto-executable macros, suspicious keywords or IOCs. For obfuscated strings, it is</li>
+</ul>
+<p>the decoded value of the string.</p>
+<ul>
+<li>description provides a description of the keyword. For obfuscated strings, it is the encoded value of the string.</li>
+</ul>
+<p>Example:</p>
+<pre><code>results = vbaparser.scan()
+
+for kw_type, keyword, description in results:
+
+    print &#39;type=%s - keyword=%s - description=%s&#39; % (kw_type, keyword, description)</code></pre>
+<p><strong>VBA_Parser.scan_summary()</strong> returns a tuple with the number of items found for each category:</p>
+<p>(autoexec, suspicious, IOCs, hex, base64, dridex, vbastrings).</p>
+<h3 id="close-the-vba_parser">Close the VBA_Parser</h3>
+<p>After usage, it is better to call the <strong>close</strong> method of the VBA_Parser object, to make sure the file is closed,</p>
+<p>especially if your application is parsing many files.</p>
+<pre><code>vbaparser.close()</code></pre>
+<hr />
+<h2 id="deprecated-api">Deprecated API</h2>
+<p>The following methods and functions are still functional, but their usage is not recommended</p>
+<p>since they have been replaced by better solutions.</p>
+<h3 id="vba_scanner-deprecated">VBA_Scanner (deprecated)</h3>
 <p>Note: this API is under active development and may change in the future.</p>
 <p>The class <strong>VBA_Scanner</strong> can be used to scan the source code of a VBA module to find obfuscated strings,</p>
 <p>suspicious keywords, IOCs, auto-executable macros, etc.</p>
@@ -449,10 +486,6 @@ if patterns:
 else:
     print &#39;Patterns: None found&#39;</code></pre>
-<h3 id="close-the-vba_parser">Close the VBA_Parser</h3>
-<p>After usage, it is better to call the <strong>close</strong> method of the VBA_Parser object, to make sure the file is closed,</p>
-<p>especially if your application is parsing many files.</p>
-<pre><code>vba.close()</code></pre>
 <hr />
 <p>python-oletools documentation</p>
 <hr />
@@ -35,7 +35,7 @@ by John William Davison, with significant modifications.
 - Detect anti-sandboxing and anti-virtualization techniques
 - Detect and decodes strings obfuscated with Hex/Base64/StrReverse/Dridex
 - Deobfuscates VBA expressions with any combination of Chr, Asc, Val, StrReverse, Environ, +, &, using a VBA parser built with
-[pyparsing](http://pyparsing.wikispaces.com)
+[pyparsing](http://pyparsing.wikispaces.com), including custom Hex and Base64 encodings
 - Extract IOCs/patterns of interest such as IP addresses, URLs, e-mail addresses and executable file names
 - Scan multiple files and sample collections (wildcards, recursive)
 - Triage mode for a summary view of multiple files
@@ -71,7 +71,7 @@ and potential IOCs (URLs, IP addresses, e-mail addresses, executable filenames, 
       -h, --help            show this help message and exit
       -r                    find files recursively in subdirectories.
       -z ZIP_PASSWORD, --zip=ZIP_PASSWORD
-                            if the file is a zip archive, open first file from it,
+                            if the file is a zip archive, open all files from it,
                             using the provided password (requires Python 2.6+)
       -f ZIP_FNAME, --zipfname=ZIP_FNAME
                             if the file is a zip archive, file(s) to be opened
@@ -91,7 +91,6 @@ and potential IOCs (URLs, IP addresses, e-mail addresses, executable filenames, 
                             content (Hex, Base64, StrReverse, Dridex, VBA).
       --attr                display the attribute lines at the beginning of VBA
                             source code
-      --each                analyze each VBA module separately
 ### Examples
@@ -249,7 +248,7 @@ IMPORTANT: olevba is currently under active development, therefore this API is l
 First, import the **oletools.olevba** package, using at least the VBA_Parser and VBA_Scanner classes:
     :::python
-    from oletools.olevba import VBA_Parser, VBA_Scanner
+    from oletools.olevba import VBA_Parser, TYPE_OLE, TYPE_OpenXML, TYPE_Word2003_XML, TYPE_MHTML 
 ### Parse a MS Office file 
@@ -257,7 +256,7 @@ To parse a file on disk, create an instance of the **VBA_Parser** class, providi
 For example:
     :::python
-    vba = VBA_Parser('my_file_with_macros.doc')
+    vbaparser = VBA_Parser('my_file_with_macros.doc')
 The file may also be provided as a bytes string containing its data. In that case, the actual 
 filename must be provided for reference, and the file content with the data parameter. For example:
@@ -265,10 +264,13 @@ filename must be provided for reference, and the file content with the data para
     :::python
     myfile = 'my_file_with_macros.doc'
     filedata = open(myfile, 'rb').read()
-    vba = VBA_Parser(myfile, data=filedata)
+    vbaparser = VBA_Parser(myfile, data=filedata)
-VBA_Parser will raise an exception if the file is not a supported format, either OLE (MS Office 97-2003) or OpenXML 
-(MS Office 2007+). 
+VBA_Parser will raise an exception if the file is not a supported format, such as OLE (MS Office 97-2003), OpenXML 
+(MS Office 2007+), MHTML or Word 2003 XML.
+ 
+After parsing the file, the attribute **VBA_Parser.type** is a string indicating the file type.
+It can be either TYPE_OLE, TYPE_OpenXML, TYPE_Word2003_XML or TYPE_MHTML. (constants defined in the olevba module)
 ### Detect VBA macros
@@ -276,7 +278,7 @@ The method **detect_vba_macros** of a VBA_Parser object returns True if VBA macr
 False otherwise.
     :::python
-    if vba.detect_vba_macros():
+    if vbaparser.detect_vba_macros():
         print 'VBA Macros found'
     else:
         print 'No VBA Macros found'
@@ -304,16 +306,66 @@ for each VBA macro found.
 Example:
     :::python
-    for (filename, stream_path, vba_filename, vba_code) in vba.extract_macros():
+    for (filename, stream_path, vba_filename, vba_code) in vbaparser.extract_macros():
         print '-'*79
         print 'Filename    :', filename
         print 'OLE stream  :', stream_path
         print 'VBA filename:', vba_filename
         print '- '*39
         print vba_code
+        
+Alternatively, the VBA_Parser method **extract_all_macros** returns the same results as a list of tuples.
 ### Analyze VBA Source Code
+Since version 0.40, the VBA_Parser class provides simpler methods than VBA_Scanner to analyze all macros contained
+in a file:
+
+The methods **scan** or **scan_summary** from the class **VBA_Parser** can be used to scan the source code of all 
+VBA modules to find obfuscated strings, suspicious keywords, IOCs, auto-executable macros, etc.
+
+scan() takes an optional argument include_decoded_strings: if set to True, the results will contain all the encoded
+strings found in the code (Hex, Base64, Dridex) with their decoded value.
+By default, it will include the strings which contain printable characters only.
+
+**VBA_Parser.scan()** returns a list of tuples (type, keyword, description), one for each item in the results. 
+
+- type may be either 'AutoExec', 'Suspicious', 'IOC', 'Hex String', 'Base64 String', 'Dridex String' or 
+  'VBA obfuscated Strings'.
+- keyword is the string found for auto-executable macros, suspicious keywords or IOCs. For obfuscated strings, it is
+  the decoded value of the string.
+- description provides a description of the keyword. For obfuscated strings, it is the encoded value of the string.
+
+Example:
+
+    :::python
+    results = vbaparser.scan()
+    for kw_type, keyword, description in results:
+        print 'type=%s - keyword=%s - description=%s' % (kw_type, keyword, description)
+        
+**VBA_Parser.scan_summary()** returns a tuple with the number of items found for each category: 
+(autoexec, suspicious, IOCs, hex, base64, dridex, vbastrings).
+
+
+
+### Close the VBA_Parser
+
+After usage, it is better to call the **close** method of the VBA_Parser object, to make sure the file is closed, 
+especially if your application is parsing many files.
+
+    :::python
+    vbaparser.close()
+
+
+--------------------------------------------------------------------------
+
+## Deprecated API
+
+The following methods and functions are still functional, but their usage is not recommended
+since they have been replaced by better solutions.
+
+### VBA_Scanner (deprecated)
+
 Note: this API is under active development and may change in the future.
 The class **VBA_Scanner** can be used to scan the source code of a VBA module to find obfuscated strings,
@@ -423,16 +475,6 @@ Sample usage:
         print 'Patterns: None found'
-### Close the VBA_Parser
-
-After usage, it is better to call the **close** method of the VBA_Parser object, to make sure the file is closed, 
-especially if your application is parsing many files.
-
-    :::python
-    vba.close()
-
-        
-
 --------------------------------------------------------------------------
 python-oletools documentation