Array optimization fixes performance issues (fixes #305, #311)

Jay Berkenbilt
1 parent 522d2b22
Showing 3 changed files with 16 additions and 7 deletions
ChangeLog
TODO
manual/qpdf-manual.xml
+2019-08-17  Jay Berkenbilt  <ejb@ql.org>
+
+	* Change internal implementation of QPDF arrays to use sparse
+	arrays, which results in using much less memory for arrays with
+	large numbers of nulls. Various files have been encountered in the
+	wild that contains thousands of arrays with millions of nulls.
+	Fixes #305, #311.
+
 2019-07-03  Jay Berkenbilt  <ejb@ql.org>
 	* Non-compatible API change: change
-Performance
-===========
-
-There is some performance problem that happens with some large files.
-Issues 305 and 311 both seem to be examples of this. Files saved in
-../misc/bugs.
-
 Fuzz Errors
 ===========
@@ -4482,6 +4482,14 @@ print &quot;\n&quot;;
         </listitem>
         <listitem>
          <para>
+          The underlying implementation of QPDF arrays has been
+          enhanced to be much more memory efficient when dealing with
+          arrays with lots of nulls. This enables qpdf to use
+          drastically less memory for certain types of files.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
           A new helper method
           <function>QUtil::read_file_into_memory</function> was added.
          </para>