Preserve page labels (numbers) when splitting and merging

Jay Berkenbilt
1 parent 6ef9e312
Showing 12 changed files with 806 additions and 144 deletions
ChangeLog
manual/qpdf-manual.xml
qpdf/qpdf.cc
qpdf/qpdf.testcov
qpdf/qtest/qpdf.test
qpdf/qtest/qpdf/11-pages-with-labels.pdf
qpdf/qtest/qpdf/labels-split-01-06.pdf
qpdf/qtest/qpdf/labels-split-07-11.pdf
qpdf/qtest/qpdf/merge-implicit-ranges.pdf
qpdf/qtest/qpdf/merge-multiple-labels.pdf
qpdf/qtest/qpdf/merge-three-files-1.pdf
qpdf/qtest/qpdf/merge-three-files-2.pdf
 2018-12-18  Jay Berkenbilt  <ejb@ql.org>
+	* Preserve page labels when merging and splitting files. Prior
+	versions of qpdf simply preserved the page label information from
+	the first file, which usually wouldn't make any sense in the
+	merged file. Now any page that had a page number in any original
+	file will have the same page number after merging or splitting.
+
 	* Add QPDFPageLabelDocumentHelper class. This is a document helper
 	class that provides useful methods for dealing with page labels.
 	It abstracts the fact that they are stored as number trees and
@@ -911,23 +911,23 @@ make
     </itemizedlist>
    </para>
    <para>
-    Note that qpdf doesn't presently do anything special about other
-    constructs in a PDF file that may know about pages, so semantics
-    of splitting and merging vary across features.  For example, the
-    document's outlines (bookmarks) point to actual page objects, so
-    if you select some pages and not others, bookmarks that point to
-    pages that are in the output file will work, and remaining
-    bookmarks will not work.  On the other hand, page labels (page
-    numbers specified in the file) are just sequential, so page labels
-    will be messed up in the output file.  A future version of
-    <command>qpdf</command> may do a better job at handling these
-    issues.  (Note that the qpdf library already contains all of the
-    APIs required in order to implement this in your own application
-    if you need it.)  In the mean time, you can always use
-    <option>--empty</option> as the primary input file to avoid
-    copying all of that from the first file.  For example, to take
-    pages 1 through 5 from a <filename>infile.pdf</filename> while
-    preserving all metadata associated with that file, you could use
+    Starting in qpdf version 8.3, when you split and merge files, any
+    page labels (page numbers) are preserved in the final file. It is
+    expected that more document features will be preserved by
+    splitting and merging. In the mean time, semantics of splitting
+    and merging vary across features. For example, the document's
+    outlines (bookmarks) point to actual page objects, so if you
+    select some pages and not others, bookmarks that point to pages
+    that are in the output file will work, and remaining bookmarks
+    will not work. A future version of <command>qpdf</command> may do
+    a better job at handling these issues. (Note that the qpdf library
+    already contains all of the APIs required in order to implement
+    this in your own application if you need it.) In the mean time,
+    you can always use <option>--empty</option> as the primary input
+    file to avoid copying all of that from the first file. For
+    example, to take pages 1 through 5 from a
+    <filename>infile.pdf</filename> while preserving all metadata
+    associated with that file, you could use
     <programlisting><command>qpdf</command> <option>infile.pdf --pages infile.pdf 1-5 -- outfile.pdf</option>
 </programlisting>
@@ -946,8 +946,8 @@ make
     If, for some reason, you wanted to take the first page of an
     encrypted file called <filename>encrypted.pdf</filename> with
     password <literal>pass</literal> and repeat it twice in an output
-    file, and if you wanted to drop metadata (like page numbers and
-    outlines) but preserve encryption, you would use
+    file, and if you wanted to drop document-level metadata but
+    preserve encryption, you would use
     <programlisting><command>qpdf</command> <option>--empty --copy-encryption=encrypted.pdf --encryption-file-password=pass
 --pages encrypted.pdf --password=pass 1 ./encrypted.pdf --password=pass 1 --
@@ -16,6 +16,7 @@
 #include <qpdf/QPDF.hh>
 #include <qpdf/QPDFPageDocumentHelper.hh>
 #include <qpdf/QPDFPageObjectHelper.hh>
+#include <qpdf/QPDFPageLabelDocumentHelper.hh>
 #include <qpdf/QPDFExc.hh>
 #include <qpdf/QPDFWriter.hh>
@@ -2296,11 +2297,25 @@ static void handle_page_specs(QPDF&amp; pdf, Options&amp; o,
     // Keep track of any pages from the original file that we are
     // selecting.
     std::set<int> selected_from_orig;
+    std::vector<QPDFObjectHandle> new_labels;
+    bool any_page_labels = false;
+    int out_pageno = 0;
     for (std::vector<QPDFPageData>::iterator iter =
              parsed_specs.begin();
          iter != parsed_specs.end(); ++iter)
     {
         QPDFPageData& page_data = *iter;
+        ClosedFileInputSource* cis = 0;
+        if (page_spec_cfis.count(page_data.filename))
+        {
+            cis = page_spec_cfis[page_data.filename];
+            cis->stayOpen(true);
+        }
+        QPDFPageLabelDocumentHelper pldh(*page_data.qpdf);
+        if (pldh.hasPageLabels())
+        {
+            any_page_labels = true;
+        }
         if (o.verbose)
         {
             std::cout << whoami << ": adding pages from "
@@ -2309,22 +2324,14 @@ static void handle_page_specs(QPDF&amp; pdf, Options&amp; o,
         for (std::vector<int>::iterator pageno_iter =
                  page_data.selected_pages.begin();
              pageno_iter != page_data.selected_pages.end();
-             ++pageno_iter)
+             ++pageno_iter, ++out_pageno)
         {
             // Pages are specified from 1 but numbered from 0 in the
             // vector
             int pageno = *pageno_iter - 1;
-            ClosedFileInputSource* cis = 0;
-            if (page_spec_cfis.count(page_data.filename))
-            {
-                cis = page_spec_cfis[page_data.filename];
-                cis->stayOpen(true);
-            }
+            pldh.getLabelsForPageRange(pageno, pageno, out_pageno,
+                                       new_labels);
             dh.addPage(page_data.orig_pages.at(pageno), false);
-            if (cis)
-            {
-                cis->stayOpen(false);
-            }
             if (page_data.qpdf == &pdf)
             {
                 // This is a page from the original file. Keep track
@@ -2332,6 +2339,18 @@ static void handle_page_specs(QPDF&amp; pdf, Options&amp; o,
                 selected_from_orig.insert(pageno);
             }
         }
+        if (cis)
+        {
+            cis->stayOpen(false);
+        }
+    }
+    if (any_page_labels)
+    {
+        QPDFObjectHandle page_labels =
+            QPDFObjectHandle::newDictionary();
+        page_labels.replaceKey(
+            "/Nums", QPDFObjectHandle::newArray(new_labels));
+        pdf.getRoot().replaceKey("/PageLabels", page_labels);
     }
     // Delete page objects for unused page in primary. This prevents
@@ -2574,6 +2593,7 @@ static void write_outfile(QPDF&amp; pdf, Options&amp; o)
             dh.pushInheritedAttributesToPage();
             dh.removeUnreferencedResources();
         }
+        QPDFPageLabelDocumentHelper pldh(pdf);
         std::vector<QPDFObjectHandle> const& pages = pdf.getAllPages();
         int pageno_len = QUtil::int_to_string(pages.size()).length();
         unsigned int num_pages = pages.size();
@@ -2592,6 +2612,16 @@ static void write_outfile(QPDF&amp; pdf, Options&amp; o)
                 QPDFObjectHandle page = pages.at(pageno - 1);
                 outpdf.addPage(page, false);
             }
+            if (pldh.hasPageLabels())
+            {
+                std::vector<QPDFObjectHandle> labels;
+                pldh.getLabelsForPageRange(first - 1, last - 1, 0, labels);
+                QPDFObjectHandle page_labels =
+                    QPDFObjectHandle::newDictionary();
+                page_labels.replaceKey(
+                    "/Nums", QPDFObjectHandle::newArray(labels));
+                outpdf.getRoot().replaceKey("/PageLabels", page_labels);
+            }
             std::string page_range = QUtil::int_to_string(first, pageno_len);
             if (o.split_pages > 1)
             {
@@ -360,3 +360,4 @@ qpdf disable keep files open 0
 qpdf keep files open n 0
 qpdf keep files open y 0
 qpdf don't disable keep files open 0
+QPDFPageLabelDocumentHelper skip first 0
@@ -1137,7 +1137,7 @@ my @sp_cases = (
     [11, 'pdf extension', '', 'split-out.Pdf'],
     [4, 'fallback', '--pages 11-pages.pdf 1-3 minimal.pdf --', 'split-out'],
     );
-$n_tests += 9;
+$n_tests += 12;
 for (@sp_cases)
 {
     $n_tests += 1 + $_->[0];
@@ -1171,6 +1171,17 @@ foreach my $i (qw(01-04 05-08 09-10))
                  {$td->FILE => "shared-split-$i.pdf"});
 }
+$td->runtest("split page with labels",
+             {$td->COMMAND => "qpdf --qdf --static-id --split-pages=6".
+                  " 11-pages-with-labels.pdf split-out-labels.pdf"},
+             {$td->STRING => "", $td->EXIT_STATUS => 0});
+foreach my $i (qw(01-06 07-11))
+{
+    $td->runtest("check output ($i)",
+                 {$td->FILE => "split-out-labels-$i.pdf"},
+                 {$td->FILE => "labels-split-$i.pdf"});
+}
+
 foreach my $d (@sp_cases)
 {
     my ($n, $description, $xargs, $out) = @$d;
@@ -1335,7 +1346,7 @@ foreach my $d (@nrange_tests)
 show_ntests();
 # ----------
 $td->notify("--- Merging and Splitting ---");
-$n_tests += 16;
+$n_tests += 18;
 # Select pages from the same file multiple times including selecting
 # twice from an encrypted file and specifying the password only the
@@ -1368,8 +1379,7 @@ $td-&gt;runtest(&quot;merge three files&quot;,
                   " $pages_options --static-id"},
              {$td->STRING => "", $td->EXIT_STATUS => 0});
 # Manually verified about this file: it has the same pages but does
-# not contain outlines, page labels, or other things from the original
-# file.
+# not contain outlines or other things from the original file.
 $td->runtest("check output",
              {$td->FILE => "a.pdf"},
              {$td->FILE => "merge-three-files-2.pdf"});
@@ -1392,6 +1402,17 @@ $td-&gt;runtest(&quot;merge with implicit ranges&quot;,
 $td->runtest("check output",
              {$td->FILE => "a.pdf"},
              {$td->FILE => "merge-implicit-ranges.pdf"});
+$td->runtest("merge with multiple labels",
+             {$td->COMMAND =>
+                  "qpdf --empty a.pdf" .
+                  " --pages 11-pages-with-labels.pdf 8-11" .
+                  " minimal.pdf " .
+                  " page-labels-and-outlines.pdf 17-19 --" .
+                  " --static-id"},
+             {$td->STRING => "", $td->EXIT_STATUS => 0});
+$td->runtest("check output",
+             {$td->FILE => "a.pdf"},
+             {$td->FILE => "merge-multiple-labels.pdf"});
 $td->runtest("split with shared resources",
              {$td->COMMAND =>
+%PDF-1.3
+%¿÷¢þ
+%QDF-1.0
+
+%% Original object ID: 1 0
+1 0 obj
+<<
+  /PageLabels <<
+    /Nums [
+      0
+      <<
+        /P (pre-)
+        /St 1
+      >>
+      4
+      <<
+        /S /r
+        /St 4
+      >>
+    ]
+  >>
+  /Pages 2 0 R
+  /Type /Catalog
+>>
+endobj
+
+%% Original object ID: 2 0
+2 0 obj
+<<
+  /Count 6
+  /Kids [
+    3 0 R
+    4 0 R
+    5 0 R
+    6 0 R
+    7 0 R
+    8 0 R
+  ]
+  /Type /Pages
+>>
+endobj
+
+%% Page 1
+%% Original object ID: 3 0
+3 0 obj
+<<
+  /Contents 9 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 2
+%% Original object ID: 6 0
+4 0 obj
+<<
+  /Contents 12 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 3
+%% Original object ID: 8 0
+5 0 obj
+<<
+  /Contents 14 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 4
+%% Original object ID: 10 0
+6 0 obj
+<<
+  /Contents 16 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 5
+%% Original object ID: 12 0
+7 0 obj
+<<
+  /Contents 18 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 6
+%% Original object ID: 14 0
+8 0 obj
+<<
+  /Contents 20 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 11 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Contents for page 1
+%% Original object ID: 4 0
+9 0 obj
+<<
+  /Length 10 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 1) Tj ET
+endstream
+endobj
+
+10 0 obj
+47
+endobj
+
+%% Original object ID: 5 0
+11 0 obj
+<<
+  /BaseFont /Times-Roman
+  /Encoding /WinAnsiEncoding
+  /Subtype /Type1
+  /Type /Font
+>>
+endobj
+
+%% Contents for page 2
+%% Original object ID: 7 0
+12 0 obj
+<<
+  /Length 13 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 2) Tj ET
+endstream
+endobj
+
+13 0 obj
+47
+endobj
+
+%% Contents for page 3
+%% Original object ID: 9 0
+14 0 obj
+<<
+  /Length 15 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 3) Tj ET
+endstream
+endobj
+
+15 0 obj
+47
+endobj
+
+%% Contents for page 4
+%% Original object ID: 11 0
+16 0 obj
+<<
+  /Length 17 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 4) Tj ET
+endstream
+endobj
+
+17 0 obj
+47
+endobj
+
+%% Contents for page 5
+%% Original object ID: 13 0
+18 0 obj
+<<
+  /Length 19 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 5) Tj ET
+endstream
+endobj
+
+19 0 obj
+47
+endobj
+
+%% Contents for page 6
+%% Original object ID: 15 0
+20 0 obj
+<<
+  /Length 21 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 6) Tj ET
+endstream
+endobj
+
+21 0 obj
+47
+endobj
+
+xref
+0 22
+0000000000 65535 f 
+0000000052 00000 n 
+0000000285 00000 n 
+0000000444 00000 n 
+0000000699 00000 n 
+0000000955 00000 n 
+0000001212 00000 n 
+0000001469 00000 n 
+0000001726 00000 n 
+0000001995 00000 n 
+0000002098 00000 n 
+0000002145 00000 n 
+0000002304 00000 n 
+0000002408 00000 n 
+0000002478 00000 n 
+0000002582 00000 n 
+0000002653 00000 n 
+0000002757 00000 n 
+0000002828 00000 n 
+0000002932 00000 n 
+0000003003 00000 n 
+0000003107 00000 n 
+trailer <<
+  /Root 1 0 R
+  /Size 22
+  /ID [<31415926535897932384626433832795><31415926535897932384626433832795>]
+>>
+startxref
+3127
+%%EOF
+%PDF-1.3
+%¿÷¢þ
+%QDF-1.0
+
+%% Original object ID: 1 0
+1 0 obj
+<<
+  /PageLabels <<
+    /Nums [
+      0
+      <<
+        /S /r
+        /St 6
+      >>
+      2
+      <<
+        /S /a
+        /St 16
+      >>
+    ]
+  >>
+  /Pages 2 0 R
+  /Type /Catalog
+>>
+endobj
+
+%% Original object ID: 2 0
+2 0 obj
+<<
+  /Count 5
+  /Kids [
+    3 0 R
+    4 0 R
+    5 0 R
+    6 0 R
+    7 0 R
+  ]
+  /Type /Pages
+>>
+endobj
+
+%% Page 1
+%% Original object ID: 3 0
+3 0 obj
+<<
+  /Contents 8 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 10 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 2
+%% Original object ID: 6 0
+4 0 obj
+<<
+  /Contents 11 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 10 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 3
+%% Original object ID: 8 0
+5 0 obj
+<<
+  /Contents 13 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 10 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 4
+%% Original object ID: 10 0
+6 0 obj
+<<
+  /Contents 15 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 10 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Page 5
+%% Original object ID: 12 0
+7 0 obj
+<<
+  /Contents 17 0 R
+  /MediaBox [
+    0
+    0
+    612
+    792
+  ]
+  /Parent 2 0 R
+  /Resources <<
+    /Font <<
+      /F1 10 0 R
+    >>
+    /ProcSet [
+      /PDF
+      /Text
+    ]
+  >>
+  /Type /Page
+>>
+endobj
+
+%% Contents for page 1
+%% Original object ID: 4 0
+8 0 obj
+<<
+  /Length 9 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 7) Tj ET
+endstream
+endobj
+
+9 0 obj
+47
+endobj
+
+%% Original object ID: 5 0
+10 0 obj
+<<
+  /BaseFont /Times-Roman
+  /Encoding /WinAnsiEncoding
+  /Subtype /Type1
+  /Type /Font
+>>
+endobj
+
+%% Contents for page 2
+%% Original object ID: 7 0
+11 0 obj
+<<
+  /Length 12 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 8) Tj ET
+endstream
+endobj
+
+12 0 obj
+47
+endobj
+
+%% Contents for page 3
+%% Original object ID: 9 0
+13 0 obj
+<<
+  /Length 14 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 9) Tj ET
+endstream
+endobj
+
+14 0 obj
+47
+endobj
+
+%% Contents for page 4
+%% Original object ID: 11 0
+15 0 obj
+<<
+  /Length 16 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 10) Tj ET
+endstream
+endobj
+
+16 0 obj
+48
+endobj
+
+%% Contents for page 5
+%% Original object ID: 13 0
+17 0 obj
+<<
+  /Length 18 0 R
+>>
+stream
+BT /F1 15 Tf 72 720 Td (Original page 11) Tj ET
+endstream
+endobj
+
+18 0 obj
+48
+endobj
+
+xref
+0 19
+0000000000 65535 f 
+0000000052 00000 n 
+0000000282 00000 n 
+0000000431 00000 n 
+0000000686 00000 n 
+0000000942 00000 n 
+0000001199 00000 n 
+0000001456 00000 n 
+0000001725 00000 n 
+0000001827 00000 n 
+0000001873 00000 n 
+0000002032 00000 n 
+0000002136 00000 n 
+0000002206 00000 n 
+0000002310 00000 n 
+0000002381 00000 n 
+0000002486 00000 n 
+0000002557 00000 n 
+0000002662 00000 n 
+trailer <<
+  /Root 1 0 R
+  /Size 19
+  /ID [<31415926535897932384626433832795><31415926535897932384626433832795>]
+>>
+startxref
+2682
+%%EOF