OpenSystemsDevelopment / qpdf

Commit 893d38b87e4ad6c6c55f49464f6b721c516ec878

Authored by Jay Berkenbilt
1 parent a5367003

Allow propagation of errors and retry through StreamDataProvider 

StreamDataProvider::provideStreamData now has a rich enough API for it
to effectively proxy to pipeStreamData.
Inline Side-by-side
Showing 21 changed files with 469 additions and 174 deletions
ChangeLog
  View file @893d38b
1 1 2020-04-04 Jay Berkenbilt <ejb@ql.org>
2 2  
  3 + * Add a new provideStreamData method for StreamDataProvider that
  4 + allows a success code to be returned and that accepts the
  5 + suppress_warnings and will_retry methods. This makes it possible
  6 + to have a StreamDataProvider call pipeStreamData and propagate its
  7 + results back. This change allows better error handling and
  8 + recovery when objects are copied from other files and when
  9 + "immediate copy from" is enabled.
  10 +
  11 + * Add a new version of QPDFObjectHandle::pipeStreamData whose
  12 + return value indicates overall success or failure rather than
  13 + whether nor not filtering was attempted. It should have always
  14 + been this way. This change was done in a backward-compatible
  15 + fashion. Previously existing pipeStreamData methods' return values
  16 + mean the same as always.
  17 +
3 18 * Add "objectinfo" section to json output. In this release,
4 19 information about whether each object is a stream or not is
5 20 provided. There's otherwise no way to tell conclusively from the
... ...
... ... @@ -20,18 +20,6 @@ ABI Changes
20 20 This is a list of changes to make next time there is an ABI change.
21 21 Comments appear in the code prefixed by "ABI"
22 22  
23   -* (Source compatibility) As somewhat discussed in issue 219, the
24   - original pipeStreamData in QPDF_Stream has various logic for
25   - reporting warnings and letting the caller retry. This logic is not
26   - implemented for stream data providers. When copying foreign streams,
27   - qpdf uses a stream data provider (QPDF::CopiedStreamDataProvider) to
28   - read the stream data from the original file. While a warning is
29   - issued for that case, there is no way to actually propagate failure
30   - information back through because
31   - StreamDataProvider::provideStreamData doesn't take the
32   - suppress_warnings or will_retry options, and adding them would break
33   - source compatibility.
34   -
35 23 C++-11
36 24 ======
37 25  
... ...
include/qpdf/QPDF.hh
  View file @893d38b
... ... @@ -813,8 +813,9 @@ class QPDF
813 813 virtual ~CopiedStreamDataProvider()
814 814 {
815 815 }
816   - virtual void provideStreamData(int objid, int generation,
817   - Pipeline* pipeline);
  816 + virtual bool provideStreamData(
  817 + int objid, int generation, Pipeline* pipeline,
  818 + bool suppress_warnings, bool will_retry) override;
818 819 void registerForeignStream(QPDFObjGen const& local_og,
819 820 QPDFObjectHandle foreign_stream);
820 821 void registerForeignStream(QPDFObjGen const& local_og,
... ... @@ -909,9 +910,7 @@ class QPDF
909 910 bool will_retry);
910 911 bool pipeForeignStreamData(
911 912 PointerHolder<ForeignStreamData>,
912   - Pipeline*,
913   - int encode_flags,
914   - qpdf_stream_decode_level_e decode_level);
  913 + Pipeline*, bool suppress_warnings, bool will_retry);
915 914 static bool pipeStreamData(PointerHolder<QPDF::EncryptionParameters> encp,
916 915 PointerHolder<InputSource> file,
917 916 QPDF& qpdf_for_warning,
... ...
include/qpdf/QPDFObjectHandle.hh
  View file @893d38b
... ... @@ -57,6 +57,9 @@ class QPDFObjectHandle
57 57 class QPDF_DLL_CLASS StreamDataProvider
58 58 {
59 59 public:
  60 + QPDF_DLL
  61 + StreamDataProvider(bool supports_retry = false);
  62 +
60 63 QPDF_DLL
61 64 virtual ~StreamDataProvider()
62 65 {
... ... @@ -74,8 +77,30 @@ class QPDFObjectHandle
74 77 // information is made available just to make it more
75 78 // convenient to use a single StreamDataProvider object to
76 79 // provide data for multiple streams.
  80 +
  81 + // Prior to qpdf 10.0.0, it was not possible to handle errors
  82 + // the way pipeStreamData does or to pass back success.
  83 + // Starting in qpdf 10.0.0, those capabilities have been added
  84 + // by allowing an alternative provideStreamData to be
  85 + // implemented. You must implement at least one of the
  86 + // versions of provideStreamData below. If you implement the
  87 + // version that supports retry and returns a value, you should
  88 + // pass true as the value of supports_retry in the base class
  89 + // constructor. This will cause the library to call that
  90 + // version of the method, which should also return a boolean
  91 + // indicating whether it ran without errors.
  92 + QPDF_DLL
77 93 virtual void provideStreamData(int objid, int generation,
78   - Pipeline* pipeline) = 0;
  94 + Pipeline* pipeline);
  95 + QPDF_DLL
  96 + virtual bool provideStreamData(
  97 + int objid, int generation, Pipeline* pipeline,
  98 + bool suppress_warnings, bool will_retry);
  99 + QPDF_DLL
  100 + bool supportsRetry();
  101 +
  102 + private:
  103 + bool supports_retry;
79 104 };
80 105  
81 106 // The TokenFilter class provides a way to filter content streams
... ... @@ -779,18 +804,39 @@ class QPDFObjectHandle
779 804 // we determine that we know how to apply all requested filters,
780 805 // do so and return true if we are successful.
781 806 //
782   - // In all cases, a return value of true means that filtered data
783   - // has been written successfully. If filtering is requested but
784   - // this method returns false, it means there was some error in the
785   - // filtering, in which case the resulting data is likely partially
786   - // filtered and/or incomplete and may not be consistent with the
787   - // configured filters. QPDFWriter handles this by attempting to
788   - // get the stream data without filtering, but callers should
789   - // consider a false return value when decode_level is not
790   - // qpdf_dl_none to be a potential loss of data. If you intend to
791   - // retry in that case, pass true as the value of will_retry. This
792   - // changes the warning issued by the library to indicate that the
793   - // operation will be retried without filtering to avoid data loss.
  807 + // The exact meaning of the return value differs the different
  808 + // versions of this function, but for any version, the meaning has
  809 + // been the same. For the main version, added in qpdf 10, the
  810 + // return value indicates whether the overall operation succeeded.
  811 + // The filter parameter, if specified, will be set to whether or
  812 + // not filtering was attempted. If filtering was not requested,
  813 + // this value will be false even if the overall operation
  814 + // succeeded.
  815 + //
  816 + // If filtering is requested but this method returns false, it
  817 + // means there was some error in the filtering, in which case the
  818 + // resulting data is likely partially filtered and/or incomplete
  819 + // and may not be consistent with the configured filters.
  820 + // QPDFWriter handles this by attempting to get the stream data
  821 + // without filtering, but callers should consider a false return
  822 + // value when decode_level is not qpdf_dl_none to be a potential
  823 + // loss of data. If you intend to retry in that case, pass true as
  824 + // the value of will_retry. This changes the warning issued by the
  825 + // library to indicate that the operation will be retried without
  826 + // filtering to avoid data loss.
  827 +
  828 + // Return value is overall success, even if filtering is not
  829 + // requested.
  830 + QPDF_DLL
  831 + bool pipeStreamData(Pipeline*, bool* filtering_attempted,
  832 + int encode_flags,
  833 + qpdf_stream_decode_level_e decode_level,
  834 + bool suppress_warnings = false,
  835 + bool will_retry = false);
  836 +
  837 + // Legacy version. Return value is whether filtering was
  838 + // attempted. There is no way to determine success if filtering
  839 + // was not attempted.
794 840 QPDF_DLL
795 841 bool pipeStreamData(Pipeline*,
796 842 int encode_flags,
... ... @@ -804,6 +850,7 @@ class QPDFObjectHandle
804 850 // filter = true -> decode_level = qpdf_dl_generalized
805 851 // normalize = true -> encode_flags |= qpdf_sf_normalize
806 852 // compress = true -> encode_flags |= qpdf_sf_compress
  853 + // Return value is whether filtering was attempted.
807 854 QPDF_DLL
808 855 bool pipeStreamData(Pipeline*, bool filter,
809 856 bool normalize, bool compress);
... ...
libqpdf/QPDF.cc
  View file @893d38b
... ... @@ -68,27 +68,37 @@ QPDF::ForeignStreamData::ForeignStreamData(
68 68  
69 69 QPDF::CopiedStreamDataProvider::CopiedStreamDataProvider(
70 70 QPDF& destination_qpdf) :
  71 + QPDFObjectHandle::StreamDataProvider(true),
71 72 destination_qpdf(destination_qpdf)
72 73 {
73 74 }
74 75  
75   -void
  76 +bool
76 77 QPDF::CopiedStreamDataProvider::provideStreamData(
77   - int objid, int generation, Pipeline* pipeline)
  78 + int objid, int generation, Pipeline* pipeline,
  79 + bool suppress_warnings, bool will_retry)
78 80 {
79 81 PointerHolder<ForeignStreamData> foreign_data =
80 82 this->foreign_stream_data[QPDFObjGen(objid, generation)];
  83 + bool result = false;
81 84 if (foreign_data.getPointer())
82 85 {
83   - destination_qpdf.pipeForeignStreamData(
84   - foreign_data, pipeline, 0, qpdf_dl_none);
  86 + result = destination_qpdf.pipeForeignStreamData(
  87 + foreign_data, pipeline, suppress_warnings, will_retry);
  88 + QTC::TC("qpdf", "QPDF copy foreign with data",
  89 + result ? 0 : 1);
85 90 }
86 91 else
87 92 {
88 93 QPDFObjectHandle foreign_stream =
89 94 this->foreign_streams[QPDFObjGen(objid, generation)];
90   - foreign_stream.pipeStreamData(pipeline, 0, qpdf_dl_none);
  95 + result = foreign_stream.pipeStreamData(
  96 + pipeline, nullptr, 0, qpdf_dl_none,
  97 + suppress_warnings, will_retry);
  98 + QTC::TC("qpdf", "QPDF copy foreign with foreign_stream",
  99 + result ? 0 : 1);
91 100 }
  101 + return result;
92 102 }
93 103  
94 104 void
... ... @@ -2851,8 +2861,7 @@ bool
2851 2861 QPDF::pipeForeignStreamData(
2852 2862 PointerHolder<ForeignStreamData> foreign,
2853 2863 Pipeline* pipeline,
2854   - int encode_flags,
2855   - qpdf_stream_decode_level_e decode_level)
  2864 + bool suppress_warnings, bool will_retry)
2856 2865 {
2857 2866 if (foreign->encp->encrypted)
2858 2867 {
... ... @@ -2863,7 +2872,7 @@ QPDF::pipeForeignStreamData(
2863 2872 foreign->foreign_objid, foreign->foreign_generation,
2864 2873 foreign->offset, foreign->length,
2865 2874 foreign->local_dict, foreign->is_attachment_stream,
2866   - pipeline, false, false);
  2875 + pipeline, suppress_warnings, will_retry);
2867 2876 }
2868 2877  
2869 2878 void
... ...
libqpdf/QPDFObjectHandle.cc
  View file @893d38b
... ... @@ -36,6 +36,36 @@ class TerminateParsing
36 36 {
37 37 };
38 38  
  39 +QPDFObjectHandle::StreamDataProvider::StreamDataProvider(
  40 + bool supports_retry) :
  41 + supports_retry(supports_retry)
  42 +{
  43 +}
  44 +
  45 +void
  46 +QPDFObjectHandle::StreamDataProvider::provideStreamData(
  47 + int objid, int generation, Pipeline* pipeline)
  48 +{
  49 + throw std::logic_error(
  50 + "you must override provideStreamData -- see QPDFObjectHandle.hh");
  51 +}
  52 +
  53 +bool
  54 +QPDFObjectHandle::StreamDataProvider::provideStreamData(
  55 + int objid, int generation, Pipeline* pipeline,
  56 + bool suppress_warnings, bool will_retry)
  57 +{
  58 + throw std::logic_error(
  59 + "you must override provideStreamData -- see QPDFObjectHandle.hh");
  60 + return false;
  61 +}
  62 +
  63 +bool
  64 +QPDFObjectHandle::StreamDataProvider::supportsRetry()
  65 +{
  66 + return this->supports_retry;
  67 +}
  68 +
39 69 class CoalesceProvider: public QPDFObjectHandle::StreamDataProvider
40 70 {
41 71 public:
... ... @@ -1135,14 +1165,29 @@ QPDFObjectHandle::getRawStreamData()
1135 1165 }
1136 1166  
1137 1167 bool
1138   -QPDFObjectHandle::pipeStreamData(Pipeline* p,
  1168 +QPDFObjectHandle::pipeStreamData(Pipeline* p, bool* filtering_attempted,
1139 1169 int encode_flags,
1140 1170 qpdf_stream_decode_level_e decode_level,
1141 1171 bool suppress_warnings, bool will_retry)
1142 1172 {
1143 1173 assertStream();
1144 1174 return dynamic_cast<QPDF_Stream*>(obj.getPointer())->pipeStreamData(
1145   - p, encode_flags, decode_level, suppress_warnings, will_retry);
  1175 + p, filtering_attempted, encode_flags, decode_level,
  1176 + suppress_warnings, will_retry);
  1177 +}
  1178 +
  1179 +bool
  1180 +QPDFObjectHandle::pipeStreamData(Pipeline* p,
  1181 + int encode_flags,
  1182 + qpdf_stream_decode_level_e decode_level,
  1183 + bool suppress_warnings, bool will_retry)
  1184 +{
  1185 + assertStream();
  1186 + bool filtering_attempted;
  1187 + dynamic_cast<QPDF_Stream*>(obj.getPointer())->pipeStreamData(
  1188 + p, &filtering_attempted, encode_flags, decode_level,
  1189 + suppress_warnings, will_retry);
  1190 + return filtering_attempted;
1146 1191 }
1147 1192  
1148 1193 bool
... ...
libqpdf/QPDF_Stream.cc
  View file @893d38b
... ... @@ -163,7 +163,7 @@ PointerHolder&lt;Buffer&gt;
163 163 QPDF_Stream::getStreamData(qpdf_stream_decode_level_e decode_level)
164 164 {
165 165 Pl_Buffer buf("stream data buffer");
166   - if (! pipeStreamData(&buf, 0, decode_level, false, false))
  166 + if (! pipeStreamData(&buf, nullptr, 0, decode_level, false, false))
167 167 {
168 168 throw QPDFExc(qpdf_e_unsupported, qpdf->getFilename(),
169 169 "", this->offset,
... ... @@ -177,7 +177,12 @@ PointerHolder&lt;Buffer&gt;
177 177 QPDF_Stream::getRawStreamData()
178 178 {
179 179 Pl_Buffer buf("stream data buffer");
180   - pipeStreamData(&buf, 0, qpdf_dl_none, false, false);
  180 + if (! pipeStreamData(&buf, nullptr, 0, qpdf_dl_none, false, false))
  181 + {
  182 + throw QPDFExc(qpdf_e_unsupported, qpdf->getFilename(),
  183 + "", this->offset,
  184 + "error getting raw stream data");
  185 + }
181 186 QTC::TC("qpdf", "QPDF_Stream getRawStreamData");
182 187 return buf.getBuffer();
183 188 }
... ... @@ -467,7 +472,7 @@ QPDF_Stream::filterable(std::vector&lt;std::string&gt;&amp; filters,
467 472 }
468 473  
469 474 bool
470   -QPDF_Stream::pipeStreamData(Pipeline* pipeline,
  475 +QPDF_Stream::pipeStreamData(Pipeline* pipeline, bool* filterp,
471 476 int encode_flags,
472 477 qpdf_stream_decode_level_e decode_level,
473 478 bool suppress_warnings, bool will_retry)
... ... @@ -480,7 +485,14 @@ QPDF_Stream::pipeStreamData(Pipeline* pipeline,
480 485 bool early_code_change = true;
481 486 bool specialized_compression = false;
482 487 bool lossy_compression = false;
483   - bool filter = (! ((encode_flags == 0) && (decode_level == qpdf_dl_none)));
  488 + bool ignored;
  489 + if (filterp == nullptr)
  490 + {
  491 + filterp = &ignored;
  492 + }
  493 + bool& filter = *filterp;
  494 + filter = (! ((encode_flags == 0) && (decode_level == qpdf_dl_none)));
  495 + bool success = true;
484 496 if (filter)
485 497 {
486 498 filter = filterable(filters, specialized_compression, lossy_compression,
... ... @@ -505,6 +517,7 @@ QPDF_Stream::pipeStreamData(Pipeline* pipeline,
505 517 if (pipeline == 0)
506 518 {
507 519 QTC::TC("qpdf", "QPDF_Stream pipeStreamData with null pipeline");
  520 + // Return value is whether we can filter in this case.
508 521 return filter;
509 522 }
510 523  
... ... @@ -625,8 +638,21 @@ QPDF_Stream::pipeStreamData(Pipeline* pipeline,
625 638 else if (this->stream_provider.getPointer())
626 639 {
627 640 Pl_Count count("stream provider count", pipeline);
628   - this->stream_provider->provideStreamData(
629   - this->objid, this->generation, &count);
  641 + if (this->stream_provider->supportsRetry())
  642 + {
  643 + if (! this->stream_provider->provideStreamData(
  644 + this->objid, this->generation, &count,
  645 + suppress_warnings, will_retry))
  646 + {
  647 + filter = false;
  648 + success = false;
  649 + }
  650 + }
  651 + else
  652 + {
  653 + this->stream_provider->provideStreamData(
  654 + this->objid, this->generation, &count);
  655 + }
630 656 qpdf_offset_t actual_length = count.getCount();
631 657 qpdf_offset_t desired_length = 0;
632 658 if (this->stream_dict.hasKey("/Length"))
... ... @@ -674,6 +700,7 @@ QPDF_Stream::pipeStreamData(Pipeline* pipeline,
674 700 will_retry))
675 701 {
676 702 filter = false;
  703 + success = false;
677 704 }
678 705 }
679 706  
... ... @@ -704,7 +731,7 @@ QPDF_Stream::pipeStreamData(Pipeline* pipeline,
704 731 " for content normalization in the manual."));
705 732 }
706 733  
707   - return filter;
  734 + return success;
708 735 }
709 736  
710 737 void
... ...
libqpdf/qpdf/QPDF_Stream.hh
  View file @893d38b
... ... @@ -31,7 +31,7 @@ class QPDF_Stream: public QPDFObject
31 31 PointerHolder<QPDFObjectHandle::StreamDataProvider> getStreamDataProvider() const;
32 32  
33 33 // See comments in QPDFObjectHandle.hh for these methods.
34   - bool pipeStreamData(Pipeline*,
  34 + bool pipeStreamData(Pipeline*, bool* tried_filtering,
35 35 int encode_flags,
36 36 qpdf_stream_decode_level_e decode_level,
37 37 bool suppress_warnings, bool will_retry);
... ...
qpdf/qpdf.testcov
  View file @893d38b
... ... @@ -453,3 +453,5 @@ QPDFPageObjectHelper filter form xobject 0
453 453 qpdf found resources in non-leaf 0
454 454 qpdf found shared resources in leaf 0
455 455 qpdf found shared xobject in leaf 0
  456 +QPDF copy foreign with data 1
  457 +QPDF copy foreign with foreign_stream 1
... ...
qpdf/qtest/qpdf.test
  View file @893d38b
... ... @@ -1698,6 +1698,8 @@ my @sp_cases = (
1698 1698 [11, '%d in middle', '--encrypt u o 128 --', 'a-%d-split-out.zdf'],
1699 1699 [11, 'pdf extension', '', 'split-out.Pdf'],
1700 1700 [4, 'fallback', '--pages 11-pages.pdf 1-3 minimal.pdf --', 'split-out'],
  1701 + [1, 'broken data', '--pages broken-lzw.pdf --', 'split-out.pdf',
  1702 + {$td->FILE => "broken-lzw.out", $td->EXIT_STATUS => 3}],
1701 1703 );
1702 1704 $n_tests += 35;
1703 1705 $n_compare_pdfs += 1;
... ... @@ -1713,7 +1715,7 @@ $td-&gt;runtest(&quot;split page group &gt; 1&quot;,
1713 1715 $td->NORMALIZE_NEWLINES);
1714 1716 foreach my $f ('01-05', '06-10', '11-11')
1715 1717 {
1716   - $td->runtest("checkout group $f",
  1718 + $td->runtest("check out group $f",
1717 1719 {$td->FILE => "split-out-group-$f.pdf"},
1718 1720 {$td->FILE => "split-exp-group-$f.pdf"});
1719 1721 }
... ... @@ -1761,12 +1763,17 @@ foreach my $i (qw(01-10 11-20 21-30))
1761 1763  
1762 1764 foreach my $d (@sp_cases)
1763 1765 {
1764   - my ($n, $description, $xargs, $out) = @$d;
  1766 + my ($n, $description, $xargs, $out, $exp) = @$d;
  1767 + if (! defined $exp)
  1768 + {
  1769 + $exp = {$td->STRING => "", $td->EXIT_STATUS => 0};
  1770 + }
1765 1771 $td->runtest("split pages " . $description,
1766 1772 {$td->COMMAND =>
1767 1773 "qpdf --static-id --split-pages 11-pages.pdf" .
1768 1774 " $xargs $out"},
1769   - {$td->STRING => "", $td->EXIT_STATUS => 0});
  1775 + $exp,
  1776 + $td->NORMALIZE_NEWLINES);
1770 1777 my $pattern = $out;
1771 1778 my $nlen = length($n);
1772 1779 if ($pattern =~ m/\%d/)
... ... @@ -1786,7 +1793,7 @@ foreach my $d (@sp_cases)
1786 1793 my $actual = sprintf($pattern, $i);
1787 1794 my $expected = $actual;
1788 1795 $expected =~ s/split-out/split-exp/;
1789   - $td->runtest("checkout output page $i",
  1796 + $td->runtest("check output page $i ($description)",
1790 1797 {$td->FILE => $actual},
1791 1798 {$td->FILE => $expected});
1792 1799 }
... ... @@ -2390,7 +2397,8 @@ foreach my $d ([25, 1], [26, 2], [27, 3])
2390 2397 $td->runtest("copy objects $outn",
2391 2398 {$td->COMMAND => "test_driver $testn" .
2392 2399 " minimal.pdf copy-foreign-objects-in.pdf"},
2393   - {$td->STRING => "test $testn done\n", $td->EXIT_STATUS => 0},
  2400 + {$td->FILE => "copy-foreign-objects-$testn.out",
  2401 + $td->EXIT_STATUS => 0},
2394 2402 $td->NORMALIZE_NEWLINES);
2395 2403 $td->runtest("check output",
2396 2404 {$td->FILE => "a.pdf"},
... ...
qpdf/qtest/qpdf/broken-lzw.out 0 → 100644
  View file @893d38b
  1 +WARNING: broken-lzw.pdf (offset 444): error decoding stream data for object 4 0: LZWDecoder: bad code received
  2 +WARNING: broken-lzw.pdf (offset 444): stream will be re-processed without filtering to avoid data loss
  3 +qpdf: operation succeeded with warnings; resulting file may have some problems
... ...
qpdf/qtest/qpdf/broken-lzw.pdf 0 → 100644
  View file @893d38b
  1 +%PDF-1.3
  2 +%¿÷¢þ
  3 +%QDF-1.0
  4 +
  5 +1 0 obj
  6 +<<
  7 + /Pages 2 0 R
  8 + /Type /Catalog
  9 +>>
  10 +endobj
  11 +
  12 +2 0 obj
  13 +<<
  14 + /Count 1
  15 + /Kids [
  16 + 3 0 R
  17 + ]
  18 + /Type /Pages
  19 +>>
  20 +endobj
  21 +
  22 +%% Page 1
  23 +3 0 obj
  24 +<<
  25 + /Contents [ 4 0 R 6 0 R ]
  26 + /MediaBox [
  27 + 0
  28 + 0
  29 + 612
  30 + 792
  31 + ]
  32 + /Parent 2 0 R
  33 + /Resources <<
  34 + /Font <<
  35 + /F1 8 0 R
  36 + >>
  37 + /ProcSet 9 0 R
  38 + >>
  39 + /Type /Page
  40 +>>
  41 +endobj
  42 +
  43 +%% Contents for page 1
  44 +4 0 obj
  45 +<<
  46 + /Filter /LZWDecode
  47 + /Length 5 0 R
  48 +>>
  49 +stream
  50 +Not really compressed.
  51 +endstream
  52 +endobj
  53 +
  54 +5 0 obj
  55 +23
  56 +endobj
  57 +
  58 +%% Contents for page 1
  59 +6 0 obj
  60 +<<
  61 + /Length 7 0 R
  62 +>>
  63 +stream
  64 +Really compressed.
  65 +endstream
  66 +endobj
  67 +
  68 +7 0 obj
  69 +19
  70 +endobj
  71 +
  72 +8 0 obj
  73 +<<
  74 + /BaseFont /Helvetica
  75 + /Encoding /WinAnsiEncoding
  76 + /Name /F1
  77 + /Subtype /Type1
  78 + /Type /Font
  79 +>>
  80 +endobj
  81 +
  82 +9 0 obj
  83 +[
  84 + /PDF
  85 + /Text
  86 +]
  87 +endobj
  88 +
  89 +xref
  90 +0 10
  91 +0000000000 65535 f
  92 +0000000025 00000 n
  93 +0000000079 00000 n
  94 +0000000161 00000 n
  95 +0000000386 00000 n
  96 +0000000485 00000 n
  97 +0000000527 00000 n
  98 +0000000601 00000 n
  99 +0000000620 00000 n
  100 +0000000738 00000 n
  101 +trailer <<
  102 + /Root 1 0 R
  103 + /Size 10
  104 + /ID [<8b40e25fc6409c63043ed789f3ae4a4b><8b40e25fc6409c63043ed789f3ae4a4b>]
  105 +>>
  106 +startxref
  107 +773
  108 +%%EOF
... ...
qpdf/qtest/qpdf/copy-foreign-objects-25.out 0 → 100644
  View file @893d38b
  1 +test 25 done
... ...
qpdf/qtest/qpdf/copy-foreign-objects-26.out 0 → 100644
  View file @893d38b
  1 +test 26 done
... ...
qpdf/qtest/qpdf/copy-foreign-objects-27.out 0 → 100644
  View file @893d38b
  1 +WARNING: copy-foreign-objects-in.pdf (offset 3627): error decoding stream data for object 26 0: LZWDecoder: bad code received
  2 +WARNING: copy-foreign-objects-in.pdf (offset 3627): stream will be re-processed without filtering to avoid data loss
  3 +test 27 done
... ...
qpdf/qtest/qpdf/copy-foreign-objects-in.pdf
  View file @893d38b
... ... @@ -7,12 +7,13 @@
7 7 % file.
8 8  
9 9 % The /QTest key in trailer has pointers to several indirect objects:
10   -% O1, O2, O3 where O1 is an array that contains a dictionary that has
11   -% a key that points to O2, O2 is a dictionary that contains an array
12   -% that points to O1, and O3 is a page object that inherits some
13   -% resource from its parent /Pages and also points to some other page.
14   -% O1 also points to a stream whose dictionary has a key that points to
15   -% another stream whose dictionary points back to the first stream.
  10 +% O1, O2, O3, O4 where O1 is an array that contains a dictionary that
  11 +% has a key that points to O2, O2 is a dictionary that contains an
  12 +% array that points to O1, O3 is a page object that inherits some
  13 +% resource from its parent /Pages and also points to some other page,
  14 +% and O4 is a stream with invalid compressed data. O1 also points to a
  15 +% stream whose dictionary has a key that points to another stream
  16 +% whose dictionary points back to the first stream.
16 17  
17 18 1 0 obj
18 19 <<
... ... @@ -293,43 +294,59 @@ endobj
293 294  
294 295 % QTest
295 296 25 0 obj
296   -<< /This-is-QTest true /O1 19 0 R /O2 20 0 R /O3 5 0 R >>
  297 +<< /This-is-QTest true /O1 19 0 R /O2 20 0 R /O3 5 0 R /O4 26 0 R >>
  298 +endobj
  299 +
  300 +26 0 obj
  301 +<<
  302 + /Length 27 0 R
  303 + /Filter /LZWDecode
  304 +>>
  305 +stream
  306 +Not really compresed.
  307 +endstream
  308 +endobj
  309 +
  310 +27 0 obj
  311 +22
297 312 endobj
298 313  
299 314 xref
300   -0 26
  315 +0 28
301 316 0000000000 65535 f
302   -0000000655 00000 n
303   -0000000709 00000 n
304   -0000000845 00000 n
305   -0000001073 00000 n
306   -0000001313 00000 n
307   -0000001580 00000 n
308   -0000001839 00000 n
309   -0000002081 00000 n
310   -0000002183 00000 n
311   -0000002202 00000 n
312   -0000002334 00000 n
313   -0000002438 00000 n
314   -0000002481 00000 n
315   -0000002585 00000 n
316   -0000002628 00000 n
317   -0000002732 00000 n
318   -0000002775 00000 n
319   -0000002879 00000 n
320   -0000002904 00000 n
321   -0000003042 00000 n
322   -0000003138 00000 n
323   -0000003255 00000 n
324   -0000003285 00000 n
325   -0000003402 00000 n
326   -0000003430 00000 n
  317 +0000000706 00000 n
  318 +0000000760 00000 n
  319 +0000000896 00000 n
  320 +0000001124 00000 n
  321 +0000001364 00000 n
  322 +0000001631 00000 n
  323 +0000001890 00000 n
  324 +0000002132 00000 n
  325 +0000002234 00000 n
  326 +0000002253 00000 n
  327 +0000002385 00000 n
  328 +0000002489 00000 n
  329 +0000002532 00000 n
  330 +0000002636 00000 n
  331 +0000002679 00000 n
  332 +0000002783 00000 n
  333 +0000002826 00000 n
  334 +0000002930 00000 n
  335 +0000002955 00000 n
  336 +0000003093 00000 n
  337 +0000003189 00000 n
  338 +0000003306 00000 n
  339 +0000003336 00000 n
  340 +0000003453 00000 n
  341 +0000003481 00000 n
  342 +0000003567 00000 n
  343 +0000003667 00000 n
327 344 trailer <<
328 345 /Root 1 0 R
329   - /Size 26
  346 + /Size 28
330 347 /QTest 25 0 R
331 348 /ID [<d15f7aca3be584a96c1c94adb0931e71><9adb6b2fdb22e857340f7103917b16e4>]
332 349 >>
333 350 startxref
334   -3505
  351 +3687
335 352 %%EOF
... ...
qpdf/qtest/qpdf/copy-foreign-objects-out1.pdf
  View file @893d38b
... ... @@ -4,27 +4,33 @@
4 4 << /Pages 3 0 R /Type /Catalog >>
5 5 endobj
6 6 2 0 obj
7   -<< /O1 4 0 R /O2 5 0 R /This-is-QTest true >>
  7 +<< /O1 4 0 R /O2 5 0 R /O4 6 0 R /This-is-QTest true >>
8 8 endobj
9 9 3 0 obj
10   -<< /Count 1 /Kids [ 6 0 R ] /Type /Pages >>
  10 +<< /Count 1 /Kids [ 7 0 R ] /Type /Pages >>
11 11 endobj
12 12 4 0 obj
13   -[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 5 0 R >> 2.17828 ] >> /salad /O2 5 0 R /Stream1 7 0 R ]
  13 +[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 5 0 R >> 2.17828 ] >> /salad /O2 5 0 R /Stream1 8 0 R ]
14 14 endobj
15 15 5 0 obj
16 16 << /K1 [ 2.236 /O1 4 0 R 1.732 ] /O1 4 0 R /This-is-O2 true >>
17 17 endobj
18 18 6 0 obj
19   -<< /Contents 8 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 9 0 R >> /ProcSet 10 0 R >> /Type /Page >>
  19 +<< /Filter /LZWDecode /Length 22 >>
  20 +stream
  21 +Not really compresed.
  22 +endstream
20 23 endobj
21 24 7 0 obj
22   -<< /Stream2 11 0 R /This-is-Stream1 true /Length 18 >>
  25 +<< /Contents 9 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 10 0 R >> /ProcSet 11 0 R >> /Type /Page >>
  26 +endobj
  27 +8 0 obj
  28 +<< /Stream2 12 0 R /This-is-Stream1 true /Length 18 >>
23 29 stream
24 30 This is stream 1.
25 31 endstream
26 32 endobj
27   -8 0 obj
  33 +9 0 obj
28 34 << /Length 44 >>
29 35 stream
30 36 BT
... ... @@ -34,33 +40,34 @@ BT
34 40 ET
35 41 endstream
36 42 endobj
37   -9 0 obj
  43 +10 0 obj
38 44 << /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font >>
39 45 endobj
40   -10 0 obj
  46 +11 0 obj
41 47 [ /PDF /Text ]
42 48 endobj
43   -11 0 obj
44   -<< /Stream1 7 0 R /This-is-Stream2 true /Length 18 >>
  49 +12 0 obj
  50 +<< /Stream1 8 0 R /This-is-Stream2 true /Length 18 >>
45 51 stream
46 52 This is stream 2.
47 53 endstream
48 54 endobj
49 55 xref
50   -0 12
  56 +0 13
51 57 0000000000 65535 f
52 58 0000000015 00000 n
53 59 0000000064 00000 n
54   -0000000125 00000 n
55   -0000000184 00000 n
56   -0000000301 00000 n
57   -0000000379 00000 n
58   -0000000523 00000 n
59   -0000000628 00000 n
60   -0000000721 00000 n
61   -0000000828 00000 n
62   -0000000859 00000 n
63   -trailer << /QTest 2 0 R /Root 1 0 R /Size 12 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
  60 +0000000135 00000 n
  61 +0000000194 00000 n
  62 +0000000311 00000 n
  63 +0000000389 00000 n
  64 +0000000479 00000 n
  65 +0000000624 00000 n
  66 +0000000729 00000 n
  67 +0000000822 00000 n
  68 +0000000930 00000 n
  69 +0000000961 00000 n
  70 +trailer << /QTest 2 0 R /Root 1 0 R /Size 13 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
64 71 startxref
65   -964
  72 +1066
66 73 %%EOF
... ...
qpdf/qtest/qpdf/copy-foreign-objects-out2.pdf
  View file @893d38b
... ... @@ -4,39 +4,45 @@
4 4 << /Pages 3 0 R /Type /Catalog >>
5 5 endobj
6 6 2 0 obj
7   -<< /O1 4 0 R /O2 5 0 R /O3 6 0 R /This-is-QTest true >>
  7 +<< /O1 4 0 R /O2 5 0 R /O3 6 0 R /O4 7 0 R /This-is-QTest true >>
8 8 endobj
9 9 3 0 obj
10   -<< /Count 2 /Kids [ 7 0 R 6 0 R ] /Type /Pages >>
  10 +<< /Count 2 /Kids [ 8 0 R 6 0 R ] /Type /Pages >>
11 11 endobj
12 12 4 0 obj
13   -[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 5 0 R >> 2.17828 ] >> /salad /O2 5 0 R /Stream1 8 0 R ]
  13 +[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 5 0 R >> 2.17828 ] >> /salad /O2 5 0 R /Stream1 9 0 R ]
14 14 endobj
15 15 5 0 obj
16 16 << /K1 [ 2.236 /O1 4 0 R 1.732 ] /O1 4 0 R /This-is-O2 true >>
17 17 endobj
18 18 6 0 obj
19   -<< /Contents 9 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 10 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3 true /Type /Page >>
  19 +<< /Contents 10 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 11 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3 true /Type /Page >>
20 20 endobj
21 21 7 0 obj
22   -<< /Contents 11 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 12 0 R >> /ProcSet 13 0 R >> /Type /Page >>
  22 +<< /Filter /LZWDecode /Length 22 >>
  23 +stream
  24 +Not really compresed.
  25 +endstream
23 26 endobj
24 27 8 0 obj
25   -<< /Stream2 14 0 R /This-is-Stream1 true /Length 18 >>
  28 +<< /Contents 12 0 R /MediaBox [ 0 0 612 792 ] /Parent 3 0 R /Resources << /Font << /F1 13 0 R >> /ProcSet 14 0 R >> /Type /Page >>
  29 +endobj
  30 +9 0 obj
  31 +<< /Stream2 15 0 R /This-is-Stream1 true /Length 18 >>
26 32 stream
27 33 This is stream 1.
28 34 endstream
29 35 endobj
30   -9 0 obj
  36 +10 0 obj
31 37 << /Length 47 >>
32 38 stream
33 39 BT /F1 15 Tf 72 720 Td (Original page 2) Tj ET
34 40 endstream
35 41 endobj
36   -10 0 obj
  42 +11 0 obj
37 43 << /BaseFont /Times-Roman /Encoding /WinAnsiEncoding /Subtype /Type1 /Type /Font >>
38 44 endobj
39   -11 0 obj
  45 +12 0 obj
40 46 << /Length 44 >>
41 47 stream
42 48 BT
... ... @@ -46,36 +52,37 @@ BT
46 52 ET
47 53 endstream
48 54 endobj
49   -12 0 obj
  55 +13 0 obj
50 56 << /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font >>
51 57 endobj
52   -13 0 obj
  58 +14 0 obj
53 59 [ /PDF /Text ]
54 60 endobj
55   -14 0 obj
56   -<< /Stream1 8 0 R /This-is-Stream2 true /Length 18 >>
  61 +15 0 obj
  62 +<< /Stream1 9 0 R /This-is-Stream2 true /Length 18 >>
57 63 stream
58 64 This is stream 2.
59 65 endstream
60 66 endobj
61 67 xref
62   -0 15
  68 +0 16
63 69 0000000000 65535 f
64 70 0000000015 00000 n
65 71 0000000064 00000 n
66   -0000000135 00000 n
67   -0000000200 00000 n
68   -0000000317 00000 n
69   -0000000395 00000 n
70   -0000000577 00000 n
71   -0000000723 00000 n
72   -0000000828 00000 n
73   -0000000924 00000 n
74   -0000001024 00000 n
75   -0000001118 00000 n
76   -0000001226 00000 n
77   -0000001257 00000 n
78   -trailer << /QTest 2 0 R /Root 1 0 R /Size 15 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
  72 +0000000145 00000 n
  73 +0000000210 00000 n
  74 +0000000327 00000 n
  75 +0000000405 00000 n
  76 +0000000588 00000 n
  77 +0000000678 00000 n
  78 +0000000824 00000 n
  79 +0000000929 00000 n
  80 +0000001026 00000 n
  81 +0000001126 00000 n
  82 +0000001220 00000 n
  83 +0000001328 00000 n
  84 +0000001359 00000 n
  85 +trailer << /QTest 2 0 R /Root 1 0 R /Size 16 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
79 86 startxref
80   -1362
  87 +1464
81 88 %%EOF
... ...
qpdf/qtest/qpdf/copy-foreign-objects-out3.pdf
  View file @893d38b
... ... @@ -4,7 +4,7 @@
4 4 << /Pages 6 0 R /Type /Catalog >>
5 5 endobj
6 6 2 0 obj
7   -<< /O1 7 0 R /O2 8 0 R /O3 9 0 R /This-is-QTest true >>
  7 +<< /O1 7 0 R /O2 8 0 R /O3 9 0 R /O4 10 0 R /This-is-QTest true >>
8 8 endobj
9 9 3 0 obj
10 10 << /Length 20 >>
... ... @@ -25,39 +25,45 @@ more data for stream
25 25 endstream
26 26 endobj
27 27 6 0 obj
28   -<< /Count 3 /Kids [ 10 0 R 11 0 R 9 0 R ] /Type /Pages >>
  28 +<< /Count 3 /Kids [ 11 0 R 12 0 R 9 0 R ] /Type /Pages >>
29 29 endobj
30 30 7 0 obj
31   -[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 8 0 R >> 2.17828 ] >> /salad /O2 8 0 R /Stream1 12 0 R ]
  31 +[ /This-is-O1 /potato << /O2 [ 3.14159 << /O2 8 0 R >> 2.17828 ] >> /salad /O2 8 0 R /Stream1 13 0 R ]
32 32 endobj
33 33 8 0 obj
34 34 << /K1 [ 2.236 /O1 7 0 R 1.732 ] /O1 7 0 R /This-is-O2 true >>
35 35 endobj
36 36 9 0 obj
37   -<< /Contents 13 0 R /MediaBox [ 0 0 612 792 ] /OtherPage 11 0 R /Parent 6 0 R /Resources << /Font << /F1 14 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3 true /Type /Page >>
  37 +<< /Contents 14 0 R /MediaBox [ 0 0 612 792 ] /OtherPage 12 0 R /Parent 6 0 R /Resources << /Font << /F1 15 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3 true /Type /Page >>
38 38 endobj
39 39 10 0 obj
40   -<< /Contents 15 0 R /MediaBox [ 0 0 612 792 ] /Parent 6 0 R /Resources << /Font << /F1 16 0 R >> /ProcSet 17 0 R >> /Type /Page >>
  40 +<< /Filter /LZWDecode /Length 22 >>
  41 +stream
  42 +Not really compresed.
  43 +endstream
41 44 endobj
42 45 11 0 obj
43   -<< /Contents 18 0 R /MediaBox [ 0 0 612 792 ] /Parent 6 0 R /Resources << /Font << /F1 14 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3-other-page true /Type /Page >>
  46 +<< /Contents 16 0 R /MediaBox [ 0 0 612 792 ] /Parent 6 0 R /Resources << /Font << /F1 17 0 R >> /ProcSet 18 0 R >> /Type /Page >>
44 47 endobj
45 48 12 0 obj
46   -<< /Stream2 19 0 R /This-is-Stream1 true /Length 18 >>
  49 +<< /Contents 19 0 R /MediaBox [ 0 0 612 792 ] /Parent 6 0 R /Resources << /Font << /F1 15 0 R >> /ProcSet [ /PDF /Text ] >> /Rotate 180 /This-is-O3-other-page true /Type /Page >>
  50 +endobj
  51 +13 0 obj
  52 +<< /Stream2 20 0 R /This-is-Stream1 true /Length 18 >>
47 53 stream
48 54 This is stream 1.
49 55 endstream
50 56 endobj
51   -13 0 obj
  57 +14 0 obj
52 58 << /Length 47 >>
53 59 stream
54 60 BT /F1 15 Tf 72 720 Td (Original page 2) Tj ET
55 61 endstream
56 62 endobj
57   -14 0 obj
  63 +15 0 obj
58 64 << /BaseFont /Times-Roman /Encoding /WinAnsiEncoding /Subtype /Type1 /Type /Font >>
59 65 endobj
60   -15 0 obj
  66 +16 0 obj
61 67 << /Length 44 >>
62 68 stream
63 69 BT
... ... @@ -67,47 +73,48 @@ BT
67 73 ET
68 74 endstream
69 75 endobj
70   -16 0 obj
  76 +17 0 obj
71 77 << /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font >>
72 78 endobj
73   -17 0 obj
  79 +18 0 obj
74 80 [ /PDF /Text ]
75 81 endobj
76   -18 0 obj
  82 +19 0 obj
77 83 << /Length 47 >>
78 84 stream
79 85 BT /F1 15 Tf 72 720 Td (Original page 3) Tj ET
80 86 endstream
81 87 endobj
82   -19 0 obj
83   -<< /Stream1 12 0 R /This-is-Stream2 true /Length 18 >>
  88 +20 0 obj
  89 +<< /Stream1 13 0 R /This-is-Stream2 true /Length 18 >>
84 90 stream
85 91 This is stream 2.
86 92 endstream
87 93 endobj
88 94 xref
89   -0 20
  95 +0 21
90 96 0000000000 65535 f
91 97 0000000015 00000 n
92 98 0000000064 00000 n
93   -0000000135 00000 n
94   -0000000204 00000 n
95   -0000000259 00000 n
96   -0000000329 00000 n
97   -0000000402 00000 n
98   -0000000520 00000 n
99   -0000000598 00000 n
100   -0000000799 00000 n
101   -0000000946 00000 n
102   -0000001141 00000 n
103   -0000001247 00000 n
104   -0000001344 00000 n
105   -0000001444 00000 n
106   -0000001538 00000 n
107   -0000001646 00000 n
108   -0000001677 00000 n
109   -0000001774 00000 n
110   -trailer << /QTest 2 0 R /QTest2 [ 3 0 R 4 0 R 5 0 R ] /Root 1 0 R /Size 20 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
  99 +0000000146 00000 n
  100 +0000000215 00000 n
  101 +0000000270 00000 n
  102 +0000000340 00000 n
  103 +0000000413 00000 n
  104 +0000000531 00000 n
  105 +0000000609 00000 n
  106 +0000000810 00000 n
  107 +0000000901 00000 n
  108 +0000001048 00000 n
  109 +0000001243 00000 n
  110 +0000001349 00000 n
  111 +0000001446 00000 n
  112 +0000001546 00000 n
  113 +0000001640 00000 n
  114 +0000001748 00000 n
  115 +0000001779 00000 n
  116 +0000001876 00000 n
  117 +trailer << /QTest 2 0 R /QTest2 [ 3 0 R 4 0 R 5 0 R ] /Root 1 0 R /Size 21 /ID [<31415926535897932384626433832795><31415926535897932384626433832795>] >>
111 118 startxref
112   -1880
  119 +1982
113 120 %%EOF
... ...
qpdf/qtest/qpdf/split-exp-1.pdf 0 → 100644
View file @893d38b
No preview for this file type
qpdf/test_driver.cc
  View file @893d38b
... ... @@ -1210,7 +1210,8 @@ void runtest(int n, char const* filename1, char const* arg2)
1210 1210  
1211 1211 QPDFWriter w(pdf, "a.pdf");
1212 1212 w.setStaticID(true);
1213   - w.setStreamDataMode(qpdf_s_preserve);
  1213 + w.setCompressStreams(false);
  1214 + w.setDecodeLevel(qpdf_dl_generalized);
1214 1215 w.write();
1215 1216 }
1216 1217 else if (n == 28)
... ...