OpenSystemsDevelopment / qpdf

StreamDataProvider::provideStreamData now has a rich enough API for it
to effectively proxy to pipeStreamData.

It's detected in QPDFWriter instead of at parse time because I can't
figure out how to construct a test case in a reasonable time. This
commit moves the fuzz file into the regular test suite for a QTC
coverage case.

This makes all integer type conversions that have potential data loss
explicit with calls that do range checks and raise an exception. After
this commit, qpdf builds with no warnings when -Wsign-conversion
-Wconversion is used with gcc or clang or when -W3 -Wd4800 is used
with MSVC. This significantly reduces the likelihood of potential
crashes from bogus integer values.

There are some parts of the code that take int when they should take
size_t or an offset. Such places would make qpdf not support files
with more than 2^31 of something that usually wouldn't be so large. In
the event that such a file shows up and is valid, at least qpdf would
raise an error in the right spot so the issue could be legitimately
addressed rather than failing in some weird way because of a silent
overflow condition.

Change from unsigned long to int since we pass enumerated type values
to this field.

On read, ignore /DecodeParms when empty list; on write, delete it.
Some files have been found that include an empty list for
/DecodeParms, but this is not technically compliant with the spec, and
the only sensible interpretation is to treat it as if there are no
decode parameters.

The original QPDF is only required now when the source
QPDFObjectHandle is a stream that gets its stream data from a
QPDFObjectHandle::StreamDataProvider.

Give objects descriptions and context so it is possible to issue
warnings instead of fatal errors for attempts to access objects of the
wrong type.

Expose Pl_QPDFTokenizer, and have it do more of the work of managing
the token filter's pipeline.

Implement a TokenFilter class and refactor Pl_QPDFTokenizer to use a
TokenFilter class called ContentNormalizer. Pl_QPDFTokenizer is now a
general filter that passes data through a TokenFilter.

If the stream isn't filterable but we call getStreamData, throw a
regular exception instead of a logic error so that normal error
handling and reporting mechanisms will be used.

Tweak the message so that we inform the user that we are mitigating
data loss.

This commit adds several API methods that enable control over which
types of filters QPDF will attempt to decode. It also adds support for
/RunLengthDecode and /DCTDecode filters for both encoding and
decoding.

When requested, QPDFWriter will do more aggress prechecking of streams
to make sure it can actually succeed in decoding them before
attempting to do so. This will allow preservation of raw data even
when the raw data is corrupted relative to the specified filters.

For std::string and std::vector, replace operator[] with at.  This was
done using an automated process.  See README.hardening for details.

Add virtual methods to QPDFObject, wrappers to QPDFObjectHandle, and
implementations to all the QPDF_Object types.

Test cases added in a future commit since they depend on /R=6 support.

Move object parsing code from QPDF to QPDFObjectHandle and
parameterize the parts of it that are specific to a QPDF object.
Provide a version that can't handle indirect objects and that can be
called on an arbitrary string.

A side effect of this change is that the offset used when reporting
invalid stream length has changed, but since the new value seems like
a better value than the old one, the test suite has been updated
rather than making the code backward compatible.  This only effects
the offset reported for invalid streams that lack /Length or have an
invalid /Length key.

Updated some test code and exmaples to use QPDFObjectHandle::parse.

Supporting changes include adding a BufferInputSource constructor that
takes a string.

Breaking API change: length parameter has disappeared from the
StreamDataProvider version of QPDFObjectHandle::replaceStreamData
since it is no longer necessary to compute it in advance.  This
breaking change is justified by the fact that removing the length
parameter provides the caller an opportunity to simplify the calling
code.

off_t is used internally only when needed to talk to standard
libraries.  This requires that the "long long" type be supported by
the compiler.

Significantly improve the code's use of off_t for file offsets, size_t
for memory sizes, and integer types in cases where there has to be
compatibility with external interfaces.  Rework sections of the code
that would have prevented qpdf from working on files larger than 2 (or
maybe 4) GB in size.

git-svn-id: svn+q:///qpdf/trunk@1031 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@1025 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@1010 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@991 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@990 71b93d88-0707-0410-a8cf-f5a4172ac649

…mented but not tested

git-svn-id: svn+q:///qpdf/trunk@988 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@987 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@829 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@827 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@822 71b93d88-0707-0410-a8cf-f5a4172ac649