OpenSystemsDevelopment / qpdf

This is CVE-2017-9208.

The QPDF library uses object ID 0 internally as a sentinel to
represent a direct object, but prior to this fix, was not blocking
handling of 0 0 obj or 0 0 R as a special case. Creating an object in
the file with 0 0 obj could cause various infinite loops. The PDF spec
doesn't allow for object 0. Having qpdf handle object 0 might be a
better fix, but changing all the places in the code that assumes objid
== 0 means direct would be risky.

This is CVE-2017-9209.

This is CVE-2017-9210.

The description string for an error message included unparsing an
object, which is too complex of a thing to try to do while throwing an
exception. There was only one example of this in the entire codebase,
so it is not a pervasive problem. Fixing this eliminated one class of
infinite loop errors.

The 64 Bit file functions are supported by C++-Builder as well and
need to be used, else fseek will error out on larger files than 4 GB
like used in the large file test.

Also update maintainer documentation on binary compatibility testing.

Since we have to bump soname, remove some private methods that were
just there for binary compatibility

If xref table entries lack the spec-required trailing whitespace or
contain a small amount of extra space, handle them anyway.

For non-encrypted files, determinstic ID generation uses file contents
instead of timestamp and file name. At a small runtime cost, this
enables generation of the same /ID if the same inputs are converted in
the same way multiple times.

As reported in issue #40, a call to CryptAcquireContext in
SecureRandomDataProvider fails in a fresh windows install prior to any
user keys being created in AppData\Roaming\Microsoft\Crypto\RSA.

Thanks michalrames.

Pushing inherited objects to pages and getting all pages were both
prone to stack overflow infinite loops if there were loops in the
Pages dictionary. There is a general weakness in the code in that any
part of the code that traverses the Pages structure would be prone to
this and would have to implement its own loop detection. A more robust
fix may provide some general method for handling the Pages structure,
but it's probably not worth doing.

Note: addition of *Internal2 private functions was done rather than
changing signatures of existing methods to avoid breaking
compatibility.

Converting a password to an encryption key is supposed to copy up to a
certain number of bytes from a digest. Make sure never to copy more
than the size of the digest.

When checking two objects preceding R while parsing, ensure that the
objects are direct. This avoids stuff like 1 0 obj containing 1 0 R 0 R
from causing an infinite loop in object resolution.

Original reported here:
https://bugs.launchpad.net/ubuntu/+source/qpdf/+bug/1397413

The PDF specification says that the /Type key for nodes in the pages
dictionary (both /Page and /Pages) is required, but some PDF files
omit them. Use the presence of other keys to determine the type of
pages tree node this is if the type key is not found.

The spec allows /Contents to be omitted for pages that are blank, but
QPDFObjectHandle::getPageContents() was throwing an exception in this
case.

QPDFWriter was trying to make /Filter and /DecodeParms direct in all
cases, but there are some cases where /DecodeParms may refer to a
stream, which can't be direct. QPDFWriter doesn't actually need
/DecodeParms to be direct in that case because it won't be able to
filter the stream. Until we can handle this type of stream, just don't
make /Filter and /DecodeParms direct if we can't filter the stream
anyway.

Fixes #34

Fix problem: if the last object in the first part of a linearized file
had an offset that was below 65536 by less than the size of the hint
stream, the xref stream was invalid and the resulting file is not
usable.

This is a performance fix.  The output is unchanged.

Fixes #28.

Add a method to get the current random data provider, and document and
test the method for resetting it.

Also remove some trivial, non-functional code.

Add new RandomDataProvider object and implement existing random number
generation in terms of that.  This enables end users to supply their
own random data providers.

Fixes #27.

If NO_GET_ENVIRONMENT is #defined at compile time on Windows, do not
call GetEnvironmentVariable.  QUtil::get_env will always return
false.  This option is not available through configure.  This was
added to support a specific user's requirements to avoid calling
GetEnvironmentVariable from the Windows API.  Nothing in qpdf outside
the test coverage system in qtest relies on QUtil::get_env.

Also accept -accessibility=n with 256 bit keys even though it will be
ignored.

For std::string and std::vector, replace operator[] with at.  This was
done using an automated process.  See README.hardening for details.

If not available, give an error.  The user may also configure qpdf to
use an insecure random number generator.

Ideally, the library should never call assert outside of test code,
but it does in several places.  For some cases where the assertion
might conceivably fail because of a problem with the input data,
replace assertions with exceptions so that they can be trapped by the
calling application.  This commit surely misses some cases and
replaced some cases unnecessarily, but it should still be an
improvement.

In places where std::vector<T>(size_t) was used, either validate that
the size parameter is sane or refactor code to avoid the need to
pre-allocate the vector.