OpenSystemsDevelopment / qpdf

Give objects descriptions and context so it is possible to issue
warnings instead of fatal errors for attempts to access objects of the
wrong type.

Implement a TokenFilter class and refactor Pl_QPDFTokenizer to use a
TokenFilter class called ContentNormalizer. Pl_QPDFTokenizer is now a
general filter that passes data through a TokenFilter.

This is for protection against future ABI breaking changes.

Significant enhancements to the lexer to improve EOF handling and to
support comments and spaces as tokens. Various other minor issues were
fixed as well.

While scanning the file looking for objects, limit the length of
tokens we allow. This prevents us from getting caught up in reading a
file character by character while digging through large streams.

Sometimes we want to ignore bad tokens rather than having them throw
an exception. A coverage case is commented out here and added in a
later commit.

Required for strtol()

For std::string and std::vector, replace operator[] with at.  This was
done using an automated process.  See README.hardening for details.

This method allows parsing of the PDF objects in a content stream or
array of content streams.

Previously only whitespace and comments did it.  This fix is needed
for object streams whose last object is a literal (name, integer,
real, string) not terminated by space or newline.

Significantly improve the code's use of off_t for file offsets, size_t
for memory sizes, and integer types in cases where there has to be
compatibility with external interfaces.  Rework sections of the code
that would have prevented qpdf from working on files larger than 2 (or
maybe 4) GB in size.

git-svn-id: svn+q:///qpdf/trunk@796 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@709 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@688 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@651 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@607 71b93d88-0707-0410-a8cf-f5a4172ac649

git-svn-id: svn+q:///qpdf/trunk@599 71b93d88-0707-0410-a8cf-f5a4172ac649