Reflow TODO.md to line length 100

m-holger
1 parent f13947de
Showing 1 changed file with 485 additions and 655 deletions
TODO.md
@@ -21,105 +21,85 @@ Contents
 Always
 ======
-* Evaluate issues tagged with `next` and `bug`. Remember to check
-  discussions and pull requests in addition to regular issues.
-* When close to release, make sure external-libs is building and
-  follow instructions in ../external-libs/README
+* Evaluate issues tagged with `next` and `bug`. Remember to check discussions and pull requests in
+  addition to regular issues.
+* When close to release, make sure external-libs is building and follow instructions in
+  ../external-libs/README
 Next
 ====
-* Fix #874 -- make args in --encrypt to match the json and make
-  positional fill in the gaps
+* Fix #874 -- make args in --encrypt to match the json and make positional fill in the gaps
 * Maybe fix #553 -- use file times for attachments
 * std::string_view transition -- work being done by m-holger
-* Break ground on "Document-level work" -- TODO-pages.md lives on a
-  separate branch.
-* Standard for CLI and Job JSON support for JSON-based command-line
-  arguments. Come up with a standard way of supporting command-line
-  arguments that take JSON specifications of things so that
-  * there is a predictable way to indicate whether an argument is a
-    file or a JSON blob
-  * with QPDFJob JSON, make sure it is possible to directly include
-    the JSON rather than having to stringify a JSON blob
-  * One option might be to prepend file:// to a filename or otherwise
-    to take a JSON blob. We could have that as a particular type of
-    argument that would behave properly for both job JSON and CLI.
-
+* Break ground on "Document-level work" -- TODO-pages.md lives on a separate branch.
+* Standard for CLI and Job JSON support for JSON-based command-line arguments. Come up with a
+  standard way of supporting command-line arguments that take JSON specifications of things so that
+  * there is a predictable way to indicate whether an argument is a file or a JSON blob
+  * with QPDFJob JSON, make sure it is possible to directly include the JSON rather than having to
+    stringify a JSON blob
+  * One option might be to prepend file:// to a filename or otherwise to take a JSON blob. We could
+    have that as a particular type of argument that would behave properly for both job JSON and CLI.
 Possible future JSON enhancements
 =================================
-* Consider not including unreferenced objects and trimming the trailer
-  in the same way that QPDFWriter does (except don't remove `/ID`).
-  This means excluding the linearization dictionary and hint stream,
-  the encryption dictionary, all keys from trailer that are removed by
-  QPDFWriter::getTrimmedTrailer except `/ID`, any object streams, and
-  the xref stream as long as all those objects are unreferenced. (They
-  always should be, but there could be some bizarre case of someone
-  creating a PDF file that has an indirect reference to one of those,
-  in which case we need to preserve it.) If this is done, make
-  `--preserve-unreferenced` preserve unreference objects and also
-  those extra keys. Search for "linear" and "trailer" in json.rst to
-  update the various places in the documentation that discuss this.
-  Also update the help for --json and --preserve-unreferenced.
-
-* Add to JSON output the information available from a few additional
-  informational options:
+* Consider not including unreferenced objects and trimming the trailer in the same way that
+  QPDFWriter does (except don't remove `/ID`). This means excluding the linearization dictionary and
+  hint stream, the encryption dictionary, all keys from trailer that are removed by QPDFWriter::
+  getTrimmedTrailer except `/ID`, any object streams, and the xref stream as long as all those
+  objects are unreferenced. (They always should be, but there could be some bizarre case of someone
+  creating a PDF file that has an indirect reference to one of those, in which case we need to
+  preserve it.) If this is done, make
+  `--preserve-unreferenced` preserve unreference objects and also those extra keys. Search for "
+  linear" and "trailer" in json.rst to update the various places in the documentation that discuss
+  this. Also update the help for --json and --preserve-unreferenced.
+
+* Add to JSON output the information available from a few additional informational options:
   * --check: add but maybe not by default?
-  * --show-linearization: add but maybe not by default? Also figure
-    out whether warnings reported for some of the PDF specs (1.7) are
-    qpdf problems. This may not be worth adding in the first
+  * --show-linearization: add but maybe not by default? Also figure out whether warnings reported
+    for some of the PDF specs (1.7) are qpdf problems. This may not be worth adding in the first
     increment.
   * --show-xref: add
-* Consider having --check, --show-encryption, etc., just select the
-  right keys when in json mode. I don't think I want check on by
-  default, so that might be different.
+* Consider having --check, --show-encryption, etc., just select the right keys when in json mode. I
+  don't think I want check on by default, so that might be different.
-* Consider having warnings be included in the json in a "warnings" key
-  in json mode.
+* Consider having warnings be included in the json in a "warnings" key in json mode.
 QPDFJob
 =======
-Here are some ideas for QPDFJob that didn't make it into 10.6. Not all
-of these are necessarily good -- just things to consider.
-
-* How do we chain jobs? The idea would be that the input and/or output
-  of a QPDFJob could be a QPDF object rather than a file. For input,
-  it's pretty easy. For output, none of the output-specific options
-  (encrypt, compress-streams, objects-streams, etc.) would have any
-  affect, so we would have to treat this like inspect for error
-  checking. The QPDF object in the state where it's ready to be sent
-  off to QPDFWriter would be used as the input to the next QPDFJob.
-  For the job json, I think we can have the output be an identifier
-  that can be used as the input for another QPDFJob. For a json file,
-  we could the top level detect if it's an array with the convention
-  that exactly one has an output, or we could have a subkey with other
-  job definitions or something. Ideally, any input
-  (copy-attachments-from, pages, etc.) could use a QPDF object. It
-  wouldn't surprise me if this exposes bugs in qpdf around foreign
-  streams as this has been a relatively fragile area before.
+Here are some ideas for QPDFJob that didn't make it into 10.6. Not all of these are necessarily
+good -- just things to consider.
+
+* How do we chain jobs? The idea would be that the input and/or output of a QPDFJob could be a QPDF
+  object rather than a file. For input, it's pretty easy. For output, none of the output-specific
+  options
+  (encrypt, compress-streams, objects-streams, etc.) would have any affect, so we would have to
+  treat this like inspect for error checking. The QPDF object in the state where it's ready to be
+  sent off to QPDFWriter would be used as the input to the next QPDFJob. For the job json, I think
+  we can have the output be an identifier that can be used as the input for another QPDFJob. For a
+  json file, we could the top level detect if it's an array with the convention that exactly one has
+  an output, or we could have a subkey with other job definitions or something. Ideally, any input
+  (copy-attachments-from, pages, etc.) could use a QPDF object. It wouldn't surprise me if this
+  exposes bugs in qpdf around foreign streams as this has been a relatively fragile area before.
 Documentation
 =============
 * Do a full pass through the documentation.
-  * Make sure `qpdf` is consistent. Use QPDF when just referring to
-    the package.
+  * Make sure `qpdf` is consistent. Use QPDF when just referring to the package.
   * Make sure markup is consistent
   * Autogenerate where possible
-  * Consider which parts might be good candidates for moving to the
-    wiki.
+  * Consider which parts might be good candidates for moving to the wiki.
-* Commit 'Manual - enable line wrapping in table cells' from
-  Mon Jan 17 12:22:35 2022 +0000 enables table cell wrapping. See if
-  this can be incorporated directly into sphinx_rtd_theme and the
+* Commit 'Manual - enable line wrapping in table cells' from Mon Jan 17 12:22:35 2022 +0000 enables
+  table cell wrapping. See if this can be incorporated directly into sphinx_rtd_theme and the
   workaround can be removed.
 * When possible, update the debian package to include docs again. See
@@ -130,76 +110,62 @@ Document-level work
 * Ideas here may by superseded by #593.
-* QPDFPageCopier -- object for moving pages around within files or
-  between files and performing various transformations. Reread/rewrite
+* QPDFPageCopier -- object for moving pages around within files or between files and performing
+  various transformations. Reread/rewrite
   _page-selection in the manual if needed.
   * Handle all the stuff of pages and split-pages
   * Do n-up, booklet, collation
   * Look through cli and see what else...flatten-*?
-  * See comments in QPDFPageDocumentHelper.hh for addPage -- search
-    for "a future version".
+  * See comments in QPDFPageDocumentHelper.hh for addPage -- search for "a future version".
   * Make it efficient for bulk operations
   * Make certain doc-level features selectable
-  * qpdf.cc should do all its page operations, including
-    overlay/underlay, splitting, and merging, using this
+  * qpdf.cc should do all its page operations, including overlay/underlay, splitting, and merging,
+    using this
   * There should also be example code
-* After doc-level checks are in, call --check on the output files in
-  the "Copy Annotations" tests.
+* After doc-level checks are in, call --check on the output files in the "Copy Annotations" tests.
-* Document-level checks. For example, for forms, make sure all form
-  fields point to an annotation on exactly one page as well as that
-  all widget annotations are associated with a form field. Hook this
-  into QPDFPageCopier as well as the doc helpers. Make sure it is
-  called from --check.
+* Document-level checks. For example, for forms, make sure all form fields point to an annotation on
+  exactly one page as well as that all widget annotations are associated with a form field. Hook
+  this into QPDFPageCopier as well as the doc helpers. Make sure it is called from --check.
 * See also issues tagged with "pages". Include closed issues.
-* Add flags to CLI to select which document-level options to
-  preserve or not preserve. We will probably need a pair of mutually
-  exclusive, repeatable options with a way to specify all, none, only
-  {x,y}, or all but {x,y}.
+* Add flags to CLI to select which document-level options to preserve or not preserve. We will
+  probably need a pair of mutually exclusive, repeatable options with a way to specify all, none,
+  only {x,y}, or all but {x,y}.
-* If a page contains a reference a file attachment annotation, when
-  that page is copied, if the file attachment appears in the top-level
-  EmbeddedFiles tree, that entry should be preserved in the
-  destination file. Otherwise, we probably will require the use of
-  --copy-attachments-from to preserve these. What will the strategy be
-  for deduplicating in the automatic case?
+* If a page contains a reference a file attachment annotation, when that page is copied, if the file
+  attachment appears in the top-level EmbeddedFiles tree, that entry should be preserved in the
+  destination file. Otherwise, we probably will require the use of --copy-attachments-from to
+  preserve these. What will the strategy be for deduplicating in the automatic case?
 Text Appearance Streams
 =======================
-This is a list of known issues with text appearance streams and things
-we might do about it.
-
-* For variable text, the spec says to pull any resources from /DR that
-  are referenced in /DA but if the resource dictionary already has
-  that resource, just use the one that's there. The current code looks
-  only for /Tf and adds it if needed. We might want to instead merge
-  /DR with resources and then remove anything that's unreferenced. We
-  have all the code required for that in ResourceFinder except
-  TfFinder also gets the font size, which ResourceFinder doesn't do.
-
-* There are things we are missing because we don't look at font
-  metrics. The code from TextBuilder (work) has almost everything in
-  it that is required. Once we have knowledge of character widths, we
-  can support quadding and multiline text fields (/Ff 4096), and we
-  can potentially squeeze text to fit into a field. For multiline,
-  first squeeze vertically down to the font height, then squeeze
-  horizontally with Tz. For single line, squeeze horizontally with Tz.
-  If we use Tz, issue a warning.
-
-* When mapping characters to widths, we will need to care about
-  character encoding. For built-in fonts, we can create a map from
-  Unicode code point to width and then go from the font's encoding to
-  unicode to the width. See misc/character-encoding/ (not on github)
-  and font metric information for the 14 standard fonts in my local
-  pdf-spec directory.
-
-* Once we know about character widths, we can correctly support
-  auto-sized variable text fields (0 Tf). If this is fixed, search for
+This is a list of known issues with text appearance streams and things we might do about it.
+
+* For variable text, the spec says to pull any resources from /DR that are referenced in /DA but if
+  the resource dictionary already has that resource, just use the one that's there. The current code
+  looks only for /Tf and adds it if needed. We might want to instead merge /DR with resources and
+  then remove anything that's unreferenced. We have all the code required for that in ResourceFinder
+  except TfFinder also gets the font size, which ResourceFinder doesn't do.
+
+* There are things we are missing because we don't look at font metrics. The code from TextBuilder (
+  work) has almost everything in it that is required. Once we have knowledge of character widths, we
+  can support quadding and multiline text fields (/Ff 4096), and we can potentially squeeze text to
+  fit into a field. For multiline, first squeeze vertically down to the font height, then squeeze
+  horizontally with Tz. For single line, squeeze horizontally with Tz. If we use Tz, issue a
+  warning.
+
+* When mapping characters to widths, we will need to care about character encoding. For built-in
+  fonts, we can create a map from Unicode code point to width and then go from the font's encoding
+  to unicode to the width. See misc/character-encoding/ (not on github)
+  and font metric information for the 14 standard fonts in my local pdf-spec directory.
+
+* Once we know about character widths, we can correctly support auto-sized variable text fields (0
+  Tf). If this is fixed, search for
   "auto-sized" in cli.rst.
 Fuzz Errors
@@ -215,367 +181,297 @@ External Libraries
 Current state (10.0.2):
-* qpdf/external-libs repository builds external-libs on a schedule.
-  It detects and downloads the latest versions of zlib, jpeg, and
-  openssl and creates source and binary distribution zip files in an
-  artifact called "distribution".
+* qpdf/external-libs repository builds external-libs on a schedule. It detects and downloads the
+  latest versions of zlib, jpeg, and openssl and creates source and binary distribution zip files in
+  an artifact called "distribution".
-* Releases in qpdf/external-libs are made manually. They contain
-  qpdf-external-libs-{bin,src}.zip.
+* Releases in qpdf/external-libs are made manually. They contain qpdf-external-libs-{bin,src}.zip.
-* The qpdf build finds the latest non-prerelease release and downloads
-  the qpdf-external-libs-*.zip files from the releases in the setup
-  stage.
+* The qpdf build finds the latest non-prerelease release and downloads the qpdf-external-libs-*.zip
+  files from the releases in the setup stage.
-* To upgrade to a new version of external-libs, create a new release
-  of qpdf/external-libs (see README-maintainer in external-libs) from
-  the distribution artifact of the most recent successful build after
-  ensuring that it works.
+* To upgrade to a new version of external-libs, create a new release of qpdf/external-libs (see
+  README-maintainer in external-libs) from the distribution artifact of the most recent successful
+  build after ensuring that it works.
 Desired state:
-* The qpdf/external-libs repository should create release candidates.
-  Ideally, every scheduled run would make its zip files available. A
-  personal access token with actions:read scope for the
-  qpdf/external-libs repository is required to download the artifact
-  from an action run, and qpdf/qpdf's secrets.GITHUB_TOKEN doesn't
-  have this access. We could create a service account for this
-  purpose. As an alternative, we could have a draft release in
-  qpdf/external-libs that the qpdf/external-libs build could update
-  with each candidate. It may also be possible to solve this by
-  developing a simple GitHub app.
-
-* Scheduled runs of the qpdf build in the qpdf/qpdf repository (not a
-  fork or pull request) could download external-libs from the release
-  candidate area instead of the latest stable release. Pushes to the
-  build branch should still use the latest release so it always
-  matches the main branch.
-
-* Periodically, we would create a release of external-libs from the
-  release candidate zip files. This could be done safely because we
-  know the latest qpdf works with it. This could be done at least
-  before every release of qpdf, but potentially it could be done at
-  other times, such as when a new dependency version is available or
-  after some period of time.
+* The qpdf/external-libs repository should create release candidates. Ideally, every scheduled run
+  would make its zip files available. A personal access token with actions:read scope for the
+  qpdf/external-libs repository is required to download the artifact from an action run, and
+  qpdf/qpdf's secrets.GITHUB_TOKEN doesn't have this access. We could create a service account for
+  this purpose. As an alternative, we could have a draft release in qpdf/external-libs that the
+  qpdf/external-libs build could update with each candidate. It may also be possible to solve this
+  by developing a simple GitHub app.
+
+* Scheduled runs of the qpdf build in the qpdf/qpdf repository (not a fork or pull request) could
+  download external-libs from the release candidate area instead of the latest stable release.
+  Pushes to the build branch should still use the latest release so it always matches the main
+  branch.
+
+* Periodically, we would create a release of external-libs from the release candidate zip files.
+  This could be done safely because we know the latest qpdf works with it. This could be done at
+  least before every release of qpdf, but potentially it could be done at other times, such as when
+  a new dependency version is available or after some period of time.
 Other notes:
-* The external-libs branch in qpdf/qpdf was never documented. We might
-  be able to get away with deleting it.
+* The external-libs branch in qpdf/qpdf was never documented. We might be able to get away with
+  deleting it.
-* See README-maintainer in qpdf/external-libs for information on
-  creating a release. This could be at least partially scripted in a
-  way that works for the qpdf/qpdf repository as well since they are
-  very similar.
+* See README-maintainer in qpdf/external-libs for information on creating a release. This could be
+  at least partially scripted in a way that works for the qpdf/qpdf repository as well since they
+  are very similar.
 ABI Changes
 ===========
-This is a list of changes to make next time there is an ABI change.
-Comments appear in the code prefixed by "ABI".
+This is a list of changes to make next time there is an ABI change. Comments appear in the code
+prefixed by "ABI".
 Always:
 * Search for ABI in source and header files
 * Search for "[[deprecated" to find deprecated APIs that can be removed
 * Search for issues, pull requests, and discussions with the "abi" label
-* Check discussion "qpdf X planning" where X is the next major
-  version. This should be tagged `abi`
+* Check discussion "qpdf X planning" where X is the next major version. This should be tagged `abi`
 For qpdf 12, see https://github.com/qpdf/qpdf/discussions/785
 C++ Version Changes
 ===================
-Use
-// C++NN: ...
-to mark places in the code that should be updated when we require at
-least that version of C++.
+Use // C++NN: ... to mark places in the code that should be updated when we require at least that
+version of C++.
 Page splitting/merging
 ======================
- * Update page splitting and merging to handle document-level
-   constructs with page impact such as interactive forms and article
-   threading. Check keys in the document catalog for others, such as
-   outlines, page labels, thumbnails, and zones. For threads,
-   Subramanyam provided a test file; see ../misc/article-threads.pdf.
-   Email Q-Count: 431864 from 2009-11-03.
-
- * bookmarks (outlines) 12.3.3
-   * support bookmarks when merging
-   * prune bookmarks that don't point to a surviving page when merging
-     or splitting
-   * make sure conflicting named destinations work possibly test by
-     including the same file by two paths in a merge
-   * see also comments in issue 343
-
-   Note: original implementation of bookmark preservation for split
-   pages caused a very high performance hit. The problem was
-   introduced in 313ba081265f69ac9a0324f9fe87087c72918191 and reverted
-   in the commit that adds this paragraph. The revert includes marking
-   a few tests cases as $td->EXPECT_FAILURE. When properly coded, the
-   test cases will need to be adjusted to only include the parts of
-   the outlines that are actually copied. The tests in question are
-   "split page with outlines". When implementing properly, ensure that
-   the performance is not adversely affected by timing split-pages on
-   a large file with complex outlines such as the PDF specification.
-
-   When pruning outlines, keep all outlines in the hierarchy that are
-   above an outline for a page we care about. If one of the ancestor
-   outlines points to a non-existent page, clear its dest. If an
-   outline does not have any children that point to pages in the
-   document, just omit it.
-
-   Possible strategy:
-   * resolve all named destinations to explicit destinations
-   * concatenate top-level outlines
-   * prune outlines whose dests don't point to a valid page
-   * recompute all /Count fields
-
-   Test files
-   * page-labels-and-outlines.pdf: old file with both page labels and
-     outlines. All destinations are explicit destinations. Each page
-     has Potato and a number. All titles are feline names.
-   * outlines-with-actions.pdf: mixture of explicit destinations,
-     named destinations, goto actions with explicit destinations, and
-     goto actions with named destinations; uses /Dests key in names
-     dictionary. Each page has Salad and a number. All titles are
-     silly words. One destination is an indirect object.
-   * outlines-with-old-root-dests.pdf: like outlines-with-actions
-     except it uses the PDF-1.1 /Dests dictionary for named
-     destinations, and each page has Soup and a number. Also pages are
-     numbered with upper-case Roman numerals starting with 0. All
-     titles are silly words preceded by a bullet.
-
-   If outline handling is significantly improved, see
-   ../misc/bad-outlines/bad-outlines.pdf and email:
-   https://mail.google.com/mail/u/0/#search/rfc822msgid%3A02aa01d3d013%249f766990%24de633cb0%24%40mono.hr)
-
- * Form fields: should be similar to outlines.
+* Update page splitting and merging to handle document-level constructs with page impact such as
+  interactive forms and article threading. Check keys in the document catalog for others, such as
+  outlines, page labels, thumbnails, and zones. For threads, Subramanyam provided a test file; see
+  ../misc/article-threads.pdf. Email Q-Count: 431864 from 2009-11-03.
+
+* bookmarks (outlines) 12.3.3
+  * support bookmarks when merging
+  * prune bookmarks that don't point to a surviving page when merging or splitting
+  * make sure conflicting named destinations work possibly test by including the same file by two
+    paths in a merge
+  * see also comments in issue 343
+
+  Note: original implementation of bookmark preservation for split pages caused a very high
+  performance hit. The problem was introduced in 313ba081265f69ac9a0324f9fe87087c72918191 and
+  reverted in the commit that adds this paragraph. The revert includes marking a few tests cases as
+  $td->EXPECT_FAILURE. When properly coded, the test cases will need to be adjusted to only include
+  the parts of the outlines that are actually copied. The tests in question are
+  "split page with outlines". When implementing properly, ensure that the performance is not
+  adversely affected by timing split-pages on a large file with complex outlines such as the PDF
+  specification.
+
+  When pruning outlines, keep all outlines in the hierarchy that are above an outline for a page we
+  care about. If one of the ancestor outlines points to a non-existent page, clear its dest. If an
+  outline does not have any children that point to pages in the document, just omit it.
+
+  Possible strategy:
+  * resolve all named destinations to explicit destinations
+  * concatenate top-level outlines
+  * prune outlines whose dests don't point to a valid page
+  * recompute all /Count fields
+
+  Test files
+  * page-labels-and-outlines.pdf: old file with both page labels and outlines. All destinations are
+    explicit destinations. Each page has Potato and a number. All titles are feline names.
+  * outlines-with-actions.pdf: mixture of explicit destinations, named destinations, goto actions
+    with explicit destinations, and goto actions with named destinations; uses /Dests key in names
+    dictionary. Each page has Salad and a number. All titles are silly words. One destination is an
+    indirect object.
+  * outlines-with-old-root-dests.pdf: like outlines-with-actions except it uses the PDF-1.1 /Dests
+    dictionary for named destinations, and each page has Soup and a number. Also pages are numbered
+    with upper-case Roman numerals starting with 0. All titles are silly words preceded by a bullet.
+
+  If outline handling is significantly improved, see ../misc/bad-outlines/bad-outlines.pdf and
+  email:
+  https://mail.google.com/mail/u/0/#search/rfc822msgid%3A02aa01d3d013%249f766990%24de633cb0%24%40mono.hr)
+
+* Form fields: should be similar to outlines.
 Analytics
 =========
-Consider features that make it easier to detect certain patterns in
-PDF files. The information below could be computed using an external
-program that reads the existing json, but if it's useful enough, we
-could add it directly to the json output.
+Consider features that make it easier to detect certain patterns in PDF files. The information below
+could be computed using an external program that reads the existing json, but if it's useful enough,
+we could add it directly to the json output.
- * Add to "pages" in the json:
-   * "inheritsresources": bool; whether there are any inherited
-     attributes from ancestor page tree nodes
-   * "sharedresources": a list of indirect objects that are
-     "/Resources" dictionaries or "XObject" resource dictionary subkeys
-     of either the page itself or of any form XObject referenced by the
-     page.
+* Add to "pages" in the json:
+  * "inheritsresources": bool; whether there are any inherited attributes from ancestor page tree
+    nodes
+  * "sharedresources": a list of indirect objects that are
+    "/Resources" dictionaries or "XObject" resource dictionary subkeys of either the page itself or
+    of any form XObject referenced by the page.
- * Add to "objectinfo" in json: "directpagerefcount": the number of
-   pages that directly reference this object (i.e., you can find an
-   indirect reference to the object in the page dictionary without
-   traversing over any indirect objects)
+* Add to "objectinfo" in json: "directpagerefcount": the number of pages that directly reference
+  this object (i.e., you can find an indirect reference to the object in the page dictionary without
+  traversing over any indirect objects)
 General
 =======
-NOTE: Some items in this list refer to files in my personal home
-directory or that are otherwise not publicly accessible. This includes
-things sent to me by email that are specifically not public. Even so,
-I find it useful to make reference to them in this list.
+NOTE: Some items in this list refer to files in my personal home directory or that are otherwise not
+publicly accessible. This includes things sent to me by email that are specifically not public. Even
+so, I find it useful to make reference to them in this list.
 * Consider enabling code scanning on GitHub.
-* Add an option --ignore-encryption to ignore encryption information
-  and treat encrypted files as if they weren't encrypted. This should
-  make it possible to solve #598 (--show-encryption without a
-  password). We'll need to make sure we don't try to filter any
-  streams in this mode. Ideally we should be able to combine this with
-  --json so we can look at the raw encrypted strings and streams if we
-  want to, though be sure to document that the resulting JSON won't be
-  convertible back to a valid PDF. Since providing the password may
-  reveal additional details, --show-encryption could potentially retry
-  with this option if the first time doesn't work. Then, with the file
-  open, we can read the encryption dictionary normally. If this is
-  done, search for "raw, encrypted" in json.rst.
-
-* In libtests, separate executables that need the object library
-  from those that strictly use public API. Move as many of the test
-  drivers from the qpdf directory into the latter category as long
-  as doing so isn't too troublesome from a coverage standpoint.
-
-* Consider generating a non-flat pages tree before creating output to
-  better handle files with lots of pages. If there are more than 256
-  pages, add a second layer with the second layer nodes having no more
-  than 256 nodes and being as evenly sizes as possible. Don't worry
-  about the case of more than 65,536 pages. If the top node has more
-  than 256 children, we'll live with it. This is only safe if all
-  intermediate page nodes have only /Kids, /Parent, /Type, and /Count.
+* Add an option --ignore-encryption to ignore encryption information and treat encrypted files as if
+  they weren't encrypted. This should make it possible to solve #598 (--show-encryption without a
+  password). We'll need to make sure we don't try to filter any streams in this mode. Ideally we
+  should be able to combine this with --json so we can look at the raw encrypted strings and streams
+  if we want to, though be sure to document that the resulting JSON won't be convertible back to a
+  valid PDF. Since providing the password may reveal additional details, --show-encryption could
+  potentially retry with this option if the first time doesn't work. Then, with the file open, we
+  can read the encryption dictionary normally. If this is done, search for "raw, encrypted" in
+  json.rst.
+
+* In libtests, separate executables that need the object library from those that strictly use public
+  API. Move as many of the test drivers from the qpdf directory into the latter category as long as
+  doing so isn't too troublesome from a coverage standpoint.
+
+* Consider generating a non-flat pages tree before creating output to better handle files with lots
+  of pages. If there are more than 256 pages, add a second layer with the second layer nodes having
+  no more than 256 nodes and being as evenly sizes as possible. Don't worry about the case of more
+  than 65,536 pages. If the top node has more than 256 children, we'll live with it. This is only
+  safe if all intermediate page nodes have only /Kids, /Parent, /Type, and /Count.
 * Look at https://bestpractices.coreinfrastructure.org/en
 * Consider adding fuzzer code for JSON
-* Rework tests so that nothing is written into the source directory.
-  Ideally then the entire build could be done with a read-only
-  source tree.
-
-* Large file tests fail with linux32 before and after cmake. This was
-  first noticed after 10.6.3. I don't think it's worth fixing.
-
-* Consider updating the fuzzer with code that exercises
-  copyAnnotations, file attachments, and name and number trees. Check
-  fuzzer coverage.
-
-* Add code for creation of a file attachment annotation. It should
-  also be possible to create a widget annotation and a form field.
-  Update the pdf-attach-file.cc example with new APIs when ready.
-
-* Flattening of form XObjects seems like something that would be
-  useful in the library. We are seeing more cases of completely valid
-  PDF files with form XObjects that cause problems in other software.
-  Flattening of form XObjects could be a useful way to work around
-  those issues or to prepare files for additional processing, making
-  it possible for users of the qpdf library to not be concerned about
-  form XObjects. This could be done recursively; i.e., we could have a
-  method to embed a form XObject into whatever contains it, whether
-  that is a form XObject or a page. This would require more
-  significant interpretation of the content stream. We would need a
-  test file in which the placement of the form XObject has to be in
-  the right place, e.g., the form XObject partially obscures earlier
-  code and is partially obscured by later code. Keys in the resource
-  dictionary may need to be changed -- create test cases with lots of
-  duplicated/overlapping keys.
-
-* Part of closed_file_input_source.cc is disabled on Windows because
-  of odd failures. It might be worth investigating so we can fully
-  exercise this in the test suite. That said, ClosedFileInputSource
-  is exercised elsewhere in qpdf's test suite, so this is not that
-  pressing.
-
-* If possible, consider adding CCITT3, CCITT4, or any other easy
-  filters. For some reference code that we probably can't use but may
-  be handy anyway, see
+* Rework tests so that nothing is written into the source directory. Ideally then the entire build
+  could be done with a read-only source tree.
+
+* Large file tests fail with linux32 before and after cmake. This was first noticed after 10.6.3. I
+  don't think it's worth fixing.
+
+* Consider updating the fuzzer with code that exercises copyAnnotations, file attachments, and name
+  and number trees. Check fuzzer coverage.
+
+* Add code for creation of a file attachment annotation. It should also be possible to create a
+  widget annotation and a form field. Update the pdf-attach-file.cc example with new APIs when
+  ready.
+
+* Flattening of form XObjects seems like something that would be useful in the library. We are
+  seeing more cases of completely valid PDF files with form XObjects that cause problems in other
+  software. Flattening of form XObjects could be a useful way to work around those issues or to
+  prepare files for additional processing, making it possible for users of the qpdf library to not
+  be concerned about form XObjects. This could be done recursively; i.e., we could have a method to
+  embed a form XObject into whatever contains it, whether that is a form XObject or a page. This
+  would require more significant interpretation of the content stream. We would need a test file in
+  which the placement of the form XObject has to be in the right place, e.g., the form XObject
+  partially obscures earlier code and is partially obscured by later code. Keys in the resource
+  dictionary may need to be changed -- create test cases with lots of duplicated/overlapping keys.
+
+* Part of closed_file_input_source.cc is disabled on Windows because of odd failures. It might be
+  worth investigating so we can fully exercise this in the test suite. That said,
+  ClosedFileInputSource is exercised elsewhere in qpdf's test suite, so this is not that pressing.
+
+* If possible, consider adding CCITT3, CCITT4, or any other easy filters. For some reference code
+  that we probably can't use but may be handy anyway, see
   http://partners.adobe.com/public/developer/ps/sdk/index_archive.html
 * If possible, support the following types of broken files:
-   - Files that have no whitespace token after "endobj" such that
-     endobj collides with the start of the next object
+  - Files that have no whitespace token after "endobj" such that endobj collides with the start of
+    the next object
-   - See ../misc/broken-files
+  - See ../misc/broken-files
-   - See ../misc/bad-files-issue-476. This directory contains a
-     snapshot of the google doc and linked PDF files from issue #476.
-     Please see the issue for details.
+  - See ../misc/bad-files-issue-476. This directory contains a snapshot of the google doc and linked
+    PDF files from issue #476. Please see the issue for details.
 * Additional form features
-  * set value from CLI? Specify title, and provide way to
-    disambiguate, probably by giving objgen of field
+  * set value from CLI? Specify title, and provide way to disambiguate, probably by giving objgen of
+    field
 * Pl_TIFFPredictor is pretty slow.
-* Support for handling file names with Unicode characters in Windows
-  is incomplete. qpdf seems to support them okay from a functionality
-  standpoint, and the right thing happens if you pass in UTF-8
-  encoded filenames to QPDF library routines in Windows (they are
-  converted internally to wchar_t*), but file names are encoded in
-  UTF-8 on output, which doesn't produce nice error messages or
-  output on Windows in some cases.
-
-* If we ever wanted to do anything more with character encoding, see
-  ../misc/character-encoding/, which includes machine-readable dump
-  of table D.2 in the ISO-32000 PDF spec. This shows the mapping
-  between Unicode, StandardEncoding, WinAnsiEncoding,
-  MacRomanEncoding, and PDFDocEncoding.
-
-* Some test cases on bad files fail because qpdf is unable to find
-  the root dictionary when it fails to read the trailer. Recovery
-  could find the root dictionary and even the info dictionary in
-  other ways. In particular, issue-202.pdf can be opened by evince,
-  and there's no real reason that qpdf couldn't be made to be able to
-  recover that file as well.
-
-* Audit every place where qpdf allocates memory to see whether there
-  are cases where malicious inputs could cause qpdf to attempt to
-  grab very large amounts of memory. Certainly there are cases like
-  this, such as if a very highly compressed, very large image stream
-  is requested in a buffer. Hopefully normal input to output
-  filtering doesn't ever try to do this. QPDFWriter should be checked
-  carefully too. See also bugs/private/from-email-663916/
+* Support for handling file names with Unicode characters in Windows is incomplete. qpdf seems to
+  support them okay from a functionality standpoint, and the right thing happens if you pass in
+  UTF-8 encoded filenames to QPDF library routines in Windows (they are converted internally to
+  wchar_t*), but file names are encoded in UTF-8 on output, which doesn't produce nice error
+  messages or output on Windows in some cases.
+
+* If we ever wanted to do anything more with character encoding, see ../misc/character-encoding/,
+  which includes machine-readable dump of table D.2 in the ISO-32000 PDF spec. This shows the
+  mapping between Unicode, StandardEncoding, WinAnsiEncoding, MacRomanEncoding, and PDFDocEncoding.
+
+* Some test cases on bad files fail because qpdf is unable to find the root dictionary when it fails
+  to read the trailer. Recovery could find the root dictionary and even the info dictionary in other
+  ways. In particular, issue-202.pdf can be opened by evince, and there's no real reason that qpdf
+  couldn't be made to be able to recover that file as well.
+
+* Audit every place where qpdf allocates memory to see whether there are cases where malicious
+  inputs could cause qpdf to attempt to grab very large amounts of memory. Certainly there are cases
+  like this, such as if a very highly compressed, very large image stream is requested in a buffer.
+  Hopefully normal input to output filtering doesn't ever try to do this. QPDFWriter should be
+  checked carefully too. See also bugs/private/from-email-663916/
 * Interactive form modification:
-  https://github.com/qpdf/qpdf/issues/213 contains a good discussion
-  of some ideas for adding methods to modify annotations and form
-  fields if we want to make it easier to support modifications to
-  interactive forms. Some of the ideas have been implemented, and
-  some of the probably never will be implemented, but it's worth a
-  read if there is an intention to work on this. In the issue, search
-  for "Regarding write functionality", and read that comment and the
+  https://github.com/qpdf/qpdf/issues/213 contains a good discussion of some ideas for adding
+  methods to modify annotations and form fields if we want to make it easier to support
+  modifications to interactive forms. Some of the ideas have been implemented, and some of the
+  probably never will be implemented, but it's worth a read if there is an intention to work on
+  this. In the issue, search for "Regarding write functionality", and read that comment and the
   responses to it.
 * Look at ~/Q/pdf-collection/forms-from-appian/
-* When decrypting files with /R=6, hash_V5 is called more than once
-  with the same inputs.  Caching the results or refactoring to reduce
-  the number of identical calls could improve performance for
+* When decrypting files with /R=6, hash_V5 is called more than once with the same inputs. Caching
+  the results or refactoring to reduce the number of identical calls could improve performance for
   workloads that involve processing large numbers of small files.
-* Consider adding a method to balance the pages tree.  It would call
-  pushInheritedAttributesToPage, construct a pages tree from scratch,
-  and replace the /Pages key of the root dictionary with the new
-  tree.
-
-* Study what's required to support savable forms that can be saved by
-  Adobe Reader. Does this require actually signing the document with
-  an Adobe private key? Search for "Digital signatures" in the PDF
-  spec, and look at ~/Q/pdf-collection/form-with-full-save.pdf, which
-  came from Adobe's example site. See also
-  ../misc/digital-sign-from-trueroad/ and
-  ../misc/digital-signatures/digitally-signed-pdf-xfa.pdf. If digital
-  signatures are implemented, update the docs on crypto providers,
-  which mention that this may happen in the future.
-
-* Qpdf does not honor /EFF when adding new file attachments. When it
-  encrypts, it never generates streams with explicit crypt filters.
-  Prior to 10.2, there was an incorrect attempt to treat /EFF as a
-  default value for decrypting file attachment streams, but it is not
-  supposed to mean that. Instead, it is intended for conforming
-  writers to obey this when adding new attachments. Qpdf is not a
-  conforming writer in that respect.
-
-* The whole xref handling code in the QPDF object allows the same
-  object with more than one generation to coexist, but a lot of logic
-  assumes this isn't the case.  Anything that creates mappings only
-  with the object number and not the generation is this way,
-  including most of the interaction between QPDFWriter and QPDF.  If
-  we wanted to allow the same object with more than one generation to
-  coexist, which I'm not sure is allowed, we could fix this by
-  changing xref_table.  Alternatively, we could detect and disallow
-  that case.  In fact, it appears that Adobe reader and other PDF
-  viewing software silently ignores objects of this type, so this is
-  probably not a big deal.
-
-* From a suggestion in bug 3152169, consider having an option to
-  re-encode inline images with an ASCII encoding.
-
-* From github issue 2, provide more in-depth output for examining
-  hint stream contents. Consider adding on option to provide a
-  human-readable dump of linearization hint tables. This should
-  include improving the 'overflow reading bit stream' message as
-  reported in issue #2. There are multiple calls to stopOnError in
-  the linearization checking code. Ideally, these should not
-  terminate checking. It would require re-acquiring an understanding
-  of all that code to make the checks more robust. In particular,
-  it's hard to look at the code and quickly determine what is a true
-  logic error and what could happen because of malformed user input.
-  See also ../misc/linearization-errors.
-
-* If I ever decide to make appearance stream-generation aware of
-  fonts or font metrics, see email from Tobias with Message-ID
+* Consider adding a method to balance the pages tree. It would call pushInheritedAttributesToPage,
+  construct a pages tree from scratch, and replace the /Pages key of the root dictionary with the
+  new tree.
+
+* Study what's required to support savable forms that can be saved by Adobe Reader. Does this
+  require actually signing the document with an Adobe private key? Search for "Digital signatures"
+  in the PDF spec, and look at ~/Q/pdf-collection/form-with-full-save.pdf, which came from Adobe's
+  example site. See also ../misc/digital-sign-from-trueroad/ and
+  ../misc/digital-signatures/digitally-signed-pdf-xfa.pdf. If digital signatures are implemented,
+  update the docs on crypto providers, which mention that this may happen in the future.
+
+* Qpdf does not honor /EFF when adding new file attachments. When it encrypts, it never generates
+  streams with explicit crypt filters. Prior to 10.2, there was an incorrect attempt to treat /EFF
+  as a default value for decrypting file attachment streams, but it is not supposed to mean that.
+  Instead, it is intended for conforming writers to obey this when adding new attachments. Qpdf is
+  not a conforming writer in that respect.
+
+* The whole xref handling code in the QPDF object allows the same object with more than one
+  generation to coexist, but a lot of logic assumes this isn't the case. Anything that creates
+  mappings only with the object number and not the generation is this way, including most of the
+  interaction between QPDFWriter and QPDF. If we wanted to allow the same object with more than one
+  generation to coexist, which I'm not sure is allowed, we could fix this by changing xref_table.
+  Alternatively, we could detect and disallow that case. In fact, it appears that Adobe reader and
+  other PDF viewing software silently ignores objects of this type, so this is probably not a big
+  deal.
+
+* From a suggestion in bug 3152169, consider having an option to re-encode inline images with an
+  ASCII encoding.
+
+* From github issue 2, provide more in-depth output for examining hint stream contents. Consider
+  adding on option to provide a human-readable dump of linearization hint tables. This should
+  include improving the 'overflow reading bit stream' message as reported in issue #2. There are
+  multiple calls to stopOnError in the linearization checking code. Ideally, these should not
+  terminate checking. It would require re-acquiring an understanding of all that code to make the
+  checks more robust. In particular, it's hard to look at the code and quickly determine what is a
+  true logic error and what could happen because of malformed user input. See also
+  ../misc/linearization-errors.
+
+* If I ever decide to make appearance stream-generation aware of fonts or font metrics, see email
+  from Tobias with Message-ID
   <5C3C9C6C.8000102@thax.hardliners.org> dated 2019-01-14.
-* Look at places in the code where object traversal is being done and,
-  where possible, try to avoid it entirely or at least avoid ever
-  traversing the same objects multiple times.
+* Look at places in the code where object traversal is being done and, where possible, try to avoid
+  it entirely or at least avoid ever traversing the same objects multiple times.
 ----------------------------------------------------------------------
@@ -588,281 +484,215 @@ I find it useful to make reference to them in this list.
 Performance
 ===========
-As described in https://github.com/qpdf/qpdf/issues/401, there was
-great performance degradation between qpdf 7.1.1 and 9.1.1. Doing a
-bisect between dac65a21fb4fa5f871e31c314280b75adde89a6c and
-release-qpdf-7.1.1, I found several commits that damaged performance.
-I fixed some of them to improve performance by about 70% (as measured
-by saying that old times were 170% of new times). The remaining
-commits that broke performance either can't be correct because they
-would re-introduce an old bug or aren't worth correcting because of
-the high value they offer relative to a relatively low penalty. For
-historical reference, here are the commits. The numbers are the time
-in seconds on the machine I happened to be using of splitting the
-first 100 pages of PDF32000_2008.pdf 20 times and taking an average
-duration.
+As described in https://github.com/qpdf/qpdf/issues/401, there was great performance degradation
+between qpdf 7.1.1 and 9.1.1. Doing a bisect between dac65a21fb4fa5f871e31c314280b75adde89a6c and
+release-qpdf-7.1.1, I found several commits that damaged performance. I fixed some of them to
+improve performance by about 70% (as measured by saying that old times were 170% of new times). The
+remaining commits that broke performance either can't be correct because they would re-introduce an
+old bug or aren't worth correcting because of the high value they offer relative to a relatively low
+penalty. For historical reference, here are the commits. The numbers are the time in seconds on the
+machine I happened to be using of splitting the first 100 pages of PDF32000_2008.pdf 20 times and
+taking an average duration.
 Commits that broke performance:
-* d0e99f195a987c483bbb6c5449cf39bee34e08a1 -- object description and
-  context: 0.39 -> 0.45
-* a01359189b32c60c2d55b039f7aefd6c3ce0ebde (minus 313ba08) -- fix
-  dangling references: 0.55 -> 0.6
+* d0e99f195a987c483bbb6c5449cf39bee34e08a1 -- object description and context: 0.39 -> 0.45
+* a01359189b32c60c2d55b039f7aefd6c3ce0ebde (minus 313ba08) -- fix dangling references: 0.55 -> 0.6
 * e5f504b6c5dc34337cc0b316b4a7b1fca7e614b1 -- sparse array: 0.6 -> 0.62
 Other intermediate steps that were previously fixed:
-* 313ba081265f69ac9a0324f9fe87087c72918191 -- copy outlines into
-  split: 0.55 -> 4.0
+* 313ba081265f69ac9a0324f9fe87087c72918191 -- copy outlines into split: 0.55 -> 4.0
 * a01359189b32c60c2d55b039f7aefd6c3ce0ebde -- fix dangling references:
   4.0 -> 9.0
 This commit fixed the awful problem introduced in 313ba081:
-* a5a016cdd26a8e5c99e5f019bc30d1bdf6c050a2 -- revert outline
-  preservation: 9.0 -> 0.6
+* a5a016cdd26a8e5c99e5f019bc30d1bdf6c050a2 -- revert outline preservation: 9.0 -> 0.6
-Note that the fix dangling references commit had a much worse impact
-prior to removing the outline preservation, so I also measured its
-impact in isolation.
+Note that the fix dangling references commit had a much worse impact prior to removing the outline
+preservation, so I also measured its impact in isolation.
 A few important lessons (in README-maintainer)
-* Indirection through PointerHolder<Members> is expensive, and should
-  not be used for things that are created and destroyed frequently
-  such as QPDFObjectHandle and QPDFObject.
-* Traversal of objects is expensive and should be avoided where
-  possible.
+* Indirection through PointerHolder<Members> is expensive, and should not be used for things that
+  are created and destroyed frequently such as QPDFObjectHandle and QPDFObject.
+* Traversal of objects is expensive and should be avoided where possible.
-Also, it turns out that PointerHolder is more performant than
-std::shared_ptr. (This was true at the time but subsequent
-implementations of std::shared_ptr became much more efficient.)
+Also, it turns out that PointerHolder is more performant than std::shared_ptr. (This was true at the
+time but subsequent implementations of std::shared_ptr became much more efficient.)
 QPDFPagesTree
 =============
-On a few occasions, I have considered implementing a QPDFPagesTree
-object that would allow the document's original page tree structure to
-be preserved. See comments at the top QPDF_pages.cc for why this was
-abandoned.
-
-Partial work is in refs/attic/QPDFPagesTree. QPDFPageTree is mostly
-implemented and mostly tested. There are not enough cases of different
-kinds of operations (pclm, linearize, json, etc.) with non-flat pages
-trees. Insertion is not implemented. Insertion is potentially complex
-because of the issue of inherited objects. We will have to call
-pushInheritedAttributesToPage before adding any pages to the pages
-tree. The test suite is failing on that branch.
-
-Some parts of page tree repair are silent (no warnings). All page tree
-repair should warn. The reason is that page tree repair will change
-object numbers, and knowing that is important when working with JSON
-output.
-
-If we were to do this, we would still need keep a pages cache for
-efficient insertion. There's no reason we can't keep a vector of page
-objects up to date and just do a traversal the first time we do
-getAllPages just like we do now. The difference is that we would not
-flatten the pages tree. It would be useful to go through QPDF_pages
-and reimplement everything without calling flattenPagesTree. Then we
-can remove flattenPagesTree, which is private. That said, with the
-addition of creating non-flat pages trees, there is really no reason
-not to flatten the pages tree for internal use.
-
-In its current state, QPDFPagesTree does not proactively fix /Type or
-correct page objects that are used multiple times. You have to
-traverse the pages tree to trigger this operation. It would be nice if
-we would do that somewhere but not do it more often than necessary so
-isPagesObject and isPageObject are reliable and can be made more
-reliable. Maybe add a validate or repair function? It should also make
-sure /Count and /Parent are correct.
+On a few occasions, I have considered implementing a QPDFPagesTree object that would allow the
+document's original page tree structure to be preserved. See comments at the top QPDF_pages.cc for
+why this was abandoned.
+
+Partial work is in refs/attic/QPDFPagesTree. QPDFPageTree is mostly implemented and mostly tested.
+There are not enough cases of different kinds of operations (pclm, linearize, json, etc.) with
+non-flat pages trees. Insertion is not implemented. Insertion is potentially complex because of the
+issue of inherited objects. We will have to call pushInheritedAttributesToPage before adding any
+pages to the pages tree. The test suite is failing on that branch.
+
+Some parts of page tree repair are silent (no warnings). All page tree repair should warn. The
+reason is that page tree repair will change object numbers, and knowing that is important when
+working with JSON output.
+
+If we were to do this, we would still need keep a pages cache for efficient insertion. There's no
+reason we can't keep a vector of page objects up to date and just do a traversal the first time we
+do getAllPages just like we do now. The difference is that we would not flatten the pages tree. It
+would be useful to go through QPDF_pages and reimplement everything without calling
+flattenPagesTree. Then we can remove flattenPagesTree, which is private. That said, with the
+addition of creating non-flat pages trees, there is really no reason not to flatten the pages tree
+for internal use.
+
+In its current state, QPDFPagesTree does not proactively fix /Type or correct page objects that are
+used multiple times. You have to traverse the pages tree to trigger this operation. It would be nice
+if we would do that somewhere but not do it more often than necessary so isPagesObject and
+isPageObject are reliable and can be made more reliable. Maybe add a validate or repair function? It
+should also make sure /Count and /Parent are correct.
 Rejected Ideas
 ==============
-* Investigate whether there is a way to automate the memory checker
-  tests for Windows.
-
-* Provide support in QPDFWriter for writing incremental updates.
-  Provide support in qpdf for preserving incremental updates.  The
-  goal should be that QDF mode should be fully functional for files
-  with incremental updates including fix_qdf.
-
-  Note that there's nothing that says an indirect object in one
-  update can't refer to an object that doesn't appear until a later
-  update.  This means that QPDF has to treat indirect null objects
-  differently from how it does now.  QPDF drops indirect null objects
-  that appear as members of arrays or dictionaries.  For arrays, it's
-  handled in QPDFWriter where we make indirect nulls direct.  This is
-  in a single if block, and nothing else in the code cares about it.
-  We could just remove that if block and not break anything except a
-  few test cases that exercise the current behavior.  For
-  dictionaries, it's more complicated.  In this case,
-  QPDF_Dictionary::getKeys() ignores all keys with null values, and
-  hasKey() returns false for keys that have null values.  We would
-  probably want to make QPDF_Dictionary able to handle the special
-  case of keys that are indirect nulls and basically never have it
-  drop any keys that are indirect objects.
-
-  If we make a change to have qpdf preserve indirect references to
-  null objects, we have to note this in ChangeLog and in the release
-  notes since this will change output files.  We did this before when
-  we stopped flattening scalar references, so this is probably not a
-  big deal.  We also have to make sure that the testing for this
-  handles non-trivial cases of the targets of indirect nulls being
-  replaced by real objects in an update.  I'm not sure how this plays
-  with linearization, if at all.  For cases where incremental updates
-  are not being preserved as incremental updates and where the data
-  is being folded in (as is always the case with qpdf now), none of
-  this should make any difference in the actual semantics of the
-  files.
-
-* The second xref stream for linearized files has to be padded only
-  because we need file_size as computed in pass 1 to be accurate.  If
-  we were not allowing writing to a pipe, we could seek back to the
-  beginning and fill in the value of /L in the linearization
-  dictionary as an optimization to alleviate the need for this
-  padding.  Doing so would require us to pad the /L value
-  individually and also to save the file descriptor and determine
-  whether it's seekable.  This is probably not worth bothering with.
-
-* Based on an idea suggested by user "Atom Smasher", consider
-  providing some mechanism to recover earlier versions of a file
-  embedded prior to appended sections.
-
-* Consider creating a sanitizer to make it easier for people to send
-  broken files. Now that we have json mode, this is probably no
-  longer worth doing. Here is the previous idea, possibly implemented
-  by making it possible to run the lexer (tokenizer) over a whole
-  file. Make it possible to replace all strings in a file lexically
-  even on badly broken files. Ideally this should work files that are
-  lacking xref, have broken links, duplicated dictionary keys, syntax
-  errors, etc., and ideally it should work with encrypted files if
-  possible. This should go through the streams and strings and
-  replace them with fixed or random characters, preferably, but not
-  necessarily, in a manner that works with fonts. One possibility
-  would be to detect whether a string contains characters with normal
-  encoding, and if so, use 0x41. If the string uses character maps,
-  use 0x01. The output should otherwise be unrelated to the input.
-  This could be built after the filtering and tokenizer rewrite and
-  should be done in a manner that takes advantage of the other
-  lexical features. This sanitizer should also clear metadata and
-  replace images. If I ever do this, the file from issue #494 would
-  be a great one to look at.
-
-* Here are some notes about having stream data providers modify
-  stream dictionaries. I had wanted to add this functionality to make
-  it more efficient to create stream data providers that may
-  dynamically decide what kind of filters to use and that may end up
-  modifying the dictionary conditionally depending on the original
-  stream data. Ultimately I decided not to implement this feature.
-  This paragraph describes why.
-
-  * When writing, the way objects are placed into the queue for
-    writing strongly precludes creation of any new indirect objects,
-    or even changing which indirect objects are referenced from which
-    other objects, because we sometimes write as we are traversing
-    and enqueuing objects. For non-linearized files, there is a risk
-    that an indirect object that used to be referenced would no
-    longer be referenced, and whether it was already written to the
-    output file would be based on an accident of where it was
-    encountered when traversing the object structure. For linearized
-    files, the situation is considerably worse. We decide which
-    section of the file to write an object to based on a mapping of
-    which objects are used by which other objects. Changing this
-    mapping could cause an object to appear in the wrong section, to
-    be written even though it is unreferenced, or to be entirely
-    omitted since, during linearization, we don't enqueue new objects
-    as we traverse for writing.
-
-  * There are several places in QPDFWriter that query a stream's
-    dictionary in order to prepare for writing or to make decisions
-    about certain aspects of the writing process. If the stream data
-    provider has the chance to modify the dictionary, every piece of
-    code that gets stream data would have to be aware of this. This
-    would potentially include end user code. For example, any code
-    that called getDict() on a stream before installing a stream data
-    provider and expected that dictionary to be valid would
-    potentially be broken. As implemented right now, you must perform
-    any modifications on the dictionary in advance and provided
-    /Filter and /DecodeParms at the time you installed the stream
-    data provider. This means that some computations would have to be
-    done more than once, but for linearized files, stream data
-    providers are already called more than once. If the work done by
-    a stream data provider is especially expensive, it can implement
+* Investigate whether there is a way to automate the memory checker tests for Windows.
+
+* Provide support in QPDFWriter for writing incremental updates. Provide support in qpdf for
+  preserving incremental updates. The goal should be that QDF mode should be fully functional for
+  files with incremental updates including fix_qdf.
+
+  Note that there's nothing that says an indirect object in one update can't refer to an object that
+  doesn't appear until a later update. This means that QPDF has to treat indirect null objects
+  differently from how it does now. QPDF drops indirect null objects that appear as members of
+  arrays or dictionaries. For arrays, it's handled in QPDFWriter where we make indirect nulls
+  direct. This is in a single if block, and nothing else in the code cares about it. We could just
+  remove that if block and not break anything except a few test cases that exercise the current
+  behavior. For dictionaries, it's more complicated. In this case, QPDF_Dictionary::getKeys()
+  ignores all keys with null values, and hasKey() returns false for keys that have null values. We
+  would probably want to make QPDF_Dictionary able to handle the special case of keys that are
+  indirect nulls and basically never have it drop any keys that are indirect objects.
+
+  If we make a change to have qpdf preserve indirect references to null objects, we have to note
+  this in ChangeLog and in the release notes since this will change output files. We did this before
+  when we stopped flattening scalar references, so this is probably not a big deal. We also have to
+  make sure that the testing for this handles non-trivial cases of the targets of indirect nulls
+  being replaced by real objects in an update. I'm not sure how this plays with linearization, if at
+  all. For cases where incremental updates are not being preserved as incremental updates and where
+  the data is being folded in (as is always the case with qpdf now), none of this should make any
+  difference in the actual semantics of the files.
+
+* The second xref stream for linearized files has to be padded only because we need file_size as
+  computed in pass 1 to be accurate. If we were not allowing writing to a pipe, we could seek back
+  to the beginning and fill in the value of /L in the linearization dictionary as an optimization to
+  alleviate the need for this padding. Doing so would require us to pad the /L value individually
+  and also to save the file descriptor and determine whether it's seekable. This is probably not
+  worth bothering with.
+
+* Based on an idea suggested by user "Atom Smasher", consider providing some mechanism to recover
+  earlier versions of a file embedded prior to appended sections.
+
+* Consider creating a sanitizer to make it easier for people to send broken files. Now that we have
+  json mode, this is probably no longer worth doing. Here is the previous idea, possibly implemented
+  by making it possible to run the lexer (tokenizer) over a whole file. Make it possible to replace
+  all strings in a file lexically even on badly broken files. Ideally this should work files that
+  are lacking xref, have broken links, duplicated dictionary keys, syntax errors, etc., and ideally
+  it should work with encrypted files if possible. This should go through the streams and strings
+  and replace them with fixed or random characters, preferably, but not necessarily, in a manner
+  that works with fonts. One possibility would be to detect whether a string contains characters
+  with normal encoding, and if so, use 0x41. If the string uses character maps, use 0x01. The output
+  should otherwise be unrelated to the input. This could be built after the filtering and tokenizer
+  rewrite and should be done in a manner that takes advantage of the other lexical features. This
+  sanitizer should also clear metadata and replace images. If I ever do this, the file from issue
+  #494 would be a great one to look at.
+
+* Here are some notes about having stream data providers modify stream dictionaries. I had wanted to
+  add this functionality to make it more efficient to create stream data providers that may
+  dynamically decide what kind of filters to use and that may end up modifying the dictionary
+  conditionally depending on the original stream data. Ultimately I decided not to implement this
+  feature. This paragraph describes why.
+
+  * When writing, the way objects are placed into the queue for writing strongly precludes creation
+    of any new indirect objects, or even changing which indirect objects are referenced from which
+    other objects, because we sometimes write as we are traversing and enqueuing objects. For
+    non-linearized files, there is a risk that an indirect object that used to be referenced would
+    no longer be referenced, and whether it was already written to the output file would be based on
+    an accident of where it was encountered when traversing the object structure. For linearized
+    files, the situation is considerably worse. We decide which section of the file to write an
+    object to based on a mapping of which objects are used by which other objects. Changing this
+    mapping could cause an object to appear in the wrong section, to be written even though it is
+    unreferenced, or to be entirely omitted since, during linearization, we don't enqueue new
+    objects as we traverse for writing.
+
+  * There are several places in QPDFWriter that query a stream's dictionary in order to prepare for
+    writing or to make decisions about certain aspects of the writing process. If the stream data
+    provider has the chance to modify the dictionary, every piece of code that gets stream data
+    would have to be aware of this. This would potentially include end user code. For example, any
+    code that called getDict() on a stream before installing a stream data provider and expected
+    that dictionary to be valid would potentially be broken. As implemented right now, you must
+    perform any modifications on the dictionary in advance and provided /Filter and /DecodeParms at
+    the time you installed the stream data provider. This means that some computations would have to
+    be done more than once, but for linearized files, stream data providers are already called more
+    than once. If the work done by a stream data provider is especially expensive, it can implement
     its own cache.
-  The example examples/pdf-custom-filter.cc demonstrates the use of
-  custom stream filters. This includes a custom pipeline, a custom
-  stream filter, as well as modification of a stream's dictionary to
-  include creation of a new stream that is referenced from
-  /DecodeParms.
-
-* Removal of raw QPDF* from the API. Discussions in #747 and #754.
-  This is a summary of the arguments I put forth in #754. The idea was
-  to make QPDF::QPDF() private and require all QPDF objects to be
-  shared pointers created with QPDF::create(). This would enable us to
-  have QPDFObjectHandle::getOwningQPDF() return a std::weak_ptr<QPDF>.
-  Prior to #726 (QPDFObject/QPDFValue split, released in qpdf 11.0.0),
-  getOwningQPDF() could return an invalid pointer if the owning QPDF
-  disappeared, but this is no longer the case, which removes the main
+  The example examples/pdf-custom-filter.cc demonstrates the use of custom stream filters. This
+  includes a custom pipeline, a custom stream filter, as well as modification of a stream's
+  dictionary to include creation of a new stream that is referenced from /DecodeParms.
+
+* Removal of raw QPDF* from the API. Discussions in #747 and #754. This is a summary of the
+  arguments I put forth in #754. The idea was to make QPDF::QPDF() private and require all QPDF
+  objects to be shared pointers created with QPDF::create(). This would enable us to have
+  QPDFObjectHandle::getOwningQPDF() return a std::weak_ptr<QPDF>. Prior to #726 (
+  QPDFObject/QPDFValue split, released in qpdf 11.0.0), getOwningQPDF() could return an invalid
+  pointer if the owning QPDF disappeared, but this is no longer the case, which removes the main
   motivation. QPDF 11 added QPDF::create() anyway though.
-  Removing raw QPDF* would look something like this. Note that you
-  can't use std::make_shared<T> unless T has a public constructor.
+  Removing raw QPDF* would look something like this. Note that you can't use std::make_shared<T>
+  unless T has a public constructor.
   QPDF_POINTER_TRANSITION = 0 -- no warnings around calling the QPDF constructor
-  QPDF_POINTER_TRANSITION = 1 -- calls to QPDF() are deprecated, but QPDF is still available so code can be backward compatible and use std::make_shared<QPDF>
-  QPDF_POINTER_TRANSITION = 2 -- the QPDF constructor is private; all calls to std::make_shared<QPDF> have to be replaced with QPDF::create
-
-  If we were to do this, we'd have to look at each use of QPDF* in the
-  interface and decide whether to use a std::shared_ptr or a
-  std::weak_ptr. The answer would almost always be to use a
-  std::weak_ptr, which means we'd have to take the extra step of
-  calling lock(), and it means there would be lots of code changes
-  cause people would have to pass weak pointers instead of raw
-  pointers around, and those have to be constructed and locked.
-  Passing std::shared_ptr around leaves the possibility of creating
-  circular references. It seems to be too much trouble in the library
-  and too much toil for library users to be worth the small benefit of
-  not having to call resetObjGen in QPDF's destructor.
+  QPDF_POINTER_TRANSITION = 1 -- calls to QPDF() are deprecated, but QPDF is still available so code
+  can be backward compatible and use std::make_shared<QPDF>
+  QPDF_POINTER_TRANSITION = 2 -- the QPDF constructor is private; all calls to std::
+  make_shared<QPDF> have to be replaced with QPDF::create
+
+  If we were to do this, we'd have to look at each use of QPDF* in the interface and decide whether
+  to use a std::shared_ptr or a std::weak_ptr. The answer would almost always be to use a std::
+  weak_ptr, which means we'd have to take the extra step of calling lock(), and it means there would
+  be lots of code changes cause people would have to pass weak pointers instead of raw pointers
+  around, and those have to be constructed and locked. Passing std::shared_ptr around leaves the
+  possibility of creating circular references. It seems to be too much trouble in the library and
+  too much toil for library users to be worth the small benefit of not having to call resetObjGen in
+  QPDF's destructor.
 * Fix Multiple Direct Object Parent Issue
-  This idea was rejected because it would be complicated to implement
-  and would likely have a high performance cost to fix what is not
-  really that big of a problem in practice.
-
-  It is possible for a QPDFObjectHandle for a direct object to be
-  contained inside of multiple QPDFObjectHandle objects or even
-  replicated across multiple QPDF objects. This creates a potentially
-  confusing and unintentional aliasing of direct objects. There are
-  known cases in the qpdf library where this happens including page
-  splitting and merging (particularly with page labels, and possibly
-  with other cases), and also with unsafeShallowCopy. Disallowing this
-  would incur a significant performance penalty and is probably not
-  worth doing. If we were to do it, here are some ideas.
-
-  * Add std::weak_ptr<QPDFObject> parent to QPDFObject. When adding a
-    direct object to an array or dictionary, set its parent. When
-    removing it, clear the parent pointer. The parent pointer would
-    always be null for indirect objects, so the parent pointer, which
-    would reside in QPDFObject, would have to be managed by
-    QPDFObjectHandle. This is because QPDFObject can't tell the
+  This idea was rejected because it would be complicated to implement and would likely have a high
+  performance cost to fix what is not really that big of a problem in practice.
+
+  It is possible for a QPDFObjectHandle for a direct object to be contained inside of multiple
+  QPDFObjectHandle objects or even replicated across multiple QPDF objects. This creates a
+  potentially confusing and unintentional aliasing of direct objects. There are known cases in the
+  qpdf library where this happens including page splitting and merging (particularly with page
+  labels, and possibly with other cases), and also with unsafeShallowCopy. Disallowing this would
+  incur a significant performance penalty and is probably not worth doing. If we were to do it, here
+  are some ideas.
+
+  * Add std::weak_ptr<QPDFObject> parent to QPDFObject. When adding a direct object to an array or
+    dictionary, set its parent. When removing it, clear the parent pointer. The parent pointer would
+    always be null for indirect objects, so the parent pointer, which would reside in QPDFObject,
+    would have to be managed by QPDFObjectHandle. This is because QPDFObject can't tell the
     difference between a resolved indirect object and a direct object.
-  * Phase 1: When a direct object that already has a parent is added
-    to a dictionary or array, issue a warning. There would need to be
-    unsafe add methods used by unsafeShallowCopy. These would add but
-    not modify the parent pointer.
-
-  * Phase 2: In the next major release, make the multiple parent case
-    an error. Require people to create a copy. The unsafe operations
-    would still have to be permitted.
-
-  This approach would allow an object to be moved from one object to
-  another by removing it, which returns the now orphaned object, and
-  then inserting it somewhere else. It also doesn't break the pattern
-  of adding a direct object to something and subsequently mutating it.
-  It just prevents the same object from being added to more than one
-  thing.
+  * Phase 1: When a direct object that already has a parent is added to a dictionary or array, issue
+    a warning. There would need to be unsafe add methods used by unsafeShallowCopy. These would add
+    but not modify the parent pointer.
+
+  * Phase 2: In the next major release, make the multiple parent case an error. Require people to
+    create a copy. The unsafe operations would still have to be permitted.
+
+  This approach would allow an object to be moved from one object to another by removing it, which
+  returns the now orphaned object, and then inserting it somewhere else. It also doesn't break the
+  pattern of adding a direct object to something and subsequently mutating it. It just prevents the
+  same object from being added to more than one thing.