OpenSystemsDevelopment / qpdf

* native UTF-8 strings
* names whose PDF and canonical syntax differ in both dictionary key
  positions and other positions

For json, names are converted both as names and directly when used as
dictionary keys.

* Replace --create-from-json=file with --json-input, which causes the
  regular input to be treated as json.
* Eliminate --to-json
* In --json=2, bring back "objects" and eliminate "objectinfo". Stream
  data is never present.
* In --json-output=2, write "qpdf-v2" with "objects" and include
  stream data.

The last offset is irrelevant to not being able to find /Root.

This makes it a lot easier to run parts of the test suite.

There is one unexpected pass in this commit. This script was applied
to the files changed in this commit:

----------
#!/usr/bin/env python3
import json
import sys

def json_dumps(data):
    return json.dumps(data, ensure_ascii=False,
                      indent=2, separators=(',', ': '))

for filename in sys.argv[1:]:
    with open(filename, 'r') as f:
        data = json.loads(f.read())
    data['version'] = 2
    objectinfo = {}
    if 'objectinfo' in data:
        objectinfo = data['objectinfo']
        del data['objectinfo']
    if 'objects' not in data:
        continue
    qpdf = {'jsonversion': 2, 'pdfversion': '1.3', 'objects': {}}
    for k, v in data['objects'].items():
        is_stream = objectinfo.get(k, {}).get('stream', {}).get('is', False)
        if k.endswith(' R'):
            k = 'obj:' + k
        if is_stream:
            v = {'stream': {'dict': v}}
        else:
            v = {'value': v}
        qpdf['objects'][k] = v
    data['qpdf'] = qpdf
    del data['objects']
print(json_dumps(data))
----------

moddify -> modify. Also carefully spell checked all remaining keys by
splitting them into words and running a spell checker, not just
relying on visual proofreading. That was the only one.

Not fully exercised in this commit

Also refine the herustic for deciding whether to use hexadecimal
notation for a string.

Update getJSON() methods and calls to them

This script was used on test data:

----------
#!/usr/bin/env python3
import json
import sys
import re

def json_dumps(data):
    return json.dumps(data, ensure_ascii=False,
                      indent=2, separators=(',', ': '))

for filename in sys.argv[1:]:
    with open(filename, 'r') as f:
        data = json.loads(f.read())
    if 'objectinfo' not in data:
        continue
    trailer = None
    to_sort = []
    for k, v in data['objectinfo'].items():
        if k == 'trailer':
            trailer = v
        else:
            m = re.match(r'^(\d+) \d+ R', k)
            if m:
                to_sort.append([int(m.group(1)), k, v])
    newobjectinfo = {x[1]: x[2] for x in sorted(to_sort)}
    if trailer is not None:
        newobjectinfo['trailer'] = trailer
    data['objectinfo'] = newobjectinfo
print(json_dumps(data))
----------

The following script was used to adjust test data:

----------
#!/usr/bin/env python3
import json
import sys
import re

def json_dumps(data):
    return json.dumps(data, ensure_ascii=False,
                      indent=2, separators=(',', ': '))

for filename in sys.argv[1:]:
    with open(filename, 'r') as f:
        data = json.loads(f.read())
    if 'objects' not in data:
        continue
    trailer = None
    to_sort = []
    for k, v in data['objects'].items():
        if k == 'trailer':
            trailer = v
        else:
            m = re.match(r'^(\d+) \d+ R', k)
            if m:
                to_sort.append([int(m.group(1)), k, v])
    newobjects = {x[1]: x[2] for x in sorted(to_sort)}
    if trailer is not None:
        newobjects['trailer'] = trailer
    data['objects'] = newobjects
print(json_dumps(data))
----------

This commit just changes the order in which fields are written to the
json without changing their content. All the json files in the test
suite were modified with this script to ensure that we didn't get any
changes other than ordering.

----------
#!/usr/bin/env python3
import json
import sys

def json_dumps(data):
    return json.dumps(data, ensure_ascii=False,
                      indent=2, separators=(',', ': '))

for filename in sys.argv[1:]:
    with open(filename, 'r') as f:
        data = json.loads(f.read())
    newdata = {}
    for i in ('version', 'parameters', 'pages', 'pagelabels',
              'acroform', 'attachments', 'encrypt', 'outlines',
              'objects', 'objectinfo'):
        if i in data:
            newdata[i] = data[i]
print(json_dumps(newdata))
----------

Testing json against schema requires an in-memory copy, so do it only
when requested by the test suite.

Make it so that a key with a null value is always treated as not being
present. This was inconsistent before.