Commit 2a8cd4acdc48f050a0be66405cfbe9ba1c6ce9da

Authored by Jay Berkenbilt
1 parent 6219111e

Convert README files to markdown

README-hardening.md
1 -Avoiding operator[]  
2 -=================== 1 +# Avoiding `operator[]`
3 2
4 -During a security review by Red Hat security team (specifically  
5 -Florian Weimer), it was discovered that qpdf used std::string and  
6 -std::vector's operator[], which has no bounds checking by design.  
7 -Instead, using those objects' at() method is preferable since it does  
8 -bounds checking. Florian has a tool that can detect all uses of these  
9 -methods and report them. I have a short perl script that  
10 -automatically corrects any such uses. The perl script is not intended  
11 -to be general, but it could be reasonably general. The only known  
12 -shortcut is that it might not work very well with some cases of nested  
13 -[]'s like a[b[c]] or with cases where there are line breaks inside the  
14 -brackets. For qpdf's coding style, it worked on all cases reported. 3 +During a security review by Red Hat security team (specifically Florian Weimer), it was discovered that qpdf used `std::string` and `std::vector`'s `operator[]`, which has no bounds checking by design. Instead, using those objects' `at()` method is preferable since it does bounds checking. Florian has a tool that can detect all uses of these methods and report them. I have a short perl script that automatically corrects any such uses. The perl script is not intended to be general, but it could be reasonably general. The only known shortcut is that it might not work very well with some cases of nested `[]`'s like `a[b[c]]` or with cases where there are line breaks inside the brackets. For qpdf's coding style, it worked on all cases reported.
15 4
16 -To use this, obtain htcondor-analyzer, run it, and respond to the  
17 -report. Here's what I did. 5 +To use this, obtain htcondor-analyzer, run it, and respond to the report. Here's what I did.
18 6
  7 +```
19 sudo aptitude install libclang-dev llvm llvm-dev clang 8 sudo aptitude install libclang-dev llvm llvm-dev clang
20 cd /tmp 9 cd /tmp
21 git clone https://github.com/fweimer/htcondor-analyzer 10 git clone https://github.com/fweimer/htcondor-analyzer
22 # HEAD = 5fa06fc68a9b0677e9de162279185d58ba1e8477 at this writing 11 # HEAD = 5fa06fc68a9b0677e9de162279185d58ba1e8477 at this writing
23 cd htcondor-analyzer 12 cd htcondor-analyzer
24 make 13 make
  14 +```
25 15
26 -in qpdf 16 +In qpdf:
27 17
  18 +```
28 ./autogen.sh 19 ./autogen.sh
29 /tmp/htcondor-analyzer/create-db 20 /tmp/htcondor-analyzer/create-db
30 CC=/tmp/htcondor-analyzer/cc CXX=/tmp/htcondor-analyzer/cxx ./configure --disable-shared --disable-werror 21 CC=/tmp/htcondor-analyzer/cc CXX=/tmp/htcondor-analyzer/cxx ./configure --disable-shared --disable-werror
31 # to remove conftest.c 22 # to remove conftest.c
32 \rm htcondor-analyzer.sqlite 23 \rm htcondor-analyzer.sqlite
33 /tmp/htcondor-analyzer/create-db 24 /tmp/htcondor-analyzer/create-db
  25 +```
34 26
35 Repeat until no more errors: 27 Repeat until no more errors:
36 28
  29 +```
37 /tmp/fix-at.pl is shown below. 30 /tmp/fix-at.pl is shown below.
  31 +```
38 32
  33 +```
39 make 34 make
40 /tmp/htcondor-analyzer/report | grep std:: | grep qpdf >| /tmp/r 35 /tmp/htcondor-analyzer/report | grep std:: | grep qpdf >| /tmp/r
41 perl /tmp/fix-at.pl /tmp/r 36 perl /tmp/fix-at.pl /tmp/r
42 # move all *.new over the original file. patmv is my script. Can 37 # move all *.new over the original file. patmv is my script. Can
43 # also use a for loop. 38 # also use a for loop.
44 patmv -f s/.new// **/*.new 39 patmv -f s/.new// **/*.new
  40 +```
45 41
46 ----------- /tmp/fix-at.pl ---------- 42 +/tmp/fix-at.pl:
  43 +```perl
47 #!/usr/bin/env perl 44 #!/usr/bin/env perl
48 require 5.008; 45 require 5.008;
49 use warnings; 46 use warnings;
@@ -90,4 +87,4 @@ foreach my $file (sort keys %to_fix) @@ -90,4 +87,4 @@ foreach my $file (sort keys %to_fix)
90 } 87 }
91 close(F) or die; 88 close(F) or die;
92 } 89 }
93 --------------------- 90 +```
README-maintainer.md
1 -Release Reminders  
2 -=================  
3 -  
4 - * Test for binary compatility. The easiest way to do this is to check  
5 - out the last release, run the test suite, check out the new  
6 - release, run make build_libqpdf, check out the old release, and run  
7 - make check NO_REBUILD=1.  
8 -  
9 - * When making a release, always remember to run large file tests and  
10 - image comparison tests (--enable-test-compare-images  
11 - --with-large-file-test-path=/path). For Windows, use a Windows  
12 - style path, not an MSYS path for large files. For a major release,  
13 - consider running a spelling checker over the source code to catch  
14 - errors in variable names, strings, and comments. Use ispell -p  
15 - ispell-words.  
16 -  
17 - * Run tests with sanitize address enabled:  
18 -  
19 - ./configure CFLAGS="-fsanitize=address -g" \  
20 - CXXFLAGS="-fsanitize=address -g" \  
21 - LDFLAGS="-fsanitize=address" \  
22 - --enable-werror --disable-shared  
23 -  
24 - As of gcc 6.3.0, this exposes some good things but appears to also  
25 - have some false positive leak reports. Valgrind is more reliable  
26 - but also may miss some things that this catches.  
27 -  
28 - * Consider running tests with latest gcc and/or valgrind. To test  
29 - with valgrind:  
30 -  
31 - ./configure --disable-shared  
32 - make -j8 -k VALGRIND=1  
33 - make -k check NO_REBUILD=1  
34 -  
35 - This moves each binary into a subdirectory and replaces it with a  
36 - link to make/exec-z. See make/exec-z.  
37 -  
38 - * Check all open issues in the sourceforge trackers and on github.  
39 -  
40 - * If any interfaces were added or changed, check C API to see whether  
41 - changes are appropriate there as well. If necessary, review the  
42 - casting policy in the manual, and ensure that integer types are  
43 - properly handled.  
44 -  
45 - * Remember to avoid using operator[] with std::string or  
46 - std::vector. See README-hardening.md for details.  
47 -  
48 - * Increment shared library version information as needed (LT_* in  
49 - configure.ac)  
50 -  
51 - * Update release notes in manual -- look at diffs and ChangeLog  
52 -  
53 - * Add a release entry to ChangeLog  
54 -  
55 - * Make sure version numbers are consistent in the following  
56 - locations:  
57 -  
58 - configure.ac  
59 - libqpdf/QPDF.cc  
60 - manual/qpdf-manual.xml  
61 -  
62 - make_dist verifies this consistency.  
63 -  
64 - * Update release date in manual/qpdf-manual.xml. Remember to ensure  
65 - that the entities at the top of the document are consistent with  
66 - the release notes for both version and release date.  
67 -  
68 - * Check TODO file to make sure all planned items for the release are  
69 - done or retargeted.  
70 -  
71 - * Each year, update copyright notices. Just do a case-insensitive  
72 - search for copyright. Don't forget copyright in manual. Also update  
73 - debian copyright in debian package. Last updated: 2017.  
74 -  
75 - * To construct a source distribution from a pristine checkout,  
76 - make_dist does the following:  
77 -  
78 - ./autogen.sh  
79 - ./configure --enable-doc-maintenance --enable-werror  
80 - make build_manual  
81 - make distclean  
82 -  
83 - * To create a source release, do an export from the version control  
84 - system to a directory called qpdf-version. For example, from this  
85 - directory:  
86 -  
87 - rm -rf /tmp/qpdf-x.y.z  
88 - git archive --prefix=qpdf-x.y.z/ HEAD . | (cd /tmp; tar xf -)  
89 -  
90 - From the parent of that directory, run make_dist with the directory  
91 - as an argument. Remember to have fop in your path. For internally  
92 - testing releases, you can run make_dist with the --no-tests option.  
93 -  
94 - * To create a source release of external libs, do an export from the  
95 - version control system into a directory called qpdf-external-libs  
96 - and just make a zip file of the result called  
97 - qpdf-external-libs-src.zip. See the README.txt file there for  
98 - information on creating binary external libs releases. Run this  
99 - from the external-libs repository:  
100 -  
101 - git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -)  
102 - cd /tmp  
103 - zip -r qpdf-external-libs-src.zip external-libs  
104 -  
105 - * To create Windows binary releases, extract the qpdf source  
106 - distribution in Windows (MSYS + MINGW, MSVC). From the extracted  
107 - directory, extract the binary distribution of the external  
108 - libraries. Run ./make_windows_releases from there. You will need  
109 - to have zip in your path.  
110 -  
111 - * Before releasing, rebuild and test debian package.  
112 -  
113 - * Remember to copy README-what-to-download.md separately onto the  
114 - download area.  
115 -  
116 - * Remember to update the web page including putting new documentation  
117 - in the "files" subdirectory of the website on sourceforge.net.  
118 -  
119 - * Create a tag in the version control system, and make backups of the  
120 - actual releases. With git, use git tag -s to create a signed tag:  
121 -  
122 - git tag -s release-qpdf-$version HEAD -m"qpdf $version"  
123 -  
124 - * When releasing on sourceforge, external-libs distributions go in  
125 - external-libs/yyyymmdd, and qpdf distributions go in qpdf/vvv.  
126 - Make the source package the default for all but Windows, and make  
127 - the 32-bit mingw build the default for Windows.  
128 -  
129 -  
130 -General Build Stuff  
131 -===================  
132 -  
133 -QPDF uses autoconf and libtool but does not use automake. The only  
134 -files distributed with the qpdf source distribution that are not  
135 -controlled are "configure", "libqpdf/qpdf/qpdf-config.h.in",  
136 -"aclocal.m4", and some documentation. See above for the steps  
137 -required to prepare a source distribution.  
138 -  
139 -A small handful of additional files have been taken from autotools  
140 -programs. These should probably be updated from time to time.  
141 -  
142 - * config.guess, config.sub, ltmain.sh, and the m4 directory: these  
143 - were created by running libtoolize -c. To update, run libtoolize  
144 - -f -c or remove the files and rerun libtoolize.  
145 -  
146 - * Other files copied as indicated:  
147 -  
148 - cp /usr/share/automake-1.11/install-sh .  
149 - cp /usr/share/automake-1.11/mkinstalldirs .  
150 -  
151 -The entire contents of the m4 directory came from libtool.m4. If we  
152 -had some additional local parts, we could also add those to the m4  
153 -directory. In order for this to work, it is necessary to run "aclocal  
154 --I m4" before running autoheader and autoconf.  
155 -  
156 -If building or editing documentation, configure with  
157 ---enable-doc-maintenance. This will ensure that all tools or files  
158 -required to validate and build documentation are available.  
159 -  
160 -If you want to run make maintainer-clean or make distclean and you  
161 -haven't run ./configure, you can pass CLEAN=1 to make on the command  
162 -line to prevent it from complaining about configure not having been  
163 -run.  
164 -  
165 -Local Windows Testing Procedure  
166 -=============================== 1 +# Release Reminders
  2 +
  3 +* Test for binary compatility. The easiest way to do this is to check out the last release, run the test suite, check out the new release, run `make build_libqpdf`, check out the old release, and run `make check NO_REBUILD=1`.
  4 +* When making a release, always remember to run large file tests and image comparison tests (`--enable-test-compare-images` `--with-large-file-test-path=/path`). For Windows, use a Windows style path, not an MSYS path for large files. For a major release, consider running a spelling checker over the source code to catch errors in variable names, strings, and comments. Use `ispell -p ispell-words`.
  5 +* Run tests with sanitize address enabled:
  6 + ```
  7 + ./configure CFLAGS="-fsanitize=address -g" \
  8 + CXXFLAGS="-fsanitize=address -g" \
  9 + LDFLAGS="-fsanitize=address" \
  10 + --enable-werror --disable-shared
  11 + ```
  12 + As of gcc 6.3.0, this exposes some good things but appears to also have some false positive leak reports. Valgrind is more reliable but also may miss some things that this catches.
  13 +* Consider running tests with latest gcc and/or valgrind. To test with valgrind:
  14 + ```
  15 + ./configure --disable-shared
  16 + make -j8 -k VALGRIND=1
  17 + make -k check NO_REBUILD=1
  18 + ```
  19 + This moves each binary into a subdirectory and replaces it with a link to make/exec-z. See make/exec-z.
  20 +* Check all open issues in the sourceforge trackers and on github.
  21 +* If any interfaces were added or changed, check C API to see whether changes are appropriate there as well. If necessary, review the casting policy in the manual, and ensure that integer types are properly handled.
  22 +* Remember to avoid using `operator[]` with `std::string` or `std::vector`. Instead, use `at()`. See README-hardening.md for details.
  23 +* Increment shared library version information as needed (`LT_*` in `configure.ac`)
  24 +* Update release notes in manual. Look at diffs and ChangeLog.
  25 +* Add a release entry to ChangeLog.
  26 +* Make sure version numbers are consistent in the following locations:
  27 + * configure.ac
  28 + * libqpdf/QPDF.cc
  29 + * manual/qpdf-manual.xml
  30 + `make_dist` verifies this consistency.
  31 +* Update release date in `manual/qpdf-manual.xml`. Remember to ensure that the entities at the top of the document are consistent with the release notes for both version and release date.
  32 +* Check `TODO` file to make sure all planned items for the release are done or retargeted.
  33 +* Each year, update copyright notices. Just do a case-insensitive search for copyright. Don't forget copyright in manual. Also update debian copyright in debian package. Last updated: 2017.
  34 +* To construct a source distribution from a pristine checkout, `make_dist` does the following:
  35 + ```
  36 + ./autogen.sh
  37 + ./configure --enable-doc-maintenance --enable-werror
  38 + make build_manual
  39 + make distclean
  40 + ```
  41 +* To create a source release, do an export from the version control system to a directory called qpdf-version. For example, from this directory:
  42 + ```
  43 + rm -rf /tmp/qpdf-x.y.z
  44 + git archive --prefix=qpdf-x.y.z/ HEAD . | (cd /tmp; tar xf -)
  45 + ```
  46 + From the parent of that directory, run `make_dist` with the directory as an argument. Remember to have fop in your path. For internally testing releases, you can run make_dist with the `--no-tests` option.
  47 +* To create a source release of external libs, do an export from the version control system into a directory called `qpdf-external-libs` and just make a zip file of the result called `qpdf-external-libs-src.zip`. See the README.txt file there for information on creating binary external libs releases. Run this from the external-libs repository:
  48 + ```
  49 + git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -)
  50 + cd /tmp
  51 + zip -r qpdf-external-libs-src.zip external-libs
  52 + ```
  53 +* To create Windows binary releases, extract the qpdf source distribution in Windows (MSYS2 + MSVC). From the extracted directory, extract the binary distribution of the external libraries. Run ./make_windows_releases from there.
  54 +* Before releasing, rebuild and test debian package.
  55 +* Remember to copy `README-what-to-download.md` separately onto the download area.
  56 +* Remember to update the web page including putting new documentation in the `files` subdirectory of the website on sourceforge.net.
  57 +* Create a tag in the version control system, and make backups of the actual releases. With git, use git `tag -s` to create a signed tag:
  58 + ```
  59 + git tag -s release-qpdf-$version HEAD -m"qpdf $version"
  60 + ```
  61 +* When releasing on sourceforge, `external-libs` distributions go in `external-libs/yyyymmdd`, and qpdf distributions go in `qpdf/vvv`. Make the source package the default for all but Windows, and make the 32-bit mingw build the default for Windows.
  62 +
  63 +# General Build Stuff
  64 +
  65 +QPDF uses autoconf and libtool but does not use automake. The only files distributed with the qpdf source distribution that are not controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`, `aclocal.m4`, and some documentation. See above for the steps required to prepare a source distribution.
  66 +
  67 +A small handful of additional files have been taken from autotools programs. These should probably be updated from time to time.
  68 +* `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory: these were created by running `libtoolize -c`. To update, run `libtoolize -f -c` or remove the files and rerun `libtoolize`.
  69 +* Other files copied as indicated:
  70 + ```
  71 + cp /usr/share/automake-1.11/install-sh .
  72 + cp /usr/share/automake-1.11/mkinstalldirs .
  73 + ```
  74 +
  75 +The entire contents of the `m4` directory came from `libtool.m4`. If we had some additional local parts, we could also add those to the `m4` directory. In order for this to work, it is necessary to run `aclocal -I m4` before running `autoheader` and `autoconf`. The `autogen.sh` script handles this.
  76 +
  77 +If building or editing documentation, configure with `--enable-doc-maintenance`. This will ensure that all tools or files required to validate and build documentation are available.
  78 +
  79 +If you want to run `make maintainer-clean`, `make distclean`, or `make autofiles.zip` and you haven't run `./configure`, you can pass `CLEAN=1` to make on the command line to prevent it from complaining about configure not having been run.
  80 +
  81 +If you want to run checks without rerunning the build, pass `NO_REBUILD=1` to make. This can be useful for special testing scenarios such as valgrind or binary compatibility.
  82 +
  83 +# Local Windows Testing Procedure
167 84
168 This is what I do for routine testing on Windows. 85 This is what I do for routine testing on Windows.
169 86
170 -From Linux, run ./autogen.sh and make autofiles.zip 87 +From Linux, run `./autogen.sh` and `make autofiles.zip CLEAN=1`.
171 88
172 -From Windows, git clone from my Linux clone, unzip external-libs, and  
173 -unzip autofiles.zip. 89 +From Windows, git clone from my Linux clone, unzip `external-libs`, and unzip `autofiles.zip`.
174 90
175 -Look at make_windows_releases. Set up path the same way and run  
176 -whichever ./config-* is appropriate for whichever compiler I need to  
177 -test with. Run from msys started from one of the visual studio command  
178 -line shells. The build doesn't work from git bash on my local system. 91 +Look at `make_windows_releases`. Set up path the same way and run whichever `./config-*` is appropriate for whichever compiler I need to test with. Start one of the Visual Studio native compiler shells, and from there, run one of the msys shells. The Visual Studio step is not necessary if just building with mingw.
README-what-to-download.md
1 -To build from source for Linux or other UNIX/UNIX-like systems, it is  
2 -generally sufficient to download just the source qpdf-<version>.tar.gz  
3 -file. 1 +To build from source for Linux or other UNIX/UNIX-like systems, it is generally sufficient to download just the source `qpdf-<version>.tar.gz` file.
4 2
5 -For Windows, there are several additional files that you might want to  
6 -download. 3 +For Windows, there are several additional files that you might want to download.
7 4
8 - * qpdf-<version>-bin-mingw32.zip 5 +* `qpdf-<version>-bin-mingw32.zip`
9 6
10 - If you just want to use the qpdf command line program or use the  
11 - qpdf DLL's C-language interface, you can download this file. You  
12 - can also download this version if you are using MINGW's gcc 4.6 (or  
13 - a binary compatible version) and want to program using the C++  
14 - interface. 7 + If you just want to use the qpdf command line program or use the qpdf DLL's C-language interface, you can download this file. You can also download this version if you are using MINGW's gcc and want to program using the C++ interface.
15 8
16 - * qpdf-<version>-bin-mingw64.zip 9 +* `qpdf-<version>-bin-mingw64.zip`
17 10
18 - A 64-bit version built with mingw. Use this for 64-bit Windows  
19 - systems. The 32-bit version will also work on Windows 64-bit.  
20 - Both the 32-bit and the 64-bit version support files over 2 GB in  
21 - size, but you may find it easier to integrate this with your own  
22 - software if you use the 64-bit version. 11 + A 64-bit version built with mingw. Use this for 64-bit Windows systems. The 32-bit version will also work on Windows 64-bit. Both the 32-bit and the 64-bit version support files over 2 GB in size, but you may find it easier to integrate this with your own software if you use the 64-bit version.
23 12
24 - * qpdf-<version>-bin-msvc32.zip 13 +* `qpdf-<version>-bin-msvc32.zip`
25 14
26 - If you want to program using qpdf's C++ interface and you are using  
27 - Microsoft Visual C++ 2010 in 32-bit mode, you can download this  
28 - file. 15 + If you want to program using qpdf's C++ interface and you are using Microsoft Visual C++ 2015 in 32-bit mode, you can download this file.
29 16
30 - * qpdf-<version>-bin-msvc64.zip 17 +* `qpdf-<version>-bin-msvc64.zip`
31 18
32 - If you want to program using qpdf's C++ interface and you are using  
33 - Microsoft Visual C++ 2010 in 64-bit mode, you can download this  
34 - file. 19 + If you want to program using qpdf's C++ interface and you are using Microsoft Visual C++ 2015 in 64-bit mode, you can download this file.
35 20
36 - * qpdf-external-libs-bin.zip 21 +* `qpdf-external-libs-bin.zip`
37 22
38 - If you want to build qpdf for Windows yourself with either MINGW or  
39 - MSVC 2010, you can download this file and extract it inside the  
40 - qpdf source distribution. Please refer to README-windows.md in  
41 - the qpdf source distribution for additional details. Note that you  
42 - need the 2012-06-20 version or later to be able to build qpdf 3.0  
43 - or newer. The 2009-10-24 version is required for qpdf 2.3.1 or  
44 - older. 23 + If you want to build qpdf for Windows yourself with either MINGW or MSVC 2015, you can download this file and extract it inside the qpdf source distribution. Please refer to README-windows.md in the qpdf source distribution for additional details. Note that you need the 2017-08-21 version or later to be able to build qpdf 7.0 or newer. Generally grab the `external-libs` distribution that was the latest version at the time of the release of whichever version of qpdf you are building.
45 24
46 - * qpdf-external-libs-src.zip 25 +* `qpdf-external-libs-src.zip`
47 26
48 - If you want to build the external libraries on your own (for  
49 - Windows or anything else), you can download this archive. In  
50 - addition to including an unmodified distribution zlib and the jpeg  
51 - libary, it includes a README file and some scripts to help you  
52 - build it for Windows. qpdf was built using a binary distribution of  
53 - libjpeg-turbo for Windows. You will also have to provide those. See  
54 - README-windows.md for details. 27 + If you want to build the external libraries on your own (for Windows or anything else), you can download this archive. In addition to including an unmodified distribution `zlib` and the `jpeg` libary, it includes a `README` file and some scripts to help you build it for Windows. You will also have to provide those.
  28 +
  29 +If you want to build on Windows, please see also README-windows.md in the qpdf source distribution.
55 30
56 -If you want to build on Windows, please see also README-windows.md.  
README-windows.md
1 Common Setup 1 Common Setup
2 ============ 2 ============
3 3
4 -You may need to disable antivirus software to run qpdf's test suite. 4 +You may need to disable antivirus software to run qpdf's test suite. Running Windows Defender on Windows 10 does not interfere with building or running qpdf or its test suite.
5 5
6 -To be able to build qpdf and run its test suite, you must have MSYS2  
7 -installed. This replaces the old process of having a mixture of msys,  
8 -mingw-w64, and ActiveState perl. It is now possible to do everything  
9 -with just MSYS2. 6 +To be able to build qpdf and run its test suite, you must have MSYS2 installed. This replaces the old process of having a mixture of msys, mingw-w64, and ActiveState perl. It is now possible to do everything with just MSYS2.
10 7
11 Here's what I did on my system: 8 Here's what I did on my system:
12 9
@@ -16,214 +13,95 @@ Here&#39;s what I did on my system: @@ -16,214 +13,95 @@ Here&#39;s what I did on my system:
16 * From the prompt: 13 * From the prompt:
17 * Run `pacman -Syuu` and follow the instructions, which may tell you 14 * Run `pacman -Syuu` and follow the instructions, which may tell you
18 to close the window and rerun the command multiple times. 15 to close the window and rerun the command multiple times.
19 - * pacman -S make base-devel git zip unzip  
20 - * pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain  
21 -  
22 -If you would like to build with Microsoft Visual C++, install a  
23 -suitable Microsoft Visual Studio edition. In early 2016, 2015  
24 -community edition with C++ support is fine. It may crash a few times  
25 -during installation, but repeating the installation will allow it to  
26 -finish, and the resulting software is stable.  
27 -  
28 -To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64  
29 -shell from a command window started from one of the Visual Studio  
30 -shell windows. You must use the mingw shell for the same word size (32  
31 -or 64 bit) as the Windows compiler since the MSVC build uses objdump  
32 -from the msys distribution. You must also have it inherit the path.  
33 -For example: 16 + * `pacman -S make base-devel git zip unzip`
  17 + * `pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain`
34 18
35 -* Start x64 native tools command prompt from msvc  
36 -* set MSYS2_PATH_TYPE=inherit  
37 -* C:\msys64\mingw64  
38 -  
39 -Image comparison tests are disabled by default, but it is possible to  
40 -run them on Windows. To do so, add --enable-test-compare-images from  
41 -the configure statements given below and install some additional  
42 -third-party dependencies. These may be provided in an environment such  
43 -as MSYS or Cygwin or can be downloaded separately for other  
44 -environments. You may extract or install the following software into  
45 -separate folders each and add the "bin" folder to your "PATH"  
46 -environment variable to make executables and DLLs available. If  
47 -installers are provided, they might do that already by default.  
48 -  
49 - * LibJpeg (http://gnuwin32.sourceforge.net/packages/jpeg.htm)  
50 -  
51 - This archive provides some needed DLLs needed by LibTiff.  
52 -  
53 - * LibTiff (http://gnuwin32.sourceforge.net/packages/tiff.htm)  
54 -  
55 - This archive provides some needed binaries and DLLs if you want to  
56 - use the image comparison tests. It depends on some DLLs from  
57 - LibJpeg.  
58 -  
59 - * GhostScript (http://www.ghostscript.com/download/gsdnld.html) 19 +If you would like to build with Microsoft Visual C++, install a suitable Microsoft Visual Studio edition. In early 2016, 2015 community edition with C++ support is fine. It may crash a few times during installation, but repeating the installation will allow it to finish, and the resulting software is stable.
60 20
61 - GhostScript is needed for image comparison tests. It's important  
62 - that the binary is available as "gs", while its default name is  
63 - "gswin32[c].exe". You can either copy one of the original files,  
64 - use "mklink" to create a hard-/softlink, or provide a custom  
65 - "gs.cmd" wrapper that forwards all arguments to one of the original  
66 - binaries. Using "mklink" with "gswin32c.exe" is probably the best  
67 - choice. 21 +To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64 shell from a command window started from one of the Visual Studio shell windows. You must use the mingw shell for the same word size (32 or 64 bit) as the Windows compiler since the MSVC build uses objdump from the msys distribution. You must also have it inherit the path. For example:
68 22
  23 +* Start x64 native tools command prompt from msvc
  24 +* `set MSYS2_PATH_TYPE=inherit`
  25 +* `C:\msys64\mingw64`
69 26
70 -External Libraries  
71 -==================  
72 -  
73 -In order to build qpdf, you must have a copy of zlib and the jpeg  
74 -library. The easy way to get it is to download the external libs from  
75 -the qpdf download area. There are packages called  
76 -external-libs-bin.zip and external-libs-src.zip. If you are building  
77 -with MSVC 2015 or MINGW with MSYS2, you can just extract the  
78 -qpdf-external-libs-bin.zip zip file into the top-level qpdf source  
79 -tree. Note that you need the 2017-08-21 version (at least) to build  
80 -qpdf 7.0 or greater since this includes jpeg. Passing  
81 ---enable-external-libs to ./configure (which is done automatically if  
82 -you follow the instructions below) is sufficient to find them.  
83 -  
84 -You can also obtain zlib and jpeg directly on your own and install  
85 -them. If you are using mingw, you can just set CPPFLAGS, LDFLAGS, and  
86 -LIBS when you run ./configure so that it can find the header files and  
87 -libraries. If you are building with msvc and you want to do this, it  
88 -probably won't work because ./configure doesn't know how to interpret  
89 -LDFLAGS and LIBS properly for MSVC (though qpdf's own build system  
90 -does). In this case, you can probably get away with cheating by  
91 -passing --enable-external-libs to ./configure and then just editing  
92 -CPPFLAGS, LDFLAGS, LIBS in the generated autoconf.mk file. Note that  
93 -you should use UNIX-like syntax (-I, -L, -l) even though this is not  
94 -what cl takes on the command line. qpdf's build rules will fix it. 27 +Image comparison tests are disabled by default, but it is possible to run them on Windows. To do so, add `--enable-test-compare-images` from the configure statements given below and install some additional third-party dependencies. These may be provided in an environment such as MSYS or Cygwin or can be downloaded separately for other environments. You may extract or install the following software into separate folders each and add the `bin` folder to your `PATH` environment variable to make executables and DLLs available. If installers are provided, they might do that already by default.
95 28
96 -You can also download qpdf-external-libs-src.zip and follow the  
97 -instructions in the README.txt there for how to build external libs. 29 +* [LibJpeg](http://gnuwin32.sourceforge.net/packages/jpeg.htm): This archive provides some needed DLLs needed by LibTiff.
  30 +* [LibTiff](http://gnuwin32.sourceforge.net/packages/tiff.htm): This archive provides some needed binaries and DLLs if you want to use the image comparison tests. It depends on some DLLs from LibJpeg.
  31 +* [GhostScript](http://www.ghostscript.com/download/gsdnld.html): GhostScript is needed for image comparison tests. It's important that the binary is available as `gs`, while its default name is `gswin32[c].exe`. You can either copy one of the original files, use `mklink` to create a hard/softlink, or provide a custom `gs.cmd` wrapper that forwards all arguments to one of the original binaries. Using `mklink` with `gswin32c.exe` is probably the best choice.
98 32
  33 +# External Libraries
99 34
100 -Building from version control  
101 -============================= 35 +In order to build qpdf, you must have a copy of `zlib` and the `jpeg` library. The easy way to get it is to download the external libs from the qpdf download area. There are packages called `external-libs-bin.zip` and `external-libs-src.zip`. If you are building with MSVC 2015 or MINGW with MSYS2, you can just extract the `qpdf-external-libs-bin.zip` zip file into the top-level qpdf source tree. Note that you need the 2017-08-21 version (at least) to build qpdf 7.0 or greater since this includes jpeg. Passing `--enable-external-libs` to `./configure` (which is done automatically if you follow the instructions below) is sufficient to find them.
102 36
103 -If you check out qpdf from version control, you will not have the  
104 -files that are generated by autoconf. If you are not changing these  
105 -files, you can grab them from a source distribution or create them  
106 -from a system that has autoconf. To create them from scratch, run  
107 -./autogen.sh on a system that has autoconf installed. Once you have  
108 -them, you can run make CLEAN=1 autofiles.zip. This will create an  
109 -autofiles.zip that you can extract on top of a fresh checkout. 37 +You can also obtain `zlib` and `jpeg` directly on your own and install them. If you are using mingw, you can just set `CPPFLAGS`, `LDFLAGS`, and `LIBS` when you run ./configure so that it can find the header files and libraries. If you are building with MSVC and you want to do this, it probably won't work because `./configure` doesn't know how to interpret `LDFLAGS` and `LIBS` properly for MSVC (though qpdf's own build system does). In this case, you can probably get away with cheating by passing `--enable-external-libs` to `./configure` and then just editing `CPPFLAGS`, `LDFLAGS`, `LIBS` in the generated autoconf.mk file. Note that you should use UNIX-like syntax (`-I`, `-L`, `-l`) even though this is not what cl takes on the command line. qpdf's build rules will fix it.
110 38
  39 +You can also download `qpdf-external-libs-src.zip` and follow the instructions in the README.txt there for how to build external libs.
111 40
112 -Building with MinGW  
113 -=================== 41 +# Building from version control
114 42
115 -QPDF is known to build and pass its test suite with MSYS2 using the  
116 -32-bit and 64-bit compilers from that project and Microsoft Visual C++  
117 -2015, both 32-bit and 64-bit versions. MSYS2 is required to build as  
118 -well in order to get make and other related tools. See common setup at  
119 -the top of this file for installation and configuration of MSYS2.  
120 -Then, from the suitable 32-bit or 64-bit environment, run 43 +If you check out qpdf from version control, you will not have the files that are generated by autoconf. If you are not changing these files, you can grab them from a source distribution or create them from a system that has autoconf. To create them from scratch, run `./autogen.sh` on a system that has autoconf installed. Once you have them, you can run `make CLEAN=1 autofiles.zip`. This will create an autofiles.zip that you can extract on top of a fresh checkout.
121 44
122 - ./config-mingw 45 +# Building with MinGW
123 46
124 -and then 47 +QPDF is known to build and pass its test suite with MSYS2 using the 32-bit and 64-bit compilers. MSYS2 is required to build as well in order to get make and other related tools. See common setup at the top of this file for installation and configuration of MSYS2. Then, from the suitable 32-bit or 64-bit environment, run
125 48
126 - make 49 +```
  50 +./config-mingw
  51 +make
  52 +```
127 53
128 -Note that ./config-mingw just runs ./configure with specific  
129 -arguments, so you can look at it, make adjustments, and manually run  
130 -configure instead. 54 +Note that `./config-mingw` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
131 55
132 -Add the absolute path to the libqpdf/build directory to your PATH.  
133 -Make sure you can run the qpdf command by typing qpdf/build/qpdf and  
134 -making sure you get a help message rather than an error loading the  
135 -DLL or no output at all. Run the test suite by typing 56 +Add the absolute path to the `libqpdf/build` directory to your `PATH`. Make sure you can run the qpdf command by typing qpdf/build/qpdf and making sure you get a help message rather than an error loading the DLL or no output at all. Run the test suite by typing
136 57
137 - make check 58 +```
  59 +make check
  60 +```
138 61
139 If all goes well, you should get a passing test suite. 62 If all goes well, you should get a passing test suite.
140 63
141 -To create an installation directory, run make install. This will  
142 -create install-mingw/qpdf-VERSION and populate it. The binary  
143 -download of qpdf for Windows with mingw is created from this  
144 -directory.  
145 -  
146 -You can also take a look at make_windows_releases for reference. This  
147 -is how the distributed Windows executables are created. 64 +To create an installation directory, run `make install`. This will create `install-mingw/qpdf-VERSION` and populate it. The binary download of qpdf for Windows with mingw is created from this directory.
148 65
  66 +You can also take a look at `make_windows_releases` for reference. This is how the distributed Windows executables are created.
149 67
150 -Building with MSVC 2015  
151 -======================= 68 +# Building with MSVC 2015
152 69
153 -These instructions would likely work with newer versions of MSVC and  
154 -are known to have worked with versions as old as 2008 Express. 70 +These instructions would likely work with newer versions of MSVC and are known to have worked with versions as old as 2008 Express.
155 71
156 -You should first set up your environment to be able to run MSVC from  
157 -the command line. There is usually a batch file included with MSVC  
158 -that does this. Make sure that you start a command line environment  
159 -configured for whichever of 32-bit or 64-bit output that you intend to  
160 -build for. 72 +You should first set up your environment to be able to run MSVC from the command line. There is usually a batch file included with MSVC that does this. Make sure that you start a command line environment configured for whichever of 32-bit or 64-bit output that you intend to build for.
161 73
162 -From that cmd prompt, you can start your MSYS2 shell with path  
163 -inheritance as described above. 74 +From that cmd prompt, you can start your MSYS2 shell with path inheritance as described above.
164 75
165 -Configure as follows: 76 +Configure and build as follows:
166 77
167 - ./config-msvc 78 +```
  79 +./config-msvc
  80 +make
  81 +```
168 82
169 -Once configured, run 83 +Note that `./config-msvc` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
170 84
171 - make 85 +NOTE: automated dependencies are not generated with the msvc build. If you're planning on making modifications, you should probably work with mingw. If there is a need, I can add dependency information to the msvc build, but since I only use it for generating release versions, I haven't bothered.
172 86
173 -Note that ./config-msvc just runs ./configure with specific arguments,  
174 -so you can look at it, make adjustments, and manually run configure  
175 -instead. 87 +Once built, add the full path to the `libqpdf/build` directory to your path and run
176 88
177 -NOTE: automated dependencies are not generated with the msvc build.  
178 -If you're planning on making modifications, you should probably work  
179 -with mingw. If there is a need, I can add dependency information to  
180 -the msvc build, but since I only use it for generating release  
181 -versions, I haven't bothered. 89 +```
  90 +make check
  91 +```
182 92
183 -Once built, add the full path to the libqpdf/build directory to your  
184 -path and run 93 +to run the test suite.
185 94
186 - make check 95 +If you are building with MSVC and want to debug a crash in MSVC's debugger, first start an instance of Visual C++. Then run qpdf. When the abort/retry/ignore dialog pops up, first attach the process from within visual C++, and then click Retry in qpdf.
187 96
188 -to run the test suite. 97 +A release version of qpdf is built by default. If you want to link against debugging libraries, you will have to change `/MD` to `/MDd` in `make/msvc.mk`. Note that you must redistribute the Microsoft runtime DLLs. Linking with static runtime (`/MT`) won't work; see "Static Runtime" below for details.
189 98
190 -If you are building with MSVC and want to debug a crash in MSVC's  
191 -debugger, first start an instance of Visual C++. Then run qpdf. When  
192 -the abort/retry/ignore dialog pops up, first attach the process from  
193 -within visual C++, and then click Retry in qpdf. 99 +# Runtime DLLs
194 100
195 -A release version of qpdf is built by default. If you want to link  
196 -against debugging libraries, you will have to change /MD to /MDd in  
197 -make/msvc.mk. Note that you must redistribute the Microsoft runtime  
198 -DLLs. Linking with static runtime (/MT) won't work; see "Static  
199 -Runtime" below for details. 101 +Both build methods create executables and DLLs that are dependent on the compiler's runtime DLLs. When you run make install, the installation process will automatically detect the DLLs and copy them into the installation bin directory. Look at the `copy_dlls` script for details on how this is accomplished.
200 102
  103 +Redistribution of the runtime DLL is unavoidable as of this writing; see "Static Runtime" below for details.
201 104
202 -Runtime DLLs  
203 -============ 105 +# Static Runtime
204 106
205 -Both build methods create executables and DLLs that are dependent on  
206 -the compiler's runtime DLLs. When you run make install, the  
207 -installation process will automatically detect the DLLs and copy them  
208 -into the installation bin directory. Look at the copy_dlls script for  
209 -details on how this is accomplished.  
210 -  
211 -Redistribution of the runtime DLL is unavoidable as of this writing;  
212 -see "Static Runtime" below for details.  
213 -  
214 -  
215 -Static Runtime  
216 -==============  
217 -  
218 -Building the DLL and executables with static runtime does not work  
219 -with either Visual C++ .NET 2008 (a.k.a. vc9) using /MT or with mingw  
220 -(at least as of 4.4.0) using -static-libgcc. The reason is that, in  
221 -both cases, there is static data involved with exception handling, and  
222 -when the runtime is linked in statically, exceptions cannot be thrown  
223 -across the DLL to EXE boundary. Since qpdf uses exception handling  
224 -extensively for error handling, we have no choice but to redistribute  
225 -the C++ runtime DLLs. Maybe this will be addressed in a future  
226 -version of the compilers. This has not been retested with the  
227 -toolchain versions used to create qpdf 3.0 distributions. (This has  
228 -not been revisited since MSVC 2008, but redistrbuting runtime DLLs is  
229 -extremely common and should not be a problem.) 107 +Building the DLL and executables with static runtime does not work with either Visual C++ .NET 2008 (a.k.a. vc9) using `/MT` or with mingw (at least as of 4.4.0) using `-static-libgcc`. The reason is that, in both cases, there is static data involved with exception handling, and when the runtime is linked in statically, exceptions cannot be thrown across the DLL to EXE boundary. Since qpdf uses exception handling extensively for error handling, we have no choice but to redistribute the C++ runtime DLLs. Maybe this will be addressed in a future version of the compilers. This has not been retested with the toolchain versions used to create qpdf >= 3.0 distributions. This has not been revisited since MSVC 2008, but redistrbuting runtime DLLs is extremely common and should not be a problem.
README.md
1 -This is the QPDF package. Information about it can be found at  
2 -http://qpdf.sourceforge.net. The source code repository is hosted  
3 -at github: https://github.com/qpdf/qpdf. 1 +This is the QPDF package. Information about it can be found at https://qpdf.sourceforge.net. The source code repository is hosted at github: https://github.com/qpdf/qpdf.
4 2
5 QPDF is copyright (c) 2005-2017 Jay Berkenbilt 3 QPDF is copyright (c) 2005-2017 Jay Berkenbilt
6 4
7 -This software may be distributed under the terms of version 2 of the  
8 -Artistic License which may be found in the source distribution as  
9 -"Artistic-2.0". It is provided "as is" without express or implied  
10 -warranty. 5 +This software may be distributed under the terms of version 2 of the Artistic License which may be found in the source distribution as "Artistic-2.0". It is provided "as is" without express or implied warranty.
11 6
  7 +# Prerequisites
12 8
13 -Prerequisites  
14 -============= 9 +QPDF depends on the external libraries [zlib](http://www.zlib.net/) and [jpeg](http://www.ijg.org/files/). The [libjpeg-turbo](https://libjpeg-turbo.org/) library is also known to work since it is compatible with the regular jpeg library, and QPDF doesn't use any interfaces that aren't present in the straight jpeg8 API. These are part of every Linux distribution and are readily available. Download information appears in the documentation. For Windows, you can download pre-built binary versions of these libraries for some compilers; see [README-windows.md](README-windows.md) for additional details.
15 10
16 -QPDF depends on the external libraries "zlib" and "jpeg". These are  
17 -part of every Linux distribution and are readily available. Download  
18 -information appears in the documentation. For Windows, you can  
19 -download pre-built binary versions of these libraries for some  
20 -compilers; see README-windows.md for additional details. 11 +QPDF requires a C++ compiler that works with STL. Your compiler must also support `long long`. Almost all modern compilers do. If you are trying to port qpdf to a compiler that doesn't support `long long`, you could change all occurrences of `long long` to `long` in the source code, noting that this would break binary compatibility with other builds of qpdf. Doing so would certainly prevent qpdf from working with files larger than 2 GB, but remaining functionality would most likely work fine. If you built qpdf this way and it passed its test suite with large file support disabled, you could be confident that you had an otherwise working qpdf.
21 12
22 -QPDF requires a C++ compiler that works with STL. Your compiler must  
23 -also support "long long". Almost all modern compilers do. If you are  
24 -trying to port qpdf to a compiler that doesn't support long long, you  
25 -could change all occurrences of "long long" to "long" in the source  
26 -code, noting that this would break binary compatibility with other  
27 -builds of qpdf. Doing so would certainly prevent qpdf from working  
28 -with files larger than 2 GB, but remaining functionality would most  
29 -likely work fine. If you built qpdf this way and it passed its test  
30 -suite with large file support disabled, you could be confident that  
31 -you had an otherwise working qpdf. 13 +# Licensing terms of embedded software
32 14
  15 +QPDF makes use of zlib and jpeg libraries for its functionality. These packages can be downloaded separately from their own download locations, or they can be downloaded in the external-libs area of the qpdf download site.
33 16
34 -Licensing terms of embedded software  
35 -====================================  
36 -  
37 -QPDF makes use of zlib and jpeg libraries for its functionality. These  
38 -packages can be downloaded separately from their own download  
39 -locations, or they can be downloaded in the external-libs area of the  
40 -qpdf download site.  
41 -  
42 -The Rijndael encryption implementation used as the basis for AES  
43 -encryption and decryption support comes from Philip J. Erdelsky's  
44 -public domain implementation. The files libqpdf/rijndael.cc and  
45 -libqpdf/qpdf/rijndael.h remain in the public domain. They were  
46 -obtained from  
47 -  
48 - http://www.efgh.com/software/rijndael.htm  
49 - http://www.efgh.com/software/rijndael.txt 17 +The Rijndael encryption implementation used as the basis for AES encryption and decryption support comes from Philip J. Erdelsky's public domain implementation. The files `libqpdf/rijndael.cc` and `libqpdf/qpdf/rijndael.h` remain in the public domain. They were obtained from
  18 +* http://www.efgh.com/software/rijndael.htm
  19 +* http://www.efgh.com/software/rijndael.txt
50 20
51 The embedded sha2 code comes from sphlib 3.0 21 The embedded sha2 code comes from sphlib 3.0
52 -  
53 - http://www.saphir2.com/sphlib/ 22 +* http://www.saphir2.com/sphlib/
54 23
55 That code has the following license: 24 That code has the following license:
56 - 25 + ```
57 Copyright (c) 2007-2011 Projet RNRT SAPHIR 26 Copyright (c) 2007-2011 Projet RNRT SAPHIR
58 27
59 Permission is hereby granted, free of charge, to any person obtaining 28 Permission is hereby granted, free of charge, to any person obtaining
@@ -74,166 +43,59 @@ That code has the following license: @@ -74,166 +43,59 @@ That code has the following license:
74 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 43 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
75 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 44 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
76 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 45 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  46 + ```
77 47
  48 +# Building from a pristine checkout
78 49
79 -Building from a pristine checkout  
80 -=================================  
81 -  
82 -When building qpdf from a pristine checkout from version control,  
83 -documentation and automatically generated files are not present.  
84 -Building on Windows from a pristine checkout is not guaranteed to work  
85 -because of issues running autoconf; see README-windows.md for how to  
86 -handle this. For UNIX and UNIX-like systems, you must have some  
87 -addditional tools installed to build from the source repository. To  
88 -do this, you should run 50 +When building qpdf from a pristine checkout from version control, documentation and automatically generated files are not present. Building on Windows from a pristine checkout is not guaranteed to work because of issues running autoconf; see [README-windows.md](README-windows.md) for how to handle this. For UNIX and UNIX-like systems, you must have some addditional tools installed to build from the source repository. To do this, you should run
89 51
  52 +```
90 ./autogen.sh 53 ./autogen.sh
91 ./configure --enable-doc-maintenance 54 ./configure --enable-doc-maintenance
92 make 55 make
93 make install 56 make install
  57 +```
94 58
95 -If you don't have Apache fop and the docbook stylesheets installed,  
96 -you won't be able to build documentation. You can omit  
97 ---enable-doc-maintenance and produce working qpdf software that passes  
98 -its test suite, but make install will fail because the documentation  
99 -files won't exist. Depending on your purposes, you can either work  
100 -around this or grab the docs from a source distribution.  
101 - 59 +If you don't have Apache fop and the docbook stylesheets installed, you won't be able to build documentation. You can omit `--enable-doc-maintenance` and produce working qpdf software that passes its test suite, but make install will fail because the documentation files won't exist. Depending on your purposes, you can either work around this or grab the docs from a source distribution.
102 60
103 -Building from source distribution on UNIX/Linux  
104 -=============================================== 61 +# Building from source distribution on UNIX/Linux
105 62
106 For UNIX and UNIX-like systems, you can usually get by with just 63 For UNIX and UNIX-like systems, you can usually get by with just
107 64
  65 +```
108 ./configure 66 ./configure
109 make 67 make
110 make install 68 make install
  69 +```
  70 +
  71 +Packagers may set DESTDIR, in which case make install will install inside of DESTDIR, as is customary with many packages. For more detailed general information, see the "INSTALL" file in this directory. If you are already accustomed to building and installing software that uses autoconf, there's nothing new for you in the INSTALL file. Note that qpdf uses `autoconf` but not `automake`. We have our own system of Makefiles that allows cross-directory dependencies, doesn't use recursive make, and works better on non-UNIX platforms.
  72 +
  73 +# Building on Windows
  74 +
  75 +QPDF is known to build and pass its test suite with mingw (latest version tested: gcc 7.2.0), mingw64 (latest version tested: 7.2.0) and Microsoft Visual C++ 2015, both 32-bit and 64-bit versions. MSYS2 is required to build as well in order to get make and other related tools. See [README-windows.md](README-windows.md) for details on how to build under Windows.
  76 +
  77 +# Additional Notes on Build
  78 +
  79 +QPDF's build system, inspired by [abuild](http://www.abuild.org), can optionally use its own built-in rules rather than using libtool and obeying the compiler specified with configure. This can be enabled by passing `--with-buildrules=buildrules` where buildrules corresponds to one of the `.mk` files (other than `rules.mk`) in the make directory. This should never be necessary on a UNIX system, but may be necessary on a Windows system. See [README-windows.md](README-windows.md) for details. There is a `gcc-linux.mk` file enable `gcc-linux` build rules, but it is intended to help test the build system; Linux users should build with the `libtools` rules, which are enabled by default.
  80 +
  81 +The QPDF package provides some executables and a software library. A user manual can be found in the "doc" directory. The docbook sources to the user manual can be found in the `manual` directory.
  82 +
  83 +The software library is just `libqpdf`, and all the header files are in the `qpdf` subdirectories of `include` and `libqpdf`. If you link statically with `-lqpdf`, then you will also need to link with `-lz` and `-ljpeg`. The shared qpdf library is linked with `-lz` and `-ljpeg`, none of qpdf's public header files directly include files from `libz`, and only `Pl_DCT.hh` includes files from `libjpeg`, so for most cases, qpdf's development files are self contained. If you need to use `Pl_DCT` in your application code, you will need to have the header files for some libjpeg distribution in your include path.
  84 +
  85 +To learn about using the library, please read comments in the header files in `include/qpdf`, especially `QPDF.hh`, `QPDFObjectHandle.hh`, and
  86 +`QPDFWriter.hh`. These are the best sources of documentation on the API. You can also study the code of `qpdf/qpdf.cc`, which exercises most of the public interface. There are additional example programs in the examples directory. Reading all the source files in the `qpdf` directory (including the qpdf command-line tool and some test drivers) along with the code in the examples directory will give you a complete picture of every aspect of the public interface.
  87 +
  88 +# Additional Notes on Test Suite
  89 +
  90 +By default, slow tests and tests that require dependencies beyond those needed to build qpdf are disabled. Slow tests include image comparison tests and large file tests. Image comparison tests can be enabled by passing `--enable-test-compare-images` to ./configure. This was on by default in qpdf versions prior to 3.0, but is now off by default. Large file tests can be enabled by passing `--with-large-file-test-path=path` to `./configure` or by setting the `QPDF_LARGE_FILE_TEST_PATH` environment variable. On Windows, this should be a Windows path. Run `./configure --help` for additional options. The test suite provides nearly full coverage even without these tests. Unless you are making deep changes to the library that would impact the contents of the generated PDF files or testing this on a new platform for the first time, there is no real reason to run these tests. If you're just running the test suite to make sure that qpdf works for your build, the default tests are adequate. The configure rules for these tests do nothing other than setting variables in `autoconf.mk`, so you can feel free to turn these on and off directly in `autoconf.mk` rather than rerunning configure.
  91 +
  92 +If you are packaging qpdf for a distribution and preparing a build that is run by an autobuilder, you may want to add the `--enable-show-failed-test-output` to configure options. This way, if the test suite fails, test failure detail will be included in the build output. Otherwise, you will have to have access to the `qtest.log` file from the build to view test failures. The debian packages for qpdf enable this option.
  93 +
  94 +# Random Number Generation
  95 +
  96 +By default, when `qpdf` detects either the Windows cryptography API or the existence of `/dev/urandom`, `/dev/arandom`, or `/dev/random`, it uses them to generate cryptography secure random numbers. If none of these conditions are true, the build will fail with an error. This behavior can be modified in several ways:
  97 +* If you configure with `--disable-os-secure-random` or define `SKIP_OS_SECURE_RANDOM`, qpdf will not attempt to use Windows cryptography or the random device. You must either supply your own random data provider or allow use of insecure random numbers.
  98 +* If you configure qpdf with the `--enable-insecure-random` option or define `USE_INSECURE_RANDOM`, qpdf will try insecure random numbers if OS-provided secure random numbers are disabled. This is not a fallback. In order for insecure random numbers to be used, you must also disable OS secure random numbers since, otherwise, failure to find OS secure random numbers is a compile error. The insecure random number source is stdlib's `random()` or `rand()` calls. These random numbers are not cryptography secure, but the qpdf library is fully functional using them. Using non-secure random numbers means that it's easier in some cases to guess encryption keys. If you're not generating encrypted files, there's no advantage to using secure random numbers.
  99 +* In all cases, you may supply your own random data provider. To do this, derive a class from `qpdf/RandomDataProvider` (since version 5.1.0) and call `QUtil::setRandomDataProvider` before you create any `QPDF` objects. If you supply your own random data provider, it will always be used even if support for one of the other random data providers is compiled in. If you wish to avoid any possibility of your build of qpdf from using anything but a user-supplied random data provider, you can define `SKIP_OS_SECURE_RANDOM` and not `USE_INSECURE_RANDOM`. In this case, qpdf will throw a runtime error if any attempt is made to generate random numbers and no random data provider has been supplied.
111 100
112 -Packagers may set DESTDIR, in which case make install will install  
113 -inside of DESTDIR, as is customary with many packages. For more  
114 -detailed general information, see the "INSTALL" file in this  
115 -directory. If you are already accustomed to building and installing  
116 -software that uses autoconf, there's nothing new for you in the  
117 -INSTALL file.  
118 -  
119 -  
120 -Building on Windows  
121 -===================  
122 -  
123 -QPDF is known to build and pass its test suite with mingw (latest  
124 -version tested: gcc 4.6.2), mingw64 (latest version tested: 4.7.0) and  
125 -Microsoft Visual C++ 2010, both 32-bit and 64-bit versions. MSYS plus  
126 -ActiveState Perl is required to build as well in order to get make  
127 -and other related tools. See README-windows.md for details on how to  
128 -build under Windows, see README-windows.md.  
129 -  
130 -  
131 -Additional Notes on Build  
132 -=========================  
133 -  
134 -QPDF's build system, inspired by abuild (http://www.abuild.org), can  
135 -optionally use its own built-in rules rather than using libtool and  
136 -obeying the compiler specified with configure. This can be enabled by  
137 -passing --with-buildrules=buildrules where buildrules corresponds to  
138 -one of the .mk files (other than rules.mk) in the make directory.  
139 -This should never be necessary on a UNIX system, but may be necessary  
140 -on a Windows system. See README-windows.md for details. There is a  
141 -gcc-linux.mk file enable "gcc-linux" build rules, but it is intended  
142 -to help test the build system; Linux users should build with the  
143 -"libtools" rules, which are enabled by default.  
144 -  
145 -The QPDF package provides some executables and a software library. A  
146 -user's manual can be found in the "doc" directory. The docbook  
147 -sources to the user's manual can be found in the "manual" directory.  
148 -  
149 -The software library is just libqpdf, and all the header files are in  
150 -the qpdf subdirectory. If you link statically with -lqpdf, then you  
151 -will also need to link with -lz and -ljpeg. The shared qpdf library is  
152 -linked with -lz and -ljpeg, none of qpdf's public header files  
153 -directly include files from libz, and only Pl_DCT.hh includes files  
154 -from libjpeg, so for most cases, qpdf's development files are self  
155 -contained. If you need to use Pl_DCT in your application code, you  
156 -will need to have the header files for some libjpeg distribution in  
157 -your include path.  
158 -  
159 -To learn about using the library, please read comments in the header  
160 -files in include/qpdf, especially QPDF.hh, QPDFObjectHandle.hh, and  
161 -QPDFWriter.hh. You can also study the code of qpdf/qpdf.cc, which  
162 -exercises most of the public interface. There are additional example  
163 -programs in the examples directory. Reading all the source files in  
164 -the qpdf directory (including the qpdf command-line tool and some test  
165 -drivers) along with the code in the examples directory will give you a  
166 -complete picture of every aspect of the public interface.  
167 -  
168 -  
169 -Additional Notes on Test Suite  
170 -==============================  
171 -  
172 -By default, slow tests are disabled. Slow tests include image  
173 -comparison tests and large file tests. Image comparison tests can be  
174 -enabled by passing --enable-test-compare-images to ./configure. This  
175 -was on by default in qpdf versions prior to 3.0, but is now off by  
176 -default. Large file tests can be enabled by passing  
177 ---with-large-file-test-path=path to ./configure or by setting the  
178 -QPDF_LARGE_FILE_TEST_PATH environment variable. On Windows, this  
179 -should be a Windows path. Run ./configure --help for additional  
180 -options. The test suite provides nearly full coverage even without  
181 -these tests. Unless you are making deep changes to the library that  
182 -would impact the contents of the generated PDF files or testing this  
183 -on a new platform for the first time, there is no real reason to run  
184 -these tests. If you're just running the test suite to make sure that  
185 -qpdf works for your build, the default tests are adequate. The  
186 -configure rules for these tests do nothing other than setting  
187 -variables in autoconf.mk, so you can feel free to turn these on and  
188 -off directly in autoconf.mk rather than rerunning configure.  
189 -  
190 -If you are packaging qpdf for a distribution and preparing a build  
191 -that is run by an autobuilder, you may want to add the  
192 ---enable-show-failed-test-output to configure options. This way, if  
193 -the test suite fails, test failure detail will be included in the  
194 -build output. Otherwise, you will have to have access to the  
195 -qtest.log file from the build to view test failures. The debian  
196 -packages for qpdf enable this option, for example.  
197 -  
198 -  
199 -Random Number Generation  
200 -========================  
201 -  
202 -By default, when the qpdf detects either the Windows cryptography API  
203 -or the existence of /dev/urandom, /dev/arandom, or /dev/random, it  
204 -uses them to generate cryptography secure random numbers. If none of  
205 -these conditions are true, the build will fail with an error. This  
206 -behavior can be modified in several ways:  
207 -  
208 - * If you configure with --disable-os-secure-random or define  
209 - SKIP_OS_SECURE_RANDOM, qpdf will not attempt to use Windows  
210 - cryptography or the random device. You must either supply your own  
211 - random data provider or allow use of insecure random numbers.  
212 -  
213 - * If you configure qpdf with the --enable-insecure-random option or  
214 - define USE_INSECURE_RANDOM, qpdf will try insecure random numbers  
215 - if OS-provided secure random numbers are disabled. This is not a  
216 - fallback. In order for insecure random numbers to be used, you  
217 - must also disable OS secure random numbers since, otherwise,  
218 - failure to find OS secure random numbers is a compile error. The  
219 - insecure random number source is stdlib's random() or rand() calls.  
220 - These random numbers are not cryptography secure, but the qpdf  
221 - library is fully functional using them. Using non-secure random  
222 - numbers means that it's easier in some cases to guess encryption  
223 - keys. If you're not generating encrypted files, there's no  
224 - advantage to using secure random numbers.  
225 -  
226 - * In all cases, you may supply your own random data provider. To do  
227 - this, derive a class from qpdf/RandomDataProvider (since 5.1.0) and  
228 - call QUtil::setRandomDataProvider before you create any QPDF  
229 - objects. If you supply your own random data provider, it will  
230 - always be used even if support for one of the other random data  
231 - providers is compiled in. If you wish to avoid any possibility of  
232 - your build of qpdf from using anything but a user-supplied random  
233 - data provider, you can define SKIP_OS_SECURE_RANDOM and not  
234 - USE_INSECURE_RANDOM. In this case, qpdf will throw a runtime error  
235 - if any attempt is made to generate random numbers and no random  
236 - data provider has been supplied.  
237 -  
238 -If you are building qpdf on a platform that qpdf doesn't know how to  
239 -generate secure random numbers on, a patch would be welcome. 101 +If you are building qpdf on a platform that qpdf doesn't know how to generate secure random numbers on, a patch would be welcome.