Commit f58d2a60d57653071609a0a9ec0d693c6bda0024

Authored by Jay Berkenbilt
Committed by Jay Berkenbilt
1 parent 70d0d088

Update build-related documentation and comments

README-maintainer
... ... @@ -15,7 +15,8 @@ cmake -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
15 15 Profiling:
16 16  
17 17 CFLAGS=-pg LDFLAGS=-pg \
18   - cmake -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 -DCMAKE_BUILD_TYPE=Debug ..
  18 + cmake -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
  19 + -DCMAKE_BUILD_TYPE=Debug ..
19 20  
20 21 Then run `gprof gmon.out`. Note that gmon.out is not cumulative.
21 22  
... ... @@ -25,12 +26,17 @@ CFLAGS="-fsanitize=address -fsanitize=undefined" \
25 26 CXXFLAGS="-fsanitize=address -fsanitize=undefined" \
26 27 LDFLAGS="-fsanitize=address -fsanitize=undefined" \
27 28 CC=clang CXX=clang++ \
28   - cmake -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 -DCMAKE_BUILD_TYPE=Debug ..
  29 + cmake -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
  30 + -DCMAKE_BUILD_TYPE=Debug ..
29 31  
30 32 Windows:
31 33  
32 34 ../cmake-win {mingw|msvc} maint
33 35  
  36 +See ./build-scripts for other ways to run the build for different
  37 +configurations.
  38 +
  39 +
34 40 VERSIONS
35 41  
36 42 * The version number on the main branch is whatever the version would
... ... @@ -249,53 +255,20 @@ RELEASE PREPARATION
249 255 * Run a spelling checker over the source code to catch errors in
250 256 variable names, strings, and comments.
251 257  
252   - make spell CLEAN=1
  258 + ./spell-check
253 259  
254 260 This uses cspell. Install with `npm install -g cspell`. The output
255 261 of cspell is suitable for use with `M-x grep` in emacs. Add
256 262 exceptions to cSpell.json.
257 263  
258   -* If needed, run large file and image comparison tests. Configure
259   - options:
  264 +* If needed, run large file and image comparison tests by setting
  265 + these environment variables:
260 266  
261   ---enable-test-compare-images --with-large-file-test-path=/path
  267 + QPDF_LARGE_FILE_TEST_PATH=/full/path
  268 + QPDF_TEST_COMPARE_IMAGES=1
262 269  
263 270 For Windows, use a Windows style path, not an MSYS path for large files.
264 271  
265   -* Test with clang. Pass `CC=clang CXX=clang++` to `./configure`. (Done
266   - in CI).
267   -
268   -* Test with newer version of gcc if available.
269   -
270   -* Test 32-bit. Pass `CC=i686-linux-gnu-gcc CXX=i686-linux-gnu-g++` to
271   - `./configure`. (Done in CI.)
272   -
273   -* Test build on a mac. (Done in CI.)
274   -
275   -* Test with address sanitizer as described above. (Done in CI.)
276   -
277   -* A small handful of additional files have been taken from autotools
278   - programs. These should probably be updated from time to time.
279   -
280   - * `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory:
281   - these were created by running `libtoolize -c`. To update, run
282   - `libtoolize -f -c` or remove the files and rerun `libtoolize`. For
283   - `config.guess` and `config.sub`, search for "latest" in the files,
284   - and follow directions for updating them.
285   -
286   - * Other files copied as indicated:
287   - ```
288   - cp /usr/share/automake-1.16/install-sh .
289   - cp /usr/share/automake-1.16/mkinstalldirs .
290   - cp /usr/share/aclocal/pkg.m4 m4
291   - ```
292   -
293   - The entire contents of the `m4` directory came from `libtool.m4`. If
294   - we had some additional local parts, we could also add those to the
295   - `m4` directory. In order for this to work, it is necessary to run
296   - `aclocal -I m4` before running `autoheader` and `autoconf`. The
297   - `autogen.sh` script handles this.
298   -
299 272 * If any interfaces were added or changed, check C API to see whether
300 273 changes are appropriate there as well. If necessary, review the
301 274 casting policy in the manual, and ensure that integer types are
... ... @@ -306,17 +279,12 @@ RELEASE PREPARATION
306 279 * Double check versions and shared library details. They should
307 280 already be up to date in the code.
308 281  
309   - * Increment shared library version information as needed
310   - (`LT_CURRENT` in `configure.ac`)
311   -
312 282 * Make sure version numbers are consistent in the following locations:
313   - * configure.ac
  283 + * CMakeLists.txt
314 284 * include/qpdf/DLL.h
315 285 * manual/conf.py
316 286 `make_dist` verifies this consistency.
317 287  
318   - * Run ./autogen.sh
319   -
320 288 * Update release notes in manual. Look at diffs and ChangeLog.
321 289 Update release date in `manual/release-notes.rst`.
322 290  
... ... @@ -330,15 +298,24 @@ RELEASE PREPARATION
330 298  
331 299 * Test for performance and binary compatibility:
332 300 * Check out the last release
333   - * ./configure --enable-werror && make -j$(nproc)
  301 + * cmake -S . -B build \
  302 + -DMAINTAINER_MODE=1 -DBUILD_STATIC_LIBS=0 -DBUILD_DOC=0 \
  303 + -DCMAKE_BUILD_TYPE=RelWithDebInfo
  304 + * cmake --build build -j$(nproc)
334 305 * Check out the current version
335 306 * ./performance_check | tee -a /tmp/perf
336   - * ./configure --enable-werror && make -j$(nproc) build_libqpdf
  307 + * cmake -S . -B build \
  308 + -DMAINTAINER_MODE=1 -DBUILD_STATIC_LIBS=0 -DBUILD_DOC=0 \
  309 + -DCMAKE_BUILD_TYPE=RelWithDebInfo
  310 + * cmake --build build -j$(nproc) --target libqpdf
337 311 * Checkout the last release
338   - * make -k check NO_REBUILD=1 (some failures are normal -- looking
339   - for binary compatibility)
  312 + * (cd build; ctest --verbose)
  313 + * (some failures are normal -- looking for binary compatibility)
340 314 * Check out the current version
341   - * make -j$(nproc)
  315 + * cmake -S . -B build \
  316 + -DMAINTAINER_MODE=1 -DBUILD_STATIC_LIBS=0 -DBUILD_DOC=0 \
  317 + -DCMAKE_BUILD_TYPE=RelWithDebInfo
  318 + * cmake --build build -j$(nproc)
342 319 * ./performance_check | tee -a /tmp/perf
343 320  
344 321 * Run pikepdf's test suite. Do this in a separate shell.
... ... @@ -363,7 +340,7 @@ CREATING A RELEASE
363 340 * Push to main. This will create an artifact called distribution
364 341 which will contain all the distribution files. Download these,
365 342 verify the checksums from the job output, rename to remove -ci from
366   - the names, and copy to the release archive area.
  343 + the names, and extract to the release archive area.
367 344  
368 345 * Sign the source distribution:
369 346  
... ... @@ -375,9 +352,7 @@ gpg --detach-sign --armor qpdf-$version.tar.gz
375 352 * Add a calendar reminder to check the status of the debian package to
376 353 make sure it is transitioning properly and to resolve any issues.
377 354  
378   -* Sign the releases. The release archive area should contain the
379   - Windows binaries, the AppImage, the source tarball, and the source
380   - tarball signature.
  355 +* From the release archive area, sign the releases.
381 356  
382 357 \rm -f *.sha256
383 358 files=(*)
... ... @@ -467,40 +442,25 @@ Use -e SKIP_TESTS=1 to skip the test suite.
467 442 Use -ti -e RUN_SHELL=1 to run a shell instead of the build script.
468 443  
469 444  
470   -GENERAL BUILD STUFF
471   -
472   -QPDF uses autoconf and libtool but does not use automake. The only
473   -files distributed with the qpdf source distribution that are not
474   -controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`,
475   -`aclocal.m4`, and some documentation. See above for the steps required
476   -to prepare a source distribution.
477   -
478   -If building or editing documentation, configure with
479   -`--enable-doc-maintenance`. This will ensure that all tools or files
480   -required to validate and build documentation are available.
  445 +LOCAL WINDOWS TESTING PROCEDURE
481 446  
482   -If you want to run `make maintainer-clean` or `make distclean` and you
483   -haven't run `./configure`, you can pass `CLEAN=1` to make on the
484   -command line to prevent it from complaining about configure not having
485   -been run.
  447 +This is what I do for routine testing on Windows.
486 448  
487   -If you want to run checks without rerunning the build, pass
488   -`NO_REBUILD=1` to make. This can be useful for special testing
489   -scenarios such as validation of memory fixes or binary compatibility.
  449 +* From Windows, git clone from my Linux clone, and unzip
  450 + `external-libs`.
490 451  
  452 +* Start a command-line shell for x86_64 and x86 tools from Visual
  453 + studio. From there, start C:\msys64\mingw64 twice and
  454 + C:\msys64\mingw32 twice.
491 455  
492   -LOCAL WINDOWS TESTING PROCEDURE
  456 +* Create a build directory for each of the four permutations. Then, in
  457 + each build directory, run `../cmake-win <tool> maint`.
493 458  
494   -This is what I do for routine testing on Windows.
  459 +* Run `cmake --build . -j4`. For MSVC, add `--config Release`
495 460  
496   -From Windows, git clone from my Linux clone, and unzip
497   -`external-libs`.
  461 +* Test with with msvc: `ctest --verbose -C Release`
498 462  
499   -Look at `make_windows_releases`. Set up path the same way and run
500   -whichever `./config-*` is appropriate for whichever compiler I need to
501   -test with. Start one of the Visual Studio native compiler shells, and
502   -from there, run one of the msys shells. The Visual Studio step is not
503   -necessary if just building with mingw.
  463 +* Test with mingw: `ctest --verbose -C RelWithDebInfo`
504 464  
505 465  
506 466 DOCS ON readthedocs.org
... ... @@ -529,3 +489,25 @@ following branching strategy to support docs:
529 489  
530 490 The release process includes updating the approach branches and
531 491 activating versions.
  492 +
  493 +
  494 +CMAKE notes
  495 +
  496 +To verify the various cmake options and their interactions, several
  497 +manual tests were done:
  498 +
  499 +* Break installed qpdf executables (qpdf, fix-qdf, zlib-flate), the
  500 + shared library, and DLL.h to ensure that other qpdf installations do
  501 + not interfere with building from source
  502 +
  503 +* Build static only and shared only
  504 +
  505 +* Install separate components separately
  506 +
  507 +* Build only HTML docs and only PDF docs
  508 +
  509 +* Try MAINTAINER_MODE without BUILD_DOC
  510 +
  511 +We are using RelWithDebInfo for mingw and other non-Windows builds but
  512 +Release for MSVC. There are linker warnings if MSVC is built with
  513 +RelWithDebInfo when using external-libs.
... ...
README-what-to-download.md
... ... @@ -4,13 +4,15 @@ Windows Binaries
4 4  
5 5 You can download Windows binaries that are statically linked with qpdf's external dependencies and use the OpenSSL crypto provider. There are several options:
6 6  
7   -* `qpdf-<version>-bin-mingw32.zip` - 32-bit executables that should work on basically any Windows system, including 64-bit systems. The 32-bit executables are capable of handling files larger than 2 GB. If you just want to use the qpdf command line program or use the qpdf DLL's C-language interface, you can download this file. You can also download this version if you are using MINGW's gcc and want to program using the C++ interface.
  7 +* For Windows installers, you can download the `.exe` file for a traditional installer that allows relocation, or you can download the `.zip` file which you can unzip to any location.
8 8  
9   -* `qpdf-<version>-bin-mingw64.zip` - A 64-bit version built with mingw. Use this for 64-bit Windows systems. The 32-bit version will also work on Windows 64-bit. Both the 32-bit and the 64-bit version support files over 2 GB in size, but you may find it easier to integrate this with your own software if you use the 64-bit version.
  9 +* `qpdf-<version>-bin-mingw32.exe` - 32-bit executables that should work on basically any Windows system, including 64-bit systems. The 32-bit executables are capable of handling files larger than 2 GB. If you just want to use the qpdf command line program or use the qpdf DLL's C-language interface, you can download this file. You can also download this version if you are using MINGW's gcc and want to program using the C++ interface.
10 10  
11   -* `qpdf-<version>-bin-msvc32.zip` - If you want to program using qpdf's C++ interface and you are using a recent version of Microsoft Visual C++ in 32-bit mode, you can download this file.
  11 +* `qpdf-<version>-bin-mingw64.exe` - A 64-bit version built with mingw. Use this for 64-bit Windows systems. The 32-bit version will also work on Windows 64-bit. Both the 32-bit and the 64-bit version support files over 2 GB in size, but you may find it easier to integrate this with your own software if you use the 64-bit version.
12 12  
13   -* `qpdf-<version>-bin-msvc64.zip` - If you want to program using qpdf's C++ interface and you are using a recent version of Microsoft Visual C++ in 64-bit mode, you can download this file.
  13 +* `qpdf-<version>-bin-msvc32.exe` - If you want to program using qpdf's C++ interface and you are using a recent version of Microsoft Visual C++ in 32-bit mode, you can download this file.
  14 +
  15 +* `qpdf-<version>-bin-msvc64.exe` - If you want to program using qpdf's C++ interface and you are using a recent version of Microsoft Visual C++ in 64-bit mode, you can download this file.
14 16  
15 17 Linux Binaries
16 18  
... ... @@ -22,7 +24,7 @@ Virtually all Linux distributions include packages for qpdf. There is also a PPA
22 24  
23 25 Windows Build Support
24 26  
25   -If you are building on Windows and want to use pre-built external static libraries, you should obtain current versions from https://github.com/qpdf/external-libs/releases. The `external-libs` directory contains older versions that will not work with qpdf versions >= 10.0.2. Please see README-windows.md in the qpdf source distribution.
  27 +If you are building on Windows and want to use pre-built external static libraries, you should obtain current versions from https://github.com/qpdf/external-libs/releases. The `external-libs` directory contains older versions that will not work with qpdf versions >= 10.0.2. Please see README-windows.md in the qpdf source distribution. The libraries from this distribution will not work with a Debug build with MSVC.
26 28  
27 29 Documentation
28 30  
... ...
README-windows.md
... ... @@ -3,7 +3,7 @@ Common Setup
3 3  
4 4 You may need to disable antivirus software to run qpdf's test suite. Running Windows Defender on Windows 10 does not interfere with building or running qpdf or its test suite.
5 5  
6   -To be able to build qpdf and run its test suite, you must have MSYS2 installed. This replaces the old process of having a mixture of msys, mingw-w64, and ActiveState perl. It is now possible to do everything with just MSYS2.
  6 +Starting with qpdf version 11, qpdf is built with cmake. You can build qpdf with Visual C++ in Release mode with the pre-built external-libraries distribution (described below) without having any additional tools installed. To run the test suite, you need MSYS2.
7 7  
8 8 Here's what I did on my system:
9 9  
... ... @@ -11,20 +11,23 @@ Here&#39;s what I did on my system:
11 11 * Run the installer.
12 12 * Run msys2_shell.cmd by allowing the installer to start it.
13 13 * From the prompt:
14   - * Run `pacman -Syuu` and follow the instructions, which may tell you
  14 + * Run `pacman -Syu` and follow the instructions, which may tell you
15 15 to close the window and rerun the command multiple times.
  16 + * Run `pacman -Su` to fully update.
16 17 * `pacman -S make base-devel git zip unzip`
17 18 * `pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain`
18 19  
19   -If you would like to build with Microsoft Visual C++, install a suitable Microsoft Visual Studio edition. In early 2016, 2015 community edition with C++ support is fine. It may crash a few times during installation, but repeating the installation will allow it to finish, and the resulting software is stable.
  20 +You need cmake. If you have Visual Studio installed, you can use the cmake that comes with it to build with both MSVC and mingw. You can also a install a native Windows cmake from cmake.org.
20 21  
21   -To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64 shell from a command window started from one of the Visual Studio shell windows. You must use the mingw shell for the same word size (32 or 64 bit) as the Windows compiler since the MSVC build uses objdump from the msys distribution. You must also have it inherit the path. For example:
  22 +To build qpdf with Visual Studio from msys2 so you can run its test suite, start the msys2 mingw32 or mingw64 shell from a command window started from one of the Visual Studio shell windows. You must have it inherit the path. For example:
22 23  
23 24 * Start x64 native tools command prompt from msvc
24 25 * `set MSYS2_PATH_TYPE=inherit`
25 26 * `C:\msys64\mingw64`
26 27  
27   -Image comparison tests are disabled by default, but it is possible to run them on Windows. To do so, add `--enable-test-compare-images` from the configure statements given below and install some additional third-party dependencies. These may be provided in an environment such as MSYS or Cygwin or can be downloaded separately for other environments. You may extract or install the following software into separate folders each and add the `bin` folder to your `PATH` environment variable to make executables and DLLs available. If installers are provided, they might do that already by default.
  28 +For the test suite to work properly, your build directory must be on the same drive as your source directory. This is because there are parts of the test environment that create relative paths from one to the other. You can use a cross-drive symlink if needed.
  29 +
  30 +Image comparison tests are disabled by default, but it is possible to run them on Windows. To do so, set the `QPDF_TEST_COMPARE_IMAGES` environment variable to `1` and install the additional third-party dependencies described in the manual. These may be provided in an environment such as MSYS or Cygwin or can be downloaded separately for other environments. You may extract or install the following software into separate folders each and add the `bin` folder to your `PATH` environment variable to make executables and DLLs available. If installers are provided, they might do that already by default.
28 31  
29 32 * [LibJpeg](http://gnuwin32.sourceforge.net/packages/jpeg.htm): This archive provides some needed DLLs needed by LibTiff.
30 33 * [LibTiff](http://gnuwin32.sourceforge.net/packages/tiff.htm): This archive provides some needed binaries and DLLs if you want to use the image comparison tests. It depends on some DLLs from LibJpeg.
... ... @@ -32,76 +35,16 @@ Image comparison tests are disabled by default, but it is possible to run them o
32 35  
33 36 # External Libraries
34 37  
35   -In order to build qpdf, you must have a copy of `zlib` and the `jpeg` library. You can download [prebuilt static external libraries from the qpdf/external-libs github repository](https://github.com/qpdf/external-libs/releases). These include `zlib`, `jpeg`, and `openssl` libraries. There are packages called `external-libs-bin.zip` and `external-libs-src.zip`. If you are building with a recent MSVC or MINGW with MSYS2, you can just extract the `qpdf-external-libs-bin.zip` zip file into the top-level qpdf source tree. Note that you need the 2020-10-24 version (at least) to build qpdf 10.0.2 or greater since this includes openssl. Passing `--enable-external-libs` to `./configure` (which is done automatically if you follow the instructions below) is sufficient to find them.
  38 +In order to build qpdf, you must have a copy of `zlib` and the `jpeg` library. You can download [prebuilt static external libraries from the qpdf/external-libs github repository](https://github.com/qpdf/external-libs/releases). These include `zlib`, `jpeg`, and `openssl` libraries. For MSVC, you must use a non-debugging build configuration. There are files called `external-libs-bin.zip` and `external-libs-src.zip`. If you are building with a recent MSVC or MINGW with MSYS2, you can just extract the `qpdf-external-libs-bin.zip` zip file into the top-level qpdf source tree. The qpdf build detects the presence of the `external-libs` directory automatically. You don't need to set any cmake options.
36 39  
37   -You can also obtain `zlib` and `jpeg` directly on your own and install them. If you are using mingw, you can just set `CPPFLAGS`, `LDFLAGS`, and `LIBS` when you run ./configure so that it can find the header files and libraries. If you are building with MSVC and you want to do this, it probably won't work because `./configure` doesn't know how to interpret `LDFLAGS` and `LIBS` properly for MSVC (though qpdf's own build system does). In this case, you can probably get away with cheating by passing `--enable-external-libs` to `./configure` and then just editing `CPPFLAGS`, `LDFLAGS`, `LIBS` in the generated autoconf.mk file. Note that you should use UNIX-like syntax (`-I`, `-L`, `-l`) even though this is not what cl takes on the command line. qpdf's build rules will fix it.
  40 +You can also obtain `zlib` and `jpeg` directly on your own and install them. Just make sure cmake can find them.
38 41  
39 42 External libraries are built using GitHub Actions from the [qpdf/external-libs](https://github.com/qpdf/external-libs) repository.
40 43  
41   -# Building from version control
42   -
43   -If you check out qpdf from version control, documentation files will not be generated or installed. Documentation files are included in the source distribution, but they can only be built from Linux.
44   -
45   -# Building with MinGW
46   -
47   -QPDF is known to build and pass its test suite with MSYS2 using the 32-bit and 64-bit compilers. MSYS2 is required to build as well in order to get make and other related tools. See common setup at the top of this file for installation and configuration of MSYS2. Then, from the suitable 32-bit or 64-bit environment, run
48   -
49   -```
50   -./config-mingw
51   -make
52   -```
53   -
54   -Note that `./config-mingw` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
55   -
56   -Add the absolute path to the `libqpdf/build` directory to your `PATH`. Make sure you can run the qpdf command by typing qpdf/build/qpdf and making sure you get a help message rather than an error loading the DLL or no output at all. Run the test suite by typing
57   -
58   -```
59   -make check
60   -```
61   -
62   -If all goes well, you should get a passing test suite.
63   -
64   -To create an installation directory, run `make install`. This will create `install-mingw/qpdf-VERSION` and populate it. The binary download of qpdf for Windows with mingw is created from this directory.
65   -
66   -You can also take a look at `make_windows_releases` for reference. This is how the distributed Windows executables are created.
67   -
68   -# Building with MSVC 2015
69   -
70   -These instructions would likely work with newer versions of MSVC and are known to have worked with versions as old as 2008 Express.
  44 +# Running tools from the build area
71 45  
72   -You should first set up your environment to be able to run MSVC from the command line. There is usually a batch file included with MSVC that does this. Make sure that you start a command line environment configured for whichever of 32-bit or 64-bit output that you intend to build for.
73   -
74   -From that cmd prompt, you can start your MSYS2 shell with path inheritance as described above.
75   -
76   -Configure and build as follows:
77   -
78   -```
79   -./config-msvc
80   -make
81   -```
82   -
83   -Note that `./config-msvc` just runs `./configure` with specific arguments, so you can look at it, make adjustments, and manually run configure instead.
84   -
85   -NOTE: automated dependencies are not generated with the msvc build. If you're planning on making modifications, you should probably work with mingw. If there is a need, I can add dependency information to the msvc build, but since I only use it for generating release versions, I haven't bothered.
86   -
87   -Once built, add the full path to the `libqpdf/build` directory to your path and run
88   -
89   -```
90   -make check
91   -```
92   -
93   -to run the test suite.
94   -
95   -If you are building with MSVC and want to debug a crash in MSVC's debugger, first start an instance of Visual C++. Then run qpdf. When the abort/retry/ignore dialog pops up, first attach the process from within visual C++, and then click Retry in qpdf.
96   -
97   -A release version of qpdf is built by default. If you want to link against debugging libraries, you will have to change `/MD` to `/MDd` in `make/msvc.mk`. Note that you must redistribute the Microsoft runtime DLLs. Linking with static runtime (`/MT`) won't work; see "Static Runtime" below for details.
  46 +You can run qpdf's tests without modifying your PATH. If you want to manually run executables from the build tree on Windows, you need to add the `libqpdf` build directory to your path so it can find the qpdf DLL. This would typically be either `libqpdf` or `libqpdf/<CONFIG>` (e.g., `libqpdf/Release`) depending on which cmake generator you are using. Alternatively, you can disable `BUILD_SHARED_LIBS` for your Windows build. This will cause the executables to use the static qpdf library and not build a qpdf DLL at all.
98 47  
99 48 # Runtime DLLs
100 49  
101   -Both build methods create executables and DLLs that are dependent on the compiler's runtime DLLs. When you run make install, the installation process will automatically detect the DLLs and copy them into the installation bin directory. Look at the `copy_dlls` script for details on how this is accomplished.
102   -
103   -Redistribution of the runtime DLL is unavoidable as of this writing; see "Static Runtime" below for details.
104   -
105   -# Static Runtime
106   -
107   -Building the DLL and executables with static runtime does not work with either Visual C++ .NET 2008 (a.k.a. vc9) using `/MT` or with mingw (at least as of 4.4.0) using `-static-libgcc`. The reason is that, in both cases, there is static data involved with exception handling, and when the runtime is linked in statically, exceptions cannot be thrown across the DLL to EXE boundary. Since qpdf uses exception handling extensively for error handling, we have no choice but to redistribute the C++ runtime DLLs. Maybe this will be addressed in a future version of the compilers. This has not been retested with the toolchain versions used to create qpdf >= 3.0 distributions. This has not been revisited since MSVC 2008, but redistributing runtime DLLs is extremely common and should not be a problem.
  50 +Both build methods create executables and DLLs that are dependent on the compiler's runtime DLLs. When you run `cmake --install` or `cpack`, the installation process will automatically detect the DLLs and copy them into the installation bin directory. For mingw, a perl script is used. For MSVC, `cmake`'s `InstallRequiredSystemLibraries` module is adequate.
... ...
README.md
... ... @@ -9,9 +9,7 @@
9 9  
10 10 QPDF is a command-line tool and C++ library that performs content-preserving transformations on PDF files. It supports linearization, encryption, and numerous other features. It can also be used for splitting and merging files, creating PDF files (but you have to supply all the content yourself), and inspecting files for study or analysis. QPDF does not render PDFs or perform text extraction, and it does not contain higher-level interfaces for working with page contents. It is a low-level tool for working with the structure of PDF files and can be a valuable tool for anyone who wants to do programmatic or command-line-based manipulation of PDF files.
11 11  
12   -The [QPDF Manual](https://qpdf.readthedocs.io) is hosted online at https://qpdf.readthedocs.io.
13   -
14   -Additional information about it can be found at https://qpdf.sourceforge.io. The source code repository is hosted at GitHub: https://github.com/qpdf/qpdf.
  12 +The [QPDF Manual](https://qpdf.readthedocs.io) is hosted online at https://qpdf.readthedocs.io. The project website is https://qpdf.sourceforge.io. The source code repository is hosted at GitHub: https://github.com/qpdf/qpdf.
15 13  
16 14 # Verifying Distributions
17 15  
... ... @@ -35,10 +33,22 @@ Versions of qpdf prior to version 7 were released under the terms of version 2.0
35 33  
36 34 QPDF requires a C++ compiler that supports C++-14.
37 35  
  36 +To compile and link something with qpdf, you can use `pkg-config` with package name `libqpdf` or `cmake` with package name `qpdf`. Here's an example of a `CMakeLists.txt` file that builds a program with the qpdf library:
  37 +
  38 +```
  39 +cmake_minimum_required(VERSION 3.16)
  40 +project(some-application LANGUAGES CXX)
  41 +find_package(qpdf)
  42 +add_executable(some-application some-application.cc)
  43 +target_link_libraries(some-application qpdf::libqpdf)
  44 +```
  45 +
38 46 QPDF depends on the external libraries [zlib](https://www.zlib.net/) and [jpeg](https://www.ijg.org/files/). The [libjpeg-turbo](https://libjpeg-turbo.org/) library is also known to work since it is compatible with the regular jpeg library, and QPDF doesn't use any interfaces that aren't present in the straight jpeg8 API. These are part of every Linux distribution and are readily available. Download information appears in the documentation. For Windows, you can download pre-built binary versions of these libraries for some compilers; see [README-windows.md](README-windows.md) for additional details.
39 47  
40 48 Depending on which crypto providers are enabled, then [GnuTLS](https://www.gnutls.org/) and [OpenSSL](https://openssl.org) may also be required. This is discussed more in `Crypto providers` below.
41 49  
  50 +Detailed information appears in the [manual](https://qpdf.readthedocs.io/en/latest/installation.html).
  51 +
42 52 # Licensing terms of embedded software
43 53  
44 54 QPDF makes use of zlib and jpeg libraries for its functionality. These packages can be downloaded separately from their own download locations. If the optional GnuTLS or OpenSSL crypto providers are enabled, then GnuTLS and/or OpenSSL are also required.
... ... @@ -47,23 +57,14 @@ Please see the [NOTICE](NOTICE.md) file for information on licenses of embedded
47 57  
48 58 # Crypto providers
49 59  
50   -As of version 9.1.0, qpdf can use different crypto implementations. These can be selected at compile time or at runtime. The native crypto implementations that were used in all versions prior to 9.1.0 are still present and enabled by default.
  60 +qpdf can use different crypto implementations. These can be selected at compile time or at runtime. The native crypto implementations that were used in all versions prior to 9.1.0 are still present, but they are not built into qpdf by default if any external providers are available at build time.
51 61  
52   -Initially, the following providers are available:
53   -* `native`: a native implementation where all the source is embedded in qpdf and no external dependencies are required
54   -* `openssl`: an implementation that can use the OpenSSL (or BoringSSL) libraries to provide crypto; causes libqpdf to link with the OpenSSL library
  62 +The following providers are available:
55 63 * `gnutls`: an implementation that uses the GnuTLS library to provide crypto; causes libqpdf to link with the GnuTLS library
  64 +* `openssl`: an implementation that can use the OpenSSL (or BoringSSL) libraries to provide crypto; causes libqpdf to link with the OpenSSL library
  65 +* `native`: a native implementation where all the source is embedded in qpdf and no external dependencies are required
56 66  
57   -The default behavior is for ./configure to discover which other crypto providers can be supported based on available external libraries, to build all available crypto providers, and to use an external provider as the default over the native one. This behavior can be changed with the following flags to ./configure:
58   -
59   -* `--enable-crypto-x` -- (where `x` is a supported crypto provider): enable the `x` crypto provider, requiring any external dependencies it needs
60   -* `--disable-crypto-x` -- disable the `x` provider, and do not link against its dependencies even if they are available
61   -* `--with-default-crypto=x` -- make `x` the default provider even if a higher priority one is available
62   -* `--disable-implicit-crypto` -- only build crypto providers that are explicitly requested with an `--enable-crypto-x` option
63   -
64   -For example, if you want to guarantee that the GnuTLS crypto provider is used, you could run ./configure with `--enable-crypto-gnutls --disable-implicit-crypto`.
65   -
66   -Please see the section on crypto providers in the manual for more details.
  67 +The default behavior is for cmake to discover which other crypto providers can be supported based on available external libraries, to build all available external crypto providers, and to use an external provider as the default over the native one. By default, the native crypto provider will be used only if no external providers are available. This behavior can be changed with various cmake options as [described in the manual](https://qpdf.readthedocs.io/en/latest/installation.html#build-time-crypto-selection).
67 68  
68 69 ## Note about weak cryptographic algorithms
69 70  
... ... @@ -71,62 +72,58 @@ The PDF file format used to rely on RC4 for encryption. Using 256-bit keys alway
71 72  
72 73 # Building from source distribution on UNIX/Linux
73 74  
74   -For UNIX and UNIX-like systems, you can usually get by with just
  75 +Starting with version 11, qpdf builds with cmake. The default configuration with cmake works on most systems. On Windows, you can build qpdf with Visual Studio using cmake without having any additional tools installed. However, to run the test suite, you need MSYS2, and you also need MSYS2 to build with mingw.
75 76  
  77 +Example UNIX/Linux build:
76 78 ```
77   -./configure
78   -make
79   -make install
  79 +cmake -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
  80 +cmake --build build
80 81 ```
81 82  
82   -Packagers may set DESTDIR, in which case make install will install inside of DESTDIR, as is customary with many packages. Please also see the "Notes for Packagers" section of the manual.
83   -
84   -For more detailed general information, see the "INSTALL" file in this directory. If you are already accustomed to building and installing software that uses autoconf, there's nothing new for you in the INSTALL file. Note that qpdf uses `autoconf` but not `automake`. We have our own system of Makefiles that allows cross-directory dependencies, doesn't use recursive make, and works better on non-UNIX platforms.
85   -
86   -# Building without wchar_t
87   -
88   -Executive summary: manually define -DQPDF_NO_WCHAR_T in your build if you are building on a system without wchar_t. For details, read the rest of this section.
89   -
90   -While wchar_t is part of the C++ standard library and should be present on virtually every system, there are some stripped down systems, such as those targeting certain embedded environments, that lack wchar_t. Internally, qpdf uses UTF-8 encoding for everything, so there is nothing important in qpdf's API that uses wchar_t. However, there is a helper method for converting between wchar_t* and char* that uses wchar_t.
  83 +Example mingw build from an MSYS2 mingw shell:
  84 +```
  85 +cmake -S . -B build -G 'MSYS Makefiles' -DCMAKE_BUILD_TYPE=RelWithDebInfo
  86 +cmake --build build
  87 +```
91 88  
92   -If you are building in an environment that does not support wchar_t, you can define the preprocessor symbol QPDF_NO_WCHAR_T in your build. This will work whether you are building qpdf and need to avoid compiling the code that uses wchar_t or whether you are building client code that uses qpdf.
  89 +Example MSVC build from an MSYS shell or from a Windows command shell with Visual Studio command-line tools in the path:
  90 +```
  91 +cmake -S . -B build
  92 +cmake --build build --config Release
  93 +```
93 94  
94   -For example, to build qpdf on a system without wchar_t, be sure that -DQPDF_NO_WCHAR_T is part of your CXXFLAGS. Similar techniques will work in other places.
  95 +Installation can be done with `cmake --install`. Packages can be made with `cpack`.
95 96  
96   -Note that, when you build code with libqpdf, it is *not necessary* to have the definition of QPDF_NO_WCHAR_T in your build match what was defined when the library was built as long as you are not calling QUtil::call_main_from_wmain in your code. In other words, if your qpdf library was built on a system without wchar_t and you are using that system to build at some later time after wchar_t was available, as long as you don't call the function that uses it, you can just build normally.
  97 +The tests use `qtest`, and the test driver is invoked by `ctest`. To see the real underlying tests, run `ctest --verbose` so that you can see `qtest`'s output. If you need to turn off qtest's color output, pass `-DQTEST_COLOR=0` to cmake.
97 98  
98   -Note qpdf will never define QPDF_NO_WCHAR_T using autoconf or any other automated method in spite of the fact that it would be easy to do so. That is because there is a hard rule in qpdf that values determined by autoconf are not available in the public API. This is because there is never a guarantee or even expectation that those values will match between the system on which qpdf was build and the system on which a user is building code with libqpdf, and qpdf's include directory should look the same across all systems.
  99 +For additional information, please refer to the [manual](https://qpdf.readthedocs.io/en/latest/installation.html).
99 100  
100 101 # Building on Windows
101 102  
102   -QPDF is known to build and pass its test suite with mingw (latest version tested: gcc 7.2.0), mingw64 (latest version tested: 7.2.0) and Microsoft Visual C++ 2015, both 32-bit and 64-bit versions. MSYS2 is required to build as well in order to get make and other related tools. See [README-windows.md](README-windows.md) for details on how to build under Windows.
  103 +qpdf is known to build and pass its test suite with mingw and Microsoft Visual C++. Both 32-bit and 64-bit versions work. In addition to the manual, see [README-windows.md](README-windows.md) for more details on how to build under Windows.
103 104  
104 105 # Building Documentation
105 106  
106   -The QPDF manual is written in reStructured Text format and is build with [sphinx](https://www.sphinx-doc.org). The sources to the user manual can be found in the `manual` directory. For more detailed information, consult the [Building and Installing QPDF section of the manual](manual/installation.rst) or consult the [build-doc script](build-scripts/build-doc).
  107 +The QPDF manual is written in reStructured Text format and is build with [sphinx](https://www.sphinx-doc.org). The sources to the user manual can be found in the `manual` directory. For more detailed information, consult the [Building and Installing QPDF section of the manual](https://qpdf.readthedocs.io/en/latest/installation.html) or consult the [build-doc script](build-scripts/build-doc).
107 108  
108 109 # Additional Notes on Build
109 110  
110   -QPDF's build system can optionally use its own built-in rules rather than using libtool and obeying the compiler specified with configure. This can be enabled by passing `--with-buildrules=buildrules` where buildrules corresponds to one of the `.mk` files (other than `rules.mk`) in the make directory. This should never be necessary on a UNIX system, but may be necessary on a Windows system. See [README-windows.md](README-windows.md) for details.
  111 +qpdf provides cmake configuration files and pkg-config files. They support static and dynamic linking. In general, you do not need the header files from qpdf's dependencies to be available to builds that _use_ qpdf. The only exception to this is that, if you include `Pl_DCT.hh`, you need header files from `libjpeg`. Since this is a rare case, qpdf's cmake and pkg-config files do not automatically add a JPEG include path to the build. If you are using `Pl_DCT` explicitly, you probably already have that configured in your build.
111 112  
112   -The software library is just `libqpdf`, and all the header files are in the `qpdf` subdirectories of `include` and `libqpdf`. If you link statically with `-lqpdf`, then you will also need to link with `-lz` and `-ljpeg`. The shared qpdf library is linked with `-lz` and `-ljpeg`, none of qpdf's public header files directly include files from `libz`, and only `Pl_DCT.hh` includes files from `libjpeg`, so for most cases, qpdf's development files are self contained. If you need to use `Pl_DCT` in your application code, you will need to have the header files for some libjpeg distribution in your include path.
113   -
114   -To learn about using the library, please read comments in the header files in `include/qpdf`, especially `QPDF.hh`, `QPDFObjectHandle.hh`, and
115   -`QPDFWriter.hh`. These are the best sources of documentation on the API. You can also study the code of `qpdf/qpdf.cc`, which exercises most of the public interface. There are additional example programs in the examples directory. Reading all the source files in the `qpdf` directory (including the qpdf command-line tool and some test drivers) along with the code in the examples directory will give you a complete picture of every aspect of the public interface.
  113 +To learn about using the library, please read comments in the header files in [include/qpdf](include/qpdf/), especially [QPDF.hh](include/qpdf/QPDF.hh), [QPDFObjectHandle.hh](include/qpdf/QPDFObjectHandle.hh), and
  114 +[QPDFWriter.hh](include/qpdf/QPDFWriter.hh). These are the best sources of documentation on the API. You can also study the code of [QPDFJob.cc](libqpdf/QPDFJob.cc), which exercises most of the public interface. There are additional example programs in the [examples](examples/) directory.
116 115  
117 116 # Additional Notes on Test Suite
118 117  
119   -By default, slow tests and tests that require dependencies beyond those needed to build qpdf are disabled. Slow tests include image comparison tests and large file tests. Image comparison tests can be enabled by passing `--enable-test-compare-images` to ./configure. This was on by default in qpdf versions prior to 3.0, but is now off by default. Large file tests can be enabled by passing `--with-large-file-test-path=path` to `./configure` or by setting the `QPDF_LARGE_FILE_TEST_PATH` environment variable. On Windows, this should be a Windows path. Run `./configure --help` for additional options. The test suite provides nearly full coverage even without these tests. Unless you are making deep changes to the library that would impact the contents of the generated PDF files or testing this on a new platform for the first time, there is no real reason to run these tests. If you're just running the test suite to make sure that qpdf works for your build, the default tests are adequate. The configure rules for these tests do nothing other than setting variables in `autoconf.mk`, so you can feel free to turn these on and off directly in `autoconf.mk` rather than rerunning configure.
  118 +By default, slow tests and tests that require dependencies beyond those needed to build qpdf are disabled. Slow tests include image comparison tests and large file tests. Image comparison tests can be enabled by setting the `QPDF_TEST_COMPARE_IMAGES` environment variable to `1`. Large file tests can be enabled setting the `QPDF_LARGE_FILE_TEST_PATH` environment variable to the absolute path of a directory with at least 11 GB of free space that can handle files over 4 GB in size. On Windows, this should be a Windows path (e.g. `C:\LargeFileTemp` even if the build is being run from an MSYS2 environment. The test suite provides nearly full coverage even without these tests. Unless you are making deep changes to the library that would impact the contents of the generated PDF files or testing this on a new platform for the first time, there is no real reason to run these tests. If you're just running the test suite to make sure that qpdf works for your build, the default tests are adequate.
120 119  
121   -If you are packaging qpdf for a distribution and preparing a build that is run by an autobuilder, you may want to add the `--enable-show-failed-test-output` to configure options. This way, if the test suite fails, test failure detail will be included in the build output. Otherwise, you will have to have access to the `qtest.log` file from the build to view test failures. The Debian packages for qpdf enable this option.
  120 +If you are packaging qpdf for a distribution and preparing a build that is run by an autobuilder, you may want to pass `-DSHOW_FAILED_TEST_OUTPUT=1` to `cmake` and run `ctest` with the `--verbose` or `--output-on-failure` option. This way, if the test suite fails, test failure detail will be included in the build output. Otherwise, you will have to have access to the `qtest.log` file from the build to view test failures. The Debian packages for qpdf enable this option. More notes for packagers can be found in [the manual](https://qpdf.readthedocs.io/en/latest/packaging.html).
122 121  
123 122 # Random Number Generation
124 123  
125 124 By default, qpdf uses the crypto provider for generating random numbers. The rest of this applies only if you are using the native crypto provider.
126 125  
127 126 If the native crypto provider is in use, then, when `qpdf` detects either the Windows cryptography API or the existence of `/dev/urandom`, `/dev/arandom`, or `/dev/random`, it uses them to generate cryptographically secure random numbers. If none of these conditions are true, the build will fail with an error. This behavior can be modified in several ways:
128   -* If you configure with `--disable-os-secure-random` or define `SKIP_OS_SECURE_RANDOM`, qpdf will not attempt to use Windows cryptography or the random device. You must either supply your own random data provider or allow use of insecure random numbers.
129   -* If you configure qpdf with the `--enable-insecure-random` option or define `USE_INSECURE_RANDOM`, qpdf will try insecure random numbers if OS-provided secure random numbers are disabled. This is not a fallback. In order for insecure random numbers to be used, you must also disable OS secure random numbers since, otherwise, failure to find OS secure random numbers is a compile error. The insecure random number source is stdlib's `random()` or `rand()` calls. These random numbers are not cryptography secure, but the qpdf library is fully functional using them. Using non-secure random numbers means that it's easier in some cases to guess encryption keys. If you're not generating encrypted files, there's no advantage to using secure random numbers.
  127 +* If you use the cmake option `SKIP_OS_SECURE_RANDOM` or define the `SKIP_OS_SECURE_RANDOM` preprocessor symbol, qpdf will not attempt to use Windows cryptography or the random device. You must either supply your own random data provider or allow use of insecure random numbers.
  128 +* If you turn on the cmake option `USE_INSECURE_RANDOM` or define the `USE_INSECURE_RANDOM` preprocessor symbol, qpdf will try insecure random numbers if OS-provided secure random numbers are disabled. This is not a fallback. In order for insecure random numbers to be used, you must also disable OS secure random numbers since, otherwise, failure to find OS secure random numbers is a compile error. The insecure random number source is stdlib's `random()` or `rand()` calls. These random numbers are not cryptography secure, but the qpdf library is fully functional using them. Using non-secure random numbers means that it's easier in some cases to guess encryption keys.
130 129 * In all cases, you may supply your own random data provider. To do this, derive a class from `qpdf/RandomDataProvider` (since version 5.1.0) and call `QUtil::setRandomDataProvider` before you create any `QPDF` objects. If you supply your own random data provider, it will always be used even if support for one of the other random data providers is compiled in. If you wish to avoid any possibility of your build of qpdf from using anything but a user-supplied random data provider, you can define `SKIP_OS_SECURE_RANDOM` and not `USE_INSECURE_RANDOM`. In this case, qpdf will throw a runtime error if any attempt is made to generate random numbers and no random data provider has been supplied.
131   -
132   -If you are building qpdf on a platform that qpdf doesn't know how to generate secure random numbers on, a patch would be welcome.
... ...
... ... @@ -28,6 +28,9 @@ Misc
28 28 --show-encryption could potentially retry with this option if the
29 29 first time doesn't work. Then, with the file open, we can read the
30 30 encryption dictionary normally.
  31 +* Go through README-maintainer "CODING RULES" and update --
  32 + PointerHolder and other changes will make some of the rules
  33 + obsolete.
31 34  
32 35 Soon: Break ground on "Document-level work"
33 36  
... ... @@ -1032,12 +1035,6 @@ Rejected Ideas
1032 1035 * Investigate whether there is a way to automate the memory checker
1033 1036 tests for Windows.
1034 1037  
1035   -* Consider adding "uninstall" target to makefile. It should only
1036   - uninstall what it installed, which means that you must run
1037   - uninstall from the version you ran install with. It would only be
1038   - supported for the toolchains that support the install target
1039   - (libtool).
1040   -
1041 1038 * Provide support in QPDFWriter for writing incremental updates.
1042 1039 Provide support in qpdf for preserving incremental updates. The
1043 1040 goal should be that QDF mode should be fully functional for files
... ...
cSpell.json
... ... @@ -77,6 +77,8 @@
77 77 "cstr",
78 78 "cxxflags",
79 79 "cygwin",
  80 + "dbuild",
  81 + "dcmake",
80 82 "dctdecode",
81 83 "decltype",
82 84 "decrypter",
... ... @@ -88,14 +90,19 @@
88 90 "devel",
89 91 "didier",
90 92 "diffutils",
  93 + "dinstall",
91 94 "directpagerefcount",
92 95 "distclean",
93 96 "ditems",
  97 + "dmaintainer",
94 98 "docbook",
95 99 "docdir",
96 100 "dohnal",
97 101 "dqpdf",
  102 + "dqtest",
98 103 "dquote",
  104 + "drequire",
  105 + "dshow",
99 106 "dtdvalid",
100 107 "ebra",
101 108 "ecks",
... ...
manual/installation.rst
... ... @@ -3,125 +3,440 @@
3 3 Building and Installing QPDF
4 4 ============================
5 5  
6   -This chapter describes how to build and install qpdf. Please see also
7   -the :file:`README.md` and
8   -:file:`INSTALL` files in the source distribution.
  6 +This chapter describes how to build and install qpdf.
9 7  
10 8 .. _prerequisites:
11 9  
12   -System Requirements
13   --------------------
  10 +Dependencies
  11 +------------
  12 +
  13 +qpdf has few external dependencies. This section describes what you
  14 +need to build qpdf in various circumstances.
  15 +
  16 +Basic Dependencies
  17 +~~~~~~~~~~~~~~~~~~
  18 +
  19 +- A C++ compiler that supports C++-14
  20 +
  21 +- `CMake <https://www.cmake.org>`__ version 3.16 or later
  22 +
  23 +- `zlib <https://www.zlib.net/>`__ or a compatible zlib implementation
  24 +
  25 +- A libjpeg-compatible library such as `jpeg <https://ijg.org/>`__ or
  26 + `libjpeg-turbo <https://libjpeg-turbo.org/>`__
  27 +
  28 +- *Recommended but not required:* `gnutls <https://www.gnutls.org/>`__
  29 + to be able to use the gnutls crypto provider and/or `openssl
  30 + <https://openssl.org/>`__ to be able to use the openssl crypto
  31 + provider
  32 +
  33 +The qpdf source tree includes a few automatically generated files. The
  34 +code generator uses Python 3. Automatic code generation is off by
  35 +default. For a discussion, refer to :ref:`build-options`.
  36 +
  37 +Test Dependencies
  38 +~~~~~~~~~~~~~~~~~
  39 +
  40 +qpdf's test suite is run by ``ctest``, which is part of CMake, but
  41 +the tests themselves are implemented using an embedded copy of `qtest
  42 +<https://qtest.sourceforge.io>`__, which is implemented in perl. On
  43 +Windows, MSYS2's perl is known to work.
  44 +
  45 +qtest requires `GNU diffutils
  46 +<http://www.gnu.org/software/diffutils/>`__ or any other diff that
  47 +supports :command:`diff -u`. The default ``diff`` command works on
  48 +GNU/Linux and MacOS.
  49 +
  50 +Part of qpdf's test suite does comparisons of the contents PDF files
  51 +by converting them to images and comparing the images. The image
  52 +comparison tests are disabled by default. Those tests are not required
  53 +for determining correctness of a qpdf build since the test suite also
  54 +contains expected output files that are compared literally. The image
  55 +comparison tests provide an extra check to make sure that any content
  56 +transformations don't break the rendering of pages. Transformations
  57 +that affect the content streams themselves are off by default and are
  58 +only provided to help developers look into the contents of PDF files.
  59 +If you are making deep changes to the library that cause changes in
  60 +the contents of the files that qpdf generates, then you should enable
  61 +the image comparison tests. Enable them by setting the
  62 +``QPDF_TEST_COMPARE_IMAGES`` environment variable to ``1`` before
  63 +running tests. Image comparison tests add these additional
  64 +requirements:
  65 +
  66 +- `libtiff <http://www.simplesystems.org/libtiff/>`__ command-line
  67 + utilities
  68 +
  69 +- `GhostScript <https://www.ghostscript.com/>`__ version 8.60 or newer
  70 +
  71 +Note: prior to qpdf 11, image comparison tests were enabled within
  72 +:file:`qpdf.test`, and you had to *disable* them by setting
  73 +``QPDF_SKIP_TEST_COMPARE_IMAGES`` to ``1``. This was done
  74 +automatically by ``./configure``. Now you have to *enable* image
  75 +comparison tests by setting an environment variable. This change was
  76 +made because developers have to set the environment variable
  77 +themselves now rather than setting it through the build. Either way,
  78 +they are off by default.
  79 +
  80 +Additional Requirements on Windows
  81 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
14 82  
15   -The qpdf package has few external dependencies. In order to build qpdf,
16   -the following packages are required:
  83 +- To build qpdf with Visual Studio, there are no additional
  84 + requirements when the default cmake options are used. You can build
  85 + qpdf from a Visual C++ command-line shell.
17 86  
18   -- A C++ compiler that supports C++-14.
  87 +- To build with mingw, MSYS2 is recommended with the mingw32 and/or
  88 + mingw64 tool chains. You can also build with MSVC from an MSYS2
  89 + environment.
19 90  
20   -- zlib: http://www.zlib.net/
  91 +- qpdf's test suite can run within the MSYS2 environment for both
  92 + mingw and MSVC-based builds.
21 93  
22   -- jpeg: http://www.ijg.org/files/ or https://libjpeg-turbo.org/
  94 +For additional notes, see :file:`README-windows.md` in the source
  95 +distribution.
23 96  
24   -- *Recommended but not required:* gnutls: https://www.gnutls.org/ to be
25   - able to use the gnutls crypto provider, and/or openssl:
26   - https://openssl.org/ to be able to use the openssl crypto provider.
  97 +Requirements for Building Documentation
  98 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
27 99  
28   -- gnu make 3.81 or newer: http://www.gnu.org/software/make
  100 +The qpdf manual is written in reStructured Text and built with `Sphinx
  101 +<https://www.sphinx-doc.org>`__ using the `Read the Docs Sphinx Theme
  102 +<https://sphinx-rtd-theme.readthedocs.io>`__. Versions of sphinx prior
  103 +to version 4.3.2 probably won't work. Sphinx requires Python 3. In
  104 +order to build the HTML documentation from source, you need to install
  105 +sphinx and the theme, which you can typically do with ``pip install
  106 +sphinx sphinx_rtd_theme``. To build the PDF version of the
  107 +documentation, you need ``pdflatex``, ``latexmk``, and a fairly
  108 +complete LaTeX installation. Detailed requirements can be found in the
  109 +Sphinx documentation. To see how the documentation is built for the
  110 +qpdf distribution, refer to the :file:`build-scripts/build-doc` file
  111 +in the qpdf source distribution.
29 112  
30   -- perl version 5.8 or newer: http://www.perl.org/; required for running
31   - the test suite. Starting with qpdf version 9.1.1, perl is no longer
32   - required at runtime.
  113 +.. _building:
33 114  
34   -- GNU diffutils (any version): http://www.gnu.org/software/diffutils/
35   - is required to run the test suite. Note that this is the version of
36   - diff present on virtually all GNU/Linux systems. This is required
37   - because the test suite uses :command:`diff -u`.
  115 +Build Instructions
  116 +------------------
38 117  
39   -Part of qpdf's test suite does comparisons of the contents PDF files by
40   -converting them images and comparing the images. The image comparison
41   -tests are disabled by default. Those tests are not required for
42   -determining correctness of a qpdf build if you have not modified the
43   -code since the test suite also contains expected output files that are
44   -compared literally. The image comparison tests provide an extra check to
45   -make sure that any content transformations don't break the rendering of
46   -pages. Transformations that affect the content streams themselves are
47   -off by default and are only provided to help developers look into the
48   -contents of PDF files. If you are making deep changes to the library
49   -that cause changes in the contents of the files that qpdf generate,
50   -then you should enable the image comparison tests. Enable them by
51   -running :command:`configure` with the
52   -:samp:`--enable-test-compare-images` flag. If you enable
53   -this, the following additional requirements are required by the test
54   -suite. Note that in no case are these items required to use qpdf.
  118 +Starting with qpdf 11, qpdf is built with `CMake
  119 +<https://www.cmake.org>`__.
55 120  
56   -- libtiff: http://www.remotesensing.org/libtiff/
  121 +Basic Build Invocation
  122 +~~~~~~~~~~~~~~~~~~~~~~
57 123  
58   -- GhostScript version 8.60 or newer: http://www.ghostscript.com
  124 +qpdf uses cmake in an ordinary way, so refer to the CMake
  125 +documentation for details about how to run ``cmake``. Here is a
  126 +brief summary.
59 127  
60   -If you do not enable this, then you do not need to have tiff and
61   -ghostscript.
  128 +You can usually just run
62 129  
63   -For information on building the documentation, see :ref:`build-doc`.
  130 +::
64 131  
65   -.. _building:
  132 + cmake -S . -B build
  133 + cmake --build build
66 134  
67   -Build Instructions
68   -------------------
  135 +If you are using a multi-configuration generator such as MSVC, you
  136 +should pass ``--config <Config>`` (where ``<Config>`` is ``Release``,
  137 +``Debug``, ``RelWithDebInfo``, or ``MinSizeRel`` as discussed in the
  138 +CMake documentation) to the *build* command. If you are running a
  139 +single configuration generator such as the default Makefile generators
  140 +in Linux or MSYS, you may want to pass ``-DCMAKE_BUILD_TYPE=<Config>``
  141 +to the original ``cmake`` command.
69 142  
70   -Building qpdf on UNIX is generally just a matter of running
  143 +Run ``ctest`` to run the test suite. Since the real tests are
  144 +implemented with `qtest <https://qtest.sourceforge.io/>`__, you will
  145 +want to pass ``--verbose`` to ``cmake`` so you can see the individual
  146 +test outputs. Otherwise, you will see a small number of ``ctest``
  147 +commands that take a very long to run.
71 148  
72   -::
  149 +.. _installation:
73 150  
74   - ./configure
75   - make
76   -
77   -You can also run :command:`make check` to run the test
78   -suite and :command:`make install` to install. Please run
79   -:command:`./configure --help` for options on what can be
80   -configured. You can also set the value of ``DESTDIR`` during
81   -installation to install to a temporary location, as is common with many
82   -open source packages. Please see also the
83   -:file:`README.md` and
84   -:file:`INSTALL` files in the source distribution.
85   -
86   -Building on Windows is a little bit more complicated. For details,
87   -please see :file:`README-windows.md` in the source
88   -distribution. You can also download a binary distribution for Windows.
89   -There is a port of qpdf to Visual C++ version 6 in the
90   -:file:`contrib` area generously contributed by Jian
91   -Ma. This is also discussed in more detail in
92   -:file:`README-windows.md`.
93   -
94   -While ``wchar_t`` is part of the C++ standard, qpdf uses it in only one
95   -place in the public API, and it's just in a helper function. It is
96   -possible to build qpdf on a system that doesn't have ``wchar_t``, and
97   -it's also possible to compile a program that uses qpdf on a system
98   -without ``wchar_t`` as long as you don't call that one method. This is a
99   -very unusual situation. For a detailed discussion, please see the
100   -top-level README.md file in qpdf's source distribution.
101   -
102   -There are some other things you can do with the build. Although qpdf
103   -uses :command:`autoconf`, it does not use
104   -:command:`automake` but instead uses a
105   -hand-crafted non-recursive Makefile that requires gnu make. If you're
106   -really interested, please read the comments in the top-level
107   -:file:`Makefile`.
108   -
109   -.. _build-doc:
110   -
111   -Building Documentation
112   -----------------------
  151 +Installation and Packaging
  152 +~~~~~~~~~~~~~~~~~~~~~~~~~~
113 153  
114   -The qpdf manual is written in reStructured Text and built with `Sphinx
115   -<https://www.sphinx-doc.org>`__ using the `Read the Docs Sphinx Theme
116   -<https://sphinx-rtd-theme.readthedocs.io>`__. In order to build the
117   -HTML documentation from source, you need to install sphinx and the
118   -theme, which you can typically do with ``pip install sphinx
119   -sphinx_rtd_theme``. To build the PDF version of the documentation, you
120   -need ``pdflatex``, ``latexmk``, and a fairly complete LaTeX
121   -installation. Detailed requirements can be found in the Sphinx
122   -documentation. To see how the documentation is built for the qpdf
123   -distribution, refer to the :file:`build-scripts/build-doc` file in the
124   -qpdf source distribution.
  154 +Installation can be performed using ``cmake --install`` or ``cpack``.
  155 +For most normal use cases, ``cmake --install`` or ``cpack`` can be run
  156 +in the normal way as described in CMake documentation. qpdf follows
  157 +all normal installation conventions and uses CMake-defined variables
  158 +for standard behavior.
  159 +
  160 +There are several components that can be installed separately:
  161 +
  162 +.. list-table:: Installation Components
  163 + :widths: 5 80
  164 + :header-rows: 0
  165 +
  166 + - - cli
  167 + - Command-line tools
  168 +
  169 + - - lib
  170 + - The runtime libraries; required if you built with shared
  171 + libraries
  172 +
  173 + - - dev
  174 + - Static libraries, header files, and other files needed by
  175 + developers
  176 +
  177 + - - doc
  178 + - Documentation and, if selected for installation, the manual
  179 +
  180 + - - examples
  181 + - Example source files
  182 +
  183 +There are also separate options, discussed in :ref:`build-options`,
  184 +that control how certain specific parts of the software are installed.
  185 +
  186 +.. _build-options:
  187 +
  188 +Build Options
  189 +-------------
  190 +
  191 +.. last verified consistent with build: 2022-03-13. The top-level
  192 + CMakeLists.txt contains a comment that references this section.
  193 +
  194 +.. cSpell:ignore ccmake
  195 +
  196 +All available build options are defined in the the top-level
  197 +:file:`CMakeLists.txt` file and have help text. You can see them using
  198 +any standard cmake front-end (like ``cmake-gui`` or ``ccmake``). This
  199 +section describes options that apply to most users. If you are trying
  200 +to map autoconf options (from prior to qpdf 11) to cmake options,
  201 +please see :ref:`autoconf-to-cmake`.
  202 +
  203 +If you are packaging qpdf for a distribution, you should also read
  204 +:ref:`packaging`.
  205 +
  206 +Basic Build Options
  207 +~~~~~~~~~~~~~~~~~~~
  208 +
  209 +BUILD_DOC
  210 + Whether to build documentation with sphinx. You must have the
  211 + required tools installed.
  212 +
  213 +BUILD_DOC_HTML
  214 + Visible when BUILD_DOC is selected. This option controls building
  215 + HTML documentation separately from PDF documentation since
  216 + the sphinx theme only needed for the HTML documentation.
  217 +
  218 +BUILD_DOC_PDF
  219 + Visible when BUILD_DOC is selected. This option controls building
  220 + PDF documentation separately from HTML documentation since
  221 + additional tools are required to build the PDF documentation.
  222 +
  223 +BUILD_SHARED_LIBS, BUILD_STATIC_LIBS
  224 + You can configure whether to build shared libraries, static
  225 + libraries, or both. You must select at least one of these options.
  226 + For rapid iteration, select only one as this cuts the build time in
  227 + half.
  228 +
  229 + On Windows, if you build with shared libraries, you must have the
  230 + output directory for libqpdf (e.g. :file:`libqpdf/Release` or
  231 + :file:`libqpdf` within the build directory) in your path so that the
  232 + compiled executables can find the DLL. Updating your path is not
  233 + necessary if you build with static libraries only.
  234 +
  235 +QTEST_COLOR
  236 + Turn this on or off to control whether qtest uses color in its
  237 + output.
  238 +
  239 +Options for Working on qpdf
  240 +~~~~~~~~~~~~~~~~~~~~~~~~~~~
  241 +
  242 +GENERATE_AUTO_JOB
  243 + Some qpdf source files are automatically generated from
  244 + :file:`job.yml` and the CLI documentation. If you are adding new
  245 + command-line arguments to the qpdf CLI or updating
  246 + :file:`manual/cli.rst` in the qpdf sources, you should turn this on.
  247 + This option requires Python 3.
  248 +
  249 +WERROR
  250 + Make any compiler warnings into errors. We want qpdf to compile free
  251 + of warnings whenever possible, but there's always a chance that a
  252 + compiler upgrade or tool change may cause warnings to appear that
  253 + weren't there before. If you are testing qpdf with a new compiler,
  254 + you should turn this on.
  255 +
  256 +Environment-Specific Options
  257 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  258 +
  259 +SHOW_FAILED_TEST_OUTPUT
  260 + Ordinarily, qtest (which drives qpdf's test suite) writes detailed
  261 + information about its output to the file ``qtest.log`` in the build
  262 + output directory. If you are running a build in a continuous
  263 + integration or automated environment where you can't get to those
  264 + files, you should enable this option and also run ``ctest
  265 + --verbose`` or ``ctest --output-on-failure``. This will cause
  266 + detailed test failure output to be written into the build log.
  267 +
  268 +CI_MODE
  269 + Turning this on sets options used in qpdf's continuous integration
  270 + environment to ensure we catch as many problems as possible.
  271 + Specifically, this option enables ``SHOW_FAILED_TEST_OUTPUT`` and
  272 + ``WERROR`` and forces the native crypto provider to be built.
  273 +
  274 +MAINTAINER_MODE
  275 + Turning this option on sets options that should be on if you are
  276 + maintaining qpdf. In turns on the following:
  277 +
  278 + - ``BUILD_DOC``
  279 +
  280 + - ``GENERATE_AUTO_JOB``
  281 +
  282 + - ``WERROR``
  283 +
  284 + - ``REQUIRE_NATIVE_CRYPTO``
  285 +
  286 + It is possible to turn ``BUILD_DOC`` off in maintainer mode so that
  287 + the extra requirements for building documentation don't have to be
  288 + available.
  289 +
  290 +.. _crypto.build:
  291 +
  292 +Build-time Crypto Selection
  293 +~~~~~~~~~~~~~~~~~~~~~~~~~~~
  294 +
  295 +Since version 9.1.0, qpdf can use external crypto providers in
  296 +addition to its native provider. For a general discussion, see
  297 +:ref:`crypto`. This section discusses how to configure which crypto
  298 +providers are compiled into qpdf.
  299 +
  300 +In nearly all cases, external crypto providers should be preferred
  301 +over the native one. However, if you are not concerned about working
  302 +with encrypted files and want to reduce the number of dependencies,
  303 +the native crypto provider is fully supported.
  304 +
  305 +By default, qpdf's build enables every external crypto providers whose
  306 +dependencies are available and only enables the native crypto provider
  307 +if no external providers are available. You can change this behavior
  308 +with the options described here.
  309 +
  310 +USE_IMPLICIT_CRYPTO
  311 + This is on by default. If turned off, only explicitly selected
  312 + crypto providers will be built. You must use at least one of the
  313 + ``REQUIRE`` options below.
  314 +
  315 +ALLOW_CRYPTO_NATIVE
  316 + This option is only available when USE_IMPLICIT_CRYPTO is selected,
  317 + in which case it is on by default. Turning it off prevents qpdf from
  318 + falling back to the native crypto provider when no external provider
  319 + is available.
  320 +
  321 +REQUIRE_CRYPTO_NATIVE
  322 + Build the native crypto provider even if other options are
  323 + available.
  324 +
  325 +REQUIRE_CRYPTO_GNUTLS
  326 + Require the gnutls crypto provider. Turning this on makes in an
  327 + error if the gnutls library is not available.
  328 +
  329 +REQUIRE_CRYPTO_OPENSSL
  330 + Require the openssl crypto provider. Turning this on makes in an
  331 + error if the openssl library is not available.
  332 +
  333 +DEFAULT_CRYPTO
  334 + Explicitly select which crypto provider is used by default. See
  335 + :ref:`crypto.runtime` for information about run-time selection of
  336 + the crypto provider. If not specified, qpdf will pick gnutls if
  337 + available, otherwise openssl if available, and finally native as a
  338 + last priority.
  339 +
  340 +Example: if you wanted to build with only the gnutls crypto provider,
  341 +you should run cmake with ``-DUSE_IMPLICIT_CRYPTO=0
  342 +-DREQUIRE_CRYPTO_GNUTLS=1``.
  343 +
  344 +Advanced Build Options
  345 +~~~~~~~~~~~~~~~~~~~~~~
  346 +
  347 +These options are used only for special purposes and are not relevant
  348 +to most users.
  349 +
  350 +AVOID_WINDOWS_HANDLE
  351 + Disable use of the ``HANDLE`` type in Windows. This can be useful if
  352 + you are building for certain embedded Windows environments. Some
  353 + functionality won't work, but you can still process PDF files from
  354 + memory in this configuration.
  355 +
  356 +BUILD_DOC_DIST, INSTALL_MANUAL
  357 + By default, installing qpdf does not include a pre-built copy of the
  358 + manual. Instead, it installs a README file that tells people where
  359 + to find the manual online. If you want to install the manual, you
  360 + must enable the ``INSTALL_MANUAL`` option, and you must have a
  361 + ``doc-dist`` directory in the manual directory of the build. The
  362 + ``doc-dist`` directory is created if ``BUILD_DOC_DIST`` is selected
  363 + and ``BUILD_DOC_PDF`` and ``BUILD_DOC_HTML`` are both on.
  364 +
  365 + The ``BUILD_DOC_DIST`` and ``INSTALL_MANUAL`` options are separate
  366 + and independent because of the additional tools required to build
  367 + documentation. In particular, for qpdf's official release
  368 + preparation, a ``doc-dist`` directory is built in Linux and then
  369 + extracted into the Windows builds so that it can be included in the
  370 + Windows installers. This prevents us from having to build the
  371 + documentation in a Windows environment. For additional discussion,
  372 + see :ref:`doc-packaging-rationale`.
  373 +
  374 +INSTALL_CMAKE_PACKAGE
  375 + Controls whether or not to install qpdf's cmake configuration file
  376 + (on by default).
  377 +
  378 +INSTALL_EXAMPLES
  379 + Controls whether or not to install qpdf's example source files with
  380 + documentation (on by default).
  381 +
  382 +INSTALL_PKGCONFIG
  383 + Controls whether or not to install qpdf's pkg-config configuration
  384 + file (on by default).
  385 +
  386 +OSS_FUZZ
  387 + Turning this option on changes the build of the fuzzers in a manner
  388 + specifically required by Google's oss-fuzz project. There is no
  389 + reason to turn this on for any other reason. It is enabled by the
  390 + build script that builds qpdf from that context.
  391 +
  392 +SKIP_OS_SECURE_RANDOM, USE_INSECURE_RANDOM
  393 + The native crypto implementation uses the operating systems's secure
  394 + random number source when available. It is not used when an external
  395 + crypto provider is in use. If you are building in a very specialized
  396 + environment where you are not using an external crypto provider but
  397 + can't use the OS-provided secure random number generator, you can
  398 + turn both of these options on. This will cause qpdf to fall back to
  399 + an insecure random number generator, which may generate guessable
  400 + random numbers. The resulting qpdf is still secure, but encrypted
  401 + files may be more subject to brute force attacks. Unless you know
  402 + you need these options for a specialized purpose, you don't need
  403 + them. These options were added to qpdf in response to a special
  404 + request from a user who needed to run a specialized PDF-related task
  405 + in an embedded environment that didn't have a secure random number
  406 + source.
  407 +
  408 +Building without wchar_t
  409 +~~~~~~~~~~~~~~~~~~~~~~~~
  410 +
  411 +It is possible to build qpdf on a system that doesn't have
  412 +``wchar_t``. The resulting build of qpdf is not API-compatible with a
  413 +regular qpdf build, so this option cannot be selected from cmake. This
  414 +option was added to qpdf to support installation on a very stripped
  415 +down embedded environment that included only a partial implementation
  416 +of the standard C++ library.
  417 +
  418 +You can disable use of ``wchar_t`` in qpdf's code by defining the
  419 +``QPDF_NO_WCHAR_T`` preprocessor symbol in your build (e.g. by
  420 +including ``-DQPDF_NO_WCHAR_T`` in ``CFLAGS`` and ``CXXFLAGS``).
  421 +
  422 +While ``wchar_t`` is part of the C++ standard library and should be
  423 +present on virtually every system, there are some stripped down
  424 +systems, such as those targeting certain embedded environments, that
  425 +lack ``wchar_t``. Internally, qpdf uses UTF-8 encoding for everything,
  426 +so there is nothing important in qpdf's API that uses ``wchar_t``.
  427 +However, there are some helper methods for converting between
  428 +``wchar_t*`` and ``char*``.
  429 +
  430 +If you are building in an environment that does not support
  431 +``wchar_t``, you can define the preprocessor symbol
  432 +``QPDF_NO_WCHAR_T`` in your build. This will work whether you are
  433 +building qpdf and need to avoid compiling the code that uses wchar_t
  434 +or whether you are building client code that uses qpdf.
  435 +
  436 +Note that, when you build code with libqpdf, it is *not necessary* to
  437 +have the definition of ``QPDF_NO_WCHAR_T`` in your build match what
  438 +was defined when the library was built as long as you are not calling
  439 +any of the methods that use ``wchar_t``.
125 440  
126 441 .. _crypto:
127 442  
... ... @@ -134,93 +449,36 @@ to as &quot;crypto providers.&quot; At the time of writing, a crypto
134 449 implementation must provide MD5 and SHA2 (256, 384, and 512-bit) hashes
135 450 and RC4 and AES256 with and without CBC encryption. In the future, if
136 451 digital signature is added to qpdf, there may be additional requirements
137   -beyond this.
  452 +beyond this. Some of these are weak cryptographic algorithms. For a
  453 +discussion of why they're needed, see :ref:`weak-crypto`.
138 454  
139   -Starting with qpdf version 9.1.0, the available implementations are
140   -``native`` and ``gnutls``. In qpdf 10.0.0, ``openssl`` was added.
141   -Additional implementations may be added if needed. It is also possible
142   -for a developer to provide their own implementation without modifying
143   -the qpdf library.
  455 +The available crypto provider implementations are ``gnutls``,
  456 +``openssl``, and ``native``. OpenSSL support was added in qpdf 10.0.0
  457 +with support for OpenSSL added in 10.4.0. GnuTLS support was
  458 +introduced in qpdf 9.1.0. Additional implementations can be added as
  459 +needed. It is also possible for a developer to provide their own
  460 +implementation without modifying the qpdf library.
144 461  
145   -.. _crypto.build:
146   -
147   -Build Support For Crypto Providers
148   -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
149   -
150   -When building with qpdf's build system, crypto providers can be enabled
151   -at build time using various :command:`./configure`
152   -options. The default behavior is for
153   -:command:`./configure` to discover which crypto providers
154   -can be supported based on available external libraries, to build all
155   -available crypto providers, and to use an external provider as the
156   -default over the native one. This behavior can be changed with the
157   -following flags to :command:`./configure`:
158   -
159   -- :samp:`--enable-crypto-{x}`
160   - (where :samp:`{x}` is a supported crypto
161   - provider): enable the :samp:`{x}` crypto
162   - provider, requiring any external dependencies it needs
163   -
164   -- :samp:`--disable-crypto-{x}`:
165   - disable the :samp:`{x}` provider, and do not
166   - link against its dependencies even if they are available
167   -
168   -- :samp:`--with-default-crypto={x}`:
169   - make :samp:`{x}` the default provider even if
170   - a higher priority one is available
171   -
172   -- :samp:`--disable-implicit-crypto`: only build crypto
173   - providers that are explicitly requested with an
174   - :samp:`--enable-crypto-{x}`
175   - option
176   -
177   -For example, if you want to guarantee that the gnutls crypto provider is
178   -used and that the native provider is not built, you could run
179   -:command:`./configure --enable-crypto-gnutls
180   ---disable-implicit-crypto`.
181   -
182   -If you build qpdf using your own build system, in order for qpdf to work
183   -at all, you need to enable at least one crypto provider. The file
184   -:file:`libqpdf/qpdf/qpdf-config.h.in` provides
185   -macros ``DEFAULT_CRYPTO``, whose value must be a string naming the
186   -default crypto provider, and various symbols starting with
187   -``USE_CRYPTO_``, at least one of which has to be enabled. Additionally,
188   -you must compile the source files that implement a crypto provider. To
189   -get a list of those files, look at
190   -:file:`libqpdf/build.mk`. If you want to omit a
191   -particular crypto provider, as long as its ``USE_CRYPTO_`` symbol is
192   -undefined, you can completely ignore the source files that belong to a
193   -particular crypto provider. Additionally, crypto providers may have
194   -their own external dependencies that can be omitted if the crypto
195   -provider is not used. For example, if you are building qpdf yourself and
196   -are using an environment that does not support gnutls or openssl, you
197   -can ensure that ``USE_CRYPTO_NATIVE`` is defined, ``USE_CRYPTO_GNUTLS``
198   -is not defined, and ``DEFAULT_CRYPTO`` is defined to ``"native"``. Then
199   -you must include the source files used in the native implementation,
200   -some of which were added or renamed from earlier versions, to your
201   -build, and you can ignore
202   -:file:`QPDFCrypto_gnutls.cc`. Always consult
203   -:file:`libqpdf/build.mk` to get the list of source
204   -files you need to build.
  462 +For information about selecting which crypto providers are compiled
  463 +into qpdf, see :ref:`crypto.build`.
205 464  
206 465 .. _crypto.runtime:
207 466  
208 467 Runtime Crypto Provider Selection
209 468 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
210 469  
211   -You can use the :qpdf:ref:`--show-crypto` option to
212   -:command:`qpdf` to get a list of available crypto
213   -providers. The default provider is always listed first, and the rest are
214   -listed in lexical order. Each crypto provider is listed on a line by
215   -itself with no other text, enabling the output of this command to be
216   -used easily in scripts.
  470 +You can use the :qpdf:ref:`--show-crypto` option to :command:`qpdf` to
  471 +get a list of available crypto providers. The default provider is
  472 +always listed first, and the rest are listed in lexical order. Each
  473 +crypto provider is listed on a line by itself with no other text,
  474 +enabling the output of this command to be used easily in scripts.
217 475  
218 476 You can override which crypto provider is used by setting the
219   -``QPDF_CRYPTO_PROVIDER`` environment variable. There are few reasons to
220   -ever do this, but you might want to do it if you were explicitly trying
221   -to compare behavior of two different crypto providers while testing
222   -performance or reproducing a bug. It could also be useful for people who
223   -are implementing their own crypto providers.
  477 +``QPDF_CRYPTO_PROVIDER`` environment variable. There are few reasons
  478 +to ever do this, but you might want to do it if you were explicitly
  479 +trying to compare behavior of two different crypto providers while
  480 +testing performance or reproducing a bug. It could also be useful for
  481 +people who are implementing their own crypto providers.
224 482  
225 483 .. _crypto.develop:
226 484  
... ... @@ -229,11 +487,11 @@ Crypto Provider Information for Developers
229 487  
230 488 If you are writing code that uses libqpdf and you want to force a
231 489 certain crypto provider to be used, you can call the method
232   -``QPDFCryptoProvider::setDefaultProvider``. The argument is the name of
233   -a built-in or developer-supplied provider. To add your own crypto
234   -provider, you have to create a class derived from ``QPDFCryptoImpl`` and
235   -register it with ``QPDFCryptoProvider``. For additional information, see
236   -comments in :file:`include/qpdf/QPDFCryptoImpl.hh`.
  490 +``QPDFCryptoProvider::setDefaultProvider``. The argument is the name
  491 +of a built-in or developer-supplied provider. To add your own crypto
  492 +provider, you have to create a class derived from ``QPDFCryptoImpl``
  493 +and register it with ``QPDFCryptoProvider``. For additional
  494 +information, see comments in :file:`include/qpdf/QPDFCryptoImpl.hh`.
237 495  
238 496 .. _crypto.design:
239 497  
... ... @@ -308,3 +566,126 @@ provide their own implementations for basic filters like
308 566 Implementing the registration functions and internal storage of
309 567 registered providers was also easier using C++-11's functional
310 568 interfaces, which was another reason to require C++-11 at this time.
  569 +
  570 +.. _autoconf-to-cmake:
  571 +
  572 +Converting From autoconf to cmake
  573 +---------------------------------
  574 +
  575 +Versions of qpdf before qpdf 11 were built with ``autoconf`` and a
  576 +home-grown GNU Make-based build system. If you built qpdf with special
  577 +``./configure`` options, this section can help you switch them over to
  578 +``cmake``.
  579 +
  580 +In most cases, there is a one-to-one mapping between configure options
  581 +and cmake options. There are a few exceptions:
  582 +
  583 +- The cmake build behaves differently with respect to whether or not
  584 + to include support for the native crypto provider. Specifically, it
  585 + is not implicitly enabled unless explicitly requested if there are
  586 + other options available. You can force it to be included by enabling
  587 + ``REQUIRE_CRYPTO_NATIVE``. For details, see :ref:`crypto.build`.
  588 +
  589 +- The ``--enable-external-libs`` option is no longer available. The
  590 + cmake build detects the presence of ``external-libs`` automatically.
  591 + See :file:`README-windows.md` in the source distribution for a more
  592 + in-depth discussion.
  593 +
  594 +- The sense of the option representing use of the OS-provided secure
  595 + random number generator has been reversed: the
  596 + ``--enable-os-secure-random``, which was on by default, has been
  597 + replaced by the ``SKIP_OS_SECURE_RANDOM`` option, which is off by
  598 + default. The option's new name and behavior match the preprocessor
  599 + symbol that it turns on.
  600 +
  601 +- Non-default test configuration is selected with environment
  602 + variables rather than cmake. The old ``./configure`` options just
  603 + set environment variables. Note that the sense of the variable for
  604 + image comparison tests has been reversed. It used to be that you had
  605 + to set ``QPDF_SKIP_TEST_COMPARE_IMAGES`` to ``1`` to *disable* image
  606 + comparison tests. This was done by default. Now you have to set
  607 + ``QPDF_TEST_COMPARE_IMAGES`` to ``1`` to *enable* image comparison
  608 + tests. Either way, they are off by default.
  609 +
  610 +- A handful of options that were specific to autoconf or the old build
  611 + system have been dropped.
  612 +
  613 +- ``cmake --install`` installs example source code in
  614 + ``doc/qpdf/examples`` in the ``examples`` installation component.
  615 + Packagers are encouraged to package this with development files if
  616 + there is no separate doc package. This can be turned off by
  617 + disabling the ``INSTALL_EXAMPLES`` build option.
  618 +
  619 +There are some new options available in the cmake build that were not
  620 +available in the autoconf build. This table shows the old options and
  621 +their equivalents in cmake.
  622 +
  623 +.. list-table:: configure flags to cmake options
  624 + :widths: 40 60
  625 + :header-rows: 0
  626 +
  627 + - - enable-avoid-windows-handle
  628 + - AVOID_WINDOWS_HANDLE
  629 +
  630 + - - enable-check-autofiles
  631 + - none -- not relevant to cmake
  632 +
  633 + - - enable-crypto-gnutls
  634 + - REQUIRE_CRYPTO_GNUTLS
  635 +
  636 + - - enable-crypto-native
  637 + - REQUIRE_CRYPTO_NATIVE (but see above)
  638 +
  639 + - - enable-crypto-openssl
  640 + - REQUIRE_CRYPTO_OPENSSL
  641 +
  642 + - - enable-doc-maintenance
  643 + - BUILD_DOC
  644 +
  645 + - - enable-external-libs
  646 + - none -- detected automatically
  647 +
  648 + - - enable-html-doc
  649 + - BUILD_DOC_HTML
  650 +
  651 + - - enable-implicit-crypto
  652 + - USE_IMPLICIT_CRYPTO
  653 +
  654 + - - enable-insecure-random
  655 + - USE_INSECURE_RANDOM
  656 +
  657 + - - enable-ld-version-script
  658 + - none -- detected automatically
  659 +
  660 + - - enable-maintainer-mode
  661 + - MAINTAINER_MODE (slight differences)
  662 +
  663 + - - enable-os-secure-random (on by default)
  664 + - SKIP_OS_SECURE_RANDOM (off by default)
  665 +
  666 + - - enable-oss-fuzz
  667 + - OSS_FUZZ
  668 +
  669 + - - enable-pdf-doc
  670 + - BUILD_DOC_PDF
  671 +
  672 + - - enable-rpath
  673 + - none -- cmake handles rpath correctly
  674 +
  675 + - - enable-show-failed-test-output
  676 + - SHOW_FAILED_TEST_OUTPUT
  677 +
  678 + - - enable-test-compare-images
  679 + - set the ``QPDF_TEST_COMPARE_IMAGES`` environment variable
  680 +
  681 + - - enable-werror
  682 + - WERROR
  683 +
  684 + - - with-buildrules
  685 + - none -- not relevant to cmake
  686 +
  687 + - - with-default-crypto
  688 + - DEFAULT_CRYPTO
  689 +
  690 + - - large-file-test-path
  691 + - set the ``QPDF_LARGE_FILE_TEST_PATH`` environment variable
... ...
manual/library.rst
... ... @@ -23,10 +23,20 @@ All header files are installed in the
23 23 you use ``#include <qpdf/QPDF.hh>`` rather than adding
24 24 :file:`include/qpdf` to your include path.
25 25  
26   -When linking against the qpdf static library, you may also need to
27   -specify ``-lz -ljpeg`` on your link command. If your system understands
28   -how to read libtool :file:`.la` files, this may not
29   -be necessary.
  26 +qpdf installs a ``pkg-config`` configuration with package name
  27 +``libqpdf`` and a ``cmake`` configuration with package name ``qpdf``.
  28 +The ``libqpdf`` target is exported in the ``qpdf::`` namespace. The
  29 +following is an example of a :file:`CMakeLists.txt` file for a
  30 +single-file executable that links with qpdf:
  31 +
  32 +.. code-block:: cmake
  33 +
  34 + cmake_minimum_required(VERSION 3.16)
  35 + project(some-application LANGUAGES CXX)
  36 + find_package(qpdf)
  37 + add_executable(some-application some-application.cc)
  38 + target_link_libraries(some-application qpdf::libqpdf)
  39 +
30 40  
31 41 The qpdf library is safe to use in a multithreaded program, but no
32 42 individual ``QPDF`` object instance (including ``QPDF``,
... ...
manual/packaging.rst
... ... @@ -9,6 +9,10 @@ chapter is for you. Otherwise, feel free to skip.
9 9 Build Options
10 10 -------------
11 11  
  12 +For a detailed discussion of build options, please refer to
  13 +:ref:`build-options`. This section calls attention to options that are
  14 +particularly useful to packagers.
  15 +
12 16 - Perl must be present at build time. Prior to qpdf version 9.1.1,
13 17 there was a runtime dependency on perl, but this is no longer the
14 18 case.
... ... @@ -16,31 +20,23 @@ Build Options
16 20 - Make sure you are getting the intended behavior with regard to crypto
17 21 providers. Read :ref:`crypto.build` for details.
18 22  
19   -- Passing :samp:`--enable-show-failed-test-output` to
20   - :command:`./configure` will cause any failed test
21   - output to be written to the console. This can be very useful for
22   - seeing test failures generated by autobuilders where you can't access
23   - qtest.log after the fact.
24   -
25   -- If qpdf's build environment detects the presence of autoconf and
26   - related tools, it will check to ensure that automatically generated
27   - files are up-to-date with recorded checksums and fail if it detects a
28   - discrepancy. This feature is intended to prevent you from
29   - accidentally forgetting to regenerate automatic files after modifying
30   - their sources. If your packaging environment automatically refreshes
31   - automatic files, it can cause this check to fail. Suppress qpdf's
32   - checks by passing :samp:`--disable-check-autofiles`
33   - to :command:`/.configure`. This is safe since qpdf's
34   - :command:`autogen.sh` just runs autotools in the
35   - normal way.
36   -
37   -- QPDF's :command:`make install` does not install completion files by
38   - default, but as a packager, it's good if you install them wherever
39   - your distribution expects such files to go. You can find completion
40   - files to install in the :file:`completions` directory.
41   -
42   -- Packagers are encouraged to install the source files from the
43   - :file:`examples` directory along with qpdf development packages.
  23 +- Use of ``SHOW_FAILED_TEST_OUTPUT`` is recommended for building in
  24 + continuous integration or other automated environments as it makes
  25 + it possible to see test failures in build logs. This should be
  26 + combined with either ``ctest --verbose`` or ``ctest
  27 + --output-on-failure``.
  28 +
  29 +- qpdf's install targets do not install completion files by default
  30 + since there is no standard location for them. As a packager, it's
  31 + good if you install them wherever your distribution expects such
  32 + files to go. You can find completion files to install in the
  33 + :file:`completions` directory. See the :file:`completions/README.md`
  34 + file for more information.
  35 +
  36 +- Starting with qpdf 11, qpdf's default installation installs source
  37 + files from the examples directory with documentation. Prior to qpdf
  38 + 11, this was a recommendation for packagers but was not done
  39 + automatically.
44 40  
45 41 .. _packaging-doc:
46 42  
... ... @@ -54,27 +50,34 @@ you may want to consider for your packages:
54 50 - **Do nothing**
55 51  
56 52 When you run ``make install``, the file :file:`README-doc.txt` is
57   - installed in ``$(docdir)``. That file tells the reader where to find
58   - the documentation online and where to go to download offline copies
59   - of the documentation. This is the option selected by the debian
60   - packages.
  53 + installed in the documentation directory. That file tells the reader
  54 + where to find the documentation online and where to go to download
  55 + offline copies of the documentation. This is the option selected by
  56 + the debian packages.
61 57  
62 58 - **Embed pre-built documentation**
63 59  
64 60 You can obtain pre-built documentation and extract its contents into
65 61 your distribution. This is what the Windows binary distributions
66   - available from the qpdf release site do. You can find the pre-build
  62 + available from the qpdf release site do. You can find the pre-built
67 63 documentation in the release area in the file
68   - :file:`qpdf-{version}-doc.zip`.
  64 + :file:`qpdf-{version}-doc.zip`. For an example of this approach,
  65 + look at qpdf's GitHub actions build scripts. The
  66 + :file:`build-scripts/build-doc` script builds with
  67 + ``-DBUILD_DOC_DIST=1`` to create the documentation distribution. The
  68 + :file:`build-scripts/build-windows` script extracts it into the
  69 + build tree and builds with ``-DINSTALL_MANUAL=1`` to include it in
  70 + the installer.
69 71  
70 72 - **Build the documentation yourself**
71 73  
72 74 You can build the documentation as part of your build process. Be
73   - sure to pass ``--enable-doc-maintenance`` to ``./configure``, and
74   - install it with ``make doc-dist DOC_DEST=...``. This is what the
75   - AppImage build does. The latest version of Sphinx at the time of the
76   - initial conversion a sphinx-based documentation was 4.3.2. Older
77   - versions are not guaranteed to work.
  75 + sure to pass ``-DBUILD_DOC_DIST=1`` and ``-DINSTALL_MANUAL=1`` to
  76 + cmake. This is what the AppImage build does. The latest version of
  77 + Sphinx at the time of the initial conversion a sphinx-based
  78 + documentation was 4.3.2. Older versions are not guaranteed to work.
  79 +
  80 +.. _doc-packaging-rationale:
78 81  
79 82 Documentation Packaging Rationale
80 83 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
... ...
qpdf/qtest/qpdf.test
... ... @@ -1185,8 +1185,9 @@ my @bug_tests = (
1185 1185 ["335a", "ozz-fuzz-12152", 2],
1186 1186 ["335b", "ozz-fuzz-14845", 2],
1187 1187 ["fuzz-16214", "stream in object stream", 3, "--preserve-unreferenced"],
1188   - # When adding to this list, consider adding to CORPUS_FROM_TEST
1189   - # in fuzz/build.mk and updating the count in fuzz/qtest/fuzz.test.
  1188 + # When adding to this list, consider adding to CORPUS_FROM_TEST in
  1189 + # fuzz/CMakeLists.txt and updating the count in
  1190 + # fuzz/qtest/fuzz.test.
1190 1191 );
1191 1192 $n_tests += scalar(@bug_tests);
1192 1193 foreach my $d (@bug_tests)
... ...