From dba228452df014b6302eae5c5d97963fedebe6d3 Mon Sep 17 00:00:00 2001 From: Jay Berkenbilt Date: Tue, 22 Aug 2017 13:00:49 -0400 Subject: [PATCH] Rename README files before converting to markdown --- README | 239 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- README-hardening.md | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ README-maintainer.md | 178 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ README-what-to-download.md | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ README-what-to-download.txt | 56 -------------------------------------------------------- README-windows.md | 229 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ README-windows.txt | 229 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- README.hardening | 93 --------------------------------------------------------------------------------------------- README.maintainer | 178 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- README.md | 239 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 795 insertions(+), 795 deletions(-) delete mode 100644 README create mode 100644 README-hardening.md create mode 100644 README-maintainer.md create mode 100644 README-what-to-download.md delete mode 100644 README-what-to-download.txt create mode 100644 README-windows.md delete mode 100644 README-windows.txt delete mode 100644 README.hardening delete mode 100644 README.maintainer create mode 100644 README.md diff --git a/README b/README deleted file mode 100644 index 5c15fa3..0000000 --- a/README +++ /dev/null @@ -1,239 +0,0 @@ -This is the QPDF package. Information about it can be found at -http://qpdf.sourceforge.net. The source code repository is hosted -at github: https://github.com/qpdf/qpdf. - -QPDF is copyright (c) 2005-2017 Jay Berkenbilt - -This software may be distributed under the terms of version 2 of the -Artistic License which may be found in the source distribution as -"Artistic-2.0". It is provided "as is" without express or implied -warranty. - - -Prerequisites -============= - -QPDF depends on the external libraries "zlib" and "jpeg". These are -part of every Linux distribution and are readily available. Download -information appears in the documentation. For Windows, you can -download pre-built binary versions of these libraries for some -compilers; see README-windows.txt for additional details. - -QPDF requires a C++ compiler that works with STL. Your compiler must -also support "long long". Almost all modern compilers do. If you are -trying to port qpdf to a compiler that doesn't support long long, you -could change all occurrences of "long long" to "long" in the source -code, noting that this would break binary compatibility with other -builds of qpdf. Doing so would certainly prevent qpdf from working -with files larger than 2 GB, but remaining functionality would most -likely work fine. If you built qpdf this way and it passed its test -suite with large file support disabled, you could be confident that -you had an otherwise working qpdf. - - -Licensing terms of embedded software -==================================== - -QPDF makes use of zlib and jpeg libraries for its functionality. These -packages can be downloaded separately from their own download -locations, or they can be downloaded in the external-libs area of the -qpdf download site. - -The Rijndael encryption implementation used as the basis for AES -encryption and decryption support comes from Philip J. Erdelsky's -public domain implementation. The files libqpdf/rijndael.cc and -libqpdf/qpdf/rijndael.h remain in the public domain. They were -obtained from - - http://www.efgh.com/software/rijndael.htm - http://www.efgh.com/software/rijndael.txt - -The embedded sha2 code comes from sphlib 3.0 - - http://www.saphir2.com/sphlib/ - -That code has the following license: - - Copyright (c) 2007-2011 Projet RNRT SAPHIR - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - "Software"), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY - CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, - TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE - SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - - -Building from a pristine checkout -================================= - -When building qpdf from a pristine checkout from version control, -documentation and automatically generated files are not present. -Building on Windows from a pristine checkout is not guaranteed to work -because of issues running autoconf; see README-windows.txt for how to -handle this. For UNIX and UNIX-like systems, you must have some -addditional tools installed to build from the source repository. To -do this, you should run - -./autogen.sh -./configure --enable-doc-maintenance -make -make install - -If you don't have Apache fop and the docbook stylesheets installed, -you won't be able to build documentation. You can omit ---enable-doc-maintenance and produce working qpdf software that passes -its test suite, but make install will fail because the documentation -files won't exist. Depending on your purposes, you can either work -around this or grab the docs from a source distribution. - - -Building from source distribution on UNIX/Linux -=============================================== - -For UNIX and UNIX-like systems, you can usually get by with just - -./configure -make -make install - -Packagers may set DESTDIR, in which case make install will install -inside of DESTDIR, as is customary with many packages. For more -detailed general information, see the "INSTALL" file in this -directory. If you are already accustomed to building and installing -software that uses autoconf, there's nothing new for you in the -INSTALL file. - - -Building on Windows -=================== - -QPDF is known to build and pass its test suite with mingw (latest -version tested: gcc 4.6.2), mingw64 (latest version tested: 4.7.0) and -Microsoft Visual C++ 2010, both 32-bit and 64-bit versions. MSYS plus -ActiveState Perl is required to build as well in order to get make -and other related tools. See README-windows.txt for details on how to -build under Windows, see README-windows.txt. - - -Additional Notes on Build -========================= - -QPDF's build system, inspired by abuild (http://www.abuild.org), can -optionally use its own built-in rules rather than using libtool and -obeying the compiler specified with configure. This can be enabled by -passing --with-buildrules=buildrules where buildrules corresponds to -one of the .mk files (other than rules.mk) in the make directory. -This should never be necessary on a UNIX system, but may be necessary -on a Windows system. See README-windows.txt for details. There is a -gcc-linux.mk file enable "gcc-linux" build rules, but it is intended -to help test the build system; Linux users should build with the -"libtools" rules, which are enabled by default. - -The QPDF package provides some executables and a software library. A -user's manual can be found in the "doc" directory. The docbook -sources to the user's manual can be found in the "manual" directory. - -The software library is just libqpdf, and all the header files are in -the qpdf subdirectory. If you link statically with -lqpdf, then you -will also need to link with -lz and -ljpeg. The shared qpdf library is -linked with -lz and -ljpeg, none of qpdf's public header files -directly include files from libz, and only Pl_DCT.hh includes files -from libjpeg, so for most cases, qpdf's development files are self -contained. If you need to use Pl_DCT in your application code, you -will need to have the header files for some libjpeg distribution in -your include path. - -To learn about using the library, please read comments in the header -files in include/qpdf, especially QPDF.hh, QPDFObjectHandle.hh, and -QPDFWriter.hh. You can also study the code of qpdf/qpdf.cc, which -exercises most of the public interface. There are additional example -programs in the examples directory. Reading all the source files in -the qpdf directory (including the qpdf command-line tool and some test -drivers) along with the code in the examples directory will give you a -complete picture of every aspect of the public interface. - - -Additional Notes on Test Suite -============================== - -By default, slow tests are disabled. Slow tests include image -comparison tests and large file tests. Image comparison tests can be -enabled by passing --enable-test-compare-images to ./configure. This -was on by default in qpdf versions prior to 3.0, but is now off by -default. Large file tests can be enabled by passing ---with-large-file-test-path=path to ./configure or by setting the -QPDF_LARGE_FILE_TEST_PATH environment variable. On Windows, this -should be a Windows path. Run ./configure --help for additional -options. The test suite provides nearly full coverage even without -these tests. Unless you are making deep changes to the library that -would impact the contents of the generated PDF files or testing this -on a new platform for the first time, there is no real reason to run -these tests. If you're just running the test suite to make sure that -qpdf works for your build, the default tests are adequate. The -configure rules for these tests do nothing other than setting -variables in autoconf.mk, so you can feel free to turn these on and -off directly in autoconf.mk rather than rerunning configure. - -If you are packaging qpdf for a distribution and preparing a build -that is run by an autobuilder, you may want to add the ---enable-show-failed-test-output to configure options. This way, if -the test suite fails, test failure detail will be included in the -build output. Otherwise, you will have to have access to the -qtest.log file from the build to view test failures. The debian -packages for qpdf enable this option, for example. - - -Random Number Generation -======================== - -By default, when the qpdf detects either the Windows cryptography API -or the existence of /dev/urandom, /dev/arandom, or /dev/random, it -uses them to generate cryptography secure random numbers. If none of -these conditions are true, the build will fail with an error. This -behavior can be modified in several ways: - - * If you configure with --disable-os-secure-random or define - SKIP_OS_SECURE_RANDOM, qpdf will not attempt to use Windows - cryptography or the random device. You must either supply your own - random data provider or allow use of insecure random numbers. - - * If you configure qpdf with the --enable-insecure-random option or - define USE_INSECURE_RANDOM, qpdf will try insecure random numbers - if OS-provided secure random numbers are disabled. This is not a - fallback. In order for insecure random numbers to be used, you - must also disable OS secure random numbers since, otherwise, - failure to find OS secure random numbers is a compile error. The - insecure random number source is stdlib's random() or rand() calls. - These random numbers are not cryptography secure, but the qpdf - library is fully functional using them. Using non-secure random - numbers means that it's easier in some cases to guess encryption - keys. If you're not generating encrypted files, there's no - advantage to using secure random numbers. - - * In all cases, you may supply your own random data provider. To do - this, derive a class from qpdf/RandomDataProvider (since 5.1.0) and - call QUtil::setRandomDataProvider before you create any QPDF - objects. If you supply your own random data provider, it will - always be used even if support for one of the other random data - providers is compiled in. If you wish to avoid any possibility of - your build of qpdf from using anything but a user-supplied random - data provider, you can define SKIP_OS_SECURE_RANDOM and not - USE_INSECURE_RANDOM. In this case, qpdf will throw a runtime error - if any attempt is made to generate random numbers and no random - data provider has been supplied. - -If you are building qpdf on a platform that qpdf doesn't know how to -generate secure random numbers on, a patch would be welcome. diff --git a/README-hardening.md b/README-hardening.md new file mode 100644 index 0000000..a2389af --- /dev/null +++ b/README-hardening.md @@ -0,0 +1,93 @@ +Avoiding operator[] +=================== + +During a security review by Red Hat security team (specifically +Florian Weimer), it was discovered that qpdf used std::string and +std::vector's operator[], which has no bounds checking by design. +Instead, using those objects' at() method is preferable since it does +bounds checking. Florian has a tool that can detect all uses of these +methods and report them. I have a short perl script that +automatically corrects any such uses. The perl script is not intended +to be general, but it could be reasonably general. The only known +shortcut is that it might not work very well with some cases of nested +[]'s like a[b[c]] or with cases where there are line breaks inside the +brackets. For qpdf's coding style, it worked on all cases reported. + +To use this, obtain htcondor-analyzer, run it, and respond to the +report. Here's what I did. + +sudo aptitude install libclang-dev llvm llvm-dev clang +cd /tmp +git clone https://github.com/fweimer/htcondor-analyzer +# HEAD = 5fa06fc68a9b0677e9de162279185d58ba1e8477 at this writing +cd htcondor-analyzer +make + +in qpdf + +./autogen.sh +/tmp/htcondor-analyzer/create-db +CC=/tmp/htcondor-analyzer/cc CXX=/tmp/htcondor-analyzer/cxx ./configure --disable-shared --disable-werror +# to remove conftest.c +\rm htcondor-analyzer.sqlite +/tmp/htcondor-analyzer/create-db + +Repeat until no more errors: + +/tmp/fix-at.pl is shown below. + +make +/tmp/htcondor-analyzer/report | grep std:: | grep qpdf >| /tmp/r +perl /tmp/fix-at.pl /tmp/r +# move all *.new over the original file. patmv is my script. Can +# also use a for loop. +patmv -f s/.new// **/*.new + +---------- /tmp/fix-at.pl ---------- +#!/usr/bin/env perl +require 5.008; +use warnings; +use strict; +use File::Basename; + +my $whoami = basename($0); + +my %to_fix = (); +while (<>) +{ + chomp; + die unless m/^([^:]+):(\d+):(\d+):\s*(.*)$/; + my ($file, $line, $col, $message) = ($1, $2, $3, $4); + if ($message !~ m/operator\[\]/) + { + warn "skipping $_\n"; + next; + } + push(@{$to_fix{$file}}, [$line, $col, $message]); +} +foreach my $file (sort keys %to_fix) +{ + open(F, "<$file") or die; + my @lines = (); + close(F); + my $last = ""; + my @data = reverse sort { ($a->[0] <=> $b->[0]) || ($a->[1] <=> $b->[1]) } @{$to_fix{$file}}; + foreach my $d (@data) + { + my ($line, $col) = @$d; + next if $last eq "$line:$col"; + $last = "$line:$col"; + die if $line-- < 1; + die if $col-- < 1; + print $lines[$line]; + $lines[$line] =~ s/^(.{$col})([^\[]+)\[([^\]]+)\]/$1$2.at($3)/ or die "$file:$last\n"; + print $lines[$line]; + } + open(F, ">$file.new") or die; + foreach my $line (@lines) + { + print F $line; + } + close(F) or die; +} +-------------------- diff --git a/README-maintainer.md b/README-maintainer.md new file mode 100644 index 0000000..e602744 --- /dev/null +++ b/README-maintainer.md @@ -0,0 +1,178 @@ +Release Reminders +================= + + * Test for binary compatility. The easiest way to do this is to check + out the last release, run the test suite, check out the new + release, run make build_libqpdf, check out the old release, and run + make check NO_REBUILD=1. + + * When making a release, always remember to run large file tests and + image comparison tests (--enable-test-compare-images + --with-large-file-test-path=/path). For Windows, use a Windows + style path, not an MSYS path for large files. For a major release, + consider running a spelling checker over the source code to catch + errors in variable names, strings, and comments. Use ispell -p + ispell-words. + + * Run tests with sanitize address enabled: + + ./configure CFLAGS="-fsanitize=address -g" \ + CXXFLAGS="-fsanitize=address -g" \ + LDFLAGS="-fsanitize=address" \ + --enable-werror --disable-shared + + As of gcc 6.3.0, this exposes some good things but appears to also + have some false positive leak reports. Valgrind is more reliable + but also may miss some things that this catches. + + * Consider running tests with latest gcc and/or valgrind. To test + with valgrind: + + ./configure --disable-shared + make -j8 -k VALGRIND=1 + make -k check NO_REBUILD=1 + + This moves each binary into a subdirectory and replaces it with a + link to make/exec-z. See make/exec-z. + + * Check all open issues in the sourceforge trackers and on github. + + * If any interfaces were added or changed, check C API to see whether + changes are appropriate there as well. If necessary, review the + casting policy in the manual, and ensure that integer types are + properly handled. + + * Remember to avoid using operator[] with std::string or + std::vector. See README.hardening for details. + + * Increment shared library version information as needed (LT_* in + configure.ac) + + * Update release notes in manual -- look at diffs and ChangeLog + + * Add a release entry to ChangeLog + + * Make sure version numbers are consistent in the following + locations: + + configure.ac + libqpdf/QPDF.cc + manual/qpdf-manual.xml + + make_dist verifies this consistency. + + * Update release date in manual/qpdf-manual.xml. Remember to ensure + that the entities at the top of the document are consistent with + the release notes for both version and release date. + + * Check TODO file to make sure all planned items for the release are + done or retargeted. + + * Each year, update copyright notices. Just do a case-insensitive + search for copyright. Don't forget copyright in manual. Also update + debian copyright in debian package. Last updated: 2017. + + * To construct a source distribution from a pristine checkout, + make_dist does the following: + + ./autogen.sh + ./configure --enable-doc-maintenance --enable-werror + make build_manual + make distclean + + * To create a source release, do an export from the version control + system to a directory called qpdf-version. For example, from this + directory: + + rm -rf /tmp/qpdf-x.y.z + git archive --prefix=qpdf-x.y.z/ HEAD . | (cd /tmp; tar xf -) + + From the parent of that directory, run make_dist with the directory + as an argument. Remember to have fop in your path. For internally + testing releases, you can run make_dist with the --no-tests option. + + * To create a source release of external libs, do an export from the + version control system into a directory called qpdf-external-libs + and just make a zip file of the result called + qpdf-external-libs-src.zip. See the README.txt file there for + information on creating binary external libs releases. Run this + from the external-libs repository: + + git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -) + cd /tmp + zip -r qpdf-external-libs-src.zip external-libs + + * To create Windows binary releases, extract the qpdf source + distribution in Windows (MSYS + MINGW, MSVC). From the extracted + directory, extract the binary distribution of the external + libraries. Run ./make_windows_releases from there. You will need + to have zip in your path. + + * Before releasing, rebuild and test debian package. + + * Remember to copy README-what-to-download.txt separately onto the + download area. + + * Remember to update the web page including putting new documentation + in the "files" subdirectory of the website on sourceforge.net. + + * Create a tag in the version control system, and make backups of the + actual releases. With git, use git tag -s to create a signed tag: + + git tag -s release-qpdf-$version HEAD -m"qpdf $version" + + * When releasing on sourceforge, external-libs distributions go in + external-libs/yyyymmdd, and qpdf distributions go in qpdf/vvv. + Make the source package the default for all but Windows, and make + the 32-bit mingw build the default for Windows. + + +General Build Stuff +=================== + +QPDF uses autoconf and libtool but does not use automake. The only +files distributed with the qpdf source distribution that are not +controlled are "configure", "libqpdf/qpdf/qpdf-config.h.in", +"aclocal.m4", and some documentation. See above for the steps +required to prepare a source distribution. + +A small handful of additional files have been taken from autotools +programs. These should probably be updated from time to time. + + * config.guess, config.sub, ltmain.sh, and the m4 directory: these + were created by running libtoolize -c. To update, run libtoolize + -f -c or remove the files and rerun libtoolize. + + * Other files copied as indicated: + + cp /usr/share/automake-1.11/install-sh . + cp /usr/share/automake-1.11/mkinstalldirs . + +The entire contents of the m4 directory came from libtool.m4. If we +had some additional local parts, we could also add those to the m4 +directory. In order for this to work, it is necessary to run "aclocal +-I m4" before running autoheader and autoconf. + +If building or editing documentation, configure with +--enable-doc-maintenance. This will ensure that all tools or files +required to validate and build documentation are available. + +If you want to run make maintainer-clean or make distclean and you +haven't run ./configure, you can pass CLEAN=1 to make on the command +line to prevent it from complaining about configure not having been +run. + +Local Windows Testing Procedure +=============================== + +This is what I do for routine testing on Windows. + +From Linux, run ./autogen.sh and make autofiles.zip + +From Windows, git clone from my Linux clone, unzip external-libs, and +unzip autofiles.zip. + +Look at make_windows_releases. Set up path the same way and run +whichever ./config-* is appropriate for whichever compiler I need to +test with. Run from msys started from one of the visual studio command +line shells. The build doesn't work from git bash on my local system. diff --git a/README-what-to-download.md b/README-what-to-download.md new file mode 100644 index 0000000..7fae0d2 --- /dev/null +++ b/README-what-to-download.md @@ -0,0 +1,56 @@ +To build from source for Linux or other UNIX/UNIX-like systems, it is +generally sufficient to download just the source qpdf-.tar.gz +file. + +For Windows, there are several additional files that you might want to +download. + + * qpdf--bin-mingw32.zip + + If you just want to use the qpdf command line program or use the + qpdf DLL's C-language interface, you can download this file. You + can also download this version if you are using MINGW's gcc 4.6 (or + a binary compatible version) and want to program using the C++ + interface. + + * qpdf--bin-mingw64.zip + + A 64-bit version built with mingw. Use this for 64-bit Windows + systems. The 32-bit version will also work on Windows 64-bit. + Both the 32-bit and the 64-bit version support files over 2 GB in + size, but you may find it easier to integrate this with your own + software if you use the 64-bit version. + + * qpdf--bin-msvc32.zip + + If you want to program using qpdf's C++ interface and you are using + Microsoft Visual C++ 2010 in 32-bit mode, you can download this + file. + + * qpdf--bin-msvc64.zip + + If you want to program using qpdf's C++ interface and you are using + Microsoft Visual C++ 2010 in 64-bit mode, you can download this + file. + + * qpdf-external-libs-bin.zip + + If you want to build qpdf for Windows yourself with either MINGW or + MSVC 2010, you can download this file and extract it inside the + qpdf source distribution. Please refer to README-windows.txt in + the qpdf source distribution for additional details. Note that you + need the 2012-06-20 version or later to be able to build qpdf 3.0 + or newer. The 2009-10-24 version is required for qpdf 2.3.1 or + older. + + * qpdf-external-libs-src.zip + + If you want to build the external libraries on your own (for + Windows or anything else), you can download this archive. In + addition to including an unmodified distribution zlib and the jpeg + libary, it includes a README file and some scripts to help you + build it for Windows. qpdf was built using a binary distribution of + libjpeg-turbo for Windows. You will also have to provide those. See + README-windows.txt for details. + +If you want to build on Windows, please see also README-windows.txt. diff --git a/README-what-to-download.txt b/README-what-to-download.txt deleted file mode 100644 index 7fae0d2..0000000 --- a/README-what-to-download.txt +++ /dev/null @@ -1,56 +0,0 @@ -To build from source for Linux or other UNIX/UNIX-like systems, it is -generally sufficient to download just the source qpdf-.tar.gz -file. - -For Windows, there are several additional files that you might want to -download. - - * qpdf--bin-mingw32.zip - - If you just want to use the qpdf command line program or use the - qpdf DLL's C-language interface, you can download this file. You - can also download this version if you are using MINGW's gcc 4.6 (or - a binary compatible version) and want to program using the C++ - interface. - - * qpdf--bin-mingw64.zip - - A 64-bit version built with mingw. Use this for 64-bit Windows - systems. The 32-bit version will also work on Windows 64-bit. - Both the 32-bit and the 64-bit version support files over 2 GB in - size, but you may find it easier to integrate this with your own - software if you use the 64-bit version. - - * qpdf--bin-msvc32.zip - - If you want to program using qpdf's C++ interface and you are using - Microsoft Visual C++ 2010 in 32-bit mode, you can download this - file. - - * qpdf--bin-msvc64.zip - - If you want to program using qpdf's C++ interface and you are using - Microsoft Visual C++ 2010 in 64-bit mode, you can download this - file. - - * qpdf-external-libs-bin.zip - - If you want to build qpdf for Windows yourself with either MINGW or - MSVC 2010, you can download this file and extract it inside the - qpdf source distribution. Please refer to README-windows.txt in - the qpdf source distribution for additional details. Note that you - need the 2012-06-20 version or later to be able to build qpdf 3.0 - or newer. The 2009-10-24 version is required for qpdf 2.3.1 or - older. - - * qpdf-external-libs-src.zip - - If you want to build the external libraries on your own (for - Windows or anything else), you can download this archive. In - addition to including an unmodified distribution zlib and the jpeg - libary, it includes a README file and some scripts to help you - build it for Windows. qpdf was built using a binary distribution of - libjpeg-turbo for Windows. You will also have to provide those. See - README-windows.txt for details. - -If you want to build on Windows, please see also README-windows.txt. diff --git a/README-windows.md b/README-windows.md new file mode 100644 index 0000000..43a7be9 --- /dev/null +++ b/README-windows.md @@ -0,0 +1,229 @@ +Common Setup +============ + +You may need to disable antivirus software to run qpdf's test suite. + +To be able to build qpdf and run its test suite, you must have MSYS2 +installed. This replaces the old process of having a mixture of msys, +mingw-w64, and ActiveState perl. It is now possible to do everything +with just MSYS2. + +Here's what I did on my system: + +* Download msys2 (64-bit) from msys2.org +* Run the installer. +* Run msys2_shell.cmd by allowing the installer to start it. +* From the prompt: + * Run `pacman -Syuu` and follow the instructions, which may tell you + to close the window and rerun the command multiple times. + * pacman -S make base-devel git zip unzip + * pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain + +If you would like to build with Microsoft Visual C++, install a +suitable Microsoft Visual Studio edition. In early 2016, 2015 +community edition with C++ support is fine. It may crash a few times +during installation, but repeating the installation will allow it to +finish, and the resulting software is stable. + +To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64 +shell from a command window started from one of the Visual Studio +shell windows. You must use the mingw shell for the same word size (32 +or 64 bit) as the Windows compiler since the MSVC build uses objdump +from the msys distribution. You must also have it inherit the path. +For example: + +* Start x64 native tools command prompt from msvc +* set MSYS2_PATH_TYPE=inherit +* C:\msys64\mingw64 + +Image comparison tests are disabled by default, but it is possible to +run them on Windows. To do so, add --enable-test-compare-images from +the configure statements given below and install some additional +third-party dependencies. These may be provided in an environment such +as MSYS or Cygwin or can be downloaded separately for other +environments. You may extract or install the following software into +separate folders each and add the "bin" folder to your "PATH" +environment variable to make executables and DLLs available. If +installers are provided, they might do that already by default. + + * LibJpeg (http://gnuwin32.sourceforge.net/packages/jpeg.htm) + + This archive provides some needed DLLs needed by LibTiff. + + * LibTiff (http://gnuwin32.sourceforge.net/packages/tiff.htm) + + This archive provides some needed binaries and DLLs if you want to + use the image comparison tests. It depends on some DLLs from + LibJpeg. + + * GhostScript (http://www.ghostscript.com/download/gsdnld.html) + + GhostScript is needed for image comparison tests. It's important + that the binary is available as "gs", while its default name is + "gswin32[c].exe". You can either copy one of the original files, + use "mklink" to create a hard-/softlink, or provide a custom + "gs.cmd" wrapper that forwards all arguments to one of the original + binaries. Using "mklink" with "gswin32c.exe" is probably the best + choice. + + +External Libraries +================== + +In order to build qpdf, you must have a copy of zlib and the jpeg +library. The easy way to get it is to download the external libs from +the qpdf download area. There are packages called +external-libs-bin.zip and external-libs-src.zip. If you are building +with MSVC 2015 or MINGW with MSYS2, you can just extract the +qpdf-external-libs-bin.zip zip file into the top-level qpdf source +tree. Note that you need the 2017-08-21 version (at least) to build +qpdf 7.0 or greater since this includes jpeg. Passing +--enable-external-libs to ./configure (which is done automatically if +you follow the instructions below) is sufficient to find them. + +You can also obtain zlib and jpeg directly on your own and install +them. If you are using mingw, you can just set CPPFLAGS, LDFLAGS, and +LIBS when you run ./configure so that it can find the header files and +libraries. If you are building with msvc and you want to do this, it +probably won't work because ./configure doesn't know how to interpret +LDFLAGS and LIBS properly for MSVC (though qpdf's own build system +does). In this case, you can probably get away with cheating by +passing --enable-external-libs to ./configure and then just editing +CPPFLAGS, LDFLAGS, LIBS in the generated autoconf.mk file. Note that +you should use UNIX-like syntax (-I, -L, -l) even though this is not +what cl takes on the command line. qpdf's build rules will fix it. + +You can also download qpdf-external-libs-src.zip and follow the +instructions in the README.txt there for how to build external libs. + + +Building from version control +============================= + +If you check out qpdf from version control, you will not have the +files that are generated by autoconf. If you are not changing these +files, you can grab them from a source distribution or create them +from a system that has autoconf. To create them from scratch, run +./autogen.sh on a system that has autoconf installed. Once you have +them, you can run make CLEAN=1 autofiles.zip. This will create an +autofiles.zip that you can extract on top of a fresh checkout. + + +Building with MinGW +=================== + +QPDF is known to build and pass its test suite with MSYS2 using the +32-bit and 64-bit compilers from that project and Microsoft Visual C++ +2015, both 32-bit and 64-bit versions. MSYS2 is required to build as +well in order to get make and other related tools. See common setup at +the top of this file for installation and configuration of MSYS2. +Then, from the suitable 32-bit or 64-bit environment, run + + ./config-mingw + +and then + + make + +Note that ./config-mingw just runs ./configure with specific +arguments, so you can look at it, make adjustments, and manually run +configure instead. + +Add the absolute path to the libqpdf/build directory to your PATH. +Make sure you can run the qpdf command by typing qpdf/build/qpdf and +making sure you get a help message rather than an error loading the +DLL or no output at all. Run the test suite by typing + + make check + +If all goes well, you should get a passing test suite. + +To create an installation directory, run make install. This will +create install-mingw/qpdf-VERSION and populate it. The binary +download of qpdf for Windows with mingw is created from this +directory. + +You can also take a look at make_windows_releases for reference. This +is how the distributed Windows executables are created. + + +Building with MSVC 2015 +======================= + +These instructions would likely work with newer versions of MSVC and +are known to have worked with versions as old as 2008 Express. + +You should first set up your environment to be able to run MSVC from +the command line. There is usually a batch file included with MSVC +that does this. Make sure that you start a command line environment +configured for whichever of 32-bit or 64-bit output that you intend to +build for. + +From that cmd prompt, you can start your MSYS2 shell with path +inheritance as described above. + +Configure as follows: + + ./config-msvc + +Once configured, run + + make + +Note that ./config-msvc just runs ./configure with specific arguments, +so you can look at it, make adjustments, and manually run configure +instead. + +NOTE: automated dependencies are not generated with the msvc build. +If you're planning on making modifications, you should probably work +with mingw. If there is a need, I can add dependency information to +the msvc build, but since I only use it for generating release +versions, I haven't bothered. + +Once built, add the full path to the libqpdf/build directory to your +path and run + + make check + +to run the test suite. + +If you are building with MSVC and want to debug a crash in MSVC's +debugger, first start an instance of Visual C++. Then run qpdf. When +the abort/retry/ignore dialog pops up, first attach the process from +within visual C++, and then click Retry in qpdf. + +A release version of qpdf is built by default. If you want to link +against debugging libraries, you will have to change /MD to /MDd in +make/msvc.mk. Note that you must redistribute the Microsoft runtime +DLLs. Linking with static runtime (/MT) won't work; see "Static +Runtime" below for details. + + +Runtime DLLs +============ + +Both build methods create executables and DLLs that are dependent on +the compiler's runtime DLLs. When you run make install, the +installation process will automatically detect the DLLs and copy them +into the installation bin directory. Look at the copy_dlls script for +details on how this is accomplished. + +Redistribution of the runtime DLL is unavoidable as of this writing; +see "Static Runtime" below for details. + + +Static Runtime +============== + +Building the DLL and executables with static runtime does not work +with either Visual C++ .NET 2008 (a.k.a. vc9) using /MT or with mingw +(at least as of 4.4.0) using -static-libgcc. The reason is that, in +both cases, there is static data involved with exception handling, and +when the runtime is linked in statically, exceptions cannot be thrown +across the DLL to EXE boundary. Since qpdf uses exception handling +extensively for error handling, we have no choice but to redistribute +the C++ runtime DLLs. Maybe this will be addressed in a future +version of the compilers. This has not been retested with the +toolchain versions used to create qpdf 3.0 distributions. (This has +not been revisited since MSVC 2008, but redistrbuting runtime DLLs is +extremely common and should not be a problem.) diff --git a/README-windows.txt b/README-windows.txt deleted file mode 100644 index 43a7be9..0000000 --- a/README-windows.txt +++ /dev/null @@ -1,229 +0,0 @@ -Common Setup -============ - -You may need to disable antivirus software to run qpdf's test suite. - -To be able to build qpdf and run its test suite, you must have MSYS2 -installed. This replaces the old process of having a mixture of msys, -mingw-w64, and ActiveState perl. It is now possible to do everything -with just MSYS2. - -Here's what I did on my system: - -* Download msys2 (64-bit) from msys2.org -* Run the installer. -* Run msys2_shell.cmd by allowing the installer to start it. -* From the prompt: - * Run `pacman -Syuu` and follow the instructions, which may tell you - to close the window and rerun the command multiple times. - * pacman -S make base-devel git zip unzip - * pacman -S mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain - -If you would like to build with Microsoft Visual C++, install a -suitable Microsoft Visual Studio edition. In early 2016, 2015 -community edition with C++ support is fine. It may crash a few times -during installation, but repeating the installation will allow it to -finish, and the resulting software is stable. - -To build qpdf with Visual Studio, start the msys2 mingw32 or mingw64 -shell from a command window started from one of the Visual Studio -shell windows. You must use the mingw shell for the same word size (32 -or 64 bit) as the Windows compiler since the MSVC build uses objdump -from the msys distribution. You must also have it inherit the path. -For example: - -* Start x64 native tools command prompt from msvc -* set MSYS2_PATH_TYPE=inherit -* C:\msys64\mingw64 - -Image comparison tests are disabled by default, but it is possible to -run them on Windows. To do so, add --enable-test-compare-images from -the configure statements given below and install some additional -third-party dependencies. These may be provided in an environment such -as MSYS or Cygwin or can be downloaded separately for other -environments. You may extract or install the following software into -separate folders each and add the "bin" folder to your "PATH" -environment variable to make executables and DLLs available. If -installers are provided, they might do that already by default. - - * LibJpeg (http://gnuwin32.sourceforge.net/packages/jpeg.htm) - - This archive provides some needed DLLs needed by LibTiff. - - * LibTiff (http://gnuwin32.sourceforge.net/packages/tiff.htm) - - This archive provides some needed binaries and DLLs if you want to - use the image comparison tests. It depends on some DLLs from - LibJpeg. - - * GhostScript (http://www.ghostscript.com/download/gsdnld.html) - - GhostScript is needed for image comparison tests. It's important - that the binary is available as "gs", while its default name is - "gswin32[c].exe". You can either copy one of the original files, - use "mklink" to create a hard-/softlink, or provide a custom - "gs.cmd" wrapper that forwards all arguments to one of the original - binaries. Using "mklink" with "gswin32c.exe" is probably the best - choice. - - -External Libraries -================== - -In order to build qpdf, you must have a copy of zlib and the jpeg -library. The easy way to get it is to download the external libs from -the qpdf download area. There are packages called -external-libs-bin.zip and external-libs-src.zip. If you are building -with MSVC 2015 or MINGW with MSYS2, you can just extract the -qpdf-external-libs-bin.zip zip file into the top-level qpdf source -tree. Note that you need the 2017-08-21 version (at least) to build -qpdf 7.0 or greater since this includes jpeg. Passing ---enable-external-libs to ./configure (which is done automatically if -you follow the instructions below) is sufficient to find them. - -You can also obtain zlib and jpeg directly on your own and install -them. If you are using mingw, you can just set CPPFLAGS, LDFLAGS, and -LIBS when you run ./configure so that it can find the header files and -libraries. If you are building with msvc and you want to do this, it -probably won't work because ./configure doesn't know how to interpret -LDFLAGS and LIBS properly for MSVC (though qpdf's own build system -does). In this case, you can probably get away with cheating by -passing --enable-external-libs to ./configure and then just editing -CPPFLAGS, LDFLAGS, LIBS in the generated autoconf.mk file. Note that -you should use UNIX-like syntax (-I, -L, -l) even though this is not -what cl takes on the command line. qpdf's build rules will fix it. - -You can also download qpdf-external-libs-src.zip and follow the -instructions in the README.txt there for how to build external libs. - - -Building from version control -============================= - -If you check out qpdf from version control, you will not have the -files that are generated by autoconf. If you are not changing these -files, you can grab them from a source distribution or create them -from a system that has autoconf. To create them from scratch, run -./autogen.sh on a system that has autoconf installed. Once you have -them, you can run make CLEAN=1 autofiles.zip. This will create an -autofiles.zip that you can extract on top of a fresh checkout. - - -Building with MinGW -=================== - -QPDF is known to build and pass its test suite with MSYS2 using the -32-bit and 64-bit compilers from that project and Microsoft Visual C++ -2015, both 32-bit and 64-bit versions. MSYS2 is required to build as -well in order to get make and other related tools. See common setup at -the top of this file for installation and configuration of MSYS2. -Then, from the suitable 32-bit or 64-bit environment, run - - ./config-mingw - -and then - - make - -Note that ./config-mingw just runs ./configure with specific -arguments, so you can look at it, make adjustments, and manually run -configure instead. - -Add the absolute path to the libqpdf/build directory to your PATH. -Make sure you can run the qpdf command by typing qpdf/build/qpdf and -making sure you get a help message rather than an error loading the -DLL or no output at all. Run the test suite by typing - - make check - -If all goes well, you should get a passing test suite. - -To create an installation directory, run make install. This will -create install-mingw/qpdf-VERSION and populate it. The binary -download of qpdf for Windows with mingw is created from this -directory. - -You can also take a look at make_windows_releases for reference. This -is how the distributed Windows executables are created. - - -Building with MSVC 2015 -======================= - -These instructions would likely work with newer versions of MSVC and -are known to have worked with versions as old as 2008 Express. - -You should first set up your environment to be able to run MSVC from -the command line. There is usually a batch file included with MSVC -that does this. Make sure that you start a command line environment -configured for whichever of 32-bit or 64-bit output that you intend to -build for. - -From that cmd prompt, you can start your MSYS2 shell with path -inheritance as described above. - -Configure as follows: - - ./config-msvc - -Once configured, run - - make - -Note that ./config-msvc just runs ./configure with specific arguments, -so you can look at it, make adjustments, and manually run configure -instead. - -NOTE: automated dependencies are not generated with the msvc build. -If you're planning on making modifications, you should probably work -with mingw. If there is a need, I can add dependency information to -the msvc build, but since I only use it for generating release -versions, I haven't bothered. - -Once built, add the full path to the libqpdf/build directory to your -path and run - - make check - -to run the test suite. - -If you are building with MSVC and want to debug a crash in MSVC's -debugger, first start an instance of Visual C++. Then run qpdf. When -the abort/retry/ignore dialog pops up, first attach the process from -within visual C++, and then click Retry in qpdf. - -A release version of qpdf is built by default. If you want to link -against debugging libraries, you will have to change /MD to /MDd in -make/msvc.mk. Note that you must redistribute the Microsoft runtime -DLLs. Linking with static runtime (/MT) won't work; see "Static -Runtime" below for details. - - -Runtime DLLs -============ - -Both build methods create executables and DLLs that are dependent on -the compiler's runtime DLLs. When you run make install, the -installation process will automatically detect the DLLs and copy them -into the installation bin directory. Look at the copy_dlls script for -details on how this is accomplished. - -Redistribution of the runtime DLL is unavoidable as of this writing; -see "Static Runtime" below for details. - - -Static Runtime -============== - -Building the DLL and executables with static runtime does not work -with either Visual C++ .NET 2008 (a.k.a. vc9) using /MT or with mingw -(at least as of 4.4.0) using -static-libgcc. The reason is that, in -both cases, there is static data involved with exception handling, and -when the runtime is linked in statically, exceptions cannot be thrown -across the DLL to EXE boundary. Since qpdf uses exception handling -extensively for error handling, we have no choice but to redistribute -the C++ runtime DLLs. Maybe this will be addressed in a future -version of the compilers. This has not been retested with the -toolchain versions used to create qpdf 3.0 distributions. (This has -not been revisited since MSVC 2008, but redistrbuting runtime DLLs is -extremely common and should not be a problem.) diff --git a/README.hardening b/README.hardening deleted file mode 100644 index a2389af..0000000 --- a/README.hardening +++ /dev/null @@ -1,93 +0,0 @@ -Avoiding operator[] -=================== - -During a security review by Red Hat security team (specifically -Florian Weimer), it was discovered that qpdf used std::string and -std::vector's operator[], which has no bounds checking by design. -Instead, using those objects' at() method is preferable since it does -bounds checking. Florian has a tool that can detect all uses of these -methods and report them. I have a short perl script that -automatically corrects any such uses. The perl script is not intended -to be general, but it could be reasonably general. The only known -shortcut is that it might not work very well with some cases of nested -[]'s like a[b[c]] or with cases where there are line breaks inside the -brackets. For qpdf's coding style, it worked on all cases reported. - -To use this, obtain htcondor-analyzer, run it, and respond to the -report. Here's what I did. - -sudo aptitude install libclang-dev llvm llvm-dev clang -cd /tmp -git clone https://github.com/fweimer/htcondor-analyzer -# HEAD = 5fa06fc68a9b0677e9de162279185d58ba1e8477 at this writing -cd htcondor-analyzer -make - -in qpdf - -./autogen.sh -/tmp/htcondor-analyzer/create-db -CC=/tmp/htcondor-analyzer/cc CXX=/tmp/htcondor-analyzer/cxx ./configure --disable-shared --disable-werror -# to remove conftest.c -\rm htcondor-analyzer.sqlite -/tmp/htcondor-analyzer/create-db - -Repeat until no more errors: - -/tmp/fix-at.pl is shown below. - -make -/tmp/htcondor-analyzer/report | grep std:: | grep qpdf >| /tmp/r -perl /tmp/fix-at.pl /tmp/r -# move all *.new over the original file. patmv is my script. Can -# also use a for loop. -patmv -f s/.new// **/*.new - ----------- /tmp/fix-at.pl ---------- -#!/usr/bin/env perl -require 5.008; -use warnings; -use strict; -use File::Basename; - -my $whoami = basename($0); - -my %to_fix = (); -while (<>) -{ - chomp; - die unless m/^([^:]+):(\d+):(\d+):\s*(.*)$/; - my ($file, $line, $col, $message) = ($1, $2, $3, $4); - if ($message !~ m/operator\[\]/) - { - warn "skipping $_\n"; - next; - } - push(@{$to_fix{$file}}, [$line, $col, $message]); -} -foreach my $file (sort keys %to_fix) -{ - open(F, "<$file") or die; - my @lines = (); - close(F); - my $last = ""; - my @data = reverse sort { ($a->[0] <=> $b->[0]) || ($a->[1] <=> $b->[1]) } @{$to_fix{$file}}; - foreach my $d (@data) - { - my ($line, $col) = @$d; - next if $last eq "$line:$col"; - $last = "$line:$col"; - die if $line-- < 1; - die if $col-- < 1; - print $lines[$line]; - $lines[$line] =~ s/^(.{$col})([^\[]+)\[([^\]]+)\]/$1$2.at($3)/ or die "$file:$last\n"; - print $lines[$line]; - } - open(F, ">$file.new") or die; - foreach my $line (@lines) - { - print F $line; - } - close(F) or die; -} --------------------- diff --git a/README.maintainer b/README.maintainer deleted file mode 100644 index e602744..0000000 --- a/README.maintainer +++ /dev/null @@ -1,178 +0,0 @@ -Release Reminders -================= - - * Test for binary compatility. The easiest way to do this is to check - out the last release, run the test suite, check out the new - release, run make build_libqpdf, check out the old release, and run - make check NO_REBUILD=1. - - * When making a release, always remember to run large file tests and - image comparison tests (--enable-test-compare-images - --with-large-file-test-path=/path). For Windows, use a Windows - style path, not an MSYS path for large files. For a major release, - consider running a spelling checker over the source code to catch - errors in variable names, strings, and comments. Use ispell -p - ispell-words. - - * Run tests with sanitize address enabled: - - ./configure CFLAGS="-fsanitize=address -g" \ - CXXFLAGS="-fsanitize=address -g" \ - LDFLAGS="-fsanitize=address" \ - --enable-werror --disable-shared - - As of gcc 6.3.0, this exposes some good things but appears to also - have some false positive leak reports. Valgrind is more reliable - but also may miss some things that this catches. - - * Consider running tests with latest gcc and/or valgrind. To test - with valgrind: - - ./configure --disable-shared - make -j8 -k VALGRIND=1 - make -k check NO_REBUILD=1 - - This moves each binary into a subdirectory and replaces it with a - link to make/exec-z. See make/exec-z. - - * Check all open issues in the sourceforge trackers and on github. - - * If any interfaces were added or changed, check C API to see whether - changes are appropriate there as well. If necessary, review the - casting policy in the manual, and ensure that integer types are - properly handled. - - * Remember to avoid using operator[] with std::string or - std::vector. See README.hardening for details. - - * Increment shared library version information as needed (LT_* in - configure.ac) - - * Update release notes in manual -- look at diffs and ChangeLog - - * Add a release entry to ChangeLog - - * Make sure version numbers are consistent in the following - locations: - - configure.ac - libqpdf/QPDF.cc - manual/qpdf-manual.xml - - make_dist verifies this consistency. - - * Update release date in manual/qpdf-manual.xml. Remember to ensure - that the entities at the top of the document are consistent with - the release notes for both version and release date. - - * Check TODO file to make sure all planned items for the release are - done or retargeted. - - * Each year, update copyright notices. Just do a case-insensitive - search for copyright. Don't forget copyright in manual. Also update - debian copyright in debian package. Last updated: 2017. - - * To construct a source distribution from a pristine checkout, - make_dist does the following: - - ./autogen.sh - ./configure --enable-doc-maintenance --enable-werror - make build_manual - make distclean - - * To create a source release, do an export from the version control - system to a directory called qpdf-version. For example, from this - directory: - - rm -rf /tmp/qpdf-x.y.z - git archive --prefix=qpdf-x.y.z/ HEAD . | (cd /tmp; tar xf -) - - From the parent of that directory, run make_dist with the directory - as an argument. Remember to have fop in your path. For internally - testing releases, you can run make_dist with the --no-tests option. - - * To create a source release of external libs, do an export from the - version control system into a directory called qpdf-external-libs - and just make a zip file of the result called - qpdf-external-libs-src.zip. See the README.txt file there for - information on creating binary external libs releases. Run this - from the external-libs repository: - - git archive --prefix=external-libs/ HEAD . | (cd /tmp; tar xf -) - cd /tmp - zip -r qpdf-external-libs-src.zip external-libs - - * To create Windows binary releases, extract the qpdf source - distribution in Windows (MSYS + MINGW, MSVC). From the extracted - directory, extract the binary distribution of the external - libraries. Run ./make_windows_releases from there. You will need - to have zip in your path. - - * Before releasing, rebuild and test debian package. - - * Remember to copy README-what-to-download.txt separately onto the - download area. - - * Remember to update the web page including putting new documentation - in the "files" subdirectory of the website on sourceforge.net. - - * Create a tag in the version control system, and make backups of the - actual releases. With git, use git tag -s to create a signed tag: - - git tag -s release-qpdf-$version HEAD -m"qpdf $version" - - * When releasing on sourceforge, external-libs distributions go in - external-libs/yyyymmdd, and qpdf distributions go in qpdf/vvv. - Make the source package the default for all but Windows, and make - the 32-bit mingw build the default for Windows. - - -General Build Stuff -=================== - -QPDF uses autoconf and libtool but does not use automake. The only -files distributed with the qpdf source distribution that are not -controlled are "configure", "libqpdf/qpdf/qpdf-config.h.in", -"aclocal.m4", and some documentation. See above for the steps -required to prepare a source distribution. - -A small handful of additional files have been taken from autotools -programs. These should probably be updated from time to time. - - * config.guess, config.sub, ltmain.sh, and the m4 directory: these - were created by running libtoolize -c. To update, run libtoolize - -f -c or remove the files and rerun libtoolize. - - * Other files copied as indicated: - - cp /usr/share/automake-1.11/install-sh . - cp /usr/share/automake-1.11/mkinstalldirs . - -The entire contents of the m4 directory came from libtool.m4. If we -had some additional local parts, we could also add those to the m4 -directory. In order for this to work, it is necessary to run "aclocal --I m4" before running autoheader and autoconf. - -If building or editing documentation, configure with ---enable-doc-maintenance. This will ensure that all tools or files -required to validate and build documentation are available. - -If you want to run make maintainer-clean or make distclean and you -haven't run ./configure, you can pass CLEAN=1 to make on the command -line to prevent it from complaining about configure not having been -run. - -Local Windows Testing Procedure -=============================== - -This is what I do for routine testing on Windows. - -From Linux, run ./autogen.sh and make autofiles.zip - -From Windows, git clone from my Linux clone, unzip external-libs, and -unzip autofiles.zip. - -Look at make_windows_releases. Set up path the same way and run -whichever ./config-* is appropriate for whichever compiler I need to -test with. Run from msys started from one of the visual studio command -line shells. The build doesn't work from git bash on my local system. diff --git a/README.md b/README.md new file mode 100644 index 0000000..5c15fa3 --- /dev/null +++ b/README.md @@ -0,0 +1,239 @@ +This is the QPDF package. Information about it can be found at +http://qpdf.sourceforge.net. The source code repository is hosted +at github: https://github.com/qpdf/qpdf. + +QPDF is copyright (c) 2005-2017 Jay Berkenbilt + +This software may be distributed under the terms of version 2 of the +Artistic License which may be found in the source distribution as +"Artistic-2.0". It is provided "as is" without express or implied +warranty. + + +Prerequisites +============= + +QPDF depends on the external libraries "zlib" and "jpeg". These are +part of every Linux distribution and are readily available. Download +information appears in the documentation. For Windows, you can +download pre-built binary versions of these libraries for some +compilers; see README-windows.txt for additional details. + +QPDF requires a C++ compiler that works with STL. Your compiler must +also support "long long". Almost all modern compilers do. If you are +trying to port qpdf to a compiler that doesn't support long long, you +could change all occurrences of "long long" to "long" in the source +code, noting that this would break binary compatibility with other +builds of qpdf. Doing so would certainly prevent qpdf from working +with files larger than 2 GB, but remaining functionality would most +likely work fine. If you built qpdf this way and it passed its test +suite with large file support disabled, you could be confident that +you had an otherwise working qpdf. + + +Licensing terms of embedded software +==================================== + +QPDF makes use of zlib and jpeg libraries for its functionality. These +packages can be downloaded separately from their own download +locations, or they can be downloaded in the external-libs area of the +qpdf download site. + +The Rijndael encryption implementation used as the basis for AES +encryption and decryption support comes from Philip J. Erdelsky's +public domain implementation. The files libqpdf/rijndael.cc and +libqpdf/qpdf/rijndael.h remain in the public domain. They were +obtained from + + http://www.efgh.com/software/rijndael.htm + http://www.efgh.com/software/rijndael.txt + +The embedded sha2 code comes from sphlib 3.0 + + http://www.saphir2.com/sphlib/ + +That code has the following license: + + Copyright (c) 2007-2011 Projet RNRT SAPHIR + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + "Software"), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be included + in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY + CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, + TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE + SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +Building from a pristine checkout +================================= + +When building qpdf from a pristine checkout from version control, +documentation and automatically generated files are not present. +Building on Windows from a pristine checkout is not guaranteed to work +because of issues running autoconf; see README-windows.txt for how to +handle this. For UNIX and UNIX-like systems, you must have some +addditional tools installed to build from the source repository. To +do this, you should run + +./autogen.sh +./configure --enable-doc-maintenance +make +make install + +If you don't have Apache fop and the docbook stylesheets installed, +you won't be able to build documentation. You can omit +--enable-doc-maintenance and produce working qpdf software that passes +its test suite, but make install will fail because the documentation +files won't exist. Depending on your purposes, you can either work +around this or grab the docs from a source distribution. + + +Building from source distribution on UNIX/Linux +=============================================== + +For UNIX and UNIX-like systems, you can usually get by with just + +./configure +make +make install + +Packagers may set DESTDIR, in which case make install will install +inside of DESTDIR, as is customary with many packages. For more +detailed general information, see the "INSTALL" file in this +directory. If you are already accustomed to building and installing +software that uses autoconf, there's nothing new for you in the +INSTALL file. + + +Building on Windows +=================== + +QPDF is known to build and pass its test suite with mingw (latest +version tested: gcc 4.6.2), mingw64 (latest version tested: 4.7.0) and +Microsoft Visual C++ 2010, both 32-bit and 64-bit versions. MSYS plus +ActiveState Perl is required to build as well in order to get make +and other related tools. See README-windows.txt for details on how to +build under Windows, see README-windows.txt. + + +Additional Notes on Build +========================= + +QPDF's build system, inspired by abuild (http://www.abuild.org), can +optionally use its own built-in rules rather than using libtool and +obeying the compiler specified with configure. This can be enabled by +passing --with-buildrules=buildrules where buildrules corresponds to +one of the .mk files (other than rules.mk) in the make directory. +This should never be necessary on a UNIX system, but may be necessary +on a Windows system. See README-windows.txt for details. There is a +gcc-linux.mk file enable "gcc-linux" build rules, but it is intended +to help test the build system; Linux users should build with the +"libtools" rules, which are enabled by default. + +The QPDF package provides some executables and a software library. A +user's manual can be found in the "doc" directory. The docbook +sources to the user's manual can be found in the "manual" directory. + +The software library is just libqpdf, and all the header files are in +the qpdf subdirectory. If you link statically with -lqpdf, then you +will also need to link with -lz and -ljpeg. The shared qpdf library is +linked with -lz and -ljpeg, none of qpdf's public header files +directly include files from libz, and only Pl_DCT.hh includes files +from libjpeg, so for most cases, qpdf's development files are self +contained. If you need to use Pl_DCT in your application code, you +will need to have the header files for some libjpeg distribution in +your include path. + +To learn about using the library, please read comments in the header +files in include/qpdf, especially QPDF.hh, QPDFObjectHandle.hh, and +QPDFWriter.hh. You can also study the code of qpdf/qpdf.cc, which +exercises most of the public interface. There are additional example +programs in the examples directory. Reading all the source files in +the qpdf directory (including the qpdf command-line tool and some test +drivers) along with the code in the examples directory will give you a +complete picture of every aspect of the public interface. + + +Additional Notes on Test Suite +============================== + +By default, slow tests are disabled. Slow tests include image +comparison tests and large file tests. Image comparison tests can be +enabled by passing --enable-test-compare-images to ./configure. This +was on by default in qpdf versions prior to 3.0, but is now off by +default. Large file tests can be enabled by passing +--with-large-file-test-path=path to ./configure or by setting the +QPDF_LARGE_FILE_TEST_PATH environment variable. On Windows, this +should be a Windows path. Run ./configure --help for additional +options. The test suite provides nearly full coverage even without +these tests. Unless you are making deep changes to the library that +would impact the contents of the generated PDF files or testing this +on a new platform for the first time, there is no real reason to run +these tests. If you're just running the test suite to make sure that +qpdf works for your build, the default tests are adequate. The +configure rules for these tests do nothing other than setting +variables in autoconf.mk, so you can feel free to turn these on and +off directly in autoconf.mk rather than rerunning configure. + +If you are packaging qpdf for a distribution and preparing a build +that is run by an autobuilder, you may want to add the +--enable-show-failed-test-output to configure options. This way, if +the test suite fails, test failure detail will be included in the +build output. Otherwise, you will have to have access to the +qtest.log file from the build to view test failures. The debian +packages for qpdf enable this option, for example. + + +Random Number Generation +======================== + +By default, when the qpdf detects either the Windows cryptography API +or the existence of /dev/urandom, /dev/arandom, or /dev/random, it +uses them to generate cryptography secure random numbers. If none of +these conditions are true, the build will fail with an error. This +behavior can be modified in several ways: + + * If you configure with --disable-os-secure-random or define + SKIP_OS_SECURE_RANDOM, qpdf will not attempt to use Windows + cryptography or the random device. You must either supply your own + random data provider or allow use of insecure random numbers. + + * If you configure qpdf with the --enable-insecure-random option or + define USE_INSECURE_RANDOM, qpdf will try insecure random numbers + if OS-provided secure random numbers are disabled. This is not a + fallback. In order for insecure random numbers to be used, you + must also disable OS secure random numbers since, otherwise, + failure to find OS secure random numbers is a compile error. The + insecure random number source is stdlib's random() or rand() calls. + These random numbers are not cryptography secure, but the qpdf + library is fully functional using them. Using non-secure random + numbers means that it's easier in some cases to guess encryption + keys. If you're not generating encrypted files, there's no + advantage to using secure random numbers. + + * In all cases, you may supply your own random data provider. To do + this, derive a class from qpdf/RandomDataProvider (since 5.1.0) and + call QUtil::setRandomDataProvider before you create any QPDF + objects. If you supply your own random data provider, it will + always be used even if support for one of the other random data + providers is compiled in. If you wish to avoid any possibility of + your build of qpdf from using anything but a user-supplied random + data provider, you can define SKIP_OS_SECURE_RANDOM and not + USE_INSECURE_RANDOM. In this case, qpdf will throw a runtime error + if any attempt is made to generate random numbers and no random + data provider has been supplied. + +If you are building qpdf on a platform that qpdf doesn't know how to +generate secure random numbers on, a patch would be welcome. -- libgit2 0.21.4