OpenSystemsDevelopment / qpdf

Commit f5265924da50eefde8f606f0ed88a0bac649ffd0

Authored by Jay Berkenbilt 2018-02-21 09:28:25 -0500

Reorganize README-maintainer

Update this to be a more accurate reflection of what I actually do and
to make it a better and more usable checklist.

Inline Side-by-side

Showing 1 changed file with 253 additions and 95 deletions

README-maintainer

View file @f526592

1		-# Release Reminders
2		-
3		-* For debugging:
4		- ```
5		- ./configure CFLAGS="-g" CXXFLAGS="-g" --enable-werror --disable-shared
6		- ```
7		-* When making a release, always remember to run large file tests and image comparison tests (`--enable-test-compare-images` `--with-large-file-test-path=/path`). For Windows, use a Windows style path, not an MSYS path for large files. For a major release, consider running a spelling checker over the source code to catch errors in variable names, strings, and comments. Use `ispell -p ispell-words`.
8		-* Run tests with sanitize address enabled:
9		- ```
10		- ./configure CFLAGS="-fsanitize=address -g" \
11		- CXXFLAGS="-fsanitize=address -g" \
12		- LDFLAGS="-fsanitize=address" \
13		- --enable-werror --disable-shared
14		- ```
15		- The test suite should run clean with this. This seems to be more reliable than valgrind.
16		-* Test with clang. Pass `CC=clang CXX=clang++` to `./configure`.
	1	+ROUTINE DEVELOPMENT
	2	+
	3	+Default:
	4	+
	5	+./configure --enable-werror --disable-shared
	6	+
	7	+Debugging:
	8	+
	9	+./configure CFLAGS="-g" CXXFLAGS="-g" --enable-werror --disable-shared
	10	+
	11	+Memory checks:
	12	+
	13	+./configure CFLAGS="-fsanitize=address -g" \
	14	+ CXXFLAGS="-fsanitize=address -g" \
	15	+ LDFLAGS="-fsanitize=address" \
	16	+ --enable-werror --disable-shared
	17	+
	18	+
	19	+CODING RULES
	20	+
	21	+* Avoid atoi. Use QUtil::string_to_int instead. It does
	22	+ overflow/underflow checking.
	23	+
	24	+* Remember to avoid using `operator[]` with `std::string` or
	25	+ `std::vector`. Instead, use `at()`. See README-hardening.md for
	26	+ details.
	27	+
	28	+
	29	+RELEASE PREPARATION
	30	+
	31	+* Each year, update copyright notices. Just do a case-insensitive
	32	+ search for copyright. Don't forget copyright in manual. Also update
	33	+ debian copyright in debian package. Last updated: 2018.
	34	+
17	35	* Check all open issues in the sourceforge trackers and on github.
18		-* If any interfaces were added or changed, check C API to see whether changes are appropriate there as well. If necessary, review the casting policy in the manual, and ensure that integer types are properly handled.
19		-* Avoid atoi. Use QUtil::string_to_int instead. It does overflow/underflow checking.
20		-* Remember to avoid using `operator[]` with `std::string` or `std::vector`. Instead, use `at()`. See README-hardening.md for details.
21		-* Increment shared library version information as needed (`LT_*` in `configure.ac`)
22		-* Test for binary compatibility. The easiest way to do this is to check out the last release, run the test suite, check out the new release, run `./autogen.mk; ./configure --enable-werror; make build_libqpdf`, check out the old release, and run `make check NO_REBUILD=1`.
23		-* Update release notes in manual. Look at diffs and ChangeLog.
24		-* Add a release entry to ChangeLog.
	36	+
	37	+* Check `TODO` file to make sure all planned items for the release are
	38	+ done or retargeted.
	39	+
	40	+* Run a spelling checker over the source code to catch errors in
	41	+ variable names, strings, and comments.
	42	+
	43	+ ispell -p words */.hh */.cc manual/*
	44	+
	45	+* If needed, run large file and image comparison tests. Configure
	46	+ options:
	47	+
	48	+--enable-test-compare-images --with-large-file-test-path=/path
	49	+
	50	+ For Windows, use a Windows style path, not an MSYS path for large files.
	51	+
	52	+* Test with clang. Pass `CC=clang CXX=clang++` to `./configure`.
	53	+
	54	+* Test build on a mac.
	55	+
	56	+* Test with address sanitizer as described above.
	57	+
	58	+* A small handful of additional files have been taken from autotools
	59	+ programs. These should probably be updated from time to time.
	60	+
	61	+ * `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory:
	62	+ these were created by running `libtoolize -c`. To update, run
	63	+ `libtoolize -f -c` or remove the files and rerun `libtoolize`.
	64	+
	65	+ * Other files copied as indicated:
	66	+ ```
	67	+ cp /usr/share/automake-1.11/install-sh .
	68	+ cp /usr/share/automake-1.11/mkinstalldirs .
	69	+ ```
	70	+
	71	+ The entire contents of the `m4` directory came from `libtool.m4`. If
	72	+ we had some additional local parts, we could also add those to the
	73	+ `m4` directory. In order for this to work, it is necessary to run
	74	+ `aclocal -I m4` before running `autoheader` and `autoconf`. The
	75	+ `autogen.sh` script handles this.
	76	+
	77	+* If any interfaces were added or changed, check C API to see whether
	78	+ changes are appropriate there as well. If necessary, review the
	79	+ casting policy in the manual, and ensure that integer types are
	80	+ properly handled.
	81	+
	82	+* Increment shared library version information as needed (`LT_*` in
	83	+ `configure.ac`)
	84	+
	85	+* Test for binary compatibility:
	86	+ * Check out the last release
	87	+ * ./autogen.sh && ./configure --enable-werror && make -j$(nproc)
	88	+ * Check out the current version
	89	+ * ./autogen.sh && ./configure --enable-werror && make -j$(nproc) build_libqpdf
	90	+ * Checkout the last release
	91	+ * make check NO_REBUILD=1
	92	+
25	93	* Make sure version numbers are consistent in the following locations:
26	94	* configure.ac
27	95	* libqpdf/QPDF.cc
28	96	* manual/qpdf-manual.xml
29	97	`make_dist` verifies this consistency.
30		-* Generate a signed AppImage using the docker image in appimage. Arguments to the docker container are arguments to git clone. The build should be made off the exact commit that will be officially tagged as the release but built prior to tagging the release.
31		-Example:
32		- ```
33		- cd appimage
34		- docker build -t qpdfbuild .
35		- rm -rf /tmp/build
36		- mkdir -p /tmp/build
37		- cp -rLp ~/.gnupg/. /tmp/build/.gnupg
38		- docker run --privileged -ti --rm -v /tmp/build:/tmp/build qpdfbuild https://github.com/jberkenbilt/qpdf -b work
39		- ```
40		- Rename the AppImage in appimage/build to qpdf-<version>-x86_64.AppImage and include it in the set of files to be released.
41		-* Update release date in `manual/qpdf-manual.xml`. Remember to ensure that the entities at the top of the document are consistent with the release notes for both version and release date.
42		-* Check `TODO` file to make sure all planned items for the release are done or retargeted.
43		-* Each year, update copyright notices. Just do a case-insensitive search for copyright. Don't forget copyright in manual. Also update debian copyright in debian package. Last updated: 2018.
44		-* To construct a source distribution from a pristine checkout, `make_dist` does the following:
45		- ```
46		- ./autogen.sh
47		- ./configure --enable-doc-maintenance --enable-werror
48		- make build_manual
49		- make distclean
50		- ```
51		-* To create a source release, do an export from the version control system to a directory called qpdf-version. For example, from this directory:
52		- ```
53		- rm -rf /tmp/qpdf-x.y.z
54		- git archive --prefix=qpdf-x.y.z/ HEAD . \| (cd /tmp; tar xf -)
55		- ```
56		- From the parent of that directory, run `qpdf-x.y.z/make_dist`. Remember to have fop in your path. For internally testing releases, you can run make_dist with the `--no-tests` option.
57		-* To create a source release of external libs, do an export from the version control system into a directory called `qpdf-external-libs` and just make a zip file of the result called `qpdf-external-libs-src.zip`. See the README.txt file there for information on creating binary external libs releases. Run this from the external-libs repository:
58		- ```
59		- git archive --prefix=external-libs/ HEAD . \| (cd /tmp; tar xf -)
60		- cd /tmp
61		- zip -r qpdf-external-libs-src.zip external-libs
62		- ```
63		-* To create Windows binary releases, extract the qpdf source distribution in Windows (MSYS2 + MSVC). From the extracted directory, extract the binary distribution of the external libraries. Run ./make_windows_releases simultaneously in 32-bit and 64-windows from there. It may be necessary to disable antivirus software first.
64		-* Before releasing, rebuild and test debian package.
65		-* Remember to copy `README-what-to-download.md` separately onto the download area.
66		-* Remember to update the web page including putting new documentation in the `files` subdirectory of the website on sourceforge.net.
67		-* Create a tag in the version control system, and make backups of the actual releases. With git, use git `tag -s` to create a signed tag:
68		- ```
69		- git tag -s release-qpdf-$version HEAD -m"qpdf $version"
70		- ```
71		-* When releasing on sourceforge, `external-libs` distributions go in `external-libs/yyyymmdd`, and qpdf distributions go in `qpdf/vvv`. Make the source package the default for all but Windows, and make the 32-bit mingw build the default for Windows.
72		-* Prepare and archive all release files. Files should be the source package, Windows binaries, AppImage, checksum files, and signature files.
73		-* Create a github release after pushing the tag. `gcurl` is an alias that includes the auth token.
74		- ```
75		- url=$(gcurl -s -XPOST https://api.github.com/repos/qpdf/qpdf/releases -d'{"tag_name": "release-qpdf-$version", "name": "qpdf $version", "draft": true, "body": "test body text"}' \| jq -r '.url')
76		- ```
77		-* Upload files to sourceforge and github. Publish the github release. Release notes can be added to the github release manually. Publish a news item manually on sourceforge. Email the qpdf-announce list.
78		-
79		-# General Build Stuff
80		-
81		-QPDF uses autoconf and libtool but does not use automake. The only files distributed with the qpdf source distribution that are not controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`, `aclocal.m4`, and some documentation. See above for the steps required to prepare a source distribution.
82		-
83		-A small handful of additional files have been taken from autotools programs. These should probably be updated from time to time.
84		-* `config.guess`, `config.sub`, `ltmain.sh`, and the `m4` directory: these were created by running `libtoolize -c`. To update, run `libtoolize -f -c` or remove the files and rerun `libtoolize`.
85		-* Other files copied as indicated:
86		- ```
87		- cp /usr/share/automake-1.11/install-sh .
88		- cp /usr/share/automake-1.11/mkinstalldirs .
89		- ```
90		-
91		-The entire contents of the `m4` directory came from `libtool.m4`. If we had some additional local parts, we could also add those to the `m4` directory. In order for this to work, it is necessary to run `aclocal -I m4` before running `autoheader` and `autoconf`. The `autogen.sh` script handles this.
92		-
93		-If building or editing documentation, configure with `--enable-doc-maintenance`. This will ensure that all tools or files required to validate and build documentation are available.
94		-
95		-If you want to run `make maintainer-clean`, `make distclean`, or `make autofiles.zip` and you haven't run `./configure`, you can pass `CLEAN=1` to make on the command line to prevent it from complaining about configure not having been run.
96		-
97		-If you want to run checks without rerunning the build, pass `NO_REBUILD=1` to make. This can be useful for special testing scenarios such as validation of memory fixes or binary compatibility.
98		-
99		-# Local Windows Testing Procedure
	98	+
	99	+* Update release notes in manual. Look at diffs and ChangeLog. Update
	100	+ release date in `manual/qpdf-manual.xml`. Remember to ensure that
	101	+ the entities at the top of the document are consistent with the
	102	+ release notes for both version and release date.
	103	+
	104	+* Add a release entry to ChangeLog.
	105	+
	106	+
	107	+CREATING A RELEASE
	108	+
	109	+* Create source release:
	110	+
	111	+version=x.y.z
	112	+rm -rf /tmp/qpdf-$version
	113	+git archive --prefix=qpdf-$version/ HEAD . \| (cd /tmp; tar xf -)
	114	+cd /tmp
	115	+./qpdf-$version/make_dist
	116	+gpg --detach-sign --armor qpdf-$version.tar.gz
	117	+
	118	+ Move qpdf-$version.tar.gz and qpdf-$version.tar.gz.asc to the
	119	+ release archive area.
	120	+
	121	+ For iterating on the release during testing, pass `--no-tests` to
	122	+ make_dist to skip the test suite.
	123	+
	124	+* Generate a signed AppImage using the docker image in appimage.
	125	+ Arguments to the docker container are arguments to git clone. The
	126	+ build should be made off the exact commit that will be officially
	127	+ tagged as the release but built prior to tagging the release.
	128	+ Example:
	129	+
	130	+cd appimage
	131	+docker build -t qpdfbuild .
	132	+rm -rf /tmp/build
	133	+mkdir -p /tmp/build
	134	+cp -rLp ~/.gnupg/. /tmp/build/.gnupg
	135	+docker run --privileged -ti --rm -v /tmp/build:/tmp/build qpdfbuild https://github.com/jberkenbilt/qpdf -b work
	136	+
	137	+ Rename the AppImage in /tmp/build/qpdf/appimage/build
	138	+ qpdf-$version-x86_64.AppImage and move it to the release archive
	139	+ area.
	140	+
	141	+* Create Windows binary releases. In Windows:
	142	+ * Extract the source distribution
	143	+ * From there, unzip the binary distribution of the external libraries
	144	+ * Open windows for 32-bit and 64-bit compilation environments that
	145	+ support msvc and mingw
	146	+ * Disable antivirus software. For bitdefender, open, select the "B"
	147	+ shield, and go to "View Features". Disable advanced threat defense
	148	+ and vulnerability scanning.
	149	+ * In each window simultaneously, run ./make_windows_releases
	150	+ * Copy the four resulting zip files into the release archive area
	151	+ * Re-enable anti-virus software
	152	+
	153	+* Build and test the debian package
	154	+
	155	+* Sign the releases. The release archive area should contain the
	156	+ Windows binaries, the AppImage, the source tarball, and the source
	157	+ tarball signature.
	158	+
	159	+\rm -f *.{md5,sha1,sha512}
	160	+files=(*)
	161	+for i in md5 sha1 sha512; do
	162	+ ${i}sum $files >\| qpdf-$version.$i
	163	+ gpg --clearsign --armor qpdf-$version.$i
	164	+ mv qpdf-$version.$i.asc qpdf-$version.$i
	165	+done
	166	+
	167	+* When creating releases on github and sourceforge, remember to copy
	168	+ `README-what-to-download.md` separately onto the download area if
	169	+ needed.
	170	+
	171	+* Create a github release after pushing the tag. `gcurl` is an alias
	172	+ that includes the auth token.
	173	+
	174	+# Create release
	175	+url=$(gcurl -s -XPOST https://api.github.com/repos/qpdf/qpdf/releases -d'{"tag_name": "release-qpdf-'$version'", "name": "qpdf '$version'", "draft": true}' \| jq -r '.url')
	176	+
	177	+# Get upload url
	178	+upload_url=$(gcurl -s $url \| jq -r '.upload_url' \| sed -E -e 's/\{.*\}//')
	179	+echo $upload_url
	180	+
	181	+# Upload all the files. You can add a label attribute too, which
	182	+# overrides the name.
	183	+for i in *; do
	184	+ mime=$(file -b --mime-type $i)
	185	+ gcurl -H "Content-Type: $mime" --data-binary @$i "$upload_url?name=$i"
	186	+done
	187	+
	188	+If needed, go onto github and make any manual updates such as
	189	+indicating a pre-release, adding release notes, etc.
	190	+
	191	+# Publish release
	192	+gcurl -XPOST $url -d'{"draft": false}'
	193	+
	194	+* Upload files to sourceforge. Make the source package the default for
	195	+ all but Windows, and make the 32-bit mingw build the default for
	196	+ Windows. Publish a news item manually on sourceforge.
	197	+
	198	+* Update the web page to indicate the new version and to put the new
	199	+ documentation in the `files` subdirectory of the website on
	200	+ sourceforge.net.
	201	+
	202	+* Email the qpdf-announce list.
	203	+
	204	+
	205	+OTHER NOTES
	206	+
	207	+To construct a source distribution from a pristine checkout,
	208	+`make_dist` does the following:
	209	+
	210	+./autogen.sh
	211	+./configure --enable-doc-maintenance --enable-werror
	212	+make build_manual
	213	+make distclean
	214	+
	215	+To create a source release of external libs, do an export from the
	216	+version control system into a directory called `qpdf-external-libs`
	217	+and just make a zip file of the result called
	218	+`qpdf-external-libs-src.zip`. See the README.txt file there for
	219	+information on creating binary external libs releases. Run this from
	220	+the external-libs repository:
	221	+
	222	+git archive --prefix=external-libs/ HEAD . \| (cd /tmp; tar xf -)
	223	+cd /tmp
	224	+zip -r qpdf-external-libs-src.zip external-libs
	225	+
	226	+When releasing on sourceforge, `external-libs` distributions go in
	227	+`external-libs/yyyymmdd`, and qpdf distributions go in `qpdf/vvv`.
	228	+
	229	+
	230	+GENERAL BUILD STUFF
	231	+
	232	+QPDF uses autoconf and libtool but does not use automake. The only
	233	+files distributed with the qpdf source distribution that are not
	234	+controlled are `configure`, `libqpdf/qpdf/qpdf-config.h.in`,
	235	+`aclocal.m4`, and some documentation. See above for the steps required
	236	+to prepare a source distribution.
	237	+
	238	+If building or editing documentation, configure with
	239	+`--enable-doc-maintenance`. This will ensure that all tools or files
	240	+required to validate and build documentation are available.
	241	+
	242	+If you want to run `make maintainer-clean`, `make distclean`, or `make
	243	+autofiles.zip` and you haven't run `./configure`, you can pass
	244	+`CLEAN=1` to make on the command line to prevent it from complaining
	245	+about configure not having been run.
	246	+
	247	+If you want to run checks without rerunning the build, pass
	248	+`NO_REBUILD=1` to make. This can be useful for special testing
	249	+scenarios such as validation of memory fixes or binary compatibility.
	250	+
	251	+
	252	+LOCAL WINDOWS TESTING PROCEDURE
100	253
101	254	This is what I do for routine testing on Windows.
102	255
103	256	From Linux, run `./autogen.sh` and `make autofiles.zip CLEAN=1`.
104	257
105		-From Windows, git clone from my Linux clone, unzip `external-libs`, and unzip `autofiles.zip`.
	258	+From Windows, git clone from my Linux clone, unzip `external-libs`,
	259	+and unzip `autofiles.zip`.
106	260
107		-Look at `make_windows_releases`. Set up path the same way and run whichever `./config-*` is appropriate for whichever compiler I need to test with. Start one of the Visual Studio native compiler shells, and from there, run one of the msys shells. The Visual Studio step is not necessary if just building with mingw.
	261	+Look at `make_windows_releases`. Set up path the same way and run
	262	+whichever `./config-*` is appropriate for whichever compiler I need to
	263	+test with. Start one of the Visual Studio native compiler shells, and
	264	+from there, run one of the msys shells. The Visual Studio step is not
	265	+necessary if just building with mingw.
...	...