Commit 19f54d981585d05285da808442aba4dd6fdbf0f2

Authored by m-holger
1 parent 35a79c63

In README-maintainer.md add mark-up for headers and code blocks

Showing 1 changed file with 80 additions and 61 deletions
README-maintainer.md
1   -ROUTINE DEVELOPMENT
  1 +## ROUTINE DEVELOPMENT
2 2  
3 3 **Remember to check pull requests as well as issues in github.**
4 4  
5 5 Default:
6 6  
  7 +```
7 8 cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
8 9 -DMAINTAINER_MODE=1 -DBUILD_STATIC_LIBS=0 \
9 10 -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
  11 +```
10 12  
11 13 Debugging:
12 14  
  15 +```
13 16 cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
14 17 -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
15 18 -DCMAKE_BUILD_TYPE=Debug ..
  19 +```
16 20  
17 21 Profiling:
18 22  
  23 +```
19 24 CFLAGS=-pg LDFLAGS=-pg \
20 25 cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
21 26 -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
22 27 -DCMAKE_BUILD_TYPE=Debug ..
  28 +```
23 29  
24 30 Then run `gprof gmon.out`. Note that gmon.out is not cumulative.
25 31  
26 32 Memory checks:
27 33  
  34 +```
28 35 CFLAGS="-fsanitize=address -fsanitize=undefined" \
29 36 CXXFLAGS="-fsanitize=address -fsanitize=undefined" \
30 37 LDFLAGS="-fsanitize=address -fsanitize=undefined" \
... ... @@ -32,21 +39,23 @@ CFLAGS="-fsanitize=address -fsanitize=undefined" \
32 39 cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
33 40 -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \
34 41 -DCMAKE_BUILD_TYPE=Debug ..
  42 +```
35 43  
36 44 Windows:
37 45  
  46 +```
38 47 ../cmake-win {mingw|msvc} maint
  48 +```
39 49  
40 50 See ./build-scripts for other ways to run the build for different
41 51 configurations.
42 52  
43   -
44   -VERSIONS
  53 +## VERSIONS
45 54  
46 55 * The version number on the main branch is whatever the version would
47 56 be if the top of the branch were released. If the most recent
48 57 release is version a.b.c, then
49   -
  58 +
50 59 * If there are any ABI-breaking changes since the last release,
51 60 main's version is a+1.0.0
52 61 * Else if there is any new public API, main's version is a.b+1.0
... ... @@ -65,15 +74,13 @@ VERSIONS
65 74 there or the changes can be merged back, depending on the amount of
66 75 drift.
67 76  
68   -
69   -CHECKING DOCS ON readthedocs
  77 +## CHECKING DOCS ON readthedocs
70 78  
71 79 To check docs on readthedocs.io without running all of CI, push to the
72 80 doc-check branch. Then visit https://qpdf.readthedocs.io/en/doc-check/
73 81 Building docs from pull requests is also enabled.
74 82  
75   -
76   -GOOGLE OSS-FUZZ
  83 +## GOOGLE OSS-FUZZ
77 84  
78 85 * See ../misc/fuzz (not in repo) for unfixed, downloaded fuzz test cases
79 86  
... ... @@ -87,26 +94,28 @@ GOOGLE OSS-FUZZ
87 94  
88 95 * To test locally, see https://github.com/google/oss-fuzz/tree/master/docs/,
89 96 especially new_project_guide.md. Summary:
90   -
  97 +
91 98 Clone the oss-fuzz project. From the root directory of the repository:
92   -
  99 +
  100 + ```
93 101 python3 infra/helper.py build_image --pull qpdf
94 102 python3 infra/helper.py build_fuzzers [ --sanitizer memory|undefined|address ] qpdf [path-to-qpdf-source]
95 103 python3 infra/helper.py check_build qpdf
96 104 python3 infra/helper.py build_fuzzers --sanitizer coverage qpdf
97 105 python3 infra/helper.py coverage qpdf
98   -
  106 + ```
  107 +
99 108 To reproduce a test case, build with the correct sanitizer, then run
100   -
  109 +
101 110 python3 infra/helper.py reproduce qpdf <specific-fuzzer> testcase
102   -
  111 +
103 112 where fuzzer is the fuzzer used in the crash.
104   -
  113 +
105 114 The fuzzer is in build/out/qpdf. It can be run with a directory as
106 115 an argument to run against files in a directory. You can use
107   -
  116 +
108 117 qpdf_fuzzer -merge=1 cur new >& /dev/null&
109   -
  118 +
110 119 to add any files from new into cur if they increase coverage. You
111 120 need to do this with the coverage build (the one with
112 121 --sanitizer coverage)
... ... @@ -120,8 +129,7 @@ GOOGLE OSS-FUZZ
120 129 * Latest corpus:
121 130 gs://qpdf-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/qpdf_fuzzer/latest.zip
122 131  
123   -
124   -CODING RULES
  132 +## CODING RULES
125 133  
126 134 * Code is formatted with clang-format-16. See .clang-format and the
127 135 "Code Formatting" section in manual/contributing.rst for details.
... ... @@ -130,18 +138,18 @@ CODING RULES
130 138 * Use std::to_string instead of QUtil::int_to_string et al
131 139  
132 140 * Use of assert:
133   -
  141 +
134 142 * Test code: #include <qpdf/assert_test.h> first.
135 143 * Debug code: #include <qpdf/assert_debug.h> first and use
136 144 qpdf_assert_debug instead of assert.
137   -
  145 +
138 146 These rules are enforced by the check-assert test. This practices
139 147 serves to
140   -
  148 +
141 149 * remind us that assert in release code disappears and so should only
142 150 be used for debugging; when doing so use a Debug build
143 151 configuration
144   -
  152 +
145 153 * protect us from using assert in test code without explicitly
146 154 removing the NDEBUG definition, since that would cause the assert
147 155 not to actually be testing anything in non-Debug build
... ... @@ -190,7 +198,7 @@ CODING RULES
190 198 dynamic_cast with those, but doing it anyway may help with some
191 199 strange cases for mingw or with some code generators that may
192 200 systematically do this for other reasons.
193   -
  201 +
194 202 IMPORTANT NOTE ABOUT QPDF_DLL_CLASS: On mingw, the vtable for a
195 203 class with some virtual methods and no pure virtual methods seems
196 204 often (always?) not to be generated if the destructor is inline or
... ... @@ -199,9 +207,11 @@ CODING RULES
199 207 methods, you must declare the destructor in the header without
200 208 `= default` and provide a non-inline implementation in the source
201 209 file. Add this comment to the implementation:
202   -
  210 +
  211 + ```cpp
203 212 // Must be explicit and not inline -- see QPDF_DLL_CLASS in
204 213 // README-maintainer
  214 + ```
205 215  
206 216 * Put private member variables in std::shared_ptr<Members> for all
207 217 public classes. Remember to use QPDF_DLL on ~Members(). Exception:
... ... @@ -220,8 +230,7 @@ CODING RULES
220 230 * Avoid attaching too much metadata to objects and object handles
221 231 since those have to get copied around a lot.
222 232  
223   -
224   -HOW TO ADD A COMMAND-LINE ARGUMENT
  233 +## HOW TO ADD A COMMAND-LINE ARGUMENT
225 234  
226 235 Quick reminder:
227 236  
... ... @@ -278,18 +287,17 @@ When done, the following should happen:
278 287 * The job JSON file should have a new key in the schema corresponding
279 288 to the new option
280 289  
281   -
282   -RELEASE PREPARATION
  290 +## RELEASE PREPARATION
283 291  
284 292 * Each year, update copyright notices. This will find all relevant
285 293 places (assuming current copyright is from last year):
286   -
  294 +
287 295 git --no-pager grep -i -n -P "copyright.*$(expr $(date +%Y) - 1).*berkenbilt"
288   -
  296 +
289 297 Also update the copyright in these places:
290 298 * debian package -- search for copyright.*berkenbilt in debian/copyright
291 299 * qtest-driver, TestDriver.pm in qtest source
292   -
  300 +
293 301 Copyright last updated: 2023.
294 302  
295 303 * Take a look at "External Libraries" in TODO to see if we need to
... ... @@ -311,24 +319,24 @@ RELEASE PREPARATION
311 319 * Check work `qpdf` project for private issues
312 320  
313 321 * Make sure the code is formatted.
314   -
  322 +
315 323 ./format-code
316 324  
317 325 * Run a spelling checker over the source code to catch errors in
318 326 variable names, strings, and comments.
319   -
  327 +
320 328 ./spell-check
321   -
  329 +
322 330 This uses cspell. Install with `npm install -g cspell`. The output
323 331 of cspell is suitable for use with `M-x grep` in emacs. Add
324 332 exceptions to cSpell.json.
325 333  
326 334 * If needed, run large file and image comparison tests by setting
327 335 these environment variables:
328   -
  336 +
329 337 QPDF_LARGE_FILE_TEST_PATH=/full/path
330 338 QPDF_TEST_COMPARE_IMAGES=1
331   -
  339 +
332 340 For Windows, use a Windows style path, not an MSYS path for large files.
333 341  
334 342 * If any interfaces were added or changed, check C API to see whether
... ... @@ -340,12 +348,12 @@ RELEASE PREPARATION
340 348  
341 349 * Double check versions and shared library details. They should
342 350 already be up to date in the code.
343   -
  351 +
344 352 * Make sure version numbers are consistent in the following locations:
345 353 * CMakeLists.txt
346 354 * include/qpdf/DLL.h
347 355 * manual/conf.py
348   -
  356 +
349 357 `make_dist` verifies this consistency, and CI fails if they are
350 358 inconsistent.
351 359  
... ... @@ -362,12 +370,12 @@ RELEASE PREPARATION
362 370 testing, do performance testing.
363 371  
364 372 * Test for performance and binary compatibility:
365   -
  373 +
366 374 ./abi-perf-test v<old> @
367   -
  375 +
368 376 Prefix with SKIP_PERF=1 to skip performance test.
369 377 Prefix with SKIP_TESTS=1 to skip test suite run.
370   -
  378 +
371 379 See "ABI checks" for details about the process.
372 380 End state:
373 381 * /tmp/check-abi/perf contains the performance comparison
... ... @@ -389,8 +397,7 @@ env PKG_CONFIG_PATH=/tmp/inst/lib/pkgconfig \
389 397 CMAKE_PREFIX_PATH=/tmp/inst \
390 398 ./pkg-test/run-all
391 399  
392   -
393   -CREATING A RELEASE
  400 +## CREATING A RELEASE
394 401  
395 402 * Push to main. This will create an artifact called distribution
396 403 which will contain all the distribution files. Download these,
... ... @@ -442,7 +449,9 @@ git push qpdf @:stable
442 449 * Create a github release after pushing the tag. `gcurl` is an alias
443 450 that includes the auth token.
444 451  
  452 +```
445 453 # Create release
  454 +
446 455 GITHUB_TOKEN=$(qdata-show cred github-token)
447 456 function gcurl() { curl -H "Authorization: token $GITHUB_TOKEN" ${1+"$@"}; }
448 457  
... ... @@ -458,6 +467,7 @@ for i in *; do
458 467 mime=$(file -b --mime-type $i)
459 468 gcurl -H "Content-Type: $mime" --data-binary @$i "$upload_url?name=$i"
460 469 done
  470 +```
461 471  
462 472 If needed, go onto github and make any manual updates such as
463 473 indicating a pre-release, adding release notes, etc.
... ... @@ -470,8 +480,10 @@ This is qpdf version x.y.z. (Brief description)
470 480 For a full list of changes from previous releases, please see the [release notes](https://qpdf.readthedocs.io/en/stable/release-notes.html). See also [README-what-to-download](./README-what-to-download.md) for details about the available source and binary distributions.
471 481 ```
472 482  
  483 +```
473 484 # Publish release
474 485 gcurl -XPOST $url -d'{"draft": false}'
  486 +```
475 487  
476 488 * Upload files to sourceforge.
477 489  
... ... @@ -486,14 +498,14 @@ rsync -vrlcO ./ jay_berkenbilt,qpdf@frs.sourceforge.net:/home/frs/project/q/qp/q
486 498  
487 499 * Email the qpdf-announce list.
488 500  
489   -
490   -RUNNING pikepdf's TEST SUITE
  501 +## RUNNING pikepdf's TEST SUITE
491 502  
492 503 We run pikepdf's test suite from CI. These instructions show how to do
493 504 it manually.
494 505  
495 506 Do this in a separate shell.
496 507  
  508 +```
497 509 cd ...qpdf-source-tree...
498 510 export QPDF_SOURCE_TREE=$PWD
499 511 export QPDF_BUILD_LIBDIR=$QPDF_SOURCE_TREE/build/libqpdf
... ... @@ -510,10 +522,12 @@ python3 -m pip install &#39;.[test]&#39;
510 522 rehash
511 523 python3 -m pip install .
512 524 pytest -n auto
  525 +```
513 526  
514 527 If there are failures, use git bisect to figure out where the failure
515 528 was introduced. For example, set up a work area like this:
516 529  
  530 +```
517 531 rm -rf /tmp/z
518 532 mkdir /tmp/z
519 533 cd /tmp/z
... ... @@ -541,12 +555,12 @@ python3 -m pip install .
541 555 pytest -n auto
542 556 EOF
543 557 chmod +x /tmp/check
  558 +```
544 559  
545 560 Then in /tmp/z/qpdf, run git bisect. Use /tmp/check at each stage to
546 561 test whether it's a good or bad commit.
547 562  
548   -
549   -OTHER NOTES
  563 +## OTHER NOTES
550 564  
551 565 For local iteration on the AppImage generation, it works to just
552 566 ./build-scripts/build-appimage and get the resulting AppImage from the
... ... @@ -558,9 +572,11 @@ Use -ti -e RUN_SHELL=1 to run a shell instead of the build script.
558 572  
559 573 To iterate on the scripts directly in the source tree, you can run
560 574  
  575 +```
561 576 docker build -t qpdfbuild appimage
562 577 docker run --privileged --rm -ti -e SKIP_TESTS=1 -e RUN_SHELL=1 \
563 578 -v $PWD/..:/tmp/build ${1+"$@"} qpdfbuild
  579 +```
564 580  
565 581 This will put you at a shell prompt inside the container with your
566 582 current directory set to the top of the source tree and your uid equal
... ... @@ -569,12 +585,13 @@ to the owner of the parent directory source tree.
569 585 Note: this will leave some extra files (like .bash_history) in the
570 586 parent directory of the source tree. You will want to clean those up.
571 587  
572   -DEPRECATION
  588 +## DEPRECATION
573 589  
574 590 This is a reminder of how to use and test deprecation.
575 591  
576 592 To temporarily disable deprecation warnings for testing:
577 593  
  594 +```cpp
578 595 #ifdef _MSC_VER
579 596 # pragma warning(disable : 4996)
580 597 #endif
... ... @@ -586,13 +603,15 @@ To temporarily disable deprecation warnings for testing:
586 603 #if (defined(__GNUC__) || defined(__clang__))
587 604 # pragma GCC diagnostic pop
588 605 #endif
  606 +```
589 607  
590 608 To declare something as deprecated:
591 609  
  610 +```cpp
592 611 [[deprecated("explanation")]]
  612 +```
593 613  
594   -
595   -LOCAL WINDOWS TESTING PROCEDURE
  614 +## LOCAL WINDOWS TESTING PROCEDURE
596 615  
597 616 This is what I do for routine testing on Windows.
598 617  
... ... @@ -612,8 +631,7 @@ This is what I do for routine testing on Windows.
612 631  
613 632 * Test with mingw: `ctest --verbose -C RelWithDebInfo`
614 633  
615   -
616   -DOCS ON readthedocs.org
  634 +### DOCS ON readthedocs.org
617 635  
618 636 * Registered for an account at readthedocs.org with my github account
619 637 * Project page: https://readthedocs.org/projects/qpdf/
... ... @@ -640,8 +658,7 @@ following branching strategy to support docs:
640 658 The release process includes updating the approach branches and
641 659 activating versions.
642 660  
643   -
644   -CMAKE notes
  661 +## CMAKE notes
645 662  
646 663 To verify the various cmake options and their interactions, several
647 664 manual tests were done:
... ... @@ -662,8 +679,7 @@ We are using RelWithDebInfo for mingw and other non-Windows builds but
662 679 Release for MSVC. There are linker warnings if MSVC is built with
663 680 RelWithDebInfo when using external-libs.
664 681  
665   -
666   -ABI checks
  682 +## ABI checks
667 683  
668 684 Until the conversion of the build to cmake, we relied on running the
669 685 test suite with old executables and a new library. When QPDFJob was
... ... @@ -698,8 +714,7 @@ steps. See comments in check_abi for additional notes. Running
698 714 "check_abi check-sizes" is run by ctest on Linux when CHECK_SIZES is
699 715 on.
700 716  
701   -
702   -CODE FORMATTING
  717 +## CODE FORMATTING
703 718  
704 719 * Emacs doesn't indent breaking strings concatenated with + over
705 720 lines but clang-format does. It's clearer with clang-format. To
... ... @@ -707,18 +722,22 @@ CODE FORMATTING
707 722 that builds the concatenated string.
708 723  
709 724 * With
710   -
  725 +
  726 + ```cpp
711 727 long_function(long_function(
712 728 args)
713 729  
  730 + ```
  731 +
714 732 clang-format anchors relative to the first function, and emacs
715 733 anchors relative to the second function. Use
716   -
  734 +
  735 + ```cpp
717 736 long_function(
718 737 // line-break
719 738 long_function(
720 739 args)
721   -
  740 + ```
722 741 to resolve.
723 742  
724 743 In the revision control history, there is a commit around April 3,
... ...