Commit 19f54d981585d05285da808442aba4dd6fdbf0f2
1 parent
35a79c63
In README-maintainer.md add mark-up for headers and code blocks
Showing
1 changed file
with
80 additions
and
61 deletions
README-maintainer.md
| 1 | -ROUTINE DEVELOPMENT | |
| 1 | +## ROUTINE DEVELOPMENT | |
| 2 | 2 | |
| 3 | 3 | **Remember to check pull requests as well as issues in github.** |
| 4 | 4 | |
| 5 | 5 | Default: |
| 6 | 6 | |
| 7 | +``` | |
| 7 | 8 | cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \ |
| 8 | 9 | -DMAINTAINER_MODE=1 -DBUILD_STATIC_LIBS=0 \ |
| 9 | 10 | -DCMAKE_BUILD_TYPE=RelWithDebInfo .. |
| 11 | +``` | |
| 10 | 12 | |
| 11 | 13 | Debugging: |
| 12 | 14 | |
| 15 | +``` | |
| 13 | 16 | cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \ |
| 14 | 17 | -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \ |
| 15 | 18 | -DCMAKE_BUILD_TYPE=Debug .. |
| 19 | +``` | |
| 16 | 20 | |
| 17 | 21 | Profiling: |
| 18 | 22 | |
| 23 | +``` | |
| 19 | 24 | CFLAGS=-pg LDFLAGS=-pg \ |
| 20 | 25 | cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \ |
| 21 | 26 | -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \ |
| 22 | 27 | -DCMAKE_BUILD_TYPE=Debug .. |
| 28 | +``` | |
| 23 | 29 | |
| 24 | 30 | Then run `gprof gmon.out`. Note that gmon.out is not cumulative. |
| 25 | 31 | |
| 26 | 32 | Memory checks: |
| 27 | 33 | |
| 34 | +``` | |
| 28 | 35 | CFLAGS="-fsanitize=address -fsanitize=undefined" \ |
| 29 | 36 | CXXFLAGS="-fsanitize=address -fsanitize=undefined" \ |
| 30 | 37 | LDFLAGS="-fsanitize=address -fsanitize=undefined" \ |
| ... | ... | @@ -32,21 +39,23 @@ CFLAGS="-fsanitize=address -fsanitize=undefined" \ |
| 32 | 39 | cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \ |
| 33 | 40 | -DMAINTAINER_MODE=1 -DBUILD_SHARED_LIBS=0 \ |
| 34 | 41 | -DCMAKE_BUILD_TYPE=Debug .. |
| 42 | +``` | |
| 35 | 43 | |
| 36 | 44 | Windows: |
| 37 | 45 | |
| 46 | +``` | |
| 38 | 47 | ../cmake-win {mingw|msvc} maint |
| 48 | +``` | |
| 39 | 49 | |
| 40 | 50 | See ./build-scripts for other ways to run the build for different |
| 41 | 51 | configurations. |
| 42 | 52 | |
| 43 | - | |
| 44 | -VERSIONS | |
| 53 | +## VERSIONS | |
| 45 | 54 | |
| 46 | 55 | * The version number on the main branch is whatever the version would |
| 47 | 56 | be if the top of the branch were released. If the most recent |
| 48 | 57 | release is version a.b.c, then |
| 49 | - | |
| 58 | + | |
| 50 | 59 | * If there are any ABI-breaking changes since the last release, |
| 51 | 60 | main's version is a+1.0.0 |
| 52 | 61 | * Else if there is any new public API, main's version is a.b+1.0 |
| ... | ... | @@ -65,15 +74,13 @@ VERSIONS |
| 65 | 74 | there or the changes can be merged back, depending on the amount of |
| 66 | 75 | drift. |
| 67 | 76 | |
| 68 | - | |
| 69 | -CHECKING DOCS ON readthedocs | |
| 77 | +## CHECKING DOCS ON readthedocs | |
| 70 | 78 | |
| 71 | 79 | To check docs on readthedocs.io without running all of CI, push to the |
| 72 | 80 | doc-check branch. Then visit https://qpdf.readthedocs.io/en/doc-check/ |
| 73 | 81 | Building docs from pull requests is also enabled. |
| 74 | 82 | |
| 75 | - | |
| 76 | -GOOGLE OSS-FUZZ | |
| 83 | +## GOOGLE OSS-FUZZ | |
| 77 | 84 | |
| 78 | 85 | * See ../misc/fuzz (not in repo) for unfixed, downloaded fuzz test cases |
| 79 | 86 | |
| ... | ... | @@ -87,26 +94,28 @@ GOOGLE OSS-FUZZ |
| 87 | 94 | |
| 88 | 95 | * To test locally, see https://github.com/google/oss-fuzz/tree/master/docs/, |
| 89 | 96 | especially new_project_guide.md. Summary: |
| 90 | - | |
| 97 | + | |
| 91 | 98 | Clone the oss-fuzz project. From the root directory of the repository: |
| 92 | - | |
| 99 | + | |
| 100 | + ``` | |
| 93 | 101 | python3 infra/helper.py build_image --pull qpdf |
| 94 | 102 | python3 infra/helper.py build_fuzzers [ --sanitizer memory|undefined|address ] qpdf [path-to-qpdf-source] |
| 95 | 103 | python3 infra/helper.py check_build qpdf |
| 96 | 104 | python3 infra/helper.py build_fuzzers --sanitizer coverage qpdf |
| 97 | 105 | python3 infra/helper.py coverage qpdf |
| 98 | - | |
| 106 | + ``` | |
| 107 | + | |
| 99 | 108 | To reproduce a test case, build with the correct sanitizer, then run |
| 100 | - | |
| 109 | + | |
| 101 | 110 | python3 infra/helper.py reproduce qpdf <specific-fuzzer> testcase |
| 102 | - | |
| 111 | + | |
| 103 | 112 | where fuzzer is the fuzzer used in the crash. |
| 104 | - | |
| 113 | + | |
| 105 | 114 | The fuzzer is in build/out/qpdf. It can be run with a directory as |
| 106 | 115 | an argument to run against files in a directory. You can use |
| 107 | - | |
| 116 | + | |
| 108 | 117 | qpdf_fuzzer -merge=1 cur new >& /dev/null& |
| 109 | - | |
| 118 | + | |
| 110 | 119 | to add any files from new into cur if they increase coverage. You |
| 111 | 120 | need to do this with the coverage build (the one with |
| 112 | 121 | --sanitizer coverage) |
| ... | ... | @@ -120,8 +129,7 @@ GOOGLE OSS-FUZZ |
| 120 | 129 | * Latest corpus: |
| 121 | 130 | gs://qpdf-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/qpdf_fuzzer/latest.zip |
| 122 | 131 | |
| 123 | - | |
| 124 | -CODING RULES | |
| 132 | +## CODING RULES | |
| 125 | 133 | |
| 126 | 134 | * Code is formatted with clang-format-16. See .clang-format and the |
| 127 | 135 | "Code Formatting" section in manual/contributing.rst for details. |
| ... | ... | @@ -130,18 +138,18 @@ CODING RULES |
| 130 | 138 | * Use std::to_string instead of QUtil::int_to_string et al |
| 131 | 139 | |
| 132 | 140 | * Use of assert: |
| 133 | - | |
| 141 | + | |
| 134 | 142 | * Test code: #include <qpdf/assert_test.h> first. |
| 135 | 143 | * Debug code: #include <qpdf/assert_debug.h> first and use |
| 136 | 144 | qpdf_assert_debug instead of assert. |
| 137 | - | |
| 145 | + | |
| 138 | 146 | These rules are enforced by the check-assert test. This practices |
| 139 | 147 | serves to |
| 140 | - | |
| 148 | + | |
| 141 | 149 | * remind us that assert in release code disappears and so should only |
| 142 | 150 | be used for debugging; when doing so use a Debug build |
| 143 | 151 | configuration |
| 144 | - | |
| 152 | + | |
| 145 | 153 | * protect us from using assert in test code without explicitly |
| 146 | 154 | removing the NDEBUG definition, since that would cause the assert |
| 147 | 155 | not to actually be testing anything in non-Debug build |
| ... | ... | @@ -190,7 +198,7 @@ CODING RULES |
| 190 | 198 | dynamic_cast with those, but doing it anyway may help with some |
| 191 | 199 | strange cases for mingw or with some code generators that may |
| 192 | 200 | systematically do this for other reasons. |
| 193 | - | |
| 201 | + | |
| 194 | 202 | IMPORTANT NOTE ABOUT QPDF_DLL_CLASS: On mingw, the vtable for a |
| 195 | 203 | class with some virtual methods and no pure virtual methods seems |
| 196 | 204 | often (always?) not to be generated if the destructor is inline or |
| ... | ... | @@ -199,9 +207,11 @@ CODING RULES |
| 199 | 207 | methods, you must declare the destructor in the header without |
| 200 | 208 | `= default` and provide a non-inline implementation in the source |
| 201 | 209 | file. Add this comment to the implementation: |
| 202 | - | |
| 210 | + | |
| 211 | + ```cpp | |
| 203 | 212 | // Must be explicit and not inline -- see QPDF_DLL_CLASS in |
| 204 | 213 | // README-maintainer |
| 214 | + ``` | |
| 205 | 215 | |
| 206 | 216 | * Put private member variables in std::shared_ptr<Members> for all |
| 207 | 217 | public classes. Remember to use QPDF_DLL on ~Members(). Exception: |
| ... | ... | @@ -220,8 +230,7 @@ CODING RULES |
| 220 | 230 | * Avoid attaching too much metadata to objects and object handles |
| 221 | 231 | since those have to get copied around a lot. |
| 222 | 232 | |
| 223 | - | |
| 224 | -HOW TO ADD A COMMAND-LINE ARGUMENT | |
| 233 | +## HOW TO ADD A COMMAND-LINE ARGUMENT | |
| 225 | 234 | |
| 226 | 235 | Quick reminder: |
| 227 | 236 | |
| ... | ... | @@ -278,18 +287,17 @@ When done, the following should happen: |
| 278 | 287 | * The job JSON file should have a new key in the schema corresponding |
| 279 | 288 | to the new option |
| 280 | 289 | |
| 281 | - | |
| 282 | -RELEASE PREPARATION | |
| 290 | +## RELEASE PREPARATION | |
| 283 | 291 | |
| 284 | 292 | * Each year, update copyright notices. This will find all relevant |
| 285 | 293 | places (assuming current copyright is from last year): |
| 286 | - | |
| 294 | + | |
| 287 | 295 | git --no-pager grep -i -n -P "copyright.*$(expr $(date +%Y) - 1).*berkenbilt" |
| 288 | - | |
| 296 | + | |
| 289 | 297 | Also update the copyright in these places: |
| 290 | 298 | * debian package -- search for copyright.*berkenbilt in debian/copyright |
| 291 | 299 | * qtest-driver, TestDriver.pm in qtest source |
| 292 | - | |
| 300 | + | |
| 293 | 301 | Copyright last updated: 2023. |
| 294 | 302 | |
| 295 | 303 | * Take a look at "External Libraries" in TODO to see if we need to |
| ... | ... | @@ -311,24 +319,24 @@ RELEASE PREPARATION |
| 311 | 319 | * Check work `qpdf` project for private issues |
| 312 | 320 | |
| 313 | 321 | * Make sure the code is formatted. |
| 314 | - | |
| 322 | + | |
| 315 | 323 | ./format-code |
| 316 | 324 | |
| 317 | 325 | * Run a spelling checker over the source code to catch errors in |
| 318 | 326 | variable names, strings, and comments. |
| 319 | - | |
| 327 | + | |
| 320 | 328 | ./spell-check |
| 321 | - | |
| 329 | + | |
| 322 | 330 | This uses cspell. Install with `npm install -g cspell`. The output |
| 323 | 331 | of cspell is suitable for use with `M-x grep` in emacs. Add |
| 324 | 332 | exceptions to cSpell.json. |
| 325 | 333 | |
| 326 | 334 | * If needed, run large file and image comparison tests by setting |
| 327 | 335 | these environment variables: |
| 328 | - | |
| 336 | + | |
| 329 | 337 | QPDF_LARGE_FILE_TEST_PATH=/full/path |
| 330 | 338 | QPDF_TEST_COMPARE_IMAGES=1 |
| 331 | - | |
| 339 | + | |
| 332 | 340 | For Windows, use a Windows style path, not an MSYS path for large files. |
| 333 | 341 | |
| 334 | 342 | * If any interfaces were added or changed, check C API to see whether |
| ... | ... | @@ -340,12 +348,12 @@ RELEASE PREPARATION |
| 340 | 348 | |
| 341 | 349 | * Double check versions and shared library details. They should |
| 342 | 350 | already be up to date in the code. |
| 343 | - | |
| 351 | + | |
| 344 | 352 | * Make sure version numbers are consistent in the following locations: |
| 345 | 353 | * CMakeLists.txt |
| 346 | 354 | * include/qpdf/DLL.h |
| 347 | 355 | * manual/conf.py |
| 348 | - | |
| 356 | + | |
| 349 | 357 | `make_dist` verifies this consistency, and CI fails if they are |
| 350 | 358 | inconsistent. |
| 351 | 359 | |
| ... | ... | @@ -362,12 +370,12 @@ RELEASE PREPARATION |
| 362 | 370 | testing, do performance testing. |
| 363 | 371 | |
| 364 | 372 | * Test for performance and binary compatibility: |
| 365 | - | |
| 373 | + | |
| 366 | 374 | ./abi-perf-test v<old> @ |
| 367 | - | |
| 375 | + | |
| 368 | 376 | Prefix with SKIP_PERF=1 to skip performance test. |
| 369 | 377 | Prefix with SKIP_TESTS=1 to skip test suite run. |
| 370 | - | |
| 378 | + | |
| 371 | 379 | See "ABI checks" for details about the process. |
| 372 | 380 | End state: |
| 373 | 381 | * /tmp/check-abi/perf contains the performance comparison |
| ... | ... | @@ -389,8 +397,7 @@ env PKG_CONFIG_PATH=/tmp/inst/lib/pkgconfig \ |
| 389 | 397 | CMAKE_PREFIX_PATH=/tmp/inst \ |
| 390 | 398 | ./pkg-test/run-all |
| 391 | 399 | |
| 392 | - | |
| 393 | -CREATING A RELEASE | |
| 400 | +## CREATING A RELEASE | |
| 394 | 401 | |
| 395 | 402 | * Push to main. This will create an artifact called distribution |
| 396 | 403 | which will contain all the distribution files. Download these, |
| ... | ... | @@ -442,7 +449,9 @@ git push qpdf @:stable |
| 442 | 449 | * Create a github release after pushing the tag. `gcurl` is an alias |
| 443 | 450 | that includes the auth token. |
| 444 | 451 | |
| 452 | +``` | |
| 445 | 453 | # Create release |
| 454 | + | |
| 446 | 455 | GITHUB_TOKEN=$(qdata-show cred github-token) |
| 447 | 456 | function gcurl() { curl -H "Authorization: token $GITHUB_TOKEN" ${1+"$@"}; } |
| 448 | 457 | |
| ... | ... | @@ -458,6 +467,7 @@ for i in *; do |
| 458 | 467 | mime=$(file -b --mime-type $i) |
| 459 | 468 | gcurl -H "Content-Type: $mime" --data-binary @$i "$upload_url?name=$i" |
| 460 | 469 | done |
| 470 | +``` | |
| 461 | 471 | |
| 462 | 472 | If needed, go onto github and make any manual updates such as |
| 463 | 473 | indicating a pre-release, adding release notes, etc. |
| ... | ... | @@ -470,8 +480,10 @@ This is qpdf version x.y.z. (Brief description) |
| 470 | 480 | For a full list of changes from previous releases, please see the [release notes](https://qpdf.readthedocs.io/en/stable/release-notes.html). See also [README-what-to-download](./README-what-to-download.md) for details about the available source and binary distributions. |
| 471 | 481 | ``` |
| 472 | 482 | |
| 483 | +``` | |
| 473 | 484 | # Publish release |
| 474 | 485 | gcurl -XPOST $url -d'{"draft": false}' |
| 486 | +``` | |
| 475 | 487 | |
| 476 | 488 | * Upload files to sourceforge. |
| 477 | 489 | |
| ... | ... | @@ -486,14 +498,14 @@ rsync -vrlcO ./ jay_berkenbilt,qpdf@frs.sourceforge.net:/home/frs/project/q/qp/q |
| 486 | 498 | |
| 487 | 499 | * Email the qpdf-announce list. |
| 488 | 500 | |
| 489 | - | |
| 490 | -RUNNING pikepdf's TEST SUITE | |
| 501 | +## RUNNING pikepdf's TEST SUITE | |
| 491 | 502 | |
| 492 | 503 | We run pikepdf's test suite from CI. These instructions show how to do |
| 493 | 504 | it manually. |
| 494 | 505 | |
| 495 | 506 | Do this in a separate shell. |
| 496 | 507 | |
| 508 | +``` | |
| 497 | 509 | cd ...qpdf-source-tree... |
| 498 | 510 | export QPDF_SOURCE_TREE=$PWD |
| 499 | 511 | export QPDF_BUILD_LIBDIR=$QPDF_SOURCE_TREE/build/libqpdf |
| ... | ... | @@ -510,10 +522,12 @@ python3 -m pip install '.[test]' |
| 510 | 522 | rehash |
| 511 | 523 | python3 -m pip install . |
| 512 | 524 | pytest -n auto |
| 525 | +``` | |
| 513 | 526 | |
| 514 | 527 | If there are failures, use git bisect to figure out where the failure |
| 515 | 528 | was introduced. For example, set up a work area like this: |
| 516 | 529 | |
| 530 | +``` | |
| 517 | 531 | rm -rf /tmp/z |
| 518 | 532 | mkdir /tmp/z |
| 519 | 533 | cd /tmp/z |
| ... | ... | @@ -541,12 +555,12 @@ python3 -m pip install . |
| 541 | 555 | pytest -n auto |
| 542 | 556 | EOF |
| 543 | 557 | chmod +x /tmp/check |
| 558 | +``` | |
| 544 | 559 | |
| 545 | 560 | Then in /tmp/z/qpdf, run git bisect. Use /tmp/check at each stage to |
| 546 | 561 | test whether it's a good or bad commit. |
| 547 | 562 | |
| 548 | - | |
| 549 | -OTHER NOTES | |
| 563 | +## OTHER NOTES | |
| 550 | 564 | |
| 551 | 565 | For local iteration on the AppImage generation, it works to just |
| 552 | 566 | ./build-scripts/build-appimage and get the resulting AppImage from the |
| ... | ... | @@ -558,9 +572,11 @@ Use -ti -e RUN_SHELL=1 to run a shell instead of the build script. |
| 558 | 572 | |
| 559 | 573 | To iterate on the scripts directly in the source tree, you can run |
| 560 | 574 | |
| 575 | +``` | |
| 561 | 576 | docker build -t qpdfbuild appimage |
| 562 | 577 | docker run --privileged --rm -ti -e SKIP_TESTS=1 -e RUN_SHELL=1 \ |
| 563 | 578 | -v $PWD/..:/tmp/build ${1+"$@"} qpdfbuild |
| 579 | +``` | |
| 564 | 580 | |
| 565 | 581 | This will put you at a shell prompt inside the container with your |
| 566 | 582 | current directory set to the top of the source tree and your uid equal |
| ... | ... | @@ -569,12 +585,13 @@ to the owner of the parent directory source tree. |
| 569 | 585 | Note: this will leave some extra files (like .bash_history) in the |
| 570 | 586 | parent directory of the source tree. You will want to clean those up. |
| 571 | 587 | |
| 572 | -DEPRECATION | |
| 588 | +## DEPRECATION | |
| 573 | 589 | |
| 574 | 590 | This is a reminder of how to use and test deprecation. |
| 575 | 591 | |
| 576 | 592 | To temporarily disable deprecation warnings for testing: |
| 577 | 593 | |
| 594 | +```cpp | |
| 578 | 595 | #ifdef _MSC_VER |
| 579 | 596 | # pragma warning(disable : 4996) |
| 580 | 597 | #endif |
| ... | ... | @@ -586,13 +603,15 @@ To temporarily disable deprecation warnings for testing: |
| 586 | 603 | #if (defined(__GNUC__) || defined(__clang__)) |
| 587 | 604 | # pragma GCC diagnostic pop |
| 588 | 605 | #endif |
| 606 | +``` | |
| 589 | 607 | |
| 590 | 608 | To declare something as deprecated: |
| 591 | 609 | |
| 610 | +```cpp | |
| 592 | 611 | [[deprecated("explanation")]] |
| 612 | +``` | |
| 593 | 613 | |
| 594 | - | |
| 595 | -LOCAL WINDOWS TESTING PROCEDURE | |
| 614 | +## LOCAL WINDOWS TESTING PROCEDURE | |
| 596 | 615 | |
| 597 | 616 | This is what I do for routine testing on Windows. |
| 598 | 617 | |
| ... | ... | @@ -612,8 +631,7 @@ This is what I do for routine testing on Windows. |
| 612 | 631 | |
| 613 | 632 | * Test with mingw: `ctest --verbose -C RelWithDebInfo` |
| 614 | 633 | |
| 615 | - | |
| 616 | -DOCS ON readthedocs.org | |
| 634 | +### DOCS ON readthedocs.org | |
| 617 | 635 | |
| 618 | 636 | * Registered for an account at readthedocs.org with my github account |
| 619 | 637 | * Project page: https://readthedocs.org/projects/qpdf/ |
| ... | ... | @@ -640,8 +658,7 @@ following branching strategy to support docs: |
| 640 | 658 | The release process includes updating the approach branches and |
| 641 | 659 | activating versions. |
| 642 | 660 | |
| 643 | - | |
| 644 | -CMAKE notes | |
| 661 | +## CMAKE notes | |
| 645 | 662 | |
| 646 | 663 | To verify the various cmake options and their interactions, several |
| 647 | 664 | manual tests were done: |
| ... | ... | @@ -662,8 +679,7 @@ We are using RelWithDebInfo for mingw and other non-Windows builds but |
| 662 | 679 | Release for MSVC. There are linker warnings if MSVC is built with |
| 663 | 680 | RelWithDebInfo when using external-libs. |
| 664 | 681 | |
| 665 | - | |
| 666 | -ABI checks | |
| 682 | +## ABI checks | |
| 667 | 683 | |
| 668 | 684 | Until the conversion of the build to cmake, we relied on running the |
| 669 | 685 | test suite with old executables and a new library. When QPDFJob was |
| ... | ... | @@ -698,8 +714,7 @@ steps. See comments in check_abi for additional notes. Running |
| 698 | 714 | "check_abi check-sizes" is run by ctest on Linux when CHECK_SIZES is |
| 699 | 715 | on. |
| 700 | 716 | |
| 701 | - | |
| 702 | -CODE FORMATTING | |
| 717 | +## CODE FORMATTING | |
| 703 | 718 | |
| 704 | 719 | * Emacs doesn't indent breaking strings concatenated with + over |
| 705 | 720 | lines but clang-format does. It's clearer with clang-format. To |
| ... | ... | @@ -707,18 +722,22 @@ CODE FORMATTING |
| 707 | 722 | that builds the concatenated string. |
| 708 | 723 | |
| 709 | 724 | * With |
| 710 | - | |
| 725 | + | |
| 726 | + ```cpp | |
| 711 | 727 | long_function(long_function( |
| 712 | 728 | args) |
| 713 | 729 | |
| 730 | + ``` | |
| 731 | + | |
| 714 | 732 | clang-format anchors relative to the first function, and emacs |
| 715 | 733 | anchors relative to the second function. Use |
| 716 | - | |
| 734 | + | |
| 735 | + ```cpp | |
| 717 | 736 | long_function( |
| 718 | 737 | // line-break |
| 719 | 738 | long_function( |
| 720 | 739 | args) |
| 721 | - | |
| 740 | + ``` | |
| 722 | 741 | to resolve. |
| 723 | 742 | |
| 724 | 743 | In the revision control history, there is a commit around April 3, | ... | ... |