ReviewOS

stacks/bun-query-builder

ci(release): a slow npm readback fails the release and skips the GitHub release and binaries

#1197
Closed glennmichael123 opened this 6 days ago ·
6 days ago

The Releaser marks a release failed whenever npm takes longer than two minutes to show the new version, even though the package was published. Every such run then skips Build Release Binaries and Create GitHub Release, so the version ships to npm with no GitHub release and no binaries.

Evidence: the last ten release runs

VersionRunpantry publish public readbackOn npm
0.3.28failure120.0s, then exit 1yes, ~4 min after upload
0.3.27failure120.0s, then exit 1yes, ~3 min after upload
0.3.26success58.4syes
0.3.25failure120.0s, then exit 1yes
0.3.24failure120.0s, then exit 1yes
0.3.23success61.6syes
0.3.22success72.0syes
0.3.21success58.2syes
0.3.20failure120.0s, then exit 1yes
0.3.19success59.1syes

Every failed run logs the same thing after a successful upload:

timings: public readback 120.0s
Upload accepted; publication is pending public metadata and verified tarball availability. Do not upload this version again.
##[error]Process completed with exit code 1.

The package is always published. What fails is pantry publish's own wait to read it back from the public registry: it gives up at 120 seconds and exits 1, and npm's propagation delay is often longer. As a result, 0.3.20, 0.3.24, 0.3.25, 0.3.27 and 0.3.28 have no GitHub release; the latest one listed is 0.3.26.

Re-running the failed job is not a fix. It would upload the same version again, which the publisher warns against ("Do not upload this version again") and which can produce a staged-version conflict.

Proposed fix

pantry's own Releaser already handles this. Its docs/releases.md ("npm acceptance and public availability") documents the contract, and .github/workflows/release.yml implements it:

  1. Publish once and keep the result. Run pantry publish --npm --access public once, tee its output to a log, and keep its exit code.
  2. Read back for longer, with full checks. Pass that exit code to .github/scripts/npm-publication-readback.ts. A pending accepted upload continues to a bounded readback of up to 15 minutes, which checks the real archive: the original identity, the metadata checksums, the full tarball bytes, and the manifest inside the archive.
  3. Fail only on real problems. A rejected upload, a missing or conflicting receipt, or a mismatched archive still fails the job; a slow registry no longer does. The publish log and proof are kept as artifacts.

Adopting the same step here, before Build Release Binaries and Create GitHub Release, would let a slow npm finish the release instead of abandoning it.

Also needed

Create the missing GitHub releases (and binaries) for 0.3.20, 0.3.24, 0.3.25, 0.3.27 and 0.3.28 from their existing tags, without publishing to npm again.

Found while releasing 0.3.28 for stacksjs/bun-query-builder#1196.

6 days ago

Fixed in 26cc976 and b00976d, building on 10a4d65.

The release workflow

  • Publish: pantry publish runs once. Its log and exit code go to the readback, which is ported from pantry's own Releaser: it waits up to 15 minutes and verifies the metadata, the tarball bytes and the manifest inside the tarball against the archive npm accepted.
  • When it fails: a rejected upload, a missing or conflicting receipt, or a mismatched archive still fails the job. A slow registry no longer does, and the publish log and proof are kept as a run artifact.
  • Manual runs: a manual run now requires an existing tag. It checks that tag out, confirms its version is the one npm serves, and builds the binaries and the GitHub Release without publishing. Before, a manual run would have published main again.
  • The Latest badge: a manual run hands the Latest badge back to the newest version. The releaser marks whatever it creates as Latest, and the first backfill took the badge from v0.3.29 until I moved it back.

Backfilled

v0.3.14, 0.3.18, 0.3.20, 0.3.24, 0.3.25, 0.3.27 and 0.3.28 now each have a GitHub Release with all five binaries. Every run verified the version npm serves. Nothing was uploaded again, npm latest is still 0.3.29, and v0.3.29 is still Latest on GitHub.

Checks

  • Unit tests: pantry's readback tests are ported to test/release-npm-readback.test.ts, plus a case using the real 0.3.28 publish receipt.
  • Against real npm: the real 0.3.28 receipt verifies, and a receipt with one changed shasum character is rejected.

Sign in to comment on this issue.