Bulk PDF Upload v2 — Implementation Plan¶
Design: 2026-08-11-bulk-pdf-upload-v2-design.md. This plan is executable PR-by-PR; each PR section lists repo, scope, key files, tests, and acceptance criteria. Waves order the PRs by dependency; PRs within a wave are parallel.
Wave map¶
| Wave | PR | Repo | Summary |
|---|---|---|---|
| 0 | PR-0a | camaradesuk/syrf | s3-notifier chart: template notification filter + lifecycle rules |
| 0 | PR-0b | camaradesuk/camarades-infrastructure | prod role ignore_changes handover block |
| 0 | PR-0c | camaradesuk/cluster-gitops | per-env filter/lifecycle values + Function CR adoption fix + verified no-op prod sync |
| 1 | PR-1 | camaradesuk/syrf | Domain + API: server-owned multipart session, sweeper, processing claim, PM consumers, normalization contract |
| 1 | PR-5 | camaradesuk/server-config | clamd role, syrf-pdf-net, CIFS mount, syrf runner, agent slot |
| 2 | PR-2 | camaradesuk/syrf | s3-notifier code: dispatch switch, BulkPdfUpload kind, presigned GET |
| 3 | PR-3 | camaradesuk/syrf | Agent service + chart + image CI + arrnc-api deploy workflow |
| 3 | PR-6 | camaradesuk/cluster-gitops | agent service wiring for staging + previews (after PR-3) |
| 4 | PR-4 | camaradesuk/syrf | Frontend: folder upload, multipart/reconciliation client, truthful results UX, user guide |
Merge order = wave order. PR-5 production apply (manual dispatch) happens with Wave 3. Exception to "PRs within a wave are parallel": PR-6 targets the chart/image PR-3 publishes (the ApplicationSet would otherwise generate an Application for a chart revision that doesn't exist yet), so within Wave 3, PR-3 must merge and its image/chart tag land in GHCR/cluster-gitops before PR-6 merges.
Wave-3 prerequisite (runner-boundary review): before PR-3 merges, the self-hosted
deploy runner on arrnc-api must be approved and CLAUDE.md's runner-boundary section updated
in PR-3 to define the deploy-runner category (server-config-managed host,
gatekeeper-restricted sudo, no build/test workloads, repo-scoped) — OR PR-3 switches to the
fallback: a GitHub-hosted job calling container-web-deploy over SSH with a restricted key.
Settle this before PR-3 is written, not during its review.
Wave 0 — ADR-010 minimal unblock (prerequisite)¶
Why bundled: the notifier change (PR-2) cannot reach production through the chart while the
Phase-5 cutover is stuck, and the chart's first successful prod sync as-written would drop the
Projects/ notification filter and the live multipart-cleanup lifecycle rule (design §7).
PR-0a — chart templating (syrf)¶
src/services/s3-notifier/.chart/templates/bucket.yaml: renderspec.notification.lambdaFunctionConfigurations[].filter.key.filterRulesfrom.Values.bucket.notificationFilterPrefix(omit when empty), andspec.lifecycle.rulesfrom.Values.bucket.lifecycleRules(omit when empty).src/services/s3-notifier/.chart/values.yaml:bucket.notificationFilterPrefix: "",bucket.lifecycleRules: []defaults.- Tests:
helm templategolden-file assertions for empty/filtered/lifecycle variants. - Acceptance: chart renders byte-identical manifests for existing envs when new values unset.
PR-0b — Terraform role parity (camarades-infrastructure)¶
terraform/lambda/main.tf: add toaws_iam_role.production_lambda_rolethe samelifecycle { ignore_changes = [tags, tags_all, description, assume_role_policy] }block the staging role gained in the ADR-010 Phase-4 handover (main.tf:189-204).- Acceptance:
terraform planshows no proposed changes to the production role.
PR-0c — values + adoption fix + verification (cluster-gitops)¶
syrf/environments/production/s3-notifier/values.yaml:bucket.notificationFilterPrefix: "Projects/";bucket.lifecycleRules:carrying both the existingIncomplete Multipart Cleanuprule (verbatim from the live Bucket CR spec) and the newexpire-bulk-stagingrule (filter.prefix: "Projects/_bulk-staging/",expiration.days: 7, plusnoncurrentVersionExpiration.noncurrentDays: 1andexpiration.expiredObjectDeleteMarkercleanup — the chart enables bucket versioning, so expiry without a noncurrent-version rule would leave every "deleted" ZIP retained indefinitely as a noncurrent version behind a delete marker).- Staging + preview values:
lifecycleRuleswith the bulk-staging rule (their buckets are dedicated; notification filter prefix stays empty/bucket-wide as today). Preview caveat: preview object keys are prefixedpreview/pr-{n}/byGetS3PathPrefix(), so the preview lifecycle rule's filter must be rendered per-PR aspreview/pr-{n}/Projects/_bulk-staging/(the pr-preview workflow already writes per-PR values files) — a bareProjects/_bulk-staging/filter would match nothing in previews. - Resolve the terminal Function CR: delete the conflicting
AdoptedResource/Function pair and re-adopt in order on chart v1.17.1 (nameOrIDfix), per the chart's own forward-path comment (adopted-resource.yaml:40-45). - Gates (in order, STOP on failure): (1) re-authenticate AWS CLI and independently confirm
live bucket notification + lifecycle + Lambda config and current bucket CORS before touching
anything; (2) after
sync,
s3-notifier-productionSynced/Healthy; (3)aws s3api get-bucket-notification-configurationandget-bucket-lifecycle-configurationshow the prefix filter and both rules — i.e. the cutover is a no-op for runtime behaviour; (4) an operator harness creates a staging MPU and presigns one browser-originUploadPartwhose signature binds exact content length and SHA-256; a real browser PUT of an exact known-length Blob succeeds with user-agent-ownedContent-Lengthand JavaScript-set checksum header, proving CORS does not require JavaScript to set the forbidden header; (5) a staging search-import smoke test passes (the P-3 verification ADR-010 never ran). - Acceptance: production app Synced/Healthy; notification + lifecycle verified unchanged plus the new rule; chartTag pinned ≥ the PR-0a chart version.
Wave 1¶
PR-1 — Domain + API (syrf)¶
Scope (design §4.1–§4.3): BulkPdfUploadJob embedded entity + guards + counts + bounded
history (last 20/search, pruned on new-job creation); server-owned S3 multipart session bound
to exact upload ID/object key/size/whole-object SHA-256/part geometry/virtual host; configurable
20-minute renewable lease, server-advertised 60-second heartbeat, completed-part renewal, and
exact-content-length + checksum-bound presigned UploadPart URLs with 5-minute request-start
validity; configurable API-advertised 600-second maximum client part-request duration; endpoints to initiate/status/sign part/
acknowledge part/heartbeat/complete/abandon/cancel/fresh retry; backend-only
CompleteMultipartUpload; atomic monotonic completion-vs-abort transitions; typed terminal
rejection carrying authoritative state; storage reconciliation for ambiguous completion;
one durable completion-operation lease shared by the original API completion and PM recovery
(provisional 180 seconds around a 120-second server-owned storage timeout), with live-owner
exclusion, atomic expired takeover, 60-second legacy missing-claim grace, exact ownership
revalidation before every storage side effect/proof write, and a final claimed HEAD before
Aborting; API persistence of each latest signed-part URL expiry before returning the signature;
a hosted sweeper that promptly issues AbortMultipartUpload and derives cleanup eligibility as
max(AbortStartedAt, LatestSignedPartUrlExpiresAt) + MaximumUploadPartRequestDuration, and then
repeatedly aborts/deletes and verifies ListParts/HEAD,
requiring two joint empty/NoSuchUpload MPU and object-absence observations at least one
30-second sweep interval apart and resetting proof on any nonempty MPU or present object before
recording Abandoned/Cancelled and releasing active capacity; a present object is deleted and
must be re-observed absent on a later pass; direct absence proof only for legacy jobs that truly
have no MPU; the S3 incomplete-MPU lifecycle rule remains only a
safety net; legacy BSON compatibility without read-time inferred expiry;
Study.BulkPdfDeliveredAt and BulkPdfDeliveredPath; centralized bulk-aware PDF-URL
builder routed through ALL five existing call sites (Study.GetLinkToPdf, StudyROBDto,
StudyListItemDto, StatsWithIncompleteDto/StudyBaseDto, StudyDto,
PdfConverterService) with per-segment percent-encoding; BulkPdfUploadJobDto +
ProjectDetailsDto.BulkPdfUploadJobs + AutoMapper mapping (SignalR wiring); search/project
deletion guard while a job is non-terminal; BulkPdfUpload policy; feature flag;
history/report/pdfPaths endpoints; PM receive endpoint with processing-claim + progress + finalize
consumers (single writer, concurrency 1, optimistic-concurrency retry policy); the agent claim
must be persisted before download/scratch/sentinel/scan/copy and its opaque token must bind every
progress/finalize message; delivered-outcome-qualified study marking; CSV report generation
(RFC 4180 + formula neutralization); message contracts (path ≤ 512 B / detail ≤ 256 B caps); normalization
helper + vector file + portability/collision/root-relative contracts; BsonClassMap
registrations.
Key files
src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectAggregate/BulkPdfUploadJob.cs(new).../Model/StudyAggregate/Study.cs(BulkPdfDeliveredAt, link generation)src/libs/project-management/SyRF.ProjectManagement.Messages/Commands/—IProcessBulkPdfUploadCommand,IReportBulkPdfUploadProgressCommand,IFinalizeBulkPdfUploadCommand,BulkPdfFileOutcomesrc/libs/kernel/SyRF.SharedKernel/PdfPathNormalization/—PdfPathNormalizer.cs,normalization-vectors.jsonsrc/libs/kernel/SyRF.SharedKernel/Enums/UploadKind.cs— addBulkPdfUploadsrc/services/api/SyRF.API.Endpoint/Controllers/—BulkPdfUploadController+ multipart DTO/options validationsrc/libs/appservices/SyRF.AppServices/FileServices/BulkPdfUpload/— backend multipart storage abstraction/implementationsrc/services/project-management/SyRF.ProjectManagement.Endpoint/Consumers/— claim, progress, and finalize consumers + endpoint registrationsrc/services/project-management/SyRF.ProjectManagement.Endpoint/Services/— due-session sweeper/processor + validated optionssrc/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Repositories/ProjectRepository.cs— class mapssrc/charts/syrf-common/env-mapping.yaml+pnpm run generate:flagsoutputappsettings—PdfBaseUrlenv-supplied (no hardcoded default)
Tests: domain guard/idempotency/monotonicity suite (port the cases from #2373's
PdfUploadJobTests/PdfUploadJobIdempotencyTests, fresh code); normalizer vs vector file;
multipart controller/storage tests for exact bindings, signed content length + checksum, positive
request size, API-advertised request timeout distinct from URL validity, part
visibility/checksum/size, lease expiry,
typed terminal responses, stale heartbeat non-resurrection, late successful part acknowledgement
after abort, completion-vs-abort winners and ambiguous completion reconciliation; real-Mongo
sweeper query/fairness tests; cleanup retention through the derived latest-URL quiescence boundary and first-empty observation,
nonempty proof reset, second-empty interval enforcement, legacy/completed-object direct absence,
transient storage errors, and repeated abort/ListParts; durable cross-replica abort-sweep claim
acquisition, active-claim exclusion, expired takeover, exact-token/unexpired proof writes,
bounded storage passes, prompt normal release, expiry takeover after process/host cancellation,
and zero storage work by claim losers; separate durable completion-operation claim tests for API
and PM ownership, legacy grace, active exclusion, expired takeover, per-side-effect revalidation,
server timeout, final HEAD, and completion-vs-abort convergence; claim-before-side-effects token contracts;
finalize consumer matching/dedup/Missing-Unmatched taxonomy (port
UpdateStudyPdfStatusConsumerTests cases); controller authz + flag gating; CSV generation.
Acceptance: all endpoints 404 behind the flag; no behaviour change flag-off — the only
production-visible code path PR-1 touches is link generation, which branches on
Study.BulkPdfDeliveredAt, a field no existing document has (null → legacy URL) and which
only the finalize consumer ever sets, so flag-off behaviour is byte-identical and no
MongoDB migration is needed; swagger + generated TS client regenerated. Server state changes
are available through ProjectDetailsDto/SignalR and status/heartbeat endpoints so PR-4 can
reconcile missed real-time delivery. Storage IDs, keys, and claim tokens are never exposed to the
browser. Do not enable the feature flag until the timing gate below passes.
Infrastructure boundary (not PR-1): before the first staging multipart rehearsal, verify the
API runtime AWS identity and bucket policy allow the exact backend/presign operations under
Projects/_bulk-staging/ (PutObject, GetObject, DeleteObject,
AbortMultipartUpload, ListMultipartUploadParts) and verify bucket CORS accepts browser PUT
with the JavaScript-set checksum header from every SyRF origin while the user agent supplies the
SigV4-bound exact Content-Length for a known-length Blob. Content-Length must not appear in
the API's JavaScript-settable RequiredHeaders. PR-0c preserves the incomplete-MPU lifecycle
rule, which is cleanup defence in depth. If IAM, policy, CORS, or the real browser PUT proof is
missing, add the narrow
environment-repository change in the appropriate later wave; do not expand PR-1 or infer that
the lifecycle rule makes the protocol safe.
PR-5 — Hosting (server-config)¶
Scope (design §6): docker_networks + syrf-pdf-net; roles/clamav (Ansible-managed,
restart_policy: unless-stopped, host volume for signature DB, no published ports, and a
templated clamd.conf drop-in raising StreamMaxLength/MaxScanSize/MaxFileSize to cover
the 500 MB entry cap) + structural ansible_managed: true projects entry; cifs_mounts item
for the ecrf1 share →
/srv/data/syrf-pdf/production/output (mount_user = agent uid, smb_version per share);
new cifs/syrf-pdfs age scope; github_runner block for camaradesuk/syrf
(os_user: runner-syrf); gatekeeper project entry syrf-pdf-agent-production
(host_port: 8089 nominal, volume_path: /srv/data/syrf-pdf/production,
container_volume_path: /app/data, skip_web_acl: true) + mirror entry in
deployments/arrnc-api.yml (managed: false, gated: true); non-secret env in
portal-env-overrides.yaml; sentinel-file creation task (output/.syrf-storage-ok) guarded
on the mount being active.
Tests/acceptance: repo CI (deployment-manifest exhaustiveness, secrets validation, lifecycle vocabulary) green; rehearsal-host apply clean; production apply deferred to Wave 3 (manual dispatch). Needs the share name + credentials (design §10) before the production apply, not before merge.
Wave 2¶
PR-2 — Notifier code (syrf)¶
Scope (design §4.4): dispatch Enum.TryParse + switch with logged no-op default;
BulkPdfUpload case (metadata validation including expected completed-object size/SHA-256,
12 h presigned GET, Send to agent queue on the metadata vhost); update the stale sections of
docs/architecture/systematic-search-upload-flow.md this touches (uploadkind/virtualhost,
flattened contract).
Key files: src/services/s3-notifier/SyRF.S3FileSavedNotifier.Endpoint/S3FileReceivedFunction.cs;
tests in the endpoint test project (dispatch table: ReferenceUpload / ReferenceUpdate /
BulkPdfUpload / missing-metadata / unknown-kind ⇒ no-throw).
Acceptance: unknown-kind and missing-metadata events log and return without throwing
(kills the current single-study-PDF error/retry noise). Note for the PR-2 description:
this is a deliberate semantics change — today an unrecognised object falls into the
search-import branch and throws, triggering S3 retries; after this PR unknown kinds are a
logged no-op with no retry. Reviewers should see that stated, not discover it. LocalStack
integration test covers the new kind end-to-end to a RabbitMQ assertion — including
asserting the presigned GET actually dereferences (an HTTP GET against LocalStack inside
the test env, not just non-null; note the URL host will be localhost:4566, so the Wave-3
agent fixtures must use the same LocalStack endpoint to stay compatible). Ships as
s3-notifier-v{next} through the standard promotion (staging auto, production manual — now
unblocked by Wave 0).
Wave 3¶
PR-3 — Agent service (syrf)¶
Scope (design §4.5): src/services/pdf-agent/ — SyRF.PdfAgent worker + consumer,
mandatory PM processing claim as the first operation before sentinel/scratch/download or any
other side effect; claim-token-bound progress/finalize; clamd INSTREAM client (limits sized per
design §6.2), download byte-cap and exact whole-object SHA-256 enforcement,
extraction guards + portability/collision contracts + .pdf-extension enforcement, conflict
policies incl. SHA-256, sentinel check, crash-safe outcome journal
(scratch/{uploadId}/journal.jsonl), lock/age-guarded scratch sweep, progress/finalize
reporting; Dockerfile (dedicated non-root uid); pdf-agent.slnf; GitVersion; syrf.sln
entries; .chart/ (deployment with Recreate strategy + PVC + sentinel init
container + clamd sidecar with raised limits + optional static file server for
staging/preview); ci-cd.yml build/push/tag jobs; full PR-level CI wiring — the agent
must be added to pr-tests.yml change predicates, detect-service-changes.sh, and
pr-preview.yml's detection/version/image matrix + preview deployment values, or agent PRs
get no tests and previews can't run the PR's own agent image;
.github/workflows/deploy-pdf-agent.yml (workflow_dispatch, runs-on: [self-hosted, linux,
live], container-web-deploy with GHCR GITHUB_TOKEN auth and -e secrets — respecting
the shared-runner boundary rules in CLAUDE.md); docker-compose.dev.yml gains a clamd
container; LocalStack init extended.
Tests: consumer unit tests prove no filesystem/network-processing side effect precedes a successful PM claim, rejected/mismatched claims do nothing, and redelivery uses the persisted claim; fixture ZIPs (clean / infected via EICAR / zip-slip / zip-bomb / macOS junk / non-PDF / duplicate paths / unicode names); conflict-policy matrix incl. hash-compare; sentinel fail-closed; integration test against dockerised clamd + LocalStack.
Acceptance: image builds + pushes in CI with pdf-agent-v{semver} tags; staging E2E
(below) passes; production deploy dispatch works after the PR-5 production apply.
PR-6 — Agent env wiring (cluster-gitops)¶
Sequencing: merges after PR-3 (not parallel — see wave-map note above); targets the chart/image tag PR-3 publishes.
Scope: syrf/services/pdf-agent/{config,values}.yaml; staging + preview environment
values (PVC size, PdfBaseUrl, static server enabled, RabbitMQ vhost); ApplicationSet picks
the service up like any other. Preview flag remains off by default; enabled per-PR via
#preview-config when testing.
Acceptance: staging agent Synced/Healthy consuming the staging vhost; a preview PR with the flag enabled gets a working in-cluster pipeline.
Wave-3 operational step (not a PR): run the PR-5 production apply manual dispatch,
verify mount + sentinel + clamd health on arrnc-api, then run the deploy workflow and a
production smoke test with a 2-file fixture folder against a scratch project before any
flag change. The smoke test's direct-command path (flag stays off) needs a real
completed BulkPdfUploadJob to claim/finalize against: the runbook's operator harness creates
the server-owned multipart job through Project.CreateMultipartBulkPdfUploadJob, drives its
verified-part/Completing/Uploaded domain transitions against the pre-staged exact object, and
persists the same size/hash/storage binding the API would — bypassing only the HTTP flag gate,
not domain invariants or the mandatory PM processing claim — then sends
IProcessBulkPdfUploadCommand referencing that job and binding.
Wave 4¶
PR-4 — Frontend (syrf)¶
Scope (design §4.7): folder selection (drag-drop + picker), client validation, pre-upload
match preview (shared normalization, GET .../pdfPaths), streaming ZIP web worker (STORE) and
chunked whole-object SHA-256; server-owned multipart client (initiate, signed part URLs,
exact known-length Blob.slice for every part, browser-owned Content-Length, JavaScript-set
checksum RequiredHeaders, UploadPart, server-verified part acknowledgement, heartbeat,
backend completion); enforce the API-advertised configurable 600-second request timeout with
the upload transport's timeout/abort support (XMLHttpRequest.timeout/abort(), or
AbortController where supported), separately from the 300-second presigned request-start validity; bounded
automatic failed-part retry while the same lease remains recoverable; honest local states
(Uploading only during active part request/retry, Connection lost — retrying, Upload
interrupted, Cancelling, confirming interruption with server); AbortController shutdown of
local work on terminal state; idempotent abandon command durably queued client-side while offline
and resent on reconnect/reload; authoritative session query before rendering restored state;
SignalR-driven state with heartbeat/status and bounded polling reconciliation fallback; server
terminal state stops progress/retry and discards queued completion; fresh independent Retry;
result summary panel, history table, CSV download; TS normalization mirror + vector-file test;
user-guide page (help.syrf.org.uk) for the new workflow.
Tests: store/component specs for every state (validating, preview, uploading, retrying,
interrupted/confirming/cancelling, scanning, copying, complete, failed, infected); worker ZIP
and chunked-hash tests; network drop with successful automatic part retry; terminal drop then
reconnect; reload with queued abandonment; lost client that never returns; server abort while
client thinks active; missed SignalR recovered by heartbeat/poll; late successful part after
abort; stale heartbeat cannot resurrect; exact part slicing and request-timeout cancellation;
assert Content-Length is never treated as a JavaScript-settable required header;
completion-vs-abort winner convergence; honest accessible status text; normalization parity spec loading normalization-vectors.json;
systematic-searches integration spec for flag gating.
Acceptance: flag-off = zero UI change; the UI never displays Uploading/In progress when it knows there is no active transfer; server terminal state always wins reconciliation; a local terminal attempt never looks resumable and Retry always creates a fresh upload; preview-env E2E: drop folder → confirm preview → multipart upload → watch progress → verify served PDF link + CSV report. Staging timing proof must show every 16 MiB part of a slow 1 GB upload completes within the advertised 600-second request timeout, including the background/throttled cases.
Rollout & verification sequence¶
- Wave 0 merged + production sync verified no-op (gates in PR-0c).
- Waves 1–2 merged; staging flag on; routing verification only (the agent doesn't exist
yet, so neither does its queue): initiate a multipart session, upload/acknowledge all parts,
request backend completion, verify the completed ZIP and its exact metadata land in staging
S3, and
the notifier's
Sendis verified via Lambda logs plus a temporary operator-declared diagnostic queue bound to the command exchange in the staging vhost (RabbitMQ management UI) — deleted after the check. Without a bound queue the message would be unroutable, which is expected and harmless at this stage. - Wave 3 merged (PR-3 image, then PR-6 wiring — sequential, not parallel; PR-6 targets
a chart revision PR-3 publishes): staging pipeline rehearsal, API-driven (the browser
flow is Wave 4): use the API to initiate a session, upload/acknowledge exact-length +
checksum-bound parts,
and request backend completion (flag is on in staging since Wave 1), then verify the
full path S3 → notifier → in-cluster agent → PVC → SignalR → CSV, including an EICAR
fixture to prove the Infected path. Then server-config production apply → agent deployed
on arrnc-api → production infra smoke with the flag still off: an operator first
uses the documented harness to persist an exact, domain-transitioned
Uploadedmultipart job for the pre-staged object, then sendsIProcessBulkPdfUploadCommanddirectly to the production queue referencing its upload ID/size/hash binding, for a pre-staged 2-file fixture ZIP against a scratch project (documented runbook step — exercises agent, clamd, CIFS write, PM finalize, without needing the flag-gated API), verify files on the share + job terminal state. - Wave 4 merged; full browser-driven staging E2E (folder pick → preview → zip → multipart
upload → live progress → CSV) now that the frontend exists. Before launch, exercise the
timing matrix: slow 1 GB upload, transient connectivity and bounded same-session part retry,
expected heartbeat jitter, and device sleep/background throttling. Validate the configurable
20-minute lease / 60-second heartbeat / completed-part renewal boundary, 300-second part-URL
request-start validity, and 600-second maximum active part request; prove exact 16 MiB parts
fit the request timeout in the slow/background cases. If evidence shows the defaults unsafe,
STOP at this narrow decision and recommend revised values rather than silently changing them.
Also prove terminal interruption; the worst-case 15-minute cleanup-not-before boundary from
five-minute latest signed URL expiry plus the 600-second request maximum; two joint empty/
NoSuchUploadMPU and object-absence observations at least 30 seconds apart before capacity release; late-object deletion and proof reset; reconnect reconciliation; and fresh Retry. Then enablebulkPdfUploadin production values (this is the launch); announce; monitor first real uploads (agent logs via docker on arrnc-api; job outcomes in Mongo; Lambda metrics). - Close #2373 as superseded (comment linking this plan); remove its worktree; delete stale branch after a grace period. Update epic #2223 and close the subsumed April issues (#2516–#2546 set) with pointers to the v2 PRs.
Rollback levers¶
- Feature flag off (stops new sessions and upload-session mutation immediately; already completed objects continue through agent/PM because consumers are never flag-gated, while an incomplete browser transfer converges to verified abandonment through its lease/sweeper).
- Agent: stop/redeploy previous image via the gatekeeper (
--skip-pullenv-only redeploys supported); messages queue durably while it's down. - Notifier: standard zip-version rollback via GitHub Release + config pin — with a floor: once any bulk upload has occurred, PR-2's version is the minimum safe notifier (pre-PR-2 code throws + S3-retries on any non-ReferenceUpdate kind, so an in-flight BulkPdfUpload event would loop). Rolling back below it requires the flag off and bulk events drained first.
- Wave 0 is deliberately verified as behaviour-neutral before anything rides on it.
Risks¶
| Risk | Mitigation |
|---|---|
| ADR-010 Function CR fix goes sideways | PR-0c hard gates + AWS-side pre-verification; production runtime stays on the TF Lambda until Synced/Healthy is proven |
| CIFS mount flaps under the container | Sentinel fail-closed; _netdev,nofail fstab opts; runbook: restart container after remount |
| Normalization drift between TS and C# | Single vector file consumed by both test suites; adding a vector on one side breaks the other until implemented |
| 1 GB browser zipping on low-end machines | STORE mode + streaming worker; documented guidance; cap enforced client- and agent-side |
| Slow links/background throttling outlive a live session | Configurable 20-minute lease + 60-second heartbeat + completed-part renewal; server-advertised 600-second request timeout distinct from 300-second URL request-start validity; staging must prove every 16 MiB part fits before the narrow timing STOP gate clears |
| Browser cannot satisfy the signed part contract | Sign exact content length + checksum; upload an untransformed known-length Blob.slice; browser owns forbidden Content-Length, JavaScript sets only returned checksum headers; PR-0c real-browser CORS/PUT proof |
| Client disappears or an in-flight part/completion races abort | Durable idempotent abandon command when the client returns; monotonic Aborting; persist latest signed URL expiry, prompt abort, then delay proof until that expiry plus the maximum part-request duration; repeated abort/list/HEAD with two joint MPU-empty/object-absent observations ≥30 seconds apart; one durable 180-second completion claim around a 120-second storage timeout fences API/PM completion while a separate abort claim owns cleanup; any part or object resets proof, a present object is deleted and must later be re-observed absent; lifecycle remains a backstop |
| RabbitMQ plaintext exposure (existing posture) | Separate AMQPS hardening PR; agent flips URI when it lands |
| clamd signature staleness | freshclam auto-updates; health check surfaces staleness in logs; scan failure ⇒ job Failed (never silently unscanned) |