# Protected intake milestone

Implemented October 24, 2026. Development and fictional test files only.

## Implemented behavior

- Originals retain their bytes and SHA-256 hash. All bytes stay in the agency's private `originals/` directory; quarantine is enforced by database state and API access, not by moving files to a separate folder. The directory is not served as static content. Operators and backup archives can still access quarantined originals and must handle them accordingly.
- Each upload is checked against the supported extension/signature or UTF-8 text rules. These are preliminary format checks, not full format validation. Unsupported or mismatched files remain quarantined.
- ClamAV scanning runs before indexing, download or parser access. There is no user/admin release override and no environment variable to bypass scanning. Marking a file Reviewed does not release it.
- Missing scanner, stale/unverified signatures, sandbox failure, malware detections, encrypted input alerts, resource-limit alerts and scan failures remain blocked. The scanner signature timestamp must be at most seven days old and no more than one day in the future.
- A passed scan allows file access; it is not a guarantee of safety. Extraction has its own independent status: a clean file can have failed, unavailable or incomplete extraction.
- PDF extraction preserves physical page numbers, caps processing at 100 pages, 20,000 characters per page and 100,000 characters per file, and marks pages without extracted text for possible OCR. It does not run OCR, recover passwords or execute embedded scripts. Character positions refer to extracted text, not PDF layout coordinates.
- Clean UTF-8 text and extracted PDF pages are searchable within the authorized case. PDF hits include evidence ID, original hash and physical page number. Results are limited to 100; this is literal search, not AI correlation.
- Image/video/audio metadata processing now goes through the same scan gate and isolated worker. Existing direct metadata unit tests are parser tests using synthetic data, not the application intake path.
- Retry processing requires editor/manager access (or an agency administrator). Reads require case access. The retry re-scans the original and replaces searchable derivatives; failed re-scans remove earlier indexed text/pages. Scan outcomes and retries are recorded in intake history and actor-attributed audit events.
- Existing evidence without an intake record is migrated to `legacy_unscanned`; its derived content is hidden and its downloads blocked until scanned. Migration does not delete originals.
- SQLite backups preserve scan status/history and extracted pages, with the original bytes. Legacy backups restore as unscanned when the upgraded application first opens them. Backups themselves remain local and unencrypted; they are not safe to share.

## Backend host requirements

The web preview does not execute this backend. Run a local fictional-data installation using LOCAL_SETUP.md. Install the pinned Python requirements into a system environment or dedicated virtual environment, not only the user's site-packages: isolated workers use Python's `-I` option.

The first protected processing profile is Linux with Bubblewrap, ClamAV, official signatures and, for media metadata, FFmpeg. On an appropriate Debian/Ubuntu test host the administrator installs `bubblewrap`, `clamav`, `clamav-freshclam` and `ffmpeg`, and configures `freshclam` to keep official signatures in `/var/lib/clamav`. Do not start a second updater if a managed freshclam service already runs. Check the host's service and file permissions first.

The scanner must be discoverable as `clamscan` in the backend process PATH. Its signature directory must be readable by the non-root backend user. Signature updates need internet or an approved offline transfer. Air-gapped processing will stop when signatures age beyond the seven-day window.

Workers run with Bubblewrap network/process namespace isolation, read-only runtime/input mounts, an ephemeral temporary directory, cleared environment, and per-process CPU/address-space/file-size/file-descriptor limits. Only `/usr`, loader libraries, the Python environment, the selected worker/input and (for the scanner) ClamAV signatures are mounted. Agency databases, other originals and application secrets are not mounted. No shell command is constructed from filenames or file content.

The host must support unprivileged namespace isolation. Some container/cloud hosts prohibit this. This build refuses parser/scanner fallback when the sandbox cannot start. **Do not enable privileged containers, remove security protections, or disable the gate to make it pass.** Choose a supported host or separately reviewed worker service instead. Per-process limits are not an aggregate cgroup quota; production worker hardening and independent security review remain necessary.

The Dockerfile includes the required tools, but image build, container namespace compatibility and signature provisioning have not been validated here. It does not run a signature updater automatically, and it does not contain current signatures. Docker startup alone does not activate working scans. The pre-existing compose profile deliberately retains dropped capabilities and no-new-privileges.

## Test procedure

1. On a supported test host, upload a benign fictional TXT. Confirm `clean`, a real ClamAV version/signature date, a searchable phrase and authorized download.
2. Upload a benign two-page PDF. Confirm extracted text references the physical page, and image-only pages report missing text without invented content.
3. In a dedicated scanner test environment, use the official harmless EICAR antivirus test file. Confirm detected/quarantined state, no search/index/parser access, and HTTP 423 for download. Do not introduce real malware.
4. Stop signature availability or isolate the worker test host. Confirm explicit scanner/sandbox failure, preserved original bytes and blocked access.
5. Retry a blocked benign file after restoring protection. Confirm scan history and case permissions. Back up and restore a synthetic case to a new directory, and compare original hashes, page records and history.

This session verified application control flow with explicit synthetic scanner/extractor doubles, real benign PDF parsing, backup/restore and access tests. Actual ClamAV execution and a successful namespace sandbox could not be validated on this executor: ClamAV is absent and Bubblewrap network namespace creation is denied. Those installation checks remain mandatory before any real files are used.

## API

Under `/api/cases/{case_id}/evidence/{evidence_id}`:

| Route | Access | Behavior |
|---|---|---|
| `GET /intake` | Viewer+ | Current scan/extraction state and history timestamps |
| `POST /intake` | Editor+ | Re-scan and process original; no bypass |
| `GET /pages` | Viewer+ | Extracted PDF text by page; requires clean intake |
| `GET /original` | Viewer+ | Original bytes after scan gate and integrity check |
| `POST /metadata` | Editor+ | Re-scan before metadata processing |

Intake is synchronous in this development milestone. A bounded background queue, supervised production HTTP service, hosted processing, OCR, transcripts, AI conversations and automatic source retrieval remain next work. Do not expose the development HTTP server directly to the internet.