From 0bc7186bbcab179e13d9f6eda5121fa88efefb86 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Wed, 1 Jul 2026 12:08:19 +0300 Subject: [PATCH] Add production readiness audit --- DEVELOPMENT_PLAN_NEXT.md | 1355 ++++++++++++++++++-------------- PRODUCTION_READINESS_REPORT.md | 332 ++++++++ 2 files changed, 1093 insertions(+), 594 deletions(-) create mode 100644 PRODUCTION_READINESS_REPORT.md diff --git a/DEVELOPMENT_PLAN_NEXT.md b/DEVELOPMENT_PLAN_NEXT.md index 6bee98b..af008bd 100644 --- a/DEVELOPMENT_PLAN_NEXT.md +++ b/DEVELOPMENT_PLAN_NEXT.md @@ -1,643 +1,810 @@ -# AWatch-rus: план развития на следующие 6-12 месяцев +# TASK_065: Production 1.0 development plan -Дата аудита: 2026-07-01. +Audit date: 2026-07-01. -Статус: development planning после анализа текущего репозитория. Документ не -добавляет новые продуктовые функции и не меняет runtime-поведение. Цель - -снизить эксплуатационные, release, security и maintainability риски без -нарушения обратной совместимости. +Scope: AWatch-rus repository readiness for Production 1.0. This plan is based +only on repository evidence: tracked files, documentation, CI workflows, +Cargo metadata, dependency checks, scripts, deployment assets and operational +maturity contracts. -## Границы плана +This is not a feature roadmap. Every task below exists because the current +repository still has a confirmed blocker or material risk for a durable +Production 1.0 release. -- Production уже существует; текущий DetMir контур рассчитан примерно на 5 RDP - пользователей. -- Рабочие подсистемы не переписываются. Все изменения должны быть - инкрементальными, проверяемыми и откатываемыми. -- DLP runtime остается `core_only/disabled` по умолчанию; `light` включается - только вручную после resource preflight. -- Loki и always-on Velociraptor не включаются как часть этого плана. -- PowerShell fallback на Windows сохраняется до доказанной parity и burn-in - Rust-пути. -- GitHub остается public mirror validation; release evidence для российского - контура должен формироваться отдельно. +## Guardrails -## Фактическое состояние репозитория +- Do not redesign working subsystems. +- Do not add new product functionality as part of Production 1.0 readiness. +- Preserve backward compatibility for deployed DetMir production. +- Keep DLP runtime conservative: `core_only/disabled` by default; `light` only + after explicit operator resource preflight. +- Do not enable Loki or always-on Velociraptor as part of Production 1.0. +- Keep PowerShell fallback until Rust parity, canary evidence and rollback + criteria are proven. +- GitHub remains public mirror validation. Release evidence must be produced in + the controlled release contour documented under `docs/registry/`. -- Основной runtime слой стал Rust-first: `adk-rust/` содержит 58 workspace - crates; локальный `cargo metadata --locked` разрешает 349 пакетов. -- Rust dependency hygiene уже включен: `cargo audit --deny warnings`, - `cargo deny`, `cargo machete --with-metadata`, `cargo tree --duplicates` есть - в workflow или локальных gate. Текущий аудит: `cargo audit` и `cargo machete` - чистые, `cargo deny` проходит, но оставляет 36 `bans` warnings. -- CI уже покрывает Rust, docs/registry, smoke, operational maturity, - dependency hygiene, security scan и release binary build. Есть drift: - несколько workflows используют floating `stable`, хотя `rust-toolchain.toml` - закрепляет `1.94.0`. -- DetMir DLP resource guardrails реализованы: runtime disabled/core-only by - default, `detmir-dlp-load-guard`, documented rollback, skipped DLP buckets в - smoke считаются нормой. -- Security Finding Inbox, Hayabusa и Velociraptor оформлены как optional - findings/forensics sources и не являются hot-path зависимостью Workforce. -- Документация registry/governance сильная, но `docs/PROJECT_STATUS_RU.md` и - `docs/RESIDUAL_RISKS_RU.md` фиксируют открытые gaps: Gitea restore drill, - Russian build-runner, первый release evidence build, legal/rightsholder - package, external review evidence. -- Самые крупные maintainability hotspots: - `adk-rust/crates/detmir-portal/src/main.rs` - 14200 строк, - `adk-rust/crates/aw-windows-telemetry/src/main.rs` - 6411 строк, - `proxmox/tsj_guardian_bot.py` - 4610 строк, - `adk-rust/crates/worktime-api/src/main.rs` - 3988 строк, - `ansible/deploy_aw_server.yml` - 3099 строк. -- Найдены точные дубликаты скриптов: - `scripts/aw-contour-diag.sh` = - `scripts/detmir-full-diagnostics/aw-contour-diag.sh`; - `scripts/check_production_inventory_placeholders.sh` = - `scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh`. -- Найден security hygiene gap: ClickHouse/1C ops wrappers передают - `CLICKHOUSE_PASSWORD` через `--password`, что раскрывает секрет в process - argv на хосте. +## Evidence Base -## P0 - критично для production +Reviewed repository areas: -### P0-1. Gate фактической свежести production Rust-бинарников - -- Почему нужно: `scripts/check_detmir_rust_release_artifacts.sh` проверяет - наличие локальных release artifacts, но не доказывает, что production - service/timer/Windows task реально запускает тот же binary SHA256. -- Риск: stale бинарник в production может отличаться от проверенного кода, - усложнить rollback и скрыть регрессию в high-load контуре. -- Ожидаемый эффект: воспроизводимая цепочка `service/timer/task -> binary path - -> crate -> local artifact -> prod sha256 -> git sha`. -- Сложность: средняя. -- Объем работ: 3-5 инженерных дней. -- Затрагиваемые файлы: - `scripts/check_detmir_rust_release_artifacts.sh`, - `scripts/package_rust_release_binaries.py`, - `scripts/detmir-full-diagnostics/`, - `adk-rust/crates/detmir-readiness/`, - `windows/validate-deployment.ps1`, - `docs/DETMIR_CURRENT_STATE_RU.md`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. -- Критерии приемки: - команда dry-run выводит все реально задействованные binaries на - `10.10.10.2`, `10.10.10.13` и Windows RDP host; - для каждого binary есть crate, role, unit/timer/task, prod SHA256 и local - release SHA256; - gate падает при missing/stale binary; - проверка не включает heavy DLP, Loki или always-on Velociraptor; - rollback path документирован и проверен на одном canary binary. - -### P0-2. Bounded retention для state, evidence, queues и diagnostic output - -- Почему нужно: DLP/evidence/backlog/diagnostic контуры уже имеют guardrails, - но retention/cleanup policy для исторических artifacts остается отдельной - future work; disk exhaustion на Proxmox/AW server является прямым outage - риском. -- Риск: накопление JSONL queues, screenshots, Hayabusa drops, diagnostic runs, - ClickHouse landing files или локальных state snapshots может забить диск и - остановить ingestion, portal или ActivityWatch. -- Ожидаемый эффект: предсказуемое потребление диска, безопасный dry-run cleanup, - понятный rollback через backup/hold rules. -- Сложность: средняя. -- Объем работ: 4-7 инженерных дней. -- Затрагиваемые файлы: - `adk-rust/crates/aw-prune-local-state/`, - `scripts/detmir-full-diagnostics/`, - `scripts/detmir_dlp_warehouse_sync.sh`, - `aw-server/logrotate.conf`, - `windows/validate-deployment.ps1`, - `adk-rust/crates/aw-windows-telemetry/`, - `docs/DLP_OPTIONAL_RUNTIME_RU.md`, - `docs/DLP_RESOURCE_PROFILES_RU.md`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. -- Критерии приемки: - есть единая retention matrix по путям, владельцам, age/size limits и dry-run; - cleanup never follows untrusted paths and only touches allowlisted roots; - Windows queues имеют max-age/max-size validation; - skipped DLP buckets при `AW_DLP_ENABLED=false` не считаются ошибкой; - smoke показывает, что cleanup не удаляет active state и не требует downtime. - -### P0-3. Убрать секреты ClickHouse/1C из process argv - -- Почему нужно: `clickhouse-1c/ops/run_*.sh` передают пароль как - `--password "${CLICKHOUSE_PASSWORD}"`; этот аргумент виден через process list - локальному пользователю или диагностике. -- Риск: утечка ClickHouse credentials с возможностью чтения/изменения - production analytics data. -- Ожидаемый эффект: секреты передаются через env/config file descriptor или - другой non-argv канал; logs и error output остаются redacted. -- Сложность: низкая-средняя. -- Объем работ: 2-4 инженерных дня. -- Затрагиваемые файлы: - `clickhouse-1c/ops/run_ingest_cycle.sh`, - `clickhouse-1c/ops/run_manager_brief.sh`, - `clickhouse-1c/ops/run_recovery_brief.sh`, - `clickhouse-1c/ops/run_company_registry_bindings_refresh.sh`, - `clickhouse-1c/ops/run_company_intelligence_refresh.sh`, - `clickhouse-1c/ops/check_ingest_freshness.sh`, - `clickhouse-1c/ai/*.py`, - `clickhouse-1c/etl/*.py`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. -- Критерии приемки: - `rg -- '--password \"\\$\\{CLICKHOUSE_PASSWORD\\}\"' clickhouse-1c` ничего не - находит в runtime wrappers; - smoke через `ps` доказывает отсутствие password в argv; - существующие env-based deployments остаются совместимыми; - failed auth logs не печатают password; - `bash -n clickhouse-1c/ops/*.sh` и affected Python smoke проходят. - -## P1 - желательно выполнить до следующего релиза - -### P1-1. Выровнять Rust toolchain во всех CI workflows - -- Почему нужно: `rust-toolchain.toml` закрепляет `1.94.0`, но - `.github/workflows/ci.yml`, `security.yml`, `coverage.yml`, - `dependency-hygiene.yml` и `release-assets.yml` используют floating - `dtolnay/rust-toolchain@stable`. -- Риск: CI и release runner могут собирать разные версии компилятора, что - снижает воспроизводимость и усложняет анализ regressions. -- Ожидаемый эффект: единый compiler baseline для local, GitHub mirror и - будущего российского build-runner. -- Сложность: низкая. -- Объем работ: 1-2 инженерных дня. -- Затрагиваемые файлы: - `rust-toolchain.toml`, - `.github/workflows/ci.yml`, - `.github/workflows/security.yml`, - `.github/workflows/coverage.yml`, - `.github/workflows/dependency-hygiene.yml`, - `.github/workflows/release-assets.yml`, - `.github/workflows/rust-binary-build.yml`, - `.github/workflows/rust-workspace.yml`, - `docs/QUALITY_STATUS_RU.md`. -- Критерии приемки: - все blocking Rust jobs используют `rust-toolchain.toml` или тот же explicit - channel; - nightly остается только для advisory `cargo udeps`; - CI check names не меняются; - локально и в CI проходят `cargo fmt`, `cargo test`, `cargo clippy -D warnings`. - -### P1-2. Staged triage для duplicate/deprecated Rust dependencies - -- Почему нужно: `cargo deny` проходит, но `cargo tree --duplicates` показывает - дубли `bitflags`, `getrandom`, `hashbrown`, `mio`, `zip`, `windows-*`; - `serde_yaml 0.9.34+deprecated` уже зафиксирован как средний риск. -- Риск: supply-chain surface и binary size растут, а устаревшие crates могут - стать будущим vulnerability blocker. -- Ожидаемый эффект: baseline по допустимым дублям, запрет новых необоснованных - дублей, replacement path для deprecated crates. -- Сложность: средняя. -- Объем работ: 4-8 инженерных дней. -- Затрагиваемые файлы: - `adk-rust/Cargo.toml`, - `adk-rust/Cargo.lock`, - `deny.toml`, - `.github/workflows/dependency-hygiene.yml`, - crates using `serde_yaml`, `calamine`, `notify`, `zip`, `windows-sys`, - `docs/THIRD_PARTY_LICENSES_RU.md`, - `docs/QUALITY_STATUS_RU.md`. -- Критерии приемки: - для каждого duplicate family есть decision: keep with reason, update, or - remove; - `cargo audit --deny warnings` и `cargo deny ...` проходят; - новые duplicates без allowlist/обоснования блокируются; - replacement plan для `serde_yaml` не ломает существующие YAML configs; - изменения dependency graph идут отдельными PR, не смешиваются с runtime - features. - -### P1-3. Load regression harness для portal/worktime/report hot paths - -- Почему нужно: `docs/PROJECT_STATUS_RU.md` прямо фиксирует residual risk: - full report/snapshot prewarm может быть CPU/IO дорогим. Сейчас production - малый, но расширение users/events без бюджета нагрузки рискованно. -- Риск: рост RDP users или history window приводит к slow portal, stale cache, - ClickHouse/AW amplification или datastore lock pressure. -- Ожидаемый эффект: измеримые budgets для p95 latency, memory, query count, - cache hit/stale behavior и degradation semantics. -- Сложность: средняя-высокая. -- Объем работ: 1-2 недели. -- Затрагиваемые файлы: +- Rust workspace: `adk-rust/`, 58 crates from `adk-rust/crates/*/Cargo.toml`. +- Cargo state: `cargo metadata --locked --format-version 1` returned 58 + workspace members and 349 packages. +- Dependency hygiene: `cargo audit --deny warnings` passed; `cargo machete + --with-metadata` found no unused dependencies; `cargo deny` passed with + non-blocking policy warnings; `cargo tree --duplicates --locked` found + duplicate roots including `bitflags`, `getrandom`, `hashbrown`, `mio`, `zip`. +- CI: `.github/workflows/ci.yml`, `security.yml`, `coverage.yml`, + `dependency-hygiene.yml`, `operational-maturity.yml`, + `rust-workspace.yml`, `rust-binary-build.yml`, `release-assets.yml`. +- Operational maturity: `configs/operational-maturity-contract.json`, `scripts/operational-maturity-check.mjs`, - `scripts/awatch-production-hardening-smoke.mjs`, - `adk-rust/crates/detmir-portal/`, - `adk-rust/crates/worktime-api/`, - `adk-rust/crates/worktime-prewarm/`, - `adk-rust/crates/aw-contour-smoke/`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, - `docs/QUALITY_STATUS_RU.md`. -- Критерии приемки: - synthetic dataset проверяет минимум 5/20/50 users without prod data; - harness фиксирует p95 latency, max RSS, query count and stale-cache decisions; - disconnected RDP sessions and event-driven buckets не дают false-fail; - `AW_DLP_ENABLED=false` profile respected; - heavy job остается scheduled/advisory, blocking smoke остается быстрым. + `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. +- Registry and release evidence docs: `docs/registry/*`, + `docs/PROJECT_STATUS_RU.md`, `docs/RESIDUAL_RISKS_RU.md`, + `docs/QUALITY_STATUS_RU.md`, `ROADMAP.md`. +- Runtime/deployment: `ansible/`, `aw-server/`, `windows/`, `proxmox/`, + `clickhouse-1c/`, `grafana/`, `pfsense/`, root operational wrappers. -### P1-4. Windows Rust validation parity before reducing PowerShell runtime +Confirmed current strengths: -- Почему нужно: `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md` показывает, что - Rust overlay уже основной для части collectors, но `validate-deployment.ps1`, - Hayabusa upload и некоторые recovery paths остаются fallback/runtime. -- Риск: преждевременное удаление PowerShell fallback ухудшит recovery на - Windows Server 2019 и localized RDP sessions. -- Ожидаемый эффект: доказанная parity Rust validation, safe fallback policy и - меньше runtime drift. -- Сложность: средняя. -- Объем работ: 1-2 недели. -- Затрагиваемые файлы: - `adk-rust/crates/aw-windows-telemetry/`, - `windows/validate-deployment.ps1`, - `windows/export-upload-hayabusa-to-aw-server.ps1`, - `windows/ActivityWatch.Windows.Common.psm1`, - `ansible/deploy_aw_windows.yml`, - `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md`, - `docs/POWERSHELL_TO_RUST_ROADMAP_RU.md`. -- Критерии приемки: - Rust `validate-deployment` покрывает все 7 секций PowerShell validation; - CP866/localized user handling covered by tests; - canary run сравнивает Rust и PowerShell reports на одном RDP host; - PowerShell fallback остается documented rollback; - no removal before burn-in evidence. +- Rust-first runtime and wrappers exist. +- Public mirror CI exists. +- Dependency hygiene automation exists. +- Operational maturity offline harness exists. +- DLP resource guardrails and conservative runtime profile exist. +- Security Finding Inbox / Hayabusa / Velociraptor are optional and separated + from Workforce hot path. +- Production runbooks and DetMir guardrails are documented. -### P1-5. Российский build-runner и первый release evidence build +Confirmed Production 1.0 blockers and material risks are organized below. -- Почему нужно: `ROADMAP.md`, `docs/PROJECT_STATUS_RU.md` и - `docs/RESIDUAL_RISKS_RU.md` фиксируют, что `awatch-build-01` planned, а - первый release evidence build pending. -- Риск: registry/release package cannot be claimed reproducible in target - Russian contour. -- Ожидаемый эффект: проверяемый release package: source archive, binaries, - SBOM, SHA256SUMS, cargo metadata/tree, logs, manifest. -- Сложность: средняя. -- Объем работ: 3-6 инженерных дней плюс инфраструктурное окно. -- Затрагиваемые файлы: - `scripts/build_release_evidence.sh`, - `scripts/check_release_evidence.sh`, - `scripts/verify_release_assets.sh`, - `docs/registry/RU_BUILD_RUNNER_READINESS_RU.md`, - `docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md`, - `docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md`, - `docs/registry/registry-evidence-manifest.json`. -- Критерии приемки: - build-runner имеет documented OS/toolchain; - release evidence build выполнен не на GitHub runner; - `scripts/check_release_evidence.sh` проходит; - artifacts имеют SHA256 и manifest; - public docs не заявляют registry completion раньше legal approval. +## P0 - Critical before Production 1.0 -### P1-6. Gitea backup restore drill +### P0-1. Production binary parity gate -- Почему нужно: backup и SHA256 documented, но `restore_tested=false` остается - открытым residual risk. -- Риск: disaster recovery process может оказаться неполным только во время - инцидента. -- Ожидаемый эффект: доказанный restore на отдельном хосте и понятный RTO/RPO - baseline. -- Сложность: средняя. -- Объем работ: 2-4 инженерных дня. -- Затрагиваемые файлы: - `docs/registry/GITEA_BACKUP_AND_RESTORE_RUNBOOK_RU.md`, - `docs/registry/registry-evidence-manifest.json`, - `scripts/registry_readiness_check.sh`, - `docs/PROJECT_STATUS_RU.md`, - `docs/RESIDUAL_RISKS_RU.md`. -- Критерии приемки: - restore выполнен на отдельном сервере; - checksums, logs, repository accessibility и rollback notes сохранены; - manifest обновлен с `restore_tested=true`; - recovery evidence не содержит секреты. +Purpose: prove that every binary actually running in production matches the +reviewed release artifact. -### P1-7. Hayabusa upload path: Rust helper parity and operational proof +Reason: `scripts/check_detmir_rust_release_artifacts.sh` verifies local release +artifacts, but Production 1.0 also needs a verified mapping from deployed unit, +timer or Windows task to exact production SHA256 and source crate. -- Почему нужно: матрица PowerShell-to-Rust помечает - `windows/export-upload-hayabusa-to-aw-server.ps1` как P1 runtime wrapper. - Hayabusa is optional, but it is a security evidence path. -- Риск: Windows PowerShell encoding, task principal или SSH wrapper drift - ломает forensics upload без явной compile-time проверки. -- Ожидаемый эффект: Rust helper covers upload, sidecar metadata, checksum and - server drop-zone contract; PowerShell remains fallback. -- Сложность: средняя. -- Объем работ: 5-8 инженерных дней. -- Затрагиваемые файлы: - `adk-rust/crates/aw-windows-telemetry/`, - `adk-rust/crates/hayabusa-tools/`, - `windows/export-upload-hayabusa-to-aw-server.ps1`, - `aw-server/hayabusa/aw-hayabusa.sh`, - `ansible/deploy_aw_windows.yml`, - `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md`, - `docs/POWERSHELL_TO_RUST_ROADMAP_RU.md`. -- Критерии приемки: - helper creates identical server-side intake result on fixture; - BOM/backslash/path traversal tests preserved; - task principal remains interactive Administrator where required; - server `aw-hayabusa-drop.path` processing stays optional and fail-closed; - no automatic remediation is introduced. +Operational impact: prevents stale binary drift and makes rollback decisions +deterministic. -## P2 - улучшения архитектуры и сопровождаемости +Risk: high. A stale production binary can invalidate test results and hide +regressions. -### P2-1. Инкрементальная декомпозиция крупных runtime modules +Estimated effort: 3-5 days. -- Почему нужно: несколько production-critical files слишком крупные для - надежного review и локального reasoning, особенно `detmir-portal/main.rs` и - `aw-windows-telemetry/main.rs`. -- Риск: высокий change-coupling, труднее делать точечные fixes и находить - regressions. -- Ожидаемый эффект: smaller modules by domain, faster review, focused tests. -- Сложность: средняя-высокая. -- Объем работ: 2-4 недели несколькими малыми PR. -- Затрагиваемые файлы: - `adk-rust/crates/detmir-portal/src/main.rs`, - `adk-rust/crates/detmir-portal/src/*`, - `adk-rust/crates/aw-windows-telemetry/src/main.rs`, - `adk-rust/crates/worktime-api/src/main.rs`, - `proxmox/tsj_guardian_bot.py`, - `clickhouse-1c/ai/company_intelligence_api.py`. -- Критерии приемки: - каждый PR только moves/extracts one bounded domain; - public API, config names, systemd units and task names unchanged; - tests before/after identical; - no new dependencies without written justification; - line count decreases in target `main.rs` files without behavioral rewrite. +Affected modules: + +- `scripts/check_detmir_rust_release_artifacts.sh` +- `scripts/package_rust_release_binaries.py` +- `scripts/detmir-full-diagnostics/` +- `adk-rust/crates/detmir-readiness/` +- `adk-rust/crates/aw-windows-telemetry/` +- `windows/validate-deployment.ps1` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` + +Acceptance criteria: + +- Report maps `service/timer/task -> binary path -> crate -> runtime role`. +- Report includes local release SHA256, production SHA256 and git SHA. +- Missing or mismatched binaries fail the gate. +- DLP/Loki/Velociraptor heavy runtime is not enabled by the check. + +Validation steps: + +- Run the parity gate against current release artifacts. +- Run diagnostics for `10.10.10.2`, `10.10.10.13` and Windows RDP host. +- Confirm stale binary simulation fails. + +Expected benefit: release and production state become auditable. + +### P0-2. Controlled release evidence build + +Purpose: produce Production 1.0 release evidence outside GitHub public mirror. + +Reason: `docs/PROJECT_STATUS_RU.md`, `docs/QUALITY_STATUS_RU.md` and +`docs/registry/registry-evidence-manifest.json` state that GitHub Actions are +public validation only and that the Russian build-runner/release evidence path +is still pending. + +Operational impact: separates mirror CI from release authority. + +Risk: high. Without controlled release evidence, Production 1.0 cannot be +treated as reproducibly built. + +Estimated effort: 3-6 days plus infrastructure window. + +Affected modules: + +- `docs/registry/RU_BUILD_RUNNER_READINESS_RU.md` +- `docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md` +- `docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md` +- `docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md` +- `scripts/build_release_evidence.sh` +- `scripts/check_release_evidence.sh` +- `scripts/verify_release_assets.sh` + +Acceptance criteria: + +- Controlled runner has documented OS, toolchain and access model. +- Release evidence includes source archive, binary archive, SBOM if available, + SHA256SUMS, Cargo metadata/tree, smoke logs and release manifest. +- `scripts/check_release_evidence.sh` passes on produced artifacts. +- GitHub CI is not described as release evidence. + +Validation steps: + +- Run release evidence script on controlled runner. +- Verify manifest and checksums. +- Record runner environment and commit SHA. + +Expected benefit: Production 1.0 release becomes reproducible and auditable. + +### P0-3. Disaster recovery restore proof + +Purpose: prove that repository backup can be restored before Production 1.0. + +Reason: `docs/registry/registry-evidence-manifest.json` records +`restore_tested=false`, and `docs/RESIDUAL_RISKS_RU.md` lists Gitea restore +test as open. + +Operational impact: validates recovery from source repository loss. + +Risk: high. Untested backup is not a recovery capability. + +Estimated effort: 2-4 days. + +Affected modules: + +- `docs/registry/GITEA_BACKUP_AND_RESTORE_RUNBOOK_RU.md` +- `docs/registry/registry-evidence-manifest.json` +- `scripts/registry_readiness_check.sh` +- `docs/PROJECT_STATUS_RU.md` +- `docs/RESIDUAL_RISKS_RU.md` + +Acceptance criteria: + +- Restore is performed on a separate host. +- SHA256 verification, logs, restored repository access and rollback notes are + captured. +- Manifest is updated only after evidence exists. +- No secrets are copied into repository documentation. + +Validation steps: + +- Execute restore runbook. +- Run registry readiness check after manifest update. +- Confirm restored repository clone and log evidence. + +Expected benefit: repository DR becomes proven, not only documented. + +### P0-4. Bounded retention for operational state and evidence + +Purpose: prevent disk exhaustion from state, queues, diagnostics, evidence and +forensics artifacts. + +Reason: DLP optional runtime docs explicitly state that historical DLP buckets +and artifacts may remain until a separate retention/cleanup procedure. Scripts +and diagnostics also create durable output. + +Operational impact: reduces outage risk on Proxmox/AW server/Windows state +paths. + +Risk: high. Disk exhaustion can stop ingestion, portal, ClickHouse or +ActivityWatch services. + +Estimated effort: 4-7 days. + +Affected modules: + +- `adk-rust/crates/aw-prune-local-state/` +- `scripts/detmir-full-diagnostics/` +- `scripts/detmir_dlp_warehouse_sync.sh` +- `aw-server/logrotate.conf` +- `windows/validate-deployment.ps1` +- `adk-rust/crates/aw-windows-telemetry/` +- `docs/DLP_OPTIONAL_RUNTIME_RU.md` +- `docs/DLP_RESOURCE_PROFILES_RU.md` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` + +Acceptance criteria: + +- Retention matrix lists roots, owners, max age, max size and dry-run behavior. +- Cleanup only touches allowlisted roots and refuses traversal/symlink escape. +- Active state is preserved. +- Disabled DLP buckets remain `SKIPPED`, not failure. + +Validation steps: + +- Run dry-run cleanup on fixture tree. +- Run apply mode on controlled temporary tree. +- Run operational smoke after cleanup. + +Expected benefit: long-running production operation has bounded disk behavior. + +### P0-5. Remove ClickHouse password exposure from process arguments + +Purpose: keep production credentials out of `ps`/process argv. + +Reason: `rg` confirmed `clickhouse-1c/ops/run_*.sh` wrappers pass +`--password "${CLICKHOUSE_PASSWORD}"`. + +Operational impact: improves secret handling for the 1C/ClickHouse contour. + +Risk: high. Local process listing can reveal ClickHouse credentials. + +Estimated effort: 2-4 days. + +Affected modules: + +- `clickhouse-1c/ops/run_ingest_cycle.sh` +- `clickhouse-1c/ops/run_manager_brief.sh` +- `clickhouse-1c/ops/run_recovery_brief.sh` +- `clickhouse-1c/ops/run_company_registry_bindings_refresh.sh` +- `clickhouse-1c/ops/run_company_intelligence_refresh.sh` +- `clickhouse-1c/ops/check_ingest_freshness.sh` +- `clickhouse-1c/ai/*.py` +- `clickhouse-1c/etl/*.py` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` + +Acceptance criteria: + +- Runtime wrappers no longer pass password through argv. +- Existing environment-based deployment remains backward compatible. +- Logs redact authentication failures. +- `ps` smoke proves password absence. + +Validation steps: + +- `bash -n clickhouse-1c/ops/*.sh` +- Run affected wrapper against a test or dry-run configuration. +- Verify `ps` output during execution. +- Run secret-pattern scan. + +Expected benefit: production secret exposure surface is reduced. + +### P0-6. Portal/worktime hot-path load gate + +Purpose: prevent Production 1.0 from regressing under report/prewarm load. + +Reason: `docs/PROJECT_STATUS_RU.md` records that full report/snapshot prewarm +can still be CPU/IO expensive. + +Operational impact: protects owner/operator portal, worktime reports and AW +query path. + +Risk: high. More users or more history can produce slow portal, stale data or +AW datastore pressure. + +Estimated effort: 1-2 weeks. + +Affected modules: + +- `scripts/operational-maturity-check.mjs` +- `scripts/awatch-production-hardening-smoke.mjs` +- `adk-rust/crates/detmir-portal/` +- `adk-rust/crates/worktime-api/` +- `adk-rust/crates/worktime-prewarm/` +- `adk-rust/crates/aw-contour-smoke/` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` + +Acceptance criteria: + +- Synthetic 5/20/50 user fixtures exist without production data. +- Gate records p95 latency, max RSS, query count and stale-cache behavior. +- Disconnected RDP sessions do not false-fail. +- `AW_DLP_ENABLED=false` semantics remain valid. +- Heavy load job is scheduled/advisory; blocking smoke remains fast. + +Validation steps: + +- Run offline load harness. +- Confirm configured p95/RSS ceilings. +- Run existing operational maturity smoke. + +Expected benefit: Production 1.0 has measurable performance safety. + +## P1 - Strongly recommended for the first Production 1.0 release train + +### P1-1. Align Rust toolchain across CI + +Purpose: remove compiler drift from blocking workflows. + +Reason: `rust-toolchain.toml` pins `1.94.0`, but several workflows install +floating `stable`. + +Operational impact: improves reproducibility between local, CI and release +contours. + +Risk: medium. Toolchain drift can create inconsistent warnings or binaries. + +Estimated effort: 1-2 days. + +Affected modules: + +- `rust-toolchain.toml` +- `.github/workflows/ci.yml` +- `.github/workflows/security.yml` +- `.github/workflows/coverage.yml` +- `.github/workflows/dependency-hygiene.yml` +- `.github/workflows/release-assets.yml` +- `docs/QUALITY_STATUS_RU.md` + +Acceptance criteria: + +- Blocking Rust workflows use the pinned toolchain. +- Nightly remains limited to advisory `cargo udeps`. +- Required check names do not change. + +Validation steps: + +- YAML syntax validation. +- Affected workflow dry review. +- Run relevant Rust checks if workflow commands change. + +Expected benefit: CI becomes more deterministic. + +### P1-2. Dependency warning baseline and future block policy + +Purpose: make dependency hygiene fail closed for new risk while preserving +current compatibility. + +Reason: `cargo deny` passes but reports 36 non-blocking `bans` warnings; +`cargo tree --duplicates --locked` reports duplicate roots; `serde_yaml` is +documented as deprecated in third-party license docs. + +Operational impact: reduces future supply-chain drift. + +Risk: medium. Uncontrolled duplicate/deprecated dependency growth increases +maintenance and security load. + +Estimated effort: 4-8 days. + +Affected modules: + +- `adk-rust/Cargo.toml` +- `adk-rust/Cargo.lock` +- `deny.toml` +- `.github/workflows/dependency-hygiene.yml` +- `docs/THIRD_PARTY_LICENSES_RU.md` +- `docs/QUALITY_STATUS_RU.md` + +Acceptance criteria: + +- Each existing warning is classified: keep, update, remove or defer. +- New duplicate/deprecated dependencies require documented exception. +- `cargo audit`, `cargo deny`, `cargo machete`, `cargo metadata` pass. + +Validation steps: + +- Run dependency hygiene pipeline. +- Verify policy failure on synthetic unapproved duplicate where practical. + +Expected benefit: dependency hygiene remains controlled after 1.0. + +### P1-3. Windows Rust validation parity + +Purpose: prove Rust validation is equivalent to current PowerShell validation +before reducing fallback reliance. + +Reason: `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md` lists remaining fallback +and runtime PowerShell paths, including validation and Hayabusa upload. + +Operational impact: keeps Windows/RDP production recoverable while reducing +runtime drift. + +Risk: medium. Premature fallback removal can break localized Windows Server +2019 recovery paths. + +Estimated effort: 1-2 weeks. + +Affected modules: + +- `adk-rust/crates/aw-windows-telemetry/` +- `windows/validate-deployment.ps1` +- `windows/ActivityWatch.Windows.Common.psm1` +- `windows/export-upload-hayabusa-to-aw-server.ps1` +- `ansible/deploy_aw_windows.yml` +- `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md` +- `docs/POWERSHELL_TO_RUST_ROADMAP_RU.md` + +Acceptance criteria: + +- Rust validation covers all current production validation sections. +- Localized Windows user/session handling is tested. +- Canary comparison between Rust and PowerShell reports is recorded. +- PowerShell remains documented rollback. + +Validation steps: + +- Run Rust validation against fixture and live canary. +- Run PowerShell validation against same host. +- Compare normalized reports. + +Expected benefit: Windows runtime maturity improves without breaking rollback. + +### P1-4. Install kit reproducibility and stale payload gate + +Purpose: ensure Windows installer payloads match repository and release +artifacts. + +Reason: repository contains install-kit tooling and large installer artifacts; +stale payloads can deploy old collectors while CI is green. + +Operational impact: safer Windows upgrades and rollback. + +Risk: medium. Mismatched install kit can create production drift. + +Estimated effort: 5-8 days. + +Affected modules: + +- `windows/installkit/innosetup/` +- `adk-rust/crates/check-install-kit-vs-repo/` +- `adk-rust/crates/rebuild-install-kit/` +- `adk-rust/crates/validate-install-kit/` +- `adk-rust/crates/verify-innosetup-installer/` +- `scripts/rebuild_install_kit.sh` +- `docs/INSTALL_KIT_RUNBOOK_RU.md` + +Acceptance criteria: + +- Installer manifest contains source commit and payload SHA256 values. +- Validation fails on stale collector payload. +- Existing Windows task names and config schema remain compatible. + +Validation steps: + +- Rebuild install kit on controlled runner. +- Run install-kit validators. +- Compare payload manifest with repository state. + +Expected benefit: Windows deployments become reproducible. + +### P1-5. Current-state documentation cleanup + +Purpose: prevent operators and reviewers from following stale status text. + +Reason: current docs include historical statuses, older required check names +and explicit pending sections that must be reconciled with active branch +protection and current workflows. + +Operational impact: reduces release and support mistakes. + +Risk: medium. Wrong runbook/status interpretation can cause incorrect release +decisions. + +Estimated effort: 3-6 days. + +Affected modules: + +- `docs/PROJECT_STATUS_RU.md` +- `docs/QUALITY_STATUS_RU.md` +- `docs/ROADMAP_CONFORMANCE_AUDIT_RU.md` +- `docs/BRANCH_PROTECTION_POLICY_RU.md` +- `docs/BRANCH_PROTECTION_EVIDENCE_RU.md` +- `README.md` +- `ROADMAP.md` + +Acceptance criteria: + +- Current docs list actual required check names. +- Historical docs are clearly marked historical. +- Registry/release claims remain conservative. +- No stale instructions contradict Production 1.0 guardrails. + +Validation steps: + +- Run registry readiness check. +- Run docs smoke/link validation. +- Run secret scan. + +Expected benefit: release process is less error-prone. + +### P1-6. Reviewed PR and release governance evidence + +Purpose: prove review discipline before Production 1.0. + +Reason: `docs/RESIDUAL_RISKS_RU.md` records first reviewed PR evidence as +pending even though CODEOWNERS, PR template and ruleset are present. + +Operational impact: improves change control for production releases. + +Risk: medium. Lack of review evidence weakens release governance. + +Estimated effort: 1-3 days after reviewer availability. + +Affected modules: + +- `.github/CODEOWNERS` +- `.github/pull_request_template.md` +- `docs/PR_REVIEW_WORKFLOW_RU.md` +- `docs/PR_REVIEW_EVIDENCE_RU.md` +- `docs/REVIEW_CHECKLIST_RU.md` +- `docs/RESIDUAL_RISKS_RU.md` + +Acceptance criteria: + +- At least one PR is reviewed and merged without bypass. +- Evidence records checks, reviewer, approval and merge path. +- Release branch review policy is documented. + +Validation steps: + +- Verify PR history and ruleset evidence. +- Run registry readiness check after evidence update. + +Expected benefit: Production 1.0 has visible governance evidence. + +## P2 - Engineering improvements for post-1.0 hardening + +### P2-1. Incremental decomposition of large modules + +Purpose: reduce review risk in the largest files without behavior changes. + +Reason: confirmed hotspots include `detmir-portal/src/main.rs` at 14200 lines, +`aw-windows-telemetry/src/main.rs` at 6411 lines, +`proxmox/tsj_guardian_bot.py` at 4610 lines, +`worktime-api/src/main.rs` at 3988 lines and +`ansible/deploy_aw_server.yml` at 3099 lines. + +Operational impact: easier reviews and lower regression risk. + +Risk: medium. Large files increase accidental coupling. + +Estimated effort: 2-4 weeks in small PRs. + +Affected modules: + +- `adk-rust/crates/detmir-portal/` +- `adk-rust/crates/aw-windows-telemetry/` +- `adk-rust/crates/worktime-api/` +- `proxmox/tsj_guardian_bot.py` +- `ansible/deploy_aw_server.yml` + +Acceptance criteria: + +- Only extract bounded domains. +- Public API, config, unit and task names remain unchanged. +- Tests before and after remain equivalent. + +Validation steps: + +- Targeted tests per extracted module. +- Full Rust pipeline for Rust changes. +- Ansible syntax/list-tasks parity for playbook changes. + +Expected benefit: maintainability improves without architecture rewrite. ### P2-2. Consolidate exact duplicate diagnostic scripts -- Почему нужно: две пары скриптов имеют identical SHA256, что создает риск - будущего drift при исправлениях. -- Риск: one copy fixed, bundled copy stale; diagnostics disagree. -- Ожидаемый эффект: single source of truth while preserving existing paths. -- Сложность: низкая. -- Объем работ: 1-2 инженерных дня. -- Затрагиваемые файлы: - `scripts/aw-contour-diag.sh`, - `scripts/detmir-full-diagnostics/aw-contour-diag.sh`, - `scripts/check_production_inventory_placeholders.sh`, - `scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh`, - `scripts/detmir-full-diagnostics/detmir-full-diagnostics.sh`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. -- Критерии приемки: - оба старых paths продолжают работать; - есть один canonical implementation или generated copy check; - CI/quality gate ловит future drift; - shellcheck/bash syntax pass. +Purpose: prevent script drift. -### P2-3. Split oversized Ansible deployment playbook without behavior change +Reason: SHA256 confirms exact duplicates: +`scripts/aw-contour-diag.sh` equals +`scripts/detmir-full-diagnostics/aw-contour-diag.sh`; and +`scripts/check_production_inventory_placeholders.sh` equals +`scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh`. -- Почему нужно: `ansible/deploy_aw_server.yml` имеет 3099 строк; это усложняет - review и безопасные partial changes. -- Риск: accidental deploy behavior changes in unrelated task blocks. -- Ожидаемый эффект: role/include structure with same task order and clearer - ownership. -- Сложность: средняя. -- Объем работ: 1-2 недели. -- Затрагиваемые файлы: - `ansible/deploy_aw_server.yml`, - `ansible/roles/`, - `ansible/group_vars/`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`. -- Критерии приемки: - `ansible-playbook --syntax-check` passes; - `--list-tasks` before/after is reviewed for ordering parity; - no default vars changed; - rollback is reverting includes to previous single playbook. +Operational impact: diagnostics remain consistent. -### P2-4. Bound 1C ingest memory and file processing profile +Risk: low-medium. Future fixes may land in one copy only. -- Почему нужно: `adk-rust/crates/aw-1c-ingest/src/main.rs` currently reads rows - into `Vec` and builds large JSON batches; acceptable for pilot, but risky for - larger exports. -- Риск: memory spikes, long transaction windows, ClickHouse insert amplification - on larger 1C files. -- Ожидаемый эффект: chunked processing where practical, explicit limits and - predictable failure mode for oversized files. -- Сложность: средняя-высокая. -- Объем работ: 1-2 недели. -- Затрагиваемые файлы: - `adk-rust/crates/aw-1c-ingest/src/main.rs`, - `clickhouse-1c/etl/config.yml`, - `clickhouse-1c/etl/config.example.yml`, - `clickhouse-1c/sql/`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, - `docs/QUALITY_STATUS_RU.md`. -- Критерии приемки: - synthetic large CSV/XLSX fixture documents max RSS and runtime; - oversized input fails closed with clear diagnostic; - existing small DetMir files produce identical rows; - ClickHouse insert batch size is configurable and bounded. +Estimated effort: 1-2 days. -### P2-5. Systematic command execution boundary audit +Affected modules: -- Почему нужно: `detmir-portal` already validates shell probe commands and - kills process groups on timeout, but other operational binaries also execute - shell/commands (`aw-slo-monitor`, `diag-and-manual-restart`, quality tooling). -- Риск: future config drift can reintroduce command injection, secrets in argv - or incomplete timeout cleanup. -- Ожидаемый эффект: documented command execution policy and shared tests for - allowlisted commands, timeouts, redaction and process-group cleanup. -- Сложность: средняя. -- Объем работ: 4-7 инженерных дней. -- Затрагиваемые файлы: - `adk-rust/crates/detmir-portal/src/main.rs`, - `adk-rust/crates/detmir-portal/src/production/limits.rs`, - `adk-rust/crates/aw-slo-monitor/src/main.rs`, - `adk-rust/crates/diag-and-manual-restart/src/main.rs`, - `adk-rust/crates/quality-gate/src/main.rs`, - `docs/SECURITY_OPERATIONS_RUNBOOK_RU.md`. -- Критерии приемки: - every runtime command source is classified as constant, allowlisted config or - operator-only diagnostic; - tests reject shell control operators where config-driven; - timeout tests verify child/grandchild cleanup; - logs never include secrets from env or args. +- `scripts/aw-contour-diag.sh` +- `scripts/detmir-full-diagnostics/aw-contour-diag.sh` +- `scripts/check_production_inventory_placeholders.sh` +- `scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh` +- `scripts/detmir-full-diagnostics/detmir-full-diagnostics.sh` -### P2-6. Install kit artifact reproducibility and stale payload checks +Acceptance criteria: -- Почему нужно: install kit is critical for Windows deployment, includes large - LFS artifacts, and must not silently ship stale payloads. -- Риск: Windows production receives an older collector or mismatched config - while repo checks pass. -- Ожидаемый эффект: deterministic installer evidence and stronger - install-kit-vs-repo validation. -- Сложность: средняя. -- Объем работ: 5-8 инженерных дней. -- Затрагиваемые файлы: - `windows/installkit/innosetup/`, - `adk-rust/crates/check-install-kit-vs-repo/`, - `adk-rust/crates/rebuild-install-kit/`, - `adk-rust/crates/validate-install-kit/`, - `adk-rust/crates/verify-innosetup-installer/`, - `scripts/rebuild_install_kit.sh`, - `docs/INSTALL_KIT_RUNBOOK_RU.md`. -- Критерии приемки: - installer manifest lists exact source commit and payload SHA256; - validation fails on stale collector payload; - generated artifacts are not confused with source files; - Windows task names and config schema remain backward compatible. +- Existing paths continue to work. +- One implementation is canonical or drift check is enforced. +- Shell syntax and shellcheck pass. -### P2-7. Standardize operational metrics contract across Rust daemons +Validation steps: -- Почему нужно: individual services expose useful fields, but there is no - consistent minimal metrics/diagnostics contract for every long-running Rust - daemon. -- Риск: incident response depends on service-specific knowledge; regressions in - one daemon are harder to compare with another. -- Ожидаемый эффект: consistent `/health`, structured log fields and optional - metrics snapshots for status, cache, queue, retry, timeout and resource - pressure. -- Сложность: средняя. -- Объем работ: 2-3 недели staged by service. -- Затрагиваемые файлы: - `adk-rust/crates/aw-rus-healthd/`, - `adk-rust/crates/worktime-api/`, - `adk-rust/crates/detmir-portal/`, - `adk-rust/crates/dlp-*`, - `adk-rust/crates/aw-1c-ingest/`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, - `grafana/`. -- Критерии приемки: - minimal contract documented; - each daemon reports version/build, config profile, degraded reason and queue - pressure where applicable; - existing endpoints remain backward compatible; - Grafana/dashboard changes are additive. +- `bash -n` on affected scripts. +- Run diagnostic wrapper in dry-run/smoke mode. -## P3 - долгосрочные улучшения +Expected benefit: lower maintenance overhead. -### P3-1. Coverage thresholds after baseline review +### P2-3. Bound 1C ingest memory profile -- Почему нужно: coverage workflow exists, but roadmap explicitly says threshold - should be added after baseline review. -- Риск: coverage can silently decline while CI remains green. -- Ожидаемый эффект: gradual coverage floor without slowing blocking jobs. -- Сложность: средняя. -- Объем работ: 1-2 недели. -- Затрагиваемые файлы: - `.github/workflows/coverage.yml`, - `adk-rust/`, - `docs/QUALITY_STATUS_RU.md`, - `docs/REVIEW_CHECKLIST_RU.md`. -- Критерии приемки: - initial threshold is based on measured baseline, not arbitrary target; - critical crates get stricter per-crate goals; - generated/fixture code excluded consistently; - threshold moves from advisory to blocking only after stable history. +Purpose: make 1C ingest safer for larger files. -### P3-2. Russian OS compatibility matrix +Reason: `adk-rust/crates/aw-1c-ingest/src/main.rs` reads rows into `Vec` and +builds large JSON batches; acceptable now, but risky as export volume grows. -- Почему нужно: roadmap keeps Russian OS compatibility as planned validation. -- Риск: deployment assumptions can fail on target OS variants only after a - customer deployment attempt. -- Ожидаемый эффект: explicit supported/unsupported OS matrix and installation - evidence. -- Сложность: средняя-высокая. -- Объем работ: 2-4 недели depending on available test hosts. -- Затрагиваемые файлы: - `docs/registry/`, - `docs/INSTALLATION.md`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, - `ansible/`, - `windows/`, - `windows/installkit/`. -- Критерии приемки: - matrix lists OS version, role, test date, result and limitations; - unsupported combinations are documented explicitly; - no production defaults are changed only for compatibility claims. +Operational impact: improves ClickHouse ingestion predictability. -### P3-3. Reduce single-maintainer operational risk +Risk: medium. Large files can create memory spikes and long insert windows. -- Почему нужно: `docs/RESIDUAL_RISKS_RU.md` identifies one-main-developer risk - and pending reviewed PR evidence. -- Риск: incident response and release continuity depend too much on one person. -- Ожидаемый эффект: documented ownership, review evidence and handoff paths. -- Сложность: организационная, средняя. -- Объем работ: 1-3 месяца part-time. -- Затрагиваемые файлы: - `.github/CODEOWNERS`, - `.github/pull_request_template.md`, - `docs/PR_REVIEW_WORKFLOW_RU.md`, - `docs/PR_REVIEW_EVIDENCE_RU.md`, - `docs/REVIEW_CHECKLIST_RU.md`, - `docs/RESIDUAL_RISKS_RU.md`. -- Критерии приемки: - at least one reviewed PR merged without bypass; - release branch review policy documented; - emergency maintainer handoff checklist exists; - residual risk status updated only with real evidence. +Estimated effort: 1-2 weeks. -### P3-4. Historical docs and naming hygiene +Affected modules: -- Почему нужно: older docs still reference staged statuses, older check names - or public-readiness context; this increases cognitive load for maintainers. -- Риск: operators follow outdated docs or confuse GitHub public mirror checks - with Russian release evidence. -- Ожидаемый эффект: clearer current-state docs and archived historical records. -- Сложность: низкая-средняя. -- Объем работ: 3-6 инженерных дней. -- Затрагиваемые файлы: - `docs/PROJECT_STATUS_RU.md`, - `docs/ROADMAP_CONFORMANCE_AUDIT_RU.md`, - `docs/QUALITY_STATUS_RU.md`, - `docs/registry/`, - `README.md`, - `ROADMAP.md`. -- Критерии приемки: - current-state docs contain current required check names; - historical docs are marked historical and not operational runbooks; - no product claims are strengthened without evidence; - broken/stale links are corrected. +- `adk-rust/crates/aw-1c-ingest/src/main.rs` +- `clickhouse-1c/etl/config.yml` +- `clickhouse-1c/etl/config.example.yml` +- `clickhouse-1c/sql/` -### P3-5. Long-horizon capacity sizing guide +Acceptance criteria: -- Почему нужно: current DetMir production is small; future deployments need - sizing guidance for AW SQLite, ClickHouse, Influx/Grafana, DLP light profile - and Windows collector load. -- Риск: deployments scale by guesswork and overload the small-Proxmox pattern. -- Ожидаемый эффект: sizing table by users/events/day, retention, disk, CPU/RAM - and optional security contours. -- Сложность: средняя. -- Объем работ: 2-4 недели after P1 load harness data exists. -- Затрагиваемые файлы: - `docs/DLP_RESOURCE_PROFILES_RU.md`, - `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, - `docs/DETMIR_CURRENT_STATE_RU.md`, - `grafana/`, - `scripts/operational-maturity-check.mjs`. -- Критерии приемки: - guide is based on measured data, not estimates only; - separate profiles exist for 5, 20, 50 and 100 users; - optional DLP/Hayabusa/Velociraptor resource costs are explicit; - default production profile remains conservative. +- Oversized input fails closed with clear diagnostic. +- Batch size is bounded and configurable. +- Existing small DetMir files produce identical output. -## Не включено, потому что уже реализовано +Validation steps: -- Включение dependency hygiene automation: `cargo machete`, `cargo deny`, - `cargo audit`, `cargo tree --duplicates`, advisory `cargo udeps` уже есть в - workflow. -- Удаление текущих unused Rust dependencies: текущий `cargo machete - --with-metadata` не нашел unused dependencies. -- DetMir DLP light/core-only guardrails: runtime control, resource profiles, - load guard and skipped-bucket semantics already implemented. -- Security Finding Inbox and optional Hayabusa/Velociraptor findings path: - schema, portal/API and fail-closed executor path already documented as - optional and approval-driven. -- Operational maturity smoke workflow: already exists and explicitly avoids - heavy DLP/Loki/always-on Velociraptor. -- Branch protection/CODEOWNERS/PR checklist basics: already documented; the - remaining work is evidence and regular use, not creating the mechanism. +- Add synthetic large CSV/XLSX fixture. +- Run targeted Rust tests. +- Measure max RSS on fixture. -## Проверки, использованные для аудита +Expected benefit: safer scaling of 1C analytics. -```bash -cd /mnt/usb_hdd2/Projects/ActivityWatch-Russian/adk-rust -export CARGO_TARGET_DIR=/home/igor/.cache/detmir-adk-rust-target -cargo metadata --locked --format-version 1 -cargo tree --duplicates --locked -cargo audit --deny warnings -cargo deny --manifest-path Cargo.toml check --config ../deny.toml --hide-inclusion-graph --show-stats -cargo machete --with-metadata +### P2-4. Command execution boundary audit -cd /mnt/usb_hdd2/Projects/ActivityWatch-Russian -rg -n "dtolnay/rust-toolchain@(stable|[0-9])|rust-toolchain" .github/workflows rust-toolchain.toml -rg -n "CLICKHOUSE_PASSWORD|--password" clickhouse-1c/ops clickhouse-1c/ai clickhouse-1c/etl -sha256sum scripts/aw-contour-diag.sh scripts/detmir-full-diagnostics/aw-contour-diag.sh -sha256sum scripts/check_production_inventory_placeholders.sh scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh -wc -l adk-rust/crates/detmir-portal/src/main.rs adk-rust/crates/aw-windows-telemetry/src/main.rs proxmox/tsj_guardian_bot.py adk-rust/crates/worktime-api/src/main.rs ansible/deploy_aw_server.yml -``` +Purpose: standardize shell/command execution safety. + +Reason: `detmir-portal` already validates shell probe commands and tests +process-tree timeout cleanup, while other operational tools also execute +commands. + +Operational impact: prevents future command injection or timeout regressions. + +Risk: medium. Config-driven command execution must remain fail-closed. + +Estimated effort: 4-7 days. + +Affected modules: + +- `adk-rust/crates/detmir-portal/src/main.rs` +- `adk-rust/crates/detmir-portal/src/production/limits.rs` +- `adk-rust/crates/aw-slo-monitor/src/main.rs` +- `adk-rust/crates/diag-and-manual-restart/src/main.rs` +- `adk-rust/crates/quality-gate/src/main.rs` + +Acceptance criteria: + +- Runtime command sources are classified. +- Config-driven commands reject shell control operators where applicable. +- Timeout tests cover child/grandchild cleanup. +- Logs do not expose secrets. + +Validation steps: + +- Targeted Rust tests. +- Clippy for affected crates. +- Secret scan. + +Expected benefit: stronger fail-closed security posture. + +## P3 - Long-term improvements after Production 1.0 + +### P3-1. Russian OS compatibility matrix + +Purpose: document supported and unsupported target OS combinations. + +Reason: `ROADMAP.md` lists Russian OS compatibility validation as planned. + +Operational impact: reduces deployment surprises for new customers. + +Risk: low-medium for current DetMir, higher for wider distribution. + +Estimated effort: 2-4 weeks depending on test hosts. + +Affected modules: + +- `docs/registry/` +- `docs/INSTALLATION.md` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` +- `ansible/` +- `windows/` +- `windows/installkit/` + +Acceptance criteria: + +- Matrix lists OS, role, test date, result and limitations. +- Unsupported combinations are explicit. +- No production defaults are changed just for claims. + +Validation steps: + +- Run install/deploy smoke per OS. +- Record evidence paths. + +Expected benefit: clearer deployment support boundary. + +### P3-2. Coverage threshold after baseline review + +Purpose: prevent coverage decline after baseline stabilizes. + +Reason: `docs/QUALITY_STATUS_RU.md` and `ROADMAP.md` state coverage threshold +is not enforced yet. + +Operational impact: improves long-term regression resistance. + +Risk: low for current production if kept advisory first. + +Estimated effort: 1-2 weeks. + +Affected modules: + +- `.github/workflows/coverage.yml` +- `docs/QUALITY_STATUS_RU.md` +- `docs/REVIEW_CHECKLIST_RU.md` +- selected `adk-rust/` crates + +Acceptance criteria: + +- Initial threshold is based on measured baseline. +- Threshold starts advisory and becomes blocking only after stable history. +- Generated/fixture code exclusions are documented. + +Validation steps: + +- Run coverage workflow locally or in CI. +- Compare summary to baseline. + +Expected benefit: gradual improvement in test discipline. + +### P3-3. Capacity sizing guide from measured data + +Purpose: provide measured sizing guidance beyond the current 5-user DetMir +deployment. + +Reason: current production is small; future deployments need measured guidance +for AW SQLite, ClickHouse, Grafana, DLP light profile and Windows collector +load. + +Operational impact: safer planning for larger deployments. + +Risk: low for current production, medium for growth. + +Estimated effort: 2-4 weeks after P0 load gate data exists. + +Affected modules: + +- `docs/SIZING_GUIDE_RU.md` +- `docs/DLP_RESOURCE_PROFILES_RU.md` +- `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md` +- `grafana/` +- `scripts/operational-maturity-check.mjs` + +Acceptance criteria: + +- Profiles exist for 5, 20, 50 and 100 users. +- Optional DLP/Hayabusa/Velociraptor resource costs are explicit. +- Guidance is based on measured harness output. + +Validation steps: + +- Run synthetic capacity scenarios. +- Update sizing doc with observed p95/RSS/storage data. + +Expected benefit: production planning becomes evidence-based. + +## Not Proposed Because Already Implemented + +- Rust-first runtime direction. +- Public mirror CI, security scan and dependency review. +- `cargo audit`, `cargo deny`, `cargo machete`, `cargo tree --duplicates` and + advisory `cargo udeps` workflow coverage. +- DLP `core_only/disabled` production guardrails and load guard. +- Optional Security Finding Inbox / Hayabusa / Velociraptor separation from + Workforce hot path. +- Operational maturity offline harness. +- Branch protection, CODEOWNERS, PR template and review checklist mechanisms. +- Unused dependency cleanup: current `cargo machete --with-metadata` reports no + unused dependencies. diff --git a/PRODUCTION_READINESS_REPORT.md b/PRODUCTION_READINESS_REPORT.md new file mode 100644 index 0000000..f77cbea --- /dev/null +++ b/PRODUCTION_READINESS_REPORT.md @@ -0,0 +1,332 @@ +# TASK_065: Production readiness report + +Audit date: 2026-07-01. + +Scope: repository-level engineering and operational audit for AWatch-rus +Production 1.0 readiness. + +Production context: the project is already deployed in a small company +environment with approximately 5 RDP users. Stability, rollback, diagnostics +and maintainability have priority over new functionality. + +## Method + +The audit used repository evidence only. No readiness claim in this report is +based on an unverified assumption. + +Reviewed evidence: + +- `README.md`, `CONTRIBUTING.md`, `SECURITY.md`, `ROADMAP.md`. +- `docs/PROJECT_STATUS_RU.md`, `docs/RESIDUAL_RISKS_RU.md`, + `docs/QUALITY_STATUS_RU.md`, `docs/OPERATIONS_VALIDATION_RUNBOOK_RU.md`, + `docs/DLP_OPTIONAL_RUNTIME_RU.md`, `docs/DLP_RESOURCE_PROFILES_RU.md`, + `docs/POWERSHELL_SCRIPT_STATUS_MATRIX_RU.md`. +- `docs/registry/*` including `registry-evidence-manifest.json`. +- `.github/workflows/*`. +- `scripts/*`, `scripts/detmir-full-diagnostics/*`. +- `adk-rust/` workspace manifests and crate tree. +- `ansible/`, `aw-server/`, `windows/`, `proxmox/`, `clickhouse-1c/`, + `grafana/`, `pfsense/`. +- `configs/operational-maturity-contract.json`. + +Commands used for evidence gathering: + +```bash +git ls-files | wc -l +find adk-rust/crates -maxdepth 2 -name Cargo.toml +cargo metadata --locked --format-version 1 +cargo audit --deny warnings +cargo machete --with-metadata +cargo tree --duplicates --locked +cargo deny check --config ../deny.toml --hide-inclusion-graph --show-stats +wc -l adk-rust/crates/detmir-portal/src/main.rs adk-rust/crates/aw-windows-telemetry/src/main.rs proxmox/tsj_guardian_bot.py adk-rust/crates/worktime-api/src/main.rs ansible/deploy_aw_server.yml +sha256sum scripts/aw-contour-diag.sh scripts/detmir-full-diagnostics/aw-contour-diag.sh scripts/check_production_inventory_placeholders.sh scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh +rg -n "CLICKHOUSE_PASSWORD|--password|password" clickhouse-1c/ops clickhouse-1c/ai clickhouse-1c/etl +``` + +## Implemented Capability Inventory + +### Core Runtime + +- Rust workspace under `adk-rust/` with 58 crates. +- ActivityWatch checks and wrappers: `check-aw-data`, `check-aw-full`, + `aw-health-check`, `aw-rus-healthd`, `aw-slo-monitor`. +- DetMir portal: Rust HTML/API portal with contracts, metrics, readiness, + reports, operator/manager/security views and role-based access logic. +- Worktime stack: `worktime-api`, `worktime-prewarm`, + `worktime-influx-exporter`, `worktime-autoheal`, + `worktime-ui-bridge`, `rdp-worktime-report`. +- Windows telemetry: `aw-windows-telemetry` for collectors, validation, + file-1C upload, DLP evidence sync and collector guard paths. +- Release/install tooling: `rebuild-install-kit`, `validate-install-kit`, + `verify-innosetup-installer`, `check-install-kit-vs-repo`. + +### Workforce / 1C / ClickHouse + +- `aw-workforce-ingest` and `aw-1c-ingest` exist for workforce and 1C + ingestion paths. +- `clickhouse-1c/` includes ETL, SQL, Grafana provisioning and operational + wrappers. +- Grafana dashboards are version-controlled under `grafana/` and related + ClickHouse/Grafana directories. + +### Security / DLP / Forensics + +- DLP server-side helpers exist as Rust crates: + `dlp-policy-engine`, `dlp-case-management`, `dlp-compliance`, + `dlp-aggregator`, `dlp-health-check`, `dlp-content-analyzer`, + exporters/senders. +- DLP production runtime is intentionally conservative: + `core_only/disabled` default with documented `light` profile and load guard. +- Security Finding Inbox and Hayabusa/Velociraptor findings paths are optional + and separated from Workforce hot path. +- Hayabusa tooling exists under `hayabusa-tools` and `aw-server/hayabusa/`. + +### Deployment / Operations + +- Ansible deployment exists for server and Windows contours. +- Windows deployment and recovery scripts exist under `windows/`. +- Proxmox/pfSense support assets exist under `proxmox/` and `pfsense/`. +- Operational wrappers exist at root and under `scripts/`. +- `scripts/operational-maturity-check.mjs` validates API compatibility, + fixtures, fault injection, bounded load, config, systemd, ClickHouse + migration and observability contracts. + +### Governance / Release + +- Public CI workflows exist. +- Security workflow includes cargo audit, cargo deny, secret pattern scan and + dependency review. +- Dependency hygiene workflow includes cargo metadata, machete, duplicates, + audit, deny and advisory udeps. +- Registry-readiness documentation exists under `docs/registry/`. +- CODEOWNERS, PR template, review checklist and branch protection evidence docs + exist. + +## Confirmed Gap Analysis + +### Technical Debt + +- Large modules increase review risk: + - `adk-rust/crates/detmir-portal/src/main.rs`: 14200 lines. + - `adk-rust/crates/aw-windows-telemetry/src/main.rs`: 6411 lines. + - `proxmox/tsj_guardian_bot.py`: 4610 lines. + - `adk-rust/crates/worktime-api/src/main.rs`: 3988 lines. + - `ansible/deploy_aw_server.yml`: 3099 lines. +- PowerShell fallback remains necessary and documented; it is not dead code, but + it increases parity and validation burden. + +### Duplicated Logic + +- Exact duplicate scripts confirmed by SHA256: + - `scripts/aw-contour-diag.sh` + - `scripts/detmir-full-diagnostics/aw-contour-diag.sh` + - `scripts/check_production_inventory_placeholders.sh` + - `scripts/detmir-full-diagnostics/check_production_inventory_placeholders.sh` + +### Outdated Or Pending Documentation + +- `docs/PROJECT_STATUS_RU.md` still records first reviewed PR evidence as + pending and contains historical required check names. +- `docs/RESIDUAL_RISKS_RU.md` records Gitea restore test, build-runner, + release evidence and legal package as open. +- `ROADMAP.md` records coverage threshold and Russian OS compatibility as + planned, not complete. + +### Obsolete Or Deprecated Components + +- Some Ansible DLP roles are explicitly marked deprecated because they deployed + old service paths. +- `serde_yaml 0.9.34+deprecated` is documented as a medium third-party risk. +- Legacy scripts remain as fallback/reference; they should not be removed + without parity gates. + +### Missing Operational Checks + +- No confirmed repository check currently proves production binary SHA parity + across all actually running units/timers/tasks and local release artifacts. +- Retention/cleanup policy for long-lived state/evidence/diagnostic output is + not yet complete. +- Existing operational maturity bounded load is useful, but does not yet cover + Production 1.0 scale scenarios for 5/20/50 users and portal/worktime prewarm. + +### Missing Tests + +- Load regression tests for portal/worktime full report and prewarm hot paths + are not yet sufficient for Production 1.0 scale confidence. +- Windows Rust validation parity still needs canary evidence against the + PowerShell validation path. + +### Security Gaps + +- ClickHouse/1C ops wrappers pass `CLICKHOUSE_PASSWORD` via `--password`, which + exposes secrets in process argv. +- `cargo deny` passes but currently permits non-blocking duplicate/wildcard + dependency warnings. This is not an immediate vulnerability, but it needs a + Production 1.0 baseline. + +## Readiness Scores + +Scoring scale: + +- 90-100: production-ready with evidence. +- 75-89: strong, but with bounded gaps. +- 60-74: usable in current production, but not yet 1.0 release-grade. +- below 60: material blocker. + +| Category | Score | Justification | +|---|---:|---| +| Architecture status | 82 | Rust-first runtime, documented boundaries and conservative DLP separation exist. Large modules remain maintainability risk. | +| Repository health | 80 | 910 tracked files, clear ownership areas and runbooks. Some duplicate scripts and historical docs remain. | +| Dependency health | 78 | Audit and machete pass; 349 packages; cargo deny exits 0. Remaining deny warnings and duplicate roots need baseline policy. | +| CI health | 82 | CI/security/coverage/dependency/operational workflows exist. Toolchain drift remains between pinned `1.94.0` and floating `stable`. | +| Operational maturity | 84 | Offline operational maturity harness covers compatibility, fixtures, fault injection, bounded load, config and observability. Production-scale load gate still needed. | +| Documentation status | 78 | Extensive docs and runbooks exist. Current-state docs need cleanup around historical statuses and pending evidence. | +| Security status | 76 | Secret scan, audit, deny and conservative claims exist. ClickHouse password-in-argv is a direct hygiene gap. | +| Testing status | 80 | Full Rust pipeline recently passed and operational smokes exist. Missing scale/load and Windows parity evidence remain. | +| Deployment readiness | 76 | Ansible, Windows install kit, runbooks and wrappers exist. Install kit stale-payload gate and production binary parity still missing. | +| Upgrade readiness | 72 | Release scripts and rollback docs exist, but controlled release evidence and binary parity are not proven. | +| Recovery readiness | 70 | Recovery runbooks and backups exist; Gitea restore test is not done. | +| Configuration validation | 82 | Operational maturity validates JSON/YAML/systemd/ClickHouse files; production inventory placeholder checks exist. Coverage must be extended to retention/binary parity. | +| Observability | 84 | Metrics contract and operational maturity observability checks exist; capacity metrics need scale scenarios. | +| Support readiness | 78 | Many runbooks exist and DetMir guardrails are strong; docs need current-state cleanup for 1.0. | +| Maintainability | 72 | Strong tests and Rust-first direction, but large files and fallback parity increase maintenance cost. | + +Overall Production Readiness Score: **78 / 100**. + +Recommended release decision: **not yet Production 1.0**. The project is fit +for the current small production/pilot environment with conservative runtime +guardrails, but Production 1.0 should wait until P0 blockers in +`DEVELOPMENT_PLAN_NEXT.md` are closed and evidenced. + +## Risk Assessment + +### High Risk + +1. Production binary drift + - Probability: medium. + - Impact: high. + - Description: running binaries may not match reviewed release artifacts. + - Mitigation: implement production binary parity gate. + - Complexity: medium. + +2. Missing controlled release evidence + - Probability: high. + - Impact: high. + - Description: GitHub mirror validation is not release evidence. + - Mitigation: controlled runner release evidence build. + - Complexity: medium. + +3. Untested repository restore + - Probability: medium. + - Impact: high. + - Description: backup exists, but restore is not proven. + - Mitigation: separate-host restore drill. + - Complexity: medium. + +4. Unbounded operational artifacts + - Probability: medium. + - Impact: high. + - Description: state/evidence/diagnostic artifacts can accumulate and fill + disks. + - Mitigation: allowlisted retention and cleanup policy. + - Complexity: medium. + +5. ClickHouse password in process argv + - Probability: high on affected scripts. + - Impact: high. + - Description: local process listing can expose credentials. + - Mitigation: remove password from argv and verify with `ps`. + - Complexity: low-medium. + +6. Portal/worktime hot-path overload + - Probability: medium. + - Impact: high. + - Description: full report/snapshot prewarm remains documented as CPU/IO + expensive. + - Mitigation: Production 1.0 load gate with synthetic datasets. + - Complexity: medium-high. + +### Medium Risk + +1. CI toolchain drift + - Probability: medium. + - Impact: medium. + - Mitigation: align workflows to pinned toolchain. + - Complexity: low. + +2. Dependency warning drift + - Probability: medium. + - Impact: medium. + - Mitigation: baseline current warnings and block new unapproved warnings. + - Complexity: medium. + +3. Windows validation parity gap + - Probability: medium. + - Impact: medium-high. + - Mitigation: Rust/PowerShell canary comparison. + - Complexity: medium. + +4. Install kit stale payload + - Probability: medium. + - Impact: medium-high. + - Mitigation: install kit manifest and validation gate. + - Complexity: medium. + +5. Documentation drift + - Probability: high. + - Impact: medium. + - Mitigation: current-state cleanup and historical labeling. + - Complexity: low-medium. + +6. Large module maintainability + - Probability: high. + - Impact: medium. + - Mitigation: incremental extraction with tests. + - Complexity: medium-high. + +### Low Risk + +1. Exact duplicate diagnostic scripts + - Probability: medium. + - Impact: low-medium. + - Mitigation: canonical implementation or drift check. + - Complexity: low. + +2. Coverage threshold not yet enforced + - Probability: medium. + - Impact: low for current production, medium long-term. + - Mitigation: baseline review, advisory threshold, later blocking policy. + - Complexity: medium. + +3. Russian OS compatibility not yet matrixed + - Probability: low for current DetMir, medium for wider distribution. + - Impact: medium for new deployments. + - Mitigation: compatibility matrix with evidence. + - Complexity: medium-high. + +## Known Limitations + +- The audit did not claim legal readiness, certification or registry + submission completion. +- The audit did not validate live production services during this documentation + update. +- Heavy DLP, Loki and always-on Velociraptor are intentionally not required for + Production 1.0. +- PowerShell fallback remains a supported rollback/support path until parity + evidence allows retirement. + +## Release Readiness Summary + +Production 1.0 should be blocked on: + +1. production binary parity evidence; +2. controlled release evidence build; +3. Gitea restore drill; +4. bounded retention/cleanup; +5. ClickHouse password argv fix; +6. portal/worktime hot-path load gate. + +After these are complete, the project can reasonably move from current small +production/pilot readiness to Production 1.0 readiness, assuming validation +passes and no new runtime regressions are introduced.