docs(architecture): track orchestration map

2026-06-30 13:13:15 +03:00
parent b73730a5e9
commit 81e927bf42
6 changed files with 276 additions and 13 deletions
@@ -200,6 +200,8 @@ collectors.

 Модульная схема комплекса с GitHub/Gitea-viewable Mermaid-графами:
 [docs/MODULE_ARCHITECTURE_GRAPH_RU.md](docs/MODULE_ARCHITECTURE_GRAPH_RU.md).
+Карта orchestration entrypoints:
+[docs/ORCHESTRATION_MAP_RU.md](docs/ORCHESTRATION_MAP_RU.md).

 ## Если дашборд пустой

@@ -363,6 +365,7 @@ collectors.
 - [Сторонние лицензии](THIRD_PARTY_LICENSES_RU.md)
 - [Архитектура](docs/ARCHITECTURE_RU.md)
 - [Модульная схема комплекса](docs/MODULE_ARCHITECTURE_GRAPH_RU.md)
+- [Карта оркестрации](docs/ORCHESTRATION_MAP_RU.md)
 - [Установка](docs/INSTALL_RU.md)
 - [Руководство администратора](docs/ADMIN_GUIDE_RU.md)
 - [Руководство оператора](docs/OPERATOR_GUIDE_RU.md)
@@ -7,6 +7,13 @@
 - централизованное развёртывание Windows/RDP collector'ов по WinRM;
 - развёртывание внешнего pfSense poller'а на Debian/Ubuntu utility VM.

+Актуальная карта связи playbooks, scripts, systemd timers, Windows Scheduled
+Tasks и модулей комплекса ведётся в
+[`docs/ORCHESTRATION_MAP_RU.md`](../docs/ORCHESTRATION_MAP_RU.md). При
+добавлении или переименовании orchestration entrypoint обновляйте карту и
+проверяйте её через `bash scripts/check_orchestration_map.sh` из корня
+репозитория.
+
 ## Файлы

 - `ansible/deploy_aw_server.yml` — основной playbook для уже существующего Debian/CT host.
@@ -211,6 +211,7 @@ stateDiagram-v2
 ```mermaid
 flowchart TD
  DailyCheck["daily / weekly contour checks"] --> Healthd["aw-rus-healthd"]
+  Orchestration["Ansible and scripts<br/>deploy / validate / support"] --> DailyCheck
  Healthd --> AwCheck["AW server and bucket freshness"]
  Healthd --> WorktimeCheck["Worktime API health"]
  Healthd --> ClickHouseCheck["ClickHouse health"]
@@ -244,7 +245,25 @@ GitHub полезен для публичной проверяемости: PR,
 Но для российского реестрового release evidence нужен отдельный российский
 контур сборки и хранения артефактов.

-## 10. Где смотреть руками
+## 10. Оркестрация и поддержание актуальности
+
+Оркестрационные entrypoints отдельно зафиксированы в
+[docs/ORCHESTRATION_MAP_RU.md](ORCHESTRATION_MAP_RU.md). Этот документ
+связывает архитектурные модули с Ansible playbooks, systemd timers, Windows
+Scheduled Tasks и read-only check scripts.
+
+В репозитории есть guard:
+
+```bash
+bash scripts/check_orchestration_map.sh
+```
+
+Он не ходит в production и не меняет runtime. Его задача - проверить, что
+карта оркестрации ссылается на реальные playbooks/scripts и содержит
+обязательные safety-маркеры: DLP optional mode, Hayabusa/Velociraptor boundary,
+approval gate для containment и разделение GitHub/Gitea release контуров.
+
+## 11. Где смотреть руками

 | Что проверить | Где смотреть |
 |---|---|
@@ -258,13 +277,11 @@ GitHub полезен для публичной проверяемости: PR,
 | Runtime checks | `aw-rus-healthd`, contour check scripts, portal status |
 | Код и evidence процесса | GitHub PR/issues, private Gitea mirror |

-## 11. Связанные документы
+## 12. Связанные документы

 - [docs/ARCHITECTURE_RU.md](ARCHITECTURE_RU.md)
 - [docs/UNIFIED_OPERATING_MODEL_RU.md](UNIFIED_OPERATING_MODEL_RU.md)
- [docs/WORKFORCE_OPERATIONS_MODEL_RU.md](WORKFORCE_OPERATIONS_MODEL_RU.md)
+- [docs/ORCHESTRATION_MAP_RU.md](ORCHESTRATION_MAP_RU.md)
 - [docs/GRAFANA_DASHBOARDS_RU.md](GRAFANA_DASHBOARDS_RU.md)
- [docs/SECURITY_FINDING_INBOX_RU.md](SECURITY_FINDING_INBOX_RU.md)
- [docs/LOW_COST_SIGMA_HAYABUSA_VELOCIRAPTOR_ADDON_RU.md](LOW_COST_SIGMA_HAYABUSA_VELOCIRAPTOR_ADDON_RU.md)
 - [docs/DLP_OPTIONAL_RUNTIME_RU.md](DLP_OPTIONAL_RUNTIME_RU.md)
 - [docs/PRODUCTION_READINESS_RU.md](PRODUCTION_READINESS_RU.md)
@@ -0,0 +1,131 @@
+# AWatch-rus / DetMir: карта оркестрации
+
+Статус: актуальная карта deploy/check orchestration для просмотра в
+GitHub/Gitea.
+
+Документ фиксирует, какие playbooks, systemd units и scripts управляют
+модулями комплекса. Это не инструкция выполнять production deploy без окна
+работ: большинство playbooks меняют сервисы, scheduled tasks или dashboards.
+
+## 1. Общий порядок оркестрации
+
+```mermaid
+flowchart TD
+  Inventory["inventory.ini<br/>group_vars/*.yml<br/>private env vars"] --> Full["install_full_stack.yml"]
+  Full --> Server["deploy_aw_server.yml<br/>AW server and Rust services"]
+  Full --> Windows["deploy_aw_windows.yml<br/>RDP/Windows collectors"]
+  Full --> Gateway["deploy_proxmox_web_gateway.yml<br/>operator gateway"]
+  Full --> Grafana["deploy_grafana_dashboards.yml<br/>Grafana dashboards"]
+  Full --> Checks["deploy_grafana_check.yml<br/>dashboard/data freshness check"]
+
+  Server --> Portal["deploy_detmir_portal.yml<br/>portal and evidence API"]
+  Server --> Worktime["worktime API<br/>Influx exporter<br/>healthd"]
+  Windows --> Tasks["Windows Scheduled Tasks<br/>Launch / Recovery / File1C / Hayabusa"]
+  Gateway --> PublicEntry["/portal and /d/... routes"]
+  Grafana --> Dashboards["detmir dashboards"]
+
+  Checks --> Contour["run_awatch_contour_check.sh"]
+  Contour --> Daily["awatch-contour-daily-check.timer"]
+  Contour --> Weekly["awatch-contour-weekly-check.timer"]
+```
+
+## 2. Оркестрационные entrypoints
+
+| Зона | Entrypoint | Тип действия | Что поддерживает |
+|---|---|---|---|
+| Полный контур | `ansible/install_full_stack.yml` | deploy orchestrator | Последовательный запуск основных playbooks по группам inventory |
+| AW server | `ansible/deploy_aw_server.yml` | deploy | ActivityWatch server, Rust binaries, server-side units |
+| Windows/RDP | `ansible/deploy_aw_windows.yml` | deploy | Collectors, guard service, recovery task, File1C/Hayabusa scheduled tasks |
+| Post-validate Windows | `ansible/post_validate_aw_windows.yml` | validation | Read-only-ish validation after Windows rollout |
+| DetMir portal | `ansible/deploy_detmir_portal.yml` | deploy | Portal, evidence API, portal env, systemd services |
+| Gateway | `ansible/deploy_proxmox_web_gateway.yml` | deploy | Nginx gateway, `/portal`, Grafana routes, operator index |
+| Grafana dashboards | `ansible/deploy_grafana_dashboards.yml` | deploy | Version-controlled dashboards import |
+| Grafana checker | `ansible/deploy_grafana_check.yml` | deploy/check | Dashboard and datasource health checks |
+| File1C analytics | `ansible/deploy_file_1c_analytics.yml` | deploy | ClickHouse/File1C analytics server side |
+| File1C Windows upload | `ansible/deploy_file_1c_windows_telemetry.yml` | deploy | Windows scheduled upload task |
+| Optional DLP evidence | `ansible/deploy_dlp_evidence_sync.yml` | opt-in deploy | Evidence sync task only when DLP is explicitly enabled |
+| pfSense poller | `ansible/deploy_aw_pfsense_poller.yml` | optional deploy | Network telemetry helper, outside workforce hot path |
+| Daily/weekly checks | `scripts/run_awatch_contour_check.sh` | read-only check | Contour health bundle and optional smoke checks |
+| Daily check timer | `ops/systemd/awatch-contour-daily-check.timer` | systemd timer | Scheduled daily run of contour check |
+| Weekly check timer | `ops/systemd/awatch-contour-weekly-check.timer` | systemd timer | Scheduled weekly run of contour check |
+| Support bundle | `scripts/detmir-support-daily.sh` and related scripts | read-only/support | Operator diagnostics and support artifacts |
+| Orchestration guard | `scripts/check_orchestration_map.sh` | repository check | Ensures this map references live playbooks/scripts |
+
+## 3. Runtime boundaries
+
+```mermaid
+flowchart LR
+  Deploy["Deploy orchestration<br/>Ansible"] --> Runtime["Runtime services<br/>systemd and Windows tasks"]
+  Runtime --> Check["Read-only checks<br/>detmir-check, contour check, smoke"]
+  Check --> Evidence["Logs and summaries<br/>operator review"]
+
+  OptionalDlp["Optional DLP runtime"] -. explicit operator decision .-> Runtime
+  Hayabusa["Hayabusa upload task"] --> Findings["Security findings import"]
+  Velociraptor["Velociraptor mode"] -. disabled/offline/server explicit .-> Findings
+  Findings --> Portal["Portal security view"]
+  Portal -. approval required .-> Executor["Separate executor<br/>plan/apply/verify/rollback"]
+```
+
+Правила:
+
+- routine deploy/recovery не должен сам включать тяжелый DLP runtime;
+- Hayabusa/Velociraptor findings не являются заменой SIEM/DLP/EDR;
+- блокировка рабочих станций возможна только через отдельный executor и явное
+  approval;
+- GitHub CI остается public mirror validation, а не registry release evidence;
+- Gitea и российский build-runner остаются основным release/evidence контуром.
+
+## 4. Windows/RDP task orchestration
+
+```mermaid
+flowchart TD
+  DeployWin["deploy_aw_windows.yml"] --> Toolkit["C:\\Program Files\\AWatch-rus\\windows"]
+  Toolkit --> Ensemble["deploy-ensemble.ps1"]
+  Ensemble --> Recovery["ActivityWatch Recovery<br/>Scheduled Task"]
+  Ensemble --> Launch["ActivityWatch Launch [HOST_user]<br/>per-user tasks"]
+  Ensemble --> Guard["AWatchRusCollectorGuard<br/>Windows service"]
+  Ensemble --> File1C["ActivityWatch File1C Upload<br/>Scheduled Task"]
+  Ensemble --> Hayabusa["ActivityWatch Hayabusa Upload<br/>Scheduled Task"]
+
+  Guard --> Worktime["worktime-session collector"]
+  Launch --> AFK["aw-watcher-afk"]
+  Launch --> Window["aw-watcher-window"]
+  Launch --> Browser["browser category collector"]
+  File1C --> ClickHouse["ClickHouse landing"]
+  Hayabusa --> AWServer["AW server Hayabusa drop"]
+```
+
+Production DetMir использует стабильный logical host id `SHARKON2025` для
+bucket-ов и витрин. Смена физического имени/IP RDP-сервера не должна
+автоматически менять bucket suffix или Grafana variables.
+
+## 5. Read-only checks and quality gates
+
+| Проверка | Команда | Назначение |
+|---|---|---|
+| Orchestration map check | `bash scripts/check_orchestration_map.sh` | Проверяет, что карта оркестрации ссылается на реальные entrypoints |
+| Repository quality gate | `bash scripts/quality-gate.sh` | Включает orchestration map check, shell/node/pwsh/ansible guards |
+| Secret scan | `python3 scripts/public_secret_pattern_check.py` | Не допускает публичные секреты |
+| Contour check | `bash scripts/run_awatch_contour_check.sh` | Read-only production contour check через env вне репозитория |
+| Browser smoke | `scripts/aw-webui-browser-smoke.sh` | Проверяет operator/browser surface |
+| Portal contract sync | `node scripts/check_portal_contract_sync.mjs` | Проверяет согласованность portal API/static contracts |
+
+## 6. Что не делаем автоматически
+
+- Не запускаем `deploy_dlp_full_stack.yml` как часть routine checks.
+- Не включаем DLP timers/services без отдельного решения оператора.
+- Не стартуем Velociraptor server/client contour автоматически на малом
+  production Proxmox.
+- Не меняем маршрутизацию, firewall или workstation containment из checks.
+- Не публикуем credentials, tokens, passwords или customer identifiers в Git.
+
+## 7. Связанные документы
+
+- [MODULE_ARCHITECTURE_GRAPH_RU.md](MODULE_ARCHITECTURE_GRAPH_RU.md)
+- [ARCHITECTURE_RU.md](ARCHITECTURE_RU.md)
+- [UNIFIED_OPERATING_MODEL_RU.md](UNIFIED_OPERATING_MODEL_RU.md)
+- [DLP_OPTIONAL_RUNTIME_RU.md](DLP_OPTIONAL_RUNTIME_RU.md)
+- [PRODUCTION_READINESS_RU.md](PRODUCTION_READINESS_RU.md)
+- [OPERATIONS_VALIDATION_RUNBOOK_RU.md](OPERATIONS_VALIDATION_RUNBOOK_RU.md)
+- [GRAFANA_DASHBOARDS_RU.md](GRAFANA_DASHBOARDS_RU.md)
+- [../ansible/README.md](../ansible/README.md)
@@ -0,0 +1,98 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+DOC="$ROOT/docs/ORCHESTRATION_MAP_RU.md"
+failures=()
+
+fail() {
+  failures+=("$1")
+}
+
+require_file() {
+  local path="$1"
+  if [[ ! -s "$ROOT/$path" ]]; then
+    fail "missing_or_empty:$path"
+  fi
+}
+
+require_marker() {
+  local marker="$1"
+  local path="$2"
+  if ! grep -Fq "$marker" "$ROOT/$path"; then
+    fail "missing_marker:$path:$marker"
+  fi
+}
+
+require_absent() {
+  local marker="$1"
+  local path="$2"
+  if grep -Fq "$marker" "$ROOT/$path"; then
+    fail "forbidden_marker:$path:$marker"
+  fi
+}
+
+require_file "docs/ORCHESTRATION_MAP_RU.md"
+require_file "docs/MODULE_ARCHITECTURE_GRAPH_RU.md"
+require_file "ansible/README.md"
+require_file "README.md"
+
+entrypoints=(
+  "ansible/install_full_stack.yml"
+  "ansible/deploy_aw_server.yml"
+  "ansible/deploy_aw_windows.yml"
+  "ansible/post_validate_aw_windows.yml"
+  "ansible/deploy_detmir_portal.yml"
+  "ansible/deploy_proxmox_web_gateway.yml"
+  "ansible/deploy_grafana_dashboards.yml"
+  "ansible/deploy_grafana_check.yml"
+  "ansible/deploy_file_1c_analytics.yml"
+  "ansible/deploy_file_1c_windows_telemetry.yml"
+  "ansible/deploy_dlp_evidence_sync.yml"
+  "ansible/deploy_aw_pfsense_poller.yml"
+  "scripts/run_awatch_contour_check.sh"
+  "scripts/detmir-support-daily.sh"
+  "scripts/check_orchestration_map.sh"
+  "ops/systemd/awatch-contour-daily-check.timer"
+  "ops/systemd/awatch-contour-weekly-check.timer"
+)
+
+for path in "${entrypoints[@]}"; do
+  require_file "$path"
+  require_marker "$path" "docs/ORCHESTRATION_MAP_RU.md"
+done
+
+doc_markers=(
+  "GitHub/Gitea"
+  "DLP runtime"
+  "Hayabusa"
+  "Velociraptor"
+  "Security findings"
+  "approval"
+  "SHARKON2025"
+  "logical host id"
+  "quality-gate.sh"
+  "public mirror validation"
+  "российский build-runner"
+)
+
+for marker in "${doc_markers[@]}"; do
+  require_marker "$marker" "docs/ORCHESTRATION_MAP_RU.md"
+done
+
+require_marker "docs/ORCHESTRATION_MAP_RU.md" "docs/MODULE_ARCHITECTURE_GRAPH_RU.md"
+require_marker "docs/ORCHESTRATION_MAP_RU.md" "README.md"
+require_marker "docs/ORCHESTRATION_MAP_RU.md" "ansible/README.md"
+
+require_absent "FSTEC certified" "docs/ORCHESTRATION_MAP_RU.md"
+require_absent "ФСТЭК сертифицирован" "docs/ORCHESTRATION_MAP_RU.md"
+require_absent "automatic remediation" "docs/ORCHESTRATION_MAP_RU.md"
+require_absent "registry submission completed" "docs/ORCHESTRATION_MAP_RU.md"
+
+if (( ${#failures[@]} > 0 )); then
+  printf 'orchestration_map_check=fail\n' >&2
+  printf '%s\n' "${failures[@]}" >&2
+  exit 1
+fi
+
+printf 'orchestration_map_check=ok\n'
@@ -17,18 +17,25 @@ else
  echo "node not found, skipping portal contract sync guard."
 fi

+echo "[preflight] Orchestration map guard"
+bash scripts/check_orchestration_map.sh
+
 rust_candidates=()
 if [[ -n "$RUST_BIN" ]]; then
  rust_candidates+=("$RUST_BIN")
 fi
-if [[ -n "${AW_RUS_CARGO_TARGET_DIR:-}" ]]; then
-  rust_candidates+=("$AW_RUS_CARGO_TARGET_DIR/release/quality-gate")
+if [[ "${QUALITY_GATE_USE_RUST:-0}" == "1" ]]; then
+  if [[ -n "${AW_RUS_CARGO_TARGET_DIR:-}" ]]; then
+    rust_candidates+=("$AW_RUS_CARGO_TARGET_DIR/release/quality-gate")
+  fi
+  rust_candidates+=(
+    "$TARGET_ROOT/release/quality-gate"
+    "$ROOT_DIR/adk-rust/target/release/quality-gate"
+  )
+fi
+if [[ "${QUALITY_GATE_ALLOW_SYSTEM:-0}" == "1" ]]; then
+  rust_candidates+=("/usr/local/bin/quality-gate")
 fi
-rust_candidates+=(
-  "$TARGET_ROOT/release/quality-gate"
-  "$ROOT_DIR/adk-rust/target/release/quality-gate"
-  "/usr/local/bin/quality-gate"
-)

 for candidate in "${rust_candidates[@]}"; do
  if [[ -x "$candidate" ]]; then
@@ -96,7 +103,7 @@ fi
 violations=()
 for path in "${tracked_py[@]}"; do
  case "$path" in
-    aw-server/dlp-content-analysis/*|clickhouse-1c/ai/*|clickhouse-1c/etl/*|detmir-mcp/main.py|grafana-1c/*|pfsense/*|proxmox/tsj_guardian_bot.py|proxmox/test_tsj_guardian_bot.py|scripts/package_rust_release_binaries.py)
+    aw-server/dlp-content-analysis/*|clickhouse-1c/ai/*|clickhouse-1c/etl/*|detmir-mcp/main.py|grafana-1c/*|pfsense/*|proxmox/tsj_guardian_bot.py|proxmox/test_tsj_guardian_bot.py|scripts/package_rust_release_binaries.py|scripts/public_secret_pattern_check.py)
      continue
      ;;
  esac