fix(ops): sync verified production baseline for AW-Rus DLP

2026-05-13 22:50:42 +03:00
parent 22aadd5c03
commit 91c3b46f16
20 changed files with 661 additions and 127 deletions
@@ -0,0 +1,60 @@
+# DLP Content Analysis Runtime Status 2026-05-13
+
+This document records the production-verified state of advanced content analysis on `10.10.10.13`.
+
+## What is live
+
+- Endpoint-side dictionary and regex matching is active in `windows/dlp-endpoint-signals-collector.ps1`.
+- Active policy supports:
+  - `contentAnalysis.dictionaryPack`
+  - `contentAnalysis.regexPack`
+  - `contentAnalysis.ocrEnabled`
+  - `ioc.*`
+- Historical incidents in `aw-dlp-incidents_SHARKON2025` already contain enriched fields:
+  - `dictionaryMatches`
+  - `regexMatches`
+  - `ocrRequested`
+- IOC refresh pipeline is deployed and active:
+  - `aw-dlp-ioc-refresh.timer`
+  - output artifacts:
+    - `/opt/activitywatch/dlp-ioc/output/ioc_blacklist.json`
+    - `/opt/activitywatch/dlp-ioc/output/ioc_blacklist.csv`
+    - `/opt/activitywatch/dlp-ioc/output/ioc_blacklist.sql`
+
+## What was fixed in this phase
+
+- Server-side analyzer dependencies were installed only inside a virtualenv, but there was no canonical wrapper to run the analyzer in production.
+- Added `/usr/local/bin/aw-dlp-content-analyzer`, which executes:
+  - `/opt/activitywatch/dlp-content-analysis/.venv/bin/python`
+  - `/opt/activitywatch/dlp-content-analysis/content_analyzer.py`
+
+## Supported production mode
+
+### Fully supported now
+
+- Endpoint-side enrichment:
+  - clipboard and print content are matched against dictionary and regex packs on the endpoint;
+  - enriched incidents are sent to AW with structured matches;
+  - `ocrRequested=true` is carried into incident metadata when policy requires screenshot/OCR follow-up.
+- IOC enrichment:
+  - Hayabusa/Sigma-derived IOC artifacts are refreshed on the server and exposed over HTTP for policy consumption.
+- Server-side manual/operational analysis:
+  - operators can run `aw-dlp-content-analyzer` for text or image artifacts using the deployed packs and OCR stack.
+
+### Not a continuous background pipeline yet
+
+- There is no standalone daemon that automatically scans screenshot artifacts after incident creation.
+- OCR is production-usable as a server-side utility path, not as an always-on post-processing service.
+
+## Live verification commands
+
+```bash
+sudo systemctl status aw-dlp-ioc-refresh.timer --no-pager
+ls -1 /opt/activitywatch/dlp-ioc/output
+aw-dlp-content-analyzer --text "СНИЛС 112-233-445 95 пароль qwerty" --dictionary-pack 152-fz-pdn --regex-pack secrets
+```
+
+Expected result:
+
+- IOC artifacts exist and are non-empty.
+- The analyzer returns dictionary and regex matches for the sample text.
@@ -0,0 +1,48 @@
+# DLP Runtime Chain Status 2026-05-13
+
+## Verified production chain
+
+Verified on `10.10.10.13`:
+
+- `policy engine`
+  - service: `aw-dlp-policy-engine.service`
+  - health: `GET http://127.0.0.1:5601/healthz`
+  - active policy: `policyId=1`, `default-policy`, `version=1`
+- `case management`
+  - service: `aw-dlp-case-management.service`
+  - health: `GET http://127.0.0.1:5602/health`
+  - runtime data present: case `id=1`
+- `compliance reporting`
+  - timer active: `aw-dlp-report-scheduler.timer`
+  - artifacts present:
+    - `152-fz-2026-05.html`
+    - `152-fz-2026-05.json`
+    - `pci-dss-2026-05.html`
+    - `pci-dss-2026-05.json`
+- `integrations`
+  - timers active:
+    - `aw-dlp-cef-exporter.timer`
+    - `aw-dlp-webhook-sender.timer`
+    - `aw-dlp-syslog-forwarder.timer`
+  - recent journal runs are clean
+  - current runtime result is `sent=0` / `delivered=0` because no new incidents were generated since the last seen bucket event
+- `endpoint -> incident ingest`
+  - `aw-dlp-endpoint-signals_SHARKON2025` fresh
+  - `aw-dlp-incidents_SHARKON2025` exists and contains valid historical incidents
+- `health/admin`
+  - `/usr/local/bin/dlp-health-check --json` = `ok=true`
+  - `/usr/local/bin/dlp-admin-cli.py health check` = policy/cases/aw OK
+
+## Operational conclusion
+
+Core chain is working:
+
+`policy -> endpoint collectors -> incident bucket -> case management -> compliance -> integrations`
+
+There is no confirmed production break in the server-side DLP chain.
+
+## Bounded residual backlog
+
+- external webhook/syslog/CEF destinations are configured and runnable, but current production evidence only shows clean timer execution with zero fresh incidents to export
+- stale incident buckets must not be treated as failure by health-check if endpoint transport and policy/case services are healthy
+- remaining work belongs to content-analysis completion and broader productization, not to server-side chain break repair
@@ -9,6 +9,8 @@
 - [Компоненты системы](Components) - описание всех компонентов
 - [Интерактивная карта](Interactive-Map) - визуальная карта связей
 - [ИБ-профиль DLP](../dlp-security-functional-spec-ru.md) - подробное описание реализованного DLP/monitoring-контура для службы ИБ
+- [Runtime status: DLP chain](../dlp-runtime-chain-status-2026-05-13.md) - фактический live-статус policy/cases/integrations/compliance
+- [Runtime status: Content analysis](../dlp-content-analysis-runtime-status-2026-05-13.md) - фактический live-статус dictionary/regex/OCR/IOC

 ### Компоненты
 - [DLP Endpoint Monitoring](DLP-Endpoint-Monitoring) - мониторинг clipboard, печати, USB
@@ -22,6 +24,7 @@
 - [Установка на Windows](Windows-Installation) - установка коллекторов
 - [Настройка сервера](Server-Setup) - настройка Linux сервера
 - [Grafana + Prometheus](Monitoring-Setup) - мониторинг стек
+- [Windows startup model](../windows-deploy-startup-model.md) - canonical startup model для RDP/standalone deployment

 ### Конфигурация
 - [DLP Правила](DLP-Rules) - настройка DLP политик
@@ -0,0 +1,59 @@
+# Windows Deploy Startup Model
+
+## Supported startup models
+
+### 1. Multi-user RDP host
+
+Use this model on `SHARKON2025`-style hosts with multiple user sessions.
+
+- `ActivityWatch Launch [HOST_user]` tasks:
+  - `AtLogOn`
+  - `InteractiveToken`
+  - start only for users that currently have a real Windows session
+- `ActivityWatch Recovery` task:
+  - `AtStartup`
+  - `SYSTEM`
+  - keeps only the global `worktime-session-collector` alive
+  - may re-trigger user launch tasks, but only for users whose sessions currently exist
+- interactive collectors/watcher binaries belong to the user-session path, not to Session 0
+
+Collector ownership in this model:
+
+- `aw-watcher-afk` and `aw-watcher-window`: user-session only
+- `browser-domains-native-collector.ps1`: user-session only
+- `email-outbound-collector.ps1`: user-session only
+- `file-operations-collector.ps1`: user-session path
+- `dlp-endpoint-signals-collector.ps1`: user-session path
+- `worktime-session-collector.ps1`: single global process under recovery path
+
+### 2. Standalone service installer
+
+Use this model on single-user or headless hosts where Task Scheduler per-user orchestration is not the primary control plane.
+
+- `aw-standalone-service.ps1` runs as a loop/service wrapper
+- Session 0 starts only collectors that are safe headless
+- browser/email interactive collectors must not be assumed available from Session 0
+
+Collector ownership in this model:
+
+- `dlp-endpoint-signals-collector.ps1`: allowed
+- `file-operations-collector.ps1`: allowed
+- `worktime-session-collector.ps1`: allowed
+- `browser-domains-native-collector.ps1`: not reliable in Session 0
+- `email-outbound-collector.ps1`: not reliable in Session 0
+- `aw-watcher-afk` / `aw-watcher-window`: not a standalone Session 0 primitive
+
+## Non-supported mix
+
+Do not mix the two startup models on the same RDP host:
+
+- no permanent standalone-service loop together with per-user launch/recovery tasks
+- no blind `Start-ScheduledTask` for all configured users
+- no validation rule that treats users without sessions as failed collector startup
+
+## Hardening rules
+
+- start launch tasks only for users with real sessions
+- keep only one global `worktime-session-collector`
+- validate by session-aware expectations, not by “all configured users must currently run”
+- keep `deploy_aw_windows.yml`, `deploy-ensemble.ps1`, `hardening-recovery.ps1`, and installer assumptions aligned