From 9bdf28ed2b69a03ba9ba7b19c13e751252ae7a92 Mon Sep 17 00:00:00 2001 From: IgorRachkov <89467086+igor04091968@users.noreply.github.com> Date: Wed, 29 Apr 2026 13:09:23 +0300 Subject: [PATCH 01/13] Add newcomer onboarding guide for repository structure --- README.md | 1 + docs/codebase-onboarding.md | 128 ++++++++++++++++++++++++++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 docs/codebase-onboarding.md diff --git a/README.md b/README.md index b3e1846..21d0e07 100755 --- a/README.md +++ b/README.md @@ -5,6 +5,7 @@ ## Что входит - `docs/preparation.md` — подготовка инфраструктуры и входных параметров. +- `docs/codebase-onboarding.md` — обзор структуры репозитория и маршрут изучения для новичка. - `docs/deployment.md` — пошаговый деплой LXC и ActivityWatch Server. - `docs/runbook.md` — быстрый runbook для оператора. - `docs/operations.md` — регламент сопровождения, бэкапов, обновлений и rollback. diff --git a/docs/codebase-onboarding.md b/docs/codebase-onboarding.md new file mode 100644 index 0000000..301c3be --- /dev/null +++ b/docs/codebase-onboarding.md @@ -0,0 +1,128 @@ +# Onboarding по кодовой базе AWatch-rus + +Этот документ — быстрый вход для новичка: **что лежит где**, **как всё связано** и **что читать дальше**. + +## 1) Что это за репозиторий + +`AWatch-rus` — это не один сервис, а **инфраструктурный набор** для развёртывания и сопровождения ActivityWatch в прод-подобной среде: + +- Proxmox/LXC-подготовка, +- установка и настройка AW Server, +- русификация Web UI, +- автоматизация через Ansible, +- клиентский rollout для Windows/Linux, +- дополнительная телеметрия (в т.ч. pfSense poller и DLP-сигналы), +- эксплуатационные runbook/operations документы. + +Идея: чтобы развёртывание было **повторяемым**, а не «ручной магией в один вечер». + +## 2) Карта проекта (по папкам) + +### `docs/` +Главный источник истины по процессам. + +- `preparation.md` — входные данные, prerequisites, что нужно до старта. +- `deployment.md` — базовый серверный деплой. +- `runbook.md` — быстрые операционные действия и проверки. +- `operations.md` — сопровождение, бэкапы, rollback. +- `windows/*` — отдельная ветка документации по Windows-оркестрации. +- `1C_GRAFANA_DEPLOYMENT_RU.md`, `pfsense.md`, `linux-client.md` — специализированные подсистемы. + +### `proxmox/` +Скрипты ранней инфраструктурной фазы: + +- `create-ct.sh` — создание/подготовка контейнера, +- `push-aw-artifacts.sh` — доставка артефактов и конфигов. + +### `aw-server/` +Серверная «сердцевина»: + +- `install_aw_server.sh` — установка AW Server, +- `apply_webui_ru_patch.sh` + `aw-ru-patch.js` — русификация UI, +- `activitywatch-server.service` — unit для systemd, +- `aw-server.env.example` — шаблон переменных окружения, +- `settings/*.json` — конфигурация представлений/классов. + +### `ansible/` +Идемпотентная автоматизация (вместо ручных команд): + +- playbook'и для серверного деплоя, +- сценарии provisioning + deploy, +- `group_vars/*.example.yml` и `inventory.example.ini` как шаблоны входных данных. + +### `windows/` +PowerShell toolkit для клиентской стороны: + +- `deploy-ensemble.ps1` — оркестратор, +- `deploy-single-user.ps1`, `deploy-domain-users.ps1` — сценарии установки, +- `validate-deployment.ps1` — post-check, +- `ActivityWatch.Windows.Common.psm1` — общая библиотека функций, +- скрипты DLP/browser telemetry. + +### `pfsense/` +Отдельный poller для pfSense API + systemd unit. + +### `grafana-1c/` +Набор для SQL exporter + Prometheus + Grafana дашбордов по 1C метрикам. + +### `scripts/` +Утилиты и quality gates: + +- `quality-gate.sh` — базовый preflight, +- инсталляторы Linux-клиента и console/ssh logger режимов. + +### `secrets/` +Только шаблоны. Реальные секреты в репозиторий не кладутся. + +## 3) Как компоненты связаны в потоке + +Типовой pipeline: + +1. Подготовка параметров (`docs/preparation.md`, `secrets/*.example`). +2. Provisioning контейнера в Proxmox (`proxmox/`). +3. Установка/настройка AW Server (`aw-server/`). +4. Включение автозапуска и проверка (`systemd` + `docs/runbook.md`). +5. Rollout клиентов (обычно `windows/`, иногда `scripts/install_aw_linux_client.sh`). +6. Эксплуатация и изменения через `docs/operations.md`. +7. При необходимости — расширение мониторинга (`grafana-1c/`, `pfsense/`). + +## 4) Что важно понять в первую очередь + +1. **Репозиторий документ-ориентированный**: сначала читаешь `docs/`, потом запускаешь скрипты. +2. **Шаблоны `.example` — обязательная точка входа**: не редактируй скрипты вместо заполнения переменных. +3. **Есть два режима работы**: + - ручной/полуручной (bash + runbook), + - автоматизированный (Ansible). +4. **Windows часть — полноценный под-проект** с собственными deploy/validate практиками. +5. **Безопасность**: никаких секретов в git; rollback и backup — не опция, а стандарт процесса. + +## 5) Рекомендованный порядок изучения (первые 2–3 часа) + +1. `README.md` — получить общую картину. +2. `docs/preparation.md` — понять входные параметры. +3. `docs/deployment.md` — увидеть «сквозной» серверный сценарий. +4. `docs/runbook.md` и `docs/operations.md` — как жить с системой после деплоя. +5. `aw-server/install_aw_server.sh` и `aw-server/activitywatch-server.service` — как реально стартует сервис. +6. `windows/deploy-ensemble.ps1` + `windows/validate-deployment.ps1` — клиентская фаза. +7. `ansible/README.md` и ключевые playbook'и — переход к промышленной автоматизации. + +## 6) Практические подсказки для первого вклада + +- Начни с правок документации или `.example`-шаблонов — это самый безопасный вход. +- Перед изменениями в скриптах сравни, нет ли уже Ansible-аналога (лучше поддерживать один «официальный» путь). +- Любая новая переменная должна быть отражена: + 1) в `.example` файле, + 2) в docs, + 3) в проверках/валидации (если применимо). +- Для Windows-скриптов используй общие функции из `ActivityWatch.Windows.Common.psm1`, чтобы не дублировать логику. + +## 7) Куда смотреть дальше (углубление) + +- Если интересует эксплуатация и инциденты: `docs/runbook.md`, `docs/operations.md`, `docs/windows/troubleshooting.md`. +- Если интересует автодеплой: `ansible/provision_proxmox_ct_and_deploy_aw.yml` и `ansible/tasks/`. +- Если интересует наблюдаемость: `grafana-1c/` + `pfsense/` + `prometheus`/`alerts` конфиги. +- Если интересует hardening и DLP: `windows/hardening-recovery.ps1`, `docs/dlp-gap-analysis.md`. + +--- + +Если ты новичок в проекте, практичный старт: разверни тестовый стенд по `docs/deployment.md`, затем прогоняй валидации из `docs/runbook.md` и `windows/validate-deployment.ps1`. From aca42caf0a2b8076bfbc36b6830bfb49b83f1fd0 Mon Sep 17 00:00:00 2001 From: IgorRachkov <89467086+igor04091968@users.noreply.github.com> Date: Thu, 30 Apr 2026 20:59:40 +0300 Subject: [PATCH 02/13] Add install-kit to repo drift checker --- scripts/check_install_kit_vs_repo.sh | 53 +++++++++++++++++++ scripts/quality-gate.sh | 17 ++++-- scripts/validate_install_kit.sh | 79 ++++++++++++++++++++++++++++ 3 files changed, 146 insertions(+), 3 deletions(-) create mode 100755 scripts/check_install_kit_vs_repo.sh create mode 100755 scripts/validate_install_kit.sh diff --git a/scripts/check_install_kit_vs_repo.sh b/scripts/check_install_kit_vs_repo.sh new file mode 100755 index 0000000..97d165d --- /dev/null +++ b/scripts/check_install_kit_vs_repo.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT_DIR" + +KIT_DIR="install-kit-awindows-20260427-211240" + +python - <<'PY' +from pathlib import Path +import hashlib + +root=Path('.') +kit=Path('install-kit-awindows-20260427-211240') +if not kit.exists(): + raise SystemExit('Install kit directory not found') + +def sha(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + +all_compared=[] +mismatches=[] +missing_in_repo=[] + +for kp in sorted(p for p in kit.rglob('*') if p.is_file() and p.name!='MANIFEST.txt'): + rel=kp.relative_to(kit) + rp=root/rel + if not rp.exists(): + missing_in_repo.append(str(rel)) + continue + all_compared.append(str(rel)) + if sha(kp)!=sha(rp): + mismatches.append(str(rel)) + +ps_mismatches=[p for p in mismatches if p.startswith('windows/') and p.endswith('.ps1') or p.endswith('.psm1') or p.endswith('.psd1')] + +print(f'Compared files: {len(all_compared)}') +print(f'Missing in repo: {len(missing_in_repo)}') +print(f'Mismatched content: {len(mismatches)}') +if missing_in_repo: + print('--- Missing in repo ---') + for p in missing_in_repo: + print(p) +if mismatches: + print('--- Mismatches ---') + for p in mismatches: + print(p) +print(f'PowerShell mismatches: {len(ps_mismatches)}') +if ps_mismatches: + print('--- PowerShell mismatches ---') + for p in ps_mismatches: + print(p) +PY diff --git a/scripts/quality-gate.sh b/scripts/quality-gate.sh index 84b68eb..9182bfa 100755 --- a/scripts/quality-gate.sh +++ b/scripts/quality-gate.sh @@ -4,17 +4,17 @@ set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" cd "$ROOT_DIR" -echo "[1/3] Bash syntax check" +echo "[1/4] Bash syntax check" find aw-server proxmox -type f -name "*.sh" -print0 | xargs -0 -r -n1 bash -n -echo "[2/3] Shellcheck (if available)" +echo "[2/4] Shellcheck (if available)" if command -v shellcheck >/dev/null 2>&1; then find aw-server proxmox -type f -name "*.sh" -print0 | xargs -0 -r shellcheck else echo "shellcheck not found, skipping." fi -echo "[3/3] PowerShell parse check (if pwsh available)" +echo "[3/4] PowerShell parse check (if pwsh available)" if command -v pwsh >/dev/null 2>&1; then pwsh -NoLogo -NoProfile -Command ' $ErrorActionPreference = "Stop" @@ -28,4 +28,15 @@ else echo "pwsh not found, skipping." fi + + +echo "[4/4] Ansible syntax check (if ansible-playbook available)" +if command -v ansible-playbook >/dev/null 2>&1; then + for playbook in ansible/*.yml; do + ansible-playbook --syntax-check "$playbook" -i ansible/inventory.example.ini >/dev/null + done +else + echo "ansible-playbook not found, skipping." +fi + echo "quality-gate: OK" diff --git a/scripts/validate_install_kit.sh b/scripts/validate_install_kit.sh new file mode 100755 index 0000000..f0ddc43 --- /dev/null +++ b/scripts/validate_install_kit.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT_DIR" + +KIT_DIR="install-kit-awindows-20260427-211240" +MANIFEST="$KIT_DIR/MANIFEST.txt" +ZIP_ARCHIVE="install-kit-awindows-20260427-211240.zip" +TAR_ARCHIVE="install-kit-awindows-20260427-211240.tar.gz" + +required_files=( + "$MANIFEST" + "$KIT_DIR/README-INSTALL-KIT.txt" + "$KIT_DIR/windows/deploy-ensemble.ps1" + "$KIT_DIR/windows/validate-deployment.ps1" + "$KIT_DIR/ansible/deploy_aw_windows_phase2.yml" + "$KIT_DIR/aw-server/install_aw_server.sh" +) + +echo "[1/4] Required files presence" +for file in "${required_files[@]}"; do + [[ -f "$file" ]] || { echo "Missing required file: $file"; exit 1; } +done + +echo "[2/4] Manifest checksum verification" +sha256sum -c "$MANIFEST" >/dev/null + +echo "[3/4] Manifest completeness" +python - <<'PY' +from pathlib import Path +import sys +kit=Path('install-kit-awindows-20260427-211240') +manifest=kit/'MANIFEST.txt' +listed=[] +for line in manifest.read_text().splitlines(): + line=line.strip() + if not line: + continue + parts=line.split(' ',1) + if len(parts)!=2: + print(f'Invalid MANIFEST line: {line}') + sys.exit(1) + listed.append(parts[1]) +listed_set=set(listed) +actual_set={str(p) for p in kit.rglob('*') if p.is_file() and p.name!='MANIFEST.txt'} +missing=sorted(listed_set-actual_set) +extra=sorted(actual_set-listed_set) +if missing or extra: + print('Missing files listed in MANIFEST:', missing) + print('Files not listed in MANIFEST:', extra) + sys.exit(1) +print(f'MANIFEST complete: {len(actual_set)} files tracked') +PY + +echo "[4/4] Archive composition check" +python - <<'PY' +from pathlib import Path +import tarfile, zipfile, sys +kit_prefix='install-kit-awindows-20260427-211240/' +zip_path=Path('install-kit-awindows-20260427-211240.zip') +tar_path=Path('install-kit-awindows-20260427-211240.tar.gz') +if not zip_path.exists() or not tar_path.exists(): + print('Archives not found') + sys.exit(1) +with zipfile.ZipFile(zip_path) as z: + zip_files=sorted(i for i in z.namelist() if not i.endswith('/')) +with tarfile.open(tar_path, 'r:gz') as t: + tar_files=sorted(m.name for m in t.getmembers() if m.isfile()) +if zip_files != tar_files: + print('ZIP and TAR contents differ') + sys.exit(1) +if not all(f.startswith(kit_prefix) for f in zip_files): + print('Unexpected archive prefix layout') + sys.exit(1) +print(f'Archives match: {len(zip_files)} files') +PY + +echo "validate_install_kit: OK" From 8f10bef9044cc1e8b83c084315f11f161c3ff4b7 Mon Sep 17 00:00:00 2001 From: IgorRachkov <89467086+igor04091968@users.noreply.github.com> Date: Fri, 1 May 2026 15:50:02 +0300 Subject: [PATCH 03/13] Limit print title fallback to masked Russian document names --- .../windows/dlp-endpoint-signals-collector.ps1 | 15 +++++++++++++-- windows/dlp-endpoint-signals-collector.ps1 | 15 +++++++++++++-- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/install-kit-awindows-20260427-211240/windows/dlp-endpoint-signals-collector.ps1 b/install-kit-awindows-20260427-211240/windows/dlp-endpoint-signals-collector.ps1 index 8ee44f5..b943477 100644 --- a/install-kit-awindows-20260427-211240/windows/dlp-endpoint-signals-collector.ps1 +++ b/install-kit-awindows-20260427-211240/windows/dlp-endpoint-signals-collector.ps1 @@ -402,6 +402,17 @@ function Test-LooksLikeMojibakeQuestionMarks { return $Value -match '\?{2,}' } +function Test-LooksLikeRussianTitleMaskedAsQuestionMarks { + param([AllowNull()][string]$Value) + if ([string]::IsNullOrWhiteSpace($Value)) { return $false } + + $trimmed = $Value.Trim() + if ($trimmed -match '[A-Za-zА-Яа-я0-9]') { return $false } + + # Typical broken Cyrillic print title shape: multiple words of question marks. + return $trimmed -match '^\?{3,}(\s+\?{3,})+$' +} + function Normalize-OwnerForMatch { param([AllowNull()][string]$Value) if ([string]::IsNullOrWhiteSpace($Value)) { return '' } @@ -709,12 +720,12 @@ while ($true) { if ($script:SeenPrintJob.ContainsKey($jobId)) { continue } $script:SeenPrintJob[$jobId] = (Get-Date).ToUniversalTime() - $printerName = [string]$job.Name + $printerName = Normalize-PrinterForMatch -Value ([string]$job.Name) $documentName = [string]$job.Document $owner = [string]$job.Owner $documentNameOriginal = $documentName - if (Test-LooksLikeMojibakeQuestionMarks -Value $documentName) { + if (Test-LooksLikeRussianTitleMaskedAsQuestionMarks -Value $documentName) { $eventDocumentName = Get-BetterDocumentNameFromPrintServiceEvents -Owner $owner -PrinterName $printerName if ($eventDocumentName) { $documentName = $eventDocumentName diff --git a/windows/dlp-endpoint-signals-collector.ps1 b/windows/dlp-endpoint-signals-collector.ps1 index 8ee44f5..b943477 100644 --- a/windows/dlp-endpoint-signals-collector.ps1 +++ b/windows/dlp-endpoint-signals-collector.ps1 @@ -402,6 +402,17 @@ function Test-LooksLikeMojibakeQuestionMarks { return $Value -match '\?{2,}' } +function Test-LooksLikeRussianTitleMaskedAsQuestionMarks { + param([AllowNull()][string]$Value) + if ([string]::IsNullOrWhiteSpace($Value)) { return $false } + + $trimmed = $Value.Trim() + if ($trimmed -match '[A-Za-zА-Яа-я0-9]') { return $false } + + # Typical broken Cyrillic print title shape: multiple words of question marks. + return $trimmed -match '^\?{3,}(\s+\?{3,})+$' +} + function Normalize-OwnerForMatch { param([AllowNull()][string]$Value) if ([string]::IsNullOrWhiteSpace($Value)) { return '' } @@ -709,12 +720,12 @@ while ($true) { if ($script:SeenPrintJob.ContainsKey($jobId)) { continue } $script:SeenPrintJob[$jobId] = (Get-Date).ToUniversalTime() - $printerName = [string]$job.Name + $printerName = Normalize-PrinterForMatch -Value ([string]$job.Name) $documentName = [string]$job.Document $owner = [string]$job.Owner $documentNameOriginal = $documentName - if (Test-LooksLikeMojibakeQuestionMarks -Value $documentName) { + if (Test-LooksLikeRussianTitleMaskedAsQuestionMarks -Value $documentName) { $eventDocumentName = Get-BetterDocumentNameFromPrintServiceEvents -Owner $owner -PrinterName $printerName if ($eventDocumentName) { $documentName = $eventDocumentName From 6640299a48d8eb7ad9014ded0fa8a16c0ba60eb5 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Sun, 3 May 2026 20:18:19 +0000 Subject: [PATCH 04/13] docs: add strategic DLP roadmap vs InfoWatch Traffic Monitor Co-Authored-By: Fashion Lisa --- docs/strategic-dlp-roadmap.md | 183 ++++++++++++++++++++++++++++++++++ 1 file changed, 183 insertions(+) create mode 100644 docs/strategic-dlp-roadmap.md diff --git a/docs/strategic-dlp-roadmap.md b/docs/strategic-dlp-roadmap.md new file mode 100644 index 0000000..ff5d184 --- /dev/null +++ b/docs/strategic-dlp-roadmap.md @@ -0,0 +1,183 @@ +# Стратегический анализ: AWatch-rus vs InfoWatch Traffic Monitor + +## Текущие возможности AWatch-rus (что уже есть) + +| Область | Реализовано | +|---------|------------| +| Activity tracking | `aw-watcher-afk`, `aw-watcher-window` — время активности, окна | +| Browser monitoring | `browser-domains-native-collector.ps1` — URL/домены через UIAutomation | +| DLP Phase 1 | Rule-based политики, incident bucket, cooldown/dedup | +| DLP Phase 2 | USB/print/clipboard мониторинг (`dlp-endpoint-signals-collector.ps1`), incident screenshot | +| File operations | Прототип `file-operations-collector.ps1` (create/delete/rename/archive) | +| Worktime | Session-level presence (`worktime-session-collector.ps1`) | +| Network | pfSense poller (firewall events → AW bucket) | +| Aggregation | `aggregate_dlp_events.py` — SQLite/PostgreSQL centralized store | +| UI | Русификация WebUI + DLP incident review panel в `aw-ru-patch.js` | +| Deployment | Ansible + PowerShell ensemble + InnoSetup + Proxmox LXC | +| Linux | Remote worker, console/SSH logger, web category logger | + +## Ключевые разрывы до InfoWatch TM уровня + +### 🔴 Критические (без них это не DLP, а мониторинг) + +#### 1. Блокировка в реальном времени (Enforcement) +**InfoWatch**: Перехватывает и блокирует отправку до завершения — email не уйдёт, файл на USB не запишется, печать не пройдёт. +**AWatch-rus**: Только detect + log. Нет inline-перехвата трафика, нет file system filter driver. + +**Что делать:** +- **USB write-block** — реализуемо через Group Policy + PowerShell enforcement (`Set-StoragePolicy`, device lockdown via WMI). Не требует kernel driver. +- **Print block** — перехват через Print Spooler event (307) + `Cancel-PrintJob`. Уже есть event monitoring — нужен только enforcement шаг. +- **Clipboard block** — `SetClipboardData` hook через .NET или `Clear-Clipboard` при policy violation. Рискованно по UX, но возможно. +- **Email/web block** — требует MITM proxy (аналог InfoWatch ICAP gateway). Реалистичнее начать с email-выгрузки через Exchange Journal/Transport Rule → анализ → карантин, чем строить inline proxy. +- **Приоритет**: USB write-block + print cancel — быстрый win, 2-3 недели. + +#### 2. Email перехват +**InfoWatch**: SMTP/MAPI/IMAP — полный контроль корпоративной почты, включая вложения, тело, получатели. +**AWatch-rus**: Отсутствует. + +**Что делать:** +- **Exchange/M365**: Transport Rule с Journal → Python парсер EML → событие в AW bucket `aw-email-monitor_`. +- **On-prem SMTP**: Milter на Postfix/Sendmail (Python, 200-300 строк). +- **MVP**: Только metadata (from/to/subject/attachment names/sizes), без content inspection. Потом добавить content analysis. +- **Приоритет**: Высокий — email это #1 канал утечки в enterprise. + +#### 3. Контент-анализ (Content Inspection) +**InfoWatch**: Лингвистический анализ, ML-классификация, OCR, document fingerprinting, digital watermarks, EDM (Exact Data Matching). +**AWatch-rus**: Только regex-matching в DLP policy rules. + +**Что делать поэтапно:** +- **Phase A**: Словарные пакеты (ПДн по 152-ФЗ: ИНН, СНИЛС, паспорт, банковские реквизиты) — regex + Luhn/checksum validation. 1-2 недели. +- **Phase B**: Keyword/weighted scoring — JSON pack с весами, threshold для incident. 1 неделя. +- **Phase C**: OCR pipeline — Tesseract + screenshot analysis. Скриншоты уже снимаются! Нужен только OCR + content check step. 2-3 недели. +- **Phase D**: Document fingerprinting — SimHash/MinHash для корпоративных шаблонов. Server-side Python service. 3-4 недели. +- **Phase E**: ML-классификация — fine-tuned модель на корпоративных документах. Долгосрочно. + +### 🟡 Важные (отличают серьёзный продукт от прототипа) + +#### 4. Мессенджеры +**InfoWatch**: Skype, Telegram, WhatsApp, VK, Jabber, MS Teams. +**AWatch-rus**: Отсутствует. + +**Что делать:** +- Перехват через `aw-watcher-window` + title parsing уже частично даёт metadata (видно, что пользователь в Telegram). +- Глубокий перехват текста мессенджеров требует accessibility API или memory scraping — сложно и ломко. +- Реалистичнее: интеграция с корпоративными мессенджерами через API (MS Teams webhook, Mattermost API). +- **MVP**: title-based detection "пользователь X общался в Telegram 2 часа" — уже почти есть. + +#### 5. Облачные хранилища +**InfoWatch**: Контроль загрузки в DropBox, Google Drive, OneDrive, Яндекс.Диск. +**AWatch-rus**: Отсутствует. + +**Что делать:** +- Browser-based detection: URL matching `drive.google.com/upload`, `disk.yandex.ru` — расширить `browser-domains-native-collector.ps1` правилами. +- Sync client detection: мониторинг процессов OneDrive/GoogleDrive + file watcher в sync-каталогах. +- **MVP**: Alert "файл был загружен в облако" через browser URL + file operation correlation. 1-2 недели. + +#### 6. SIEM/SOAR интеграция +**InfoWatch**: CEF/syslog/API export, 75+ интеграций. +**AWatch-rus**: SQLite/PostgreSQL aggregator — foundation есть, но нет стандартных connectors. + +**Что делать:** +- **CEF/Syslog exporter** — из `aggregate_dlp_events.py` в CEF format → rsyslog/Graylog/ELK. Python, 1-2 дня. +- **Webhook/HTTP callback** — при incident severity=high → POST в Teams/Telegram/PagerDuty. 1 день. +- **Grafana dashboards** — PostgreSQL уже поддерживается. Нужны готовые dashboard JSON. 2-3 дня. +- **Приоритет**: CEF exporter + Grafana — быстрые wins для демонстрации "enterprise-ready". + +#### 7. RBAC и multi-tenant admin +**InfoWatch**: Ролевая модель, SoD, multi-tenant console, делегирование по отделам. +**AWatch-rus**: Single admin, нет ролей. + +**Что делать:** +- ActivityWatch REST API не имеет auth из коробки. Нужен reverse proxy (nginx) с auth layer. +- **MVP**: Basic auth + nginx + IP whitelist. Достаточно для PoC. +- **Target**: Keycloak/LDAP auth proxy → role-based API access. Серьёзная работа, 4-6 недель. + +### 🟢 Стратегические преимущества AWatch-rus (где вы уже лучше) + +1. **Open-source база** — нет vendor lock-in, полный контроль над кодом. +2. **Лёгкий agent** — PowerShell collectors vs тяжёлый C++ agent InfoWatch (~200MB RAM). +3. **Гибкая DLP policy** — JSON rules, программируемые на лету, без перекомпиляции. +4. **Linux поддержка** — remote workers, SSH/console logging, web category — InfoWatch слабее в Linux. +5. **pfSense интеграция** — network visibility через firewall API, уникальная фича. +6. **Worktime tracking** — session-level presence, отработка с 1С (Grafana dashboards). + +## Рекомендуемый стратегический roadmap + +### Квартал 1 (ближайшие 3 месяца) — "Enforce & Detect" +| # | Задача | Усилие | Влияние | +|---|--------|--------|---------| +| 1 | USB write-block (GPO + enforcement step) | 2 нед | Критическое | +| 2 | Print job cancel при policy violation | 1 нед | Критическое | +| 3 | ПДн словарный пакет (ИНН/СНИЛС/паспорт regex) | 2 нед | Высокое | +| 4 | Email metadata collector (Exchange Journal) | 3 нед | Критическое | +| 5 | CEF/Syslog exporter для SIEM | 3 дня | Высокое | +| 6 | Grafana incident dashboards | 3 дня | Среднее | + +### Квартал 2 — "Content & Cloud" +| # | Задача | Усилие | Влияние | +|---|--------|--------|---------| +| 7 | Cloud storage upload detection | 2 нед | Высокое | +| 8 | OCR pipeline (Tesseract + screenshot analysis) | 3 нед | Высокое | +| 9 | Weighted keyword scoring engine | 1 нед | Среднее | +| 10 | Webhook/callback при critical incidents | 2 дня | Среднее | +| 11 | Clipboard enforcement (clear on violation) | 1 нед | Среднее | + +### Квартал 3 — "Intelligence & Scale" +| # | Задача | Усилие | Влияние | +|---|--------|--------|---------| +| 12 | Policy engine service (server-side, versioned) | 4 нед | Критическое | +| 13 | Document fingerprinting (SimHash) | 3 нед | Высокое | +| 14 | UEBA / risk scoring (anomaly detection) | 4 нед | Высокое | +| 15 | RBAC auth proxy (Keycloak/LDAP) | 4 нед | Среднее | +| 16 | Correlation engine (user+channel+object+time) | 3 нед | Высокое | + +### Квартал 4 — "Enterprise" +| # | Задача | Усилие | Влияние | +|---|--------|--------|---------| +| 17 | Case management (investigations) | 4 нед | Среднее | +| 18 | Evidence chain / immutable audit log | 2 нед | Среднее | +| 19 | Compliance report generator (152-ФЗ) | 3 нед | Высокое | +| 20 | ML document classifier | 6+ нед | Высокое | + +## Архитектурная рекомендация + +``` +┌─────────────────────────────────────────────────┐ +│ AWatch-rus Server (Rust) │ +│ ┌──────────┐ ┌───────────┐ ┌──────────────┐ │ +│ │ AW API │ │ Policy │ │ Content │ │ +│ │ (buckets │ │ Engine │ │ Analysis │ │ +│ │ events) │ │ (Python │ │ Service │ │ +│ │ │ │ sidecar) │ │ (OCR/ML/ │ │ +│ │ │ │ │ │ fingerprint)│ │ +│ └────┬─────┘ └─────┬─────┘ └──────┬───────┘ │ +│ │ │ │ │ +│ ┌────┴──────────────┴───────────────┴────────┐ │ +│ │ PostgreSQL / SQLite │ │ +│ └────┬──────────────┬───────────────┬────────┘ │ +│ │ │ │ │ +│ ┌────┴─────┐ ┌─────┴─────┐ ┌─────┴────────┐ │ +│ │ Grafana │ │ CEF/SIEM │ │ Webhook │ │ +│ │ Dash │ │ Export │ │ Alerts │ │ +│ └──────────┘ └───────────┘ └──────────────┘ │ +└─────────────────────────────────────────────────┘ + ▲ ▲ ▲ + ┌──────┴──────┐ ┌─────┴─────┐ ┌─────┴──────┐ + │ Windows │ │ Linux │ │ Network │ + │ Endpoint │ │ Endpoint │ │ (pfSense, │ + │ Collectors │ │ Watchers │ │ email │ + │ + Enforce │ │ + SSH log │ │ gateway) │ + └─────────────┘ └───────────┘ └────────────┘ +``` + +## Вывод + +AWatch-rus уже покрывает ~25-30% функционала InfoWatch TM по каналам мониторинга. Критический разрыв — **отсутствие блокировки** (enforcement) и **контент-анализа**. Без этих двух компонентов продукт классифицируется как "мониторинг активности", а не "DLP". + +Реалистичный путь к конкурентоспособности за 6-9 месяцев: +1. Добавить enforcement (USB/print/clipboard block) — переводит из "мониторинг" в "prevention" +2. Добавить content inspection (regex packs + OCR) — переводит из "метаданные" в "DLP" +3. Добавить email канал — закрывает #1 канал утечки +4. Добавить SIEM/Grafana export — делает продукт "enterprise-visible" + +Сильная сторона проекта — **гибкость и лёгкость агента**. InfoWatch agent — тяжёлый C++ monolith. AWatch-rus collectors — лёгкие PowerShell/Python скрипты, которые можно быстро адаптировать. Это стратегическое преимущество для SMB/mid-market, где InfoWatch слишком дорог и тяжёл. From d88ef781845814e6c30a17470b6195b5bb20c70f Mon Sep 17 00:00:00 2001 From: IgorRachkov <89467086+igor04091968@users.noreply.github.com> Date: Mon, 4 May 2026 21:47:03 +0300 Subject: [PATCH 05/13] Fix DLP collector defaults, HTTP error handling, and cleanup --- powershell-analysis.md | 93 ++++++++++++++++++++++ windows/dlp-endpoint-signals-collector.ps1 | 39 +++++++-- windows/file-operations-collector.ps1 | 22 ++++- 3 files changed, 146 insertions(+), 8 deletions(-) create mode 100644 powershell-analysis.md diff --git a/powershell-analysis.md b/powershell-analysis.md new file mode 100644 index 0000000..d7af0ee --- /dev/null +++ b/powershell-analysis.md @@ -0,0 +1,93 @@ +# Анализ DLP-скриптов PowerShell (работоспособность) + +Дата анализа: **2026-05-04 (UTC)** + +## Проверенный scope + +- `windows/dlp-endpoint-signals-collector.ps1` +- `windows/file-operations-collector.ps1` +- `windows/dlp-policy.example.json` +- `windows/web-category-rules.example.json` + +## Ключевой итог + +DLP-скрипты в целом рабочие по архитектуре (heartbeat в ActivityWatch, policy-driven правила, cooldown, enforcement), но есть **критичный риск misconfiguration** и несколько эксплуатационных рисков. + +--- + +## Что точно хорошо + +1. В обоих коллекторах включены `Set-StrictMode -Version Latest` и `$ErrorActionPreference = 'Stop'`. +2. Есть отправка событий в отдельные bucket’ы (`aw-dlp-endpoint-signals_*`, `aw-dlp-incidents_*`, `aw-file-operations_*`). +3. В endpoint-коллекторе реализованы: + - правила по буферу обмена / USB / печати, + - suppression через cooldown (`Should-EmitByCooldown`), + - опциональный screenshot capture при инциденте. +4. В file collector есть наблюдение за `Desktop/Documents/Downloads` через `FileSystemWatcher`. + +--- + +## Найденные проблемы и риски + +### 1) Критично: дефолтный путь конфига в endpoint-скрипте не совпадает с проектом + +- `dlp-endpoint-signals-collector.ps1` использует по умолчанию: + - `C:\ProgramData\ActivityWatch\deployment-config.json` +- Остальной проект использует namespace `AWatch-rus` (`C:\ProgramData\AWatch-rus\...`). + +**Риск:** endpoint-коллектор может стартовать без нужного deployment-конфига и работать с неверными/пустыми параметрами. + +### 2) Нет строгой проверки HTTP-результата в file collector + +В `file-operations-collector.ps1` POST выполняется через `HttpClient`, но код ответа не валидируется (`IsSuccessStatusCode` не проверяется), ошибки частично только логируются. + +**Риск:** «тихая» потеря telemetry при 4xx/5xx. + +### 3) Watcher не снимает event subscriptions явно + +Есть `Register-ObjectEvent`, но в `finally` disposal только watcher-объектов; отписка событий (`Unregister-Event`) явно не делается. + +**Риск:** при рестартах/долгой работе возможно накопление подписок в сессии. + +### 4) Screenshot/GUI-зависимость для enforcement + +`Capture-IncidentScreenshot` и balloon notification завязаны на `System.Windows.Forms/System.Drawing`. + +**Риск:** в non-interactive / service context часть enforcement UX может не работать (событие уйдёт, но скриншот/уведомление может не сформироваться). + +--- + +## Рекомендации (приоритет) + +1. **P1:** выровнять дефолтный `ConfigPath` в `dlp-endpoint-signals-collector.ps1` на `C:\ProgramData\AWatch-rus\deployment-config.json`. +2. **P1:** добавить проверку `response.IsSuccessStatusCode` в `file-operations-collector.ps1` и логировать body/status при ошибках. +3. **P2:** сохранить subscription-объекты `Register-ObjectEvent` и делать `Unregister-Event` в `finally`. +4. **P2:** для enforcement/UI добавить fallback режим «headless» (только лог + heartbeat). + +--- + +## Что не удалось проверить в текущей среде + +В этом контейнере отсутствует `pwsh`, поэтому не выполнены: + +- синтаксический parse всех `*.ps1/*.psm1` через PowerShell parser; +- `Test-ModuleManifest`; +- smoke-run на Windows API (`Get-WinEvent`, `Get-Partition`, `Get-Disk`, `Set-Clipboard`, `Win32_PrintJob`). + +--- + +## Команды для целевой Windows-проверки + +```powershell +# 1) Синтаксис +Get-ChildItem .\windows -Recurse -Include *.ps1,*.psm1 | ForEach-Object { + [void][System.Management.Automation.Language.Parser]::ParseFile($_.FullName,[ref]$null,[ref]$errs) + if($errs){ "FAIL $($_.FullName)" } else { "OK $($_.FullName)" } +} + +# 2) Быстрый запуск file collector (с логом) +.\windows\file-operations-collector.ps1 -ConfigPath 'C:\ProgramData\AWatch-rus\deployment-config.json' -LogPath 'C:\ProgramData\AWatch-rus\collector-fileops.log' + +# 3) Быстрый запуск endpoint collector (с логом) +.\windows\dlp-endpoint-signals-collector.ps1 -ConfigPath 'C:\ProgramData\AWatch-rus\deployment-config.json' -PolicyPath 'C:\ProgramData\AWatch-rus\dlp-policy.json' -LogPath 'C:\ProgramData\AWatch-rus\collector-endpoint.log' +``` diff --git a/windows/dlp-endpoint-signals-collector.ps1 b/windows/dlp-endpoint-signals-collector.ps1 index 1052c02..74f67d0 100644 --- a/windows/dlp-endpoint-signals-collector.ps1 +++ b/windows/dlp-endpoint-signals-collector.ps1 @@ -1,6 +1,6 @@ [CmdletBinding()] param( - [string]$ConfigPath = 'C:\ProgramData\ActivityWatch\deployment-config.json', + [string]$ConfigPath = 'C:\ProgramData\AWatch-rus\deployment-config.json', [string]$ServerHost, [int]$ServerPort, [ValidateSet('http', 'https')] @@ -224,6 +224,10 @@ function Show-EnforcementNotification { [Parameter(Mandatory = $true)][string]$Title, [Parameter(Mandatory = $true)][string]$Body ) + if ($script:HeadlessMode) { + Write-EndpointLog ("headless mode: skip notification title={0}" -f $Title) + return $false + } try { Add-Type -AssemblyName System.Windows.Forms -ErrorAction SilentlyContinue $icon = New-Object System.Windows.Forms.NotifyIcon @@ -235,9 +239,11 @@ function Show-EnforcementNotification { $icon.ShowBalloonTip(5000) Start-Sleep -Milliseconds 200 $icon.Dispose() + return $true } catch { Write-EndpointLog ("notification failed: {0}" -f $_.Exception.Message) + return $false } } @@ -416,8 +422,13 @@ function Evaluate-ClipboardRules { $enforced = $false if ($action -eq 'block') { - $enforced = Invoke-ClipboardEnforcement - Show-EnforcementNotification -Title 'DLP: буфер обмена очищен' -Body $message + if ($script:HeadlessMode) { + Write-EndpointLog ("headless fallback: clipboard rule={0} requires block, skipped interactive enforcement" -f $ruleId) + } + else { + $enforced = Invoke-ClipboardEnforcement + [void](Show-EnforcementNotification -Title 'DLP: буфер обмена очищен' -Body $message) + } } Send-DlpIncidentHeartbeat -RuleId $ruleId -Action $action -Severity $severity -Message $message -SignalType 'clipboard' -Data @{ @@ -451,8 +462,13 @@ function Evaluate-UsbRules { $enforced = $false if ($action -eq 'block') { - $enforced = Invoke-UsbWriteBlockEnforcement -DriveLetter $DriveLetter - Show-EnforcementNotification -Title 'DLP: USB заблокирован для записи' -Body $message + if ($script:HeadlessMode) { + Write-EndpointLog ("headless fallback: usb rule={0} requires block, skipped interactive enforcement drive={1}" -f $ruleId, $DriveLetter) + } + else { + $enforced = Invoke-UsbWriteBlockEnforcement -DriveLetter $DriveLetter + [void](Show-EnforcementNotification -Title 'DLP: USB заблокирован для записи' -Body $message) + } } Send-DlpIncidentHeartbeat -RuleId $ruleId -Action $action -Severity $severity -Message $message -SignalType 'usb_insert' -Data @{ @@ -496,8 +512,13 @@ function Evaluate-PrintRules { $enforced = $false if ($action -eq 'block') { - $enforced = Invoke-PrintJobEnforcement -PrinterName $PrinterName -DocumentName $DocumentName -Owner $Owner - Show-EnforcementNotification -Title 'DLP: печать заблокирована' -Body $message + if ($script:HeadlessMode) { + Write-EndpointLog ("headless fallback: print rule={0} requires block, skipped interactive enforcement printer={1}" -f $ruleId, $PrinterName) + } + else { + $enforced = Invoke-PrintJobEnforcement -PrinterName $PrinterName -DocumentName $DocumentName -Owner $Owner + [void](Show-EnforcementNotification -Title 'DLP: печать заблокирована' -Body $message) + } } Send-DlpIncidentHeartbeat -RuleId $ruleId -Action $action -Severity $severity -Message $message -SignalType 'print_job' -Data @{ @@ -760,9 +781,13 @@ $script:LogPath = $resolvedLogPath $script:IncidentArtifactsRoot = $resolvedIncidentArtifactsRoot $script:IncidentScreenshotEnabled = $resolvedIncidentScreenshotEnabled $script:ScreenshotTypesLoaded = $false +$script:HeadlessMode = ($env:SESSIONNAME -eq 'Service') -or (-not [Environment]::UserInteractive) Load-DlpPolicy -Path $resolvedPolicyPath Write-EndpointLog ("endpoint collector started against {0}" -f $script:ApiBase) +if ($script:HeadlessMode) { + Write-EndpointLog "headless mode enabled: enforcement UI is disabled, incident heartbeat and logs only" +} while ($true) { try { diff --git a/windows/file-operations-collector.ps1 b/windows/file-operations-collector.ps1 index 3fb5246..ca543a3 100644 --- a/windows/file-operations-collector.ps1 +++ b/windows/file-operations-collector.ps1 @@ -48,13 +48,23 @@ function Invoke-AwJsonPost { [Parameter(Mandatory = $true)][string]$Uri, [Parameter(Mandatory = $true)][string]$Json ) + $httpClient = $null try { $httpClient = New-Object System.Net.Http.HttpClient $content = New-Object System.Net.Http.StringContent($Json, [System.Text.Encoding]::UTF8, "application/json") $response = $httpClient.PostAsync($Uri, $content).Result - $httpClient.Dispose() + if (-not $response.IsSuccessStatusCode) { + $status = [int]$response.StatusCode + $reason = [string]$response.ReasonPhrase + $body = $response.Content.ReadAsStringAsync().Result + Write-FileCollectorLog ("POST failed: uri={0} status={1} reason={2} body={3}" -f $Uri, $status, $reason, $body) + } } catch { Write-FileCollectorLog "POST Error: $($_.Exception.Message)" + } finally { + if ($null -ne $httpClient) { + $httpClient.Dispose() + } } } @@ -144,6 +154,7 @@ if ($resolvedPaths.Count -eq 0) { Write-FileCollectorLog "Starting watch on paths: $($resolvedPaths -join ', ')" $watchers = @() +$subscriptions = @() foreach ($path in $resolvedPaths) { $watcher = New-Object System.IO.FileSystemWatcher $watcher.Path = $path @@ -164,6 +175,7 @@ foreach ($path in $resolvedPaths) { } $watchers += $watcher + $subscriptions += @($onChanged, $onDeleted, $onRenamed) } Write-FileCollectorLog "Collector started. Waiting for events..." @@ -175,6 +187,14 @@ try { } finally { Write-FileCollectorLog "Stopping collector..." + foreach ($sub in @($subscriptions)) { + try { + if ($sub -and $sub.Id) { + Unregister-Event -SubscriptionId $sub.Id -ErrorAction SilentlyContinue + Remove-Job -Id $sub.Id -Force -ErrorAction SilentlyContinue + } + } catch {} + } foreach ($w in $watchers) { $w.EnableRaisingEvents = $false $w.Dispose() From 945efd203f95382a13d2213ffa483967058da67a Mon Sep 17 00:00:00 2001 From: IgorRachkov <89467086+igor04091968@users.noreply.github.com> Date: Tue, 5 May 2026 07:55:03 +0300 Subject: [PATCH 06/13] Harden DLP endpoint collector error handling and print dedupe --- windows/dlp-endpoint-signals-collector.ps1 | 29 ++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/windows/dlp-endpoint-signals-collector.ps1 b/windows/dlp-endpoint-signals-collector.ps1 index af3494c..14aeb88 100644 --- a/windows/dlp-endpoint-signals-collector.ps1 +++ b/windows/dlp-endpoint-signals-collector.ps1 @@ -386,6 +386,9 @@ function Evaluate-ClipboardRules { [string]$ClipboardText, [string]$ClipboardHash ) + if ([string]::IsNullOrEmpty($ClipboardText) -or [string]::IsNullOrEmpty($ClipboardHash)) { + return + } foreach ($rule in @($script:Policy.endpoint.clipboard)) { if (-not $rule) { continue } @@ -724,6 +727,7 @@ function Get-BetterDocumentNameFromPrintServiceEvents { } } catch { + Write-EndpointLog ("printservice fallback failed: {0}" -f $_.Exception.Message) } return $null @@ -786,6 +790,7 @@ while ($true) { } } catch { + Write-EndpointLog ("clipboard poll failed: {0}" -f $_.Exception.Message) } try { @@ -813,6 +818,7 @@ while ($true) { } } catch { + Write-EndpointLog ("usb poll failed: {0}" -f $_.Exception.Message) } try { @@ -834,12 +840,22 @@ while ($true) { $documentName = $eventDocumentName } } + $printDocumentNorm = if ($documentName) { [string]$documentName } else { '' } + $printSignalKey = ('{0}|{1}|{2}|{3}' -f + (Normalize-PrinterForMatch -Value $printerName), + (Normalize-OwnerForMatch -Value $owner), + $printDocumentNorm.ToLowerInvariant(), + 'print_job') + if (-not (Should-EmitByCooldown -Fingerprint $printSignalKey -CooldownSeconds 90)) { + continue + } Send-EndpointSignalHeartbeat -SignalType 'print_job' -Data @{ printerName = $printerName documentName = $documentName documentNameOriginal = $documentNameOriginal owner = $owner + eventSource = 'win32_printjob' } Evaluate-PrintRules -PrinterName $printerName -DocumentName $documentName -Owner $owner } @@ -853,6 +869,7 @@ while ($true) { } } catch { + Write-EndpointLog ("printjob poll failed: {0}" -f $_.Exception.Message) } try { @@ -880,6 +897,17 @@ while ($true) { continue } + $effectiveDocument = if ($resolvedDocument) { [string]$resolvedDocument } else { [string]$documentName } + $printSignalKey = ('{0}|{1}|{2}|{3}' -f + (Normalize-PrinterForMatch -Value $printerName), + (Normalize-OwnerForMatch -Value $owner), + $effectiveDocument.ToLowerInvariant(), + 'print_job') + if (-not (Should-EmitByCooldown -Fingerprint $printSignalKey -CooldownSeconds 90)) { + Write-PrintServiceEventTrace -EventSummary $summary -Phase 'skip' -MatchReason 'dedupe-recent-printjob' -ResolvedDocument $resolvedDocument + continue + } + Send-EndpointSignalHeartbeat -SignalType 'print_job' -Data @{ printerName = $printerName documentName = if ($resolvedDocument) { $resolvedDocument } else { $documentName } @@ -900,6 +928,7 @@ while ($true) { } } catch { + Write-EndpointLog ("printservice poll failed: {0}" -f $_.Exception.Message) } } catch { From 9eb02185ddc0caa94eb4dd1cee50f07a6848d14d Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Wed, 6 May 2026 20:05:43 +0300 Subject: [PATCH 07/13] fix(windows): stabilize AW deploy/validate and watcher launch script --- ansible/deploy_aw_windows.yml | 7 ++++++- ansible/post_validate_aw_windows.yml | 10 +++++++++- windows/ActivityWatch.Windows.Common.psm1 | 10 +++++----- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/ansible/deploy_aw_windows.yml b/ansible/deploy_aw_windows.yml index 92a352b..2f599cc 100644 --- a/ansible/deploy_aw_windows.yml +++ b/ansible/deploy_aw_windows.yml @@ -201,7 +201,12 @@ - aw_windows_afk_enabled | bool - aw_windows_hostname_result.stdout is defined ansible.builtin.set_fact: - aw_windows_api_smoke_check_bucket_effective: "aw-watcher-afk_{{ aw_windows_hostname_result.stdout | trim }}" + aw_windows_api_smoke_check_bucket_effective: >- + {{ + aw_windows_api_smoke_check_bucket + if (aw_windows_api_smoke_check_bucket | default('') | string | length) > 0 + else 'aw-watcher-afk_' ~ (aw_windows_hostname_result.stdout | trim) + }} - name: Выполнить AW API smoke-check (проверка наличия свежих событий в AFK бакете) when: diff --git a/ansible/post_validate_aw_windows.yml b/ansible/post_validate_aw_windows.yml index 2f28634..db8dfda 100644 --- a/ansible/post_validate_aw_windows.yml +++ b/ansible/post_validate_aw_windows.yml @@ -4,6 +4,14 @@ gather_facts: false vars: + aw_windows_deploy_root: "C:\\Program Files\\AWatch-rus" + aw_windows_state_root: "C:\\ProgramData\\AWatch-rus" + aw_windows_validation_remote_path: "{{ aw_windows_state_root }}\\aw_validate_ansible.json" + aw_windows_validation_local_dir: "/tmp/aw-rus-validation" + aw_windows_server_scheme: "http" + aw_windows_server_host: "10.10.10.13" + aw_windows_server_port: 5600 + aw_windows_fail_on_validation_error: true aw_windows_launch_task_pattern: "ActivityWatch Launch *" aw_windows_recovery_task_name: "ActivityWatch Recovery" aw_windows_force_task_restart: true @@ -44,6 +52,7 @@ ansible.builtin.uri: url: "{{ aw_windows_server_scheme }}://{{ aw_windows_server_host }}:{{ aw_windows_server_port }}/api/0/buckets/{{ aw_windows_api_smoke_check_bucket_effective }}/events?limit={{ aw_windows_api_smoke_check_limit }}" method: GET + status_code: 200 return_content: true register: aw_windows_api_smoke until: > @@ -87,4 +96,3 @@ msg: - "Validation OK on {{ inventory_hostname }}." - "Report: {{ aw_windows_validation_local_dir }}/{{ inventory_hostname }}-aw_validate_ansible.json" - diff --git a/windows/ActivityWatch.Windows.Common.psm1 b/windows/ActivityWatch.Windows.Common.psm1 index 71e573f..94d7102 100755 --- a/windows/ActivityWatch.Windows.Common.psm1 +++ b/windows/ActivityWatch.Windows.Common.psm1 @@ -718,11 +718,11 @@ function Start-CollectorScriptIfNeeded { return } - $staParam = if (`$ScriptPath -like "*endpoint-signals*") { "-STA" } else { $null } - $argumentList = @('-NoProfile', '-WindowStyle', 'Hidden', '-ExecutionPolicy', 'Bypass') - if ($staParam) { $argumentList += $staParam } - $argumentList += @('-File', `$ScriptPath, '-ConfigPath', `$ConfigPath) - Start-Process -FilePath `$PowerShellExe -ArgumentList $argumentList -WindowStyle Hidden + `$staParam = if (`$ScriptPath -like "*endpoint-signals*") { "-STA" } else { $null } + `$argumentList = @('-NoProfile', '-WindowStyle', 'Hidden', '-ExecutionPolicy', 'Bypass') + if (`$staParam) { `$argumentList += `$staParam } + `$argumentList += @('-File', `$ScriptPath, '-ConfigPath', `$ConfigPath) + Start-Process -FilePath `$PowerShellExe -ArgumentList `$argumentList -WindowStyle Hidden } `$config = Get-DeploymentConfig -Path `$ConfigPath From 03fefcc25770189bbc2f238df5a12c0b481b9924 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Wed, 6 May 2026 20:14:51 +0300 Subject: [PATCH 08/13] fix(deploy): harden windows rollout, validation and installkit payload --- ansible/group_vars/aw_windows.yml | 3 +-- scripts/prod_rollout.sh | 3 +++ windows/installkit/innosetup/AWatch-rus-InnoSetup.iss | 1 + 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/ansible/group_vars/aw_windows.yml b/ansible/group_vars/aw_windows.yml index 4ab2f5f..f335973 100644 --- a/ansible/group_vars/aw_windows.yml +++ b/ansible/group_vars/aw_windows.yml @@ -42,7 +42,7 @@ aw_windows_validation_remote_path: "{{ aw_windows_state_root }}\\aw_validate_ans aw_windows_validation_local_dir: "/tmp/aw-rus-validation" aw_windows_fail_on_validation_error: true -aw_windows_migration_enabled: true +aw_windows_migration_enabled: false aw_windows_legacy_install_root: "C:\\Program Files\\ActivityWatch-Phase2" aw_windows_legacy_state_root: "C:\\ProgramData\\ActivityWatch-Phase2" aw_windows_migration_report_remote_path: "{{ aw_windows_state_root }}\\aw_migration_ansible.json" @@ -50,4 +50,3 @@ aw_windows_migration_report_remote_path: "{{ aw_windows_state_root }}\\aw_migrat aw_windows_api_smoke_check_enabled: true aw_windows_api_smoke_check_bucket: "" aw_windows_api_smoke_check_limit: 10 - diff --git a/scripts/prod_rollout.sh b/scripts/prod_rollout.sh index d7f080c..1f9ebfd 100644 --- a/scripts/prod_rollout.sh +++ b/scripts/prod_rollout.sh @@ -77,4 +77,7 @@ ansible-playbook -i ansible/inventory.ini ansible/deploy_aw_windows.yml --check log "Deploy aw_windows..." ansible-playbook -i ansible/inventory.ini ansible/deploy_aw_windows.yml | tee -a "${LOG_DIR}/deploy_aw_windows.log" +log "Post-validate aw_windows..." +ansible-playbook -i ansible/inventory.ini ansible/post_validate_aw_windows.yml | tee -a "${LOG_DIR}/post_validate_aw_windows.log" + log "DONE. Logs: ${LOG_DIR}" diff --git a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss index 22c0dde..094b078 100644 --- a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss +++ b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss @@ -42,6 +42,7 @@ Source: "..\..\migrate-awatch-rus-paths.ps1"; DestDir: "{app}\windows"; Flags: i Source: "..\..\worktime-session-collector.ps1"; DestDir: "{app}\windows"; Flags: ignoreversion Source: "..\..\browser-domains-native-collector.ps1"; DestDir: "{app}\windows"; Flags: ignoreversion Source: "..\..\dlp-endpoint-signals-collector.ps1"; DestDir: "{app}\windows"; Flags: ignoreversion +Source: "..\..\file-operations-collector.ps1"; DestDir: "{app}\windows"; Flags: ignoreversion Source: "..\..\web-category-rules.example.json"; DestDir: "{app}\windows"; Flags: ignoreversion Source: "..\..\dlp-policy.example.json"; DestDir: "{app}\windows"; Flags: ignoreversion ; Offline payload (optional): place ZIP into windows/installkit/innosetup/payload/ before compiling. From a6a05b82a9a97699667b3c28cfb35c7b010f0f45 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Wed, 6 May 2026 23:24:52 +0300 Subject: [PATCH 09/13] fix(webui): persist worktime report links to :5610 in ansible/installkit --- ansible/deploy_aw_server.yml | 1 + ansible/group_vars/all.example.yml | 1 + ansible/group_vars/all.yml | 1 + ansible/tasks/provision_ct_and_deploy_aw.yml | 1 + aw-server/apply_webui_ru_patch.sh | 13 +++++++++++++ aw-server/aw-server.env.example | 1 + .../ansible/deploy_aw_server.yml | 1 + .../ansible/group_vars/all.example.yml | 1 + .../ansible/tasks/provision_ct_and_deploy_aw.yml | 1 + .../aw-server/apply_webui_ru_patch.sh | 10 ++++++++++ .../aw-server/aw-server.env.example | 1 + .../installkit/innosetup/AWatch-rus-InnoSetup.iss | 2 ++ 12 files changed, 34 insertions(+) diff --git a/ansible/deploy_aw_server.yml b/ansible/deploy_aw_server.yml index 42b88fc..c88246b 100644 --- a/ansible/deploy_aw_server.yml +++ b/ansible/deploy_aw_server.yml @@ -293,6 +293,7 @@ AW_SERVER_WEBUI_DIR={{ aw_server_webui_dir }} AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} + AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} XDG_DATA_HOME={{ aw_server_data_dir }}/.local/share XDG_CONFIG_HOME={{ aw_server_data_dir }}/.config diff --git a/ansible/group_vars/all.example.yml b/ansible/group_vars/all.example.yml index 2091344..9c92acf 100644 --- a/ansible/group_vars/all.example.yml +++ b/ansible/group_vars/all.example.yml @@ -8,6 +8,7 @@ aw_server_db_path: "/var/lib/activitywatch/.local/share/activitywatch/aw-server- aw_server_log_dir: "/var/log/activitywatch" aw_server_user: "activitywatch" aw_server_group: "activitywatch" +aw_worktime_report_base: "http://10.10.10.13:5610" aw_repo_root: "{{ playbook_dir | dirname }}" diff --git a/ansible/group_vars/all.yml b/ansible/group_vars/all.yml index 3298ded..2be8bca 100644 --- a/ansible/group_vars/all.yml +++ b/ansible/group_vars/all.yml @@ -8,6 +8,7 @@ aw_server_log_dir: "/var/log/activitywatch" aw_server_db_path: "/var/lib/activitywatch/aw-server-rust/sqlite.db" aw_server_user: "activitywatch" aw_server_group: "activitywatch" +aw_worktime_report_base: "http://10.10.10.13:5610" aw_repo_root: "/mnt/usb_hdd2/Projects/ActivityWatch-Russian" diff --git a/ansible/tasks/provision_ct_and_deploy_aw.yml b/ansible/tasks/provision_ct_and_deploy_aw.yml index 975e6ef..e1c503f 100644 --- a/ansible/tasks/provision_ct_and_deploy_aw.yml +++ b/ansible/tasks/provision_ct_and_deploy_aw.yml @@ -159,6 +159,7 @@ AW_SERVER_LOG_DIR={{ aw_server_log_dir }} AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} + AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} no_log: true - name: Передать AW server env внутрь CT diff --git a/aw-server/apply_webui_ru_patch.sh b/aw-server/apply_webui_ru_patch.sh index 37a898a..cb92a51 100755 --- a/aw-server/apply_webui_ru_patch.sh +++ b/aw-server/apply_webui_ru_patch.sh @@ -10,6 +10,7 @@ fi source "$ENV_FILE" WEBUI_DIR="${AW_SERVER_WEBUI_DIR:-${AW_WEBUI_DIR:-/opt/activitywatch/webui-ru}}" +REPORT_BASE="${AW_WORKTIME_REPORT_BASE:-http://10.10.10.13:5610}" PATCH_JS_SRC="/root/bootstrap/aw-ru-patch.js" SW_CLEANUP_SRC="/root/bootstrap/aw-sw-cleanup.js" HOST_GROUPS_SRC="/root/bootstrap/aw-host-groups.json" @@ -43,6 +44,18 @@ sw_hash="$(sha1sum "$SW_TARGET" | awk '{print substr($1,1,12)}')" sed -i '/ru-patch-v5.js/d;/sw-cleanup.js/d;/aw-ru-patch.js/d;/aw-sw-cleanup.js/d' "$INDEX_HTML" sed -i "s###" "$INDEX_HTML" sed -i "s###" "$INDEX_HTML" + +# Replace old relative report links if they exist. +sed -i "s#href=\"/today.csv\"#href=\"$REPORT_BASE/reports/worktime/today?format=csv\"#g" "$INDEX_HTML" +sed -i "s#href=\"/today.json\"#href=\"$REPORT_BASE/reports/worktime/today\"#g" "$INDEX_HTML" + +# Ensure report links block exists and points to worktime API on :5610. +if ! grep -q 'id="aw-report-links"' "$INDEX_HTML"; then + sed -i "s##
RDP report: CSV | JSON
#g" "$INDEX_HTML" +fi + +# Normalize existing report links block to current REPORT_BASE. +sed -i "s#
CSV | JSON
#
RDP report: CSV | JSON
#g" "$INDEX_HTML" cp "$SW_CLEANUP_SRC" "$SERVICE_WORKER" trends_chunk="$(grep -Rsl "$TRENDS_NEEDLE" "$WEBUI_DIR/js"/*.js 2>/dev/null | head -n 1 || true)" diff --git a/aw-server/aw-server.env.example b/aw-server/aw-server.env.example index 93ef939..c203b3a 100755 --- a/aw-server/aw-server.env.example +++ b/aw-server/aw-server.env.example @@ -9,3 +9,4 @@ AW_SERVER_DATA_DIR=/var/lib/activitywatch AW_SERVER_LOG_DIR=/var/log/activitywatch AW_SERVER_USER=activitywatch AW_SERVER_GROUP=activitywatch +AW_WORKTIME_REPORT_BASE=http://10.10.10.13:5610 diff --git a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml index 7f372c1..5cdaa01 100644 --- a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml +++ b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml @@ -233,6 +233,7 @@ AW_SERVER_WEBUI_DIR={{ aw_server_webui_dir }} AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} + AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} XDG_DATA_HOME={{ aw_server_data_dir }}/.local/share XDG_CONFIG_HOME={{ aw_server_data_dir }}/.config diff --git a/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml b/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml index 27d5703..37ddb3c 100644 --- a/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml +++ b/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml @@ -7,6 +7,7 @@ aw_server_data_dir: "/var/lib/activitywatch" aw_server_log_dir: "/var/log/activitywatch" aw_server_user: "activitywatch" aw_server_group: "activitywatch" +aw_worktime_report_base: "http://10.10.10.13:5610" aw_repo_root: "{{ playbook_dir | dirname }}" diff --git a/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml index 975e6ef..e1c503f 100644 --- a/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml @@ -159,6 +159,7 @@ AW_SERVER_LOG_DIR={{ aw_server_log_dir }} AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} + AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} no_log: true - name: Передать AW server env внутрь CT diff --git a/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh b/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh index 37a898a..c75ea54 100755 --- a/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh +++ b/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh @@ -10,6 +10,7 @@ fi source "$ENV_FILE" WEBUI_DIR="${AW_SERVER_WEBUI_DIR:-${AW_WEBUI_DIR:-/opt/activitywatch/webui-ru}}" +REPORT_BASE="${AW_WORKTIME_REPORT_BASE:-http://10.10.10.13:5610}" PATCH_JS_SRC="/root/bootstrap/aw-ru-patch.js" SW_CLEANUP_SRC="/root/bootstrap/aw-sw-cleanup.js" HOST_GROUPS_SRC="/root/bootstrap/aw-host-groups.json" @@ -43,6 +44,15 @@ sw_hash="$(sha1sum "$SW_TARGET" | awk '{print substr($1,1,12)}')" sed -i '/ru-patch-v5.js/d;/sw-cleanup.js/d;/aw-ru-patch.js/d;/aw-sw-cleanup.js/d' "$INDEX_HTML" sed -i "s###" "$INDEX_HTML" sed -i "s###" "$INDEX_HTML" + +sed -i "s#href=\"/today.csv\"#href=\"$REPORT_BASE/reports/worktime/today?format=csv\"#g" "$INDEX_HTML" +sed -i "s#href=\"/today.json\"#href=\"$REPORT_BASE/reports/worktime/today\"#g" "$INDEX_HTML" + +if ! grep -q 'id="aw-report-links"' "$INDEX_HTML"; then + sed -i "s##
RDP report: CSV | JSON
#g" "$INDEX_HTML" +fi + +sed -i "s#
CSV | JSON
#
RDP report: CSV | JSON
#g" "$INDEX_HTML" cp "$SW_CLEANUP_SRC" "$SERVICE_WORKER" trends_chunk="$(grep -Rsl "$TRENDS_NEEDLE" "$WEBUI_DIR/js"/*.js 2>/dev/null | head -n 1 || true)" diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example b/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example index 93ef939..c203b3a 100755 --- a/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example +++ b/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example @@ -9,3 +9,4 @@ AW_SERVER_DATA_DIR=/var/lib/activitywatch AW_SERVER_LOG_DIR=/var/log/activitywatch AW_SERVER_USER=activitywatch AW_SERVER_GROUP=activitywatch +AW_WORKTIME_REPORT_BASE=http://10.10.10.13:5610 diff --git a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss index 094b078..31ac5cf 100644 --- a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss +++ b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss @@ -4,6 +4,7 @@ #define AwDefaultServerHost "10.10.10.13" #define AwDefaultServerPort "5600" +#define AwDefaultWorktimeReportBase "http://10.10.10.13:5610" #define AwDefaultUsers "user1,user2,user3,user4,user5" #define AwDefaultInstallRoot "C:\\Program Files\\AWatch-rus\\bin" #define AwDefaultStateRoot "C:\\ProgramData\\AWatch-rus" @@ -148,6 +149,7 @@ begin 'Укажите сервер ActivityWatch (куда агенты будут отправлять данные).', 'Если нужно, измените host/port. По умолчанию — наша конфигурация.' ); + { Worktime CSV/JSON reports are served by aw-worktime-api on :5610 (AwDefaultWorktimeReportBase). } ServerHostPage.Add('ServerHost', False); ServerHostPage.Add('ServerPort', False); ServerHostPage.Values[0] := '{#AwDefaultServerHost}'; From 1df0e18a95f48acbf3dc65f51b031e1ee6e7095a Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Wed, 6 May 2026 23:32:03 +0300 Subject: [PATCH 10/13] fix(collectors): stabilize browser+DLP collectors and add rdp worktime report script --- scripts/rdp-worktime-report.sh | 139 +++++++++++++++++++ windows/browser-domains-native-collector.ps1 | 8 +- windows/dlp-endpoint-signals-collector.ps1 | 42 +++++- 3 files changed, 183 insertions(+), 6 deletions(-) create mode 100644 scripts/rdp-worktime-report.sh diff --git a/scripts/rdp-worktime-report.sh b/scripts/rdp-worktime-report.sh new file mode 100644 index 0000000..0ff435d --- /dev/null +++ b/scripts/rdp-worktime-report.sh @@ -0,0 +1,139 @@ +#!/usr/bin/env bash +set -euo pipefail + +DAY="" +FROM="" +TO="" +AW_BASE_URL="${AW_BASE_URL:-http://10.10.10.13:5600/api/0}" +OUT_DIR="${OUT_DIR:-reports}" + +usage() { + cat <&2; usage; exit 2 ;; + esac +done + +if [[ -n "$DAY" ]]; then + if [[ "$DAY" == "today" ]]; then + FROM="$(date +%F)" + TO="$FROM" + elif [[ "$DAY" == "yesterday" ]]; then + FROM="$(date -d 'yesterday' +%F)" + TO="$FROM" + else + echo "Invalid --day: $DAY" >&2 + exit 2 + fi +fi + +if [[ -z "$FROM" || -z "$TO" ]]; then + usage + exit 2 +fi + +mkdir -p "$OUT_DIR" +CSV_OUT="${OUT_DIR}/rdp-worktime-${FROM}_${TO}.csv" +JSON_OUT="${OUT_DIR}/rdp-worktime-${FROM}_${TO}.json" + +python3 - "$AW_BASE_URL" "$FROM" "$TO" "$CSV_OUT" "$JSON_OUT" <<'PY' +import csv +import json +import sys +import urllib.request +from datetime import datetime, timedelta, timezone + +base, from_d, to_d, csv_out, json_out = sys.argv[1:6] + +def get_json(url: str): + with urllib.request.urlopen(url, timeout=30) as r: + return json.loads(r.read().decode()) + +def parse_ts(s): + if not s: + return None + return datetime.fromisoformat(s.replace("Z", "+00:00")).astimezone(timezone.utc) + +buckets = get_json(f"{base}/buckets") +sessions_bucket = None +for k in buckets.keys(): + if k.startswith("aw-worktime-sessions_"): + sessions_bucket = k + break + +if not sessions_bucket: + raise SystemExit("No aw-worktime-sessions_* bucket found") + +start = datetime.fromisoformat(from_d + "T00:00:00+00:00") +end = datetime.fromisoformat(to_d + "T23:59:59+00:00") + +ev = get_json(f"{base}/buckets/{sessions_bucket}/events?limit=20000") +by_user = {} +for e in ev: + ts = parse_ts(e.get("timestamp")) + if ts is None or ts < start or ts > end: + continue + d = e.get("data") or {} + user = (d.get("username") or "").strip() + if not user: + continue + state = (d.get("state") or "").strip().lower() + is_active = ("актив" in state) or (state == "active") + rec = by_user.setdefault(user, {"active_ts": set(), "first": None, "last": None, "rows": 0}) + rec["rows"] += 1 + if is_active: + rec["active_ts"].add(ts.replace(microsecond=0)) + rec["first"] = ts if rec["first"] is None or ts < rec["first"] else rec["first"] + rec["last"] = ts if rec["last"] is None or ts > rec["last"] else rec["last"] + +rows = [] +full_range = int((end - start).total_seconds()) +for user in sorted(by_user.keys()): + rec = by_user[user] + active = len(rec["active_ts"]) + idle = max(0, full_range - active) + rows.append({ + "user": user, + "active_seconds": int(active), + "active_hhmm": f"{int(active)//3600:02d}:{(int(active)%3600)//60:02d}", + "first_activity": rec["first"].isoformat().replace("+00:00","Z") if rec["first"] else "", + "last_activity": rec["last"].isoformat().replace("+00:00","Z") if rec["last"] else "", + "idle_seconds": int(idle), + "sessions_count": rec["rows"], + }) + +with open(csv_out, "w", newline="", encoding="utf-8") as f: + w = csv.DictWriter(f, fieldnames=[ + "user","active_seconds","active_hhmm","first_activity","last_activity","idle_seconds","sessions_count" + ]) + w.writeheader() + w.writerows(rows) + +with open(json_out, "w", encoding="utf-8") as f: + json.dump({ + "from": from_d, + "to": to_d, + "generated_at_utc": datetime.now(timezone.utc).isoformat().replace("+00:00","Z"), + "rows": rows + }, f, ensure_ascii=False, indent=2) + +print(csv_out) +print(json_out) +PY + +echo "CSV: ${CSV_OUT}" +echo "JSON: ${JSON_OUT}" diff --git a/windows/browser-domains-native-collector.ps1 b/windows/browser-domains-native-collector.ps1 index d0c809f..b220b3d 100755 --- a/windows/browser-domains-native-collector.ps1 +++ b/windows/browser-domains-native-collector.ps1 @@ -541,7 +541,7 @@ function Send-DlpIncidentHeartbeat { } + $captureData } | ConvertTo-Json -Depth 5 -Compress - Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$bucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event | Out-Null + Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$bucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event -TimeoutSec 15 -DisableKeepAlive | Out-Null } function Get-FileSha256Hex { @@ -707,7 +707,7 @@ function Ensure-Bucket { hostname = $script:Hostname } | ConvertTo-Json -Compress - Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$BucketId" -ContentType 'application/json' -Body $body | Out-Null + Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$BucketId" -ContentType 'application/json' -Body $body -TimeoutSec 15 -DisableKeepAlive | Out-Null $script:KnownBuckets[$BucketId] = $true } @@ -733,7 +733,7 @@ function Send-Heartbeat { } } | ConvertTo-Json -Depth 4 -Compress - Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$BucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event | Out-Null + Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$BucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event -TimeoutSec 15 -DisableKeepAlive | Out-Null } function Send-CategoryHeartbeat { @@ -770,7 +770,7 @@ function Send-CategoryHeartbeat { } } | ConvertTo-Json -Depth 4 -Compress - Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$bucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event | Out-Null + Invoke-RestMethod -Method Post -Uri "$($script:ApiBase)/buckets/$bucketId/heartbeat?pulsetime=$resolvedPulseSeconds" -ContentType 'application/json' -Body $event -TimeoutSec 15 -DisableKeepAlive | Out-Null } Load-CustomCategoryRules -Path $resolvedRulesPath diff --git a/windows/dlp-endpoint-signals-collector.ps1 b/windows/dlp-endpoint-signals-collector.ps1 index af3494c..829153e 100644 --- a/windows/dlp-endpoint-signals-collector.ps1 +++ b/windows/dlp-endpoint-signals-collector.ps1 @@ -40,7 +40,7 @@ function Invoke-AwJsonPost { ) $bytes = [Text.Encoding]::UTF8.GetBytes($Json) - Invoke-RestMethod -Method Post -Uri $Uri -ContentType 'application/json; charset=utf-8' -Body $bytes | Out-Null + Invoke-RestMethod -Method Post -Uri $Uri -ContentType 'application/json; charset=utf-8' -Body $bytes -TimeoutSec 15 -DisableKeepAlive | Out-Null } function Ensure-Bucket { @@ -321,6 +321,44 @@ function Get-StringHash { } } +function Get-ClipboardTextSafe { + [OutputType([string])] + param() + + try { + $v = Get-Clipboard -Raw -ErrorAction Stop + if ($null -ne $v) { return [string]$v } + } + catch { + Write-EndpointLog ("clipboard direct read failed: {0}" -f $_.Exception.Message) + } + + # Fallback: read clipboard in a dedicated STA thread for RDP/user-session edge cases. + try { + Add-Type -AssemblyName System.Windows.Forms -ErrorAction SilentlyContinue | Out-Null + $result = [string]::Empty + $thread = [System.Threading.Thread]{ + try { + $script:__aw_clip = [System.Windows.Forms.Clipboard]::GetText() + } + catch { + $script:__aw_clip = $null + } + } + $thread.SetApartmentState([System.Threading.ApartmentState]::STA) + $thread.Start() + $thread.Join(3000) | Out-Null + if ($thread.IsAlive) { $thread.Abort() } + $result = [string]$script:__aw_clip + Remove-Variable -Name __aw_clip -Scope Script -ErrorAction SilentlyContinue + return $result + } + catch { + Write-EndpointLog ("clipboard STA read failed: {0}" -f $_.Exception.Message) + return $null + } +} + function Load-DlpPolicy { param([string]$Path) @@ -772,7 +810,7 @@ while ($true) { } try { - $clipboardText = Get-Clipboard -Raw -ErrorAction SilentlyContinue + $clipboardText = Get-ClipboardTextSafe if ($clipboardText) { $clipboardHash = Get-StringHash -Value $clipboardText if ($clipboardHash -and $clipboardHash -ne $script:LastClipboardHash) { From 8a91734a2771ad75d324d58f34be9b3b4af7b412 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Thu, 7 May 2026 06:30:12 +0300 Subject: [PATCH 11/13] fix(worktime): use Europe/Moscow day boundary for aw-rus today reports --- ansible/deploy_aw_server.yml | 27 ++++ ansible/group_vars/all.example.yml | 1 + ansible/group_vars/all.yml | 1 + .../provision_proxmox_ct_and_deploy_aw.yml | 2 + ...vision_proxmox_ct_matrix_and_deploy_aw.yml | 2 + ansible/tasks/provision_ct_and_deploy_aw.yml | 1 + aw-server/aw-server.env.example | 1 + aw-server/aw-worktime-api.py | 116 ++++++++++++++++++ aw-server/aw-worktime-api.service | 16 +++ aw-server/install_aw_server.sh | 14 +++ .../ansible/deploy_aw_server.yml | 27 ++++ .../ansible/group_vars/all.example.yml | 1 + .../provision_proxmox_ct_and_deploy_aw.yml | 2 + ...vision_proxmox_ct_matrix_and_deploy_aw.yml | 2 + .../tasks/provision_ct_and_deploy_aw.yml | 1 + .../aw-server/aw-server.env.example | 1 + .../aw-server/aw-worktime-api.py | 116 ++++++++++++++++++ .../aw-server/aw-worktime-api.service | 16 +++ .../aw-server/install_aw_server.sh | 14 +++ 19 files changed, 361 insertions(+) create mode 100644 aw-server/aw-worktime-api.py create mode 100644 aw-server/aw-worktime-api.service create mode 100644 install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py create mode 100644 install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.service diff --git a/ansible/deploy_aw_server.yml b/ansible/deploy_aw_server.yml index c88246b..2eb2888 100644 --- a/ansible/deploy_aw_server.yml +++ b/ansible/deploy_aw_server.yml @@ -294,9 +294,36 @@ AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} + AW_WORKTIME_TZ={{ aw_worktime_timezone }} XDG_DATA_HOME={{ aw_server_data_dir }}/.local/share XDG_CONFIG_HOME={{ aw_server_data_dir }}/.config + - name: Установить скрипт AW worktime API + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-api.py" + dest: /usr/local/bin/aw-worktime-api.py + owner: root + group: root + mode: "0755" + + - name: Установить systemd unit AW worktime API + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-api.service" + dest: /etc/systemd/system/aw-worktime-api.service + owner: root + group: root + mode: "0644" + + - name: Перезагрузить systemd после установки AW worktime API + ansible.builtin.systemd: + daemon_reload: true + + - name: Включить и перезапустить AW worktime API + ansible.builtin.systemd: + name: aw-worktime-api.service + enabled: true + state: restarted + - name: Применить хотфиксы compiled JS чанков (Trends, Timespiral, Category helper) ansible.builtin.command: cmd: "/opt/activitywatch/aw-server/apply_webui_ru_patch.sh" diff --git a/ansible/group_vars/all.example.yml b/ansible/group_vars/all.example.yml index 9c92acf..171b218 100644 --- a/ansible/group_vars/all.example.yml +++ b/ansible/group_vars/all.example.yml @@ -9,6 +9,7 @@ aw_server_log_dir: "/var/log/activitywatch" aw_server_user: "activitywatch" aw_server_group: "activitywatch" aw_worktime_report_base: "http://10.10.10.13:5610" +aw_worktime_timezone: "Europe/Moscow" aw_repo_root: "{{ playbook_dir | dirname }}" diff --git a/ansible/group_vars/all.yml b/ansible/group_vars/all.yml index 2be8bca..fdefb38 100644 --- a/ansible/group_vars/all.yml +++ b/ansible/group_vars/all.yml @@ -9,6 +9,7 @@ aw_server_db_path: "/var/lib/activitywatch/aw-server-rust/sqlite.db" aw_server_user: "activitywatch" aw_server_group: "activitywatch" aw_worktime_report_base: "http://10.10.10.13:5610" +aw_worktime_timezone: "Europe/Moscow" aw_repo_root: "/mnt/usb_hdd2/Projects/ActivityWatch-Russian" diff --git a/ansible/provision_proxmox_ct_and_deploy_aw.yml b/ansible/provision_proxmox_ct_and_deploy_aw.yml index dd9dadf..206a37c 100644 --- a/ansible/provision_proxmox_ct_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_and_deploy_aw.yml @@ -9,6 +9,8 @@ - install_aw_server.sh - apply_webui_ru_patch.sh - activitywatch-server.service + - aw-worktime-api.py + - aw-worktime-api.service - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml index 44564da..ec1a4f3 100644 --- a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml @@ -9,6 +9,8 @@ - install_aw_server.sh - apply_webui_ru_patch.sh - activitywatch-server.service + - aw-worktime-api.py + - aw-worktime-api.service - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/ansible/tasks/provision_ct_and_deploy_aw.yml b/ansible/tasks/provision_ct_and_deploy_aw.yml index e1c503f..0551e2f 100644 --- a/ansible/tasks/provision_ct_and_deploy_aw.yml +++ b/ansible/tasks/provision_ct_and_deploy_aw.yml @@ -160,6 +160,7 @@ AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} + AW_WORKTIME_TZ={{ aw_worktime_timezone }} no_log: true - name: Передать AW server env внутрь CT diff --git a/aw-server/aw-server.env.example b/aw-server/aw-server.env.example index c203b3a..03693fb 100755 --- a/aw-server/aw-server.env.example +++ b/aw-server/aw-server.env.example @@ -10,3 +10,4 @@ AW_SERVER_LOG_DIR=/var/log/activitywatch AW_SERVER_USER=activitywatch AW_SERVER_GROUP=activitywatch AW_WORKTIME_REPORT_BASE=http://10.10.10.13:5610 +AW_WORKTIME_TZ=Europe/Moscow diff --git a/aw-server/aw-worktime-api.py b/aw-server/aw-worktime-api.py new file mode 100644 index 0000000..29f7d11 --- /dev/null +++ b/aw-server/aw-worktime-api.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python3 +from http.server import BaseHTTPRequestHandler, HTTPServer +import csv +import io +import json +import os +import urllib.request +from datetime import datetime, timezone, timedelta +from zoneinfo import ZoneInfo + +AW = "http://127.0.0.1:5600/api/0" +REPORT_TZ = ZoneInfo(os.environ.get("AW_WORKTIME_TZ", "Europe/Moscow")) + + +def get(u): + with urllib.request.urlopen(u, timeout=30) as r: + return json.loads(r.read().decode()) + + +def pts(s): + return datetime.fromisoformat(s.replace("Z", "+00:00")).astimezone(timezone.utc) + + +def report_today(): + now_local = datetime.now(REPORT_TZ) + start_local = datetime(now_local.year, now_local.month, now_local.day, tzinfo=REPORT_TZ) + end_local = start_local + timedelta(days=1) - timedelta(seconds=1) + start = start_local.astimezone(timezone.utc) + end = end_local.astimezone(timezone.utc) + b = get(AW + "/buckets") + sb = next((k for k in b if k.startswith("aw-worktime-sessions_")), None) + if not sb: + return [] + ev = get(f"{AW}/buckets/{sb}/events?limit=50000") + by = {} + for e in ev: + ts = pts(e.get("timestamp")) + if ts < start or ts > end: + continue + d = e.get("data") or {} + user = (d.get("username") or "").strip() + if not user: + continue + state = (d.get("state") or "").lower() + active = ("актив" in state) or (state == "active") + row = by.setdefault(user, {"active": set(), "first": None, "last": None, "rows": 0}) + row["rows"] += 1 + if active: + second = ts.replace(microsecond=0) + row["active"].add(second) + row["first"] = second if row["first"] is None or second < row["first"] else row["first"] + row["last"] = second if row["last"] is None or second > row["last"] else row["last"] + rows = [] + full = int((end_local - start_local).total_seconds()) + for user in sorted(by): + row = by[user] + active_seconds = len(row["active"]) + rows.append({ + "user": user, + "active_seconds": active_seconds, + "active_hhmm": "%02d:%02d" % (active_seconds // 3600, (active_seconds % 3600) // 60), + "first_activity": row["first"].isoformat().replace("+00:00", "Z") if row["first"] else "", + "last_activity": row["last"].isoformat().replace("+00:00", "Z") if row["last"] else "", + "idle_seconds": max(0, full - active_seconds), + "sessions_count": row["rows"], + }) + return rows + + +class H(BaseHTTPRequestHandler): + def do_GET(self): + if not self.path.startswith("/reports/worktime/today"): + self.send_response(404) + self.end_headers() + return + fmt = "json" + if "format=csv" in self.path: + fmt = "csv" + rows = report_today() + if fmt == "csv": + out = io.StringIO() + writer = csv.DictWriter( + out, + fieldnames=[ + "user", + "active_seconds", + "active_hhmm", + "first_activity", + "last_activity", + "idle_seconds", + "sessions_count", + ], + ) + writer.writeheader() + writer.writerows(rows) + data = out.getvalue().encode() + self.send_response(200) + self.send_header("Content-Type", "text/csv; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + return + obj = { + "generated_at_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), + "report_timezone": str(REPORT_TZ), + "rows": rows, + } + data = json.dumps(obj, ensure_ascii=False, indent=2).encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +HTTPServer(("0.0.0.0", 5610), H).serve_forever() diff --git a/aw-server/aw-worktime-api.service b/aw-server/aw-worktime-api.service new file mode 100644 index 0000000..4225fb5 --- /dev/null +++ b/aw-server/aw-worktime-api.service @@ -0,0 +1,16 @@ +[Unit] +Description=AW Worktime Report API +After=network.target activitywatch-server.service +Wants=activitywatch-server.service + +[Service] +Type=simple +EnvironmentFile=/etc/activitywatch/aw-server.env +ExecStart=/usr/bin/python3 /usr/local/bin/aw-worktime-api.py +Restart=always +RestartSec=2 +User=activitywatch +Group=activitywatch + +[Install] +WantedBy=multi-user.target diff --git a/aw-server/install_aw_server.sh b/aw-server/install_aw_server.sh index 4d6262e..50ba3e7 100755 --- a/aw-server/install_aw_server.sh +++ b/aw-server/install_aw_server.sh @@ -24,6 +24,8 @@ required_vars=( BOOTSTRAP_DIR="/root/bootstrap" VIEWS_JSON="$BOOTSTRAP_DIR/settings/views-default.json" CLASSES_JSON="$BOOTSTRAP_DIR/settings/classes-worktime.json" +WORKTIME_API_SRC="$BOOTSTRAP_DIR/aw-worktime-api.py" +WORKTIME_API_SERVICE_SRC="$BOOTSTRAP_DIR/aw-worktime-api.service" for var_name in "${required_vars[@]}"; do if [[ -z "${!var_name:-}" ]]; then @@ -89,6 +91,18 @@ systemctl enable activitywatch-server.service systemctl restart activitywatch-server.service systemctl --no-pager --full status activitywatch-server.service || true +if [[ -f "$WORKTIME_API_SRC" ]]; then + install -m 0755 "$WORKTIME_API_SRC" /usr/local/bin/aw-worktime-api.py +fi + +if [[ -f "$WORKTIME_API_SERVICE_SRC" ]]; then + install -m 0644 "$WORKTIME_API_SERVICE_SRC" /etc/systemd/system/aw-worktime-api.service + systemctl daemon-reload + systemctl enable aw-worktime-api.service + systemctl restart aw-worktime-api.service + systemctl --no-pager --full status aw-worktime-api.service || true +fi + for _ in $(seq 1 20); do if curl -fsS "http://127.0.0.1:${AW_SERVER_PORT}/api/0/info" >/dev/null 2>&1; then break diff --git a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml index 5cdaa01..040ec77 100644 --- a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml +++ b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml @@ -234,9 +234,36 @@ AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} + AW_WORKTIME_TZ={{ aw_worktime_timezone }} XDG_DATA_HOME={{ aw_server_data_dir }}/.local/share XDG_CONFIG_HOME={{ aw_server_data_dir }}/.config + - name: Установить скрипт AW worktime API + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-api.py" + dest: /usr/local/bin/aw-worktime-api.py + owner: root + group: root + mode: "0755" + + - name: Установить systemd unit AW worktime API + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-api.service" + dest: /etc/systemd/system/aw-worktime-api.service + owner: root + group: root + mode: "0644" + + - name: Перезагрузить systemd после установки AW worktime API + ansible.builtin.systemd: + daemon_reload: true + + - name: Включить и перезапустить AW worktime API + ansible.builtin.systemd: + name: aw-worktime-api.service + enabled: true + state: restarted + - name: Применить хотфиксы compiled JS чанков (Trends, Timespiral, Category helper) ansible.builtin.command: cmd: "/opt/activitywatch/aw-server/apply_webui_ru_patch.sh" diff --git a/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml b/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml index 37ddb3c..be28ad3 100644 --- a/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml +++ b/install-kit-awindows-20260427-211240/ansible/group_vars/all.example.yml @@ -8,6 +8,7 @@ aw_server_log_dir: "/var/log/activitywatch" aw_server_user: "activitywatch" aw_server_group: "activitywatch" aw_worktime_report_base: "http://10.10.10.13:5610" +aw_worktime_timezone: "Europe/Moscow" aw_repo_root: "{{ playbook_dir | dirname }}" diff --git a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml index dd9dadf..206a37c 100644 --- a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml @@ -9,6 +9,8 @@ - install_aw_server.sh - apply_webui_ru_patch.sh - activitywatch-server.service + - aw-worktime-api.py + - aw-worktime-api.service - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml index 44564da..ec1a4f3 100644 --- a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml @@ -9,6 +9,8 @@ - install_aw_server.sh - apply_webui_ru_patch.sh - activitywatch-server.service + - aw-worktime-api.py + - aw-worktime-api.service - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml index e1c503f..0551e2f 100644 --- a/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/tasks/provision_ct_and_deploy_aw.yml @@ -160,6 +160,7 @@ AW_SERVER_USER={{ aw_server_user }} AW_SERVER_GROUP={{ aw_server_group }} AW_WORKTIME_REPORT_BASE={{ aw_worktime_report_base }} + AW_WORKTIME_TZ={{ aw_worktime_timezone }} no_log: true - name: Передать AW server env внутрь CT diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example b/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example index c203b3a..03693fb 100755 --- a/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example +++ b/install-kit-awindows-20260427-211240/aw-server/aw-server.env.example @@ -10,3 +10,4 @@ AW_SERVER_LOG_DIR=/var/log/activitywatch AW_SERVER_USER=activitywatch AW_SERVER_GROUP=activitywatch AW_WORKTIME_REPORT_BASE=http://10.10.10.13:5610 +AW_WORKTIME_TZ=Europe/Moscow diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py new file mode 100644 index 0000000..29f7d11 --- /dev/null +++ b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py @@ -0,0 +1,116 @@ +#!/usr/bin/env python3 +from http.server import BaseHTTPRequestHandler, HTTPServer +import csv +import io +import json +import os +import urllib.request +from datetime import datetime, timezone, timedelta +from zoneinfo import ZoneInfo + +AW = "http://127.0.0.1:5600/api/0" +REPORT_TZ = ZoneInfo(os.environ.get("AW_WORKTIME_TZ", "Europe/Moscow")) + + +def get(u): + with urllib.request.urlopen(u, timeout=30) as r: + return json.loads(r.read().decode()) + + +def pts(s): + return datetime.fromisoformat(s.replace("Z", "+00:00")).astimezone(timezone.utc) + + +def report_today(): + now_local = datetime.now(REPORT_TZ) + start_local = datetime(now_local.year, now_local.month, now_local.day, tzinfo=REPORT_TZ) + end_local = start_local + timedelta(days=1) - timedelta(seconds=1) + start = start_local.astimezone(timezone.utc) + end = end_local.astimezone(timezone.utc) + b = get(AW + "/buckets") + sb = next((k for k in b if k.startswith("aw-worktime-sessions_")), None) + if not sb: + return [] + ev = get(f"{AW}/buckets/{sb}/events?limit=50000") + by = {} + for e in ev: + ts = pts(e.get("timestamp")) + if ts < start or ts > end: + continue + d = e.get("data") or {} + user = (d.get("username") or "").strip() + if not user: + continue + state = (d.get("state") or "").lower() + active = ("актив" in state) or (state == "active") + row = by.setdefault(user, {"active": set(), "first": None, "last": None, "rows": 0}) + row["rows"] += 1 + if active: + second = ts.replace(microsecond=0) + row["active"].add(second) + row["first"] = second if row["first"] is None or second < row["first"] else row["first"] + row["last"] = second if row["last"] is None or second > row["last"] else row["last"] + rows = [] + full = int((end_local - start_local).total_seconds()) + for user in sorted(by): + row = by[user] + active_seconds = len(row["active"]) + rows.append({ + "user": user, + "active_seconds": active_seconds, + "active_hhmm": "%02d:%02d" % (active_seconds // 3600, (active_seconds % 3600) // 60), + "first_activity": row["first"].isoformat().replace("+00:00", "Z") if row["first"] else "", + "last_activity": row["last"].isoformat().replace("+00:00", "Z") if row["last"] else "", + "idle_seconds": max(0, full - active_seconds), + "sessions_count": row["rows"], + }) + return rows + + +class H(BaseHTTPRequestHandler): + def do_GET(self): + if not self.path.startswith("/reports/worktime/today"): + self.send_response(404) + self.end_headers() + return + fmt = "json" + if "format=csv" in self.path: + fmt = "csv" + rows = report_today() + if fmt == "csv": + out = io.StringIO() + writer = csv.DictWriter( + out, + fieldnames=[ + "user", + "active_seconds", + "active_hhmm", + "first_activity", + "last_activity", + "idle_seconds", + "sessions_count", + ], + ) + writer.writeheader() + writer.writerows(rows) + data = out.getvalue().encode() + self.send_response(200) + self.send_header("Content-Type", "text/csv; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + return + obj = { + "generated_at_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), + "report_timezone": str(REPORT_TZ), + "rows": rows, + } + data = json.dumps(obj, ensure_ascii=False, indent=2).encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + + +HTTPServer(("0.0.0.0", 5610), H).serve_forever() diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.service b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.service new file mode 100644 index 0000000..4225fb5 --- /dev/null +++ b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.service @@ -0,0 +1,16 @@ +[Unit] +Description=AW Worktime Report API +After=network.target activitywatch-server.service +Wants=activitywatch-server.service + +[Service] +Type=simple +EnvironmentFile=/etc/activitywatch/aw-server.env +ExecStart=/usr/bin/python3 /usr/local/bin/aw-worktime-api.py +Restart=always +RestartSec=2 +User=activitywatch +Group=activitywatch + +[Install] +WantedBy=multi-user.target diff --git a/install-kit-awindows-20260427-211240/aw-server/install_aw_server.sh b/install-kit-awindows-20260427-211240/aw-server/install_aw_server.sh index 4d6262e..50ba3e7 100755 --- a/install-kit-awindows-20260427-211240/aw-server/install_aw_server.sh +++ b/install-kit-awindows-20260427-211240/aw-server/install_aw_server.sh @@ -24,6 +24,8 @@ required_vars=( BOOTSTRAP_DIR="/root/bootstrap" VIEWS_JSON="$BOOTSTRAP_DIR/settings/views-default.json" CLASSES_JSON="$BOOTSTRAP_DIR/settings/classes-worktime.json" +WORKTIME_API_SRC="$BOOTSTRAP_DIR/aw-worktime-api.py" +WORKTIME_API_SERVICE_SRC="$BOOTSTRAP_DIR/aw-worktime-api.service" for var_name in "${required_vars[@]}"; do if [[ -z "${!var_name:-}" ]]; then @@ -89,6 +91,18 @@ systemctl enable activitywatch-server.service systemctl restart activitywatch-server.service systemctl --no-pager --full status activitywatch-server.service || true +if [[ -f "$WORKTIME_API_SRC" ]]; then + install -m 0755 "$WORKTIME_API_SRC" /usr/local/bin/aw-worktime-api.py +fi + +if [[ -f "$WORKTIME_API_SERVICE_SRC" ]]; then + install -m 0644 "$WORKTIME_API_SERVICE_SRC" /etc/systemd/system/aw-worktime-api.service + systemctl daemon-reload + systemctl enable aw-worktime-api.service + systemctl restart aw-worktime-api.service + systemctl --no-pager --full status aw-worktime-api.service || true +fi + for _ in $(seq 1 20); do if curl -fsS "http://127.0.0.1:${AW_SERVER_PORT}/api/0/info" >/dev/null 2>&1; then break From d91a396c0035a3d74334e237dc694a4a26b669f5 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Thu, 7 May 2026 06:41:42 +0300 Subject: [PATCH 12/13] feat(webui): add HTML and panel view for worktime reports --- ansible/deploy_aw_server.yml | 2 + .../provision_proxmox_ct_and_deploy_aw.yml | 1 + ...vision_proxmox_ct_matrix_and_deploy_aw.yml | 1 + aw-server/apply_webui_ru_patch.sh | 69 ++++++++-- aw-server/aw-worktime-api.py | 130 ++++++++++++++++++ aw-server/aw-worktime-panel.js | 56 ++++++++ .../ansible/deploy_aw_server.yml | 2 + .../provision_proxmox_ct_and_deploy_aw.yml | 1 + ...vision_proxmox_ct_matrix_and_deploy_aw.yml | 1 + .../aw-server/apply_webui_ru_patch.sh | 66 +++++++-- .../aw-server/aw-worktime-api.py | 130 ++++++++++++++++++ .../aw-server/aw-worktime-panel.js | 56 ++++++++ 12 files changed, 494 insertions(+), 21 deletions(-) create mode 100644 aw-server/aw-worktime-panel.js create mode 100644 install-kit-awindows-20260427-211240/aw-server/aw-worktime-panel.js diff --git a/ansible/deploy_aw_server.yml b/ansible/deploy_aw_server.yml index 2eb2888..9c775b7 100644 --- a/ansible/deploy_aw_server.yml +++ b/ansible/deploy_aw_server.yml @@ -254,6 +254,7 @@ loop: - { src: "{{ aw_repo_root }}/aw-server/aw-ru-patch.js", dest: "{{ aw_server_webui_dir }}/js/ru-patch-v5.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-sw-cleanup.js", dest: "{{ aw_server_webui_dir }}/js/sw-cleanup.js", mode: "0644" } + - { src: "{{ aw_repo_root }}/aw-server/aw-worktime-panel.js", dest: "{{ aw_server_webui_dir }}/js/aw-worktime-panel.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-host-groups.json", dest: "{{ aw_server_webui_dir }}/js/aw-host-groups.json", mode: "0644" } - name: Создать каталог /root/bootstrap для apply_webui_ru_patch.sh @@ -270,6 +271,7 @@ loop: - { src: "{{ aw_repo_root }}/aw-server/aw-ru-patch.js", dest: "/root/bootstrap/aw-ru-patch.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-sw-cleanup.js", dest: "/root/bootstrap/aw-sw-cleanup.js", mode: "0644" } + - { src: "{{ aw_repo_root }}/aw-server/aw-worktime-panel.js", dest: "/root/bootstrap/aw-worktime-panel.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-host-groups.json", dest: "/root/bootstrap/aw-host-groups.json", mode: "0644" } - name: Скопировать apply_webui_ru_patch.sh скрипт diff --git a/ansible/provision_proxmox_ct_and_deploy_aw.yml b/ansible/provision_proxmox_ct_and_deploy_aw.yml index 206a37c..40c6382 100644 --- a/ansible/provision_proxmox_ct_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_and_deploy_aw.yml @@ -11,6 +11,7 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml index ec1a4f3..5452935 100644 --- a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml @@ -11,6 +11,7 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/aw-server/apply_webui_ru_patch.sh b/aw-server/apply_webui_ru_patch.sh index cb92a51..32e53fa 100755 --- a/aw-server/apply_webui_ru_patch.sh +++ b/aw-server/apply_webui_ru_patch.sh @@ -13,12 +13,14 @@ WEBUI_DIR="${AW_SERVER_WEBUI_DIR:-${AW_WEBUI_DIR:-/opt/activitywatch/webui-ru}}" REPORT_BASE="${AW_WORKTIME_REPORT_BASE:-http://10.10.10.13:5610}" PATCH_JS_SRC="/root/bootstrap/aw-ru-patch.js" SW_CLEANUP_SRC="/root/bootstrap/aw-sw-cleanup.js" +WORKTIME_PANEL_SRC="/root/bootstrap/aw-worktime-panel.js" HOST_GROUPS_SRC="/root/bootstrap/aw-host-groups.json" INDEX_HTML="$WEBUI_DIR/index.html" SERVICE_WORKER="$WEBUI_DIR/service-worker.js" TS=$(date +%Y%m%d%H%M%S) PATCH_TARGET="$WEBUI_DIR/js/ru-patch-v5.js" SW_TARGET="$WEBUI_DIR/js/sw-cleanup.js" +WORKTIME_PANEL_TARGET="$WEBUI_DIR/js/aw-worktime-panel.js" HOST_GROUPS_TARGET="$WEBUI_DIR/js/aw-host-groups.json" TRENDS_NEEDLE='this.activityStore.query_category_time_by_period(r)' TRENDS_REPLACEMENT='this.activityStore.ensure_loaded(r)' @@ -29,33 +31,76 @@ CATEGORY_HELPER_REPLACEMENT='hostname:t.hostnameChoices.filter((function(t){retu [[ -f "$PATCH_JS_SRC" ]] || { echo "missing $PATCH_JS_SRC" >&2; exit 1; } [[ -f "$SW_CLEANUP_SRC" ]] || { echo "missing $SW_CLEANUP_SRC" >&2; exit 1; } +[[ -f "$WORKTIME_PANEL_SRC" ]] || { echo "missing $WORKTIME_PANEL_SRC" >&2; exit 1; } [[ -f "$HOST_GROUPS_SRC" ]] || { echo "missing $HOST_GROUPS_SRC" >&2; exit 1; } [[ -f "$INDEX_HTML" ]] || { echo "missing $INDEX_HTML" >&2; exit 1; } install -d "$WEBUI_DIR/js" install -m 0644 "$PATCH_JS_SRC" "$PATCH_TARGET" install -m 0644 "$SW_CLEANUP_SRC" "$SW_TARGET" +install -m 0644 "$WORKTIME_PANEL_SRC" "$WORKTIME_PANEL_TARGET" install -m 0644 "$HOST_GROUPS_SRC" "$HOST_GROUPS_TARGET" cp "$INDEX_HTML" "$INDEX_HTML.bak.$TS" patch_hash="$(sha1sum "$PATCH_TARGET" | awk '{print substr($1,1,12)}')" sw_hash="$(sha1sum "$SW_TARGET" | awk '{print substr($1,1,12)}')" +worktime_panel_hash="$(sha1sum "$WORKTIME_PANEL_TARGET" | awk '{print substr($1,1,12)}')" -sed -i '/ru-patch-v5.js/d;/sw-cleanup.js/d;/aw-ru-patch.js/d;/aw-sw-cleanup.js/d' "$INDEX_HTML" -sed -i "s###" "$INDEX_HTML" -sed -i "s###" "$INDEX_HTML" +python3 - "$WORKTIME_PANEL_TARGET" "$REPORT_BASE" <<'PY' +from pathlib import Path +import sys -# Replace old relative report links if they exist. -sed -i "s#href=\"/today.csv\"#href=\"$REPORT_BASE/reports/worktime/today?format=csv\"#g" "$INDEX_HTML" -sed -i "s#href=\"/today.json\"#href=\"$REPORT_BASE/reports/worktime/today\"#g" "$INDEX_HTML" +path = Path(sys.argv[1]) +report_base = sys.argv[2] +text = path.read_text() +text = text.replace("__AW_WORKTIME_REPORT_BASE__", report_base) +path.write_text(text) +PY -# Ensure report links block exists and points to worktime API on :5610. -if ! grep -q 'id="aw-report-links"' "$INDEX_HTML"; then - sed -i "s##
RDP report: CSV | JSON
#g" "$INDEX_HTML" -fi +python3 - "$INDEX_HTML" "$sw_hash" "$patch_hash" "$worktime_panel_hash" "$REPORT_BASE" <<'PY' +from pathlib import Path +import re +import sys -# Normalize existing report links block to current REPORT_BASE. -sed -i "s#
CSV | JSON
#
RDP report: CSV | JSON
#g" "$INDEX_HTML" +path = Path(sys.argv[1]) +sw_hash = sys.argv[2] +patch_hash = sys.argv[3] +panel_hash = sys.argv[4] +report_base = sys.argv[5] +content = path.read_text() + +content = re.sub( + r']+(?:ru-patch-v5\.js|sw-cleanup\.js|aw-ru-patch\.js|aw-sw-cleanup\.js|aw-worktime-panel\.js)[^>]*>', + '', + content, +) +content = re.sub(r"; frame-src 'self' [^\";>]*", "", content) +content = content.replace( + "script-src 'self' 'unsafe-eval'", + f"script-src 'self' 'unsafe-eval'; frame-src 'self' {report_base}", + 1, +) +content = content.replace( + "", + f'', + 1, +) +content = content.replace( + "", + ( + f'' + f'' + ), + 1, +) +if 'id="aw-report-links"' not in content: + content = content.replace( + "", + '', + 1, + ) +path.write_text(content) +PY cp "$SW_CLEANUP_SRC" "$SERVICE_WORKER" trends_chunk="$(grep -Rsl "$TRENDS_NEEDLE" "$WEBUI_DIR/js"/*.js 2>/dev/null | head -n 1 || true)" diff --git a/aw-server/aw-worktime-api.py b/aw-server/aw-worktime-api.py index 29f7d11..6f6611d 100644 --- a/aw-server/aw-worktime-api.py +++ b/aw-server/aw-worktime-api.py @@ -67,6 +67,126 @@ def report_today(): return rows +def render_html(rows): + generated = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") + date_local = datetime.now(REPORT_TZ).strftime("%Y-%m-%d") + trs = [] + for row in rows: + trs.append( + "" + f"{row['user']}" + f"{row['active_hhmm']}" + f"{row['active_seconds']}" + f"{row['first_activity']}" + f"{row['last_activity']}" + f"{row['idle_seconds']}" + f"{row['sessions_count']}" + "" + ) + if not trs: + trs.append('No data for today yet.') + return f""" + + + + + AW-rus Worktime + + + +
+
+

RDP Worktime Report

+
Date: {date_local} · Timezone: {REPORT_TZ} · Generated UTC: {generated}
+ +
+
+ + + + + + + + + + + + + + {''.join(trs)} + +
UserActiveActive secFirst activityLast activityIdle secSamples
+
+
+ +""" + + class H(BaseHTTPRequestHandler): def do_GET(self): if not self.path.startswith("/reports/worktime/today"): @@ -76,6 +196,8 @@ class H(BaseHTTPRequestHandler): fmt = "json" if "format=csv" in self.path: fmt = "csv" + elif "format=html" in self.path: + fmt = "html" rows = report_today() if fmt == "csv": out = io.StringIO() @@ -100,6 +222,14 @@ class H(BaseHTTPRequestHandler): self.end_headers() self.wfile.write(data) return + if fmt == "html": + data = render_html(rows).encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + return obj = { "generated_at_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), "report_timezone": str(REPORT_TZ), diff --git a/aw-server/aw-worktime-panel.js b/aw-server/aw-worktime-panel.js new file mode 100644 index 0000000..c38b29e --- /dev/null +++ b/aw-server/aw-worktime-panel.js @@ -0,0 +1,56 @@ +(function () { + var reportBase = "__AW_WORKTIME_REPORT_BASE__"; + var reportUrl = reportBase + "/reports/worktime/today?format=html"; + var existing = document.getElementById("aw-report-links"); + if (!existing) return; + + existing.innerHTML = + 'RDP report: ' + + 'HTML | ' + + 'CSV | ' + + 'JSON | ' + + 'Panel'; + + var panel = document.createElement("div"); + panel.id = "aw-report-panel"; + panel.style.cssText = [ + "position:fixed", + "top:16px", + "right:16px", + "width:min(980px,calc(100vw - 32px))", + "height:min(760px,calc(100vh - 32px))", + "background:#fff", + "border:1px solid rgba(15,23,42,.15)", + "border-radius:12px", + "box-shadow:0 24px 80px rgba(15,23,42,.28)", + "overflow:hidden", + "z-index:100000", + "display:none" + ].join(";"); + + panel.innerHTML = + '
' + + '
RDP Worktime Report
' + + '
' + + 'Open' + + 'Close' + + "
" + + ''; + + document.body.appendChild(panel); + + function openPanel(ev) { + if (ev) ev.preventDefault(); + panel.style.display = "block"; + } + + function closePanel(ev) { + if (ev) ev.preventDefault(); + panel.style.display = "none"; + } + + var toggle = document.getElementById("aw-report-toggle"); + if (toggle) toggle.addEventListener("click", openPanel); + var close = panel.querySelector("#aw-report-close"); + if (close) close.addEventListener("click", closePanel); +})(); diff --git a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml index 040ec77..b136df7 100644 --- a/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml +++ b/install-kit-awindows-20260427-211240/ansible/deploy_aw_server.yml @@ -194,6 +194,7 @@ loop: - { src: "{{ aw_repo_root }}/aw-server/aw-ru-patch.js", dest: "{{ aw_server_webui_dir }}/js/ru-patch-v5.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-sw-cleanup.js", dest: "{{ aw_server_webui_dir }}/js/sw-cleanup.js", mode: "0644" } + - { src: "{{ aw_repo_root }}/aw-server/aw-worktime-panel.js", dest: "{{ aw_server_webui_dir }}/js/aw-worktime-panel.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-host-groups.json", dest: "{{ aw_server_webui_dir }}/js/aw-host-groups.json", mode: "0644" } - name: Создать каталог /root/bootstrap для apply_webui_ru_patch.sh @@ -210,6 +211,7 @@ loop: - { src: "{{ aw_repo_root }}/aw-server/aw-ru-patch.js", dest: "/root/bootstrap/aw-ru-patch.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-sw-cleanup.js", dest: "/root/bootstrap/aw-sw-cleanup.js", mode: "0644" } + - { src: "{{ aw_repo_root }}/aw-server/aw-worktime-panel.js", dest: "/root/bootstrap/aw-worktime-panel.js", mode: "0644" } - { src: "{{ aw_repo_root }}/aw-server/aw-host-groups.json", dest: "/root/bootstrap/aw-host-groups.json", mode: "0644" } - name: Скопировать apply_webui_ru_patch.sh скрипт diff --git a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml index 206a37c..40c6382 100644 --- a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_and_deploy_aw.yml @@ -11,6 +11,7 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml index ec1a4f3..5452935 100644 --- a/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml +++ b/install-kit-awindows-20260427-211240/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml @@ -11,6 +11,7 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js - aw-sw-cleanup.js diff --git a/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh b/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh index c75ea54..32e53fa 100755 --- a/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh +++ b/install-kit-awindows-20260427-211240/aw-server/apply_webui_ru_patch.sh @@ -13,12 +13,14 @@ WEBUI_DIR="${AW_SERVER_WEBUI_DIR:-${AW_WEBUI_DIR:-/opt/activitywatch/webui-ru}}" REPORT_BASE="${AW_WORKTIME_REPORT_BASE:-http://10.10.10.13:5610}" PATCH_JS_SRC="/root/bootstrap/aw-ru-patch.js" SW_CLEANUP_SRC="/root/bootstrap/aw-sw-cleanup.js" +WORKTIME_PANEL_SRC="/root/bootstrap/aw-worktime-panel.js" HOST_GROUPS_SRC="/root/bootstrap/aw-host-groups.json" INDEX_HTML="$WEBUI_DIR/index.html" SERVICE_WORKER="$WEBUI_DIR/service-worker.js" TS=$(date +%Y%m%d%H%M%S) PATCH_TARGET="$WEBUI_DIR/js/ru-patch-v5.js" SW_TARGET="$WEBUI_DIR/js/sw-cleanup.js" +WORKTIME_PANEL_TARGET="$WEBUI_DIR/js/aw-worktime-panel.js" HOST_GROUPS_TARGET="$WEBUI_DIR/js/aw-host-groups.json" TRENDS_NEEDLE='this.activityStore.query_category_time_by_period(r)' TRENDS_REPLACEMENT='this.activityStore.ensure_loaded(r)' @@ -29,30 +31,76 @@ CATEGORY_HELPER_REPLACEMENT='hostname:t.hostnameChoices.filter((function(t){retu [[ -f "$PATCH_JS_SRC" ]] || { echo "missing $PATCH_JS_SRC" >&2; exit 1; } [[ -f "$SW_CLEANUP_SRC" ]] || { echo "missing $SW_CLEANUP_SRC" >&2; exit 1; } +[[ -f "$WORKTIME_PANEL_SRC" ]] || { echo "missing $WORKTIME_PANEL_SRC" >&2; exit 1; } [[ -f "$HOST_GROUPS_SRC" ]] || { echo "missing $HOST_GROUPS_SRC" >&2; exit 1; } [[ -f "$INDEX_HTML" ]] || { echo "missing $INDEX_HTML" >&2; exit 1; } install -d "$WEBUI_DIR/js" install -m 0644 "$PATCH_JS_SRC" "$PATCH_TARGET" install -m 0644 "$SW_CLEANUP_SRC" "$SW_TARGET" +install -m 0644 "$WORKTIME_PANEL_SRC" "$WORKTIME_PANEL_TARGET" install -m 0644 "$HOST_GROUPS_SRC" "$HOST_GROUPS_TARGET" cp "$INDEX_HTML" "$INDEX_HTML.bak.$TS" patch_hash="$(sha1sum "$PATCH_TARGET" | awk '{print substr($1,1,12)}')" sw_hash="$(sha1sum "$SW_TARGET" | awk '{print substr($1,1,12)}')" +worktime_panel_hash="$(sha1sum "$WORKTIME_PANEL_TARGET" | awk '{print substr($1,1,12)}')" -sed -i '/ru-patch-v5.js/d;/sw-cleanup.js/d;/aw-ru-patch.js/d;/aw-sw-cleanup.js/d' "$INDEX_HTML" -sed -i "s###" "$INDEX_HTML" -sed -i "s###" "$INDEX_HTML" +python3 - "$WORKTIME_PANEL_TARGET" "$REPORT_BASE" <<'PY' +from pathlib import Path +import sys -sed -i "s#href=\"/today.csv\"#href=\"$REPORT_BASE/reports/worktime/today?format=csv\"#g" "$INDEX_HTML" -sed -i "s#href=\"/today.json\"#href=\"$REPORT_BASE/reports/worktime/today\"#g" "$INDEX_HTML" +path = Path(sys.argv[1]) +report_base = sys.argv[2] +text = path.read_text() +text = text.replace("__AW_WORKTIME_REPORT_BASE__", report_base) +path.write_text(text) +PY -if ! grep -q 'id="aw-report-links"' "$INDEX_HTML"; then - sed -i "s##
RDP report: CSV | JSON
#g" "$INDEX_HTML" -fi +python3 - "$INDEX_HTML" "$sw_hash" "$patch_hash" "$worktime_panel_hash" "$REPORT_BASE" <<'PY' +from pathlib import Path +import re +import sys -sed -i "s#
CSV | JSON
#
RDP report: CSV | JSON
#g" "$INDEX_HTML" +path = Path(sys.argv[1]) +sw_hash = sys.argv[2] +patch_hash = sys.argv[3] +panel_hash = sys.argv[4] +report_base = sys.argv[5] +content = path.read_text() + +content = re.sub( + r']+(?:ru-patch-v5\.js|sw-cleanup\.js|aw-ru-patch\.js|aw-sw-cleanup\.js|aw-worktime-panel\.js)[^>]*>', + '', + content, +) +content = re.sub(r"; frame-src 'self' [^\";>]*", "", content) +content = content.replace( + "script-src 'self' 'unsafe-eval'", + f"script-src 'self' 'unsafe-eval'; frame-src 'self' {report_base}", + 1, +) +content = content.replace( + "", + f'', + 1, +) +content = content.replace( + "", + ( + f'' + f'' + ), + 1, +) +if 'id="aw-report-links"' not in content: + content = content.replace( + "", + '', + 1, + ) +path.write_text(content) +PY cp "$SW_CLEANUP_SRC" "$SERVICE_WORKER" trends_chunk="$(grep -Rsl "$TRENDS_NEEDLE" "$WEBUI_DIR/js"/*.js 2>/dev/null | head -n 1 || true)" diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py index 29f7d11..6f6611d 100644 --- a/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py +++ b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-api.py @@ -67,6 +67,126 @@ def report_today(): return rows +def render_html(rows): + generated = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") + date_local = datetime.now(REPORT_TZ).strftime("%Y-%m-%d") + trs = [] + for row in rows: + trs.append( + "" + f"{row['user']}" + f"{row['active_hhmm']}" + f"{row['active_seconds']}" + f"{row['first_activity']}" + f"{row['last_activity']}" + f"{row['idle_seconds']}" + f"{row['sessions_count']}" + "" + ) + if not trs: + trs.append('No data for today yet.') + return f""" + + + + + AW-rus Worktime + + + +
+
+

RDP Worktime Report

+
Date: {date_local} · Timezone: {REPORT_TZ} · Generated UTC: {generated}
+ +
+
+ + + + + + + + + + + + + + {''.join(trs)} + +
UserActiveActive secFirst activityLast activityIdle secSamples
+
+
+ +""" + + class H(BaseHTTPRequestHandler): def do_GET(self): if not self.path.startswith("/reports/worktime/today"): @@ -76,6 +196,8 @@ class H(BaseHTTPRequestHandler): fmt = "json" if "format=csv" in self.path: fmt = "csv" + elif "format=html" in self.path: + fmt = "html" rows = report_today() if fmt == "csv": out = io.StringIO() @@ -100,6 +222,14 @@ class H(BaseHTTPRequestHandler): self.end_headers() self.wfile.write(data) return + if fmt == "html": + data = render_html(rows).encode("utf-8") + self.send_response(200) + self.send_header("Content-Type", "text/html; charset=utf-8") + self.send_header("Content-Length", str(len(data))) + self.end_headers() + self.wfile.write(data) + return obj = { "generated_at_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"), "report_timezone": str(REPORT_TZ), diff --git a/install-kit-awindows-20260427-211240/aw-server/aw-worktime-panel.js b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-panel.js new file mode 100644 index 0000000..c38b29e --- /dev/null +++ b/install-kit-awindows-20260427-211240/aw-server/aw-worktime-panel.js @@ -0,0 +1,56 @@ +(function () { + var reportBase = "__AW_WORKTIME_REPORT_BASE__"; + var reportUrl = reportBase + "/reports/worktime/today?format=html"; + var existing = document.getElementById("aw-report-links"); + if (!existing) return; + + existing.innerHTML = + 'RDP report: ' + + 'HTML | ' + + 'CSV | ' + + 'JSON | ' + + 'Panel'; + + var panel = document.createElement("div"); + panel.id = "aw-report-panel"; + panel.style.cssText = [ + "position:fixed", + "top:16px", + "right:16px", + "width:min(980px,calc(100vw - 32px))", + "height:min(760px,calc(100vh - 32px))", + "background:#fff", + "border:1px solid rgba(15,23,42,.15)", + "border-radius:12px", + "box-shadow:0 24px 80px rgba(15,23,42,.28)", + "overflow:hidden", + "z-index:100000", + "display:none" + ].join(";"); + + panel.innerHTML = + '
' + + '
RDP Worktime Report
' + + '
' + + 'Open' + + 'Close' + + "
" + + ''; + + document.body.appendChild(panel); + + function openPanel(ev) { + if (ev) ev.preventDefault(); + panel.style.display = "block"; + } + + function closePanel(ev) { + if (ev) ev.preventDefault(); + panel.style.display = "none"; + } + + var toggle = document.getElementById("aw-report-toggle"); + if (toggle) toggle.addEventListener("click", openPanel); + var close = panel.querySelector("#aw-report-close"); + if (close) close.addEventListener("click", closePanel); +})(); From 8849354ddac233af7dbcee52f4840b5cb704d604 Mon Sep 17 00:00:00 2001 From: igor04091968 Date: Thu, 7 May 2026 07:20:14 +0300 Subject: [PATCH 13/13] fix(worktime): codify ui-bridge in ansible and install flow --- ansible/deploy_aw_server.yml | 43 ++++++ .../provision_proxmox_ct_and_deploy_aw.yml | 3 + ...vision_proxmox_ct_matrix_and_deploy_aw.yml | 3 + aw-server/aw-worktime-ui-bridge.py | 138 ++++++++++++++++++ aw-server/aw-worktime-ui-bridge.service | 15 ++ aw-server/aw-worktime-ui-bridge.timer | 11 ++ aw-server/install_aw_server.sh | 21 +++ .../innosetup/AWatch-rus-InnoSetup.iss | 3 +- 8 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 aw-server/aw-worktime-ui-bridge.py create mode 100644 aw-server/aw-worktime-ui-bridge.service create mode 100644 aw-server/aw-worktime-ui-bridge.timer diff --git a/ansible/deploy_aw_server.yml b/ansible/deploy_aw_server.yml index 9c775b7..996eb4b 100644 --- a/ansible/deploy_aw_server.yml +++ b/ansible/deploy_aw_server.yml @@ -316,6 +316,30 @@ group: root mode: "0644" + - name: Установить скрипт AW worktime UI bridge + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-ui-bridge.py" + dest: /usr/local/bin/aw-worktime-ui-bridge.py + owner: root + group: root + mode: "0755" + + - name: Установить systemd unit AW worktime UI bridge + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-ui-bridge.service" + dest: /etc/systemd/system/aw-worktime-ui-bridge.service + owner: root + group: root + mode: "0644" + + - name: Установить systemd timer AW worktime UI bridge + ansible.builtin.copy: + src: "{{ aw_repo_root }}/aw-server/aw-worktime-ui-bridge.timer" + dest: /etc/systemd/system/aw-worktime-ui-bridge.timer + owner: root + group: root + mode: "0644" + - name: Перезагрузить systemd после установки AW worktime API ansible.builtin.systemd: daemon_reload: true @@ -326,6 +350,25 @@ enabled: true state: restarted + - name: Отключить legacy timer aw-worktime-afk-bridge (если есть) + ansible.builtin.systemd: + name: aw-worktime-afk-bridge.timer + enabled: false + state: stopped + failed_when: false + + - name: Включить и перезапустить AW worktime UI bridge timer + ansible.builtin.systemd: + name: aw-worktime-ui-bridge.timer + enabled: true + state: restarted + + - name: Выполнить разовый прогон AW worktime UI bridge + ansible.builtin.systemd: + name: aw-worktime-ui-bridge.service + state: started + failed_when: false + - name: Применить хотфиксы compiled JS чанков (Trends, Timespiral, Category helper) ansible.builtin.command: cmd: "/opt/activitywatch/aw-server/apply_webui_ru_patch.sh" diff --git a/ansible/provision_proxmox_ct_and_deploy_aw.yml b/ansible/provision_proxmox_ct_and_deploy_aw.yml index 40c6382..1ac786e 100644 --- a/ansible/provision_proxmox_ct_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_and_deploy_aw.yml @@ -11,6 +11,9 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-ui-bridge.py + - aw-worktime-ui-bridge.service + - aw-worktime-ui-bridge.timer - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js diff --git a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml index 5452935..e0ff9bc 100644 --- a/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml +++ b/ansible/provision_proxmox_ct_matrix_and_deploy_aw.yml @@ -11,6 +11,9 @@ - activitywatch-server.service - aw-worktime-api.py - aw-worktime-api.service + - aw-worktime-ui-bridge.py + - aw-worktime-ui-bridge.service + - aw-worktime-ui-bridge.timer - aw-worktime-panel.js - aw-server.env.example - aw-ru-patch.js diff --git a/aw-server/aw-worktime-ui-bridge.py b/aw-server/aw-worktime-ui-bridge.py new file mode 100644 index 0000000..d1a4cc3 --- /dev/null +++ b/aw-server/aw-worktime-ui-bridge.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +import json +import os +import urllib.error +import urllib.request +from datetime import datetime, timezone + + +AW_URL = os.environ.get("AW_SERVER_URL", "http://127.0.0.1:5600") +HOST = os.environ.get("AW_WORKTIME_HOST", "SHARKON2025") +STATE_PATH = os.environ.get( + "AW_WORKTIME_UI_BRIDGE_STATE", + "/var/lib/activitywatch/aw-worktime-ui-bridge-state.json", +) +TIMEOUT = float(os.environ.get("AW_WORKTIME_UI_BRIDGE_TIMEOUT", "20")) + + +SESSIONS_BUCKET = f"aw-worktime-sessions_{HOST}" +AFK_BUCKET = f"aw-watcher-afk_{HOST}" +WINDOW_BUCKET = f"aw-watcher-window_{HOST}" + + +def _req(method: str, path: str, payload=None): + data = None + headers = {} + if payload is not None: + data = json.dumps(payload, ensure_ascii=False).encode("utf-8") + headers["Content-Type"] = "application/json" + req = urllib.request.Request(AW_URL + path, data=data, headers=headers, method=method) + with urllib.request.urlopen(req, timeout=TIMEOUT) as r: + raw = r.read() + if not raw: + return None + return json.loads(raw.decode("utf-8")) + + +def ensure_bucket(bucket_id: str, event_type: str, client: str): + payload = {"client": client, "type": event_type, "hostname": HOST} + try: + _req("POST", f"/api/0/buckets/{bucket_id}", payload) + except urllib.error.HTTPError as e: + if e.code != 304: + raise + + +def load_state(): + try: + with open(STATE_PATH, "r", encoding="utf-8") as f: + data = json.load(f) + if isinstance(data, dict) and "last_ts" in data: + return data + except FileNotFoundError: + pass + except json.JSONDecodeError: + pass + return {"last_ts": "1970-01-01T00:00:00Z"} + + +def save_state(state): + os.makedirs(os.path.dirname(STATE_PATH), exist_ok=True) + tmp = STATE_PATH + ".tmp" + with open(tmp, "w", encoding="utf-8") as f: + json.dump(state, f, ensure_ascii=False) + os.replace(tmp, STATE_PATH) + + +def to_iso_utc(ts): + if ts.endswith("Z"): + return ts + return ts.replace("+00:00", "Z") + + +def build_window_title(users, active_count): + if not users: + return "RDP idle" + return f"RDP active ({active_count}): " + ", ".join(users) + + +def transform(events): + out_afk = [] + out_win = [] + last_ts = None + + for e in events: + ts = e.get("timestamp") + if not ts: + continue + duration = float(e.get("duration", 0.0)) + data = e.get("data") or {} + active_users = data.get("activeUsers") or [] + active_count = int(data.get("activeCount", len(active_users))) + is_active = active_count > 0 + + afk_data = {"status": "not-afk" if is_active else "afk", "source": "aw-worktime-ui-bridge"} + out_afk.append({"timestamp": ts, "duration": duration, "data": afk_data}) + + win_data = { + "app": "RDP", + "title": build_window_title(active_users, active_count), + "source": "aw-worktime-ui-bridge", + } + out_win.append({"timestamp": ts, "duration": duration, "data": win_data}) + last_ts = ts + + return out_afk, out_win, last_ts + + +def main(): + state = load_state() + last_ts = state.get("last_ts", "1970-01-01T00:00:00Z") + + ensure_bucket(AFK_BUCKET, "afkstatus", "aw-worktime-ui-bridge") + ensure_bucket(WINDOW_BUCKET, "currentwindow", "aw-worktime-ui-bridge") + + query = { + "query": [ + "events = query_bucket(find_bucket($bid));", + "RETURN = sort_by_timestamp(events);", + ], + "timeperiods": [[last_ts, to_iso_utc(datetime.now(timezone.utc).isoformat())]], + } + rows = _req("POST", f"/api/0/query/?bid={SESSIONS_BUCKET}", query) or [] + if not rows or not rows[0]: + return + + events = rows[0] + afk_events, win_events, new_last_ts = transform(events) + if not afk_events or not win_events or not new_last_ts: + return + + _req("POST", f"/api/0/buckets/{AFK_BUCKET}/events", afk_events) + _req("POST", f"/api/0/buckets/{WINDOW_BUCKET}/events", win_events) + save_state({"last_ts": new_last_ts}) + print(f"posted_afk={len(afk_events)} posted_win={len(win_events)} last_ts={new_last_ts}") + + +if __name__ == "__main__": + main() diff --git a/aw-server/aw-worktime-ui-bridge.service b/aw-server/aw-worktime-ui-bridge.service new file mode 100644 index 0000000..0a48ba5 --- /dev/null +++ b/aw-server/aw-worktime-ui-bridge.service @@ -0,0 +1,15 @@ +[Unit] +Description=AW Worktime UI bridge (sessions -> afk/window) +After=network-online.target activitywatch-server.service +Wants=network-online.target + +[Service] +Type=oneshot +Environment=AW_SERVER_URL=http://127.0.0.1:5600 +Environment=AW_WORKTIME_HOST=SHARKON2025 +ExecStart=/usr/bin/python3 /usr/local/bin/aw-worktime-ui-bridge.py +User=root +Group=root + +[Install] +WantedBy=multi-user.target diff --git a/aw-server/aw-worktime-ui-bridge.timer b/aw-server/aw-worktime-ui-bridge.timer new file mode 100644 index 0000000..d56cb22 --- /dev/null +++ b/aw-server/aw-worktime-ui-bridge.timer @@ -0,0 +1,11 @@ +[Unit] +Description=Run AW Worktime UI bridge every 30 seconds + +[Timer] +OnBootSec=20s +OnUnitActiveSec=30s +Unit=aw-worktime-ui-bridge.service +Persistent=true + +[Install] +WantedBy=timers.target diff --git a/aw-server/install_aw_server.sh b/aw-server/install_aw_server.sh index 50ba3e7..4ac7a30 100755 --- a/aw-server/install_aw_server.sh +++ b/aw-server/install_aw_server.sh @@ -26,6 +26,9 @@ VIEWS_JSON="$BOOTSTRAP_DIR/settings/views-default.json" CLASSES_JSON="$BOOTSTRAP_DIR/settings/classes-worktime.json" WORKTIME_API_SRC="$BOOTSTRAP_DIR/aw-worktime-api.py" WORKTIME_API_SERVICE_SRC="$BOOTSTRAP_DIR/aw-worktime-api.service" +WORKTIME_UI_BRIDGE_SRC="$BOOTSTRAP_DIR/aw-worktime-ui-bridge.py" +WORKTIME_UI_BRIDGE_SERVICE_SRC="$BOOTSTRAP_DIR/aw-worktime-ui-bridge.service" +WORKTIME_UI_BRIDGE_TIMER_SRC="$BOOTSTRAP_DIR/aw-worktime-ui-bridge.timer" for var_name in "${required_vars[@]}"; do if [[ -z "${!var_name:-}" ]]; then @@ -103,6 +106,24 @@ if [[ -f "$WORKTIME_API_SERVICE_SRC" ]]; then systemctl --no-pager --full status aw-worktime-api.service || true fi +if [[ -f "$WORKTIME_UI_BRIDGE_SRC" ]]; then + install -m 0755 "$WORKTIME_UI_BRIDGE_SRC" /usr/local/bin/aw-worktime-ui-bridge.py +fi + +if [[ -f "$WORKTIME_UI_BRIDGE_SERVICE_SRC" ]]; then + install -m 0644 "$WORKTIME_UI_BRIDGE_SERVICE_SRC" /etc/systemd/system/aw-worktime-ui-bridge.service +fi + +if [[ -f "$WORKTIME_UI_BRIDGE_TIMER_SRC" ]]; then + install -m 0644 "$WORKTIME_UI_BRIDGE_TIMER_SRC" /etc/systemd/system/aw-worktime-ui-bridge.timer + systemctl daemon-reload + systemctl disable --now aw-worktime-afk-bridge.timer >/dev/null 2>&1 || true + systemctl enable aw-worktime-ui-bridge.timer + systemctl restart aw-worktime-ui-bridge.timer + systemctl start aw-worktime-ui-bridge.service || true + systemctl --no-pager --full status aw-worktime-ui-bridge.timer || true +fi + for _ in $(seq 1 20); do if curl -fsS "http://127.0.0.1:${AW_SERVER_PORT}/api/0/info" >/dev/null 2>&1; then break diff --git a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss index 31ac5cf..0ed495e 100644 --- a/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss +++ b/windows/installkit/innosetup/AWatch-rus-InnoSetup.iss @@ -149,7 +149,8 @@ begin 'Укажите сервер ActivityWatch (куда агенты будут отправлять данные).', 'Если нужно, измените host/port. По умолчанию — наша конфигурация.' ); - { Worktime CSV/JSON reports are served by aw-worktime-api on :5610 (AwDefaultWorktimeReportBase). } + { Worktime CSV/JSON reports are served by aw-worktime-api on :5610 (AwDefaultWorktimeReportBase). + Standard AW "Сегодня" is backed by server-side aw-worktime-ui-bridge timer on AW host. } ServerHostPage.Add('ServerHost', False); ServerHostPage.Add('ServerPort', False); ServerHostPage.Values[0] := '{#AwDefaultServerHost}';