feat(dfir): add hayabusa forensic workflow integration

2026-05-14 15:04:07 +03:00
parent 563bd910d1
commit 0cce6fd08e
28 changed files with 1907 additions and 25 deletions
@@ -0,0 +1,76 @@
+# Hayabusa Artifact Workflow 2026-05-14
+
+This document records the server-side EVTX intake and archive workflow on `10.10.10.13`.
+
+## Directories
+
+- incoming packages:
+  - `/opt/hayabusa/inbox/incoming`
+- transient staging:
+  - `/opt/hayabusa/inbox/staging`
+- generated reports:
+  - `/opt/hayabusa/reports`
+- archived raw packages:
+  - `/opt/hayabusa/archive/packages/<HOST>/`
+- archived extracted payloads:
+  - `/opt/hayabusa/archive/extracted/<HOST>/<INTAKE_ID>/payload/`
+- state:
+  - `/opt/hayabusa/state/latest-intake.json`
+  - `/opt/hayabusa/state/latest-run`
+  - `/opt/hayabusa/state/latest-<HOST>`
+  - `/opt/hayabusa/state/logs`
+
+## Operator flow
+
+1. Accept a package into server inbox:
+
+```bash
+aw-hayabusa accept --package /path/to/HOST-YYYYMMDD-HHMMSS.zip --host HOST
+```
+
+2. Inspect queue:
+
+```bash
+aw-hayabusa inventory
+```
+
+3. Process queued packages:
+
+```bash
+aw-hayabusa process-inbox --mode incident
+```
+
+## Processing behavior
+
+- the package is extracted into staging;
+- host is resolved from explicit `--host`, sidecar `.host`, embedded `manifest.json`, or package name fallback;
+- if EVTX payload exists, Hayabusa analysis is launched through the existing runner modes;
+- regardless of success, the package and extracted payload are moved into archive;
+- `intake.json` records:
+  - package path
+  - host
+  - intake id
+  - sha256
+  - status
+  - extracted payload path
+  - report directory
+  - processed timestamp
+
+## Failure semantics
+
+- malformed or empty packages are not lost;
+- the workflow archives them with `status=failed-*`;
+- the operator can inspect archived payloads without touching AW runtime storage.
+
+## Validation evidence
+
+- `aw-hayabusa inventory` shows queue and archive counts
+- a synthetic package was accepted, archived, and recorded with:
+  - `status=failed-no-evtx`
+- synthetic artifacts were removed after validation so production storage stayed clean
+
+## Boundaries
+
+- this phase does not yet move packages from Windows automatically
+- this phase does not yet attach reports to AW-rus incidents or cases
+- successful report generation from real EVTX remains a later validation phase
@@ -0,0 +1,76 @@
+# Hayabusa AW-rus Integration 2026-05-14
+
+This document defines the bounded integration between Hayabusa DFIR and the normal AW-rus operator path.
+
+## Purpose
+
+Hayabusa is used as DFIR enrichment after incidents, not as a new real-time detector.
+
+## When to use Hayabusa follow-up
+
+Recommended triggers:
+
+- high-severity DLP incidents that justify host-side forensic review;
+- repeated incidents on the same host or user;
+- suspicious print, USB, email, or document-export activity that needs Windows event corroboration;
+- operator-driven escalation where case review needs EVTX-based timeline evidence.
+
+Not recommended:
+
+- routine low-signal incidents;
+- replacing normal AW-rus health/runtime checks;
+- pushing raw Sigma detections into AW buckets.
+
+## Operator path
+
+1. Export EVTX package on Windows with `export-evtx-for-hayabusa.ps1`.
+2. Transfer the resulting zip package to `10.10.10.13`.
+3. Run one of:
+
+```bash
+aw-hayabusa accept --package /path/to/HOST-YYYYMMDD-HHMMSS.zip --host HOST
+aw-hayabusa process-inbox --mode incident
+```
+
+or from Telegram bot:
+
+```text
+/aw_dfir /path/to/HOST-YYYYMMDD-HHMMSS.zip HOST [CASE_ID] [MODE]
+```
+
+Default mode is `incident`.
+
+## What gets linked to a case
+
+Case management stores only bounded metadata:
+
+- `tool=hayabusa`
+- `host`
+- `mode`
+- `status`
+- `intake_id`
+- `package_path`
+- `sha256`
+- `report_dir`
+- `summary_html`
+- `timeline_path`
+- `manifest_path`
+- `linked_at`
+- `link_source`
+
+The raw Sigma output, full timelines, and extracted payloads stay under `/opt/hayabusa`, not inside AW buckets or case comments.
+
+## UI behavior
+
+Case Management shows a short `DFIR` field:
+
+- `Hayabusa ok · incident`
+- `Hayabusa failed-* · incident`
+
+This is intentionally short; report paths remain operator-facing metadata, not primary UI content.
+
+## Boundaries
+
+- No raw forensic output is copied into normal AW runtime buckets.
+- No automatic case creation from Hayabusa findings.
+- Hayabusa remains an enrichment layer around incidents and investigations.
@@ -0,0 +1,163 @@
+# Hayabusa Operator and IB Guide 2026-05-14
+
+This document explains the role of Hayabusa inside `AW-rus` for operators and IB.
+
+## What Hayabusa adds
+
+Hayabusa adds a bounded DFIR layer for Windows Event Log analysis:
+
+- EVTX-based timeline review
+- Sigma-based detection enrichment
+- logon and activity context around an already interesting host or incident
+- forensic artifacts that can be attached to case review
+
+It is useful when `AW-rus` or DLP already surfaced something worth investigating further.
+
+## What Hayabusa does not replace
+
+Hayabusa is not:
+
+- a replacement for normal `AW-rus` runtime monitoring
+- a replacement for DLP policy enforcement
+- a real-time SIEM
+- a reason to copy raw Sigma output into AW buckets or case comments
+
+The normal operational path remains:
+
+- `AW-rus` for activity/runtime visibility
+- DLP collectors and policy engine for signal generation
+- case management for operator workflow
+- Hayabusa for bounded forensic enrichment
+
+## When operators should run it
+
+Recommended cases:
+
+- high-severity DLP incidents
+- repeated suspicious incidents on one host or user
+- print, USB, file export, or email activity that needs Windows event corroboration
+- investigation requests from IB after an incident is already known
+
+Do not run it for every minor signal. It is meant for escalation and investigation, not daily noise.
+
+## Operator workflow
+
+1. Export EVTX package on Windows:
+
+```powershell
+powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\AWatch-rus\export-evtx-for-hayabusa.ps1
+```
+
+2. Transfer the resulting zip package to `10.10.10.13`.
+
+3. Run server-side processing:
+
+```bash
+aw-hayabusa accept --package /path/to/HOST-YYYYMMDD-HHMMSS.zip --host HOST
+aw-hayabusa process-inbox --mode incident
+```
+
+Or use the Telegram operator path:
+
+```text
+/aw_dfir /path/to/HOST-YYYYMMDD-HHMMSS.zip HOST [CASE_ID] [MODE]
+```
+
+4. If a case already exists, link only bounded metadata to the case.
+
+## Where artifacts live
+
+Windows export staging:
+
+- `C:\ProgramData\AWatch-rus\forensics\evtx-exports`
+
+Server-side intake and reports:
+
+- incoming packages:
+  - `/opt/hayabusa/inbox/incoming`
+- transient staging:
+  - `/opt/hayabusa/inbox/staging`
+- archived raw packages:
+  - `/opt/hayabusa/archive/packages/<HOST>/`
+- archived extracted payloads:
+  - `/opt/hayabusa/archive/extracted/<HOST>/<INTAKE_ID>/payload/`
+- reports:
+  - `/opt/hayabusa/reports/<HOST>/<UTC_TIMESTAMP>_<MODE>[_LABEL]/`
+- run state and logs:
+  - `/opt/hayabusa/state`
+
+## What is stored in AW-rus
+
+Only bounded metadata is attached to a case:
+
+- tool
+- host
+- mode
+- status
+- intake id
+- package path
+- sha256
+- report directory
+- summary path
+- timeline path
+- manifest path
+- linked timestamp
+- link source
+
+Raw forensic output stays under `/opt/hayabusa`.
+
+## Retention and storage notes
+
+Windows-side export retention:
+
+- controlled by `aw_windows_evtx_retention_days`
+- default: `14` days
+
+Windows-side export channels:
+
+- controlled by `aw_windows_evtx_channels`
+- default set:
+  - `Security`
+  - `System`
+  - `Application`
+  - `Microsoft-Windows-PowerShell/Operational`
+  - `Microsoft-Windows-TerminalServices-LocalSessionManager/Operational`
+  - `Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational`
+
+Server-side storage:
+
+- kept outside standard AW buckets
+- kept outside normal DLP screenshot artifacts
+- intended for forensic review, not for routine dashboarding
+
+## IB view
+
+From an IB perspective, Hayabusa in this project is:
+
+- a post-incident enrichment layer
+- useful for Windows event corroboration and timeline reconstruction
+- intentionally separated from the main activity-monitoring data plane
+
+This design keeps the main operator UI readable while preserving forensic detail when needed.
+
+## Limits and false expectations to avoid
+
+- Sigma detections depend on the quality and completeness of Windows logging.
+- Missing or weak audit policy reduces value immediately.
+- No EVTX means no meaningful Hayabusa result.
+- A successful Hayabusa run does not prove malicious activity by itself.
+- A clean Hayabusa run does not prove the absence of suspicious behavior.
+- This contour is deliberately not an always-on detector and not a SIEM replacement.
+
+## Canonical companion docs
+
+- source and packaging:
+  - `docs/hayabusa-source-packaging-2026-05-14.md`
+- server runner:
+  - `docs/hayabusa-server-runner-2026-05-14.md`
+- artifact workflow:
+  - `docs/hayabusa-artifact-workflow-2026-05-14.md`
+- AW-rus integration:
+  - `docs/hayabusa-aw-rus-integration-2026-05-14.md`
+- Windows EVTX export:
+  - `docs/windows-hayabusa-evtx-export.md`
@@ -0,0 +1,124 @@
+# Hayabusa Server-Side Runner 2026-05-14
+
+This document records the production runner model for Hayabusa on `10.10.10.13`.
+
+## Install layout
+
+- root: `/opt/hayabusa`
+- pinned release: `/opt/hayabusa/releases/v3.9.0`
+- active symlink: `/opt/hayabusa/current`
+- operator entrypoint: `/usr/local/bin/aw-hayabusa`
+
+## Runtime directories
+
+- inbox: `/opt/hayabusa/inbox`
+- archive: `/opt/hayabusa/archive`
+- reports: `/opt/hayabusa/reports`
+- state: `/opt/hayabusa/state`
+
+## Operator entrypoint
+
+Supported helper subcommands:
+
+- `aw-hayabusa doctor`
+- `aw-hayabusa inventory`
+- `aw-hayabusa accept --package <zip> [--host HOST]`
+- `aw-hayabusa process-inbox [--mode incident] [--limit N]`
+- `aw-hayabusa profiles`
+- `aw-hayabusa version`
+
+Supported analysis modes:
+
+- `aw-hayabusa quick --input <file-or-dir> [--host HOST]`
+- `aw-hayabusa incident --input <file-or-dir> [--host HOST]`
+- `aw-hayabusa full --input <file-or-dir> [--host HOST]`
+
+## Mode intent
+
+- `quick`
+  - fast CSV timeline
+  - HTML summary
+  - logon summary
+  - intended for first-pass triage
+
+- `incident`
+  - JSONL timeline
+  - HTML summary
+  - logon summary
+  - intended for normal incident review
+
+- `full`
+  - JSONL timeline
+  - deprecated/noisy/unsupported rules enabled
+  - HTML summary
+  - logon summary
+  - intended for deeper DFIR review
+
+## Output naming
+
+Reports are stored under:
+
+- `/opt/hayabusa/reports/<HOST>/<UTC_TIMESTAMP>_<MODE>[_LABEL]/`
+
+Typical contents:
+
+- `timeline.csv` or `timeline.jsonl`
+- `summary.html`
+- `logon-summary-*.csv`
+- `run.log`
+- `manifest.json`
+
+Latest-run symlinks:
+
+- `/opt/hayabusa/state/latest-run`
+- `/opt/hayabusa/state/latest-<HOST>`
+
+## Intake and archive workflow
+
+Incoming packages:
+
+- `/opt/hayabusa/inbox/incoming/*.zip`
+
+Transient staging:
+
+- `/opt/hayabusa/inbox/staging/<INTAKE_ID>/`
+
+Archived raw packages:
+
+- `/opt/hayabusa/archive/packages/<HOST>/<INTAKE_ID>.zip`
+
+Archived extracted payloads:
+
+- `/opt/hayabusa/archive/extracted/<HOST>/<INTAKE_ID>/payload/`
+- intake metadata:
+  - `/opt/hayabusa/archive/extracted/<HOST>/<INTAKE_ID>/intake.json`
+
+State/log helpers:
+
+- `/opt/hayabusa/state/latest-intake.json`
+- `/opt/hayabusa/state/logs/`
+
+## Minimal operator flow
+
+1. Drop or copy an export package:
+   - `aw-hayabusa accept --package /path/to/HOST-YYYYMMDD-HHMMSS.zip`
+2. Check queue:
+   - `aw-hayabusa inventory`
+3. Process packages:
+   - `aw-hayabusa process-inbox --mode incident`
+
+## Validation baseline
+
+Minimum server-side validation:
+
+```bash
+aw-hayabusa doctor
+aw-hayabusa profiles
+aw-hayabusa inventory
+```
+
+## Boundaries
+
+- Hayabusa is not deployed as a daemon.
+- No AW bucket ingestion happens in this phase.
+- EVTX intake orchestration remains a later phase.
@@ -0,0 +1,59 @@
+# Hayabusa Source and Packaging Decision 2026-05-14
+
+## Decision
+
+- upstream source of truth: `Yamato-Security/hayabusa`
+- fork policy: do not use a fork unless a concrete required patch exists and is documented
+- runtime role in this project: `DFIR enrichment`
+
+## Pinned release
+
+- release tag: `v3.9.0`
+- server target platform: `x86_64`, `debian 13`
+- selected asset:
+  - `hayabusa-3.9.0-lin-x64-gnu.zip`
+- selected asset URL:
+  - `https://github.com/Yamato-Security/hayabusa/releases/download/v3.9.0/hayabusa-3.9.0-lin-x64-gnu.zip`
+
+## Packaging model
+
+- analysis host: `10.10.10.13`
+- install root: `/opt/hayabusa`
+- versioned release root: `/opt/hayabusa/releases/v3.9.0`
+- active symlink target:
+  - `/opt/hayabusa/current`
+- suggested executable path:
+  - `/opt/hayabusa/current/hayabusa`
+- suggested wrapper path:
+  - `/usr/local/bin/aw-hayabusa`
+
+## Artifact boundaries
+
+- raw incoming EVTX:
+  - `/opt/hayabusa/inbox`
+- processed EVTX archive:
+  - `/opt/hayabusa/archive`
+- generated reports:
+  - `/opt/hayabusa/reports`
+- run logs / metadata:
+  - `/opt/hayabusa/state`
+
+These paths are intentionally outside normal ActivityWatch buckets and outside ordinary DLP artifact roots.
+
+## Integrity note
+
+The official release currently does not publish a separate checksum asset in the GitHub release asset list.
+
+Therefore the deployment model should:
+
+1. download the pinned asset URL;
+2. calculate `sha256` locally during automation;
+3. store the computed value in deployment logs or a local manifest;
+4. fail deployment if the downloaded asset name or pinned tag does not match expectations.
+
+## Why this model
+
+- no dependency on an unreviewed fork;
+- reproducible server-side installation;
+- no attempt to run Hayabusa as a real-time daemon;
+- clean separation between `AW-rus` runtime data and forensic artifacts.
@@ -11,6 +11,8 @@
 - [ИБ-профиль DLP](../dlp-security-functional-spec-ru.md) - подробное описание реализованного DLP/monitoring-контура для службы ИБ
 - [Runtime status: DLP chain](../dlp-runtime-chain-status-2026-05-13.md) - фактический live-статус policy/cases/integrations/compliance
 - [Runtime status: Content analysis](../dlp-content-analysis-runtime-status-2026-05-13.md) - фактический live-статус dictionary/regex/OCR/IOC
+- [Hayabusa AW-rus integration](../hayabusa-aw-rus-integration-2026-05-14.md) - bounded DFIR enrichment path для incidents/cases/operator flow
+- [Hayabusa operator and IB guide](../hayabusa-operator-ib-guide-2026-05-14.md) - когда запускать forensic path, где лежат артефакты и какие у него границы

 ### Компоненты
 - [DLP Endpoint Monitoring](DLP-Endpoint-Monitoring) - мониторинг clipboard, печати, USB
@@ -0,0 +1,78 @@
+# Windows EVTX Export for Hayabusa
+
+This document defines the Windows-side export path for Hayabusa DFIR enrichment.
+
+## Purpose
+
+Windows hosts do not analyze EVTX locally for this contour.
+
+They export selected event logs into a bounded forensic staging area, and the server-side Hayabusa workflow on `10.10.10.13` analyzes those artifacts later.
+
+## Export script
+
+- script: `windows/export-evtx-for-hayabusa.ps1`
+- deployed path on Windows host:
+  - `<StateRoot>\export-evtx-for-hayabusa.ps1`
+
+Default config path:
+
+- `C:\ProgramData\AWatch-rus\deployment-config.json`
+
+## Default export root
+
+- `<StateRoot>\forensics\evtx-exports`
+- Ansible override variable: `aw_windows_forensics_root`
+- retention override variable: `aw_windows_evtx_retention_days`
+- channel override variable: `aw_windows_evtx_channels`
+
+Example:
+
+- `C:\ProgramData\AWatch-rus\forensics\evtx-exports`
+
+Each run creates:
+
+- `<forensics-root>\<HOST>-<YYYYMMDD-HHMMSS>\evtx\*.evtx`
+- `<forensics-root>\<HOST>-<YYYYMMDD-HHMMSS>\manifest.json`
+- optional zip:
+  - `<forensics-root>\<HOST>-<YYYYMMDD-HHMMSS>.zip`
+
+## Default channel set
+
+- `Security`
+- `System`
+- `Application`
+- `Microsoft-Windows-PowerShell/Operational`
+- `Microsoft-Windows-TerminalServices-LocalSessionManager/Operational`
+- `Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational`
+
+Notes:
+
+- `Sysmon` is intentionally not assumed by default.
+- If `Sysmon` exists in the environment, it should be added later as an explicit extension.
+- the channel list is now carried through deployment config and validation, not left as an implicit script default.
+
+## Retention
+
+- default retention: `14` days
+- cleanup is local to the forensic export root
+- old export directories and zip packages are removed after the retention cutoff
+- retention is now exposed as `aw_windows_evtx_retention_days` in Ansible vars
+
+## Example run
+
+```powershell
+powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\AWatch-rus\export-evtx-for-hayabusa.ps1
+```
+
+Example with custom window:
+
+```powershell
+powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\AWatch-rus\export-evtx-for-hayabusa.ps1 -DaysBack 1
+```
+
+## Boundaries
+
+- output stays outside standard AW buckets
+- output stays outside normal DLP screenshot artifacts
+- this phase only defines and validates Windows export
+- transfer to `10.10.10.13` and Hayabusa execution belong to later phases