diff --git a/README.md b/README.md index 0280ab7..30cbada 100755 --- a/README.md +++ b/README.md @@ -227,6 +227,9 @@ collectors. [docs/registry/](docs/registry/REGISTER_RU_SOFTWARE_READINESS_RU.md). - Gitea Wiki используется только как навигация, не как единственный источник документов. +- Российский build-runner и release evidence описаны в + [RU_BUILD_RUNNER_READINESS_RU.md](docs/registry/RU_BUILD_RUNNER_READINESS_RU.md). +- GitHub remains public mirror only. - [Позиционирование для реестра российского ПО](docs/RUSSIAN_SOFTWARE_REGISTRY_POSITIONING_RU.md) - [Сведения для подачи в реестр](REGISTER_RU_SOFTWARE.md) diff --git a/docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md b/docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md new file mode 100644 index 0000000..2ede491 --- /dev/null +++ b/docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md @@ -0,0 +1,114 @@ +# Build-runner setup runbook + +Статус: registry-readiness runbook. Команды предназначены для нового +российского build-runner `awatch-build-01`. Runbook не меняет product runtime, +API, UI или deployment behavior. + +## Базовая подготовка + +```bash +hostnamectl set-hostname awatch-build-01 +apt update +apt upgrade +``` + +## Базовые пакеты + +```bash +apt install -y \ + curl \ + wget \ + git \ + jq \ + ca-certificates \ + gnupg \ + build-essential \ + pkg-config \ + libssl-dev \ + clang \ + cmake \ + protobuf-compiler \ + nodejs \ + npm \ + python3 \ + unzip \ + tar \ + rsync +``` + +`protobuf-compiler`, `nodejs` и `npm` нужны только если соответствующие +components/checks используются в текущем release candidate. Если package policy +организации требует другой способ установки, использовать корпоративный +approved mirror/toolchain и зафиксировать это в release evidence. + +## Rust toolchain + +Вариант через rustup допустим только если он разрешен политикой владельца +инфраструктуры: + +```bash +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -o /tmp/rustup-init.sh +sh /tmp/rustup-init.sh +``` + +Альтернатива: установить Rust toolchain из корпоративного mirror/package +repository и зафиксировать источник установки в release evidence. Скрипты +репозитория не должны выполнять `curl | sh` и не должны сами устанавливать +toolchain. + +## Проверка инструментов + +```bash +rustc --version +cargo --version +git --version +node --version +npm --version +jq --version +``` + +Если Node.js/npm не устанавливались, зафиксировать это как `skipped: tool not +installed` для соответствующих smoke checks. + +## Пользователь build + +```bash +useradd --create-home --shell /bin/bash build +usermod -aG sudo build +``` + +Доступ должен быть key-based SSH. Пароли, токены, приватные ключи и cookies не +хранить в репозитории. + +## Безопасная модель доступа + +- SSH key based access. +- No secrets in repo. +- GitHub tokens, Gitea tokens и SSH private keys только в environment variables + или protected files outside repo. +- Protected files должны иметь ограниченные permissions и не попадать в + source archive. +- Build scripts не должны менять remotes, git history, tags или выполнять + auto-push. + +## Подключение к Gitea + +HTTPS clone: + +```bash +git clone https://git.iri1968.dpdns.org/awatch-rus/AWatch-rus.git +``` + +SSH clone может быть добавлен отдельным шагом после настройки ключей Gitea. +SSH private key не хранить в репозитории. + +## Первичная проверка репозитория + +```bash +cd AWatch-rus +git remote -v +bash scripts/registry_readiness_check.sh +``` + +GitHub используется как public mirror only и не является primary registry +source/build/release contour. diff --git a/docs/registry/REGISTER_RU_SOFTWARE_READINESS_RU.md b/docs/registry/REGISTER_RU_SOFTWARE_READINESS_RU.md index 7a5e270..68a05f5 100644 --- a/docs/registry/REGISTER_RU_SOFTWARE_READINESS_RU.md +++ b/docs/registry/REGISTER_RU_SOFTWARE_READINESS_RU.md @@ -1,8 +1,8 @@ # AWatch-rus: readiness для реестра российского ПО Статус: подготовительный документ. Документ фиксирует текущее состояние -registry-readiness пакета и не является юридическим заключением о готовности -подачи или принятия AWatch-rus в реестр российского ПО. +registry-readiness пакета, не подтверждает готовность подачи и не подтверждает +юридический результат рассмотрения AWatch-rus для реестра российского ПО. ## Назначение @@ -58,6 +58,16 @@ Done / partially done: - Offsite backup в РФ. - Финальная юридическая проверка registry package. +## Новый статус registry-readiness + +- Russian Git contour: partially done / done. +- Gitea backup: partially done. +- Russian build-runner: planned. +- Release artifacts storage in RF: planned. +- Release evidence automation: partially done after this task. +- Restore test: required. +- Legal rightsholder confirmation: required. + ## Ограничения формулировок AWatch-rus не заявляется как сертифицированное средство защиты информации, diff --git a/docs/registry/REGISTRY_READINESS_CHANGELOG_RU.md b/docs/registry/REGISTRY_READINESS_CHANGELOG_RU.md index 6e57eb6..782c868 100644 --- a/docs/registry/REGISTRY_READINESS_CHANGELOG_RU.md +++ b/docs/registry/REGISTRY_READINESS_CHANGELOG_RU.md @@ -1,5 +1,40 @@ # Registry readiness changelog +## 2026-06-21 + +Added: + +- Russian build-runner readiness docs. +- Build-runner setup runbook. +- Release evidence runbook. +- Release artifacts storage policy. +- `scripts/build_release_evidence.sh`. +- `scripts/check_release_evidence.sh`. + +Changed: + +- Updated registry evidence manifest with build-runner plan. +- Updated registry readiness checks for build-runner and release evidence + requirements. + +Runtime impact: + +- No runtime/product code changes. +- No API changes. +- No UI changes. +- Rust/runtime checks not required: registry documentation and release evidence + script update only. + +Remaining gaps: + +- Actual `awatch-build-01` server provisioning. +- Build-runner first successful release candidate build. +- SBOM tool installation decision. +- Release artifacts storage in RF. +- Restore test for Gitea backup. +- Legal rightsholder confirmation. +- Final legal review. + ## 2026-06-20 / 2026-06-21 Added: diff --git a/docs/registry/RELEASE_ARTIFACTS_STORAGE_RU.md b/docs/registry/RELEASE_ARTIFACTS_STORAGE_RU.md new file mode 100644 index 0000000..7a47aac --- /dev/null +++ b/docs/registry/RELEASE_ARTIFACTS_STORAGE_RU.md @@ -0,0 +1,46 @@ +# Release artifacts storage + +Статус: planned / partially implemented. Required before registry submission: +yes. + +## Целевая модель + +Release artifacts должны храниться в российском контуре. GitHub Releases не +должны быть primary release storage для registry-readiness. Если GitHub +Releases используются, они могут быть только public mirror. + +Возможные варианты: + +- отдельный каталог на `awatch-build-01`; +- отдельный REG.RU VPS; +- российское S3-compatible хранилище; +- `releases.iri1968.dpdns.org` или `releases.awatch-rus.ru` в будущем. + +## Обязательные файлы + +- Source archive. +- Binary archive. +- `SHA256SUMS`. +- SBOM CycloneDX. +- SBOM SPDX. +- `cargo-metadata.json`. +- `cargo-tree.txt`. +- Smoke logs. +- Release evidence manifest. +- Release notes. +- Version output. +- Health/readiness output, если применимо. + +## Evidence требования + +Для каждого release candidate фиксировать: + +- storage location; +- timestamp; +- release version; +- commit SHA или tag; +- checksum verification result; +- responsible person placeholder: `[ЗАПОЛНИТЬ ПРАВООБЛАДАТЕЛЕМ]`. + +До выбора и проверки хранилища нельзя утверждать, что release artifact storage +полностью готов для подачи в реестр. diff --git a/docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md b/docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md index 548d7f9..df6f2b5 100644 --- a/docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md +++ b/docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md @@ -44,3 +44,27 @@ build-runner и storage release artifacts в РФ или документировать принятую владельцем схему. До этого нельзя утверждать, что source/build/release contour полностью закрыт для подачи. + +## Build-runner evidence + +Для `awatch-build-01` фиксировать: + +- hostname; +- provider; +- OS; +- `rustc` / `cargo` versions; +- git commit; +- tag/version; +- checks; +- artifacts; +- SBOM; +- SHA256; +- smoke logs; +- skipped checks with reasons; +- `generated_at`; +- operator/responsible person placeholder: + `[ЗАПОЛНИТЬ ПРАВООБЛАДАТЕЛЕМ]`. + +Evidence должен формироваться скриптом `scripts/build_release_evidence.sh` и +проверяться через `scripts/check_release_evidence.sh`. До первого успешного +release candidate build нельзя утверждать, что release evidence production-ready. diff --git a/docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md b/docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md new file mode 100644 index 0000000..7c45409 --- /dev/null +++ b/docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md @@ -0,0 +1,71 @@ +# Release evidence runbook + +Статус: registry-readiness runbook. Документ описывает выпуск release +candidate на российском build-runner `awatch-build-01`. Он не утверждает, что +release evidence уже production-ready до первого успешного запуска. + +## Процесс release candidate + +1. Checkout конкретного commit SHA или tag. +2. Clean build environment. +3. Выполнить `cargo fmt --all --check`. +4. Выполнить `cargo test --workspace`. +5. Выполнить `cargo clippy --workspace --all-targets -- -D warnings`. +6. Выполнить `cargo build --workspace --release`. +7. Выполнить `bash scripts/registry_readiness_check.sh`. +8. Выполнить release evidence check: + `bash scripts/check_release_evidence.sh `. +9. Запустить доступные smoke-тесты: + `node scripts/deployment-readiness-smoke.mjs`, + `node scripts/pilot-validation-smoke.mjs`, + `node scripts/browser-conformance-smoke.mjs`, если не требует live stand, + `bash scripts/validate_install_kit.sh`, если применимо. +10. Сформировать source archive. +11. Сформировать binary artifacts archive. +12. Сформировать `SHA256SUMS`. +13. Сформировать `cargo-metadata.json`. +14. Сформировать `cargo-tree.txt`. +15. Сформировать SBOM CycloneDX/SPDX, если tools доступны. +16. Сформировать release evidence manifest. +17. Сохранить logs. +18. Сохранить final report. + +Если smoke требует live stand, не удалять и не скрывать его. В release report +фиксировать: `skipped: requires live stand`. + +## Автоматизация + +Основной скрипт: + +```bash +RELEASE_VERSION=registry-candidate-YYYYMMDD-HHMMSS \ +RELEASE_COMMIT= \ +OUTPUT_DIR=release-evidence/ \ +bash scripts/build_release_evidence.sh +``` + +Для documentation-only изменения допустимо: + +```bash +DOCS_ONLY=1 bash scripts/build_release_evidence.sh +``` + +`DOCS_ONLY=1` должен явно попасть в report. Это не должно использоваться для +product/runtime release candidate. + +## Обязательные результаты + +- `release-evidence-manifest.json`. +- `RELEASE_EVIDENCE_REPORT_RU.md`. +- `SHA256SUMS`. +- `logs/`. +- Source archive. +- Binary artifacts archive или documented skip только для `DOCS_ONLY=1`. +- `cargo-metadata.json` или documented skip. +- `cargo-tree.txt` или documented skip. + +## Conservative claims + +Release evidence не является подтверждением юридически завершенной регистрации +в реестре РФ. Перед официальной подачей требуются rightsholder confirmation, +финальная юридическая проверка и проверка фактического build-runner. diff --git a/docs/registry/RU_BUILD_RUNNER_READINESS_RU.md b/docs/registry/RU_BUILD_RUNNER_READINESS_RU.md new file mode 100644 index 0000000..15a776b --- /dev/null +++ b/docs/registry/RU_BUILD_RUNNER_READINESS_RU.md @@ -0,0 +1,101 @@ +# Russian build-runner readiness + +Статус: registry-readiness plan. Документ описывает целевую архитектуру +российского build-runner для AWatch-rus и не утверждает, что build-runner уже +развернут или что release evidence уже production-ready. + +## Назначение + +Build-runner нужен для воспроизводимой сборки release candidate, выполнения +проверок, подготовки artifacts и формирования release evidence в российском +контуре. Он должен быть отделен от Gitea-сервера, чтобы тяжелые Rust workspace +build/test/check задачи не конкурировали с Git-хранилищем, HTTPS reverse proxy +и backup-процедурами. + +## Целевая роль + +| Параметр | Значение | +| --- | --- | +| Recommended hostname | `awatch-build-01` | +| Recommended provider | REG.RU VPS / cloud server или другой российский сервер | +| Target location | Russian Federation | +| Role | российский контур сборки и выпуска release artifacts | +| Source repository | `https://git.iri1968.dpdns.org/awatch-rus/AWatch-rus` | +| GitHub role | public mirror only | + +## Почему не awatch-git-01 + +`awatch-git-01` должен оставаться Git-сервером: self-hosted Gitea, +repository, `docs/registry/`, HTTPS/Nginx и Gitea backup. Тяжелые сборки, +`cargo test --workspace`, `cargo clippy`, SBOM generation и packaging могут +создавать высокую нагрузку на CPU, RAM и диск. Поэтому registry-readiness +architecture разделяет: + +- `awatch-git-01`: source repository and documentation contour; +- `awatch-build-01`: build/test/check/release evidence contour. + +## Минимальная конфигурация + +- 4 vCPU. +- 8 GB RAM. +- 100+ GB SSD/NVMe. +- Debian 12 или Ubuntu 24.04 LTS. + +## Экономная конфигурация только для docs/check + +- 2 vCPU. +- 4 GB RAM. +- 80 GB SSD. + +Экономная конфигурация допустима только для documentation checks, +registry-readiness checks и легких smoke. Ее нельзя считать достаточной для +полного Rust workspace release build без отдельной проверки. + +## Рекомендуемая конфигурация для Rust workspace + +- 4 vCPU. +- 8-16 GB RAM. +- 120+ GB SSD/NVMe. +- Debian 12 или Ubuntu 24.04 LTS. + +## Release artifacts storage + +Release artifacts должны храниться в российском контуре. Целевые варианты +описаны в `docs/registry/RELEASE_ARTIFACTS_STORAGE_RU.md`. GitHub Releases +могут быть public mirror, но не primary release storage для registry-readiness. + +## Evidence files + +Build-runner должен формировать: + +- source archive; +- binary artifacts archive; +- `SHA256SUMS`; +- `cargo-metadata.json`; +- `cargo-tree.txt`; +- SBOM CycloneDX, если инструмент доступен; +- SBOM SPDX, если инструмент доступен; +- logs for fmt/test/clippy/build/registry/smoke checks; +- release evidence manifest; +- release evidence report; +- version output, если применимо; +- health/readiness output, если применимо. + +## Требует подтверждения правообладателем + +- Фактический provider и location build-runner. +- Ответственный за build-runner. +- Политика доступа. +- Release artifacts storage в РФ. +- Offsite backup policy для release artifacts. +- Достаточность evidence для официального пакета документов. + +## Текущий статус + +- Russian Git contour: partially done / done. +- Gitea backup: partially done. +- Russian build-runner: planned. +- Release artifacts storage in RF: planned. +- Release evidence automation: partially done after this task. +- Restore test: required. +- Legal rightsholder confirmation: required. diff --git a/docs/registry/registry-evidence-manifest.json b/docs/registry/registry-evidence-manifest.json index 537bc7d..52e699f 100644 --- a/docs/registry/registry-evidence-manifest.json +++ b/docs/registry/registry-evidence-manifest.json @@ -38,6 +38,41 @@ "restore_tested": false, "production_ready": false }, + "build_runner": { + "status": "planned", + "target_hostname": "awatch-build-01", + "target_provider": "REG.RU VPS / cloud server or another Russian-hosted server", + "target_location": "Russian Federation", + "separate_from_git_server": true, + "recommended_specs": { + "cpu": "4 vCPU", + "ram": "8-16 GB", + "disk": "120+ GB SSD/NVMe", + "os": [ + "Debian 12", + "Ubuntu 24.04 LTS" + ] + }, + "required_checks": [ + "cargo fmt --all --check", + "cargo test --workspace", + "cargo clippy --workspace --all-targets -- -D warnings", + "cargo build --workspace --release", + "registry_readiness_check", + "release_evidence_check" + ], + "release_evidence": { + "source_archive": true, + "binary_archive": true, + "sha256sums": true, + "cargo_metadata": true, + "cargo_tree": true, + "sbom_cyclonedx": "if_tool_available", + "sbom_spdx": "if_tool_available", + "smoke_logs": true, + "release_manifest": true + } + }, "github_role_note": "GitHub is a public mirror only, not the target primary source/build/release contour for registry-readiness.", "remaining_gaps": [ "legal rightsholder confirmation", @@ -49,11 +84,11 @@ "final registry legal review" ], "forbidden_claims_not_made": [ - "FSTEC/FSB certification", - "SIEM replacement", - "DLP replacement", - "ML/LLM-based detection", - "cloud dependency", - "automatic remediation" + "not_claimed: FSTEC/FSB certification", + "not_claimed: SIEM replacement", + "not_claimed: DLP replacement", + "not_claimed: ML/LLM-based detection", + "not_claimed: cloud dependency", + "not_claimed: automatic remediation" ] } diff --git a/scripts/build_release_evidence.sh b/scripts/build_release_evidence.sh new file mode 100644 index 0000000..69e56f5 --- /dev/null +++ b/scripts/build_release_evidence.sh @@ -0,0 +1,312 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT" +export GIT_LFS_SKIP_SMUDGE=1 + +RELEASE_VERSION="${RELEASE_VERSION:-registry-candidate-$(date +%Y%m%d-%H%M%S)}" +RELEASE_COMMIT="${RELEASE_COMMIT:-$(git rev-parse HEAD)}" +OUTPUT_DIR="${OUTPUT_DIR:-release-evidence/${RELEASE_VERSION}}" +DOCS_ONLY="${DOCS_ONLY:-0}" +CARGO_MANIFEST_PATH="${CARGO_MANIFEST_PATH:-}" + +if [[ -z "$CARGO_MANIFEST_PATH" ]]; then + if [[ -f Cargo.toml ]]; then + CARGO_MANIFEST_PATH="Cargo.toml" + elif [[ -f adk-rust/Cargo.toml ]]; then + CARGO_MANIFEST_PATH="adk-rust/Cargo.toml" + fi +fi + +if [[ -n "$CARGO_MANIFEST_PATH" ]]; then + CARGO_WORKSPACE_DIR="$(cd "$(dirname "$CARGO_MANIFEST_PATH")" && pwd)" +else + CARGO_WORKSPACE_DIR="" +fi + +LOG_DIR="$OUTPUT_DIR/logs" +ARTIFACT_DIR="$OUTPUT_DIR/artifacts" +SKIP_FILE="$OUTPUT_DIR/skipped-checks.txt" +CHECKS_FILE="$OUTPUT_DIR/checks.tsv" + +mkdir -p "$LOG_DIR" "$ARTIFACT_DIR" +: > "$SKIP_FILE" +: > "$CHECKS_FILE" + +record_check() { + local name="$1" + local status="$2" + local detail="${3:-}" + printf '%s\t%s\t%s\n' "$name" "$status" "$detail" >> "$CHECKS_FILE" +} + +run_logged() { + local name="$1" + shift + local log="$LOG_DIR/${name}.log" + { + printf 'command:' + printf ' %q' "$@" + printf '\nstarted_at: %s\n\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" + } > "$log" + if "$@" >> "$log" 2>&1; then + printf '\nfinished_at: %s\nstatus: ok\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$log" + record_check "$name" "ok" "$log" + else + local status=$? + printf '\nfinished_at: %s\nstatus: fail\nexit_code: %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$status" >> "$log" + record_check "$name" "fail" "$log" + return "$status" + fi +} + +run_logged_in_dir() { + local name="$1" + local dir="$2" + shift 2 + local log="$LOG_DIR/${name}.log" + { + printf 'working_dir: %s\n' "$dir" + printf 'command:' + printf ' %q' "$@" + printf '\nstarted_at: %s\n\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" + } > "$log" + if (cd "$dir" && "$@") >> "$log" 2>&1; then + printf '\nfinished_at: %s\nstatus: ok\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$log" + record_check "$name" "ok" "$log" + else + local status=$? + printf '\nfinished_at: %s\nstatus: fail\nexit_code: %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$status" >> "$log" + record_check "$name" "fail" "$log" + return "$status" + fi +} + +run_optional() { + local name="$1" + shift + if "$@"; then + record_check "$name" "ok" "" + else + local status=$? + record_check "$name" "skipped" "exit_code=$status" + printf '%s: skipped: exit_code=%s\n' "$name" "$status" >> "$SKIP_FILE" + fi +} + +skip_check() { + local name="$1" + local reason="$2" + record_check "$name" "skipped" "$reason" + printf '%s: skipped: %s\n' "$name" "$reason" >> "$SKIP_FILE" +} + +capture_command() { + local name="$1" + shift + local log="$LOG_DIR/${name}.log" + if command -v "$1" >/dev/null 2>&1; then + "$@" > "$log" 2>&1 || true + else + printf 'skipped: tool not installed: %s\n' "$1" > "$log" + fi +} + +if [[ "$RELEASE_COMMIT" != "$(git rev-parse HEAD)" ]]; then + git cat-file -e "${RELEASE_COMMIT}^{commit}" + git -c filter.lfs.smudge= -c filter.lfs.process= -c filter.lfs.required=false checkout --detach "$RELEASE_COMMIT" +fi + +{ + git remote -v +} > "$LOG_DIR/git-remotes.log" 2>&1 +git status --short > "$LOG_DIR/git-status-short.log" 2>&1 +git rev-parse HEAD > "$LOG_DIR/git-rev-parse-head.log" 2>&1 +git log --oneline -20 > "$LOG_DIR/git-log-oneline-20.log" 2>&1 +capture_command rustc-version rustc --version +capture_command cargo-version cargo --version +uname -a > "$LOG_DIR/uname.log" 2>&1 +capture_command hostnamectl hostnamectl +df -h > "$LOG_DIR/df-h.log" 2>&1 +free -h > "$LOG_DIR/free-h.log" 2>&1 || true + +if [[ "$DOCS_ONLY" == "1" ]]; then + skip_check "cargo_fmt" "DOCS_ONLY=1" + skip_check "cargo_test_workspace" "DOCS_ONLY=1" + skip_check "cargo_clippy_workspace" "DOCS_ONLY=1" + skip_check "cargo_build_workspace_release" "DOCS_ONLY=1" +elif [[ -z "$CARGO_WORKSPACE_DIR" ]]; then + run_logged cargo-workspace-missing false +else + run_logged_in_dir cargo-fmt "$CARGO_WORKSPACE_DIR" cargo fmt --all --check + run_logged_in_dir cargo-test-workspace "$CARGO_WORKSPACE_DIR" cargo test --workspace + run_logged_in_dir cargo-clippy-workspace "$CARGO_WORKSPACE_DIR" cargo clippy --workspace --all-targets -- -D warnings + run_logged_in_dir cargo-build-workspace-release "$CARGO_WORKSPACE_DIR" cargo build --workspace --release +fi + +if [[ -f scripts/registry_readiness_check.sh ]]; then + run_logged registry-readiness-check bash scripts/registry_readiness_check.sh +else + skip_check "registry_readiness_check" "script missing" +fi + +for smoke in \ + scripts/deployment-readiness-smoke.mjs \ + scripts/pilot-validation-smoke.mjs \ + scripts/browser-conformance-smoke.mjs +do + if [[ -f "$smoke" ]]; then + if [[ "$smoke" == *browser-conformance-smoke.mjs ]]; then + skip_check "$(basename "$smoke")" "requires live stand unless explicitly run by operator" + continue + fi + run_logged "$(basename "$smoke" .mjs)" node "$smoke" + else + skip_check "$(basename "$smoke")" "script missing" + fi +done + +if [[ -f scripts/validate_install_kit.sh ]]; then + run_logged validate-install-kit bash scripts/validate_install_kit.sh +else + skip_check "validate_install_kit" "script missing" +fi + +if [[ "$DOCS_ONLY" == "1" ]]; then + skip_check "cargo_metadata" "DOCS_ONLY=1" + skip_check "cargo_tree" "DOCS_ONLY=1" +elif [[ -n "$CARGO_WORKSPACE_DIR" ]] && command -v cargo >/dev/null 2>&1; then + run_logged_in_dir cargo-metadata "$CARGO_WORKSPACE_DIR" cargo metadata --format-version 1 --no-deps + cp "$LOG_DIR/cargo-metadata.log" "$OUTPUT_DIR/cargo-metadata.json" + if (cd "$CARGO_WORKSPACE_DIR" && cargo tree --version >/dev/null 2>&1); then + run_logged_in_dir cargo-tree "$CARGO_WORKSPACE_DIR" cargo tree + cp "$LOG_DIR/cargo-tree.log" "$OUTPUT_DIR/cargo-tree.txt" + else + skip_check "cargo_tree" "cargo tree not available" + fi +elif [[ -z "$CARGO_WORKSPACE_DIR" ]]; then + skip_check "cargo_metadata" "Cargo.toml not found" + skip_check "cargo_tree" "Cargo.toml not found" +else + skip_check "cargo_metadata" "cargo not installed" + skip_check "cargo_tree" "cargo not installed" +fi + +SOURCE_ARCHIVE="$ARTIFACT_DIR/${RELEASE_VERSION}-source.tar.gz" +git -c filter.lfs.smudge= -c filter.lfs.process= -c filter.lfs.required=false archive --format=tar.gz --output="$SOURCE_ARCHIVE" HEAD +record_check "source_archive" "ok" "$SOURCE_ARCHIVE" + +BINARY_ARCHIVE="$ARTIFACT_DIR/${RELEASE_VERSION}-binaries.tar.gz" +if [[ "$DOCS_ONLY" == "1" ]]; then + printf 'skipped: DOCS_ONLY=1\n' > "$BINARY_ARCHIVE.skip" + skip_check "binary_archive" "DOCS_ONLY=1" +elif [[ -d target/release ]]; then + tar -czf "$BINARY_ARCHIVE" target/release + record_check "binary_archive" "ok" "$BINARY_ARCHIVE" +else + skip_check "binary_archive" "target/release missing" +fi + +if command -v cargo-cyclonedx >/dev/null 2>&1; then + run_logged sbom-cyclonedx cargo cyclonedx --format json --output-file "$OUTPUT_DIR/sbom-cyclonedx.json" +else + skip_check "sbom_cyclonedx" "cargo-cyclonedx not installed" +fi + +if command -v syft >/dev/null 2>&1; then + run_logged sbom-syft-cyclonedx syft . -o cyclonedx-json="$OUTPUT_DIR/sbom-syft-cyclonedx.json" + run_logged sbom-syft-spdx syft . -o spdx-json="$OUTPUT_DIR/sbom-spdx.json" +else + skip_check "sbom_spdx" "syft not installed" + skip_check "sbom_syft_cyclonedx" "syft not installed" +fi + +GENERATED_AT="$(date -u +%Y-%m-%dT%H:%M:%SZ)" +BUILD_RUNNER_HOST="$(hostname 2>/dev/null || printf unknown)" +PRIMARY_SOURCE_REPOSITORY="https://git.iri1968.dpdns.org/awatch-rus/AWatch-rus" + +python3 - "$OUTPUT_DIR/release-evidence-manifest.json" "$RELEASE_VERSION" "$RELEASE_COMMIT" "$GENERATED_AT" "$BUILD_RUNNER_HOST" "$PRIMARY_SOURCE_REPOSITORY" <<'PY' +import json +import sys +from pathlib import Path + +manifest_path, release_version, release_commit, generated_at, build_runner, primary_source = sys.argv[1:] +root = Path(manifest_path).parent +checks = [] +checks_file = root / "checks.tsv" +if checks_file.exists(): + for line in checks_file.read_text(encoding="utf-8").splitlines(): + name, status, detail = (line.split("\t") + ["", ""])[:3] + checks.append({"name": name, "status": status, "detail": detail}) + +artifacts = [] +for path in sorted((root / "artifacts").glob("*")): + if path.is_file(): + artifacts.append(path.relative_to(root).as_posix()) + +data = { + "product": "AWatch-rus", + "release_version": release_version, + "release_commit": release_commit, + "build_runner": build_runner, + "primary_source_repository": primary_source, + "github_role": "public_mirror_only", + "generated_at": generated_at, + "checks": checks, + "artifacts": artifacts, + "docs_only": bool(__import__("os").environ.get("DOCS_ONLY") == "1"), +} +Path(manifest_path).write_text(json.dumps(data, ensure_ascii=False, indent=2) + "\n", encoding="utf-8") +PY + +cat > "$OUTPUT_DIR/RELEASE_EVIDENCE_REPORT_RU.md" < SHA256SUMS +) + +if [[ -f scripts/check_release_evidence.sh ]]; then + bash scripts/check_release_evidence.sh "$OUTPUT_DIR" +else + skip_check "release_evidence_check" "script missing" +fi + +printf 'release_evidence_dir=%s\n' "$OUTPUT_DIR" diff --git a/scripts/check_release_evidence.sh b/scripts/check_release_evidence.sh new file mode 100644 index 0000000..1823da7 --- /dev/null +++ b/scripts/check_release_evidence.sh @@ -0,0 +1,129 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ $# -ne 1 ]]; then + printf 'usage: %s \n' "$0" >&2 + exit 2 +fi + +EVIDENCE_DIR="$1" +failures=() + +fail() { + failures+=("$1") +} + +require_file() { + local path="$1" + if [[ ! -s "$EVIDENCE_DIR/$path" ]]; then + fail "missing_or_empty:$path" + fi +} + +require_dir() { + local path="$1" + if [[ ! -d "$EVIDENCE_DIR/$path" ]]; then + fail "missing_directory:$path" + fi +} + +if [[ ! -d "$EVIDENCE_DIR" ]]; then + printf 'release_evidence_check=fail\nmissing_directory:%s\n' "$EVIDENCE_DIR" + exit 2 +fi + +require_file "release-evidence-manifest.json" +require_file "RELEASE_EVIDENCE_REPORT_RU.md" +require_file "SHA256SUMS" +require_dir "logs" +require_dir "artifacts" + +if [[ -s "$EVIDENCE_DIR/SHA256SUMS" ]]; then + (cd "$EVIDENCE_DIR" && sha256sum -c SHA256SUMS >/dev/null) || fail "sha256sum_check_failed" +fi + +if [[ -s "$EVIDENCE_DIR/release-evidence-manifest.json" ]]; then + if command -v jq >/dev/null 2>&1; then + jq -e . "$EVIDENCE_DIR/release-evidence-manifest.json" >/dev/null || fail "invalid_json:release-evidence-manifest.json" + jq -e ' + .product == "AWatch-rus" + and (.release_version | type == "string" and length > 0) + and (.release_commit | type == "string" and length > 0) + and (.build_runner | type == "string" and length > 0) + and .primary_source_repository == "https://git.iri1968.dpdns.org/awatch-rus/AWatch-rus" + and .github_role == "public_mirror_only" + and (.generated_at | type == "string" and length > 0) + and (.checks | type == "array") + and (.artifacts | type == "array") + ' "$EVIDENCE_DIR/release-evidence-manifest.json" >/dev/null || fail "manifest_required_fields" + elif command -v python3 >/dev/null 2>&1; then + python3 - "$EVIDENCE_DIR/release-evidence-manifest.json" <<'PY' || fail "manifest_required_fields" +import json +import sys + +with open(sys.argv[1], "r", encoding="utf-8") as fh: + data = json.load(fh) + +required = { + "product": "AWatch-rus", + "primary_source_repository": "https://git.iri1968.dpdns.org/awatch-rus/AWatch-rus", + "github_role": "public_mirror_only", +} +for key, value in required.items(): + if data.get(key) != value: + raise SystemExit(f"{key} mismatch") +for key in ("release_version", "release_commit", "build_runner", "generated_at"): + if not isinstance(data.get(key), str) or not data[key]: + raise SystemExit(f"{key} missing") +if not isinstance(data.get("checks"), list): + raise SystemExit("checks missing") +if not isinstance(data.get("artifacts"), list): + raise SystemExit("artifacts missing") +PY + else + fail "json_validator_missing:jq_or_python3_required" + fi +fi + +if [[ ! -s "$EVIDENCE_DIR/cargo-metadata.json" ]] && ! grep -Eiq 'cargo_metadata.*skipped' "$EVIDENCE_DIR/skipped-checks.txt" 2>/dev/null; then + fail "missing_cargo_metadata_or_documented_skip" +fi + +if [[ ! -s "$EVIDENCE_DIR/cargo-tree.txt" ]] && ! grep -Eiq 'cargo_tree.*skipped' "$EVIDENCE_DIR/skipped-checks.txt" 2>/dev/null; then + fail "missing_cargo_tree_or_documented_skip" +fi + +if ! find "$EVIDENCE_DIR/artifacts" -maxdepth 1 -type f -name '*source*.tar.gz' | grep -q .; then + fail "missing_source_archive" +fi + +if ! find "$EVIDENCE_DIR/artifacts" -maxdepth 1 -type f \( -name '*binaries*.tar.gz' -o -name '*binaries*.tar.gz.skip' \) | grep -q .; then + fail "missing_binary_archive_or_documented_skip" +fi + +scan_files=( + "$EVIDENCE_DIR/RELEASE_EVIDENCE_REPORT_RU.md" + "$EVIDENCE_DIR/release-evidence-manifest.json" +) + +if grep -RInEi "(ФСТЭК|ФСБ).{0,80}(сертифицирован|сертификация|сертификат).{0,80}(есть|получен|имеется|подтвержден)|сертифицированное[[:space:]]+СЗИ" "${scan_files[@]}" >/tmp/release_evidence_forbidden_cert.$$ 2>/dev/null; then + fail "forbidden_claim_certification:$(cat /tmp/release_evidence_forbidden_cert.$$)" +fi +rm -f /tmp/release_evidence_forbidden_cert.$$ + +if grep -RInEi "(replaces|заменяет).{0,80}(SIEM|DLP)|(SIEM|DLP).{0,80}(replacement|заменяет)" "${scan_files[@]}" \ + | grep -Eiv "(does not|not |not_claimed|не |forbidden|не заявляет)" \ + >/tmp/release_evidence_forbidden_replace.$$ 2>/dev/null; then + fail "forbidden_claim_siem_dlp_replacement:$(cat /tmp/release_evidence_forbidden_replace.$$)" +fi +rm -f /tmp/release_evidence_forbidden_replace.$$ + +if ((${#failures[@]} > 0)); then + printf 'release_evidence_check=fail\n' + for failure in "${failures[@]}"; do + printf '%s\n' "$failure" + done + exit 2 +fi + +printf 'release_evidence_check=ok\n' diff --git a/scripts/registry_readiness_check.sh b/scripts/registry_readiness_check.sh index 1c6dbcc..530e67c 100644 --- a/scripts/registry_readiness_check.sh +++ b/scripts/registry_readiness_check.sh @@ -37,11 +37,17 @@ required_files=( "docs/registry/GIT_RU_MIRRORING_RUNBOOK_RU.md" "docs/registry/GITEA_BACKUP_AND_RESTORE_RUNBOOK_RU.md" "docs/registry/WIKI_AND_DOCUMENTATION_POLICY_RU.md" + "docs/registry/RU_BUILD_RUNNER_READINESS_RU.md" + "docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md" + "docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md" + "docs/registry/RELEASE_ARTIFACTS_STORAGE_RU.md" "docs/registry/RELEASE_EVIDENCE_MANIFEST_RU.md" "docs/registry/INSTALLATION_AND_TEST_INSTANCE_RU.md" "docs/registry/LIFECYCLE_AND_SUPPORT_RU.md" "docs/registry/REGISTRY_READINESS_CHANGELOG_RU.md" "docs/registry/registry-evidence-manifest.json" + "scripts/build_release_evidence.sh" + "scripts/check_release_evidence.sh" "README.md" ) @@ -68,6 +74,11 @@ if [[ -s "$MANIFEST" ]]; then and .backup.retention_days == 14 and .backup.systemd_timer == "awatch-gitea-backup.timer" and .backup.restore_tested == false + and .build_runner.status != "production_ready" + and .build_runner.status == "planned" + and .build_runner.target_hostname == "awatch-build-01" + and .build_runner.separate_from_git_server == true + and (.build_runner.required_checks | index("release_evidence_check") != null) ' "$MANIFEST" >/dev/null || fail "manifest_required_fields" elif command -v python3 >/dev/null 2>&1; then python3 -m json.tool "$MANIFEST" >/dev/null || fail "invalid_json:docs/registry/registry-evidence-manifest.json" @@ -75,8 +86,7 @@ if [[ -s "$MANIFEST" ]]; then import json import sys -path = sys.argv[1] -with open(path, "r", encoding="utf-8") as fh: +with open(sys.argv[1], "r", encoding="utf-8") as fh: data = json.load(fh) expected = { @@ -113,19 +123,32 @@ backup_expected = { for key, value in backup_expected.items(): if backup.get(key) != value: raise SystemExit(f"backup.{key} mismatch") + +build_runner = data.get("build_runner") or {} +if build_runner.get("status") == "production_ready": + raise SystemExit("build_runner.status must not be production_ready") +if build_runner.get("status") != "planned": + raise SystemExit("build_runner.status mismatch") +if build_runner.get("target_hostname") != "awatch-build-01": + raise SystemExit("build_runner.target_hostname mismatch") +if build_runner.get("separate_from_git_server") is not True: + raise SystemExit("build_runner.separate_from_git_server mismatch") +if "release_evidence_check" not in build_runner.get("required_checks", []): + raise SystemExit("build_runner.required_checks missing release_evidence_check") PY - elif command -v node >/dev/null 2>&1; then - node -e 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"))' "$MANIFEST" \ - || fail "invalid_json:docs/registry/registry-evidence-manifest.json" else - fail "json_validator_missing:python3_or_node_required" + fail "json_validator_missing:jq_or_python3_required" fi fi require_grep "git\\.iri1968\\.dpdns\\.org" "docs/registry/SOURCE_CODE_AND_BUILD_INFRASTRUCTURE_RU.md" "gitea_domain_source_infra" require_grep "git\\.iri1968\\.dpdns\\.org" "docs/registry/GIT_RU_MIRRORING_RUNBOOK_RU.md" "gitea_domain_git_runbook" require_grep "git\\.iri1968\\.dpdns\\.org" "docs/registry/WIKI_AND_DOCUMENTATION_POLICY_RU.md" "gitea_domain_wiki_policy" +require_grep "git\\.iri1968\\.dpdns\\.org" "docs/registry/RU_BUILD_RUNNER_READINESS_RU.md" "gitea_domain_build_runner" require_grep "git\\.iri1968\\.dpdns\\.org" "README.md" "gitea_domain_readme" +require_grep "awatch-build-01" "docs/registry/RU_BUILD_RUNNER_READINESS_RU.md" "awatch_build_runner_readiness" +require_grep "awatch-build-01" "docs/registry/BUILD_RUNNER_SETUP_RUNBOOK_RU.md" "awatch_build_runner_setup" +require_grep "awatch-build-01" "docs/registry/RELEASE_EVIDENCE_RUNBOOK_RU.md" "awatch_build_runner_release_runbook" require_grep "GitHub[[:space:]]*=[[:space:]]*public mirror only|public mirror only" "docs/registry/SOURCE_CODE_AND_BUILD_INFRASTRUCTURE_RU.md" "github_public_mirror_source_infra" require_grep "GitHub[[:space:]]*=[[:space:]]*public mirror only|public mirror only" "docs/registry/GIT_RU_MIRRORING_RUNBOOK_RU.md" "github_public_mirror_git_runbook" require_grep "GitHub.*публичн|public mirror" "docs/registry/WIKI_AND_DOCUMENTATION_POLICY_RU.md" "github_public_mirror_wiki_policy" @@ -136,31 +159,55 @@ require_grep "sha256|SHA256" "docs/registry/GITEA_BACKUP_AND_RESTORE_RUNBOOK_RU. require_grep "restore_tested=false|restore_tested..false|\"restore_tested\"[[:space:]]*:[[:space:]]*false" "docs/registry/GITEA_BACKUP_AND_RESTORE_RUNBOOK_RU.md" "restore_tested_false_runbook" require_grep "\"restore_tested\"[[:space:]]*:[[:space:]]*false" "docs/registry/registry-evidence-manifest.json" "restore_tested_false_manifest" require_grep "docs/registry" "docs/registry/WIKI_AND_DOCUMENTATION_POLICY_RU.md" "authoritative_docs_path_wiki_policy" +require_grep "release_evidence_check" "docs/registry/registry-evidence-manifest.json" "release_evidence_check_manifest" scan_files=( "$ROOT/README.md" "$REGISTRY_DIR"/*.md + "$REGISTRY_DIR"/*.json + "$ROOT/scripts/build_release_evidence.sh" + "$ROOT/scripts/check_release_evidence.sh" ) -if grep -RInEi "(ФСТЭК|ФСБ).{0,80}(сертифицирован|сертификация|сертификат).{0,80}(есть|получен|имеется|подтвержден)" "${scan_files[@]}" >/tmp/registry_forbidden_fstec_fsb.$$ 2>/dev/null; then +claim_scan_files=( + "$ROOT/README.md" + "$REGISTRY_DIR"/*.md + "$REGISTRY_DIR"/*.json +) + +if grep -RInEi "(password|passwd|pwd|token|secret|api[_-]?key|private[[:space:]_-]?key)[[:space:]]*[:=][[:space:]]*['\"]?[A-Za-z0-9_./+=-]{8,}" "${scan_files[@]}" >/tmp/registry_secret_like.$$ 2>/dev/null; then + fail "secret_like_value:$(cat /tmp/registry_secret_like.$$)" +fi +rm -f /tmp/registry_secret_like.$$ + +if grep -RInEi "(ФСТЭК|ФСБ).{0,80}(сертифицирован|сертификация|сертификат).{0,80}(есть|получен|имеется|подтвержден)|сертифицированное[[:space:]]+СЗИ" "${claim_scan_files[@]}" \ + | grep -Eiv "(does not claim|not_claimed|forbidden_claim|не заявляет|не фиксируется|не является|не утверждает|нельзя)" \ + >/tmp/registry_forbidden_fstec_fsb.$$ 2>/dev/null; then fail "forbidden_claim_fstec_fsb_certification:$(cat /tmp/registry_forbidden_fstec_fsb.$$)" fi rm -f /tmp/registry_forbidden_fstec_fsb.$$ -if grep -RInEi "(заменяет|replacement for|replaces).{0,80}(SIEM|DLP)|((SIEM|DLP).{0,80}(replacement|заменяет))" "${scan_files[@]}" \ - | grep -Eiv "(не |not |does not|не является|не подменяет|не заявляет|forbidden|not_made)" \ +if grep -RInEi "(заменяет|replacement for|replaces).{0,80}(SIEM|DLP)|((SIEM|DLP).{0,80}(replacement|заменяет))" "${claim_scan_files[@]}" \ + | grep -Eiv "(не |not |does not|not_claimed|не является|не подменяет|не заявляет|forbidden|not_made)" \ >/tmp/registry_forbidden_replacement.$$ 2>/dev/null; then fail "forbidden_claim_siem_dlp_replacement:$(cat /tmp/registry_forbidden_replacement.$$)" fi rm -f /tmp/registry_forbidden_replacement.$$ -if grep -RInEi "(ML/LLM-based detection|LLM-based detection|ML-based detection|automatic remediation)" "${scan_files[@]}" \ - | grep -Eiv "(forbidden|not_made|не заявляет|не фиксируется|не используется)" \ +if grep -RInEi "(ML/LLM-based detection|LLM-based detection|ML-based detection|automatic remediation)" "${claim_scan_files[@]}" \ + | grep -Eiv "(forbidden|not_made|not_claimed|не заявляет|не фиксируется|не используется|does not claim)" \ >/tmp/registry_forbidden_ai_auto.$$ 2>/dev/null; then fail "forbidden_claim_ai_or_automatic_remediation:$(cat /tmp/registry_forbidden_ai_auto.$$)" fi rm -f /tmp/registry_forbidden_ai_auto.$$ +if grep -RInEi "(юридически заверш(е|ё)нн?ая регистрация|принят.*в реестр|реестр.*заверш(е|ё)н)" "${claim_scan_files[@]}" \ + | grep -Eiv "(не |нельзя|does not|not |не утверждает|не является|не подтверждает|forbidden)" \ + >/tmp/registry_forbidden_legal_done.$$ 2>/dev/null; then + fail "forbidden_claim_legal_registry_completion:$(cat /tmp/registry_forbidden_legal_done.$$)" +fi +rm -f /tmp/registry_forbidden_legal_done.$$ + if ((${#failures[@]} > 0)); then printf 'registry_readiness_check=fail\n' for failure in "${failures[@]}"; do