Hayabusa
Local-first security detection platform processing Windows Event Logs through a streaming pipeline into ClickHouse-powered detections.
The Problem
Security teams need pipelines they can inspect and control locally, especially when event volume, network assumptions, or deployment constraints make hosted-first approaches a bad fit.
The challenge is not just collecting Windows events, but moving them through a pipeline that remains debuggable while still supporting query performance and iterative detections.
$ inspect hayabusa --focus architecture
Evaluating workflow and runtime constraints...
Tracing data flow boundaries... [OK]
Reviewing operator ergonomics... [OK]
System state: HEALTHY_OPERATIONAL
The Solution
Hayabusa keeps collection, streaming transport, storage, and detections close to the operator so the full system can be reasoned about in real operational conditions.
The pipeline is designed to make event movement observable, preserve replay-friendly storage in ClickHouse, and support SQL-driven detections without forcing a heavyweight control plane.
Architecture
Windows endpoint collection and normalization pipeline with explicit flow into streaming transport.
Vector and NATS provide movement and buffering across the pipeline before events land in ClickHouse.
Detections are designed around queryable event history and operator visibility rather than hidden rule execution.
Event Pipeline
Collection, normalization, transport, and storage are intentionally separated so each stage can be observed and tuned without losing the full picture.
Detection Layer
ClickHouse-backed history and SQL-driven detections make it easier to iterate on rules while keeping data access fast and operator-readable.
Architecture Notes
Event Pipeline
Windows Event Logs
↓
PowerShell Exporter
↓
Vector
↓
NATS
↓
Go Ingestor
↓
ClickHouse
Data Collection
Detection Engine
ClickHouse
↓
SQL Rules
↓
Detections
↓
Alerts
Rule Processing
Design Goals
LOCAL FIRST
OBSERVABLE
AUDITABLE
NO CLOUD REQUIRED
Architecture Principles
Screenshots / media
Technical Notes
- Pipeline stages are kept explicit so transport failures, storage pressure, and normalization problems can be debugged independently.
- ClickHouse is used as a practical analytical store because detection work benefits from fast iteration on historical event queries.
- Local-first constraints shape deployment and ergonomics, but they also improve operator trust in the system.
Lessons Learned
- Detection quality depends on operational clarity as much as rule logic.
- Windows-heavy data pipelines need practical tooling around normalization and replay, not just collection throughput.
- The most useful security platforms surface their own behavior clearly to the operator.
Future Work
- Expand detection coverage and alert routing ergonomics.
- Improve replay and triage workflows for investigations.
- Add richer operator dashboards for pipeline health and rule feedback.