★ FEATURED in progress
Stock-KraKeN
Homelab data pipeline — KraKeN network lab meets market data
A self-hosted pipeline that pulls market data through the KraKeN lab, stores it on a RHEL box and surfaces it in a small operator console. Currently in active development.
network linux automation data security
problem
- ›Market data sources are scattered and their formats never agree.
- ›I want a reason to run real services on my lab instead of toy examples.
- ›Pulling public data over a clean network while keeping the lab isolated is a fun security exercise.
architecture
- ›KraKeN network lab terminates VLANs and provides the network backbone.
- ›A RHEL 9 host runs the collector, the database and a small web console.
- ›The collector normalizes every feed into one schema before storage.
technologies
- ›RHEL 9
- ›Cisco L3 switching
- ›Bash
- ›Cron
- ›SQLite
security
- ›Collector traffic restricted to an egress-only VLAN.
- ›Console served over HTTPS, no credentials on the LAN.
- ›Firewalld zones keep the database off the guest network.
what i built
- +VLAN segmentation plan for the data path.
- +Collector script with backoff and retry logic.
- +Scheduled ingestion via cron with flock locking.
- +Minimal web console rendering normalized rows.
what i learned
- +Scheduling reliable jobs is harder than writing the script.
- +Normalizing messy upstream formats takes 80% of the effort.
- +Segmenting traffic by VLAN makes troubleshooting obvious.
current status
Skeleton in place: network is up, collector ingests a single source, console renders raw rows. Next: a second data source and basic alerting.