DataCraft
A data processing framework written from scratch in Rust, with a drag-and-drop pipeline editor and an SDK for sources, transforms and sinks. It replaced Spark for real-time log parsing across 17 application groups.
- MyJio pilot: 480 cores and 480 GB down to 10 cores and 4.2 GB peak, latency 10 s to 2.3 s.
- Fleet: 8.7 TB of executor RAM down to under 200 GB; 3,500 cores down to about 200.
- dc-sdk published through the organisation's first internal Cargo registry.