Descriptions:
Cloud ETL failures often require engineers to manually inspect logs, diagnose schema or data-quality issues, select a repair, rerun the job, and validate recovery. This talk presents an RL-guided pipeline health agent that automates this workflow through deterministic anomaly detection, interpretable Q-learning, bounded remediation actions, and an external safety layer.
The system detects schema drift, null-rate spikes, type changes, and runtime failures, then selects actions such as retry, schema coercion, rollback, quarantine, or escalation. Evaluation across 30 controlled synthetic runs demonstrates minutes-scale recovery for successfully resolved cases while highlighting the importance of deterministic rules and safety guardrails.
Speakers:
– Anna Marie Benzon: Anna Marie Benzon is a World Economic Forum–recognized technology leader, startup founder, and PhD researcher in AI with 9+ years of experience building AI-powered products and scaling multidisciplinary teams.
LinkedIn: https://www.linkedin.com/in/anna-marie-benzon
GitHub: https://github.com/ambenzon27
