RunbookFX: Type- and Effect-Safe LLM Synthesis for Executable Incident Diagnosis and MitigationRemote
This program is tentative and subject to change.
Large language models are increasingly deployed as autonomous agents for cloud incident response, yet their direct use admits hallucinated diagnoses, unauthorized actions, irreversible changes, and unauditable decision trails. We present RunbookFX, a typed functional domain-specific language that elevates incident response from natural-language suggestions to executable programs whose safety is established statically. The key insight is that incident-response safety decomposes into three interacting dimensions: risk severity, exercised capabilities, and rollback resource availability. RunbookFX formalizes this decomposition as a product effect algebra $\mathrm{Risk} \times \mathcal{K} \times \mathbb{N}$ whose four cross-component interaction axioms yield domain-specific safety theorems unexpressible in flat effect frameworks; a strong handler parametricity result then transfers these guarantees from a replay handler to any bisimilar live handler, bridging offline verification and production deployment. An LLM proposes candidate programs that a CEGIS-style verifier filters by static type checking and dynamic contract replay. A ${\sim}$2,200-line Coq development discharges the product effect algebra, its composition-preservation property, and four core safety theorems: Effect WF Preservation, Progress, single-step No Unauthorized Action, and Rollback Linearity. Of the 27 supporting obligations in the substitution and multi-step layers, 18 now close with \texttt{Qed}—including all Canonical Forms, all effect-operation Inversion lemmas, Value Typing, de~Bruijn weakening, and the typing-respecting reduction cases for \texttt{observe}, \texttt{act}, \texttt{rollback}, and the affirmative guard; the remaining nine trace back to the de~Bruijn substitution lemma, whose proof skeleton follows Pierce et al. [2019]. Evaluated on RCAEval for root cause analysis and ITBench for end-to-end mitigation, RunbookFX achieves 64% Top-1 RCA accuracy against 53% for the best LLM baseline and 38% mitigation success at $3.3\times$ the official ITBench agent, with zero safety violations and 100% rollback coverage by construction.
This program is tentative and subject to change.
Thu 27 AugDisplayed time zone: Eastern Time (US & Canada) change
10:30 - 12:00 | |||
10:30 18mTalk | RunbookFX: Type- and Effect-Safe LLM Synthesis for Executable Incident Diagnosis and MitigationRemote ICFP Papers Yifan Xiao Peking University, Shijie Li China Southern Power Grid Company Limited, Yuhao Ge China Southern Power Grid Company Limited DOI | ||
10:48 18mTalk | Machine-Generated, Machine-Checked Proofs for a Verified Compiler (Experience Report) ICFP Papers Zoe Paraskevopoulou National Technical University of Athens DOI | ||
11:06 18mTalk | Proofs Promptly: Proof-Oriented Programming with AI Agents (Experience Report) ICFP Papers Eleftherios Ioannidis Microsoft Research, Nikhil Swamy Microsoft Research, Gabriel Ebner Microsoft Research, Matthai Philipose Microsoft Research, Tahina Ramananandro Microsoft Research DOI | ||
11:24 18mTalk | Programming Backpropagation with Reverse Handlers for Arrows ICFP Papers Takahiro Sanada Fukui Prefectural University, Keisuke Hoshino Research Institute for Mathematical Sciences, Kyoto University, Kenshin Hirai Research Institute for Mathematical Sciences, Kyoto University, Shin-ya Katsumata Kyoto Sangyo University DOI | ||
11:42 18mTalk | On Recursion in Graded Modal Type Theory ICFP Papers Oskar Eriksson Department of Computer Science and Engineering, University of Gothenburg and Chalmers University of Technology, Gothenburg, Sweden, Andreas Abel Gothenburg University, Nils Anders Danielsson University of Gothenburg DOI | ||