Testing Incomplete Programs in OCaml
This program is tentative and subject to change.
Programs under construction routinely use exceptions to signal yet-to-be-implemented features. Yet, testing frameworks do not distinguish incompleteness from actual failure, thereby obscuring reports with spurious errors that can also hide real errors. We advocate that testing frameworks should not conflate errors due to incompleteness with either success or failure, and instead should gracefully handle incompleteness, exploiting the potential for informative feedback. Incremental testing is a simple idea, easy to implement, and provides actionable benefits for programmers to adopt testing strategies right from the start of their projects. In this presentation, we showcase the problem of testing incomplete programs with two popular OCaml testing frameworks: Alcotest, for unit testing, and QCheck, for property-based testing. We explain how the theory of incremental reasoning about incomplete programs proposed by Diaz et al. in the context of certified programming in a proof assistant can inform the design of incremental (unit and property-based) testing frameworks. We briefly describe the small modifications required to make both Alcotest and QCheck incremental. We illustrate the modified frameworks at work on a simple substitution-based lambda calculus interpreter, and report on a larger-scale evaluation of incremental QCheck with the multicoretests suite for the OCaml 5 compiler.
This program is tentative and subject to change.
Fri 28 AugDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:30 30mTalk | Testing Incomplete Programs in OCaml ML Family | ||
12:00 30mTalk | Self-Tagging Numbers ML Family John Reppy University of Chicago, Ben Wakefield University of Chicago, Byron Zhong University of Chicago | ||