Property-Based Testing Bibliography

This bibliography aims to cover the literature on property-based testing (PBT) à la Haskell QuickCheck
(inspired by Jeremy Yallop's algebraic effects bibliography & Sam Hobin-Hochstadt's gradual typing bibliography).

Pull requests welcome! Note: end a line with two spaces to force a line break.

Contents:

• Papers
• Dissertations

---

Papers

2025

• Fulminate: Testing CN Separation-Logic Specifications in C (POPL 2025)
  Rini Banerjee, Kayvan Memarian, Dhruv Makwana, Christopher Pulte, Neel Krishnaswami, Peter Sewell
  (pdf)

2024

• Generating Well-Typed Terms that are not "Useless" (POPL 2024)
  Justin Frank, Benjamin Quiring, Leonidas Lampropoulos
  (doi)

• Property-Based Testing in Practice (ICSE 2024)
  Harrison Goldstein, Joseph W. Cutler, Daniel Dickstein, Benjamin C. Pierce, Andrew Head
  (doi)

• How We Built Cedar: A Verification-Guided Approach (FSE 2024)
  Craig Disselkoen, Aaron Eline, Shaobo He, Kyle Headley, Mike Hicks, Kesha Hietala, John Kastner, Anwar Mamat, Matt McCutchen, Neha Rungta, Bhakti Shah, Emina Torlak, Andrew Wells
  (doi)

• Type-Level Property Based Testing (TyDe 2024)
  Thomas Ekström Hansen, Edwin Brady
  (doi)

• Dependent Types to Push Corners of the Property-based Testing (TyDe 2024)
  Denis Buzdalov
  (link)

• Mica: Automated Differential Testing for OCaml Modules (OCaml 2024)
  Ernest Ng, Harrison Goldstein, Benjamin C. Pierce
  (arXiv)

• Tyche: Making Sense of PBT Effectiveness (UIST 2024)
  Harrison Goldstein, Jeffrey Tao, Zac Hatfield-Dodds, Benjamin C. Pierce, Andrew Head
  (doi)

• General and Practical Property-based Testing for Android Apps (ASE 2024)
  Yiheng Xiong, Ting Su, Jue Wang, Jingling Sun, Geguang Pu, Zhendong Su
  (doi)

• Evaluating PBT Frameworks in OCaml (PLDI SRC 2024)
  Nikhil Kamath
  (pdf)

• Constrained generation of well-typed program
  Gabriel Scherer
  (doi)

• Can Large Language Models Write Good Property-Based Tests?
  Vasudev Vikram, Caroline Lemieux, Joshua Sunshine, Rohan Padhye
  (arXiv)

2023

• Covering All the Bases: Type-Based Verification of Test Input Generators (PLDI 2023)
  Zhe Zhou, Ashish Mishra, Benjamin Delaware, Suresh Jagannathan
  (doi)

• Merging Inductive Relations (PLDI 2023)
  Jacob Prinz, Leonidas Lampropoulos
  (doi)

• Reflecting on Random Generation (ICFP 2023)
  Harrison Goldstein, Samantha Frohlich, Meng Wang, Benjamin C. Pierce
  (doi)

• Etna: An Evaluation Platform for Property-Based Testing (ICFP 2023)
  Jessica Shi, Alperen Keles, Harrison Goldstein, Benjamin C. Pierce, Leonidas Lampropoulos
  (doi)

• Formal Specification and Testing for Reinforcement Learning (ICFP 2023)
  Mahsa Varshosaz, Mohsen Ghaffari, Einar Broch Johnsen, Andrzej Wąsowski
  (doi)

• QuickerCheck: Implementing and Evaluating a Parallel Run-Time for QuickCheck (IFL 2023)
  Robert Krook, Nicholas Smallbone, Bo Joel Svensson, Koen Claessen
  (doi)

• Don’t Go Down the Rabbit Hole: Reprioritizing Enumeration for Property-Based Testing (Haskell 2023)
  Segev Elazar Mittelman, Alvin Resnick, Ivan Perez, Alwyn Goodloe, Leonidas Lampropoulos
  (doi)

• falsify: Internal Shrinking Reimagined for Haskell (Haskell 2023)
  Edsko de Vries
  (doi)

• How Developers Implement Property-Based Tests (ICSME 2023)
  Arthur Lisboa Corgozinho,Marco Tulio Valente, Henrique Rocha
  (doi)

• Evaluating Soundness of a Gradual Verifier with Property Based Testing (POPL 2023 SRC)
  Jan-Paul Ramos-Dávila
  (pdf)

2022

• Logarithm and Program Testing (POPL 2022)
  Kuen-Bang Hou (Favonia), Zhuyang Wang
  (doi)

• Quickstrom: property-based acceptance testing with LTL specifications (PLDI 2022)
  Liam O'Connor, Oskar Wickström
  (doi)

• Computing correctly with inductive relations (PLDI 2022)
  Zoe Paraskevopoulou, Aaron Eline, Leonidas Lampropoulos
  (doi)

• A Completely Unique Account of Enumeration (ICFP 2022)
  Cas van der Rest, Wouter Swierstra
  (doi)

• Random Testing of a Higher-Order Blockchain Language (ICFP 2022)
  Tram Hoang, Anton Trunov, Leonidas Lampropoulos, Ilya Sergey
  (doi)

• Parsing Randomness (OOPSLA 2022)
  Harrison Goldstein, Benjamin C. Pierce
  (doi)

• Some Problems with Properties: A Study on Property-Based Testing in Industry (HATRA 2022)
  Harrison Goldstein, Joseph W. Cutler, Adam Stein, Andrew Head, Benjamin C. Pierce
  (pdf)

• Automated, Targeted Testing of Property-Based Testing Predicates (The Art, Science, and Engineering of Programming, 2022)
  Tim Nelson, Elijah Rivera, Sam Soucie, Thomas Del Vecchio, Jack Wrenn, Shriram Krishnamurthi
  (pdf)

• Functional Pearl: Holey Generators!
  Joseph W. Cutler, Harrison Goldstein, Koen Claessen, John Hughes, Benjamin C. Pierce
  (pdf)

• Learning Good Generators for Property-Based Testing (PLDI 2022 SRC)
  Joseph W. Cutler
  (pdf)

2021

• Do Judge a Test by its Cover: Combining Combinatorial and Property-Based Testing (ESOP 2021)
  Harrison Goldstein, John Hughes, Leonidas Lampropoulos, Benjamin C. Pierce
  (doi)

• Data-Driven Abductive Inference of Library Specifications (OOPSLA 2021)
  Zhe Zhou, Robert Dickerson, Benjamin Delaware, Suresh Jagannathan
  (doi)

• Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3 (SOSP 2021)
  James Bornholt, Rajeev Joshi, Vytautas Astrauskas, Brendan Cully, Bernhard Kragl, Seth Markle, Kyle Sauri, Drew Schleit, Grant Slatton, Serdar Tasiran, Jacob Van Geffen, Andrew Warfield
  (doi)

• Resolvable Ambiguity: Principled Resolution of Syntactically Ambiguous Programs (CC 2021)
  Viktor Palmkvist, Elias Castegren, Philipp Haller, David Broman
  (doi)

• Strong automated testing of OCaml libraries (JFLA 2021)
  François Pottier
  (pdf)

• Parafuzz: Coverage-guided Property Fuzzing for Multicore OCaml programs (OCaml 2021)
  Sumit Padhiyar, Adharsh Kamath, KC Sivaramakrishnan
  (url)

• Property-Based Testing for OCaml through Coq (OCaml 2021)
  Paaras Bhandari, Leonidas Lampropoulos
  (pdf)

• Using Relational Problems to Teach Property-Based Testing
  (The Art, Science, and Engineering of Programming, 2021)
  John Wrenna, Tim Nelsona, Shriram Krishnamurthi
  (doi)

2020

• Test-Case Reduction via Test-Case Generation: Insights from the Hypothesis Reducer (ECOOP 2020)
  David R. MacIver, Alastair F. Donaldson
  (doi)

• Quickly Generating Diverse Valid Test Inputs with Reinforcement Learning (ICSE 2020)
  Sameer Reddy, Caroline Lemieux, Rohan Padhye, Koushik Sen
  (doi)

• A Simple State-Machine Framework for Property-Based Testing in OCaml (OCaml 2020)
  Jan Midtgaard
  (pdf)

2019

• Coverage guided, property based testing (OOPSLA 2019)
  Leonidas Lampropoulos, Michael Hicks, Benjamin C. Pierce
  (doi)

• Deriving compositional random generators (IFL 2019)
  Agustín Mista, Alejandro Russo
  (doi)

• Semantic Fuzzing with Zest (ISSTA 2019)
  Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, Yves Le Traon
  (doi)

2018

• Generating Good Generators for Inductive Relations (POPL 2018)
  Leonidas Lampropoulos, Zoe Paraskevopoulou, Benjamin C. Pierce
  (doi)

• Keep your laziness in check (ICFP 2018)
  Kenny Foner, Hengchu Zhang, Leonidas Lampropulos
  (doi)

• Branching processes for QuickCheck generators (Haskell 2018)
  Agustín Mista, Alejandro Russo, John Hughes
  (doi)

• AutoBench: Comparing the Time Performance of Haskell Programs (Haskell 2018)
  Martin A. T. Handley, Graham Hutton
  (doi)

• Cheap Remarks about Concurrent Programs (FLOPS 2018)
  Michael Walker, Colin Runciman
  (doi)

2017

• Beginner's luck: a language for property-based generators (POPL 2017)
  Leonidas Lampropoulos, Diane Gallois-Wong, Cătălin Hritcu, John Hughes, Benjamin C. Pierce, Li-yao Xia
  (pdf)

• Effect-driven QuickChecking of compilers (ICFP 2017)
  Jan Midtgaard, Mathias Nygaard Justesen, Patrick Kasting, Flemming Nielson, Hanne Riis Nielson
  (doi)

• Ode on a Random Urn (Functional Pearl) (Haskell 2017)
  Leonidas Lampropoulos, Antal Spector-Zabusky, Kenny Foner
  (doi)

• Speculate: discovering conditional equations and inequalities about black-box functions by reasoning from test results (Haskell 2017)
  Rudy Braquehais, Colin Runciman
  (doi)

• Extrapolate: generalizing counterexamples of functional test properties (IFL 2017)
  Rudy Braquehais, Colin Runciman
  (doi)

• QuickChecking Patricia Trees (TFP 2017)
  Jan Midtgaard
  (pdf)

• Quick Specifications for the Busy Programmer (JFP 2017)
  Nicholas Smallbone, Moa Johansson, Koen Claessen, Maximilian Algehed
  (doi)

• Failing faster: overlapping patterns for property-based testing (PADL 2017)
  Jonathan Fowler, Graham Hutton
  (doi)

• ChimpCheck: Property-Based Randomized Test Generation for Interactive Apps (Onward! 2017) Edmund S. L. Lam, Peilun Zhang, Bor-Yuh Evan Chang
  (doi)

• Testing with Crowbar (OCaml 2017)
  Stephen Dolan, Mindy Preston
  (url)

• A simple incremental development of a property-based testing tool (Functional Pearl)
  Rudy Braquehais, Michael Walker, José Manuel Calderón Trilla, Colin Runciman
  (pdf)

2016

• FitSpec: refining property sets for functional testing (Haskell 2016)
  Rudy Braquehais, Colin Runciman
  (doi)

• QuickFuzz: an automatic random fuzzer for common file formats (Haskell 2016)
  Gustavo Grieco, Martín Ceresa, Pablo Buiras
  (doi)

• Mysteries of DropBox: Property-Based Testing of a Distributed Synchronization Service (ICST 2016)
  John Hughes, Benjamin C. Pierce, Thomas Arts, Ulf Norell
  (doi)

• Testing Noninterference, Quickly (JFP 2016)
  Catalin Hritcu, Leonidas Lampropoulos, Antal Spector-Zabusky, Arthur Azevedo Amorim, Maxime Denes, John Hughes, Benjamin C. Pierce, Dimitrios Vytiniotis
  (doi)

• Experiences with QuickCheck: Testing the Hard Stuff and Staying Sane
  John Hughes
  (doi)

2015

• Making Random Judgments: Automatically Generating Well-Typed Terms from the Definition of a Type-System (ESOP 2015)
  Burke Fetscher, Koen Claessen, Michał Pałka, John Hughes, Robert Bruce Findler
  (doi)

• Type Targeted Testing (ESOP 2015)
  Eric L. Seidel, Niki Vazou, Ranjit Jhala
  (doi)

• Generating constrained random data with uniform distribution (JFP 2015)
  Koen Claessen, Jonas Duregård, Michał Pałka
  (doi

• Foundational Property-Based Testing (ITP 2015)
  Zoe Paraskevopoulou, Cătălin Hritcu, Maxime Dénès, Leonidas Lampropoulos, Benjamin C. Pierce
  (doi)

• Déjà Fu: a concurrency testing library for Haskell (Haskell 2015)
  Michael Walker, Colin Runciman
  (doi)

• Quickchecking Static Analysis Properties (ICST 2015)
  Jan Midtgaard, Anders Møller
  (pdf)

• Towards a Theory of Reach (TFP 2015)
  Jonathan Fowler, Graham Hutton
  (doi)

• Testing AUTOSAR software with QuickCheck (ICSTW 2015)
  Thomas Arts, John Hughes, Ulf Norell, Hans Svensson
  (doi)

2014

• SmartCheck: automatic and efficient counterexample reduction and generalization (Haskell 2014)
  Lee Pike
  (doi)

2013

• Automating inductive proofs using theory exploration (CADE 2013)
  Koen Claessen, Moa Johansson, Dan Rosén, Nicholas Smallbone
  (doi)

• Testing Blocking Operations with QuickCheck’s Component Library (Erlang 2013)
  Ulf Norell, Hans Svensson, Thomas Arts
  (doi)

2012

• Shrinking and showing functions (Functional Pearl) (Haskell 2012)
  Koen Claessen
  (doi)

• Feat: functional enumeration of algebraic types (Haskell 2012)
  Jonas Duregård, Patrik Jansson, Meng Wang
  (doi)

• Testing type class laws (Haskell 2012)
  Johan Jeuring, Patrik Jansson, Cláudio Amaral
  (doi)

• Advances in Lazy SmallCheck (IFL 2012)
  Jason S. Reich, Matthew Naylor, Colin Runciman
  (doi)

2011

• Testing an Optimising Compiler by Generating Random Lambda Terms (AST 2011)
  Michał H. Pałka, Koen Claessen, Alejandro Russo, John Hughes
  (doi)

2010

• Testing Polymorphic Properties (ESOP 2010)
  Jean-Philippe Bernardy, Patrik Jansson, Koen Claessen
  (doi)

• QuickSpec: Guessing Formal Specifications Using Testing (TAP 2010)
  Koen Claessen, Nicholas Smallbone, John Hughes
  (doi)

• Using Temporal Relations to Specify and Test an Instant Messaging Server (AST 2010) John Hughes, Ulf Norrell, Jérôme Sautret
  (doi)

2008

• SmallCheck and Lazy SmallCheck: automatic exhaustive testing for small values (Haskell 2008)
  Colin Runciman, Matthew Naylor, Fredrik Lindblad
  (doi)

2006

• Testing Properties of Generic Functions (IFL 2006)
  Patrik Jansson, Johan Jeuring, Laurence Cabenda, Gerbo Engels, Jacob Kleerekoper, Sander Mak, Michiel Overeem, Kees Visser
  (doi)

2002

• Testing Monadic Code with QuickCheck (Haskell 2002)
  Koen Claessen, John Hughes
  (doi)

2000

• QuickCheck: a lightweight tool for random testing of Haskell programs (ICFP 2000)
  Koen Claessen, John Hughes
  (doi)

---

Dissertations

2024

• Property-Based Testing for the People (PhD Dissertation, University of Pennsylvania)
  Harrison Goldstein
  (pdf)

2022

• Testing by Dualization (PhD Dissertation, University of Pennsylvania)
  Yishuai Li
  (pdf)

2020

• Efficiency Three Ways: Tested, Verified, and Formalised (PhD Dissertation, University of Nottingham)
  Martin Handley
  (pdf)

2019

• Narrowing in on Property-Based Testing (PhD Dissertation, University of Nottingham)
  Jonathan Fowler
  (pdf)

2018

• Random Testing for Language Design (PhD Dissertation, University of Pennsylvania)
  Leonidas Lampropoulos
  (pdf)

2017

• Tools for Discovery, Refinement and Generalization of Functional Properties by Enumerative Testing (PhD Dissertation, University of York)
  Rudy Braquehais
  (pdf)

2013

• Lightweight Verification of Functional Programs (PhD Dissertation, Chalmers University of Technology)
  Nicholas Smallbone
  (pdf)