Skip to content

search_graph_fuzz add unit test support and unit tests #163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lcnr opened this issue Feb 12, 2025 · 2 comments
Open

search_graph_fuzz add unit test support and unit tests #163

lcnr opened this issue Feb 12, 2025 · 2 comments
Labels
E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added.

Comments

@lcnr
Copy link
Contributor

lcnr commented Feb 12, 2025

https://github.com/lcnr/search_graph_fuzz/ currently doesn't have any stable representation for solve trees, so each change needs to be validated via fuzzing. Doing so in CI feels hard and not really desirable.

I would love for somebody to introduce an additional representation for solve trees and then writes unit tests using that representation. I am available to mentor this work

@lcnr lcnr added the E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added. label Feb 12, 2025
@norablackcat
Copy link

What exactly is a stable representation and how does that avoid fuzzing when a change happens?

@lcnr
Copy link
Contributor Author

lcnr commented Feb 18, 2025

right now I am simply generating random graphs and then check whether we break any invariants https://github.com/lcnr/search_graph_fuzz/blob/f267a260769b4094c4fcdf00b2b2eb52124312bf/src/global_cache.rs#L196-L250

We've got minimal (and kind of outdated) test cases for the search graph in https://gist.github.com/lcnr/67ded915a8ce43cf6faa7e97eb59e8de and https://gist.github.com/lcnr/291c3a7e1491ea33c1333c83cb594ca0. I don't think there's a proper explanation for how to read these proof trees 😅 so that probably also needs an explanation in sync

We need different graphs between the global_cache fuzzer (which mostly checks that the global cache is not observable) and the provisional_cache fuzzer (which checks that the result does not change when using the provisional cache).

The worst way (but easiest) way would be to literally write out the constructors for the graphs with the current data structure for each of the tested pattern. I think it would be useful to have a text representation and then write a small parser for it

It would be easiest to chat in sync to explain exactly what I want. Alternatively, please move this to zulip in case you're interested https://rust-lang.zulipchat.com/#narrow/channel/364551-t-types.2Ftrait-system-refactor/topic/needs_help.3A.20writing.20unit.20tests.20for.20.60search_graph_fuzz.60

There's also an unfinished rustc-dev-guide chapter for it https://hackmd.io/1ALkSjKlSCyQG-dVb_PUHw which will also be partially outdated once we land rust-lang/rust#136824

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added.
Projects
None yet
Development

No branches or pull requests

2 participants