-
Notifications
You must be signed in to change notification settings - Fork 14
chore: split deps and test tasks in Taskfile.yml #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Hi @aumrp77! Thanks for this contribution! Can you please verify that the code in this PR still produces the same results as the original paper? For sanity checking correctness, we can simply run the new task run workflow on a small set of repos (like just Compfiles and MIL) and compare key metrics/outputs against those in the paper. Moreover, to check that the entire workflow works, we can use a separate blank repo. You can quickly do these by following the |
Hi @aumrp77 , my apologies for the delay in getting back to you after our 1:1 discussion. As @Adarsh321123 mentioned, are you able to run some sanity checks to reproduce the approximate numbers from the paper for 1 or 2 repos? This would help give us confidence that your code changes don't break anything. (He updated the comment with more instructions, which may not have sent you a new notification). I spoke with @Adarsh321123 and it seems the only way to do this would be to have access to GPUs and run the experiments for some time. @Adarsh321123 can you remind me of our discussion, and how we were discussing testing the code changes in the fastest way possible, to ensure the new changes don't break anything? I believe we talked about caching the static data, and then we should be able to run on a single repo in an hour or two. Could you remind me of the details and plan on how to do that? |
Hi @aumrp77, thank you for the contribution. I wanted to clarify the motivation behind this PR. Is the goal to enable running pytest without having to run the full setup task? If so, that should already be possible. Assuming a fresh environment, you can simply run:
If you're looking to combine these into a single composite task, you might consider a pattern similar to the existing setup task. One additional note on collaboration: when building on an existing PR, the typical practice is to pull the branch and branch off of it. This helps make the proposed changes more transparent by preserving the original work. From what I can tell, this PR seems to start from scratch and doesn't incorporate the existing taskfile, which will unintentionally discard prior work. |
This PR separates the “deps” and “test” steps in our Taskfile, so that installing dependencies and running tests are two independent tasks:
This makes CI faster (tests aren’t run during deps) and ensures Lean is on PATH when pytest spins up.
Verification