Add new cop RSpec/LeakyLocalVariable
#2101
Merged
+550
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Here's the cop report for real-world-rspec. I wrote a custom formatter which prints the URL to the file and line on the remote, so you can go through it and see offenses directly on GH. There are 866 offenses for all codebases in that repo. Report |
lovro-bikic
commented
Aug 3, 2025
| next unless part_of_example_scope?(reference) | ||
| next if permitted_argument?(reference) | ||
|
|
||
| add_offense(assignment.node) |
There was a problem hiding this comment.
Initially I added the offense to the reference node, but that created offense noise (if an offending variable is referenced many times, there will be many offenses). An offense on the assignment node reduces that noise, but it may not be clear which reference triggered the offense. Let me know what you think is the better option.
Here's an example:
user = create(:user)
# ^^^^^^^^^^^^^^^^^^^^ assignment offense (only one)
before do
user.update(admin: true)
# ^^^^ reference offense
user.flag!
# ^^^^ reference offense (repeated for all references)
end
There was a problem hiding this comment.
I agree that the offence should be where the var is defined, as this is the place that needs correction, not usages of the var.
lovro-bikic
force-pushed
the
leaky-local-variable-cop
branch
2 times, most recently
from
1c78177 to
c791895
Compare
|
The problem is not the variable, but a call to factory outside of the example, made outside of the example lifecycle. Variables come handy at times, because you can’t reasonably use constants (see LeakyConstant). Would you agree to re-target the cop to catch those out-of-scope factory calls? |
I agree that that's a problem, but I don't think it's the problem. A local variable that's not a call to a factory can still be equally problematic, e.g.: foo = 'string'
it 'does something' do
foo << 'modification'
expect(foo).to eq('stringmodification')
end
it 'does something else' do
expect(foo).to eq('string') # this example can fail depending on the example order
endI would agree that some examples in there are harmless, but a lot of them are unnecessarily made local variables. To give a few examples:
Most of the examples in the report follow this pattern. Also, despite being harmless, I think that |
I've reconsidered my past opinion, as the risk of make specs non-deterministic is not justifiable by making specs more concise or marginally more performant (e.g. reading some JSON config). Please let me know if you're still up to continue with this cop. As a side note, there are other known ways to introduce non-determinism similarly. |
|
Great, thank you! I will rebase the branch and also resolve offenses on the Rails monolith I'm working on (300+ offenses have been caught there for this cop) to see if there are any false positives or areas to improve the cop. I'll let you know once it's done so you can continue with the review. |
|
I ran this cop on my monolith (~1500 spec files) and found 9 offenses. None were false positives. One thing I noticed however, is that the cop recommends that I use a |
lovro-bikic
force-pushed
the
leaky-local-variable-cop
branch
4 times, most recently
from
45a65e4 to
6eeea96
Compare
|
I rebased the branch and I think it's ready for review now. I went over the offenses in my monolith and found one false positive with factory bot, for which I added a check (
Hmm yeah, I think the offense message was too prescriptive, but the solution really depends on the context. I updated it to something more generic, let me know how it looks. |
bquorning
approved these changes
Sep 18, 2025
lovro-bikic
force-pushed
the
leaky-local-variable-cop
branch
2 times, most recently
from
cac4294 to
77a7e91
Compare
pirj
reviewed
Sep 20, 2025
pirj
reviewed
Sep 20, 2025
|
|
||
| def check_references(variable) | ||
| variable.assignments.each do |assignment| | ||
| next if part_of_example_scope?(assignment.node) |
There was a problem hiding this comment.
Cosmetic, optional. Can those be computed once per scope? Is this a worthwhile optimization? It feels like we’re iterating over vars defined in the same scope, right?
There was a problem hiding this comment.
Hmm I tried optimizing it like that, but some examples fail then, like this one:
describe SomeClass do
user = create(:user)
it 'updates the user' do
user = create(:user)
expect { user.update(admin: true) }.to change(user, :updated_at)
end
endwhen the scope of describe is checked. This one fails because part_of_example_scope? returns false for that scope (expected, since describe block doesn't count as example scope) but then it iterates over every assignment inside of describe block, including the user = create(:user) one inside it. Because the next if part_of_example_scope? line was removed, the offense is then added for references on expect line.
In summary, it cannot be computed once per scope because after_leaving_scope may be a non-example scope, but it might iterate over assignments from example scopes if they're nested within it.
pirj
reviewed
Sep 20, 2025
pirj
reviewed
Sep 20, 2025
lovro-bikic
force-pushed
the
leaky-local-variable-cop
branch
from
77a7e91 to
ddaddea
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new cop to register offenses for local variables assigned outside of examples, but referenced within them. In such cases, local variables are shared state, and they can make tests non-deterministic if they run in random order.
The cop will register offenses for:
The cop does not register offenses when local variable is:
I think this closes #1648. There are some other ideas in there, but I believe this cop solves the major pain point (which is shared state between examples through local variables).
Before submitting the PR make sure the following are checked:
config/default.yml.Enabled: pendinginconfig/default.yml.Enabled: truein.rubocop.yml.VersionAdded: "<<next>>"indefault/config.yml.