where tests nested inside check or where doesn't execute

[JakeWheat]

check:
  fun fcheck() block:
    1 is 1
    print("1 is 1?")
  where:
    2 is 2
    print("2 is 2?")
  end
  fcheck()
end

fun ftop(a):
  1 == 2
where:
  fun fnestedwhere():
    1
  where:
    3 is 3
    print("3 is 3?")
  end
  4 is 4
  print("4 is 4?")
  fnestedwhere() is 1
end

ftop(1)

output is:

false

1 is 1?

4 is 4?

And the test output confirms that the "1 is 1" and "4 is 4" tests run, but the "2 is 2" and "3 is 3" tests - which are in where blocks that are nested inside another check or where block - don't.

[sorawee]

Are we taking "Who Tests the Testers?" a bit too seriously? :)

[JakeWheat]

Actually, I was thinking maybe triple nested tests should be checked too?

[blerner]

This is more complicated than it might first appear. If you tweaked your example slightly, to

fun ftop(a) block:
  fun fnestedwhere() block:
    print("in nestedwhere where")
    1
  where:
    3 is "3"
    print("in nestedwhere where")
  end
  print("in ftop main")
  1 == 2
where:
  4 is "4"
  print("in ftop where")
end
ftop(1)

it will print

in ftop main
in nestedwhere where

false

in fcheck main
in ftop where

...even though you never call fnestedwhere explicitly.

Apparently, we do not desugar where blocks on functions nestled inside where or check blocks correctly. (This actually violates an invariant we claim inside our compiler, so I'm surprised we haven't run into this problem before...)

Also note that there is a different form of confusion over which test blocks should run, when you're dealing with check or where or examples blocks defined in multiple files: in those situations, we only run the test cases in the primary file being run. This is by design, but is an orthogonal concern to the bug presented here.