Improve .gitignore iteration speed

[silversquirl]

Using os.walk instead of Path().rglob can be 10x faster in deeply nested directory structures. This is a significant saving on certain projects, where just this iteration can take up to 3 or 4 seconds.

An even better approach would be to parse gitignore files lazily, as the directory tree is traversed later in this function. Another optimization would be to prune ignored directories while walking the directory tree, so they aren't even explored.

However, just the single optimization in this PR already reduces the impact of this code significantly, bringing it from almost 3s in my profile to just 0.4s.

[henryiii]

Also see #1048, which will help a lot (it's the second optimization you mentioned), though it's not backward compatible so it's planned for the next version.

I'll fix the failure with 3.13.4 (done in #1104), that version of Python is broken on Windows for building (on all backends).

[silversquirl]

Can anyone confirm whether these CI failures are spurious? The docs one definitely is, but I'm not sure about the tests

Oops, I can't read apparently! You already mentioned that version is broken on windows :)

[LecrisUT]

Yes, only the windows failure is actionable. Fedora rawhide is borked until python3.14 packages are being rebuilt

[henryiii]

Path.walk exists but needs 3.12, sadly. :)

[henryiii]

Thanks!