Run RAT check only on files tracked by git

[janhoy]

Now, if you have a file that in your repo that is not tracked by git, and is even in .gitignore, the RAT source scan will still complain about license header. This is annoying if you have e.g. AI tool that stores some content in the repo folder but which is ignored.

This PR filters so that RAT only cares about tracked files.

[dsmiley]

Thanks for addressing this problem -- it annoys me too.
I forgot if Lucene has moved onto a newer RAT release that addresses this natively... 50/50 chance I checked and saw but Lucene's build has move don a way from us.

[janhoy]

I forgot if Lucene has moved onto a newer RAT release that addresses this natively.

Lucene moved away from RAT to a custom build-src java class, see apache/lucene#15195. If anyone feels there is a reason for us to so similar, let that be a separate effort.

Also, RAT has as a newer version which includes tracked file handling as a native feature, but the way we hold it (we use the Ant task programatically) we'd not benefit from that automatically. If RAT only would publish a gradle plugin, not just a maven one. Update There is a 3rd party RAT gradle plugin we could also consider, but it does not natively support excluding non-tracked files either.

This PR now

• Replace shell-out with JGit: Use JGit's DirCache API (already a build dependency) instead of shelling out to git ls-files --cached to determine tracked files
• Read the git index once and cache it via rootProject.ext, avoiding redundant I/O across ~40 subproject rat tasks
• Detect git worktrees (.git is a file) with a clear warning, handle non-git directories by disabling the filter (RAT scans everything as before), and use finally to prevent repository resource leaks
• Normalize path separators (\ → /) when computing subproject prefixes, since JGit always uses forward slashes
• Replaces deprecated APIs in the rest of the file (exclude last commit if you want to hide those changes)

[epugh]

will this work then in tne cherrypick script? that is where i get bit by this a lot!

[janhoy]

will this work then in tne cherrypick script? that is where i get bit by this a lot!

Yes, we explicitly enabled RAT in cherry-pick script and it requires a completely clean working area which is stupid, for the task it does. We have other checks that confirm that a git checkout is clean.

This should fix it..

[janhoy]

Thanks for the review david. Inspired by your cleanup, I asked LLM to find more, and surely, there were duplicate excludes for solr:core that were already defined globally. If anyone has a local lucene/ checkout inside, it would not be tracked. Nine more lines shaved.

[dsmiley]

I think it's very lame that we have license headers on build files