Reduce datafusion-spark dependency footprint

[dentiny]

Which issue does this PR close?

N/A

Rationale for this change

datafusion-spark pulls unnecessary crates into its production dependency graph, increasing compilation time and artifact size for downstream users.

What changes are included in this PR?

• Replace compatibility re-exports with the narrower crates that define TableFunction and FunctionRegistry.
• Remove the unused crypto_expressions activation; Spark provides its own SHA-1, SHA-2, and CRC32 implementations.
• Fix the quote import so builds without the optional core feature continue to compile.

These changes reduce the production dependency graph from 279 to 257 packages. Cargo Machete does not report these dependencies because they are referenced in source through re-exports or activated through Cargo features.

Are these changes tested?

Yes

Are there any user-facing changes?

No. Public APIs and default behavior remain unchanged.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.17%. Comparing base (0a429a3) to head (ffc4d5a).

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24351      +/-   ##
==========================================
- Coverage   81.17%   81.17%   -0.01%     
==========================================
  Files        1109     1109              
  Lines      388038   388038              
  Branches   388038   388038              
==========================================
- Hits       314992   314983       -9     
- Misses      54507    54512       +5     
- Partials    18539    18543       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[2010YOUY01]

Thank you for the fix.

There is already the same check in CI

• datafusion/.github/workflows/dependencies.yml

  Line 70 in 0a429a3

  run: cargo machete --with-metadata

But it's not detecting those dependencies, perhaps we should update the CI first.

[dentiny]

But it's not detecting those dependencies, perhaps we should update the CI first.

Hi thanks for the quick reply! I double confirmed with my agent, it's not Machete -- Machete considers datafusion-catalog and datafusion-execution used because their names appeared in imports, even though they only re-exported types from narrower crates.

The prompt I gave is

Find unused and misplaced dependencies in the repo, 
- we could use tools like Machete
- check whether we could put test / benchmark only dependencies out of production ones