Convert Hive Unit Tests to Docker #3032
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cgivre
added
code-cleanup
refactoring
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DRILL-XXXX: Convert Hive Unit Tests to Docker
Description
This PR migrates all Hive storage plugin integration tests from the deprecated embedded HiveServer2 approach (which fails on Java 11+ due to
URLClassLoaderincompatibility) to a Docker-based testing infrastructure usingTestcontainers. Note that prior to this PR, the Hive tests were not running at all because of Java version checks that disabled the tests on Java > 8.This migration is necessary to update the Hive plugin from using the Hive v3. client to Hive v4. Hive v3 is very out of date and has multiple CVEs which are only addressed by updating to version 4.
The migration includes all 6 complex type test classes:
and authorization tests (TestStorageBasedHiveAuthorization, TestSqlStdBasedAuthorization), with all tests rewritten to generate test data via JDBC connections instead of relying on pre-existing embedded data.
To address the 20-30 minute Docker container startup time on ARM64 Macs (due to AMD64 image emulation), the PR adds architecture-aware Maven profiles that automatically skip Hive integration tests on ARM64 while allowing them to run normally on AMD64 systems (CI/CD). Developers can override with
-Pforce-hive-testsif needed.Documentation
No user facing changes.
Testing