Appender rework (1.3) #305
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a backport of the PR duckdb#295 to `v1.3-ossivalis` stable branch. This change implements the access to the Appender interface from Java with the following features: - C API is used to access the native Appender - necessary C API calls are exposed to Java using JNI wrappers as thin as possible - Java calls mirror corresponding C API calls 1 to 1 - the data chunk interface of the Appender API is used: vector data is exposed as a direct ByteBuffer, all primitive appended values are written to this buffer from Java without going through JNI + C API ( that is still used for some types with calls like: `duckdb_vector_assign_string_element_len`) - Java-side Appender/DataChunk/Vector data structures follow the Go lang's Appender implementation closely (with nested arrays initialization etc) - Java Appender usage is made thread-safe for concurrent Appender or Connection closure; `append()` calls are remained not thread-safe (to minimize the overhead), but their usage cannot cause the JNI-side crash - Java API of the new Appender is modeled after the `java.lang.StringBuilder` class and intended to be used with method chaining - support primitive arrays (one and two dimensional) - support for `NULL` and `DEFAULT` values - type checks between Java types and DB column types are enforced Note: previous version of the Appender (that internally creates `Value` instances and appends them one by one) is still available for compatibility. It can be created using `Connection#createSingleValueAppender` method. It is marked as 'deprecated' and intended to be removed in future versions. Testing: existing Appender test suite is extended and adapted to new Appender API. Fixes: duckdb#84, duckdb#100, duckdb#110, duckdb#139, duckdb#157, duckdb#163, duckdb#219, duckdb#249
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a backport of the PR #295 to
v1.3-ossivalisstable branch.This change implements the access to the Appender interface from Java with the following features:
duckdb_vector_assign_string_element_len)append()calls are remained not thread-safe (to minimize the overhead), but their usage cannot cause the JNI-side crashjava.lang.StringBuilderclass and intended to be used with method chainingNULLandDEFAULTvaluesNote: previous version of the Appender (that internally creates
Valueinstances and appends them one by one) is still available for compatibility. It can be created usingConnection#createSingleValueAppendermethod. It is marked as 'deprecated' and intended to be removed in future versions.Testing: existing Appender test suite is extended and adapted to new Appender API.
Fixes: #84, #100, #110, #139, #157, #163, #219, #249