refactor: use template function for manifest reader#461
Merged
wgtmac merged 2 commits intoapache:mainfrom Dec 31, 2025
Merged
Conversation
Member
wgtmac
commented
Dec 30, 2025
- Removed PARSE_XXX macros as many as possible
- Refined error reporting
- Removed PARSE_XXX macros as many as possible - Refined error reporting
gty404
approved these changes
Dec 31, 2025
zhjwpku
approved these changes
Dec 31, 2025
liurenjie1024
approved these changes
Dec 31, 2025
HuaHuaY
reviewed
Dec 31, 2025
|
|
||
| Status ParsePartitionFieldSummaryList(ArrowArrayView* view_of_column, | ||
| template <typename T, typename Container, typename Accessor> | ||
| Status ParseIntegerField(const ArrowArrayView* array_view, Container& container, |
Contributor
There was a problem hiding this comment.
I suggest to use a template to merge ParseIntegerField, ParseStringField and ParseBinaryField and simplify Accessor to just receive a projection.
template <typename Container, typename Accessor, typename Transfer>
requires std::ranges::forward_range<Container> &&
requires(Accessor& a, Container& c, Transfer& t, const ArrowArrayView* v,
int64_t i) {
std::invoke(a, *std::ranges::begin(c)) = std::invoke(t, v, i);
}
Status ParseField(const ArrowArrayView* array_view, Container& container,
Accessor accessor, std::string_view field_name, bool required,
ErrorKind error_kind, Transfer transfer) {
auto iter = std::ranges::begin(container);
for (int64_t row_idx = 0; row_idx < array_view->length; row_idx++, iter++) {
if (!ArrowArrayViewIsNull(array_view, row_idx)) {
std::invoke(accessor, *iter) = std::invoke(transfer, array_view, row_idx);
} else if (required) {
return MakeNullError(error_kind, field_name, row_idx);
}
}
return {};
}
template <typename T, typename Container, typename Accessor>
Status ParseIntegerField(const ArrowArrayView* array_view, Container& container,
Accessor accessor, std::string_view field_name, bool required,
ErrorKind error_kind) {
return ParseField(array_view, container, accessor, field_name, required, error_kind,
[](const ArrowArrayView* view, int64_t row_idx) {
return static_cast<T>(ArrowArrayViewGetIntUnsafe(view, row_idx));
});
}
template <typename Container, typename Accessor>
Status ParseStringField(const ArrowArrayView* array_view, Container& container,
Accessor accessor, std::string_view field_name, bool required,
ErrorKind error_kind) {
return ParseField(array_view, container, accessor, field_name, required, error_kind,
[](const ArrowArrayView* view, int64_t row_idx) {
auto value = ArrowArrayViewGetStringUnsafe(view, row_idx);
return std::string(value.data, value.size_bytes);
});
}
template <typename Container, typename Accessor>
Status ParseBinaryField(const ArrowArrayView* array_view, Container& container,
Accessor accessor, std::string_view field_name, bool required,
ErrorKind error_kind) {
return ParseField(array_view, container, accessor, field_name, required, error_kind,
ParseInt8VectorField);
}
Contributor
There was a problem hiding this comment.
After offline discussion, I will open a new PR to modify this.
HuaHuaY
approved these changes
Dec 31, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.