Skip to content

stats: fix UB in from_slice #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 15, 2022
Merged

stats: fix UB in from_slice #83

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 44 additions & 28 deletions src/stats.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,40 +93,56 @@ fn value_histogram_index(size: u32) -> Option<usize> {
}
}

#[inline(always)]
fn read_array<T, const N: usize, F>(cursor: &mut Cursor<&[u8]>, reader: F) -> [T; N]
where
F: Fn(&mut Cursor<&[u8]>) -> T,
{
// SAFETY:
// https://doc.rust-lang.org/std/mem/union.MaybeUninit.html#initializing-an-array-element-by-element
let mut data: [MaybeUninit<T>; N] = unsafe { MaybeUninit::uninit().assume_init() };
for item in &mut data[..] {
item.write(reader(cursor));
}
data.map(|x| unsafe { x.assume_init() })
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this copies the data unfortunately
using transmute currently not possible with const generics: rust-lang/rust#61956
but I can change it to a macro instead

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, I think it's a noop: https://godbolt.org/z/YcxonWxPa

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following functions are unstable, but could be used in the future:

Copy link
Member

@arkpar arkpar Jul 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well get rid of unsafe here.

 let plain: [u32; 1024] = [Default::default(); 1024];
 let mut atomic: [AtomicU32; 1024] = a.map(AtomicU32::new);
 // Read into `atomic` below.

Shoudl be optimized away as well.

Copy link
Member Author

@ordian ordian Jul 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure it's optimized this way though. The produced code seems different: https://godbolt.org/z/GGnfTro49
Also to sanity check, created a bench:

from_slice/Unsafe       time:   [1.0590 µs 1.0613 µs 1.0636 µs]                               
from_slice/Safe         time:   [2.1197 µs 2.1255 µs 2.1326 µs]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's leave it as it is then.

}

impl ColumnStats {
#[allow(clippy::uninit_assumed_init)]
pub fn from_slice(data: &[u8]) -> ColumnStats {
let mut cursor = Cursor::new(data);
let mut value_histogram: [AtomicU32; HISTOGRAM_BUCKETS] =
unsafe { MaybeUninit::uninit().assume_init() };
for item in value_histogram.iter_mut() {
*item = read_u32(&mut cursor);
}
let mut query_histogram: [AtomicU64; SIZE_TIERS] =
unsafe { MaybeUninit::uninit().assume_init() };
for item in query_histogram.iter_mut() {
*item = read_u64(&mut cursor);
}
let mut stats = ColumnStats {
let cursor = &mut cursor;

let value_histogram = read_array(cursor, read_u32);
let query_histogram = read_array(cursor, read_u64);
let oversized = read_u64(cursor);
let oversized_bytes = read_u64(cursor);
let total_values = read_u64(cursor);
let total_bytes = read_u64(cursor);
let commits = read_u64(cursor);
let inserted_new = read_u64(cursor);
let inserted_overwrite = read_u64(cursor);
let removed_hit = read_u64(cursor);
let removed_miss = read_u64(cursor);
let queries_miss = read_u64(cursor);
let uncompressed_bytes = read_u64(cursor);
let compression_delta = read_array(cursor, read_i64);

ColumnStats {
value_histogram,
query_histogram,
oversized: read_u64(&mut cursor),
oversized_bytes: read_u64(&mut cursor),
total_values: read_u64(&mut cursor),
total_bytes: read_u64(&mut cursor),
commits: read_u64(&mut cursor),
inserted_new: read_u64(&mut cursor),
inserted_overwrite: read_u64(&mut cursor),
removed_hit: read_u64(&mut cursor),
removed_miss: read_u64(&mut cursor),
queries_miss: read_u64(&mut cursor),
uncompressed_bytes: read_u64(&mut cursor),
compression_delta: unsafe { MaybeUninit::uninit().assume_init() },
};
for item in stats.compression_delta.iter_mut() {
*item = read_i64(&mut cursor);
oversized,
oversized_bytes,
total_values,
total_bytes,
commits,
inserted_new,
inserted_overwrite,
removed_hit,
removed_miss,
queries_miss,
uncompressed_bytes,
compression_delta,
}
stats
}

pub fn empty() -> ColumnStats {
Expand Down