Skip to content

metrics: fix w-amp calculation w/ blob files #4764

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 27, 2025

Conversation

jbowens
Copy link
Collaborator

@jbowens jbowens commented May 23, 2025

Fix the w-amp calculation to properly account for blob files. We should be including all bytes written to disk in the numerator, and only bytes "in" (flushed in the case of blob files) in the denominator.

Informs #4581.

@jbowens jbowens requested a review from a team as a code owner May 23, 2025 19:30
@jbowens jbowens requested a review from RaduBerinde May 23, 2025 19:30
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 3 files reviewed, 2 unresolved discussions


metrics.go line 189 at r1 (raw file):

//	TableBytesFlushed + TableBytesCompacted + BlobBytesFlushed + BlobBytesWritten
//	-----------------------------------------------------------------------------
//	                      TableBytesIn + BlobBytesFlushed

For L0, TableBytesIn is defined as "the bytes written to the WAL.". This would include BlobBytesFlushed, no?

I checked the code and it's updated here:

l0Metrics.TableBytesIn += d.mu.mem.queue[i].logSize

I think it's fine to keep the definition as is, but we need to divide only by TableBytesIn here, and we need to update this line to also add BlobBytesFlushed to it:

l0Metrics.TableBytesIn = l0Metrics.TableBytesFlushed


metrics.go line 194 at r1 (raw file):

		return 0
	}
	return float64(m.TableBytesFlushed+m.TableBytesCompacted+m.BlobBytesFlushed+m.BlobBytesWritten) /

[nit] We should rename BlobBytesWritten to BlobBytesCompacted, it is confusing

Copy link
Collaborator Author

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 5 files reviewed, 1 unresolved discussion (waiting on @RaduBerinde)


metrics.go line 189 at r1 (raw file):

Previously, RaduBerinde wrote…

For L0, TableBytesIn is defined as "the bytes written to the WAL.". This would include BlobBytesFlushed, no?

I checked the code and it's updated here:

l0Metrics.TableBytesIn += d.mu.mem.queue[i].logSize

I think it's fine to keep the definition as is, but we need to divide only by TableBytesIn here, and we need to update this line to also add BlobBytesFlushed to it:

l0Metrics.TableBytesIn = l0Metrics.TableBytesFlushed

Good catch, done


metrics.go line 194 at r1 (raw file):

Previously, RaduBerinde wrote…

[nit] We should rename BlobBytesWritten to BlobBytesCompacted, it is confusing

Done.

Fix the w-amp calculation to properly account for blob files. We should be
including all bytes written to disk in the numerator, and only bytes "in"
(flushed in the case of blob files) in the denominator.

Informs cockroachdb#4581.
Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 0 of 5 files reviewed, all discussions resolved

@jbowens
Copy link
Collaborator Author

jbowens commented May 27, 2025

TFTR!

@jbowens jbowens merged commit a38c80f into cockroachdb:master May 27, 2025
6 checks passed
@jbowens jbowens deleted the wamp-fix branch May 27, 2025 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants