STAR-508: Cursor-based memtable walks #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(cherry picked from commit f4cca35)
Synchronize schema pull handling with applying new schema changes There's a race condition around pulling schema changes, that can occur in case the schema changes push/propagation mechanism is not immediately effective (e.g. because of network delay, or because of the pulling node being down, etc.). If schema changes happen on node 1, these changes do not reach node 2 immediately through the SCHEMA.PUSH mechanism, and are first recognized during gossiping, the corresponding SCHEMA.PULL request from node 2 can catch the node 1 schema in the middle of it being modified by another schema change request. This can easily lead to problems (e.g. if a new table is being added, and the node 2 request reads the changes that need to be applied to system_schema.tables, but not the ones that need to be applied to system_schema.columns). This PR addresses that by synchronizing the SCHEMA.PULL "RPC call" executed in node 1 by a request from node 2 with the method for applying schema changes in node 1. It also adds debug level logging tracking SCHEMA.PUSH and SCHEMA.PULL messages, as there were some unexpected findings around these that may need further investigation. E.g. during my investigations, seemingly redundant SCHEMA.PULL messages were sent multiple times from node 1 to node 2, even though no schema changes were made at node 2, and node 2 did not go offline. Co-authored-by: Dimitar Dimitrov Co-authored-by: Dimitar Dimitrov <[email protected]> (cherry picked from commit ab2a669)
Improve error message when altering an MV with default ttl > 0. Co-authored-by: Brandon Williams <[email protected]> (cherry picked from commit 8296fe1)
…ache#172) ... of 2i on collections Co-authored-by: Andrés de la Peña [email protected] (cherry picked from commit 07edf0d)
…e#171) If a user has only MODIFY permission on a table and there is a materialized view built on the same table an insert will fail with the following error: Unauthorized: Error from server: code=2100 [Unauthorized] Only base MODIFY permission is required to update base with MV. Co-authored-by: Zhao Yang <[email protected]> (cherry picked from commit 53c5aa6)
* remove dead code * add awaitSyncAt test The test verifies two cases when nanoseconds overflowing causes awaitSyncAt to never return (as described in the original ticket). * fix nanoseconds overflowing in CommitLogService * Use clock consistently in all CommitLogService implementations * Compare nanoseconds as a difference as advised in the docs: https://docs.oracle.com/javase/8/docs/api/java/lang/System.html#nanoTime * add abs values where the overflowed value may cause a negative result Co-authored-by: Sergio Bossa <[email protected]> (cherry picked from commit 91db2d4)
* selectors should contain elements with same type Expressions like "SELECT [(CAST(pk AS BIGINT), CAST(ck AS BIGINT)), (t, WRITETIME(t))] FROM %s" can't be deserialized. If we try to extract the result by calling getRows we get: org.apache.cassandra.serializers.MarshalException: Expected 8 or 0 byte long This commit makes sure that all collection elements have the same type. Co-authored-by: Zhao Yang <[email protected]> * test for Maps.getExactMapTypeIfKnow Maps are a bit different than Lists and Sets. Maps can't be selected without a type hint, which makes it impossible to exercise query execution path with `getExactMapTypeIfKnow`. That is why tests for "map SELECTs" are missing TermSelectionTest.java. The previous fixed `Maps.getExactMapTypeIfKnow` but it can't be tested in a similar way as for Lists and Sets. That's why commit adds a separate unit tests for `Maps.getExactMapTypeIfKnow`. Co-authored-by: Zhao Yang <[email protected]> (cherry picked from commit 78b0117)
Co-authored-by: Robert Stupp <[email protected]> (cherry picked from commit 4109c73)
If a flush is aborted, e.g. by exception thrown by flushAllNonCFSBackedIndexesBlocking(), this was done by closing the flush writer, potentially concurrently with operations on it. The latter is unsafe and may cause writes to released memory. Fixed by adding an abort mechanism to the flush runnables. Port of DB-962 with an earlier commit. patch by Stefania Alborghetti; reviewed by Alex Petrov ported by Branimir Lambov; reviewed by Ruslan Fomkin (cherry picked from commit 8df735d)
This ports the up-to-date version of the test introduced by DB-1157 patch by Zhao Yang; reviewed by Branimir Lambov ported by Branimir Lambov (cherry picked from commit beec110)
(cherry picked from commit 3dcd047)
…s constructed from date string or from millis since epoch Co-authored-by: Ulises Cervino Beresi <[email protected]> (cherry picked from commit 7bcbf84)
…omName (apache#168) The problematic syntax was a function name with empty argument list. Co-authored-by: kamlesh ghoradkar <[email protected]> (cherry picked from commit 5d7df12)
…pache#523) (apache#188) patch by Zhao Yang; reviewed by Robert Stupp for DB-1375 Co-authored-by: Zhao Yang <[email protected]> (cherry picked from commit d09bc66)
Porting patch DB-1511, riptano/apollo#627 Co-authored-by: Zhao Yang <[email protected]> (cherry picked from commit 998eca0)
Migrate to commit log initialization implemented by ServerTestUtils that is race free. Files were removed by the cleanup that followed CL init. (cherry picked from commit 37e2e3d)
…usage (apache#177) * STAR-745: Add BloomFilter global memory limit Added a memory limiter which is a global instance (static field in BloomFilter) which monitors the total memory used by all created (and deserialized) Bloom filters. When the limit is reached we return a dummy always-true filter with a log error message. * STAR-745: Recreate Bloom filter on startup In certain situations BF will be recreated on startup (for example, if FP chance changed more than the defined tolerance). Reviewed by: Daniel Jatnieks <[email protected]> Co-authored-by: Stefania Alborghetti <[email protected]> (cherry picked from commit c1a2595)
…pache#193) Fix scrubber so that it can work with broken indexes. If the index is broken, but the data file is ok, we can continue scrubbing and the index will be rebuilt. With this patch it will work even if we cannot access the initial position in the index. Also, the corrupted file will be obsoleted before finishing the rewriter so that it will not be attempted to move starts (and thus fail at the end as move starts requires correct indexes). There are also some explanations added to how canonical set of sstables is constructed and why Co-authored-by: Branimir Lambov <[email protected]> (cherry picked from commit c0be3ae)
apache#194) STAR-583 Fix infinite loop when replaying a truncated commit log file and truncation is tolerated Co-authored-by: Massimiliano Tomassi <[email protected]> (cherry picked from commit 2bf6dd4)
Harden txn log files against exceptions when adding records and improve log messages Port of riptano/apollo@83b93bc434dc2f07371e848a9b24854403bf740e (cherry picked from commit 8175eb8)
The ported bug fix patch changes impelmentation of LogReplicaSet.append in the error case, however no tests exist. This commit adds tests to cover the error path. It also changes the version of JUnit to use assertThrow. (cherry picked from commit ca66e38)
No performance benefit and for uglier code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.