8346316: compiler/rangechecks/TestLongRangeCheck.java fails intermittently with DeoptimizeALot

[chhagedorn]

Running compiler/rangechecks/TestLongRangeCheck.java with -XX:+DeoptimizeALot fails very intermittently with this assert:

jdk/test/hotspot/jtreg/compiler/rangechecks/TestLongRangeCheck.java

Lines 60 to 64 in e0f6001

	private static void assertIsNotCompiled(Method m) {
	if (WHITE_BOX.isMethodCompiled(m) && WHITE_BOX.getMethodCompilationLevel(m) == CompilerWhiteBoxTest.COMP_LEVEL_FULL_OPTIMIZATION) {
	throw new RuntimeException("should have been deoptimized");
	}
	}

which means that a method is unexpectedly still compiled. The failing call to assertIsNotCompiled() is made here:

jdk/test/hotspot/jtreg/compiler/rangechecks/TestLongRangeCheck.java

Lines 315 to 329 in e0f6001

	m.invoke(null, 0, stop, length, offset);
	compile(m);
	offset = 0;
	stop = stride * 5;
	try {
	m.invoke(null, 0, stop, length, offset);
	throw new RuntimeException("should have thrown");
	} catch(InvocationTargetException e) {
	if (!(e.getCause() instanceof IndexOutOfBoundsException)) {
	throw new RuntimeException("unexpected exception");
	}
	}
	assertIsNotCompiled(m);

Let's look closer why this can intermittently fail.

What's Supposed to Be Checked

First, let's see what the test does when it passes normally:

We first call testStridePosScalePosInIntLoopOverflow() with specially constructed arguments:

start = 0
stop = (1 << 14) * 5 = 81920
length = 3_758_096_385
offset = 2^31

which makes Preconditions.checkIndex() not fail even in the last iteration where scale * i is equal to MIN_INT.

jdk/test/hotspot/jtreg/compiler/rangechecks/TestLongRangeCheck.java

Lines 666 to 672 in e0f6001

	public static void testStridePosScalePosInIntLoopOverflow(long start, long stop, long length, long offset) {
	checkInputs(start, stop);
	final int scale = 1 << 15;
	final int stride = 1 << 14;
	for (int i = (int)start; i < (int)stop; i += stride) {
	Preconditions.checkIndex(scale * i + offset, length, null);
	}

We then force compile() the method and inline the Preconditions.checkIndex() intrinsic which emits a trap when we are out of bounds.

Afterwards, we call the method again but this time with offset = 0. In the last iteration, we get an index which is equal to MIN_INT which is out of bounds and we hit the trap. We deopt to the interpreter and make the method not entrant.

The interpreter then throws an IOOBE which we expect and catch it. Since we deopted and marked the method not entrant, we assert this by calling assertIsNotCompiled(). So far, so good.

Problem with -XX:+DeoptimizeALot

With DeoptimizeALot, we randomly deoptimize (without making the compiled method not entrant). When running the test with DeoptimizeALot, we could have such a random deoptimization while calling checkInputs() on L667. We jump to the interpreter, return from the method, call Preconditions.checkIndex() and throw the IOOBE (all in the interpreter). Since we have not hit the C2 emitted trap for Preconditions.checkIndex(), we do not throw the compilation away which means that we fail when calling assertIsNotCompiled() - the method is still compiled.

Proposed Solution

Since the test is specifically testing this pattern:

1. Compile with trap from Preconditions.checkIndex()
2. Take the trap and deopt
3. Assert not compiled anymore

we should not interfere with DeoptimizeALot. I therefore propose to just not execute this test with DeoptimizeALot

Testing

The failure can more reliably be observed (~5-10% failure rate) when running with:

-XX:CompileThreshold=100 -XX:-TieredCompilation -XX:+DeoptimizeALot -XX:DeoptimizeALotInterval=0

By restricting the test, it will obviously no longer fail with these flags because the test will not be run with DeoptimizeALot.

Thanks,
Christian

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

• JDK-8346316: compiler/rangechecks/TestLongRangeCheck.java fails intermittently with DeoptimizeALot (Bug - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/32041/head:pull/32041
$ git checkout pull/32041

Update a local copy of the PR:
$ git checkout pull/32041
$ git pull https://git.openjdk.org/jdk.git pull/32041/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 32041

View PR using the GUI difftool:
$ git pr show -t 32041

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/32041.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back chagedorn! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@chhagedorn The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[openjdk]

The total number of required reviews for this PR has been set to 2 based on the presence of this label: hotspot-compiler. This can be overridden with the /reviewers command.

[mlbridge]

Webrevs

• 00: Full (b1c49ba6)