ENH: Adding tests for `tril` and `triu` by prady0t · Pull Request #442 · data-apis/array-api-tests

prady0t · 2026-07-13T12:01:20Z

See: #441 (That PR has been split and tests for tril and triu are added here)

Towards: #439

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

prady0t · 2026-07-17T12:52:32Z

+        expected_elements = [
+                x[idx] if idx[-1] <= idx[-2] + k 
+                else xp.asarray(0, dtype=out.dtype)
+                for idx in sh.ndindex(x.shape)
+        ]
+        if expected_elements:
+            expected = xp.stack(expected_elements)
+        else:
+            expected = xp.asarray([], dtype=out.dtype)
+        expected = xp.reshape(expected, x.shape)
+        ph.assert_array_elements(
+            "tril",
+            out=out,
+            expected=expected,
+        )


I've moved the ph.assert_array_elements outside the loop and used idx entirely for one for loop. expected_elements contains a list of arrays which later gets stacked into one using stack. We then reshape it back to its original shape.

I think this brings us closer to how eye is tested.

One thing that fails right now is running for jax.numpy with --max-examples=10_000 gives:

+ Exception Group Traceback (most recent call last): | File "/Users/pradyotranjan/array-api-tests-1/array_api_tests/test_creation_functions.py", line 398, in test_tril | def test_tril(x, data): | ^^^ | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 2274, in wrapped_test | raise the_error_hypothesis_found | hypothesis.errors.FlakyFailure: Hypothesis test_tril(x=Array([[[[ 2.5453735e+18-4.9889314e+16j]], | | [[ 1.5772956e+19+5.1305820e+16j]]], | | | [[[ 2.8746495e+16+2.5450868e+16j]], | | [[ 1.8076227e+19+1.7226468e+19j]]], | | | [[[-5.6867708e+16-4.8846053e+16j]], | | [[ 2.6559498e+16+7.6443090e+18j]]]], dtype=complex64), data=data(...)) produces unreliable results: Falsified on the first call but did not on a subsequent one (1 sub-exception) | Falsifying example: test_tril( | x=Array([[[[ 2.5453735e+18-4.9889314e+16j]], | | [[ 1.5772956e+19+5.1305820e+16j]]], | | | [[[ 2.8746495e+16+2.5450868e+16j]], | | [[ 1.8076227e+19+1.7226468e+19j]]], | | | [[[-5.6867708e+16-4.8846053e+16j]], | | [[ 2.6559498e+16+7.6443090e+18j]]]], dtype=complex64), | data=data(...), | ) | Draw 1 (k): 0 | Unreliable test timings! On an initial run, this test took 1058.55ms, which exceeded the deadline of 800.00ms, but on a subsequent run it took 4.21 ms, which did not. If you expect this sort of variability in your test timings, consider turning deadlines off for this test by setting deadline=None. +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 1212, in _execute_once_for_engine | result = self.execute_once(data) | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 1150, in execute_once | result = self.test_runner(data, run) | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 824, in default_executor | return function(data) | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 1107, in run | return test(*args, **kwargs) | File "/Users/pradyotranjan/array-api-tests-1/array_api_tests/test_creation_functions.py", line 398, in test_tril | def test_tril(x, data): | ^^^^^^^^ | File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/hypothesis/core.py", line 1041, in test | raise DeadlineExceeded( | datetime.timedelta(seconds=runtime), self.settings.deadline | ) | hypothesis.errors.DeadlineExceeded: Test took 1058.55ms, which exceeds the deadline of 800.00ms. If you expect test cases to take this long, you can use @settings(deadline=...) to either set a higher deadline, or to disable it with deadline=None. +------------------------------------ ===================================================================================================== short test summary info ===================================================================================================== FAILED array_api_tests/test_creation_functions.py::test_tril - DeadlineExceeded('Test took 1058.55ms, which exceeds the deadline of 800.00ms. If you expect test cases to take this long, you can use @settings(deadline=...) to either set a higher deadline, or to disable it with deadline...

Does it still fail if you add --disable-deadline?

Nope. It takes 13 min to complete the test on my machine.

ev-br · 2026-07-17T13:04:10Z

+        ph.assert_shape("tril", out_shape=out.shape, expected=x.shape)
+        expected_elements = [
+                x[idx] if idx[-1] <= idx[-2] + k 
+                else xp.asarray(0, dtype=out.dtype)


at a guess, this xp.asarray is not needed, since you xp.stack the result anyway.

Actually, it's required. As 0 would be counted as a python integer. stack expects a sequence of arrays.

see this failure when removed

| File "/opt/homebrew/Caskroom/miniconda/base/envs/array-api-tests/lib/python3.14/site-packages/array_api_strict/_manipulation_functions.py", line 172, in stack | if len({a.device for a in arrays}) > 1: | ^^^^^^^^ | AttributeError: 'int' object has no attribute 'device' | | ========== FAILING CODE SNIPPET: | xp.tril(Array([[0, 0]], dtype=array_api_strict.uint8), k=0)

Or it's because non-zero elements, x[idx] are arrays, and the spec prohibits xp.stack with a mix of OD arrays and python scalars? Then let's see of moving xp.asarray out of the loop helps the runtime.

EDIT: comments crossed. Value testing is hard :-(.

ev-br · 2026-07-17T13:11:17Z

+        if expected_elements:
+            expected = xp.stack(expected_elements)
+        else:
+            expected = xp.asarray([], dtype=out.dtype)


To double-check: this branching is needed even if if expected_elements is an empty list because 1) xp.stack does not accept an empty list, and even if it did, 2) it has no way to guess dtype=out.dtype. Would be nice to add a comment, since it's one of things which goes against numpy finger (and eye) memory.

ev-br · 2026-07-17T13:26:40Z

            ph.assert_kw_dtype("asarray", kw_dtype=_dtype, out_dtype=out.dtype)
        ph.assert_shape("asarray", out_shape=out.shape, expected=shape)
        for idx, v_expect in zip(sh.ndindex(out.shape), _obj):
+            print(f"out.shape is : {out.shape} idx is:  {idx}")


debug print, don't forget to remove before merging

ev-br · 2026-07-17T14:02:36Z

Okay, with

$ git diff
diff --git a/array_api_tests/test_creation_functions.py b/array_api_tests/test_creation_functions.py
index 37248b6..f76ce46 100644
--- a/array_api_tests/test_creation_functions.py
+++ b/array_api_tests/test_creation_functions.py
@@ -406,9 +406,11 @@ def test_tril(x, data):
         out = xp.tril(x, k=k)
         ph.assert_dtype("tril", in_dtype=x.dtype, out_dtype=out.dtype)
         ph.assert_shape("tril", out_shape=out.shape, expected=x.shape)
+
+        zero = xp.asarray(0, dtype=out.dtype)
         expected_elements = [
                 x[idx] if idx[-1] <= idx[-2] + k 
-                else xp.asarray(0, dtype=out.dtype)
+                else zero
                 for idx in sh.ndindex(x.shape)
         ]
         if expected_elements:

the runtime of test_tril and test_eye are comparable on my laptop. Both are abysmal (~3 min for --max-examples=1000) but that's what it is for iterating over the array elements in python.

Therefore, let's declare this being as good as it gets, cc #197. There's no point running with --max-examples=10_000 for this test TBH.

prady0t · 2026-07-17T14:09:40Z

he runtime of test_tril and test_eye are comparable on my laptop. Both are abysmal (~3 min for --max-examples=1000) but that's what it is for iterating over the array elements in python.

Mine just completed with the same diff in 15 min but for 10_000. Pushing changes now.

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

Adding tests for tril and triu

5ca8a3b

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

prady0t changed the title ~~Adding tests for tril and triu~~ ENH: Adding tests for tril and triu Jul 13, 2026

prady0t mentioned this pull request Jul 13, 2026

ENH: Adding dtype parameter to the test for linalg.trace #441

Open

prady0t commented Jul 13, 2026

View reviewed changes

Comment thread array_api_tests/test_creation_functions.py Outdated

remove in-place modifications to address jax.numpy

409ac69

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

prady0t commented Jul 16, 2026

View reviewed changes

Comment thread array_api_tests/test_creation_functions.py Outdated

prady0t added 2 commits July 17, 2026 17:59

use idx for loop entirely

64e6b97

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

use stack for elements

84be72a

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

prady0t commented Jul 17, 2026

View reviewed changes

ev-br reviewed Jul 17, 2026

View reviewed changes

adding comment + removing debug print + moving zero array out of loop

5dd3b5e

Signed-off-by: Pradyot Ranjan <99216956+pradyotRanjan@users.noreply.github.com>

prady0t marked this pull request as ready for review July 17, 2026 14:11

Uh oh!

Conversation

prady0t commented Jul 13, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ev-br Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ev-br Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ev-br commented Jul 17, 2026

Uh oh!

prady0t commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ev-br Jul 17, 2026 •

edited

Loading

ev-br Jul 17, 2026 •

edited

Loading