-
Notifications
You must be signed in to change notification settings - Fork 14.5k
[lldb] deactivate some tests on older SDKs #147768
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[lldb] deactivate some tests on older SDKs #147768
Conversation
@llvm/pr-subscribers-lldb Author: Charles Zablit (charles-zablit) Changes#144913 was reverted because some of the Darwin tests were failing on SDKs prior to In this patch, we deactivate the failing Darwin tests on older SDKs (prior to Full diff: https://github.com/llvm/llvm-project/pull/147768.diff 19 Files Affected:
diff --git a/lldb/test/API/commands/expression/import-std-module/array/TestArrayFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/array/TestArrayFromStdModule.py
index 1b21fe374ce5f..a488276c2e54d 100644
--- a/lldb/test/API/commands/expression/import-std-module/array/TestArrayFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/array/TestArrayFromStdModule.py
@@ -10,6 +10,7 @@
class TestCase(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/basic/TestImportStdModule.py b/lldb/test/API/commands/expression/import-std-module/basic/TestImportStdModule.py
index 0632328e6f4c4..0c242d5c8845d 100644
--- a/lldb/test/API/commands/expression/import-std-module/basic/TestImportStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/basic/TestImportStdModule.py
@@ -10,6 +10,7 @@
class ImportStdModule(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
def test(self):
self.build()
@@ -38,6 +39,7 @@ def test(self):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
def test_non_cpp_language(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/conflicts/TestStdModuleWithConflicts.py b/lldb/test/API/commands/expression/import-std-module/conflicts/TestStdModuleWithConflicts.py
index 95953de02866a..b3241aab72ab3 100644
--- a/lldb/test/API/commands/expression/import-std-module/conflicts/TestStdModuleWithConflicts.py
+++ b/lldb/test/API/commands/expression/import-std-module/conflicts/TestStdModuleWithConflicts.py
@@ -15,6 +15,7 @@
class TestImportStdModuleConflicts(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/deque-basic/TestDequeFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/deque-basic/TestDequeFromStdModule.py
index cfc480ae73a19..38b8508b516df 100644
--- a/lldb/test/API/commands/expression/import-std-module/deque-basic/TestDequeFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/deque-basic/TestDequeFromStdModule.py
@@ -10,6 +10,7 @@
class TestBasicDeque(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py
index 06f47e548b00f..85eaa8fe39ea8 100644
--- a/lldb/test/API/commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py
@@ -11,6 +11,7 @@ class TestDbgInfoContentDeque(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "18.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule.py
index d69f0acaf0461..a3a409d678ee6 100644
--- a/lldb/test/API/commands/expression/import-std-module/forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule.py
@@ -10,6 +10,7 @@
class TestDbgInfoContentForwardList(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/forward_list/TestForwardListFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/forward_list/TestForwardListFromStdModule.py
index e3ed8f8147294..c9f4a1588f39f 100644
--- a/lldb/test/API/commands/expression/import-std-module/forward_list/TestForwardListFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/forward_list/TestForwardListFromStdModule.py
@@ -10,6 +10,7 @@
class TestBasicForwardList(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/iterator/TestIteratorFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/iterator/TestIteratorFromStdModule.py
index 190bc5f29f66b..5c82ac352abb8 100644
--- a/lldb/test/API/commands/expression/import-std-module/iterator/TestIteratorFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/iterator/TestIteratorFromStdModule.py
@@ -10,6 +10,7 @@
class TestCase(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py
index 67ea6c68a711e..0ecc244412da1 100644
--- a/lldb/test/API/commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py
@@ -12,7 +12,7 @@ class TestDbgInfoContentList(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "12.0"])
- @skipIf(macos_version=["<", "14.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/list/TestListFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/list/TestListFromStdModule.py
index 85158ee28ae9e..f29f353ae0791 100644
--- a/lldb/test/API/commands/expression/import-std-module/list/TestListFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/list/TestListFromStdModule.py
@@ -10,6 +10,7 @@
class TestBasicList(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/non-module-type-separation/TestNonModuleTypeSeparation.py b/lldb/test/API/commands/expression/import-std-module/non-module-type-separation/TestNonModuleTypeSeparation.py
index 5558c86ec2822..5e0ab48cfa5c9 100644
--- a/lldb/test/API/commands/expression/import-std-module/non-module-type-separation/TestNonModuleTypeSeparation.py
+++ b/lldb/test/API/commands/expression/import-std-module/non-module-type-separation/TestNonModuleTypeSeparation.py
@@ -11,6 +11,7 @@
class TestCase(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
"""
diff --git a/lldb/test/API/commands/expression/import-std-module/retry-with-std-module/TestRetryWithStdModule.py b/lldb/test/API/commands/expression/import-std-module/retry-with-std-module/TestRetryWithStdModule.py
index aa8646b4cfb73..d003558f812d8 100644
--- a/lldb/test/API/commands/expression/import-std-module/retry-with-std-module/TestRetryWithStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/retry-with-std-module/TestRetryWithStdModule.py
@@ -6,6 +6,7 @@
class TestCase(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/shared_ptr-dbg-info-content/TestSharedPtrDbgInfoContentFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/shared_ptr-dbg-info-content/TestSharedPtrDbgInfoContentFromStdModule.py
index 85755a62018c0..50419b6f761ce 100644
--- a/lldb/test/API/commands/expression/import-std-module/shared_ptr-dbg-info-content/TestSharedPtrDbgInfoContentFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/shared_ptr-dbg-info-content/TestSharedPtrDbgInfoContentFromStdModule.py
@@ -10,6 +10,7 @@
class TestSharedPtrDbgInfoContent(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/shared_ptr/TestSharedPtrFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/shared_ptr/TestSharedPtrFromStdModule.py
index 35d62ae50f6a8..9f04361c48d77 100644
--- a/lldb/test/API/commands/expression/import-std-module/shared_ptr/TestSharedPtrFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/shared_ptr/TestSharedPtrFromStdModule.py
@@ -11,6 +11,7 @@ class TestSharedPtr(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "17.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent.py b/lldb/test/API/commands/expression/import-std-module/unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent.py
index e21a4cc5a565b..39b720b895fe4 100644
--- a/lldb/test/API/commands/expression/import-std-module/unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent.py
+++ b/lldb/test/API/commands/expression/import-std-module/unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent.py
@@ -11,6 +11,7 @@ class TestUniquePtrDbgInfoContent(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "9.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipIfLinux # s.reset() causes link errors on ubuntu 18.04/Clang 9
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/unique_ptr/TestUniquePtrFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/unique_ptr/TestUniquePtrFromStdModule.py
index 2952b2e4f0727..90384d5e96a22 100644
--- a/lldb/test/API/commands/expression/import-std-module/unique_ptr/TestUniquePtrFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/unique_ptr/TestUniquePtrFromStdModule.py
@@ -11,6 +11,7 @@ class TestUniquePtr(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "9.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipIfLinux # s.reset() causes link errors on ubuntu 18.04/Clang 9
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/vector-of-vectors/TestVectorOfVectorsFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/vector-of-vectors/TestVectorOfVectorsFromStdModule.py
index 5a168aa21d3de..6fa9dd5b6f56e 100644
--- a/lldb/test/API/commands/expression/import-std-module/vector-of-vectors/TestVectorOfVectorsFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/vector-of-vectors/TestVectorOfVectorsFromStdModule.py
@@ -10,6 +10,7 @@
class TestVectorOfVectors(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/weak_ptr-dbg-info-content/TestDbgInfoContentWeakPtrFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/weak_ptr-dbg-info-content/TestDbgInfoContentWeakPtrFromStdModule.py
index e95214df3b38c..19d24c1bcb89b 100644
--- a/lldb/test/API/commands/expression/import-std-module/weak_ptr-dbg-info-content/TestDbgInfoContentWeakPtrFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/weak_ptr-dbg-info-content/TestDbgInfoContentWeakPtrFromStdModule.py
@@ -11,6 +11,7 @@ class TestDbgInfoContentWeakPtr(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "17.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
diff --git a/lldb/test/API/commands/expression/import-std-module/weak_ptr/TestWeakPtrFromStdModule.py b/lldb/test/API/commands/expression/import-std-module/weak_ptr/TestWeakPtrFromStdModule.py
index 0bc0fd466acd5..e3cc9b92569c9 100644
--- a/lldb/test/API/commands/expression/import-std-module/weak_ptr/TestWeakPtrFromStdModule.py
+++ b/lldb/test/API/commands/expression/import-std-module/weak_ptr/TestWeakPtrFromStdModule.py
@@ -11,6 +11,7 @@ class TestSharedPtr(TestBase):
@add_test_categories(["libc++"])
@skipIf(compiler=no_match("clang"))
@skipIf(compiler="clang", compiler_version=["<", "17.0"])
+ @skipIf(macos_version=["<", "15.0"])
@skipUnlessDarwin
def test(self):
self.build()
|
Deactivating these tests allows the CI to pass on the Apple bots: swiftlang#10971 |
macOS 15 (Sequoia) is the latest macOS release. This seems pretty aggressive. Are any of the bots running macOS 15 at this point? I'm pretty sure GreenDragon is not. I'm worried this is equivalent to disabling the tests except at people's desk, which means they're going to regress... These test have existed since before macOS 15 was released. What changed that requires the latest SDK? You say that "This config is not supported (having a version of lldb which is more recent than the installed SDK)." but I'm not sure what that means and that definitely doesn't sound right. Is the problem that #144913 doesn't work with older versions of the SDK? If so, can we make it behave correctly with older and newer SDKs instead? |
I brought this up with @adrian-prantl couple of weeks ago and the conclusion was that the green dragon AArch64 bots do run macOS 15. We would only lose coverage on the Intel bot that's running macOS 14. I don't think it's worth keeping support for older SDKs alive for this particular feature. It's currently in maintenance mode and the reason @charles-zablit is removing support for older SDKs is because it unblocks some performance work for LLDB on Windows. Which seems like on OK trade-off to me |
I don't believe that's accurate. The incremental AS bot is running macOS 14.1 and building against a 14.2 SDK. From the console logs:
If the feature is in maintenance mode then I'm less concerned with losing some test coverage, if we have a bot. Historically, these tests do break and haven't always been the most straightforward to fix. |
Ahh you're right. I misremembered. The conclusion was we will upgrade the AS bots on Green Dragon and only then land this PR. We do want coverage for these since they usually expose Clang modules related issues. So tentatively approving conditional on the bot upgrade. |
Actually just saw that our Intel bots are running a newer OS:
So we would still technically get coverage. Though would be nice to have the AS bots testing this too |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The conclusion was we will upgrade the AS bots on Green Dragon and only then land this PR. We do want coverage for these since they usually expose Clang modules related issues.
Alright, that sounds good and alleviates my concern. Thank you both!
#144913 was reverted because some of the Darwin tests were failing on SDKs prior to
15.0
.Only the x86 bots run on macOS
14.0
. The aarch64 ones run on macOS15.0
.In this patch, we deactivate the failing Darwin tests on older SDKs (prior to
15.0
).