[libclang/python] Add missing enum variants

[DeinAlptraum]

Add tests to ensure that all C-enum variants are defined on Python side.

[llvmbot]

@llvm/pr-subscribers-clang

Author: Jannick Kremer (DeinAlptraum)

Changes

Add tests to ensure that all C-enum variants are defined on Python side.

---

Full diff: https://github.com/llvm/llvm-project/pull/143264.diff

1 Files Affected:

• (modified) clang/bindings/python/tests/cindex/test_enums.py (+48-1)

diff --git a/clang/bindings/python/tests/cindex/test_enums.py b/clang/bindings/python/tests/cindex/test_enums.py
index 9e7f44fcf7867..54c0c77e4df29 100644
--- a/clang/bindings/python/tests/cindex/test_enums.py
+++ b/clang/bindings/python/tests/cindex/test_enums.py
@@ -1,4 +1,5 @@
 import unittest
+from pathlib import Path
 
 from clang.cindex import (
     AccessSpecifier,
@@ -12,6 +13,7 @@
     TemplateArgumentKind,
     TLSKind,
     TokenKind,
+    TranslationUnit,
     TypeKind,
 )
 
@@ -44,8 +46,53 @@ def test_from_id(self):
 
     def test_duplicate_ids(self):
         """Check that no two kinds have the same id"""
-        # for enum in self.enums:
         for enum in self.enums:
             num_declared_variants = len(enum._member_map_.keys())
             num_unique_variants = len(list(enum))
             self.assertEqual(num_declared_variants, num_unique_variants)
+
+    def test_all_variants(self):
+        """Check that all libclang enum values are also defined in cindex"""
+        cenum_to_pythonenum = {
+            "CX_CXXAccessSpecifier": AccessSpecifier,
+            "CXAvailabilityKind": AvailabilityKind,
+            "CXBinaryOperatorKind": BinaryOperator,
+            "CXCursorKind": CursorKind,
+            "CXCursor_ExceptionSpecificationKind": ExceptionSpecificationKind,
+            "CXLinkageKind": LinkageKind,
+            "CXRefQualifierKind": RefQualifierKind,
+            "CX_StorageClass": StorageClass,
+            "CXTemplateArgumentKind": TemplateArgumentKind,
+            "CXTLSKind": TLSKind,
+            "CXTokenKind": TokenKind,
+            "CXTypeKind": TypeKind,
+        }
+
+        indexheader = (
+            Path(__file__).parent.parent.parent.parent.parent
+            / "include/clang-c/Index.h"
+        )
+        tu = TranslationUnit.from_source(indexheader, ["-x", "c++"])
+
+        enum_variant_map = {}
+        # For all enums in self.enums, extract all enum variants defined in Index.h
+        for cursor in tu.cursor.walk_preorder():
+            type_class = cenum_to_pythonenum.get(cursor.type.spelling)
+            if (
+                cursor.kind == CursorKind.ENUM_CONSTANT_DECL
+                and type_class in self.enums
+            ):
+                if type_class not in enum_variant_map:
+                    enum_variant_map[type_class] = []
+                enum_variant_map[type_class].append(cursor.enum_value)
+
+        for enum in self.enums:
+            with self.subTest(enum):
+                python_kinds = set([kind.value for kind in enum])
+                c_kinds = set(enum_variant_map[enum])
+                missing_python_kinds = c_kinds - python_kinds
+                self.assertEqual(
+                    missing_python_kinds,
+                    set(),
+                    f"Please ensure these variants are defined inside {enum} in cindex.py.",
+                )

[DeinAlptraum]

@Endilll this adds the missing enum variants, and also adds a test using libclang, parsing Index.h for all our used enums to check for missing variants. You can see what this looks like in case of failure on the first CI run for this PR.

This uses only the numeric enum values for comparisons since this is easier to implement (especially because some variants have aliases on the C-side)

Sidenote: I've noticed that a few enum variants have inconsistent namings, e.g. all Python enum variants are fully uppercase, execpt for StmtExpr. There's also CXCursor_CompoundAssignOperator which corresponds to COMPOUND_ASSIGNMENT_OPERATOR, i.e. slightly different names. I wonder if we should adapt these, but it might not be worth the breaking change.

This is probably by far our slowest test, responsible for about 20% of the entire test suite's runtime, but since all tests together take like 500ms that's acceptable imo