Skip to content

CAMEL-24373: camel-alibaba: Enhance EventBridge validation, multi-bus DSL, and documentation - #26010

Merged
davsclaus merged 6 commits into
apache:mainfrom
arunsrajan:feature/CAMEL-24373-alibaba-event-bridge-cache
Sep 2, 2026
Merged

CAMEL-24373: camel-alibaba: Enhance EventBridge validation, multi-bus DSL, and documentation#26010
davsclaus merged 6 commits into
apache:mainfrom
arunsrajan:feature/CAMEL-24373-alibaba-event-bridge-cache

Conversation

@arunsrajan

Copy link
Copy Markdown
Contributor

Description

This pull request enhances the 'camel-alibaba-eventbridge' component by introducing hierarchical event validation, an in-memory TTL event source cache, multi-bus and single-bus DSL configuration for allowed event sources, and comprehensive documentation updates across 'camel-alibaba-eventbridge' and 'camel-catalog'.

Fixes https://issues.apache.org/jira/browse/CAMEL-24373

Motivation

When publishing events via Alibaba Cloud EventBridge, applications often need to:

  1. Validate event buses, sources, and event types against Alibaba Cloud rules prior to publishing to avoid runtime submission failures.
  2. Enforce strict static whitelists ('allowedEventSources') for single-bus and multi-bus publishing routes.
  3. Cache validated cloud metadata with configurable TTL to reduce redundant Alibaba Cloud API roundtrips.
  4. Support flexible configuration formats (single-bus shorthand DSL, multi-bus DSL, JSON string, Java models).

Modifications:

1. 'camel-alibaba-eventbridge':

  • Hierarchical Validation:
    • Implemented 'EventSourceCache' providing validated caching of event buses, registered event sources, and rule filter pattern event types using Alibaba Cloud APIs ('listEventBuses', 'listRules') with configurable TTL ('eventSourceCacheTtl').
    • Added 'MapCloudEventValidator' to validate and convert 'Map' payloads conforming to both Alibaba EventBridge dictionary keys and standard CloudEvents 1.0 specifications.
  • Allowed Event Sources Whitelisting:
    • Added 'AllowedEventBus' and 'AllowedEventSource' models.
    • Implemented parsing in 'AlibabaEventBridgeUtils' for:
      • Single-Bus shorthand DSL: 'source1 -> type1, type2; source2 -> type3'
      • Multi-Bus DSL: 'bus1[source1 -> type1, type2; source2 -> type3] | bus2[source3 -> type4]'
      • JSON configuration: Hierarchical multi-bus JSON and single-bus flat JSON structures.
      • Java Collections / Models: 'List', 'List', 'Map<String, ...>'.
  • Component & Endpoint Options:
    • Added '@UriParam' options and corresponding headers/properties: 'validateEventSource', 'validateEventType', 'validateEventSpec', 'allowedEventSources', 'eventSourceCacheTtl'.
    • Added response metadata handling in the producer body ('requestId', 'failedEntryCount', 'entryList').
  • Testing:
    • Added unit test suite 'MapCloudEventValidationTest' (12 test cases) covering single-bus DSL, multi-bus DSL, JSON configurations, spec validation, and cache behavior.
    • Updated 'PutEventsTest' with test cases for multi-bus and header-based overrides.

2. 'camel-alibaba-common' & other Alibaba modules:

  • Moved common helper methods 'resolveBoolean' and 'resolveLong' into 'OpenApiClientSupport' in 'camel-alibaba-common'.
  • Cleaned up client configuration resolution across 'camel-alibaba-oss', 'camel-alibaba-mns', 'camel-alibaba-fc', 'camel-alibaba-sms', and 'camel-alibaba-kms'.

3. Documentation:

  • Updated 'alibaba-eventbridge-component.adoc' with detailed descriptions for:
    • Cloud validation and caching workflow.
    • Single-bus and multi-bus DSL syntax, grammar rules, and components.
    • JSON configuration examples.
    • Java DSL routing examples.
  • Escaped pipe characters ('|') inside the AsciiDoc table to fix cell fragmentation.
  • Regenerated and synchronized catalog documentation in 'catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/alibaba-eventbridge-component.adoc'.

Co-authored-by: Antigravity antigravity@google.com

DSL, and documentation

- Implement 3-level hierarchical event validation (EventBus ->
EventSource -> EventType) in camel-alibaba-eventbridge.
- Introduce EventSourceCache for Alibaba Cloud metadata caching with
configurable TTL.
- Add support for allowedEventSources configuration across single-bus
shorthand DSL, multi-bus DSL, JSON string schema, and Java records
(AllowedEventBus, AllowedEventSource).
- Add MapCloudEventValidator to enforce CloudEvents 1.0 specification
constraints and map Alibaba dictionary keys.
- Unify ClientConfigurations and credential resolution across
camel-alibaba-fc, camel-alibaba-kms, camel-alibaba-mns,
camel-alibaba-oss, and camel-alibaba-sms.
- Update component and catalog documentation with single-bus, multi-bus,
and JSON configuration examples.
- Add unit tests in MapCloudEventValidationTest with 100% pass rate.

Co-authored-by: Antigravity <antigravity@google.com>
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@Croway

Croway commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Hi @arunsrajan thanks for the PR! There are uncommitted changes, and I think this PR need to be rebased on main

There are uncommitted changes
HEAD detached at pull/26010/merge
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/alibaba-eventbridge.json
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/AlibabaEventBridgeEndpointBuilderFactory.java

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

✅ Generated files are up to date

An earlier CI run reported uncommitted generated changes; the latest run no longer does.

@davsclaus

Copy link
Copy Markdown
Contributor

@Croway here is a PR to try your new feature ^^^

@Croway

Croway commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

@Croway here is a PR to try your new feature ^^^

Done! as reported by the bot I've just executed

gh pr checkout 26010 --repo apache/camel
gh run download 33506881502 --repo apache/camel -n regen-patch
git apply --index regen.patch && rm regen.patch pr-number
git commit -m "Regen" && git push

and 43999d8 was applied, seems like it is working

@github-actions github-actions Bot added the dsl label Sep 1, 2026
@davsclaus

Copy link
Copy Markdown
Contributor

[15:11:46.267] WARN (asciidoctor): list item index: expected 2, got 1
file: /home/runner/work/camel/camel/camel/docs/components/modules/ROOT/pages/alibaba-eventbridge-component.adoc:140
source: /home/runner/work/camel/camel/camel (branch: HEAD | start path: docs/components)
[15:11:46.269] WARN (asciidoctor): list item index: expected 3, got 2
file: /home/runner/work/camel/camel/camel/docs/components/modules/ROOT/pages/alibaba-eventbridge-component.adoc:141
source: /home/runner/work/camel/camel/camel (branch: HEAD | start path: docs/components)
[15:11:46.269] WARN (asciidoctor): list item index: expected 4, got 3
file: /home/runner/work/camel/camel/camel/docs/components/modules/ROOT/pages/alibaba-eventbridge-component.adoc:142
source: /home/runner/work/camel/camel/camel (branch: HEAD | start path: docs/components)
[15:11:46.270] WARN (asciidoctor): list item index: expected 5, got 2
file: /home/runner/work/camel/camel/camel/docs/components/modules/ROOT/pages/alibaba-eventbridge-component.adoc:145
source: /home/runner/work/camel/camel/camel (branch: HEAD | start path: docs/components)

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • catalog/camel-catalog
  • components/camel-alibaba/camel-alibaba-common
  • components/camel-alibaba/camel-alibaba-eventbridge
  • components/camel-alibaba/camel-alibaba-fc
  • components/camel-alibaba/camel-alibaba-kms
  • components/camel-alibaba/camel-alibaba-mns
  • components/camel-alibaba/camel-alibaba-oss
  • components/camel-alibaba/camel-alibaba-ots
  • components/camel-alibaba/camel-alibaba-sls
  • components/camel-alibaba/camel-alibaba-sms
  • docs
  • dsl/camel-endpointdsl

🔬 Scalpel shadow comparison — Scalpel: 20 tested, 24 compile-only — current: 17 all tested

Maveniverse Scalpel detected 44 affected modules (current approach: 17).

⚠️ Modules only in Scalpel (27)
  • apache-camel
  • camel-allcomponents
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

Skip-tests mode would test 20 modules (12 direct + 8 downstream), skip tests for 24 (generated code, meta-modules)

Modules Scalpel would test (20)
  • camel-alibaba-common
  • camel-alibaba-eventbridge
  • camel-alibaba-fc
  • camel-alibaba-kms
  • camel-alibaba-mns
  • camel-alibaba-oss
  • camel-alibaba-ots
  • camel-alibaba-sls
  • camel-alibaba-sms
  • camel-catalog
  • camel-endpointdsl
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-launcher-container
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin
  • docs
Modules with tests skipped (24)
  • apache-camel
  • camel-allcomponents
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (44 modules)
  • Camel :: Alibaba Cloud :: Common
  • Camel :: Alibaba Cloud :: EventBridge
  • Camel :: Alibaba Cloud :: Function Compute (FC)
  • Camel :: Alibaba Cloud :: Key Management Service (KMS)
  • Camel :: Alibaba Cloud :: Message Service (MNS)
  • Camel :: Alibaba Cloud :: OSS
  • Camel :: Alibaba Cloud :: Short Message Service (SMS)
  • Camel :: Alibaba Cloud :: Simple Log Service (SLS)
  • Camel :: Alibaba Cloud :: Tablestore (OTS)
  • Camel :: All Components Sync point
  • Camel :: Assembly
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Catalog :: Camel Report Maven Plugin
  • Camel :: Catalog :: Camel Route Parser
  • Camel :: Catalog :: Console
  • Camel :: Catalog :: Dummy Component
  • Camel :: Catalog :: Maven
  • Camel :: Catalog :: Suggest
  • Camel :: Component DSL
  • Camel :: Coverage
  • Camel :: Docs
  • Camel :: Endpoint DSL
  • Camel :: Endpoint DSL :: Support
  • Camel :: Integration Tests
  • Camel :: JBang :: Core
  • Camel :: JBang :: Integration tests
  • Camel :: JBang :: MCP
  • Camel :: JBang :: Main
  • Camel :: JBang :: Plugin :: Edit
  • Camel :: JBang :: Plugin :: Generate
  • Camel :: JBang :: Plugin :: Kubernetes
  • Camel :: JBang :: Plugin :: MCP
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: TUI
  • Camel :: JBang :: Plugin :: Testing
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: Kamelet Main
  • Camel :: Launcher
  • Camel :: Launcher :: Container
  • Camel :: YAML DSL
  • Camel :: YAML DSL :: Deserializers
  • Camel :: YAML DSL :: Maven Plugins
  • Camel :: YAML DSL :: Validator
  • Camel :: YAML DSL :: Validator Maven Plugin

⚙️ View full build and test results

@davsclaus

Copy link
Copy Markdown
Contributor

there are some doc validation errors that must be fixed

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Code review — CAMEL-24373 EventBridge validation (Bugbot + security + test coverage)

AI-generated review on behalf of @atiaomar1978-hub

Thanks @arunsrajan for a substantial improvement to camel-alibaba-eventbridge. The 3-level validation model, multi-bus allowedEventSources DSL, and MapCloudEventValidator are well thought out. Java CI is green (JDK 17/25); docs CI is red and there are a few correctness/security items to address before merge.


Verdict: Request changes (approve in principle once blockers are fixed)


CI status

Check Result
Build and test (JDK 17/25) ✅ Pass
PR doc validation (build) ❌ Fail — AsciiDoc nested ordered-list index warnings at alibaba-eventbridge-component.adoc lines ~140–145
Generated files ✅ Up to date (after regen commit)

Bugbot findings (correctness)

  1. High — fail-open cloud validation (EventSourceCache): On listEventBuses / listRules API failure, existence checks return true and empty rule metadata makes isKnownSource / isKnownType return true. Validation appears enabled but is bypassed during outages. Prefer fail-closed when validateEventSource / validateEventType is true, or document explicitly.
  2. Medium — cache TTL header ignored (AlibabaEventBridgeUtilsEventSourceCache): EVENT_SOURCE_CACHE_TTL is resolved per exchange but the cache is created once at endpoint start from the URI option only.
  3. Medium — prefix rule mismatch (EventSourceCache.parseFilterPattern): Alibaba prefix filters are stored as literal strings; runtime checks use exact Set.contains, so prefix-matched event types may be wrongly rejected.

Security review

Per Camel's security model, route authors are trusted; untrusted ingress is the main concern.

Area Assessment
Fail-open validation Medium — policy bypass during API errors (see above)
Header overrides Medium — ALLOWED_EVENT_SOURCES and validation toggles can be overridden via headers/properties. Fine for trusted routes; risky if used as ingress policy without stripping headers
Cloud API calls Low — read-only metadata; no new deserialization surface
Map body validation Low — rejects malformed CloudEvents; whitelist enforced when configured

Recommendation: Document fail-open behaviour and header override semantics in the component doc. Add an upgrade-guide entry for validateEventSpec=true default (4.23).


Test coverage

Present (good): MapCloudEventValidationTest (12 tests) covers DSL/JSON parsing, whitelist enforcement, CloudEvents spec constraints, and multi-bus configs. PutEventsTest adds 5 integration-style cases.

Gaps to consider:

  • CloudEvent body passthrough (validateCloudEvent on non-Map bodies)
  • API failure behaviour when validateEventSource=true (mock client throws → should fail closed?)
  • Whitelist bypass via map body fields vs endpoint config
  • Prefix filter matching in EventSourceCache
  • Per-message cache TTL header override
  • Regression tests for OpenApiClientSupport refactor across FC/KMS/MNS/OSS/SMS (scope is wider than EventBridge)

Scope note

The OpenApiClientSupport extraction touches multiple Alibaba components. Please confirm behaviour is unchanged and note cross-component impact in the PR description.


Required before merge

  1. Fix docs CI (AsciiDoc list nesting under “Event Validation and Caching”)
  2. Decide fail-open vs fail-closed for cloud validation and align code + docs
  3. Add upgrade-guide entry for new/changed defaults (validateEventSpec, validation toggles)
  4. Fix or document cache TTL header behaviour and prefix filter semantics

Inline comments

Posted 10 review threads on: EventSourceCache, AlibabaEventBridgeUtils, AlibabaEventBridgeEndpoint, component docs, tests, and OpenApiClientSupport.

Happy to re-review once the docs fix and validation semantics are addressed.

arunsrajan and others added 2 commits September 2, 2026 16:19
…w feedback

Special thanks to the reviewer @atiaomar1978-hub and the Apache Camel committers for the thorough review and valuable suggestions.

- Fail-closed Cloud Validation: Updated EventSourceCache to fail-closed on Alibaba Cloud API connectivity/permission errors and missing metadata.
- Prefix Rule Matching: Added prefix pattern matching ({"prefix": "..."}) support for event sources and event types in Alibaba Cloud rule filter patterns.
- Dynamic Cache TTL: Passed effective per-message TTL from ClientConfigurations to EventSourceCache to support per-exchange TTL header overrides.
- Direct CloudEvent Validation: Added validateCloudEvent(...) for direct CloudEvent object whitelist validation.
- Documentation & Upgrade Guide: Added untrusted ingress security guidance, fixed AsciiDoc list formatting, synced catalog docs, and added camel-alibaba-eventbridge section to the 4.23 upgrade guide.
- Regenerated Artifacts: Regenerated and committed catalog JSON and Endpoint DSL Java builder (AlibabaEventBridgeEndpointBuilderFactory).
- Test Coverage: Added unit tests covering fail-closed API failures, prefix rule matching, TTL overrides, and non-standard spec bypass. Verified test suites across all Alibaba modules (EventBridge, FC, KMS, MNS, OSS, SMS).

Co-authored-by: Antigravity <noreply@google.com>
…rror

- Synchronize catalog documentation mirror with component documentation.
- Ensures all AsciiDoc formatting updates and security guidelines are mirrored in camel-catalog.

Co-authored-by: Antigravity <noreply@google.com>
@davsclaus

Copy link
Copy Markdown
Contributor

can you fix the last upgrade guide merge conflict, thanks

@arunsrajan

Copy link
Copy Markdown
Contributor Author

@davsclaus I have fixed the merge conflicts, thanks.

@davsclaus

Copy link
Copy Markdown
Contributor

thanks

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work on this comprehensive enhancement — the hierarchical event validation with caching, the fail-closed validation semantics, and the 19 test cases in MapCloudEventValidationTest are well done. The upgrade guide entry for validateEventSpec defaulting to true is also appreciated.

A few issues to consider (3 confirmed findings after independent verification, 4 initially flagged items were determined to be false positives):

📋 PR Metadata

Aspect Current Suggested
Milestone (none) 4.23.0
Labels components, catalog, docs, dsl + enhancement

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of Guillaume Nodet

…rom @gnodet

Address PR apache#26010 review feedback from @gnodet:
- ServiceKeys Compatibility: Revert ServiceKeys from a Java record back to
  a mutable class with getters/setters and default/all-args constructors.
  This preserves PropertyBindingSupport nested property binding (e.g.
  serviceKeys.accessKey=xxx) and Spring Boot auto-configuration across all
  8 Alibaba components.
- Credential Resolution: Update OpenApiClientSupport, OSSUtils, and MNSUtils
  to use getAccessKey() and getSecretKey().
- DSL Colon Fallback Fix: Remove colon-based fallback in
  AlibabaEventBridgeUtils.parseSourcesBlock to ensure source URIs containing
  scheme colons or port numbers (e.g. http://example.com:8080/events,
  urn:custom:source) are not inadvertently split. Mapping to event types is
  now performed exclusively via -> and =.
- Unit Tests: Add testSingleBusDslWithHttpUriAndPort and
  testSingleBusDslWithUrnAndEquals in MapCloudEventValidationTest; resolve
  type inference in SLS and OTS test assertions.
- Documentation & Catalog: Update alibaba-eventbridge-component.adoc and
  synchronize the catalog documentation mirror in camel-catalog.

Co-authored-by: Antigravity <noreply@google.com>
@davsclaus davsclaus added this to the 4.23.0 milestone Sep 2, 2026
@davsclaus davsclaus added the enhancement New feature or request label Sep 2, 2026
@davsclaus
davsclaus merged commit b9c1f53 into apache:main Sep 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants