Skip to content

[DX-2884, DX-2901, DX-2911] hybrid local cre env#21315

Closed
Tofel wants to merge 29 commits intodevelopfrom
dx-2884-dx-2901-dx-2911-hybrid-local-cre-env
Closed

[DX-2884, DX-2901, DX-2911] hybrid local cre env#21315
Tofel wants to merge 29 commits intodevelopfrom
dx-2884-dx-2901-dx-2911-hybrid-local-cre-env

Conversation

@Tofel
Copy link
Contributor

@Tofel Tofel commented Feb 26, 2026

Summary

This PR delivers the full CRE hybrid environment: components can run locally (Docker) or remotely (remote Docker agent on EC2), including mixed-mode topologies.
Beyond feature enablement, it also hardens and simplifies the architecture by removing obsolete transport paths, consolidating remote execution code, improving runtime/config resolution, and expanding test coverage.

Scope of changes

  • Hybrid execution model

    • Introduced placement-aware orchestration (local / remote) for blockchains, JD, and nodesets.
    • Added remote component lifecycle management through a dedicated remote agent.
    • Enabled mixed deployments where local and remote components interoperate in one CRE setup.
  • Remote execution architecture

    • Consolidated remote execution logic under a dedicated remoteexec package tree (agent + client + orchestration helpers).
    • Added descriptor-driven remote startup flow to reduce repeated orchestration code and improve cohesion.
    • Simplified remote runtime resolution and client initialization flow.
  • Networking and mixed-mode correctness

    • Implemented/normalized host and endpoint rewriting for cross-placement communication.
    • Improved P2P/bootstrapping and gateway URL handling for local<->remote paths.
    • Added fixture relay capabilities critical for remote->local connectivity in tests.
    • Fixed reconstructed-state gateway incoming host behavior for mixed mode.
  • Remote agent operations and debuggability

    • Added agent introspection endpoints and client support:
      • status
      • lock visibility
      • component log retrieval
    • Improved relay visibility with richer relay metadata for troubleshooting.
    • Added compatibility/versioning guardrails (capability/protocol oriented).
  • Simplification and cleanup

    • Removed SSM mode and related tunnel-era abstractions.
    • Removed local-agent legacy assumptions and dead code paths.
    • Centralized execution planning/normalization and address rewrite helpers.
    • Removed no-op/unused pieces and reduced cross-file coupling.
  • Configuration and naming cleanup

    • Renamed EC2-centric env vars to generic remote naming:
      • CRE_REMOTE_AGENT_URL
      • CRE_REMOTE_AGENT_PORT
      • CRE_REMOTE_HOST_IP
      • CRE_REMOTE_AGENT_EC2_INSTANCE_ID
    • Removed CRE_AWS_PROFILE; standard AWS env resolution applies where EC2 discovery is used.
    • Ensured non-EC2 explicit remote URL/host paths do not require AWS-related env vars.
  • Docs and UX

    • Updated runbooks and hybrid troubleshooting documentation.
    • Added/updated remote debugging skills/docs.
    • Added topology/placement visualization improvements for remote-aware startup context.

Testing

  • Expanded unit coverage for:
    • remote client/runtime resolution
    • remote stop/artifact flows
    • agent server handlers/state/logs/relay logic
    • environment placement planning and normalization
    • DON config and mixed-mode address resolution
  • Standardized assertions on testify/require.
  • Added high/medium ROI tests for critical relay and deployment seams.
  • Validated with package-level tests, lint gates, and mixed-mode e2e smoke runs.

Outcome

The resulting hybrid CRE environment is functional, more maintainable, and easier to debug: remote execution is cohesive, mixed-mode behavior is explicit and tested, and legacy complexity has been removed.

Tofel added 27 commits February 16, 2026 17:06
@Tofel Tofel force-pushed the dx-2884-dx-2901-dx-2911-hybrid-local-cre-env branch from 67c5abf to 130c895 Compare February 26, 2026 08:52
# Conflicts:
#	core/scripts/go.mod
#	core/scripts/go.sum
#	system-tests/lib/cre/contracts/keystone_test.go
#	system-tests/lib/cre/don.go
#	system-tests/lib/go.sum
#	system-tests/tests/go.sum
#	system-tests/tests/smoke/cre/cre_suite_test.go
@github-actions
Copy link
Contributor

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

  • #added For any new functionality added.
  • #breaking_change For any functionality that requires manual action for the node to boot.
  • #bugfix For bug fixes.
  • #changed For any change to the existing functionality.
  • #db_update For any feature that introduces updates to database schema.
  • #deprecation_notice For any upcoming deprecation functionality.
  • #internal For changesets that need to be excluded from the final changelog.
  • #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
  • #removed For any functionality/config that is removed.
  • #updated For any functionality that is updated.
  • #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

@github-actions
Copy link
Contributor

✅ No conflicts with other open PRs targeting develop

@cl-sonarqube-production
Copy link

@trunk-io
Copy link

trunk-io bot commented Feb 26, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@Tofel
Copy link
Contributor Author

Tofel commented Mar 3, 2026

Superseded by #21318

@Tofel Tofel closed this Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant