Skip to content

[CAPPL-400] Fix/truncate workflow names #2 #15896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jan 14, 2025
Merged

[CAPPL-400] Fix/truncate workflow names #2 #15896

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
3a31b67
fix(workflows/syncer): truncate workflow name
MStreet3 Jan 8, 2025
84aa723
chore: changeset patch
MStreet3 Jan 8, 2025
e14da5b
(refactor): Move truncated hashed workflow name into engine. Ensure U…
justinkaseman Jan 10, 2025
f5de941
(refactor): injectable name transform
justinkaseman Jan 13, 2025
4a06b24
Bump common to commit 42c3764c171e870bfd91443c6ae82a6e76bc6f1f
justinkaseman Jan 13, 2025
fa2079a
Use HashTruncateName in workflow registry syncer handler
justinkaseman Jan 13, 2025
33c3c38
Remove comment line
justinkaseman Jan 13, 2025
5a5d637
Merge branch 'develop' into fix/truncate-workflow-names-2
justinkaseman Jan 13, 2025
394d2e8
go mod tidy
justinkaseman Jan 13, 2025
38aeade
Changes from review
justinkaseman Jan 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/wild-planes-mix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

Truncates workflow name before starting engine
2 changes: 1 addition & 1 deletion core/scripts/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ require (
github.com/prometheus/client_golang v1.20.5
github.com/shopspring/decimal v1.4.0
github.com/smartcontractkit/chainlink-automation v0.8.1
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.13
github.com/smartcontractkit/libocr v0.0.0-20241223215956-e5b78d8e3919
Expand Down
4 changes: 2 additions & 2 deletions core/scripts/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1164,8 +1164,8 @@ github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103 h1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103/go.mod h1:ncjd6mPZSRlelEqH/2KeLE1pU3UlqzBSn8RYkEoECzY=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b h1:UBXi9Yj8YSMHDDaxQLu273x1fWjyEL9xP58nuJsqZfg=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b/go.mod h1:Bmwq4lNb5tE47sydN0TKetcLEGbgl+VxHEWp4S0LI60=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e h1:8BStiP1F4W8AvjBRga0TYtjvAtkwN8oHYnHJztAlSF4=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e h1:l1npEX9O1Y1R4ss6yNPlggxFOdhPWmvD3tIMZkOzuDU=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e h1:PRoeby6ZlTuTkv2f+7tVU4+zboTfRzI+beECynF4JQ0=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e/go.mod h1:mUh5/woemsVaHgTorA080hrYmO3syBCmPdnWc/5dOqk=
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3 h1:aeiBdBHGY8QNftps+VqrIk6OnfeeOD5z4jrAabW4ZSc=
Expand Down
37 changes: 21 additions & 16 deletions core/services/workflows/engine.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1191,22 +1191,23 @@ func (e *Engine) Name() string {
}

type Config struct {
Workflow sdk.WorkflowSpec
WorkflowID string
WorkflowOwner string
WorkflowName string
Lggr logger.Logger
Registry core.CapabilitiesRegistry
MaxWorkerLimit int
QueueSize int
NewWorkerTimeout time.Duration
MaxExecutionDuration time.Duration
Store store.Store
Config []byte
Binary []byte
SecretsFetcher secretsFetcher
HeartbeatCadence time.Duration
StepTimeout time.Duration
Workflow sdk.WorkflowSpec
WorkflowID string
WorkflowOwner string
WorkflowName string // Full human-readable workflow name. Intended for metrics and logging.
WorkflowNameTransform string // The Workflow Name in an on-chain format, which has requirements of being hex encoded and max 10 bytes
Lggr logger.Logger
Registry core.CapabilitiesRegistry
MaxWorkerLimit int
QueueSize int
NewWorkerTimeout time.Duration
MaxExecutionDuration time.Duration
Store store.Store
Config []byte
Binary []byte
SecretsFetcher secretsFetcher
HeartbeatCadence time.Duration
StepTimeout time.Duration

// For testing purposes only
maxRetries int
Expand Down Expand Up @@ -1300,6 +1301,10 @@ func NewEngine(ctx context.Context, cfg Config) (engine *Engine, err error) {
workflow.hexName = hex.EncodeToString([]byte(cfg.WorkflowName))
Copy link
Contributor

@MStreet3 MStreet3 Jan 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it simpler to not add the WorkflowNameTransform field (it's public and could get messed with in theory), not change the syncer and instead just do:

workflow.hexName = pkgworkflows.HashTruncateName(cfg.WorkflowName)

otherwise, I'm no longer a fan of the name and not a fan of something that has to be a fixed length being a public field (in hindsight having a public function be passed was no guarantee of hexName length either)

Copy link
Contributor Author

@justinkaseman justinkaseman Jan 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what I originally tried to do, but as @cedric-cordenier caught here that would cause a breaking change for the existing Keystone feeds

workflow.name = cfg.WorkflowName

if len(cfg.WorkflowNameTransform) > 0 {
workflow.hexName = cfg.WorkflowNameTransform
}

engine = &Engine{
cma: cma,
logger: cfg.Lggr.Named("WorkflowEngine").With("workflowID", cfg.WorkflowID),
Expand Down
24 changes: 14 additions & 10 deletions core/services/workflows/syncer/handler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,16 +533,20 @@ func (h *eventHandler) engineFactoryFn(ctx context.Context, id string, owner str
}

cfg := workflows.Config{
Lggr: h.lggr,
Workflow: *sdkSpec,
WorkflowID: id,
WorkflowOwner: owner, // this gets hex encoded in the engine.
WorkflowName: name,
Registry: h.capRegistry,
Store: h.workflowStore,
Config: config,
Binary: binary,
SecretsFetcher: h,
Lggr: h.lggr,
Workflow: *sdkSpec,
WorkflowID: id,
WorkflowOwner: owner, // this gets hex encoded in the engine.
WorkflowName: name,
// Internal workflow names must not exceed 10 bytes for workflow engine and on-chain use.
// A name is used internally that is first hashed to avoid collisions,
// hex encoded to ensure UTF8 encoding, then truncated to 10 bytes.
WorkflowNameTransform: pkgworkflows.HashTruncateName(name),
Registry: h.capRegistry,
Store: h.workflowStore,
Config: config,
Binary: binary,
SecretsFetcher: h,
}
return workflows.NewEngine(ctx, cfg)
}
Expand Down
2 changes: 1 addition & 1 deletion deployment/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ require (
github.com/smartcontractkit/chain-selectors v1.0.36
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e
github.com/smartcontractkit/chainlink-protos/job-distributor v0.6.0
github.com/smartcontractkit/chainlink-solana v1.1.1-0.20250110142550-e2a9566d39f3
github.com/smartcontractkit/chainlink-testing-framework/framework v0.4.2-0.20250110073248-456673e8eea2
Expand Down
4 changes: 2 additions & 2 deletions deployment/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1390,8 +1390,8 @@ github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103 h1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103/go.mod h1:ncjd6mPZSRlelEqH/2KeLE1pU3UlqzBSn8RYkEoECzY=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b h1:UBXi9Yj8YSMHDDaxQLu273x1fWjyEL9xP58nuJsqZfg=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b/go.mod h1:Bmwq4lNb5tE47sydN0TKetcLEGbgl+VxHEWp4S0LI60=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e h1:8BStiP1F4W8AvjBRga0TYtjvAtkwN8oHYnHJztAlSF4=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e h1:l1npEX9O1Y1R4ss6yNPlggxFOdhPWmvD3tIMZkOzuDU=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e h1:PRoeby6ZlTuTkv2f+7tVU4+zboTfRzI+beECynF4JQ0=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e/go.mod h1:mUh5/woemsVaHgTorA080hrYmO3syBCmPdnWc/5dOqk=
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3 h1:aeiBdBHGY8QNftps+VqrIk6OnfeeOD5z4jrAabW4ZSc=
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ require (
github.com/smartcontractkit/chain-selectors v1.0.34
github.com/smartcontractkit/chainlink-automation v0.8.1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3
github.com/smartcontractkit/chainlink-feeds v0.1.1
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1154,8 +1154,8 @@ github.com/smartcontractkit/chainlink-automation v0.8.1 h1:sTc9LKpBvcKPc1JDYAmgB
github.com/smartcontractkit/chainlink-automation v0.8.1/go.mod h1:Iij36PvWZ6blrdC5A/nrQUBuf3MH3JvsBB9sSyc9W08=
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103 h1:TeWnxdgSO2cYCKcBMwdkRcs/YdhSvXoWqqw7QWz/KeI=
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103/go.mod h1:ncjd6mPZSRlelEqH/2KeLE1pU3UlqzBSn8RYkEoECzY=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e h1:8BStiP1F4W8AvjBRga0TYtjvAtkwN8oHYnHJztAlSF4=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e h1:l1npEX9O1Y1R4ss6yNPlggxFOdhPWmvD3tIMZkOzuDU=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e h1:PRoeby6ZlTuTkv2f+7tVU4+zboTfRzI+beECynF4JQ0=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e/go.mod h1:mUh5/woemsVaHgTorA080hrYmO3syBCmPdnWc/5dOqk=
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3 h1:aeiBdBHGY8QNftps+VqrIk6OnfeeOD5z4jrAabW4ZSc=
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ require (
github.com/smartcontractkit/chain-selectors v1.0.36
github.com/smartcontractkit/chainlink-automation v0.8.1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e
github.com/smartcontractkit/chainlink-protos/job-distributor v0.6.0
github.com/smartcontractkit/chainlink-testing-framework/havoc v1.50.2
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.19
Expand Down
4 changes: 2 additions & 2 deletions integration-tests/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1414,8 +1414,8 @@ github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103 h1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103/go.mod h1:ncjd6mPZSRlelEqH/2KeLE1pU3UlqzBSn8RYkEoECzY=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b h1:UBXi9Yj8YSMHDDaxQLu273x1fWjyEL9xP58nuJsqZfg=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b/go.mod h1:Bmwq4lNb5tE47sydN0TKetcLEGbgl+VxHEWp4S0LI60=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e h1:8BStiP1F4W8AvjBRga0TYtjvAtkwN8oHYnHJztAlSF4=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e h1:l1npEX9O1Y1R4ss6yNPlggxFOdhPWmvD3tIMZkOzuDU=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e h1:PRoeby6ZlTuTkv2f+7tVU4+zboTfRzI+beECynF4JQ0=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e/go.mod h1:mUh5/woemsVaHgTorA080hrYmO3syBCmPdnWc/5dOqk=
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3 h1:aeiBdBHGY8QNftps+VqrIk6OnfeeOD5z4jrAabW4ZSc=
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/load/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ require (
github.com/pkg/errors v0.9.1
github.com/rs/zerolog v1.33.0
github.com/slack-go/slack v0.15.0
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e
github.com/smartcontractkit/chainlink-testing-framework/lib v1.50.19
github.com/smartcontractkit/chainlink-testing-framework/seth v1.50.9
github.com/smartcontractkit/chainlink-testing-framework/wasp v1.50.2
Expand Down
4 changes: 2 additions & 2 deletions integration-tests/load/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1403,8 +1403,8 @@ github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103 h1
github.com/smartcontractkit/chainlink-ccip v0.0.0-20250110181647-9dba278f2103/go.mod h1:ncjd6mPZSRlelEqH/2KeLE1pU3UlqzBSn8RYkEoECzY=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b h1:UBXi9Yj8YSMHDDaxQLu273x1fWjyEL9xP58nuJsqZfg=
github.com/smartcontractkit/chainlink-ccip/chains/solana v0.0.0-20250103152858-8973fd0c912b/go.mod h1:Bmwq4lNb5tE47sydN0TKetcLEGbgl+VxHEWp4S0LI60=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e h1:8BStiP1F4W8AvjBRga0TYtjvAtkwN8oHYnHJztAlSF4=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250108194320-2ebd63bbb16e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e h1:l1npEX9O1Y1R4ss6yNPlggxFOdhPWmvD3tIMZkOzuDU=
github.com/smartcontractkit/chainlink-common v0.4.1-0.20250113183410-42c3764c171e/go.mod h1:yti7e1+G9hhkYhj+L5sVUULn9Bn3bBL5/AxaNqdJ5YQ=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e h1:PRoeby6ZlTuTkv2f+7tVU4+zboTfRzI+beECynF4JQ0=
github.com/smartcontractkit/chainlink-cosmos v0.5.2-0.20241202195413-82468150ac1e/go.mod h1:mUh5/woemsVaHgTorA080hrYmO3syBCmPdnWc/5dOqk=
github.com/smartcontractkit/chainlink-data-streams v0.1.1-0.20241216163550-fa030d178ba3 h1:aeiBdBHGY8QNftps+VqrIk6OnfeeOD5z4jrAabW4ZSc=
Expand Down
Loading