Open
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for basic for-loop semantics by introducing LoopStart and LoopEnd operators, new iteration control messages, and the necessary execution and RPC plumbing in both Scala and Python runtimes.
- Introduced LoopStart/LoopEnd executors and descriptors to buffer and replay tuples across iterations
- Defined EndIteration and NextIteration control messages and integrated them into DataProcessor, OutputManager, and RPC handlers
- Updated Python runner, protobuf definitions, and internal models to handle iteration lifecycle
Reviewed Changes
Copilot reviewed 25 out of 28 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/sleep/SleepOpExec.scala | Added Sleep operator executor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/sleep/SleepOpDesc.scala | Defined Sleep operator descriptor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopStartOpExec.scala | Implemented LoopStart executor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopStartOpDesc.scala | Defined LoopStart descriptor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopEndOpExec.scala | Implemented LoopEnd executor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopEndOpDesc.scala | Defined LoopEnd descriptor |
| core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/LogicalOp.scala | Imported new loop and sleep operators |
| core/workflow-core/src/main/scala/edu/uci/ics/amber/core/tuple/TupleLike.scala | Added FinalizeIteration marker |
| core/workflow-core/src/main/scala/edu/uci/ics/amber/core/executor/OperatorExecutor.scala | Added reset lifecycle method |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/promisehandlers/StartChannelHandler.scala | Refactored start-channel handling |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/promisehandlers/NextIterationHandler.scala | Added NextIteration RPC handler |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/promisehandlers/EndIterationHandler.scala | Added EndIteration RPC handler |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/promisehandlers/EndChannelHandler.scala | Updated end-channel to finalize loops |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/DataProcessorRPCHandlerInitializer.scala | Registered iteration handlers |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/worker/DataProcessor.scala | Integrated iteration control flow and executor reset |
| core/amber/src/main/scala/edu/uci/ics/amber/engine/architecture/messaginglayer/OutputManager.scala | Added finalizeIteration API |
| core/amber/src/main/python/proto/edu/uci/ics/amber/engine/architecture/rpc/init.py | Extended Python protobuf with EndIteration |
| core/amber/src/main/python/core/runnables/main_loop.py | Handled EndIteration in main loop |
| core/amber/src/main/python/core/runnables/data_processor.py | Treated EndIteration as InternalMarker |
| core/amber/src/main/python/core/models/operator.py | Cleared table buffer in on_finish |
| core/amber/src/main/python/core/models/internal_marker.py | Added EndIteration marker class |
| core/amber/src/main/python/core/architecture/rpc/async_rpc_handler_initializer.py | Registered Python EndIterationHandler |
| core/amber/src/main/python/core/architecture/handlers/control/end_iteration_handler.py | Implemented Python EndIteration handler |
| core/amber/src/main/protobuf/edu/uci/ics/amber/engine/architecture/rpc/workerservice.proto | Added EndIteration/NextIteration RPCs |
| core/amber/src/main/protobuf/edu/uci/ics/amber/engine/architecture/rpc/controlcommands.proto | Added EndIterationRequest message |
Comments suppressed due to low confidence (1)
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/sleep/SleepOpDesc.scala:34
- [nitpick] The JSON schema title
"n"is ambiguous. Consider using a more descriptive title like"time"or"duration"to clarify the property purpose.
@JsonSchemaTitle("n")
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopStartOpExec.scala
Show resolved
Hide resolved
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopStartOpDesc.scala
Show resolved
Hide resolved
core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/loop/LoopEndOpDesc.scala
Show resolved
Hide resolved
...cala/edu/uci/ics/amber/engine/architecture/worker/promisehandlers/NextIterationHandler.scala
Show resolved
Hide resolved
core/amber/src/main/python/core/architecture/handlers/control/end_iteration_handler.py
Outdated
Show resolved
Hide resolved
…or/loop/LoopStartOpDesc.scala Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main changes for the
Basic For Loop:LoopStartOperatorLoopEndOperatorIterationEndNextIterationLife Cycle of

Basic For Loop: