Skip to content

feat(content-sidebar): timeline markers for timestamped comments and annotations#4647

Open
mrscobbler wants to merge 1 commit into
box:masterfrom
mrscobbler:feature/video-timestamp-markers
Open

feat(content-sidebar): timeline markers for timestamped comments and annotations#4647
mrscobbler wants to merge 1 commit into
box:masterfrom
mrscobbler:feature/video-timestamp-markers

Conversation

@mrscobbler

@mrscobbler mrscobbler commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Activity sidebar derives video timeline markers from filteredItems (comments with timestamp markup, annotations with frame target) and pushes them to the box-content-preview viewer via a window CustomEvent
  • Marker clicks bubble back through AnnotatorContext; the existing handleAnnotationSelect flow renders the overlay and seeks the video for annotations, comment markers seek + push URL
  • handleAnnotationSelect now seeks frame annotations on the current file version first so the playhead moves before sidebar scroll and overlay render
  • No viewer instance is held on the BUIE side — the bridge is a pair of window CustomEvents owned by withAnnotations

Test plan

  • Open a video file via ContentPreview with videoPlayerV2.enabled, activityFeed.threadedRepliesV2, and activityFeed.timestampedComments all on
  • Add a timestamped comment; confirm a marker appears on the scrubber
  • Add a frame annotation; confirm a marker appears
  • Resolve a comment/annotation; confirm its marker disappears
  • Click an annotation marker; confirm video pauses, sidebar scrolls to the item, annotation overlay renders, and video seeks to the timestamp
  • Click a comment marker; confirm video pauses and seeks, sidebar scrolls
  • Click an existing badge in the activity feed (regression check); confirm seek-first behavior + overlay still works
  • Filter the feed by mentions-me; confirm markers update to match the filtered list

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added timeline marker support enabling users to click markers to navigate directly to comments and annotations.
    • Introduced automatic video seeking to specific timestamps when navigating to frame-based annotations.
    • Enhanced annotation navigation with improved scrolling and routing to relevant content in the activity feed.

…annotations

Activity sidebar derives video timeline markers from filteredItems
(comments with timestamp markup, annotations with frame target) and
pushes them to the box-content-preview viewer via a window CustomEvent.
Marker clicks bubble back through AnnotatorContext, where the existing
handleAnnotationSelect flow renders the overlay and seeks the video.

handleAnnotationSelect now seeks frame annotations on the current file
version first so the playhead moves before sidebar scroll and overlay
render.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@mrscobbler mrscobbler requested review from a team as code owners June 23, 2026 23:33
@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR introduces timeline marker support across the annotator context infrastructure. New TimelineMarker, TimelineMarkerClickPayload, and TimelineMarkerClickHandler types are defined in both TypeScript and Flow. The withAnnotations HOC implements a window CustomEvent bridge (bp:timeline_markers_update, bp:timeline_marker_click, bp:timeline_markers_ready) with a marker cache for late-mounting viewers. withAnnotatorContext forwards the new props through both routing branches. ActivityFeedV2 derives markers from feed items and pushes them via context. ActivitySidebar subscribes to marker clicks and performs video seeking and sidebar navigation.

Changes

Timeline Marker Feature

Layer / File(s) Summary
Timeline marker type contracts
src/elements/common/annotator-context/types.ts, src/elements/common/annotator-context/types.js.flow, src/elements/common/annotator-context/withAnnotations.js.flow, src/elements/common/annotator-context/withAnnotatorContext.js.flow
Adds TimelineMarker, TimelineMarkerClickPayload, and TimelineMarkerClickHandler in both TypeScript and Flow. Extends AnnotatorContext, ComponentWithAnnotations, and WithAnnotatorContextProps with addTimelineMarkerClickListener and setTimelineMarkers.
withAnnotations window-event bridge
src/elements/common/annotator-context/withAnnotations.tsx, src/elements/common/annotator-context/__tests__/withAnnotations.test.tsx
Adds lastTimelineMarkers cache field, registers bp:timeline_markers_ready listener on mount to replay cached markers, implements setTimelineMarkers (dispatches bp:timeline_markers_update) and addTimelineMarkerClickListener (subscribes to bp:timeline_marker_click), and wires both into AnnotatorContext.Provider. Test adds onViewer to default props.
withAnnotatorContext prop forwarding
src/elements/common/annotator-context/withAnnotatorContext.tsx
Extends WithAnnotatorContextProps with addTimelineMarkerClickListener and setTimelineMarkers, and forwards both in both the routerDisabled and non-routerDisabled Consumer branches. Removes getAnnotationsMatchPath/getAnnotationsPath from the non-routerDisabled wiring.
ActivityFeedV2 timeline marker derivation
src/elements/content-sidebar/activity-feed-v2/ActivityFeedV2.tsx
Computes timelineMarkers from filteredItems (comments via annotationTimestampMs, annotations via numeric frame location) and calls setTimelineMarkers from AnnotatorContext in a useEffect when the derived markers change.
ActivitySidebar click handling and video seeking
src/elements/content-sidebar/ActivitySidebar.js
Imports seekVideoToMs, registers handleTimelineMarkerClick via addTimelineMarkerClickListener on mount, implements the handler to seek video for timestampMs markers and navigate to annotation/comment views, and adds pre-selection seeking inside handleAnnotationSelect for numeric-frame annotations on the current version.

Sequence Diagram(s)

sequenceDiagram
  participant Viewer as BP Viewer
  participant Window as window (CustomEvents)
  participant withAnnotations as withAnnotations HOC
  participant ActivityFeedV2 as ActivityFeedV2
  participant ActivitySidebar as ActivitySidebar

  rect rgba(100, 149, 237, 0.5)
    Note over ActivityFeedV2,withAnnotations: Marker push path
    ActivityFeedV2->>withAnnotations: setTimelineMarkers(markers) via context
    withAnnotations->>withAnnotations: cache lastTimelineMarkers
    withAnnotations->>Window: dispatch bp:timeline_markers_update
    Window->>Viewer: CustomEvent(markers)
  end

  rect rgba(144, 238, 144, 0.5)
    Note over Viewer,withAnnotations: Viewer ready replay
    Viewer->>Window: dispatch bp:timeline_markers_ready
    Window->>withAnnotations: handleTimelineMarkersReady
    withAnnotations->>Window: dispatch bp:timeline_markers_update (replay)
  end

  rect rgba(255, 165, 0, 0.5)
    Note over Viewer,ActivitySidebar: Marker click path
    Viewer->>Window: dispatch bp:timeline_marker_click
    Window->>withAnnotations: unwrap CustomEvent.detail
    withAnnotations->>ActivitySidebar: handleTimelineMarkerClick(payload)
    ActivitySidebar->>ActivitySidebar: seekVideoToMs(timestampMs)
    ActivitySidebar->>ActivitySidebar: navigate to annotation/comment
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • box/box-ui-elements#4380: Directly modifies ActivitySidebar.js's handleAnnotationSelect flow for frame annotations, video seeking, and deferred navigation—the same methods extended in this PR for timeline-marker-driven navigation.
  • box/box-ui-elements#4611: Parses annotationTimestampMs and frame-target metadata into comment/annotation feed items that ActivityFeedV2 here reads to derive TimelineMarker objects.

Suggested reviewers

  • ahorowitz123
  • jackiejou
  • jmcbgaston

Poem

🐇 Hop along the timeline, markers all in a row,
A click on a timestamp, and off to the video we go!
The viewer sends events, the HOC catches them right,
Feed items become markers, shining timeline-bright.
No annotation missed, no frame left behind—
The rabbit wired it all, with events well-aligned! 🎬

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding timeline markers for timestamped comments and annotations in the content sidebar.
Description check ✅ Passed The PR description provides a clear summary, explains the implementation approach, documents the communication bridge mechanism, and includes a comprehensive test plan with verification steps.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Biome (2.5.0)
src/elements/content-sidebar/ActivitySidebar.js

File contains syntax errors that prevent linting: Line 15: 'import { type x ident }' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 51: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 59: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 74: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 75: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 76: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 77: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 78: 'import type' are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 79: '

... [truncated 25060 characters] ...

ken. Did you mean {'>'} or &gt;?; Line 1439: expected } but instead found const; Line 1443: expected } but instead found return; Line 1444: Unexpected token. Did you mean {'}'} or &rbrace;?; Line 1446: Unexpected token. Did you mean {'}'} or &rbrace;?; Line 1449: expected } but instead found const; Line 1469: Expected an expression but instead found '}'.; Line 1476: expected } but instead found const; Line 1527: Unexpected token. Did you mean {'}'} or &rbrace;?; Line 1580: Unexpected token. Did you mean {'}'} or &rbrace;?; Line 1581: Unexpected token. Did you mean {'}'} or &rbrace;?; Line 1584: 'as' expression are a TypeScript only feature. Convert your file to a TypeScript file or remove the syntax.; Line 1593: expected < but instead the file ends


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/elements/content-sidebar/activity-feed-v2/ActivityFeedV2.tsx`:
- Around line 270-276: The useEffect hook that calls setTimelineMarkers
publishes timeline markers but does not clear them when the component unmounts,
causing stale markers to remain cached on the video scrubber. Add a cleanup
function (return statement) to the useEffect hook that clears the timeline
markers by calling setTimelineMarkers with an empty array or null value when the
component unmounts, ensuring the scrubber displays current markers only.

In `@src/elements/content-sidebar/ActivitySidebar.js`:
- Around line 254-284: The handleSelectActivityClick method falls through to
comment navigation when type is annotation but no matching feedItem is found,
causing annotation IDs to be incorrectly routed as comment IDs. After the
annotationItem check within the type === 'annotation' block, add logic to handle
the case when annotationItem is null or feedItems is not yet loaded, such as
returning early or using annotation-specific navigation instead of allowing
execution to continue to the comment navigation branch that uses
getActiveCommentPath and FeedEntryType.COMMENTS.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3e3ddfcc-6528-42ad-b920-257759f6a72d

📥 Commits

Reviewing files that changed from the base of the PR and between 583058c and b41b51f.

📒 Files selected for processing (9)
  • src/elements/common/annotator-context/__tests__/withAnnotations.test.tsx
  • src/elements/common/annotator-context/types.js.flow
  • src/elements/common/annotator-context/types.ts
  • src/elements/common/annotator-context/withAnnotations.js.flow
  • src/elements/common/annotator-context/withAnnotations.tsx
  • src/elements/common/annotator-context/withAnnotatorContext.js.flow
  • src/elements/common/annotator-context/withAnnotatorContext.tsx
  • src/elements/content-sidebar/ActivitySidebar.js
  • src/elements/content-sidebar/activity-feed-v2/ActivityFeedV2.tsx

Comment on lines +270 to +276
const { setTimelineMarkers } = React.useContext(AnnotatorContext);

React.useEffect(() => {
if (setTimelineMarkers) {
setTimelineMarkers(timelineMarkers);
}
}, [setTimelineMarkers, timelineMarkers]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Clear timeline markers on unmount to avoid stale scrubber markers.

The effect publishes markers but never clears them. When ActivityFeedV2 unmounts (e.g. switching away from the activity tab), the previously published markers remain cached by the withAnnotations bridge and stay rendered on the video scrubber.

🧹 Proposed cleanup
     React.useEffect(() => {
         if (setTimelineMarkers) {
             setTimelineMarkers(timelineMarkers);
         }
+        return () => {
+            setTimelineMarkers?.([]);
+        };
     }, [setTimelineMarkers, timelineMarkers]);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const { setTimelineMarkers } = React.useContext(AnnotatorContext);
React.useEffect(() => {
if (setTimelineMarkers) {
setTimelineMarkers(timelineMarkers);
}
}, [setTimelineMarkers, timelineMarkers]);
const { setTimelineMarkers } = React.useContext(AnnotatorContext);
React.useEffect(() => {
if (setTimelineMarkers) {
setTimelineMarkers(timelineMarkers);
}
return () => {
setTimelineMarkers?.([]);
};
}, [setTimelineMarkers, timelineMarkers]);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/elements/content-sidebar/activity-feed-v2/ActivityFeedV2.tsx` around
lines 270 - 276, The useEffect hook that calls setTimelineMarkers publishes
timeline markers but does not clear them when the component unmounts, causing
stale markers to remain cached on the video scrubber. Add a cleanup function
(return statement) to the useEffect hook that clears the timeline markers by
calling setTimelineMarkers with an empty array or null value when the component
unmounts, ensuring the scrubber displays current markers only.

Comment on lines +254 to +284
if (type === 'annotation') {
const { feedItems } = this.state;
const annotationItem = feedItems
? feedItems.find(item => item.id === id && item.type === FEED_ITEM_TYPE_ANNOTATION)
: null;
if (annotationItem) {
// Reuse the badge-click flow so the annotator's active-annotation
// state, URL push, overlay rendering, and seek all stay consistent.
this.handleAnnotationSelect(annotationItem);
return;
}
}

// Comment markers carry a timestamp directly; seek immediately so the
// playhead moves before the sidebar scroll catches up.
if (typeof timestampMs === 'number') {
seekVideoToMs(timestampMs);
}

const { history, internalSidebarNavigationHandler, routerDisabled } = this.props;

if (routerDisabled && internalSidebarNavigationHandler) {
internalSidebarNavigationHandler({
sidebar: ViewType.ACTIVITY,
activeFeedEntryId: id,
activeFeedEntryType: FeedEntryType.COMMENTS,
});
} else {
history.push(this.getActiveCommentPath(id));
}
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Annotation markers fall through to comment navigation when the feed item isn't found.

If type === 'annotation' but no matching entry exists in this.state.feedItems (e.g. not yet loaded, or filtered out), execution falls through to the comment branch and navigates with activeFeedEntryType: FeedEntryType.COMMENTS / getActiveCommentPath(id) for an annotation id, producing incorrect routing. Consider returning early (or using the annotation path) for unresolved annotation markers instead of treating them as comments.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/elements/content-sidebar/ActivitySidebar.js` around lines 254 - 284, The
handleSelectActivityClick method falls through to comment navigation when type
is annotation but no matching feedItem is found, causing annotation IDs to be
incorrectly routed as comment IDs. After the annotationItem check within the
type === 'annotation' block, add logic to handle the case when annotationItem is
null or feedItems is not yet loaded, such as returning early or using
annotation-specific navigation instead of allowing execution to continue to the
comment navigation branch that uses getActiveCommentPath and
FeedEntryType.COMMENTS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant