adding secret watcher to restart ptp4l process #111

gtannous-spec · 2025-11-24T16:44:07Z

PTP TLV Authentication: Instant Secret Change Detection via fsnotify

Summary

Implements instant detection of PTP authentication file changes using fsnotify. When authentication secrets are updated in Kubernetes, the daemon immediately restarts PTP processes without requiring pod restarts.

Changes

Added fsnotify Monitoring (`pkg/daemon/daemon.go`)

New constant:

const PtpSecretMountDir = "/etc/ptp-secret-mount/"

Daemon initialization (New()):

Creates fsnotify.Watcher on startup
Adds watch on /etc/ptp-secret-mount/ directory
Stores watcher in Daemon struct

Event loop (Run()):

Listens on watcher.Events channel for file changes
Filters events: Write, Create, Remove (ignores hidden files)
Calls applyNodePTPProfiles() immediately on relevant events
Restarts ptp4l with updated authentication keys

Error recovery:

Detects when fsnotify watcher crashes (error channel closed)
Recreates watcher with full reinitialization:
- Creates new fsnotify.Watcher
- Re-adds watch path
- Reinitializes event and error channels
Prevents pod restart on watcher failures

Benefits

✅ Instant detection - No polling delay (event-driven)
✅ No pod restarts - PTP processes restart in-place
✅ Simple - Single directory watch, no hash tracking
✅ Resilient - Auto-recovery on watcher crashes
✅ Efficient - No background CPU/I/O for polling

github-actions · 2025-11-24T16:44:15Z

Thanks for your PR,
Best regards.

josephdrichard · 2025-11-24T17:57:11Z

pkg/daemon/daemon.go

 	chronydProcessName, // there can be only one chronyd process in the system
 }

+// saFileInfo tracks authentication file information for a profile


What specifically is sa short for in this context?

security association, sa_file is an option inside the ptp4lconf's global section where we can add a filepath to mount a specific secret inside the linuxptp-daemon-container.
basically I converted some of the code from the ptpconfig_controller here because the file watcher forces a restart on the pods in case a change is detected on the mounted secret, so instead we're having a restart on the ptp4l process.

pkg/daemon/daemon.go

nocturnalastro · 2025-11-26T11:18:40Z

pkg/daemon/daemon.go

+			}
+			// Filter for relevant events (Write, Create - Kubernetes atomic updates)
+			// Ignore events on temporary/hidden files
+			if event.Op&(fsnotify.Write|fsnotify.Create) == 0 {


Should delete also be handled as a change?

If you mean "deleting the sa_file" then the controller actually removes the volumeMount in case the user delete it from the ptp4lconf or manually from the file system of the container. which forces a restart on the daemonset.

pkg/daemon/daemon.go

greyerof

Minor (nit) suggestions but overall looks good to me. Just one question: in case there's two keys added to an existing secret, will it receive one or two consecutive events?

pkg/daemon/daemon.go

greyerof · 2025-12-10T14:27:08Z

pkg/daemon/daemon.go

 	tracker.processManager = pm

+	// Initialize fsnotify watcher for sa_file change detection
+	watcher, err := fsnotify.NewWatcher()


Nit: You can use the same "saFilesWatcher" name for this local var. It's a more descriptive name to use than just "watcher".

Yes that looks better, I changed it to saFileWatch

greyerof · 2025-12-10T14:33:55Z

pkg/daemon/daemon.go

+	var watcherEvents chan fsnotify.Event
+	var watcherErrors chan error


To be consistent with the other channels in the for-select loop (like "updateCh, stopCh)", please append the "Ch" suffix to the name of this vars. Also, adding the prefix "saFilesWatcher" might be a good fit to make them more descriptive:

var saFilesWatcherEventCh ... var saFilesWatcherErrCh ...

josephdrichard · 2025-12-12T19:54:59Z

pkg/daemon/daemon.go

+		defer dn.saFileWatcher.Close()
+		glog.Info("Using fsnotify for instant sa_file change detection")
+	} else {
+		glog.Warning("fsnotify unavailable, sa_file change detection disabled")


What do you mean by fsnotify unavailable? That the fsnotify watcher in unavailable?

in case trying to initialize the fsnotify file watcher fails --> logs a warning fsnotify unavailable

example would be when authentication is disabled and there is no ptp-secret-mount directory in the linuxptp daemon pod. (the watch directory PtpSecretMountDir doesn't exists )

josephdrichard · 2025-12-12T20:07:49Z

pkg/daemon/daemon.go

+				continue // Ignore hidden files like .data
+			}
+			glog.Infof("Security file changed: %s (op: %s), restarting PTP processes", event.Name, event.Op.String())
+			err := dn.applyNodePTPProfiles()


Kind of concerned here that applyNodePTPProfiles could theoretically be called twice in quick succession, which could have race conditions that applyNodePTPProfiles isn't hardened for. Currently we are looking at a ticker and check for updates based off that ticker, so no possibility of it being called twice in a row. What would happen if a user made multiple changes to the secret in quick succession? Is there a possibility it could break something?

That was my concern at the beginning but I guess the Run() function is using a single Go routine which prevents parallelism, and each call will stop all processes with dn.stopAllProcesses() at the end , so it rebuilds from scratch

…ted secret - Add fsnotify watcher to monitor /etc/ptp-secret-mount/ for instant detection of security file changes (sa_file) - Trigger PTP process restart on sa_file write/create/remove events - Promotes fsnotify from indirect to direct dependency - Reinitialize event and error channels - Prevent pod restart on watcher failure

…itions

josephdrichard reviewed Nov 24, 2025

View reviewed changes

nocturnalastro reviewed Nov 25, 2025

View reviewed changes

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

gtannous-spec force-pushed the auth-tlv branch from 98fdb45 to a3fcf96 Compare November 25, 2025 15:44

nocturnalastro added the ok-to-test ok to test label Nov 25, 2025

github-actions bot removed the ok-to-test ok to test label Nov 26, 2025

nocturnalastro reviewed Nov 26, 2025

View reviewed changes

josephdrichard added the ok-to-test ok to test label Nov 26, 2025

github-actions bot removed the ok-to-test ok to test label Nov 26, 2025

gtannous-spec force-pushed the auth-tlv branch 3 times, most recently from f594bd0 to 258876f Compare November 29, 2025 20:43

gtannous-spec requested review from josephdrichard and nocturnalastro December 1, 2025 09:10

vitus133 reviewed Dec 1, 2025

View reviewed changes

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

gtannous-spec force-pushed the auth-tlv branch 2 times, most recently from 9563424 to 67338f9 Compare December 3, 2025 10:56

josephdrichard reviewed Dec 3, 2025

View reviewed changes

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

josephdrichard added the ok-to-test ok to test label Dec 3, 2025

edcdavid reviewed Dec 3, 2025

View reviewed changes

pkg/daemon/daemon.go Show resolved Hide resolved

gtannous-spec force-pushed the auth-tlv branch from 67338f9 to e9c61c0 Compare December 3, 2025 22:59

github-actions bot removed the ok-to-test ok to test label Dec 3, 2025

gtannous-spec marked this pull request as ready for review December 3, 2025 23:33

gtannous-spec marked this pull request as draft December 3, 2025 23:33

gtannous-spec marked this pull request as ready for review December 3, 2025 23:48

gtannous-spec force-pushed the auth-tlv branch from e9c61c0 to e3af397 Compare December 4, 2025 21:17

edcdavid added the ok-to-test ok to test label Dec 8, 2025

greyerof reviewed Dec 10, 2025

View reviewed changes

gtannous-spec force-pushed the auth-tlv branch from e3af397 to fb15fad Compare December 10, 2025 18:00

github-actions bot removed the ok-to-test ok to test label Dec 10, 2025

edcdavid added ok-to-test ok to test and removed ok-to-test ok to test labels Dec 10, 2025

edcdavid previously approved these changes Dec 12, 2025

View reviewed changes

josephdrichard reviewed Dec 12, 2025

View reviewed changes

josephdrichard added the do not merge label Dec 12, 2025

gtannous-spec dismissed edcdavid’s stale review via 27eee79 December 15, 2025 17:05

gtannous-spec force-pushed the auth-tlv branch from fb15fad to 27eee79 Compare December 15, 2025 17:05

github-actions bot removed the ok-to-test ok to test label Dec 15, 2025

adding an atomic bool flag for the saFileWatcher preventing race cond…

27eee79

…itions

		var watcherEvents chan fsnotify.Event
		var watcherErrors chan error

adding secret watcher to restart ptp4l process #111

Are you sure you want to change the base?

adding secret watcher to restart ptp4l process #111

Conversation

gtannous-spec commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PTP TLV Authentication: Instant Secret Change Detection via fsnotify

Summary

Changes

Added fsnotify Monitoring (pkg/daemon/daemon.go)

Benefits

Uh oh!

github-actions bot commented Nov 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greyerof left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gtannous-spec commented Nov 24, 2025 •

edited

Loading

Added fsnotify Monitoring (`pkg/daemon/daemon.go`)