Session sync performance optimizations

[Day-Anger]

Summary

Five independent optimizations for the SQLite session synchronization, each verified with tests.

Optimizations

1. Compact JSON — \JSON.stringify(session)\ without
   ull, 2. Saves ~30% file size on session JSON files.

2. Single WAL checkpoint — checkpoint moved out of \openDB()\ into \syncSessions(), run once per cycle instead of on every DB open.

3. Single DB handle — \syncSessions()\ opens SQLite once, uses handle-variant functions (\listSessionIdsFromHandle,
   eadSessionFromHandle, \writeSessionsToHandle) for all read/write operations within one cycle.

4. Batch DB writes — all merged sessions written in a single \BEGIN…COMMIT\ transaction instead of one INSERT per session.

5. Background git push — \pushBranch(...).catch(...)\ doesn't block startup. HEAD cache (\getHeadHash()\ + \state.lastHead) skips unnecessary remote fetches when HEAD hasn't changed.

Impact

Metrics from local testing (Windows, ~200 sessions):

• Startup time: ~2.5s → ~0.8s
• Sync cycle time: ~4s → ~1s
• Disk usage: reduced by ~30% (compact JSON)

[gemini-code-assist]

Code Review

This pull request refactors the synchronization service by replacing legacy bash scripts with a more integrated startup sync process that handles auto-committing, background pushing, and merging of session and project data. It also introduces a new SQLite-based session database and a Node-based shell wrapper. Feedback highlights a synchronization logic flaw where the 'shouldFetch' optimization might miss remote updates from other devices. Additionally, reviewers pointed out a race condition in the background push state updates and performance issues in the database layer, specifically N+1 queries and inefficient SQL statement preparation.

[gemini-code-assist]

The 'shouldFetch' optimization breaks multi-device synchronization. It assumes that if the local repository is clean and the local HEAD matches the cached 'state.lastHead', no remote changes exist. However, another device could have pushed changes to the remote repository in the meantime. Without a 'fetch' (or at least an 'ls-remote' check), this client will not detect those changes on startup. Correctness should be prioritized over skipping a network request in a sync service. Consider using 'git ls-remote' if you want a faster check than a full fetch.

[gemini-code-assist]

Updating the sync state ('lastPush', 'lastHead') and notifying the user immediately after starting a background push is unsafe. If the push fails (e.g., due to network issues or remote rejection), the local state will incorrectly reflect a successful synchronization. This, combined with the 'shouldFetch' optimization, will cause the client to skip future sync attempts for these changes. The state update and success notification should be moved inside the completion handler of the push operation.

      pushBranch(ctx.$, repoRoot, branch)
        .then(async () => {
          await updateState(locations, {
            lastPush: new Date().toISOString(),
            lastHead: commitHead ?? undefined,
          });
          log.info('Auto-committed and pushed pending changes');
          await showToast(ctx.client, 'Pending changes committed and pushed', 'info');
        })
        .catch((err) =>
          log.warn('Background push failed', { error: formatError(err) })
        );

[gemini-code-assist]

This function has an N+1 query problem and redundant statement preparation. It prepares and executes a query for the 'part' table for every single message in the session. It is significantly more efficient to fetch all parts for the session in one query and group them by 'message_id' in memory.

function readMessages(db: DatabaseSync, sessionId: string): Message[] {
  const messages = db
    .prepare('SELECT ' + MESSAGE_COLUMNS.join(', ') + ' FROM message WHERE session_id = ?')
    .all(sessionId) as unknown as Message[];
  const allParts = db
    .prepare('SELECT ' + PART_COLUMNS.join(', ') + ' FROM part WHERE session_id = ?')
    .all(sessionId) as unknown as Part[];

  const partsByMessage = new Map<string, Part[]>();
  for (const part of allParts) {
    const list = partsByMessage.get(part.message_id) ?? [];
    list.push(part);
    partsByMessage.set(part.message_id, list);
  }

  for (const msg of messages) {
    msg.parts = partsByMessage.get(msg.id) ?? [];
  }
  return messages;
}

[gemini-code-assist]

Preparing the SQL statement inside the loop (via 'upsertSession' called in 'writeSessionsToHandle') is inefficient. The statement should be prepared once outside the loop and reused for all sessions to improve performance.

[jules-relay]

🤖 Review Jules Relay

I found 1 Gemini suggestion so far.

Type /relay batch to send all suggestions to Jules.