Skip to content

add data importer #1719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

add data importer #1719

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e2b0397
add data importer
yujonglee Nov 18, 2025
d459c08
Exclude Netlify edge functions from TS compilation
yujonglee Nov 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions apps/desktop/src/store/tinybase/importer.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import {
BaseDirectory,
exists,
readTextFile,
remove,
} from "@tauri-apps/plugin-fs";
import { createMergeableStore } from "tinybase/with-schemas";

import { SCHEMA, type Store } from "./main";

export const maybeImportFromJson = async (store: Store) => {
const path = "hyprnote/import/main.json";
const baseDir = BaseDirectory.Data;

if (!(await exists(path, { baseDir }))) {
return;
}

try {
const content = await readTextFile(path, { baseDir });
const [tables, values] = JSON.parse(content) as [unknown, unknown];
Comment on lines +20 to +21
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add runtime validation for JSON structure.

The code assumes the JSON content is a two-element array but doesn't validate this structure. If the file contains malformed data (e.g., a single object, an array with a different length, or invalid JSON), the destructuring may fail silently or produce unexpected results.

 const content = await readTextFile(path, { baseDir });
-const [tables, values] = JSON.parse(content) as [unknown, unknown];
+const parsed = JSON.parse(content);
+if (!Array.isArray(parsed) || parsed.length !== 2) {
+  throw new Error("Invalid import file format: expected [tables, values] tuple");
+}
+const [tables, values] = parsed as [unknown, unknown];
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const content = await readTextFile(path, { baseDir });
const [tables, values] = JSON.parse(content) as [unknown, unknown];
const content = await readTextFile(path, { baseDir });
const parsed = JSON.parse(content);
if (!Array.isArray(parsed) || parsed.length !== 2) {
throw new Error("Invalid import file format: expected [tables, values] tuple");
}
const [tables, values] = parsed as [unknown, unknown];
🤖 Prompt for AI Agents 
In apps/desktop/src/store/tinybase/importer.ts around lines 20-21, the code
blindly parses JSON and destructures it into [tables, values]; add runtime
validation and error handling: wrap JSON.parse in try/catch to surface invalid
JSON, verify the parsed value is an array with exactly two elements, and confirm
each element is the expected type (e.g., objects or arrays as your consumer
expects); if validation fails, throw or return a clear error so callers can
handle malformed files instead of proceeding with undefined or incorrect data.


const importStore = createMergeableStore()
.setTablesSchema(SCHEMA.table)
.setValuesSchema(SCHEMA.value);

if (tables) {
importStore.setTables(tables as never);
}
if (values) {
importStore.setValues(values as never);
}
Comment on lines +27 to +32
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Improve type safety with schema validation.

Using as never type assertions bypasses all TypeScript type checking. If the imported data doesn't match SCHEMA, it could lead to runtime errors or data corruption. While TinyBase may perform runtime schema validation, the type assertions eliminate compile-time safety.

Consider validating the imported data against the schema or using TinyBase's built-in validation mechanisms more explicitly.

🤖 Prompt for AI Agents 
In apps/desktop/src/store/tinybase/importer.ts around lines 27-32, the use of
"as never" on imported tables/values bypasses TypeScript checking and can allow
schema-mismatched data into the store; replace the assertions with proper
validation: validate the imported object shape against your SCHEMA at runtime
(using TinyBase's import/validation helpers or a lightweight validator like
zod/io-ts) and only call importStore.setTables/setValues with typed data after
successful validation, or cast to the correct typed interfaces rather than
"never", and log/throw a clear error on validation failure so invalid imports
are rejected before mutating the store.

store.merge(importStore);
await remove(path, { baseDir });
} catch (error) {
console.error(error);
}
Comment on lines +33 to +37
Copy link

@coderabbitai coderabbitai bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Clarify error handling and cleanup strategy.

Current behavior:

  1. If the merge fails, the file isn't removed, causing retry on next startup
  2. If the merge succeeds but file removal fails, the error is logged but the file remains, also causing retry
  3. All errors are caught and logged, but not propagated, making failures invisible to the caller

Consider:

  • Remove the file before merging (if you want to prevent retries of malformed data)
  • Or ensure the file is removed even if merge fails (using try-finally)
  • Or propagate errors to the caller for explicit handling
  • Add success logging for observability

Example with try-finally for guaranteed cleanup:

 try {
   const content = await readTextFile(path, { baseDir });
   const [tables, values] = JSON.parse(content) as [unknown, unknown];

   const importStore = createMergeableStore()
     .setTablesSchema(SCHEMA.table)
     .setValuesSchema(SCHEMA.value);

   if (tables) {
     importStore.setTables(tables as never);
   }
   if (values) {
     importStore.setValues(values as never);
   }
+  
+  // Remove file before merging to prevent retry of malformed data
+  await remove(path, { baseDir });
+  
   store.merge(importStore);
-  await remove(path, { baseDir });
+  console.log("Successfully imported data from", path);
 } catch (error) {
   console.error("Failed to import data:", error);
+  // Consider whether to rethrow or handle differently
 }

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In apps/desktop/src/store/tinybase/importer.ts around lines 33-37, the current
catch-only error handling causes files to remain on failure and swallows errors;
change to guarantee cleanup and surface failures by using a try { await
store.merge(importStore); } finally { await remove(path, { baseDir }).catch(err
=> processLogger.error('Failed to remove import file', { path, err })); } and
rethrow any merge error so the caller can handle it (or alternatively remove the
file before merging if you prefer to avoid retries of malformed data), and add a
success log after merge completes.

};
4 changes: 3 additions & 1 deletion apps/desktop/src/store/tinybase/main.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import { getCurrentWebviewWindowLabel } from "@hypr/plugin-windows";
import { format } from "@hypr/utils";

import { DEFAULT_USER_ID } from "../../utils";
import { maybeImportFromJson } from "./importer";
import { createLocalPersister } from "./localPersister";
import { createLocalPersister2 } from "./localPersister2";
import { externalTableSchemaForTinybase } from "./schema-external";
Expand All @@ -35,7 +36,7 @@ export * from "./schema-internal";

export const STORE_ID = "main";

const SCHEMA = {
export const SCHEMA = {
value: {
...internalSchemaForTinybase.value,
} as const satisfies ValuesSchema,
Expand Down Expand Up @@ -119,6 +120,7 @@ export const StoreComponent = ({ persist = true }: { persist?: boolean }) => {

const initializer = async (cb: () => void) => {
await persister.load();
await maybeImportFromJson(store as Store);
store.transaction(() => cb());
await persister.save();
};
Expand Down
2 changes: 1 addition & 1 deletion apps/web/tsconfig.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"include": ["**/*.ts", "**/*.tsx"],
"exclude": ["supabase/functions/**/*"],
"exclude": ["supabase/functions/**/*", "netlify/edge-functions/**/*"],
"compilerOptions": {
"target": "ES2022",
"jsx": "react-jsx",
Expand Down
Loading