Skip to content

Issue[#689] Solved #703

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+25 −19
Open

Issue[#689] Solved #703

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
729a8ab
newCommit
akshatsrivastava11 Apr 20, 2025
a54ed07
Update evals/tasks/extract_press_releases.ts
akshatsrivastava11 Apr 25, 2025
b111dc2
Update evals/tasks/extract_press_releases.ts
akshatsrivastava11 Apr 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions evals/args.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,13 +83,20 @@ if (parsedArgs.leftover.length > 0) {
process.exit(1);
}
try {
EvalCategorySchema.parse(filterByCategory);
const result = EvalCategorySchema.safeParse(filterByCategory);

if (!result.success) {
throw new Error(JSON.stringify(result.error.format()));
}

const parsedCategory = result.data; =
} catch {
console.error(
`Error: Invalid category "${filterByCategory}". Valid categories are: ${DEFAULT_EVAL_CATEGORIES.join(", ")}`,
`Error: Invalid category "${filterByCategory}". Valid categories are: ${DEFAULT_EVAL_CATEGORIES.join(", ")}`
);
process.exit(1);
}

} else {
// If leftover[0] is not "category", interpret it as a task/eval name
filterByEvalName = parsedArgs.leftover[0];
Expand Down
8 changes: 5 additions & 3 deletions evals/tasks/extract_press_releases.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,12 @@ export const extract_press_releases: EvalFunction = async ({
useTextExtract,
});

const parsed = schema.parse(rawResult);
const { items } = parsed;
const parsed = schema.safeParse(rawResult);
if(!parsed.success){
throw new Error(parsed.error.format());
}
Comment on lines +41 to +44
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Error.format() returns a complex object that may not stringify well in the catch block's error logging

const { items } = parsed.data;

await stagehand.close();

const expectedLength = 28;
const expectedFirstItem: PressRelease = {
Expand Down
8 changes: 2 additions & 6 deletions examples/external_clients/customOpenAI.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,8 @@ import { z } from "zod";
import { CreateChatCompletionResponseError } from "@/types/stagehandErrors";

function validateZodSchema(schema: z.ZodTypeAny, data: unknown) {
try {
schema.parse(data);
return true;
} catch {
return false;
}
const result = schema.safeParse(data);
return result.success;
}
Comment on lines 24 to 27
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: validateZodSchema could return the error details from safeParse instead of just a boolean for better error reporting


export class CustomOpenAIClient extends LLMClient {
Expand Down
9 changes: 7 additions & 2 deletions lib/handlers/extractHandler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,8 +177,13 @@ export class StagehandExtractHandler {
);

const result = { page_text: formattedText };
return pageTextSchema.parse(result);
}
const res = pageTextSchema.safeParse(result);

if (!res.success) {
throw new Error(JSON.stringify(res.error.format()));
}

return res.data; }
Comment on lines 179 to +186
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Indentation issue - closing brace of extractPageText() is misaligned

Suggested change
const result = { page_text: formattedText };
return pageTextSchema.parse(result);
}
const res = pageTextSchema.safeParse(result);
if (!res.success) {
throw new Error(JSON.stringify(res.error.format()));
}
return res.data; }
const result = { page_text: formattedText };
const res = pageTextSchema.safeParse(result);
if (!res.success) {
throw new Error(JSON.stringify(res.error.format()));
}
return res.data;
}


private async textExtract<T extends z.AnyZodObject>({
instruction,
Expand Down
8 changes: 2 additions & 6 deletions lib/utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -369,12 +369,8 @@ export function logLineToString(logLine: LogLine): string {
}

export function validateZodSchema(schema: z.ZodTypeAny, data: unknown) {
try {
schema.parse(data);
return true;
} catch {
return false;
}
const result = schema.safeParse(data);
return result.success;
Comment on lines 371 to +373
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider returning result.error for better error reporting, since safeParse provides detailed validation errors

Suggested change
export function validateZodSchema(schema: z.ZodTypeAny, data: unknown) {
try {
schema.parse(data);
return true;
} catch {
return false;
}
const result = schema.safeParse(data);
return result.success;
export function validateZodSchema(schema: z.ZodTypeAny, data: unknown): { success: boolean; error?: z.ZodError } {
const result = schema.safeParse(data);
return { success: result.success, error: result.success ? undefined : result.error };

}

export async function drawObserveOverlay(page: Page, results: ObserveResult[]) {
Expand Down