Skip to content

Parallelize the identification of files that need to be scaled down #6857

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ZainRizvi
Copy link
Contributor

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 27, 2025
Copy link

vercel bot commented Jun 27, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Updated (UTC)
torchci ⬜️ Ignored (Inspect) Visit Preview Jul 12, 2025 1:00am

];

mocked(listRunners).mockResolvedValue(runners);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
// Verify parallel execution - check that we have both start and end calls
const startCalls = callOrder.filter(call => call.startsWith('start-'));
const endCalls = callOrder.filter(call => call.startsWith('end-'));

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
];

mocked(listRunners).mockResolvedValue(runners);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
// Track concurrent calls
let concurrentCalls = 0;
let maxConcurrentCalls = 0;

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
mocked(listGithubRunnersOrg).mockImplementation(async (org, metrics) => {
concurrentCalls++;
maxConcurrentCalls = Math.max(maxConcurrentCalls, concurrentCalls);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
maxConcurrentCalls = Math.max(maxConcurrentCalls, concurrentCalls);

await new Promise(resolve => setTimeout(resolve, 10));

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.

// Verify all GitHub API calls were made
expect(mocked(listGithubRunnersOrg)).toHaveBeenCalledTimes(5); // 3 for processing + 2 for offline cleanup (org1 and org2)

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
];

mocked(listRunners).mockResolvedValue(runners);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
];

mocked(listRunners).mockResolvedValue(runners);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.

// All runners should have been processed (50 for processing + 50 for offline cleanup)
expect(mocked(listGithubRunnersRepo)).toHaveBeenCalledTimes(100);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
@seemethere seemethere self-requested a review June 27, 2025 18:52
@jeanschmidt
Copy link
Contributor

pls run lint!

@jeanschmidt
Copy link
Contributor

but I like the idea, I believe this is something we should pursuit

@jeanschmidt
Copy link
Contributor

wait no, don't do it simply like this.

Do in batches, of dozens.

This will for sure trigger the throttling thing we have with GHA API

const runners = Array.from({ length: 25 }, (_, i) => ({
awsRegion: baseConfig.awsRegion,
instanceId: `runner-${i}`,
repo: `owner/repo${i}`,

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
launchTime: dateRef.clone().subtract(minimumRunningTimeInMinutes + 5, 'minutes').toDate(),
}));

mocked(listRunners).mockResolvedValue(runners);

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
// Track concurrent calls to verify batching
let currentConcurrentCalls = 0;
let maxConcurrentCalls = 0;
const concurrentCallHistory: number[] = [];

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
const concurrentCallHistory: number[] = [];

mocked(listGithubRunnersRepo).mockImplementation(async (repo, metrics) => {
currentConcurrentCalls++;

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
mocked(getRunnerTypes).mockResolvedValue(new Map([
['test-type', { is_ephemeral: false, min_available: 0 } as RunnerType]
]));

Check failure

Code scanning / lintrunner

SPACES/trailing spaces Error

This line has trailing spaces; please remove them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants