Skip to content

Commit 09e4817

Browse files
perf: Create and parse generator
Improves memory efficiency and runtime Fixes #45
1 parent 2506988 commit 09e4817

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

src/thread/thread.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -512,9 +512,16 @@ def start(self) -> None:
512512
}
513513

514514
i = 0
515+
for chunkStart, chunkEnd in chunk_split(len(self.dataset), max_threads):
515516
chunk_thread = Thread(
516517
target=self.function,
517-
args=[i, data_chunk, *parsed_args, *self.overflow_args],
518+
args=[
519+
i,
520+
chunkEnd - chunkStart,
521+
(self.dataset[x] for x in range(chunkStart, chunkEnd)),
522+
*parsed_args,
523+
*self.overflow_args,
524+
],
518525
name=name_format and name_format % i or None,
519526
**self.overflow_kwargs,
520527
)

0 commit comments

Comments
 (0)