Skip to content

feat: add verbose option in optimize_fn #654

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 14, 2025

Conversation

deependujha
Copy link
Collaborator

@deependujha deependujha commented Jul 14, 2025

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Gives flexibility to disable logs while optimizing. This can be helpful when you have multiple datasets that you wish to optimize and are not interested in each operation, but rather in the progress bar.

Used by Ultralytics PR when optimizing train, test & val datasets.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@deependujha deependujha requested a review from Copilot July 14, 2025 02:41
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds a verbose flag to both the top-level optimize function and the DataProcessor class so users can disable internal print calls when they only want progress bars or external logging.

  • Introduces a verbose: bool = True parameter to optimize and propagates it to internal logic.
  • Adds a verbose parameter to DataProcessor.__init__, stores it as self.verbose, and guards all existing print calls.
  • Updates function calls within optimize and DataProcessor to pass and check the new verbose flag.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/litdata/processing/functions.py Added verbose parameter to optimize, updated signature docs, gated prints, and passed verbose along to child calls.
src/litdata/processing/data_processor.py Added verbose parameter to DataProcessor, stored as self.verbose, and wrapped all existing console output under if self.verbose: checks.
Comments suppressed due to low confidence (2)

src/litdata/processing/functions.py:413

  • The optimize() docstring should include the verbose parameter in the Args section with its type and description to match the style used for other parameters.
    verbose: bool = True,

src/litdata/processing/functions.py:413

  • Add unit tests to verify that setting verbose=False actually suppresses all console output in both optimize() and DataProcessor to guard against future regressions.
    verbose: bool = True,

@deependujha deependujha requested a review from Borda as a code owner July 14, 2025 03:06
Copy link

codecov bot commented Jul 14, 2025

Codecov Report

Attention: Patch coverage is 88.88889% with 4 lines in your changes missing coverage. Please review.

Project coverage is 83%. Comparing base (54cb63b) to head (804e279).
Report is 1 commits behind head on main.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #654   +/-   ##
===================================
  Coverage    83%    83%           
===================================
  Files        49     49           
  Lines      6761   6778   +17     
===================================
+ Hits       5642   5657   +15     
- Misses     1119   1121    +2     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Borda Borda changed the title feat: add verbose option in optimize_fn feat: add verbose option in optimize_fn Jul 14, 2025
@Borda Borda enabled auto-merge (squash) July 14, 2025 09:01
@Borda Borda merged commit 371f262 into Lightning-AI:main Jul 14, 2025
35 checks passed
@deependujha deependujha deleted the feat/add-verbose-option branch July 14, 2025 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants