Specify english when using the distil model. It only helps for english #76
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(and that's what I'm using) so let's make sure whisper knows to use english overall. If using the large model in general though, it will auto-detect the language as it has previously.
|
This is a benchmark review for experiment This pull request was cloned from Experiment configurationreview_config:
# User configuration for the review
# - benchmark - use the user config from the benchmark reviews
# - <value> - use the value directly
user_config:
enable_ai_review: true
enable_rule_comments: false
enable_complexity_comments: benchmark
enable_docstring_comments: benchmark
enable_security_comments: benchmark
enable_tests_comments: benchmark
enable_comment_suggestions: benchmark
enable_functionality_review: benchmark
enable_approvals: true
ai_review_config:
# The model responses to use for the experiment
# - benchmark - use the model responses from the benchmark reviews
# - llm - call the language model to generate responses
model_responses:
comments_model: benchmark
comment_validation_model: benchmark
comment_suggestion_model: benchmark
complexity_model: benchmark
docstrings_model: benchmark
functionality_model: benchmark
security_model: benchmark
tests_model: benchmark
# The pull request dataset to run the experiment on
pull_request_dataset:
- https://github.com/nbhirud/system_update/pull/31
- https://github.com/nbhirud/system_update/pull/34
- https://github.com/suttacentral/suttacentral/pull/3164
- https://github.com/0xdade/sephiroth/pull/79
- https://github.com/Fenigor/align-game/pull/3
- https://github.com/NathanVaughn/blog.nathanv.me/pull/200
- https://github.com/jquagga/ttt/pull/107
- https://github.com/Stagietechs/sketchup-ruby-api-tutorials/pull/1
- https://github.com/jabesq-org/pyatmo/pull/497
- https://github.com/UnitapApp/unitap-backend/pull/440
- https://github.com/UnitapApp/unitap-backend/pull/441
- https://github.com/UnitapApp/unitap-backend/pull/442
- https://github.com/gdsfactory/gdsfactory/pull/2725
- https://github.com/gdsfactory/kfactory/pull/305
- https://github.com/gdsfactory/kfactory/pull/309
- https://github.com/kloudlite/operator/pull/185
- https://github.com/kloudlite/api/pull/317
- https://github.com/nuxeo/nuxeo-drive/pull/4850
- https://github.com/albumentations-team/albumentations/pull/1711
- https://github.com/avelino/awesome-go/pull/5303
- https://github.com/Cristofer543/Cristofer543.github.io/pull/1
- https://github.com/W-zrd/unishare_mobile/pull/14
- https://github.com/2lambda123/DPDK-dpdk/pull/3
- https://github.com/2lambda123/DPDK-dpdk/pull/1
- https://github.com/2lambda123/wayveai-mile/pull/1
- https://github.com/2lambda123/wayveai-mile/pull/3
- https://github.com/erxes/erxes/pull/5188
- https://github.com/Patrick-Ehimen/the-wild-oasis/pull/1
- https://github.com/Patrick-Ehimen/the-wild-oasis/pull/2
- https://github.com/manoelhc/test-actions/pull/41
- https://github.com/StartupOS/verinfast/pull/365
- https://github.com/CypherGuy/PantryPal/pull/1
- https://github.com/DevCycleHQ/devcycle-docs/pull/666
- https://github.com/DevCycleHQ/cli/pull/388
- https://github.com/DevCycleHQ/js-sdks/pull/841
- https://github.com/allthingslinux/tux/pull/207
- https://github.com/simnova/ownercommunity/pull/109
- https://github.com/simnova/ownercommunity/pull/111
- https://github.com/simnova/ownercommunity/pull/112
- https://github.com/FrancisKOUAHO/Plumera/pull/24
- https://github.com/FrancisKOUAHO/Plumera/pull/25
- https://github.com/dashmug/glue-utils/pull/32
- https://github.com/dashmug/glue-utils/pull/34
review_comment_labels:
- label: correct
question: Is this comment correct?
- label: helpful
question: Is this comment helpful?
- label: comment-type
question: Is the comment type correct?
- label: comment-area
question: Is the comment area correct?
- label: llm-test
question: |
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
# Benchmark reviews generated by running
# python -m scripts.experiment benchmark <experiment_name>
benchmark_reviews: []
|
SourceryAI
approved these changes
May 8, 2024
There was a problem hiding this comment.
Hey @sourcery-ai-experiments-bot - I've reviewed your changes and they look great!
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🟢 Security: all looks good
- 🟡 Testing: 1 issue found
- 🟢 Complexity: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.
| model_id = os.environ.get("TTT_TRANSFORMERS_MODEL_ID", "openai/whisper-large-v3") | ||
| print(f"We are using {torch_dtype} on {device} with {model_id}") |
There was a problem hiding this comment.
suggestion (code_refinement): Consider logging instead of printing for better production environment compatibility.
Using logging instead of print statements can help in managing output verbosity and is generally more flexible for different environments, such as production or development.
Suggested change
| print(f"We are using {torch_dtype} on {device} with {model_id}") | |
| import logging | |
| logging.basicConfig(level=logging.INFO) | |
| logger = logging.getLogger(__name__) | |
| logger.info(f"We are using {torch_dtype} on {device} with {model_id}") |
There was a problem hiding this comment.
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
| @@ -75,7 +77,7 @@ | |||
| tokenizer=processor.tokenizer, | |||
| feature_extractor=processor.feature_extractor, | |||
| max_new_tokens=128, | |||
| generate_kwargs={"assistant_model": assistant_model}, | |||
| generate_kwargs={"assistant_model": assistant_model, "language": "english"}, | |||
There was a problem hiding this comment.
suggestion (testing): Missing test for the new 'language' parameter in generate_kwargs.
The addition of the 'language' parameter to generate_kwargs suggests a specific behavior when using the assistant model. It would be beneficial to add a test case that verifies the correct handling and effect of this parameter, ensuring it integrates correctly and modifies the behavior as expected.
There was a problem hiding this comment.
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(and that's what I'm using) so let's make sure whisper knows to use english overall. If using the large model in general though, it will auto-detect the language as it has previously.