Skip to content

Add better profanity filtering and filter out empty notifications #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: base-sha/7ebe9275f0b7f9d1f2ca65aa4be1b89407f7c1c8
Choose a base branch
from

Conversation

sourcery-ai-experiments-bot

No description provided.

@sourcery-ai-experiments-bot
Copy link
Author

This is a benchmark review for experiment review_of_reviews_20240501.
Run ID: review_of_reviews_20240501/benchmark_2024-05-01T00-17-45_v1-16-0-221-g235b41e01.

This pull request was cloned from https://github.com/jquagga/ttt/pull/98. (Note: the URL is not a link to avoid triggering a notification on the original pull request.)

Experiment configuration
review_config:
  # User configuration for the review
  # - benchmark - use the user config from the benchmark reviews
  # - <value> - use the value directly
  user_config:
    enable_ai_review: true
    enable_rule_comments: false

    enable_complexity_comments: benchmark
    enable_docstring_comments: benchmark
    enable_security_comments: benchmark
    enable_tests_comments: benchmark
    enable_comment_suggestions: benchmark
    enable_functionality_review: benchmark

    enable_approvals: true

  ai_review_config:
    # The model responses to use for the experiment
    # - benchmark - use the model responses from the benchmark reviews
    # - llm - call the language model to generate responses
    model_responses:
      comments_model: benchmark
      comment_validation_model: benchmark
      comment_suggestion_model: benchmark
      complexity_model: benchmark
      docstrings_model: benchmark
      functionality_model: benchmark
      security_model: benchmark
      tests_model: benchmark

# The pull request dataset to run the experiment on
pull_request_dataset:
- https://github.com/strawberry-graphql/strawberry/pull/3478
- https://github.com/agatma/sprint1-http-server/pull/3
- https://github.com/strawberry-graphql/strawberry-django/pull/525
- https://github.com/agatma/sprint1-http-server/pull/4
- https://github.com/strawberry-graphql/strawberry/pull/3480
- https://github.com/Xmaster6y/mulsi/pull/21
- https://github.com/Xmaster6y/mulsi/pull/22
- https://github.com/LLotme/vscode/pull/4
- https://github.com/ignition-api/8.1/pull/271
- https://github.com/ignition-api/8.1/pull/272
- https://github.com/manoelhc/test-actions/pull/33
- https://github.com/tqt97/BlogForJob/pull/3
- https://github.com/tqt97/BlogForJob/pull/2
- https://github.com/tqt97/BlogForJob/pull/4
- https://github.com/ser3n7ty/Nvim/pull/1
- https://github.com/ser3n7ty/Nvim/pull/2
- https://github.com/ser3n7ty/Nvim/pull/3
- https://github.com/okisdev/ChatChat/pull/325
- https://github.com/DevCycleHQ/ios-client-sdk/pull/197
- https://github.com/DevCycleHQ/test-harness/pull/424
- https://github.com/DevCycleHQ/flutter-client-sdk/pull/150
- https://github.com/DevCycleHQ/cli/pull/382
- https://github.com/DevCycleHQ/devcycle-docs/pull/642
- https://github.com/uncscode/particula/pull/449
- https://github.com/nikhilbadyal/docker-py-revanced/pull/518
- https://github.com/ignition-incendium/incendium/pull/199
- https://github.com/rtk-rnjn/algorithms/pull/81
- https://github.com/rtk-rnjn/algorithms/pull/82
- https://github.com/rtk-rnjn/algorithms/pull/83
- https://github.com/rtk-rnjn/algorithms/pull/84
- https://github.com/suttacentral/suttacentral/pull/3151
- https://github.com/EduardSchwarzkopf/pecuny/pull/131
- https://github.com/gdsfactory/cspdk/pull/27
- https://github.com/gdsfactory/cspdk/pull/28
- https://github.com/gdsfactory/cspdk/pull/29
- https://github.com/gdsfactory/ubc/pull/376
- https://github.com/gdsfactory/ubc/pull/377
- https://github.com/gdsfactory/gdsfactory/pull/2709
- https://github.com/gdsfactory/ubc/pull/378
- https://github.com/W-zrd/unishare_mobile/pull/13
- https://github.com/Nuitka/Nuitka/pull/2830
- https://github.com/jquagga/ttt/pull/98
- https://github.com/jquagga/ttt/pull/101
- https://github.com/Idrinth/api-bench/pull/939
- https://github.com/letsdoitnowus/planium-backend/pull/41
- https://github.com/albumentations-team/albumentations.ai/pull/74
- https://github.com/albumentations-team/albumentations/pull/1695
- https://github.com/yashasvi-jain/ucron-models/pull/11
- https://github.com/nbhirud/system_update/pull/24
- https://github.com/supabase-community/postgrest-py/pull/422
review_comment_labels:
- label: correct
  question: Is this comment correct?
- label: helpful
  question: Is this comment helpful?
- label: comment-type
  question: Is the comment type correct?
- label: comment-area
  question: Is the comment area correct?
- label: llm-test
  question: |
    What type of LLM test could this comment become?
    - 👍 - this comment is really good/important and we should always make it
    - 👎 - this comment is really bad and we should never make it
    - no reaction - don't turn this comment into an LLM test

# Benchmark reviews generated by running
#   python -m scripts.experiment benchmark <experiment_name>
benchmark_reviews: []

Copy link

@SourceryAI SourceryAI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sourcery-ai-experiments-bot - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 3 issues found
  • 🟢 Security: all looks good
  • 🟡 Testing: 2 issues found
  • 🟢 Complexity: all looks good

LangSmith trace

Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

@@ -76,6 +76,7 @@
torch_dtype=torch_dtype,
device=device,
)
profanity.load_censor_words()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code_refinement): Consider initializing profanity filter setup in a dedicated initialization function.

Placing the initialization of the profanity filter directly in the global scope of the script might lead to issues with maintainability and testing. It's generally a good practice to encapsulate setup logic in a function.

Suggested change
profanity.load_censor_words()
def initialize_profanity_filter():
profanity.load_censor_words()
if __name__ == "__main__":
initialize_profanity_filter()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment helpful?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment type correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment area correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What type of LLM test could this comment become?

  • 👍 - this comment is really good/important and we should always make it
  • 👎 - this comment is really bad and we should never make it
  • no reaction - don't turn this comment into an LLM test

scrubber.remove_detector("email")
body = scrubber.clean(calljson["text"])
# Run ai text through profanity filter
body = profanity.censor(calljson["text"])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Missing test for the new profanity filter functionality.

Ensure there is a test that verifies the profanity filter correctly censors inappropriate language and handles edge cases, such as mixed case, special characters, and multiple languages if applicable.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment helpful?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment type correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment area correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What type of LLM test could this comment become?

  • 👍 - this comment is really good/important and we should always make it
  • 👎 - this comment is really bad and we should never make it
  • no reaction - don't turn this comment into an LLM test

Comment on lines +321 to +322
if calljson["text"] != "you":
send_notifications(calljson, audiofile, destinations)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Missing test for the new condition to filter out empty notifications.

It's important to add a test to verify that notifications are not sent when the transcript is just 'you'. This test should cover scenarios where the transcript is exactly 'you', contains 'you' with additional text, and completely different text to ensure the condition works as expected.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment helpful?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment type correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the comment area correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What type of LLM test could this comment become?

  • 👍 - this comment is really good/important and we should always make it
  • 👎 - this comment is really bad and we should never make it
  • no reaction - don't turn this comment into an LLM test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants