fix: deduplicate access logs before batch insert

[moshloop]

Summary by CodeRabbit

• Bug Fixes
  • Resolved duplicate log entries by deduplicating access logs before saving to the database, ensuring data integrity and preventing conflicts during database operations.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

Walkthrough

Adds deduplication logic for ConfigAccessLog entries in the database update process. Before saving logs, entries are merged based on a composite key (ConfigID, ExternalUserID, ScraperID), retaining only the entry with the most recent CreatedAt timestamp per key.

Changes

Cohort / File(s)	Summary
ConfigAccessLog Deduplication db/update.go	Introduces deduplication logic that consolidates duplicate ConfigAccessLog entries using a composite key before persistence, keeping the most recently created entry for each key combination.

Possibly related PRs

• chore: increment count column in config access logs #1912: Modifies the ON CONFLICT upsert strategy for config_access_logs to increment count and update fields, complementing this deduplication approach.
• fix: use correct primary key columns in config_access_logs OnConflict #1877: Addresses conflict handling for the same composite key (config_id, external_user_id, scraper_id) through upsert improvements and SaveConfigAccessLog additions.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: deduplicate access logs before batch insert' directly and clearly describes the main change: deduplication of ConfigAccessLog entries before database insertion.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch pr/fix-duplicate-logs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}