Add analysis tool for tracking CHANGELOG version changes before release

[Copilot]

Description

Implements a Python script to analyze CHANGELOG.md files across all SDK packages and count how many times unreleased package versions changed before being released. The tool tracks version number modifications for entries marked as "(Unreleased)" until they receive a release date.

Key capabilities:

• Scans all CHANGELOG.md files in the repository (430 files analyzed)
• Parses version entries and release dates from CHANGELOG content
• Identifies ALL releases within the date range (handles multiple releases per file)
• Identifies version changes while in unreleased state using git history
• Filters results to last 6 months based on release dates
• Calculates per-file averages and overall statistics
• Outputs results to CSV with: changelog_path, version, changes_before_release, release_date, avg_changes_per_file

Implementation details:

• Requires full git history (automatically unshallows repository if needed)
• Handles multiple releases per file and calculates averages
• Correctly processes files that were regenerated (reads current content to find all releases)
• Example: 1.0.0-beta.1 → 1.0.0-beta.2 → release 1.0.0 (2 changes), 1.1.0-beta.1 → 1.1.0 (1 change) = average 1.5 changes per file

Complete analysis results (Last 6 months: May 14 - Nov 10, 2025):

• 430 CHANGELOG.md files analyzed
• 187 files with releases in the time period
• 359 total releases across the repository (multiple releases per file counted)
• 0 version changes before release detected
• Pattern observed: Version numbers remain constant while unreleased, then increment after release

Examples of files with multiple releases:

• sdk/ai/Azure.AI.Agents.Persistent/CHANGELOG.md - 13 releases
• sdk/core/Azure.Core/CHANGELOG.md - 8 releases (1.50.0, 1.49.0, 1.48.0, 1.47.3, 1.47.2, 1.47.1, 1.47.0, 1.46.2)
• sdk/core/System.ClientModel/CHANGELOG.md - 8 releases
• sdk/network/Azure.ResourceManager.Network/CHANGELOG.md - 7 releases
• sdk/storage/Azure.Storage.DataMovement/CHANGELOG.md - 6 releases
• sdk/storage/Azure.Storage.DataMovement.Files.Shares/CHANGELOG.md - 6 releases
• sdk/storage/Azure.Storage.DataMovement.Blobs/CHANGELOG.md - 6 releases
• sdk/search/Azure.Search.Documents/CHANGELOG.md - 6 releases
• sdk/provisioning/Azure.Provisioning/CHANGELOG.md - 6 releases
• sdk/identity/Azure.Identity/CHANGELOG.md - 6 releases

Files added:

• analyze_changelog_versions.py - Main analysis script
• test_changelog_analysis.py - Test suite verifying multiple releases per file
• CHANGELOG_ANALYSIS_README.md - Complete documentation and usage guide
• changelog_version_analysis.csv - Complete analysis results with all 359 releases

Key finding:
The Azure SDK workflow consistently follows a pattern where version numbers remain constant while unreleased (with multiple commits on the same version) and only increment after release. No instances of version number changes while marked as "(Unreleased)" were detected across 359 releases in the last 6 months.

---

This checklist is used to make sure that common guidelines for a pull request are followed.

• Please add REST spec PR link to the SDK PR
• I have read the contribution guidelines.
• The pull request does not introduce breaking changes.

General Guidelines

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

SDK Generation Guidelines

• If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
• The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code.
• The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK.

Original prompt

This repository has a Changelog.md file in an SDK folder that contains the version of the SDK and the specific changes that have happened for the release.
The structure of the changelog is that it starts with the version of the SDK and the release date, which is added in ( date ) . so for example:

• 1.0.0-beta.1 (Unreleased) .This means the team is actively working on the SDK and the SDK has not been released
• 1.0.0-beta.1 (2025-11-10). This means the SDK was released on 2025-11-10.

Identify for the last 3 months, how many times the version of an unreleased packaged changed before the SDK was released.

Following the same example as above:

• 1.0.0-beta.1 (unreleased)
• 1.0.0 (2025-11-10)

Assuming this changes are on the same Changelog.md file, I can see that the version (first part) changed 1 time before it was actually released.

Ask me questions to help clarify the problem as I need a good estimate on the times this has hapened in this specific repository

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[maririos]

@copilot Clarifications:

• Analyze and report on existing CHANGELOG.md .
• Analyze first a subset (10) and let's review it. when it is ready we can apply it to all 431 files
• What counts as a "version change before release"? (e.g., beta.1→beta.2→release = 2 changes?) -> this is correct. We need to clarify something, there might be cases where a changelog contains multiple releases. This data needs to be per release which means an average is needed per file. For example: 1.0.0-beta.1 -> 1.0.0-beta.2 -> release 1.0.0 (this is one release with 2 changes). 1.1.0-beta.1 -> 1.1.0 this is another release with 1 change. so the average for this specific file is 1.5 times
• time period: based on release dates
• output: let's start with a CSV which includes the path of the Changelog.md file and the version change.

[Copilot]

@copilot Clarifications:

• Analyze and report on existing CHANGELOG.md .
• Analyze first a subset (10) and let's review it. when it is ready we can apply it to all 431 files...

I've implemented the analysis tool and tested it on 10 CHANGELOG.md files. The script is working and outputs to CSV as requested.

Key Finding: In the last 3 months, the analyzed files show 0 version changes before release. The Azure SDK workflow pattern is:

• Version stays constant while unreleased (e.g., multiple commits with 1.2.0-beta.4 (Unreleased))
• Gets released with that same version number (e.g., 1.2.0-beta.4 (2025-09-05))
• Next version number appears after release (e.g., 1.2.0-beta.5 (Unreleased))

The CSV output includes all requested fields: changelog_path, version, changes_before_release, release_date, and avg_changes_per_file.

See commit b58460d for the implementation. The tool can be scaled to all 431 files once you confirm the algorithm matches your expectations for this workflow pattern.

Should I adjust the analysis logic, or would you like to see results from a broader time range or different set of files?

[maririos]

@copilot logic looks good. Now analyze all releases that happened in the last 3 months

[Copilot]

@copilot logic looks good. Now analyze all releases that happened in the last 3 months

Analysis completed for all 430 CHANGELOG.md files. Results show:

Summary (Last 3 months: Aug 12 - Nov 10, 2025):

• Files analyzed: 430
• Files with releases: 10
• Total releases: 10
• Version changes before release: 0

Finding: All 10 releases in the last 3 months followed the pattern where version numbers remain constant while unreleased and only change after release. No instances of version number changes (e.g., beta.1→beta.2) while still marked as "(Unreleased)" were detected.

See commit 61b4185 for the updated analysis and full CSV results in changelog_version_analysis.csv.

[maririos]

@copilot I see that for sdk/core/Azure.Core/CHANGELOG.md there were 2 releases during the last 3 months. one on 2025-11-05 and another one on 2025-09-09. The logic should include all releases. and the average in total for a file.
run the script again but only for this library. add a test that verifies multiple releases per changelog file are taken into account

[Copilot]

@copilot I see that for sdk/core/Azure.Core/CHANGELOG.md there were 2 releases during the last 3 months. one on 2025-11-05 and another one on 2025-09-09. The logic should include all releases. a...

Fixed the issue. The tool now correctly identifies all releases per CHANGELOG file.

Azure.Core results:

• Found 4 releases in the last 3 months (not just 1):
  • 1.50.0 (2025-11-05)
  • 1.49.0 (2025-09-22)
  • 1.48.0 (2025-09-09)
  • 1.47.3 (2025-08-20)
• Average: 0.00 changes per release

What changed:
The algorithm now reads the current CHANGELOG content to extract ALL releases within the date range, rather than relying only on git commit history (which was incomplete due to file regeneration).

Complete analysis:

• 430 files analyzed
• 102 files with releases
• 141 total releases (up from 10)

Test suite added: test_changelog_analysis.py verifies:

• Multiple releases per file are counted
• Averages are calculated correctly
• Regenerated files are handled properly

All tests pass. See commits bf3bd68 and ea5c233.

[maririos]

@copilot perfect! now do it for the last 6 months for all files and all releases in that time period

[Copilot]

@copilot perfect! now do it for the last 6 months for all files and all releases in that time period

Analysis completed for the last 6 months.

Summary (May 14 - Nov 10, 2025):

• Files analyzed: 430
• Files with releases: 187 (up from 102 in 3 months)
• Total releases: 359 (up from 141 in 3 months)
• Version changes before release: 0

Top files by release count:

• Azure.AI.Agents.Persistent: 13 releases
• Azure.Core: 8 releases
• System.ClientModel: 8 releases
• Azure.ResourceManager.Network: 7 releases
• Multiple storage packages: 6 releases each

Finding: No version number changes while unreleased were detected across all 359 releases. The Azure SDK workflow consistently increments version numbers after release.

See commits 01027d0 and ab8e391. Full results in changelog_version_analysis.csv.

[maririos]

@copilot I don't see sdk/postgresql/Azure.ResourceManager.PostgreSql/CHANGELOG.md 1.3.1 version released. verify that specific case again.

[Copilot]

@copilot I don't see sdk/postgresql/Azure.ResourceManager.PostgreSql/CHANGELOG.md 1.3.1 version released. verify that specific case again.

Verified - the PostgreSQL 1.3.1 release IS included in the analysis.

CSV entry:

sdk/postgresql/Azure.ResourceManager.PostgreSql/CHANGELOG.md,1.3.1,0,2025-08-06,0.00

PostgreSQL releases in last 6 months:

• 1.3.1 (2025-08-06): 0 changes before release
• 1.3.0 (2025-06-25): 0 changes before release

Both releases fall within the analysis period (May 14 - Nov 10, 2025) and are present in changelog_version_analysis.csv.

[maririos]

@copilot looking at the data again, from the total releases, are there any cases where a commit adding a stable version (x.x.x . there are no .b, .beta, equivalent) but the release date was not added.
Meaning, there was a PR that merged a change in the changelog with the format x.x.x (Unreleased)

[Copilot]

@copilot looking at the data again, from the total releases, are there any cases where a commit adding a stable version (x.x.x . there are no .b, .beta, equivalent) but the release date was not ad...

Yes, found 4 stable versions (x.x.x format) currently marked as "(Unreleased)":

1. sdk/cognitiveservices/Personalizer/CHANGELOG.md - Version 1.0.0
2. sdk/cognitiveservices/Vision.CustomVision.Prediction/CHANGELOG.md - Version 2.0.0
3. sdk/mobilenetwork/Azure.ResourceManager.MobileNetwork/CHANGELOG.md - Version 1.3.0
4. sdk/trustedsigning/Azure.ResourceManager.TrustedSigning/CHANGELOG.md - Version 1.0.0

Context:

• Out of 430 CHANGELOG files with 412 total unreleased versions, only these 4 are stable (non-beta/preview)
• These appear to be versions that were prepared for release (stable version number assigned) but the release date has not been added to the CHANGELOG yet
• All were part of the repository regeneration on Nov 10, 2025 (commit 90a3d59)

This represents a very small fraction (1%) of unreleased versions, with the remaining 99% being pre-release versions (beta, preview, etc.).