Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update benchmarks script to use final data file instead of benchmark data files #2199

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Update benchmarks script to use final data file instead of benchmark data files #2199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
95fb8e5
Update benchmarks script to use final data file instead of benchmark …
itsmesamster Feb 10, 2025
03e03fa
Update benchmarks to be compatible with cli location changes
itsmesamster Feb 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
CSC_FOR_PULL_REQUEST: true
strategy:
matrix:
os: [ubuntu-20.04, windows-latest, macos-13, macOS-latest]
os: [ubuntu-latest, windows-latest, macos-13, macOS-latest]
rust: [stable]
steps:
- name: Checkout
Expand Down
35 changes: 25 additions & 10 deletions scripts/benchmarks/process.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
ENV['REPO_NAME'] = 'chipmunk'

COMMANDS = [
'cargo install --path=cli/development-cli',
"cargo install --path=cli_path",
'cargo chipmunk test wrapper -p -s spec/build/spec/_session.benchmark.spec.js -u print'
]

Expand Down Expand Up @@ -76,7 +76,6 @@ def set_environment_vars
def clone_and_setup_repo(branch_or_tag_name, temp_dir)
system("git clone --depth 1 --branch #{branch_or_tag_name} https://github.com/#{ENV['REPO_OWNER']}/#{ENV['REPO_NAME']}.git #{temp_dir}")
FileUtils.cp_r("#{SHELL_SCRIPT_PATH}/.", "#{temp_dir}/#{SHELL_SCRIPT_PATH}/.", verbose: true)
FileUtils.cp_r("cli/development-cli", "#{temp_dir}/", verbose: true)
end

def process_release_or_pr(branch_or_tag_name, identifier, env_vars)
Expand All @@ -90,7 +89,9 @@ def process_release_or_pr(branch_or_tag_name, identifier, env_vars)
ENV['PERFORMANCE_RESULTS'] = "Benchmark_#{identifier}.json"
system("corepack enable")
system("yarn cache clean")
system(COMMANDS[0])
path = Dir.exist?('cli/development-cli') ? 'cli/development-cli' : 'cli'
puts "Path is #{path}"
system(COMMANDS[0].gsub("cli_path",path))

next unless File.exist?("#{SHELL_SCRIPT_PATH}/#{env_vars['JASMIN_TEST_CONFIGURATION'].gsub('./spec/', '')}")

Expand Down Expand Up @@ -134,21 +135,35 @@ def collect_latest_benchmark_data(directory)
end
end

def update_performance_data(data, data_file_path)
test_data = data.each_with_object({}) do |benchmark, hash|
def update_performance_data(new_data, data_file_path)
existing_data = File.exist?(data_file_path) ? JSON.parse(File.read(data_file_path)) : {}

# Transforming the new data into the expected format
new_test_data = new_data.each_with_object({}) do |benchmark, hash|
benchmark[:data].each do |entry|
test_name = entry['name']
actual_value = entry['actual']
release = benchmark[:file_name].gsub("Benchmark_", "").gsub(".json", "")

hash[test_name] ||= []
hash[test_name] << { release: release, actual_value: actual_value }
hash[test_name] << { "release" => release, "actual_value" => actual_value }
end
end

# Merging new data with the existing data
new_test_data.each do |test_name, new_entries|
existing_data[test_name] ||= []

new_entries.each do |new_entry|
release = new_entry["release"]
# Removing existing entry for the same release before appending the new one
existing_data[test_name].reject! { |entry| entry["release"] == release }
existing_data[test_name] << new_entry
end
end
test_data = test_data.to_json
puts "Data written to #{data_file_path}\n#{test_data}"
File.write(data_file_path, test_data)
puts "Benchmark data created successfully!"

File.write(data_file_path, JSON.pretty_generate(existing_data))
puts "Updated benchmark data written to #{data_file_path}."
end

arg = ARGV[0] || usage
Expand Down