Skip to content

Commit dcec3c6

Browse files
authored
Multiple engines (the-benchmarker#3958)
* fix outdated rake config * remove duplicated config * wrong merge for php dockerfile * use phalcon 4.x * merge imi * update ci * add workerman config for imi * bad version required for phalcon * update ci config * lint * add swoole / workerman for mixphp * merge node and uwebsockets versions for chubbyjs * update ci config * fix * fix * add ssl layer in php containers * enable ci for engines * fix ci * fix chiubbyjs * use built-in php ssl module * missing config for nest * update sails config * lint * use phalcon 5.0.0alpha6 * fix roadrunner config for spiral * fix tower config * fix phalcon config * lint files * remove duplicate upload on container * show docker version used on CI * quotify paths for file uploaded to containers * downgrade imi to 2.0.11 * [Javascript] Add opine (the-benchmarker#4839) * unrestrict imi version * fix docker error due to duplicate upload onto containers * deduplicates framework build steps * use phalcon beta * avoid multiple gemfiles for ruby based projects * typo * [Python] Use various engines (the-benchmarker#4568) * start pythonization * run ci only for python * add blacksheep * add bottle * add cyclone * add gunicorn for django * disable daphne for blacksheep * add hypercorn for django * add asgi adapter for django * add asgi support for baize * add accurate engine for python frameworks * missing cheroo * add command for waitress / uwsgi * typo * complete python suite * remove daphne for blacksheep * typo * use safe navigation * remove meinheld patching * fix * remove outdated LOCs * fix CI for uwebsocket based frameworks * make gunicorn default choice for python * fix low-http-server * start working on golang * fix specs * pass on golang frameworks * fix deprecation warning on npm usage * Create codeql-analysis.yml * fix gramework * bad config for basolato * remove debug * add pcntl extension for workeman * typo * wrong container definition for php * configuration changes * use non lts version for uwsgi * beautify * fix go dockerfile * typos * remove codeql * restore old behavior * fix go dockerfiles * restore data * restore imi variants * restore laravel s * restore mixphp variants * restore one fpm * restore slim variants * restore sunrise router variants * restore yii swoole * fix aurora * typo * update go dockerfile * restore basolato * restore basolator * fix some php containers * typo * restore turbo polka * reduce PR size * reduce PR size * reduce PR size * restore fyrejet uwebsocket * beautify * upgrade node * reduce PR size * reduce PR size * typo * typo * typo * cleanup * cleanup * fix bottle * add engines * fix deprecated action * reduce ci tasks * reduce ci tasks * fix CI * increase CI tasks * increase CI tasks * increase CI tasks * increase CI tasks * increase CI tasks * increase CI tasks * increase CI tasks * typo * align config files * fix amber * typo * typo * missing static files into go containers * typo * missing files onto container * modify c dockerfile * missing dub.sdl for dlang * fix go builds * fix routejs config for uws * fix nginx config for php * missing ts compilation for adonis * use openjdk 17 * fix adonis config * fix haskell * fix yada * fix luminus * fix routerling * fix masonite * fix basolato * fix basicphp * fix workerman * fix antidot * fix chubbyphp * update dub * add redis / pcntl for formo * add pcntl for hyperf * beautify * cleanup * update * typo on config * pcntl is a php module not extension * lint * run I only on first engine * restore cpp / java frameworks * typos on some dockerfiles * fix java config * add properties files onto java containers * use jammy image for donkey * typo on dart config * typo * typo on java config * specify jvm level for kotlin based frameworks * add config file onto kotlin containers * add road runner config file onto containers * update camping config * html entity issue in r dockerfile * fix hug config * add yml files for ligt-4j * fix micronaut dockerfile * typo on config for spring * fix light-4j config * add gradle build files for micronaut * fix micronaut build * fix event moudle load for php * use first engine * update * add run.sh * update scorper * update
1 parent 74c85e4 commit dcec3c6

File tree

303 files changed

+23900
-21851
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

303 files changed

+23900
-21851
lines changed

.github/workflows/ci.yml

+15-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
name: CI
22

3+
concurrency:
4+
group: ${{ github.head_ref }}
5+
cancel-in-progress: true
6+
37
on:
48
- pull_request
59
jobs:
@@ -19,7 +23,7 @@ jobs:
1923
- name: Install deps
2024
run: bundle install --jobs 4 --retry 3
2125
- id: set-matrix
22-
run: echo "::set-output name=matrix::$(bundle exec rake ci:matrix)"
26+
run: echo "matrix=$(bundle exec rake ci:matrix)" >> "$GITHUB_OUTPUT"
2327
env:
2428
FILES: ${{ steps.file_changes.outputs.files }}
2529

@@ -42,18 +46,25 @@ jobs:
4246
- name: Install deps
4347
run: bundle install --jobs 4 --retry 3
4448

49+
- name: Docker version
50+
run: docker -v
51+
4552
- name: Configure
4653
run: bundle exec rake config
4754

4855
- name: Build
4956
run: |
50-
cd ${{ matrix.directory }}
51-
make build -f .Makefile
57+
make "build.${ENGINE}" -f "${DIRECTORY}/.Makefile"
58+
env:
59+
DIRECTORY: ${{ matrix.directory }}
60+
ENGINE: ${{ matrix.engine }}
5261

5362
- name: Wait
5463
run: sleep 1
5564

5665
- name: Test
5766
run: bundle exec rspec .spec
5867
env:
59-
DIRECTORY: ${{ matrix.directory }}
68+
LANGUAGE: ${{ matrix.language }}
69+
FRAMEWORK: ${{ matrix.framework }}
70+
ENGINE: ${{ matrix.engine }}

.gitignore

+3-3
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,12 @@ Gemfile.lock
1616

1717
# Temporary files
1818
FRAMEWORKS.yaml
19-
*/*/.Dockerfile
20-
*/*/Dockerfile
19+
*/*/.Dockerfile*
2120
*/*/.Makefile
2221
*/*/ip.txt
2322
venv
2423
*/*/cid.txt
24+
*/.Makefile
2525

2626
/lib/**/*.cr
2727
/vendor
@@ -41,4 +41,4 @@ obj
4141
[Dd]ebug/
4242
[Rr]elease/
4343
*.fsproj
44-
*.fsproj.user
44+
*.fsproj.user

.rubocop.yml

+21
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,31 @@ require:
44

55
AllCops:
66
NewCops: enable
7+
TargetRubyVersion: 2.7
8+
Exclude:
9+
- Rakefile
710

811
Style/Documentation:
912
Enabled: false
1013

1114
Naming/AccessorMethodName:
1215
Exclude:
1316
- ruby/rack-routing/app/route_handler.rb
17+
18+
Lint/ShadowingOuterLocalVariable:
19+
Exclude:
20+
- .tasks/*.rake
21+
22+
Metrics/BlockLength:
23+
Exclude:
24+
- .tasks/*.rake
25+
26+
Lint/AmbiguousBlockAssociation:
27+
Exclude:
28+
- .tasks/*.rake
29+
Layout/LineLength:
30+
Exclude:
31+
- .tasks/*.rake
32+
Metrics/ParameterLists:
33+
Exclude:
34+
- .tasks/*.rake

.spec/route_spec.rb

+17-19
Original file line numberDiff line numberDiff line change
@@ -1,48 +1,46 @@
11
# frozen_string_literal: true
22

3-
require 'net/http'
3+
require "net/http"
44

5-
require_relative 'spec_helper'
5+
require_relative "spec_helper"
66

7-
RSpec.describe 'routes' do
8-
let!(:http) { address(ENV['DIRECTORY']) }
7+
RSpec.describe "routes" do
8+
context "GET : /" do
9+
subject(:response) { http.request(Net::HTTP::Get.new("/")) }
910

10-
context 'GET : /' do
11-
subject(:response) { http.request(Net::HTTP::Get.new('/')) }
12-
13-
it 'returns successfully' do
11+
it "returns successfully" do
1412
expect(response).to be_a(Net::HTTPSuccess)
1513
end
1614

17-
it 'returns an empty string' do
15+
it "returns an empty string" do
1816
expect(response.body.to_s).to be_empty
1917
end
2018
end
2119

22-
context 'GET : /user/0' do
23-
subject(:response) { http.request(Net::HTTP::Get.new('/user/0')) }
20+
context "GET : /user/0" do
21+
subject(:response) { http.request(Net::HTTP::Get.new("/user/0")) }
2422

25-
it 'returns successfully' do
23+
it "returns successfully" do
2624
expect(response).to be_a(Net::HTTPSuccess)
2725
end
2826

29-
it 'returns the expected param' do
30-
expect(response.body.to_s).to eq('0')
27+
it "returns the expected param" do
28+
expect(response.body.to_s).to eq("0")
3129
end
3230
end
3331

34-
context 'POST : /user' do
35-
let!(:request) { Net::HTTP::Post.new('/user') }
32+
context "POST : /user" do
33+
let!(:request) { Net::HTTP::Post.new("/user") }
3634

37-
before { request['Content-Type'] = 'text/plain' }
35+
before { request["Content-Type"] = "text/plain" }
3836

3937
subject(:response) { http.request(request) }
4038

41-
it 'returns successfully' do
39+
it "returns successfully" do
4240
expect(response).to be_a(Net::HTTPSuccess)
4341
end
4442

45-
it 'returns an empty string' do
43+
it "returns an empty string" do
4644
expect(response.body.to_s).to be_empty
4745
end
4846
end

.spec/spec_helper.rb

+4-5
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# frozen_string_literal: true
22

3-
require 'net/http'
4-
require 'yaml'
3+
require "net/http"
4+
require "yaml"
55

66
RSpec.configure do |config|
77
config.expect_with :rspec do |expectations|
@@ -15,8 +15,7 @@
1515
config.shared_context_metadata_behavior = :apply_to_host_groups
1616
end
1717

18-
def address(name)
19-
language, framework = name.split('/')
20-
ip = File.read(File.join(language, framework, 'ip.txt')).strip
18+
def http
19+
ip = File.read(File.join(ENV["LANGUAGE"], ENV["FRAMEWORK"], "ip-#{ENV["ENGINE"]}.txt")).strip
2120
Net::HTTP.new(ip, 3000)
2221
end

.tasks/ci.rake

+13-9
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ require 'json'
44

55
namespace :ci do
66
task :matrix do
7-
files = JSON.parse(ENV['FILES'])
7+
files = JSON.parse(ENV.fetch('FILES', nil))
88
matrix = { include: [] }
99

1010
files = Dir.glob('*/*/config.yaml') if files.include?('data.json')
@@ -14,21 +14,25 @@ namespace :ci do
1414
.map { |path| path.split(File::SEPARATOR).shift }
1515
.flat_map { |language| Dir.glob(File.join(language, '*', 'config.yaml')) }
1616

17-
files.take(256).each do |file|
17+
files.each do |file|
1818
next if file.start_with?('.')
19-
2019
next if file.count(File::SEPARATOR) < 2
2120

2221
language, framework, = file.split(File::SEPARATOR)
2322

24-
next if matrix[:include].detect do |row|
25-
row[:framework] == framework
26-
end
23+
config = get_config_from(File.join(Dir.pwd, language, framework))
24+
25+
engine = config.dig('framework', 'engines')&.first
2726

28-
matrix[:include] << { language: language, framework: framework,
29-
directory: File.join(language, framework) }
27+
if engine
28+
matrix[:include] << { language: language, framework: framework, directory: File.join(language, framework),
29+
engine: engine }
30+
else
31+
warn "Configuration for #{language}/#{framework} is not correct"
32+
end
3033
end
31-
warn matrix.to_json
34+
35+
matrix[:include] = matrix[:include].take(256)
3236
puts matrix.to_json
3337
end
3438
end

.tasks/cloud

+2-2
Original file line numberDiff line numberDiff line change
@@ -47,10 +47,10 @@ namespace :cloud do
4747
end
4848
end
4949

50-
if config.key?('before_command')
50+
if config.key?('bootstrap')
5151
commands = config['cloud']['config']['runcmd'] || []
5252
config['cloud']['config']['runcmd'] = []
53-
config['before_command'].each do |cmd|
53+
config['bootstrap'].each do |cmd|
5454
config['cloud']['config']['runcmd'] << cmd
5555
end
5656
commands.each do |cmd|

.tasks/collect.rake

+11-5
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ PIPELINE = {
1010
POST: File.join(Dir.pwd, 'pipeline_post.lua')
1111
}.freeze
1212

13-
def insert(db, framework_id, metric, value, concurrency_level_id)
13+
def insert(db, framework_id, metric, value, concurrency_level_id, engine_id)
1414
res = db.query('INSERT INTO keys (label) VALUES ($1) ON CONFLICT (label) DO UPDATE SET label = $1 RETURNING id',
1515
[metric])
1616

@@ -19,8 +19,8 @@ def insert(db, framework_id, metric, value, concurrency_level_id)
1919
res = db.query('INSERT INTO values (key_id, value) VALUES ($1, $2) RETURNING id', [metric_id, value])
2020
value_id = res.first['id']
2121

22-
db.query('INSERT INTO metrics (value_id, framework_id, concurrency_id) VALUES ($1, $2, $3)',
23-
[value_id, framework_id, concurrency_level_id])
22+
db.query('INSERT INTO metrics (value_id, framework_id, concurrency_id, engine_id) VALUES ($1, $2, $3, $4)',
23+
[value_id, framework_id, concurrency_level_id, engine_id])
2424
end
2525

2626
task :collect do
@@ -31,8 +31,9 @@ task :collect do
3131
concurrencies = ENV.fetch('CONCURRENCIES', '10')
3232
routes = ENV.fetch('ROUTES', 'GET:/')
3333
database = ENV.fetch('DATABASE_URL') { raise 'please provide a DATABASE_URL (pg only)' }
34+
hostname = ENV.fetch('HOSTNAME')
35+
engine = ENV.fetch('ENGINE')
3436

35-
hostname = File.read(File.join(Dir.pwd, language, framework, 'ip.txt')).strip
3637
`wrk -H 'Connection: keep-alive' -d 5s -c 8 --timeout 8 -t #{threads} http://#{hostname}:3000`
3738
`wrk -H 'Connection: keep-alive' -d #{duration}s -c 256 --timeout 8 -t #{threads} http://#{hostname}:3000`
3839

@@ -43,6 +44,11 @@ task :collect do
4344
)
4445
language_id = res.first['id']
4546

47+
res = db.query(
48+
'INSERT INTO engines (label) VALUES ($1) ON CONFLICT (label) DO UPDATE SET label = $1 RETURNING id', [engine]
49+
)
50+
engine_id = res.first['id']
51+
4652
res = db.query(
4753
'INSERT INTO frameworks (language_id, label) VALUES ($1, $2) ON CONFLICT (language_id, label) DO UPDATE SET label = $2 RETURNING id', [
4854
language_id, framework
@@ -83,7 +89,7 @@ task :collect do
8389

8490
info = lua_output.split(',')
8591
lua_keys.each_with_index do |key, index|
86-
insert(db, framework_id, key, info[index].to_d, concurrency_level_id)
92+
insert(db, framework_id, key, info[index].to_d, concurrency_level_id, engine_id)
8793
end
8894
end
8995
end

.tasks/db.rake

+11-9
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,17 @@ end
3030

3131
namespace :db do
3232
task :check_failures do
33-
results = JSON.load(File.read('data.json'))
34-
frameworks = results['metrics'].filter_map {|row|row['framework_id'] if row['label'] == "total_requests_per_s" && row["value"] == 0 }
35-
STDOUT.puts results['frameworks'].filter_map {|row|row['label'] if frameworks.include? (row["id"]) }
33+
results = JSON.parse(File.read('data.json'))
34+
frameworks = results['metrics'].filter_map do |row|
35+
row['framework_id'] if row['label'] == 'total_requests_per_s' && (row['value']).zero?
36+
end
37+
$stdout.puts results['frameworks'].filter_map { |row| row['label'] if frameworks.include?(row['id']) }
3638
end
3739
task :raw_export do
3840
raise 'Please provide a database' unless ENV['DATABASE_URL']
3941

4042
data = { metrics: [], frameworks: [], languages: [] }
41-
db = PG.connect(ENV['DATABASE_URL'])
43+
db = PG.connect(ENV.fetch('DATABASE_URL', nil))
4244
db.exec("select row_to_json(t) from (#{SQL}) as t") do |result|
4345
result.each do |row|
4446
info = JSON.parse(row['row_to_json'], symbolize_names: true)
@@ -52,11 +54,11 @@ namespace :db do
5254
scheme = 'http' if framework_config['framework'].key?('unsecure')
5355
website = framework_config['framework']['website']
5456
if website.nil?
55-
website = if framework_config['framework'].key?('github')
56-
"github.com/#{framework_config['framework']['github']}"
57-
elsif framework_config['framework'].key?('gitlab')
58-
"gitlab.com/#{framework_config['framework']['gitlab']}"
59-
end
57+
website = if framework_config['framework'].key?('github')
58+
"github.com/#{framework_config['framework']['github']}"
59+
elsif framework_config['framework'].key?('gitlab')
60+
"gitlab.com/#{framework_config['framework']['gitlab']}"
61+
end
6062
end
6163
unless data[:frameworks].map { |row| row[:id] }.to_a.include?(framework_id)
6264
data[:frameworks] << {

Makefile

-6
This file was deleted.

0 commit comments

Comments
 (0)