Skip to content

Commit d28158e

Browse files
committed
mem-ruby,scons: Add scons option for multiple protocols
This change does many things, but they must all be atomically done. **USER FACING CHANGE**: The Ruby protocols in Kconfig have changed names (they are now the same case as the SLICC file names). So, after this commit, your build configurations need to be updated. You can do so by running `scons menuconfig <build dir>` and selecting the right ruby options. Alternatively, if you're using a `build_opts` file, you can run `scons defconfig build/<ISA> build_opts/<ISA>` which should update your config correctly. Detailed changes are described below. Kconfig changes: - Kconfig files in ruby now must all be declared in the ruby/Kconfig file - All of the protocol names are changed to match their slicc file names including the case - A new option is available called "Use multiple protocols" which should be selected if multiple protocols are selected. This is only used to set the PROTOCOL variable to "MULTIPLE" when in multiple mode. - The PROTOCOL variable can now be "MULTIPLE" which means it will be ignored. If it's not "MULTIPLE" then it holds the "main" protocol, which is necessary for backwards compatibility with the Ruby.py files. Ruby config changes: To make this change backwards compatible with Ruby.py, this change adds a new "protocol" config called MULTIPLE.py which is used to allow the user to set a "--protocol" option on the command line. This is only needed if you are using a gem5 binary with multiple protocols but need to use Ruby.py. stdlib changes: - Make the coherence protocol file behave like the ISA file - Add a function to get the coherence protocol from the `CacheHierarchy` like we do with the ISA in the `Processor`. - Use this function where `get_runtime_coherence_protocol` was used - Update the requires code to work with the ne CoherenceProtocol - Fix a typo in the AMD Hammer name and also add the missing MSI protocol Scons changes: - In Ruby we now gather up all of the protocols and build them all if there are multiple protocols - There's some bending over backwards to tell the user if they are using an out of date gem5.build/config file and how to update it - Note that multiple ruby protocols adds a significant amount of time to the build since we have to run slicc twice for each file. build_opts: - Update all files with new names - Add a new NULL_All_Ruby that will be used for testing Signed-off-by: Jason Lowe-Power <[email protected]>
1 parent 42dc2f1 commit d28158e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+343
-138
lines changed

RELEASE-NOTES.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ The most notable changes are:
1515
* Removes the RubySlicc_interfaces.slicc file from the SLICC includes of
1616
every protocol.
1717
* Changes required: If you have a custom protocol, you will need to remove the line `include "RubySlicc_interfaces.slicc"` from your .slicc file.
18-
18+
* Updates the build configurations variables
19+
* **USER FACING CHANGE**: The Ruby protocols in Kconfig have changed names (they are now the same case as the SLICC file names). So, after this commit, your build configurations need to be updated. You can do so by running `scons menuconfig <build dir>` and selecting the right ruby options. Alternatively, if you're using a `build_opts` file, you can run `scons defconfig build/<ISA> build_opts/<ISA>` which should update your config correctly.
1920

2021

2122
# Version 24.0

build_opts/ALL

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MESI_TWO_LEVEL=y
2+
PROTOCOL="MESI_Two_Level"
3+
RUBY_PROTOCOL_MESI_Two_Level=y
34
BUILD_ISA=y
45
USE_ARM_ISA=y
56
USE_MIPS_ISA=y

build_opts/ARM

+1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
BUILD_ISA=y
22
USE_ARM_ISA=y
33
RUBY=y
4+
PROTOCOL="CHI"
45
RUBY_PROTOCOL_CHI=y

build_opts/ARM_MESI_Three_Level

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
BUILD_ISA=y
22
USE_ARM_ISA=y
33
RUBY=y
4-
RUBY_PROTOCOL_MESI_THREE_LEVEL=y
4+
PROTOCOL="MESI_Three_Level"
5+
RUBY_PROTOCOL_MESI_Three_Level=y

build_opts/ARM_MESI_Three_Level_HTM

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
BUILD_ISA=y
22
USE_ARM_ISA=y
33
RUBY=y
4-
RUBY_PROTOCOL_MESI_THREE_LEVEL_HTM=y
4+
PROTOCOL="MESI_Three_Level_HTM"
5+
RUBY_PROTOCOL_MESI_Three_Level_HTM=y

build_opts/ARM_MOESI_hammer

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
BUILD_ISA=y
22
USE_ARM_ISA=y
33
RUBY=y
4-
RUBY_PROTOCOL_MOESI_HAMMER=y
4+
PROTOCOL="MOESI_hammer"
5+
RUBY_PROTOCOL_MOESI_hammer=y

build_opts/Garnet_standalone

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_GARNET_STANDALONE=y
2+
PROTOCOL="Garnet_standalone"
3+
RUBY_PROTOCOL_Garnet_standalone=y

build_opts/MIPS

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y
34
BUILD_ISA=y
45
USE_MIPS_ISA=y

build_opts/NULL

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y

build_opts/NULL_All_Ruby

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
RUBY=y
2+
USE_MULTIPLE_PROTOCOLS=y
3+
PROTOCOL="MULTIPLE"
4+
RUBY_PROTOCOL_MOESI_AMD_Base=y
5+
RUBY_PROTOCOL_MESI_Two_Level=y
6+
RUBY_PROTOCOL_MESI_Three_Level=y
7+
RUBY_PROTOCOL_MESI_Three_Level_HTM=y
8+
RUBY_PROTOCOL_MI_example=y
9+
RUBY_PROTOCOL_MOESI_CMP_directory=y
10+
RUBY_PROTOCOL_MOESI_CMP_token=y
11+
RUBY_PROTOCOL_MOESI_hammer=y
12+
RUBY_PROTOCOL_Garnet_standalone=y
13+
RUBY_PROTOCOL_CHI=y
14+
RUBY_PROTOCOL_MSI=y

build_opts/NULL_MESI_Two_Level

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MESI_TWO_LEVEL=y
2+
PROTOCOL="MESI_Two_Level"
3+
RUBY_PROTOCOL_MESI_Two_Level=y

build_opts/NULL_MOESI_CMP_directory

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MOESI_CMP_DIRECTORY=y
2+
PROTOCOL="MOESI_CMP_directory"
3+
RUBY_PROTOCOL_MOESI_CMP_directory=y

build_opts/NULL_MOESI_CMP_token

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MOESI_CMP_TOKEN=y
2+
PROTOCOL="MOESI_CMP_token"
3+
RUBY_PROTOCOL_MOESI_CMP_token=y

build_opts/NULL_MOESI_hammer

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MOESI_HAMMER=y
2+
PROTOCOL="MOESI_hammer"
3+
RUBY_PROTOCOL_MOESI_hammer=y

build_opts/POWER

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y
34
BUILD_ISA=y
45
USE_POWER_ISA=y

build_opts/RISCV

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y
34
BUILD_ISA=y
45
USE_RISCV_ISA=y

build_opts/SPARC

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y
34
BUILD_ISA=y
45
USE_SPARC_ISA=y

build_opts/VEGA_X86

+1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
RUBY=y
22
NUMBER_BITS_PER_SET=128
3+
PROTOCOL="GPU_VIPER"
34
RUBY_PROTOCOL_GPU_VIPER=y
45
BUILD_ISA=y
56
USE_X86_ISA=y

build_opts/X86

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
RUBY=y
22
NUMBER_BITS_PER_SET=128
3-
RUBY_PROTOCOL_MESI_TWO_LEVEL=y
3+
PROTOCOL="MESI_Two_Level"
4+
RUBY_PROTOCOL_MESI_Two_Level=y
45
BUILD_ISA=y
56
USE_X86_ISA=y

build_opts/X86_MESI_Two_Level

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
RUBY=y
22
NUMBER_BITS_PER_SET=128
3-
RUBY_PROTOCOL_MESI_TWO_LEVEL=y
3+
PROTOCOL="MESI_Two_Level"
4+
RUBY_PROTOCOL_MESI_Two_Level=y
45
BUILD_ISA=y
56
USE_X86_ISA=y

build_opts/X86_MI_example

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MI_EXAMPLE=y
2+
PROTOCOL="MI_example"
3+
RUBY_PROTOCOL_MI_example=y
34
BUILD_ISA=y
45
USE_X86_ISA=y

build_opts/X86_MOESI_AMD_Base

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
RUBY=y
2-
RUBY_PROTOCOL_MOESI_AMD_BASE=y
2+
PROTOCOL="MOESI_AMD_Base"
3+
RUBY_PROTOCOL_MOESI_AMD_Base=y
34
BUILD_ISA=y
45
USE_X86_ISA=y

configs/learning_gem5/part3/msi_caches.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646

4747
class MyCacheSystem(RubySystem):
4848
def __init__(self):
49-
if buildEnv["PROTOCOL"] != "MSI":
49+
if not "RUBY_PROTOCOL_MSI" in buildEnv:
5050
fatal("This system assumes MSI from learning gem5!")
5151

5252
super().__init__()

configs/learning_gem5/part3/ruby_caches_MI_example.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848

4949
class MyCacheSystem(RubySystem):
5050
def __init__(self):
51-
if buildEnv["PROTOCOL"] != "MI_example":
51+
if "MI_example" not in buildEnv:
5252
fatal("This system assumes MI_example!")
5353

5454
super().__init__()

configs/ruby/MULTIPLE.py

+89
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Copyright 2024 (c) Jason Lowe-Power
2+
# All rights reserved.
3+
#
4+
# Redistribution and use in source and binary forms, with or without
5+
# modification, are permitted provided that the following conditions are
6+
# met: redistributions of source code must retain the above copyright
7+
# notice, this list of conditions and the following disclaimer;
8+
# redistributions in binary form must reproduce the above copyright
9+
# notice, this list of conditions and the following disclaimer in the
10+
# documentation and/or other materials provided with the distribution;
11+
# neither the name of the copyright holders nor the names of its
12+
# contributors may be used to endorse or promote products derived from
13+
# this software without specific prior written permission.
14+
#
15+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
16+
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
17+
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
18+
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
19+
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
20+
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
21+
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
22+
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
23+
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24+
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26+
27+
import importlib
28+
29+
from m5.defines import buildEnv
30+
31+
32+
def define_options(parser):
33+
print("**************IMPORTANT*******************************")
34+
print("This gem5 binary is configured with multiple protocols.")
35+
print("To use this gem5 binary, you must use the standard library.")
36+
print("The Ruby.py file is deprecated.")
37+
print("You can use --protocol to specify the protocol you want to use.")
38+
print("IMPORTANT: THIS WILL BE REMOVED IN THE FUTURE.")
39+
40+
available_protocols = [
41+
protocol[len("RUBY_PROTOCOL_") :]
42+
for protocol in buildEnv.keys()
43+
if protocol.startswith("RUBY_PROTOCOL_")
44+
]
45+
46+
parser.add_argument(
47+
"--protocol",
48+
type=str,
49+
help="Specify the protocol you want to use.",
50+
required=True,
51+
choices=available_protocols,
52+
)
53+
54+
# Note: we can't rely on the options yet because they haven't been parsed.
55+
from sys import argv
56+
57+
found = False
58+
for arg in argv:
59+
if arg.startswith("--protocol"):
60+
found = True
61+
if "=" in arg:
62+
protocol = arg[len("--protocol=") :]
63+
else:
64+
protocol = argv[argv.index(arg) + 1]
65+
if not found:
66+
print(
67+
"ERROR: You must specify a protocol with --protocol <protocol>. "
68+
"or --protocol=<protocol>"
69+
)
70+
exit(1)
71+
72+
protocol_module = importlib.import_module(f".{protocol}", package="ruby")
73+
protocol_module.define_options(parser)
74+
buildEnv["PROTOCOL"] = protocol
75+
76+
77+
# Note: This function isn't used because buildEnv["PROTOCOL"] is set
78+
# in define_options. However, this is here just in case.
79+
def create_system(options, *args, **kwargs):
80+
protocol = options.protocol
81+
print("Using protocol: ", protocol)
82+
print("WARNING: The Ruby.py file is deprecated.")
83+
84+
protocol_module = importlib.import_module(f".{protocol}", package="ruby")
85+
(cpu_sequencers, dir_cntrls, topology) = protocol_module.create_system(
86+
options, *args, **kwargs
87+
)
88+
89+
return (cpu_sequencers, dir_cntrls, topology)

src/Kconfig

-1
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,6 @@ config USE_EFENCE
4444

4545
rsource "base/Kconfig"
4646
rsource "mem/ruby/Kconfig"
47-
rsource "learning_gem5/part3/Kconfig"
4847
rsource "proto/Kconfig"
4948
rsource "dev/net/Kconfig"
5049
rsource "arch/Kconfig"

src/arch/arm/SConscript

+1-1
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ Source('insts/sve_mem.cc', tags='arm isa')
7777
Source('insts/vfp.cc', tags='arm isa')
7878
Source('insts/crypto.cc', tags='arm isa')
7979
Source('insts/tme64.cc', tags='arm isa')
80-
if env['CONF']['PROTOCOL'] == 'MESI_Three_Level_HTM':
80+
if env['CONF']['RUBY_PROTOCOL_MESI_Three_Level_HTM']:
8181
Source('insts/tme64ruby.cc', tags='arm isa')
8282
else:
8383
Source('insts/tme64classic.cc', tags='arm isa')

src/learning_gem5/part3/Kconfig

+3-6
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,7 @@
2323
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
2424
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
2525

26-
config PROTOCOL
27-
default "MSI" if RUBY_PROTOCOL_MSI
2826

29-
cont_choice "Ruby protocol"
30-
config RUBY_PROTOCOL_MSI
31-
bool "MSI"
32-
endchoice
27+
config RUBY_PROTOCOL_MSI
28+
bool "MSI"
29+
default n

src/mem/ruby/Kconfig

+21-8
Original file line numberDiff line numberDiff line change
@@ -28,15 +28,28 @@ menu "Ruby"
2828
bool "Enable"
2929

3030
if RUBY
31+
menu "Ruby protocols"
32+
33+
config USE_MULTIPLE_PROTOCOLS
34+
bool 'Use multiple protocols'
35+
default y
36+
3137
config PROTOCOL
32-
string
38+
string "The 'main' protocol"
39+
default "MULTIPLE" if USE_MULTIPLE_PROTOCOLS
40+
help
41+
This shows up as `buildEnv["PROTOCOL"]`` for backwards
42+
compatibility, mostly in the Ruby.py config scripts.
43+
Otherwise, this is ignored.
44+
If this is set to anything when build with MULTIPLE,
45+
it will be ignored unless using Ruby.py, in which case
46+
the protocol specified here will appear as the only protocol.
47+
Note: This is case sensitive.
3348

34-
config NEED_PARTIAL_FUNC_READS
35-
bool
49+
rsource "protocol/Kconfig"
50+
rsource "../../learning_gem5/part3/Kconfig"
3651

37-
choice "Ruby protocol"
38-
prompt "Ruby protocol"
39-
endchoice
52+
endmenu
4053

4154
config SLICC_HTML
4255
bool 'Create HTML files'
@@ -45,6 +58,6 @@ menu "Ruby"
4558
int 'Max elements in set'
4659
default 64
4760
endif
48-
endmenu
4961

50-
rsource "protocol/Kconfig"
62+
63+
endmenu

src/mem/ruby/SConscript

+1-1
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ MakeInclude('structures/PersistentTable.hh')
113113
MakeInclude('structures/RubyPrefetcher.hh')
114114
MakeInclude('structures/RubyPrefetcherProxy.hh')
115115
MakeInclude('structures/TBEStorage.hh')
116-
if env['CONF']['PROTOCOL'] == 'CHI':
116+
if env['CONF']['RUBY_PROTOCOL_CHI']:
117117
MakeInclude('structures/MN_TBEStorage.hh')
118118
MakeInclude('structures/MN_TBETable.hh')
119119
MakeInclude('structures/TBETable.hh')

0 commit comments

Comments
 (0)