Skip to content

Commit 4bee682

Browse files
authored
NO-JIRA: performance profile: internal cleanup (#1359)
* profilecreator: dissolve helpers into subpackages reorganize the current helper.go dissolving it into specific subpackages. The goal is tstreamline the code layout without changes in behavior. Signed-off-by: Francesco Romani <[email protected]> * profilecreator: simplify package layout the `cmd/pkg` layout is awkward. Canonically: - code used only by a tool should sit in `internal/` - code which is/can be exported should sit in `pkg`. However, in case of nontrivial command line utilities, it's a widespread practice to have `pkg/cmd` (or `pkg/commands`) trees. This enable third party utilities to easily embed other utilities maximizing code reuse. We streamline the code layout with trivial code movements - no intended changes of behavior. The net result is now `cmd/performance-profile-creator` is now minified, and holds only the entry point source code file. Signed-off-by: Francesco Romani <[email protected]> * profilecreator: chore: refactor profilecreator.go extract code which deals with mustgather or ghw in their own source files. Trivial code movement with no changes in behavior. Signed-off-by: Francesco Romani <[email protected]> * profilecreator: improvements in the `info` command - add pure text output command. By default (and it's very hard to change, if we ever should) log messages go on stderr. We want the output on stdout. Added flag `--text` to enable it - internal cleanup and code reorganization. In log mode, log all the message in one go, which is easier to manage. No intended change in behavior. Signed-off-by: Francesco Romani <[email protected]> * profilecreator: add and use Alert function We used the `logrus` package internally, but minimally and pretty much only to notify processing status to the user. This code was thus the only remaining consumer of that package, which we don't really need. Add a neutral (nor log, nor warning) minimal abstraction `Alert`, similar in spirit to `ghw`, to be used when we need to notify the user about processing status. Make the backend easy to replace to make integration of this code easier. We don't see this happening, but the extra cost is negligible. Remove the direct dependency to `logrus`. Note we still have quite many indirect dependencies pulling it in. Signed-off-by: Francesco Romani <[email protected]> --------- Signed-off-by: Francesco Romani <[email protected]>
1 parent e768e3a commit 4bee682

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+653
-3667
lines changed

cmd/performance-profile-creator/main.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ package main
1919
import (
2020
"os"
2121

22-
"github.com/openshift/cluster-node-tuning-operator/cmd/performance-profile-creator/cmd"
22+
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/profilecreator/cmd"
2323
)
2424

2525
func main() {

go.mod

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ require (
2626
github.com/openshift/library-go v0.0.0-20240419113445-f1541d628746
2727
github.com/pkg/errors v0.9.1
2828
github.com/prometheus/client_golang v1.21.1
29-
github.com/sirupsen/logrus v1.9.3
3029
github.com/spf13/cobra v1.9.1
3130
github.com/spf13/pflag v1.0.6
3231
gopkg.in/fsnotify.v1 v1.4.7

go.sum

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -434,8 +434,6 @@ github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0
434434
github.com/sirupsen/logrus v1.4.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
435435
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
436436
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
437-
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
438-
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
439437
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
440438
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
441439
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
@@ -686,7 +684,6 @@ golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBc
686684
golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
687685
golang.org/x/sys v0.0.0-20220422013727-9388b58f7150/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
688686
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
689-
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
690687
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
691688
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
692689
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
package profilecreator
2+
3+
import (
4+
"log"
5+
"os"
6+
)
7+
8+
var alertSink *log.Logger
9+
10+
func init() {
11+
// intentionally no flag for minimal output
12+
alertSink = log.New(os.Stderr, "PPC: ", 0)
13+
}
14+
15+
// Alert raise status messages. `profilecreator` is meant to run interactively,
16+
// so the default behavior is to bubble up the messages on the user console.
17+
// It is however possible to reroute the messages to any log.Logger compliant backend
18+
// see the function SetAlertSink. All Alerts are considered Info messages.
19+
func Alert(format string, values ...any) {
20+
alertSink.Printf(format, values...)
21+
}
22+
23+
// Call this function before any other else in the `profilecreator` package
24+
func SetAlertSink(lh *log.Logger) {
25+
alertSink = lh
26+
}
27+
28+
func GetAlertSink() *log.Logger {
29+
return alertSink
30+
}

cmd/performance-profile-creator/cmd/info.go renamed to pkg/performanceprofile/profilecreator/cmd/info.go

Lines changed: 31 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@ import (
55
"fmt"
66
"os"
77
"sort"
8+
"strings"
89

9-
log "github.com/sirupsen/logrus"
1010
"github.com/spf13/cobra"
1111
)
1212

@@ -17,6 +17,7 @@ const (
1717

1818
type infoOptions struct {
1919
jsonOutput bool
20+
textOutput bool
2021
}
2122

2223
// NewInfoCommand return info command, which provides a list of the cluster nodes and their hardware topology.
@@ -31,6 +32,7 @@ func NewInfoCommand(pcArgs *ProfileCreatorArgs) *cobra.Command {
3132
},
3233
}
3334
info.Flags().BoolVar(&opts.jsonOutput, "json", false, "output as JSON")
35+
info.Flags().BoolVar(&opts.textOutput, "text", false, "output as plain text")
3436
return info
3537
}
3638

@@ -40,12 +42,8 @@ func executeInfoMode(mustGatherDirPath string, createForHypershift bool, infoOpt
4042
return fmt.Errorf("failed to parse the cluster data: %w", err)
4143
}
4244
clusterInfo := makeClusterInfoFromClusterData(clusterData)
43-
if infoOpts.jsonOutput {
44-
if err := showClusterInfoJSON(clusterInfo); err != nil {
45-
return fmt.Errorf("unable to show cluster info %w", err)
46-
}
47-
} else {
48-
showClusterInfoLog(clusterInfo)
45+
if err := showClusterInfo(clusterInfo, infoOpts); err != nil {
46+
return fmt.Errorf("unable to show cluster info %w", err)
4947
}
5048
return nil
5149
}
@@ -96,13 +94,13 @@ func makeClusterInfoFromClusterData(cluster ClusterData) ClusterInfo {
9694
for _, handle := range nodeHandlers {
9795
topology, err := handle.SortedTopology()
9896
if err != nil {
99-
log.Infof("%s(Topology discovery error: %v)", handle.Node.GetName(), err)
97+
Alert("%s(Topology discovery error: %v)", handle.Node.GetName(), err)
10098
continue
10199
}
102100

103101
htEnabled, err := handle.IsHyperthreadingEnabled()
104102
if err != nil {
105-
log.Infof("%s(HT discovery error: %v)", handle.Node.GetName(), err)
103+
Alert("%s(HT discovery error: %v)", handle.Node.GetName(), err)
106104
}
107105

108106
nInfo := NodeInfo{
@@ -128,21 +126,36 @@ func makeClusterInfoFromClusterData(cluster ClusterData) ClusterInfo {
128126
return cInfo.Sort()
129127
}
130128

131-
func showClusterInfoJSON(cInfo ClusterInfo) error {
132-
return json.NewEncoder(os.Stdout).Encode(cInfo)
129+
func showClusterInfo(cInfo ClusterInfo, infoOpts *infoOptions) error {
130+
if infoOpts.jsonOutput {
131+
return json.NewEncoder(os.Stdout).Encode(cInfo)
132+
}
133+
textInfo := dumpClusterInfo(cInfo)
134+
if infoOpts.textOutput {
135+
fmt.Print(textInfo)
136+
return nil
137+
}
138+
Alert("Cluster info:\n%s", textInfo)
139+
return nil
133140
}
134141

135-
func showClusterInfoLog(cInfo ClusterInfo) {
136-
log.Infof("Cluster info:")
142+
func dumpClusterInfo(cInfo ClusterInfo) string {
143+
var sb strings.Builder
137144
for _, mcpInfo := range cInfo {
138-
log.Infof("MCP '%s' nodes:", mcpInfo.Name)
145+
fmt.Fprintf(&sb, "MCP '%s' nodes:", mcpInfo.Name)
146+
sb.WriteString("\n")
139147
for _, nInfo := range mcpInfo.Nodes {
140-
log.Infof("Node: %s (NUMA cells: %d, HT: %v)", nInfo.Name, len(nInfo.NUMACells), nInfo.HTEnabled)
148+
fmt.Fprintf(&sb, "Node: %s (NUMA cells: %d, HT: %v)", nInfo.Name, len(nInfo.NUMACells), nInfo.HTEnabled)
149+
sb.WriteString("\n")
141150
for _, cInfo := range nInfo.NUMACells {
142-
log.Infof("NUMA cell %d : %v", cInfo.ID, cInfo.CoreList)
151+
fmt.Fprintf(&sb, "NUMA cell %d : %v", cInfo.ID, cInfo.CoreList)
152+
sb.WriteString("\n")
143153
}
144-
log.Infof("CPU(s): %d", nInfo.CPUsCount)
154+
fmt.Fprintf(&sb, "CPU(s): %d", nInfo.CPUsCount)
155+
sb.WriteString("\n")
145156
}
146-
log.Infof("---")
157+
fmt.Fprintf(&sb, "---")
158+
sb.WriteString("\n")
147159
}
160+
return sb.String()
148161
}

cmd/performance-profile-creator/cmd/root.go renamed to pkg/performanceprofile/profilecreator/cmd/root.go

Lines changed: 18 additions & 104 deletions
Original file line numberDiff line numberDiff line change
@@ -18,31 +18,28 @@ package cmd
1818

1919
import (
2020
"bytes"
21-
"encoding/json"
2221
"fmt"
23-
"io"
2422
"os"
2523
"strings"
2624

27-
log "github.com/sirupsen/logrus"
2825
"github.com/spf13/cobra"
2926
"github.com/spf13/pflag"
3027

3128
corev1 "k8s.io/api/core/v1"
3229
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
33-
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
3430
"k8s.io/apimachinery/pkg/runtime"
3531
serializer "k8s.io/apimachinery/pkg/runtime/serializer/json"
3632
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
3733
"k8s.io/apimachinery/pkg/util/sets"
3834
kubeletconfig "k8s.io/kubelet/config/v1beta1"
3935
"k8s.io/utils/ptr"
40-
"sigs.k8s.io/yaml"
4136

4237
machineconfigv1 "github.com/openshift/api/machineconfiguration/v1"
43-
"github.com/openshift/cluster-node-tuning-operator/cmd/performance-profile-creator/cmd/pkg/hypershift"
4438
performancev2 "github.com/openshift/cluster-node-tuning-operator/pkg/apis/performanceprofile/v2"
4539
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/profilecreator"
40+
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/profilecreator/cmd/hypershift"
41+
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/profilecreator/serialize"
42+
"github.com/openshift/cluster-node-tuning-operator/pkg/performanceprofile/profilecreator/toleration"
4643
)
4744

4845
const (
@@ -107,11 +104,10 @@ type ProfileData struct {
107104
// ClusterData collects the cluster wide information, each mcp points to a list of ghw node handlers
108105
type ClusterData map[string][]*profilecreator.GHWHandler
109106

107+
// shortcut
108+
var Alert = profilecreator.Alert
109+
110110
func init() {
111-
log.SetOutput(os.Stderr)
112-
log.SetFormatter(&log.TextFormatter{
113-
DisableTimestamp: true,
114-
})
115111
utilruntime.Must(performancev2.AddToScheme(scheme))
116112
}
117113

@@ -128,7 +124,7 @@ func NewRootCommand() *cobra.Command {
128124
"must-gather-dir-path",
129125
}
130126

131-
tolerations := profilecreator.TolerationSet{}
127+
tols := toleration.Set{}
132128

133129
root := &cobra.Command{
134130
Use: "performance-profile-creator",
@@ -164,7 +160,7 @@ func NewRootCommand() *cobra.Command {
164160
return err
165161
}
166162

167-
err = profilecreator.EnsureNodesHaveTheSameHardware(nodesHandlers, tolerations)
163+
err = profilecreator.EnsureNodesHaveTheSameHardware(nodesHandlers, tols)
168164
if err != nil {
169165
return fmt.Errorf("targeted nodes differ: %w", err)
170166
}
@@ -175,12 +171,17 @@ func NewRootCommand() *cobra.Command {
175171
if err != nil {
176172
return fmt.Errorf("failed to make profile data from node handler: %w", err)
177173
}
178-
tolerations[profilecreator.EnableHardwareTuning] = profileData.enableHardwareTuning
174+
tols[toleration.EnableHardwareTuning] = profileData.enableHardwareTuning
179175
profile, err := makePerformanceProfileFrom(*profileData)
180176
if err != nil {
181177
return err
182178
}
183-
return writeProfile(profile, tolerations)
179+
profData, err := serialize.Profile(profile, tols)
180+
if err != nil {
181+
return err
182+
}
183+
_, err = fmt.Println(profData)
184+
return err
184185
},
185186
}
186187
pcArgs.AddFlags(root.PersistentFlags())
@@ -201,7 +202,7 @@ func makeNodesHandlers(mustGatherDirPath, poolName string, nodes []*corev1.Node)
201202
sb.WriteString(node.Name + " ")
202203
}
203204
// NodePoolName is alias of MCPName
204-
log.Infof("Nodes names targeted by %s pool are: %s", poolName, sb.String())
205+
Alert("Nodes names targeted by %s pool are: %s", poolName, sb.String())
205206
return handlers, nil
206207
}
207208

@@ -311,8 +312,8 @@ func makeProfileDataFrom(nodeHandler *profilecreator.GHWHandler, args *ProfileCr
311312
if err != nil {
312313
return nil, fmt.Errorf("failed to compute the reserved and isolated CPUs: %v", err)
313314
}
314-
log.Infof("%d reserved CPUs allocated: %v ", reservedCPUs.Size(), reservedCPUs.String())
315-
log.Infof("%d isolated CPUs allocated: %v", isolatedCPUs.Size(), isolatedCPUs.String())
315+
Alert("%d reserved CPUs allocated: %v ", reservedCPUs.Size(), reservedCPUs.String())
316+
Alert("%d isolated CPUs allocated: %v", isolatedCPUs.Size(), isolatedCPUs.String())
316317
kernelArgs := profilecreator.GetAdditionalKernelArgs(args.DisableHT)
317318
profileData := &ProfileData{
318319
reservedCPUs: reservedCPUs.String(),
@@ -521,93 +522,6 @@ func makePerformanceProfileFrom(profileData ProfileData) (runtime.Object, error)
521522
return profile, nil
522523
}
523524

524-
func writeProfile(obj runtime.Object, tolerations profilecreator.TolerationSet) error {
525-
// write CSV to out dir
526-
writer := strings.Builder{}
527-
if err := MarshallObject(obj, &writer); err != nil {
528-
return err
529-
}
530-
531-
if tolerations[profilecreator.EnableHardwareTuning] {
532-
if _, err := writer.Write([]byte(profilecreator.HardwareTuningMessage)); err != nil {
533-
return err
534-
}
535-
}
536-
537-
if tolerations[profilecreator.DifferentCoreIDs] {
538-
if _, err := writer.Write([]byte(profilecreator.DifferentCoreIDsMessage)); err != nil {
539-
return err
540-
}
541-
}
542-
543-
fmt.Printf("%s", writer.String())
544-
return nil
545-
}
546-
547-
// MarshallObject mashals an object, usually a CSV into YAML
548-
func MarshallObject(obj interface{}, writer io.Writer) error {
549-
jsonBytes, err := json.Marshal(obj)
550-
if err != nil {
551-
return err
552-
}
553-
554-
var r unstructured.Unstructured
555-
if err := json.Unmarshal(jsonBytes, &r.Object); err != nil {
556-
return err
557-
}
558-
559-
// remove status and metadata.creationTimestamp
560-
unstructured.RemoveNestedField(r.Object, "metadata", "creationTimestamp")
561-
unstructured.RemoveNestedField(r.Object, "template", "metadata", "creationTimestamp")
562-
unstructured.RemoveNestedField(r.Object, "spec", "template", "metadata", "creationTimestamp")
563-
unstructured.RemoveNestedField(r.Object, "status")
564-
565-
deployments, exists, err := unstructured.NestedSlice(r.Object, "spec", "install", "spec", "deployments")
566-
if err != nil {
567-
return err
568-
}
569-
if exists {
570-
for _, obj := range deployments {
571-
deployment := obj.(map[string]interface{})
572-
unstructured.RemoveNestedField(deployment, "metadata", "creationTimestamp")
573-
unstructured.RemoveNestedField(deployment, "spec", "template", "metadata", "creationTimestamp")
574-
unstructured.RemoveNestedField(deployment, "status")
575-
}
576-
if err := unstructured.SetNestedSlice(r.Object, deployments, "spec", "install", "spec", "deployments"); err != nil {
577-
return err
578-
}
579-
}
580-
581-
jsonBytes, err = json.Marshal(r.Object)
582-
if err != nil {
583-
return err
584-
}
585-
586-
yamlBytes, err := yaml.JSONToYAML(jsonBytes)
587-
if err != nil {
588-
return err
589-
}
590-
591-
// fix double quoted strings by removing unneeded single quotes...
592-
s := string(yamlBytes)
593-
s = strings.Replace(s, " '\"", " \"", -1)
594-
s = strings.Replace(s, "\"'\n", "\"\n", -1)
595-
596-
yamlBytes = []byte(s)
597-
598-
_, err = writer.Write([]byte("---\n"))
599-
if err != nil {
600-
return err
601-
}
602-
603-
_, err = writer.Write(yamlBytes)
604-
if err != nil {
605-
return err
606-
}
607-
608-
return nil
609-
}
610-
611525
func listNodesForPool(args *ProfileCreatorArgs) ([]*corev1.Node, error) {
612526
nodes, err := profilecreator.GetNodeList(args.MustGatherDirPath)
613527
if err != nil {

0 commit comments

Comments
 (0)