Fix metric files being wiped after the app starts

When we hit our app with the initial request, in `warmup`,
some metrics already being created as well as corresponding files.
If we do `multiproc_file_dir` cleanup after that, we delete the files
from the dir while keeping them in memory which leads to the incorrect
behavior: the metric is being updated in in-memory, while is not present
in the db, not sent to Prometheus as the result.

Author: Aleksei Lipniagov

changelogs/unreleased/65278-fix-puma-master-counter-wipe.yml

@@ -0,0 +1,5 @@
+---
+title: Fix active metric files being wiped after the app starts
+merge_request: 31668
+author:
+type: fixed

config.ru

@@ -17,7 +17,22 @@ end
 
 require ::File.expand_path('../config/environment', __FILE__)
 
+
+# The following is necessary to ensure stale Prometheus metrics don't accumulate over time.
+# It needs to be done as early as here to ensure metrics files aren't deleted.
+# After we hit our app in `warmup`, first metrics and corresponding files already being created,
+# for example in `lib/gitlab/metrics/requests_rack_middleware.rb`.
+def cleanup_prometheus_multiproc_dir
+  if dir = ::Prometheus::Client.configuration.multiprocess_files_dir
+    old_metrics = Dir[File.join(dir, '*.db')]
+
+    FileUtils.rm_rf(old_metrics)
+  end
+end
+
 warmup do |app|
+  cleanup_prometheus_multiproc_dir
+
   client = Rack::MockRequest.new(app)
   client.get('/')
 end

config/initializers/7_prometheus_metrics.rb

@@ -51,22 +51,3 @@ def prometheus_default_multiproc_dir
     end
   end
 end
-
-def cleanup_prometheus_multiproc_dir
-  # The following is necessary to ensure stale Prometheus metrics don't
-  # accumulate over time. It needs to be done in this hook as opposed to
-  # inside an init script to ensure metrics files aren't deleted after new
-  # unicorn workers start after a SIGUSR2 is received.
-  if dir = ::Prometheus::Client.configuration.multiprocess_files_dir
-    old_metrics = Dir[File.join(dir, '*.db')]
-    FileUtils.rm_rf(old_metrics)
-  end
-end
-
-Gitlab::Cluster::LifecycleEvents.on_master_start do
-  cleanup_prometheus_multiproc_dir
-end
-
-Gitlab::Cluster::LifecycleEvents.on_master_restart do
-  cleanup_prometheus_multiproc_dir
-end