Add latest changes from gitlab-org/gitlab@master

Author: GitLab Bot

.rubocop_todo/style/inline_disable_annotation.yml

@@ -2416,7 +2416,6 @@ Style/InlineDisableAnnotation:
     - 'metrics_server/override_gitlab_current_settings.rb'
     - 'metrics_server/override_rails_constants.rb'
     - 'metrics_server/settings_overrides.rb'
-    - 'qa/chemlab-library-gitlab.gemspec'
     - 'qa/qa/ee/page/admin/subscription.rb'
     - 'qa/qa/ee/page/main/banner.rb'
     - 'qa/qa/ee/page/project/monitor/on_call_schedule/index.rb'

CHANGELOG.md

@@ -2,6 +2,25 @@
 documentation](doc/development/changelog.md) for instructions on adding your own
 entry.
 
+## 17.3.1 (2024-08-20)
+
+### Fixed (3 changes)
+
+- [Merge branch '444926-fix-bug-in-resolve-vulnerabilities-migration' into 'master'](https://gitlab.com/gitlab-org/security/gitlab/-/commit/ac8a0cdc00b76c5ad84e8d18f3a7e49ebea2a538)
+- [Turn NotFound from Gitaly into 404 for InfoRefs](https://gitlab.com/gitlab-org/security/gitlab/-/commit/3566c2625d62857246b215e191fb137091de1650)
+- [Fix timeout when checking group dependencies](https://gitlab.com/gitlab-org/security/gitlab/-/commit/3f3bdb24b185196875a3989f0378d237243e80f6) **GitLab Enterprise Edition**
+
+### Changed (1 change)
+
+- [Include language server version in code suggestions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/5b4b98955c5fa55911631800c3cd48f6224bf664) **GitLab Enterprise Edition**
+
+### Security (4 changes)
+
+- [Do not run pipelines when resolving vulnerability](https://gitlab.com/gitlab-org/security/gitlab/-/commit/ef9c251b19c1ad7aedb591870158fc0085ee5fd9) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4360))
+- [Add Octokit::ResponseValidation middleware](https://gitlab.com/gitlab-org/security/gitlab/-/commit/08d547262c574b00135fb71105e52f03dc3ca8c0) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4375))
+- [IP restriction to prevent all group permissions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/e080f2d2c5a578df52f202505e993c560fec6cb2) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4368))
+- [Destroy associated releases when removing a tag via Git CLI](https://gitlab.com/gitlab-org/security/gitlab/-/commit/b79ada987b82fa756e6ae74f7527dcde8c30d08f) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4365))
+
 ## 17.3.0 (2024-08-14)
 
 ### Added (143 changes)
@@ -727,6 +746,35 @@ entry.
 - [Dynamically gets the column type for assertion](https://gitlab.com/gitlab-org/gitlab/-/commit/1389a3daffd104925cce71776903cbf527723222) ([merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/159099))
 - [Quarantine a flaky test](https://gitlab.com/gitlab-org/gitlab/-/commit/c94fca35b909440ec66ea35c97ab11aa847dde58) ([merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/158180))
 
+## 17.2.4 (2024-08-21)
+
+### Security (1 change)
+
+- [Always build assets image when tagging](https://gitlab.com/gitlab-org/security/gitlab/-/commit/d0e661baad53be4fb7eef3b530b544d05a609953) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4386))
+
+## 17.2.3 (2024-08-20)
+
+### Fixed (3 changes)
+
+- [Turn NotFound from Gitaly into 404 for InfoRefs](https://gitlab.com/gitlab-org/security/gitlab/-/commit/cdd5159fa1f8259dbf92333cf13a2968e814d307)
+- [Fix empty dependency list page](https://gitlab.com/gitlab-org/security/gitlab/-/commit/71fc48e515cffcbc46ad4f824dc1990a0eb3b08a) **GitLab Enterprise Edition**
+- [Properly handle empty repository.ff_merge in FromTrainRef merge strategy](https://gitlab.com/gitlab-org/security/gitlab/-/commit/63c5e1ce261f03549f19a36867ed83cd928a2c5f) **GitLab Enterprise Edition**
+
+### Changed (1 change)
+
+- [Include language server version in code suggestions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/c9c47052794e5d5bfc86d78c5fdd91c4b910a8b8) **GitLab Enterprise Edition**
+
+### Security (4 changes)
+
+- [Do not run pipelines when resolving vulnerability](https://gitlab.com/gitlab-org/security/gitlab/-/commit/4a8fa8c6fd40731c93986e54f56d6b581f4dfa92) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4307))
+- [Add Octokit::ResponseValidation middleware](https://gitlab.com/gitlab-org/security/gitlab/-/commit/f8cf13d24a0f5aa07c0236238b2a3d61d82615e9) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4376))
+- [IP restriction to prevent all group permissions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/4b540bf88e05c7e39803652bdbba8978c74ebab6) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4340))
+- [Destroy associated releases when removing a tag via Git CLI](https://gitlab.com/gitlab-org/security/gitlab/-/commit/bb033e98e00bc242a92d7091406f7c91e98a3079) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4366))
+
+### Other (1 change)
+
+- [Check if columns exist before running credit card hashing migration](https://gitlab.com/gitlab-org/security/gitlab/-/commit/f6b7ac482714a0da707cab36e685378d5067d891)
+
 ## 17.2.2 (2024-08-06)
 
 ### Fixed (2 changes)
@@ -1548,6 +1596,29 @@ entry.
 - [Remove "use_remote_mirror_destroy_service" feature flag](https://gitlab.com/gitlab-org/gitlab/-/commit/74e1e921d003960afd6f259384aee2dfec18f30e) ([merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/155852))
 - [Protected containers: Cleanup renaming of protected_up_to_access_level](https://gitlab.com/gitlab-org/gitlab/-/commit/4606b5ef64f75acdd581258a0b93034195626e83) by @gerardo-navarro ([merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/146177))
 
+## 17.1.6 (2024-08-21)
+
+### Security (1 change)
+
+- [Always build assets image when tagging](https://gitlab.com/gitlab-org/security/gitlab/-/commit/b10a04aa687e6fbdf6c26b5756dcbb3748728e9a) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4385))
+
+## 17.1.5 (2024-08-20)
+
+### Fixed (1 change)
+
+- [Properly handle empty repository.ff_merge in FromTrainRef merge strategy](https://gitlab.com/gitlab-org/security/gitlab/-/commit/b3fa341da06a3cbec69e2c32c5b3a336fcba6df7) **GitLab Enterprise Edition**
+
+### Changed (1 change)
+
+- [Include language server version in code suggestions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/02b9f967f7f68865d6dfcfd550e9967689239791) **GitLab Enterprise Edition**
+
+### Security (4 changes)
+
+- [Do not run pipelines when resolving vulnerability](https://gitlab.com/gitlab-org/security/gitlab/-/commit/b962ee1ac8f2ab653435937008c5c4d869aa17f8) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4308))
+- [Add Octokit::ResponseValidation middleware](https://gitlab.com/gitlab-org/security/gitlab/-/commit/f0483d356530afafa070302e50f34a12634b28a8) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4377))
+- [IP restriction to prevent all group permissions](https://gitlab.com/gitlab-org/security/gitlab/-/commit/aecd6dd35b1bfd45d1cb7442afc9a9bf312310b6) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4341))
+- [Destroy associated releases when removing a tag via Git CLI](https://gitlab.com/gitlab-org/security/gitlab/-/commit/7478a869a7a2722377da289a6b16d77b608ccfc2) ([merge request](https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/4367))
+
 ## 17.1.4 (2024-08-06)
 
 ### Changed (2 changes)

app/assets/javascripts/work_items/components/shared/work_item_more_actions.vue

@@ -6,6 +6,7 @@ import {
   GlToggle,
 } from '@gitlab/ui';
 import { s__ } from '~/locale';
+import { InternalEvents } from '~/tracking';
 import { workItemRoadmapPath } from '../../utils';
 import { WORK_ITEM_TYPE_ENUM_EPIC } from '../../constants';
 
@@ -22,6 +23,7 @@ export default {
   directives: {
     GlTooltip: GlTooltipDirective,
   },
+  mixins: [InternalEvents.mixin()],
   props: {
     workItemIid: {
       type: String,
@@ -113,7 +115,11 @@ export default {
         </template>
       </gl-disclosure-dropdown-item>
 
-      <gl-disclosure-dropdown-item v-if="shouldShowViewRoadmapAction" :item="viewOnARoadmap" />
+      <gl-disclosure-dropdown-item
+        v-if="shouldShowViewRoadmapAction"
+        :item="viewOnARoadmap"
+        @action="trackEvent('view_epic_on_roadmap')"
+      />
     </gl-disclosure-dropdown>
   </div>
 </template>

app/models/commit_status.rb

@@ -50,7 +50,7 @@ class CommitStatus < Ci::ApplicationRecord
   validates :name, presence: true, unless: :importing?
   validates :ci_stage, presence: true, on: :create, unless: :importing?
   validates :ref, :target_url, :description, length: { maximum: 255 }
-  validates :project, presence: true, on: :create
+  validates :project, presence: true
 
   alias_attribute :author, :user
   alias_attribute :pipeline_id, :commit_id

app/services/git/tag_push_service.rb

@@ -10,13 +10,20 @@ def execute
       project.repository.before_push_tag
       TagHooksService.new(project, current_user, params).execute
 
+      destroy_releases
       unlock_artifacts
 
       true
     end
 
     private
 
+    def destroy_releases
+      return unless removing_tag?
+
+      Releases::DestroyService.new(project, current_user, tag: Gitlab::Git.tag_name(ref)).execute
+    end
+
     def unlock_artifacts
       return unless removing_tag?
 

app/views/shared/access_tokens/_form.html.haml

@@ -24,7 +24,7 @@
         = s_("AccessTokens|For example, the application using the token or the purpose of the token.")
 
   .js-access-tokens-expires-at{ data: expires_at_field_data }
-    = f.text_field :expires_at, class: 'gl-datepicker-input form-control gl-form-input', placeholder: 'YYYY-MM-DD', autocomplete: 'off', data: { js_name: 'expiresAt' }
+    = f.text_field :expires_at, class: 'form-control gl-form-input', placeholder: 'YYYY-MM-DD', autocomplete: 'off', data: { js_name: 'expiresAt' }
 
   - if resource
     .form-group

app/views/user_settings/ssh_keys/_form.html.haml

@@ -21,7 +21,7 @@
       .col.form-group
         .js-access-tokens-expires-at{ data: {min_date: Date.tomorrow, max_date: max_date, default_date_offset: 365, description: ssh_key_expires_field_description } }
           = f.label :expires_at, s_('Profiles|Expiration date'), class: 'label-bold'
-          = f.text_field :expires_at, class: "gl-datepicker-input form-control gl-form-input", placeholder: 'YYYY-MM-DD', min: Date.tomorrow, max: max_date, data: { js_name: 'expiresAt' }
+          = f.text_field :expires_at, class: "form-control gl-form-input", placeholder: 'YYYY-MM-DD', min: Date.tomorrow, max: max_date, data: { js_name: 'expiresAt' }
           %p.form-text.text-muted= ssh_key_expires_field_description
 
     .js-add-ssh-key-validation-warning.hide

config/gitlab_loose_foreign_keys.yml

@@ -190,6 +190,10 @@ dast_profiles_tags:
   - table: tags
     column: tag_id
     on_delete: async_delete
+dast_scanner_profiles:
+  - table: projects
+    column: project_id
+    on_delete: async_delete
 dast_scanner_profiles_builds:
   - table: ci_builds
     column: ci_build_id

config/initializers/octokit.rb

@@ -1,3 +1,4 @@
 # frozen_string_literal: true
 
-Octokit.middleware.insert_after Octokit::Middleware::FollowRedirects, Gitlab::Octokit::Middleware
+Octokit.middleware.insert_after Octokit::Middleware::FollowRedirects, Gitlab::Octokit::UrlValidation
+Octokit.middleware.insert_after Gitlab::Octokit::UrlValidation, Gitlab::Octokit::ResponseValidation

db/docs/dast_scanner_profiles.yml

@@ -7,7 +7,7 @@ feature_categories:
 description: A scanner profile defines the scanner settings used to run an on-demand scan
 introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37404
 milestone: '13.3'
-gitlab_schema: gitlab_main_cell
+gitlab_schema: gitlab_sec
 allow_cross_foreign_keys:
 - gitlab_main_clusterwide
 sharding_key:

db/migrate/20240816195355_add_identifier_names_to_vulnerability_reads.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+class AddIdentifierNamesToVulnerabilityReads < Gitlab::Database::Migration[2.2]
+  milestone '17.4'
+
+  def change
+    add_column :vulnerability_reads, :identifier_names, :text, array: true, default: [], null: false
+  end
+end

db/post_migrate/20240819135845_remove_projects_dast_scanner_profiles_project_id_fk.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+class RemoveProjectsDastScannerProfilesProjectIdFk < Gitlab::Database::Migration[2.2]
+  milestone '17.4'
+  disable_ddl_transaction!
+
+  FOREIGN_KEY_NAME = "fk_rails_72a8ba7141"
+
+  def up
+    with_lock_retries do
+      remove_foreign_key_if_exists(:dast_scanner_profiles, :projects,
+        name: FOREIGN_KEY_NAME, reverse_lock_order: true)
+    end
+  end
+
+  def down
+    add_concurrent_foreign_key(:dast_scanner_profiles, :projects,
+      name: FOREIGN_KEY_NAME, column: :project_id,
+      target_column: :id, on_delete: :cascade)
+  end
+end

db/schema_migrations/20240816195355

@@ -0,0 +1 @@
+600e49adf906434ea9a1a2998c8ab157f6ee39381bbc4aa4249a238f0b7e9ecb

db/schema_migrations/20240819135845

@@ -0,0 +1 @@
+d9519fcd3ce7bc1cd2764dde5ea2040fc1eff7ae52f764a2d60a9e5d0adeb5c3

db/structure.sql

@@ -19923,6 +19923,7 @@ CREATE TABLE vulnerability_reads (
     traversal_ids bigint[] DEFAULT '{}'::bigint[],
     archived boolean DEFAULT false NOT NULL,
     identifier_external_ids text[] DEFAULT '{}'::text[] NOT NULL,
+    identifier_names text[] DEFAULT '{}'::text[] NOT NULL,
     CONSTRAINT check_380451bdbe CHECK ((char_length(location_image) <= 2048)),
     CONSTRAINT check_4b1a1bf5ea CHECK ((has_merge_request IS NOT NULL)),
     CONSTRAINT check_a105eb825a CHECK ((char_length(cluster_agent_id) <= 10)),
@@ -35072,9 +35073,6 @@ ALTER TABLE ONLY ci_pending_builds
 ALTER TABLE security_findings
     ADD CONSTRAINT fk_rails_729b763a54 FOREIGN KEY (scanner_id) REFERENCES vulnerability_scanners(id) ON DELETE CASCADE;
 
-ALTER TABLE ONLY dast_scanner_profiles
-    ADD CONSTRAINT fk_rails_72a8ba7141 FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE;
-
 ALTER TABLE ONLY custom_emoji
     ADD CONSTRAINT fk_rails_745925b412 FOREIGN KEY (namespace_id) REFERENCES namespaces(id) ON DELETE CASCADE;
 

doc/.vale/gitlab_base/spelling-exceptions.txt

@@ -152,7 +152,6 @@ ChatOps
 checksummable
 checksummed
 checksumming
-Chemlab
 chipset
 chipsets
 CIDRs

doc/administration/backup_restore/backup_cli.md

@@ -26,6 +26,66 @@ To take a backup of the current GitLab installation:
 sudo gitlab-backup-cli backup all
 ```
 
+### Backing up object storage
+
+Only Google cloud is supported. See [epic 11577](https://gitlab.com/groups/gitlab-org/-/epics/11577) for the plan to add more vendors.
+
+#### GCP
+
+`gitlab-backup-cli` creates and runs jobs with [Google Transfer Service](https://cloud.google.com/storage-transfer-service/) to copy GitLab data to a separate backup bucket.
+
+Prerequisites:
+
+- Follow [Google's documentation](https://cloud.google.com/docs/authentication) for authentication.
+- This document assumes you are setting up and using a dedicated Google Cloud service account for managing backups.
+- If no other credentials are provided, and you are running inside Google Cloud, then the tool attempts to use the access of the infrastructure it is running on. It is recommended to run with separate credentials, and restrict access to the created backups from the application.
+
+To create a backup:
+
+1. [Create a role](https://cloud.google.com/iam/docs/creating-custom-roles):
+   1. Create a file `role.yaml` with the following definition:
+
+   ```yaml
+   ---
+   description: Role for backing up GitLab object storage
+   includedPermissions:
+      - storagetransfer.jobs.create
+      - storagetransfer.jobs.get
+      - storagetransfer.jobs.run
+      - storagetransfer.jobs.update
+      - storagetransfer.operations.get
+      - storagetransfer.projects.getServiceAccount
+   stage: GA
+   title: GitLab Backup Role
+   ```
+
+   1. Apply the role:
+
+      ```shell
+      gcloud iam roles create --project=<YOUR_PROJECT_ID> <ROLE_NAME> --file=role.yaml
+      ```
+
+1. Create a service account for backups, and add it to the role:
+
+   ```shell
+   gcloud iam service-accounts create "gitlab-backup-cli" --display-name="GitLab Backup Service Account"
+   # Get the service account email from the output of the following
+   gcloud iam service-accounts list
+   # Add the account to the role created previously
+   gcloud projects add-iam-policy-binding <YOUR_PROJECT_ID> --member="serviceAccount:<SERVICE_ACCOUNT_EMAIL>" --role="roles/<ROLE_NAME>"
+   ```
+
+1. Follow [Google's documentation](https://cloud.google.com/docs/authentication) for authentication with the service account. In general, the credentials can be saved to a file, or stored in a predefined environment variable.
+1. Create a destination bucket to backup to in [Google Cloud Storage](https://cloud.google.com/storage/). The options here are highly dependent on your requirements.
+1. Run the backup:
+
+   ```shell
+   sudo gitlab-backup-cli backup all --backup-bucket=<BUCKET_NAME>
+   ```
+
+   If you want to backup the container registry bucket, add the option `--registry-bucket=<REGISTRY_BUCKET_NAME>`.
+1. The backup creates a backup under `backups/<BACKUP_ID>/<BUCKET>` for each of the object storage types in the bucket.
+
 ## Backup directory structure
 
 Example backup directory structure:

doc/administration/backup_restore/troubleshooting_backup_gitlab.md

@@ -204,6 +204,14 @@ You should verify that the secrets are the root cause before deleting any data.
    TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs CASCADE;
    ```
 
+## Container registry is not restored
+
+If you restore a backup from an environment that uses the [container registry](../../user/packages/container_registry/index.md) 
+to a newly installed environment where the container registry is not enabled, the container registry is not restored.
+
+To also restore the container registry, you need to [enable it](../packages/container_registry.md#enable-the-container-registry) in the new
+environment before you restore the backup.
+
 ## Container registry push failures after restoring from a backup
 
 If you use the [container registry](../../user/packages/container_registry/index.md),