GROW-663: Capture K8 related sub-count for GCP (#2)

Author: laverion

README.md

@@ -15,11 +15,15 @@ the user running the resource counting script:
       - `roles/owner`
       - `roles/cloudasset.owner`
       - `roles/cloudasset.viewer`
-    - [Security Command Center][2] (faster, but not free). The user running the
-    script must have one of these roles at the organization-level:
-      - `roles/resourcemanager.organizationAdmin`
-      - `roles/securitycenter.admin`
-      - `roles/securitycenter.adminViewer`
+    - This method has been deprecated. As outlined in
+    [Listing assets using the Security Command Center API][4],
+    this functionality has been deprecated on `June 20, 2023` and will reach its
+    EOL on `June 20, 2024`.  
+    ~~[Security Command Center][2] (faster, but not free). The user running the
+    script must have one of these roles at the organization-level:~~
+      - ~~`roles/resourcemanager.organizationAdmin`~~
+      - ~~`roles/securitycenter.admin`~~
+      - ~~`roles/securitycenter.adminViewer`~~
 3. Install [jq][3]
 4. Retrieve the GCP organization ID by running the following command and
 looking for the ID of the organization to count:
@@ -46,26 +50,22 @@ from the previous step:
 {
   "appengine.googleapis.com/Application": 20,
   "cloudfunctions.googleapis.com/CloudFunction": 63,
-  "compute.googleapis.com/Image": 43,
   "compute.googleapis.com/Instance": 466,
+  "compute.googleapis.com/K8RelatedInstance": 8,
   "sqladmin.googleapis.com/Instance": 65,
-  "storage.googleapis.com/Bucket": 367
+  "storage.googleapis.com/Bucket": 367,
+  "k8s.io/Node": 8,
 }
 ```
 
-## How to run using security command center (gcloud_scc_inventory.sh)
+### Note about output
 
-```bash
-λ organizationID=123456789012 ./gcloud_scc_inventory.sh
-{
-  "appengine.googleapis.com/Application": 20,
-  "cloudfunctions.googleapis.com/CloudFunction": 63,
-  "compute.googleapis.com/Image": 43,
-  "compute.googleapis.com/Instance": 466,
-  "sqladmin.googleapis.com/Instance": 65,
-  "storage.googleapis.com/Bucket": 367
-}
-```
+`compute.googleapis.com/K8RelatedInstance` is not an actual asset that
+listed in [Supported asset types][5]. The number generated for this
+custom asset is from filtering all compute instances that have a
+`goog-gke-node` label. This label is used because it's a protected
+and automatically applied label to compute instances that were created
+by a GKE cluster.
 
 ## Troubleshooting
 
@@ -79,24 +79,9 @@ null
 Verify IAM for user running asset inventory script has one of the
 [roles required](#requirements) and has access to the organization.
 
-### Running security command center returns error
-
-<!-- markdownlint-disable MD013 -->
-```bash
-ERROR: (gcloud.scc.assets.group) User [your-user-id@your-organization] does not
-have permission to access organizations instance [123456789012] (or it may not
-exist): Permission 'securitycenter.assets.group' denied on resource
-'//cloudresourcemanager.googleapis.com/organizations/123456789012' (or it may
-not exist)
-```
-<!-- markdownlint-enable MD013 -->
-
-[Verify security command center is enabled](4) for the organization. Also verify
-IAM for user running security command center script has one of the
-[roles required](#requirements) and has access to the organization.
-
 [0]: https://cloud.google.com/sdk/docs/install-sdk
 [1]: https://cloud.google.com/asset-inventory/docs/listing-assets
 [2]: https://cloud.google.com/security-command-center/docs/set-up
 [3]: https://stedolan.github.io/jq/download/
-[4]: https://console.cloud.google.com/security/command-center/overview
+[4]: https://cloud.google.com/security-command-center/docs/how-to-api-list-assets
+[5]: https://cloud.google.com/asset-inventory/docs/supported-asset-types#searchable_asset_types

gcloud_asset_inventory.sh

@@ -25,8 +25,10 @@ declare -ar types=(
   appengine.googleapis.com/Application
   cloudfunctions.googleapis.com/CloudFunction
   compute.googleapis.com/Instance
+  compute.googleapis.com/K8RelatedInstance
   sqladmin.googleapis.com/Instance
   storage.googleapis.com/Bucket
+  k8s.io/Node
 )
 
 check_binary "gcloud"
@@ -41,16 +43,30 @@ outfile=$(mktemp -q)
 # shellcheck disable=SC2048
 for type in ${types[*]}; do
 
-  (echo "${organizations[*]}" \
-    | xargs -n1 -I{} gcloud asset list \
-        --asset-types "$type" \
-        --content-type resource \
-        --format 'value(assetType)' \
-        --organization {} \
-        --snapshot-time "$time" \
-    | sort \
-    | uniq -c \
-    | awk '{ printf "{\"%s\":%s}\n", $2, $1 }' >> "$outfile") &
+  if [ "$type" == "compute.googleapis.com/K8RelatedInstance" ]; then
+    (echo "${organizations[*]}" \
+      | xargs -n1 -I{} gcloud asset list \
+          --asset-types "compute.googleapis.com/Instance" \
+          --content-type resource \
+          --format 'value(assetType)' \
+          --organization {} \
+          --snapshot-time "$time" \
+          --filter 'resource.data.labels:goog-gke-node' \
+      | sort \
+      | uniq -c \
+      | awk '{ printf "{\"compute.googleapis.com/K8RelatedInstance\":%s}\n", $1 }' >> "$outfile") &
+  else
+    (echo "${organizations[*]}" \
+      | xargs -n1 -I{} gcloud asset list \
+          --asset-types "$type" \
+          --content-type resource \
+          --format 'value(assetType)' \
+          --organization {} \
+          --snapshot-time "$time" \
+      | sort \
+      | uniq -c \
+      | awk '{ printf "{\"%s\":%s}\n", $2, $1 }' >> "$outfile") &
+  fi
 
 done