Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
6d82cf9
*: initial scaffolding for the stickydisk action
adityamaru Dec 7, 2024
925fd98
Merge pull request #1 from useblacksmith/initial-commit
adityamaru Dec 13, 2024
1e9b1c3
README: update use cases in README
adityamaru Dec 16, 2024
86672c0
README: update README
adityamaru Dec 16, 2024
f9a0feb
Merge pull request #3 from useblacksmith/update-readme
adityamaru Dec 16, 2024
133fc0b
README: add blacksmith logo (#4)
aayushshah15 Dec 16, 2024
d29289d
main: maintain runner perms when mounting a stickydisk
adityamaru Dec 19, 2024
719a784
Merge pull request #5 from useblacksmith/fix-perms
adityamaru Dec 19, 2024
60fc5d5
update README (#6)
aayushshah15 Dec 19, 2024
ea2e5f8
README: update README with use cases and arch diagram
adityamaru Jan 16, 2025
3d6bf16
.github: fix typo
adityamaru Jan 17, 2025
16ff158
Merge pull request #7 from useblacksmith/add-readme
adityamaru Jan 17, 2025
6d5bf41
README: fix nits
adityamaru Jan 17, 2025
c8fc92b
Merge pull request #8 from useblacksmith/fix-nits
adityamaru Jan 17, 2025
af6814d
Update README.md
adityamaru Jan 17, 2025
b26d421
.github: clarify in first sentence
adityamaru Jan 17, 2025
a978eb7
Update README.md
adityamaru Jan 17, 2025
7464a4c
Update README.md
adijp Jan 17, 2025
d197c9d
Merge pull request #9 from useblacksmith/adijp-patch-1
adijp Jan 17, 2025
a85354c
src: add sync before umount
adityamaru Jan 22, 2025
9c61fba
Merge pull request #10 from useblacksmith/sd-fixes
adityamaru Jan 22, 2025
5175c4c
src: increase timeout to the same as build-push-action
adityamaru Feb 10, 2025
abd1f46
Merge pull request #11 from useblacksmith/timeout-fails
adityamaru Feb 10, 2025
eb09d7f
src: explicitly flush journal before umounting (#12)
aayushshah15 Feb 28, 2025
617cc46
use the env var for sticky disk grpc port
aayushshah15 Apr 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .github/workflows/basic.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: Basic Sticky Disk Test
on:
workflow_dispatch:
pull_request:

jobs:
test-sticky-disk:
name: Test Sticky Disk
runs-on: blacksmith
strategy:
matrix:
iteration: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
fail-fast: false
steps:
- name: Create directory called
run: sudo mkdir -p ./shouldseethis

- name: Mount First Sticky Disk
uses: useblacksmith/stickydisk@main
with:
key: foo
path: ./shouldseethis

- name: List first directory if exists
run: |
if [ -d "./shouldseethis" ]; then
ls -la ./shouldseethis
fi

- name: Write test file to first disk
run: sudo sh -c 'echo "Hello from first sticky disk" > ./shouldseethis/test.txt'

- name: Mount Second Sticky Disk
uses: useblacksmith/stickydisk@main
with:
key: bar
path: ./seconddisk

- name: List second directory if exists
run: |
if [ -d "./seconddisk" ]; then
ls -la ./seconddisk
fi

- name: Write test file to second disk
run: sudo sh -c 'echo "Hello from second sticky disk" > ./seconddisk/test.txt'

- name: Mount Third Sticky Disk
uses: useblacksmith/stickydisk@main
with:
key: baz
path: ./thirddisk

- name: List third directory if exists
run: |
if [ -d "./thirddisk" ]; then
ls -la ./thirddisk
fi

- name: Write test file to third disk
run: sudo sh -c 'echo "Hello from third sticky disk" > ./thirddisk/test.txt'

- name: Mount First Sticky Disk Again
uses: useblacksmith/stickydisk@main
with:
key: foo
path: ./shouldseethis_again

- name: List first directory again if exists
run: |
if [ -d "./shouldseethis_again" ]; then
ls -la ./shouldseethis_again
fi

- name: Write test file to first disk again
run: sudo sh -c 'echo "Hello again from first sticky disk" > ./shouldseethis_again/test2.txt'
16 changes: 15 additions & 1 deletion .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@ jobs:
with:
node-version-file: .nvmrc

- name: Setup Buf
uses: bufbuild/buf-setup-action@v1

- name: Configure Buf Registry
run: |
npm config set @buf:registry https://buf.build/gen/npm/v1/
npm config set //buf.build/gen/npm/v1/:_authToken ${{ secrets.BUF_TOKEN }}
npm install @buf/blacksmith_vm-agent.connectrpc_es@latest

- name: Install Dependencies
run: npm ci

Expand All @@ -26,4 +35,9 @@ jobs:
- name: Build Action
run: |
npm run build
git diff && git diff-index --quiet --exit-code HEAD
# Check if any files were changed during build
if ! git diff --quiet; then
echo "Error: Build generated changes that aren't committed. Please run 'npm run build' locally and commit the changes."
git diff
exit 1
fi
22 changes: 22 additions & 0 deletions .github/workflows/bump-tag.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Bump Tag

on:
workflow_dispatch:

jobs:
bump-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: main

- name: Bump v1 tag
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
git tag -f v1
git push origin v1 -f
23 changes: 23 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,27 @@
!.git*
!.nvmrc

# Dependencies
node_modules/

# TypeScript
*.tsbuildinfo

# IDE
.idea/
.vscode/
*.swp
*.swo

# OS
.DS_Store
Thumbs.db

# Test coverage
coverage/

# Logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
Binary file added Blacksmith_Logo-Black-Large.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Blacksmith_Logo-White-Large.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
133 changes: 75 additions & 58 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,76 +1,93 @@
# Blacksmith Cache Delete Action
<p align="center">
<picture>
<!-- Dark mode -->
<source media="(prefers-color-scheme: dark)" srcset="./Blacksmith_Logo-White-Large.png" width="300">
<!-- Light mode -->
<source media="(prefers-color-scheme: light)" srcset="./Blacksmith_Logo-Black-Large.png" width="300">
<img alt="Blacksmith Logo" src="./Blacksmith_Logo-Black-Large.png" width="300">
</picture>
</p>

A GitHub Action to delete caches from Blacksmith's cache storage. This action allows you to delete either a specific cache version or all versions of a cache key. This action only works Blacksmith runners.
A GitHub Action that helps persist state written to disk across jobs. This action can serve as a superior alternative to the [Actions cache](https://github.com/useblacksmith/cache), especially when the cache artifacts are extremely large. Each sticky disk is hot-loaded into the runner and mounted at the specified path.
The sticky disk is formatted as an ext4 filesystem.

## Usage
# Architecture

```yaml
- name: Delete Cache
uses: useblacksmith/cache-delete@v1
with:
key: Linux-composer-ecf6e2e236589e4d34ba89662b6bc2afe8e15237cd19a13df9dc0cb599ff4826
version: v213asda2cf # Optional: specific version to delete
```

## Inputs
<p align="center">
<picture>
<!-- Dark mode -->
<source media="(prefers-color-scheme: dark)" srcset="./arch-dark-mode.png" width="1000">
<!-- Light mode -->
<source media="(prefers-color-scheme: light)" srcset="./arch-light.png" width="1000">
<img alt="Blacksmith Logo" src="./arch-light.png" width="1000">
</picture>
</p>

| Input | Description | Required | Default |
| --------- | --------------------------------------- | -------- | ------- |
| `key` | The cache key to delete | No\* | - |
| `version` | Specific version of the cache to delete | No | - |
| `prefix` | Treat key as a prefix for bulk deletion | No | false |
Blacksmith stores sticky disk artifacts in a secure, highly performant Ceph cluster, running on local NVMe drives. Our runners proxy their requests through our Storage Agents to interact with the Ceph cluster. Each sticky disk is uniquely identified by a key. When a GitHub Action job requests a sticky disk, the last committed snapshot will be cloned and mounted into the runner at the specified path. Once the job completes, the sticky disk will be unmounted and committed for future invocations. At the moment, customers can use up to 5 sticky disks in a single GitHub Action job.

\* Required unless `prefix` is true, in which case it can be empty to match all cache keys
# Use Cases

## Examples

### Delete All Versions of a Cache

```yaml
- name: Delete All Cache Versions
uses: useblacksmith/cache-delete@v1
with:
key: npm-cache
```
## NPM Package Caching

### Delete a Specific Cache Version
Node.js projects can have extensive dependency trees, leading to large `node_modules` directories. Sticky disks provide persistent, high-performance storage for your NPM packages.

```yaml
- name: Delete Specific Cache Version
uses: useblacksmith/cache-delete@v1
with:
key: npm-cache
version: v1.0
jobs:
build:
runs-on: blacksmith
steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: useblacksmith/setup-node@v5
with:
node-version: '18.x'

- name: Mount NPM Cache
uses: useblacksmith/stickydisk@v1
with:
key: ${{ github.repository }}-npm-cache
path: ~/.npm

- name: Mount node_modules
uses: useblacksmith/stickydisk@v1
with:
key: ${{ github.repository }}-node-modules
path: ./node_modules

- name: Install Dependencies
run: npm ci

- name: Build
run: npm run build
```

### Delete All Caches with a Prefix

```yaml
- name: Delete All npm Caches
uses: useblacksmith/cache-delete@v1
with:
key: npm-
prefix: true
```
## Bazel Build Caching

### Delete All Caches
Bazel's remote cache can significantly improve build times, but uploading and downloading cached artifacts can still be a bottleneck. Using sticky disks with Blacksmith runners provides near-instant access to your Bazel caches as they are bind mounted into your runners on demand. Our [`useblacksmith/setup-bazel@v2`](https://github.com/useblacksmith/setup-bazel) action is a zero-confg way to use sticky disks to store the disk, repository, and external cache.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo detected: 'zero-confg' should be 'zero-config'.

Suggested change
Bazel's remote cache can significantly improve build times, but uploading and downloading cached artifacts can still be a bottleneck. Using sticky disks with Blacksmith runners provides near-instant access to your Bazel caches as they are bind mounted into your runners on demand. Our [`useblacksmith/setup-bazel@v2`](https://github.com/useblacksmith/setup-bazel) action is a zero-confg way to use sticky disks to store the disk, repository, and external cache.
Bazel's remote cache can significantly improve build times, but uploading and downloading cached artifacts can still be a bottleneck. Using sticky disks with Blacksmith runners provides near-instant access to your Bazel caches as they are bind mounted into your runners on demand. Our [`useblacksmith/setup-bazel@v2`](https://github.com/useblacksmith/setup-bazel) action is a zero-config way to use sticky disks to store the disk, repository, and external cache.


```yaml
- name: Delete All Caches
uses: useblacksmith/cache-delete@v1
with:
key: ""
prefix: true
jobs:
build:
runs-on: blacksmith
steps:
- uses: actions/checkout@v4

- name: Setup Bazel
uses: useblacksmith/setup-bazel@v2
with:
version: '6.x'

- name: Build
run: |
bazel build //...
```

## Error Handling

The action will:

- Fail if the cache deletion request fails (non-404 error)
- Log a message if the cache is not found (404)
- Successfully complete if the cache is deleted
### Cache Performance Comparison

## License
| Caching Solution | Cache Size | Average Download Speed | Time to Access |
|-----------------|------------|----------------|----------------|
| GitHub Actions Cache | 6GB | 90 MB/s | ~1m6s |
| Blacksmith Cache | 6GB | 400 MB/s | ~15s |
| Sticky Disks | 6GB | N/A | 3 seconds |

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
21 changes: 9 additions & 12 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
name: "Delete Blacksmith Cache"
author: Aayush Shah
description: "Deletes a cache or specific cache version from Blacksmith"
name: "Blacksmith Sticky Disk"
author: Aditya Maru
description: "Creates a sticky disk on Blacksmith"
branding:
icon: folder-plus
color: black
inputs:
key:
description: "The cache key to delete"
required: false
version:
description: "Specific version of the cache to delete (optional)"
required: false
prefix:
description: "Treat key as a prefix for bulk deletion"
required: false
default: "false"
description: "A unique key to identify the sticky disk"
required: true
path:
description: "The path at which to mount the sticky disk"
required: true
runs:
using: "node20"
main: "dist/index.js"
post: "dist/post/index.js"
Binary file added arch-dark-mode.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added arch-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading