Add content addressable gems support

[jenshenny]

#9654

TL;DR

Adds RubyGems and Bundler client support for content-addressable ("skinny") binary gems: one artifact per Ruby ABI, named with a SHA-256 prefix.

nokogiri-1.18.9-x86_64-linux.gem # traditional platform gem
nokogiri-1.18.9-78be552b.gem     # content-addressable gem

The branch covers build, discovery, install, display, yank, lockfiles, caching, and bundle install --local. Existing source and platform gems are unchanged.

Why

"Fat" binary gems contain every supported Ruby ABI and keep growing. Skinny binaries are smaller, but builds for the same gem, version, and platform need distinct filenames. A content-derived suffix gives each artifact a unique identity.

A content address must be 8–64 lowercase hexadecimal characters. RubyGems only treats it as one when the gem also has a non-Ruby platform and constrained required_ruby_version, avoiding false matches with ordinary filenames.

What changes

Flow	Behaviour	Code
gem build	--ruby-abi X.Y validates the ABI, platform, and required_ruby_version; defaults the requirement to ~> X.Y.0; builds in memory; writes name-version-<sha>.gem. Without the option, behaviour is unchanged.	issue · commit · PR
gem push	Adds --platform and --ruby-abi selectors. Given multiple SHA-named files, RubyGems reads their specifications and pushes the single matching artifact. No match or multiple matches raise instead of guessing; passing one exact filename remains unchanged.	issue · commit · PR
Local gem install	Reads the archive, derives the hash from its filename, verifies it against the file's SHA-256, and preserves both hash and platform. Installed directory and gemspec use the hash. A mismatch fails; reinstalling is idempotent.	issue · commit · PR
Remote gem install	Decodes the compact-index hash and platform:= metadata separately. Distinct hashes remain distinct candidates. A compatible skinny gem is preferred; resolution falls back to a traditional platform gem, then a source gem. Downloads the SHA-named archive and development gemspec.	issue · commit · PR
Remote CLI output	gem list -r, gem search -r, and gem info -r show the real platform and Ruby ABI, not the hash. Versions/platforms are separated; ABIs for the same version/platform are grouped.	issue · identity · decoding · display · PR
gem yank	Adds --ruby-abi; sends gem name, version, platform, and ABI so the server can select one skinny variant. Existing yank behaviour remains unchanged without it.	issue · commit · PR
Remote bundle install	Separates compact-index hash from the real platform, includes the hash in candidate identity, selects an ABI-compatible skinny gem, and falls back to platform/source gems when needed.	issue · commit · PR
Lockfile and local cache	Writes platform and hash to Gemfile.lock, parses both on the next run, and installs the SHA-named artifact from vendor/cache with bundle install --local. Remote and local paths produce the same installed directory; checksums remain keyed by the platform lock name.	issue · commit · PR

Formats

Build:

gem build nokogiri.gemspec --ruby-abi 3.4
# => nokogiri-1.18.9-78be552b.gem

Compact index:

1.18.9-78be552b |ruby:~> 3.4.0,platform:= x86_64-linux

Remote query output:

mygem (1.0 Platform: x86_64-linux, Ruby ABI: 3.3, 3.4)

Lockfile:

GEM
  remote: https://rubygems.org/
  specs:
    mygem (1.0-x86_64-linux) 78be552b

Cache and install paths:

cache/mygem-1.0-78be552b.gem
gems/mygem-1.0-78be552b/

Targeted yank:

gem yank mygem -v 1.0.0 --platform x86_64-linux --ruby-abi 3.4

Each part was reviewed independently before being assembled for the feature branch.

Tophatting

1. Build two variants with the same name/version/platform but different ABIs; confirm different SHA filenames.
2. Run gem push name-*.gem --platform <platform> --ruby-abi <ABI>; confirm it selects the matching file. Confirm ambiguous and non-matching selectors fail rather than guessing.
3. Serve the variants with platform/source fallbacks; confirm gem install and bundle install select the compatible skinny gem and fall back on an incompatible Ruby.
4. Confirm the installed directory, gemspec, cache file, and lockfile preserve the expected hash/platform identities.
5. Move the cached gem to vendor/cache, remove the installed copy, and confirm bundle install --local recreates the same directory without contacting the server.
6. Confirm remote query commands display platform/ABI rather than the hash, then yank one variant with --platform and --ruby-abi.