Add Versionstamp and Subspace support to Swift bindings #16

1amageek · 2025-11-01T15:27:04Z

Summary

This PR adds two major missing features to the Swift bindings:

Versionstamp support - Full tuple-layer versionstamp implementation
Subspace support - Key namespace management with range operations

These features bring the Swift bindings to parity with Python, Go, and Java bindings, enabling the development of Record Layer and other advanced FoundationDB applications in Swift.

Motivation

The Swift bindings currently lack essential features available in other language bindings, making it difficult to:

Use versionstamped keys for temporal ordering
Manage key namespaces efficiently
Build Record Layer or similar frameworks

This PR addresses these gaps by implementing the missing features following the canonical behavior of official bindings.

Versionstamp Implementation

Core Features

Versionstamp struct: 12-byte value (10-byte transaction version + 2-byte user version)
Incomplete versionstamps: Support for transaction-time assignment
Tuple integration: Full encode/decode support with type code 0x33
packWithVersionstamp(): Automatic offset calculation for atomic operations

Usage Example

// Create incomplete versionstamp
let vs = Versionstamp.incomplete(userVersion: 0)
let tuple = Tuple("user", 12345, vs)

// Pack with automatic offset calculation
let key = try tuple.packWithVersionstamp()

// Use with atomic operation
transaction.atomicOp(
    key: key,
    param: value,
    mutationType: .setVersionstampedKey
)

// Read back and decode
let storedKey = try await transaction.get(someKey)
let decoded = try Tuple.decode(from: storedKey)
let versionstamp = decoded[2] as? Versionstamp

Subspace Implementation

Core Features

Subspace struct: Key namespace management with tuple encoding
range() method: Returns (prefix + [0x00], prefix + [0xFF]) for tuple data
strinc() algorithm: String increment for raw binary prefixes
prefixRange() method: Complete prefix coverage using strinc

Usage Example

// Create subspace
let userSpace = Subspace(rootPrefix: "users")
let activeUsers = userSpace.subspace("active")

// Pack keys with prefix
let key = activeUsers.pack(Tuple(userId, "name"))

// Range queries
let (begin, end) = activeUsers.range()
let records = try await transaction.getRange(beginKey: begin, endKey: end)

// Raw binary prefix support
let rawSubspace = Subspace(prefix: [0x01, 0xFF])
let (begin, end) = try rawSubspace.prefixRange()  // Uses strinc

Testing

Comprehensive test suite with 150 tests (all passing):

Versionstamp Tests (20)

Incomplete/complete versionstamp creation
Byte encoding/decoding
Tuple packing with offset calculation
Roundtrip tests (encode → decode)
Error handling

Subspace Tests (22)

Range operations (range() and prefixRange())
Tuple packing/unpacking
Namespace containment checks
Edge cases (0xFF handling)

String Increment Tests (14)

strinc algorithm correctness
Trailing 0xFF handling
Cross-language compatibility (Java, Go)
Error cases

Integration

All tests verified with Swift Testing framework
Clean build with no warnings
Memory-safe operations

Compatibility

Cross-Language Consistency

This implementation follows the canonical behavior of official bindings:

Java: ByteArrayUtil.strinc(), Versionstamp, Subspace
Python: fdb.strinc(), fdb.tuple.Versionstamp, tuple packing
Go: fdb.Strinc(), fdb.IncompleteVersionstamp(), Subspace.FDBRangeKeys()
C++: Range semantics match C++ implementation

API Version

Supports API 520+ (4-byte offsets)
Dead code for API < 520 removed for clarity
Follows modern FoundationDB best practices

Files Changed

New Files

Sources/FoundationDB/Versionstamp.swift (196 lines)
- Core Versionstamp implementation
- TupleElement conformance
- Comprehensive documentation
Sources/FoundationDB/Tuple+Versionstamp.swift (205 lines)
- packWithVersionstamp() method
- Validation and helper methods
- API 520+ offset handling
Sources/FoundationDB/Subspace.swift (424 lines)
- Subspace implementation
- range() and prefixRange() methods
- strinc() algorithm
- SubspaceError enum
Tests/FoundationDBTests/VersionstampTests.swift (416 lines)
Tests/FoundationDBTests/SubspaceTests.swift (312 lines)
Tests/FoundationDBTests/StringIncrementTests.swift (194 lines)

Modified Files

Sources/FoundationDB/Tuple.swift (+3 lines)
- Add versionstamp case to Tuple.decode() switch
- Enables automatic versionstamp decoding

Statistics

Total additions: ~1,750 lines
Test coverage: 56 new tests
Documentation: Comprehensive inline documentation for all public APIs

Breaking Changes

None. This PR is purely additive and maintains full backward compatibility with existing code.

Future Work

This PR lays the groundwork for:

FoundationDB Record Layer implementation in Swift
Directory Layer support
Advanced indexing patterns
Versionstamp-based temporal queries

Checklist

References

FoundationDB Versionstamp Documentation
Subspace Documentation
Python bindings: bindings/python/fdb/tuple.py
Go bindings: bindings/go/src/fdb/tuple/tuple.go
Java bindings: bindings/java/src/main/com/apple/foundationdb/tuple/Versionstamp.java

This commit adds two major features to the Swift bindings: ## Versionstamp Support - Implement 12-byte Versionstamp structure (10-byte transaction version + 2-byte user version) - Add incomplete versionstamp support for transaction-time assignment - Implement Tuple integration with versionstamp encoding (type code 0x33) - Add packWithVersionstamp() for atomic operations ## Subspace Implementation - Implement Subspace for key namespace management with tuple encoding - Add range() method using prefix + [0x00] / prefix + [0xFF] pattern - Implement strinc() algorithm for raw binary prefix support - Add prefixRange() method for complete prefix coverage - Define SubspaceError for proper error handling ## Testing - Add VersionstampTests with 15 test cases - Add StringIncrementTests with 14 test cases for strinc() algorithm - Add SubspaceTests with 22 test cases covering range() and prefixRange() - Verify cross-language compatibility with official bindings All implementations follow the canonical behavior of official Java, Python, Go, and C++ bindings.

This commit completes the Versionstamp implementation by adding decode support and comprehensive roundtrip tests. ## Tuple.decode() Integration - Add versionstamp case (0x33) to Tuple.decode() switch - Enable automatic Versionstamp decoding in tuples - Allows reading versionstamped keys from database ## Roundtrip Tests - Add 5 roundtrip tests (encode → decode) - Complete versionstamp roundtrip - Incomplete versionstamp roundtrip - Mixed tuple with multiple types - Multiple versionstamps in one tuple - Error handling for insufficient bytes ## Test Fixes - Fix withUnsafeBytes crash by ensuring exact 4-byte array - Add size validation before unsafe memory access - Fix range test expectations (prefix vs prefix + [0x00]) ## Code Cleanup - Remove dead code for API < 520 (no longer supported) - Simplify to single code path using 4-byte offsets - Update documentation to reflect API 520+ requirement All 150 tests now pass successfully.

glbrntt · 2025-11-07T11:13:33Z

Sources/FoundationDB/Subspace.swift

+        let tupleBytes = Array(key.dropFirst(prefix.count))
+        let elements = try Tuple.decode(from: tupleBytes)


This is quite inefficient: You're allocating a new array and then copying all but one element from the key array. It seems like the various decode methods might be better implemented as being generic over a Collection<UInt8>. That would allow you to decode a slice without going via an intermediate array.

glbrntt · 2025-11-07T11:15:42Z

Sources/FoundationDB/Subspace.swift

+extension Subspace: Equatable {
+    public static func == (lhs: Subspace, rhs: Subspace) -> Bool {
+        return lhs.prefix == rhs.prefix
+    }
+}
+
+// MARK: - Hashable
+
+extension Subspace: Hashable {
+    public func hash(into hasher: inout Hasher) {
+        hasher.combine(prefix)
+    }
+}


Can the compiler not synthesise these conformances for you?

glbrntt · 2025-11-07T12:52:20Z

Sources/FoundationDB/Subspace.swift

+// MARK: - SubspaceError
+
+/// Errors that can occur in Subspace operations
+public enum SubspaceError: Error {


enums generally make for bad error types as you can't add new cases without breaking API.

The most common model is having an error code and an error message wrapped up in a struct. It's not uncommon to then attach a few other things too (like an underlying error if applicable). SwiftNIO's FileSystemError is a good example of this (see here).

glbrntt · 2025-11-07T12:55:24Z

Sources/FoundationDB/Subspace.swift

+    /// ```
+    ///
+    /// - SeeAlso: `Subspace.prefixRange()` for usage with Subspace
+    public func strinc() throws -> FDB.Bytes {


I just want to call out again that this is another reason we're pretty strongly opposed to using typealiases in public APIs: any extension on the typealias pollutes the aliased type.

glbrntt · 2025-11-07T12:56:49Z

Sources/FoundationDB/Subspace.swift

+        }
+
+        // Check if result is empty (input was empty or all 0xFF)
+        guard !result.isEmpty else {


nit: guard !X is less readable than if X

glbrntt · 2025-11-07T13:51:14Z

Sources/FoundationDB/Tuple+Versionstamp.swift

+        return elements.reduce(0) { count, element in
+            if let vs = element as? Versionstamp, !vs.isComplete {
+                return count + 1
+            }
+            return count
+        }


There's a count(where:) method you can use for this:

return elements.count { if let vs = element as? Versionstamp { return !vs.isComplete } return false }

glbrntt · 2025-11-07T13:53:58Z

Sources/FoundationDB/Tuple+Versionstamp.swift

+        guard incompleteCount == 1 else {
+            throw TupleError.invalidEncoding
+        }


guard is great for early returns. The else branch of the guard here is no earlier than the guarded path; if would be more idiomatic and easier to read here:

if incompleteCount != 1 { throw TupleError.invalidEncoding }

glbrntt · 2025-11-07T13:56:05Z

Sources/FoundationDB/Tuple+Versionstamp.swift

+            offset += 1
+
+            switch typeCode {
+            case TupleTypeCode.versionstamp.rawValue:


It would be more idiomatic to turn your typeCode into a typed TupleTypeCode so that you can switch over it more naturally.

glbrntt · 2025-11-07T13:58:34Z

Sources/FoundationDB/Versionstamp.swift

+        var bytes = transactionVersion ?? Self.incompletePlaceholder
+
+        // User version is stored as big-endian
+        bytes.append(contentsOf: withUnsafeBytes(of: userVersion.bigEndian) { Array($0) })


This is another avoidable array allocation.

glbrntt · 2025-11-07T13:59:47Z

Sources/FoundationDB/Versionstamp.swift

+    public func hash(into hasher: inout Hasher) {
+        hasher.combine(transactionVersion)
+        hasher.combine(userVersion)
+    }
+
+    public static func == (lhs: Versionstamp, rhs: Versionstamp) -> Bool {
+        return lhs.transactionVersion == rhs.transactionVersion &&
+               lhs.userVersion == rhs.userVersion
+    }
+


Any reason the compiler can't synthesise these?

MMcM · 2025-11-07T14:42:55Z

Sources/FoundationDB/Subspace.swift

+
+    /// Create a subspace with a string prefix
+    /// - Parameter rootPrefix: The string prefix (will be encoded as a Tuple)
+    public init(rootPrefix: String) {


I am not sure about the need to privilege strings like this. In practice, I would expect that the root of a subspace would come from the Directory Layer, which we do not yet have for Swift, but should.

MMcM · 2025-11-07T14:44:04Z

Sources/FoundationDB/Subspace.swift

+    /// - Returns: The encoded key with prefix
+    ///
+    /// The returned key will have the format: `[prefix][encoded tuple]`
+    public func pack(_ tuple: Tuple) -> FDB.Bytes {


I have no strong preference between pack / unpack and encode / decode, but I do not see any reason for Subspace and Tuple to differ in their choice. Perhaps others can comment on which feels more natural in this binding.

MMcM · 2025-11-07T14:59:49Z

Sources/FoundationDB/Subspace.swift

+    /// let activeUsers = users.subspace("active")  // prefix = users + "active"
+    /// let userById = activeUsers.subspace(12345)  // prefix = users + "active" + 12345
+    /// ```
+    public func subspace(_ elements: any TupleElement...) -> Subspace {


Some other bindings also overload subscript to do this. For example, Python adds a __getitem__. Although Rust's Subspace does not implement Index, so it is not universal. But I have found it convenient.

Remove manual implementations of Equatable and Hashable in Versionstamp and Subspace. The compiler can synthesize these conformances automatically since all stored properties already conform to these protocols. This improves code maintainability and follows Swift best practices. Addresses: glbrntt review comment FoundationDB#2

- Use count(where:) instead of reduce for counting incomplete versionstamps This is more idiomatic Swift and clearly expresses intent - Replace guard with if for incompleteCount validation Guard is for early returns; if is more appropriate here - Optimize withUnsafeBytes to eliminate unnecessary allocation Append directly instead of creating intermediate Array Addresses: glbrntt review comments FoundationDB#6, FoundationDB#7, FoundationDB#10

Move strinc() from FDB.Bytes extension to FDB static method. Since FDB.Bytes is a typealias for [UInt8], extending it would pollute all Array<UInt8> types with the strinc() method in the public API. The new API matches other language bindings more closely: - Go: fdb.Strinc() - Python: fdb.strinc() - Java: ByteArrayUtil.strinc() Usage: try FDB.strinc(bytes) instead of try bytes.strinc() Addresses: glbrntt review comment FoundationDB#4

Convert SubspaceError from enum to struct-based error with error codes. This design follows SwiftNIO's FileSystemError pattern and provides better extensibility for future error cases. The struct-based approach will make it easier to add new error types when the Directory Layer is introduced in the next PR. Also replace guard with if in strinc error check for better readability, as guard is intended for early returns. Addresses: glbrntt review comments FoundationDB#3, FoundationDB#5

1. Use TupleTypeCode enum in switch statements for type safety Convert raw UInt8 type code to TupleTypeCode enum before switching. This provides compile-time safety and makes the code more maintainable. 2. Rename encode/decode to pack/unpack for cross-language consistency All official FoundationDB bindings use pack/unpack terminology: - Python: tuple.pack() / tuple.unpack() - Java: Tuple.pack() / Tuple.fromBytes() - Go: Tuple.Pack() / Tuple.Unpack() Using pack/unpack: - Matches established FDB terminology ("tuple packing") - Avoids confusion with Swift's Codable encode/decode - Improves searchability and documentation consistency API changes: - Tuple.encode() → Tuple.pack() - Tuple.decode(from:) → Tuple.unpack(from:) - Subspace methods already used pack/unpack (no change) Addresses: glbrntt review comment FoundationDB#9, MMcM review comment FoundationDB#2

Update all tests to reflect API changes: - Use Tuple.pack() / Tuple.unpack() instead of encode/decode - Use Tuple().pack() for Subspace prefix initialization - Update SubspaceError checks to use struct-based error with .code - Update StackTester to use new API All 150 tests pass successfully.

1amageek · 2025-11-08T04:35:35Z

pack/unpack Naming Decision

Thank you for raising this important design question. After careful consideration, I've decided to standardize on pack/unpack throughout the Swift bindings, aligning with all other official FoundationDB language bindings.

Current State Analysis

The Swift bindings initially had inconsistent naming:

Tuple: encode() / decode()
Subspace: pack() / unpack() ✅
Versionstamp: packWithVersionstamp() ✅

This inconsistency needed resolution.

Cross-Language Consistency

All official FoundationDB bindings use pack/unpack:

Python:
```python
packed = fdb.tuple.pack(('user', 123))
items = fdb.tuple.unpack(packed)
packed = fdb.tuple.pack_with_versionstamp(('user', vs))
```

Java:
```java
byte[] packed = Tuple.from("user", 123).pack();
Tuple tuple = Tuple.fromBytes(packed);
```

Go:
```go
packed := tuple.Tuple{"user", 123}.Pack()
elements, _ := tuple.Unpack(packed)
```

C++ (via bindings documentation): Uses "pack" terminology

Why `pack/unpack` Instead of `encode/decode`

1. Established FDB Terminology

Official documentation: "tuple packing"
Community discussions: "pack your data"
Existing APIs: `packWithVersionstamp()`

2. Avoids Confusion with Swift's Codable

Swift developers strongly associate `encode/decode` with the `Codable` protocol:
```swift
// Swift's Codable
let data = try JSONEncoder().encode(user)
let user = try JSONDecoder().decode(User.self, from: data)
```

Using `pack/unpack` clearly signals this is FDB-specific tuple encoding, not general Swift serialization.

3. Semantic Clarity

pack: "Compress multiple elements into ordered bytes" (with ordering preservation)
encode: "Convert to bytes" (generic serialization)

Tuple packing is special—it preserves lexicographic ordering, which is critical for range queries.

4. Internal vs Public API

The `TupleElement` protocol still uses `encodeTuple()`/`decodeTuple()` internally (for implementation), while the public API uses `pack()`/`unpack()` (for users). This separation is appropriate:
```swift
// Public API - user-facing
let bytes = tuple.pack()
let elements = try Tuple.unpack(from: bytes)

// Internal protocol - implementation detail
protocol TupleElement {
func encodeTuple() -> FDB.Bytes
static func decodeTuple(from: FDB.Bytes, at: inout Int) throws -> Self
}
```

Implementation

I've updated the code to use `pack/unpack` consistently:

`Tuple.encode()` → `Tuple.pack()`
`Tuple.decode(from:)` → `Tuple.unpack(from:)`
Subspace already uses `pack/unpack` ✅
`packWithVersionstamp()` unchanged ✅

All 150 tests pass with the new API.

Benefits

Cross-language consistency: Developers moving between Python/Java/Go/Swift will use the same terminology
Clear FDB context: No confusion with Swift's Codable
Better searchability: Searching "FoundationDB pack" yields relevant results across all languages
Matches existing APIs: `packWithVersionstamp()` already established this pattern

This decision prioritizes FoundationDB ecosystem consistency over Swift-specific conventions, which is appropriate for a database client library where cross-language compatibility is valuable.

1amageek · 2025-11-08T04:43:00Z

@glbrntt @MMcM Thank you both very much for your thorough and insightful code reviews! Your feedback has significantly improved the code quality, Swift idiomaticity, and API design of this PR.

Changes Implemented

@glbrntt's review comments:

✅ Generic Collection for decode - Noted as valuable future optimization (not included in this PR to keep scope manageable)
✅ Compiler synthesis for Equatable/Hashable - Removed manual implementations
✅ Struct-based SubspaceError - Converted to SwiftNIO-style error model for extensibility
✅ Avoid extending FDB.Bytes typealias - Moved to FDB.strinc() static method
✅ Guard vs if improvements - Replaced unnecessary guard statements
✅ Use count(where:) - Replaced reduce with more idiomatic approach
✅ TupleTypeCode enum in switch - Added type-safe enum conversion
✅ Optimize withUnsafeBytes - Eliminated unnecessary allocations

@MMcM's review comments:

✅ Remove rootPrefix initializer - Removed string initializer; prefixes should come from Directory Layer (next PR)
✅ Standardize pack/unpack naming - Changed Tuple.encode()/decode() to pack()/unpack() for cross-language consistency (detailed rationale in separate comment)
✅ Add subscript operator - Implemented for convenient nested subspace creation, matching Python's __getitem__ pattern

All 150 tests pass successfully after these changes.

Ready for Re-review

I've committed these changes in separate, focused commits for easier review:

Compiler synthesis improvements
Code quality enhancements
Error model refactoring
API namespace cleanup
Naming alignment with other FoundationDB bindings
Test updates

The code is now more idiomatic Swift while maintaining strong cross-language compatibility.

Note: I'm not a native English speaker and am using AI assistance to help communicate my intent clearly. Please let me know if anything needs clarification.

Thank you again for your time and expertise! I look forward to your feedback.

1amageek added 2 commits November 1, 2025 15:26

jzhou77 requested review from glbrntt, rnro and vishesh November 6, 2025 17:35

jzhou77 assigned vishesh Nov 6, 2025

1amageek mentioned this pull request Nov 7, 2025

Add Directory Layer implementation #17

Closed

6 tasks

glbrntt reviewed Nov 7, 2025

View reviewed changes

MMcM suggested changes Nov 7, 2025

View reviewed changes

1amageek added 6 commits November 8, 2025 13:30

1amageek mentioned this pull request Nov 8, 2025

Establish AI-Assisted Development Guidelines in AI_DEVELOPMENT_GUIDELINES.md #19

Open

		let tupleBytes = Array(key.dropFirst(prefix.count))
		let elements = try Tuple.decode(from: tupleBytes)

Add Versionstamp and Subspace support to Swift bindings #16

Are you sure you want to change the base?

Add Versionstamp and Subspace support to Swift bindings #16

Uh oh!

Conversation

1amageek commented Nov 1, 2025

Summary

Motivation

Versionstamp Implementation

Core Features

Usage Example

Subspace Implementation

Core Features

Usage Example

Testing

Versionstamp Tests (20)

Subspace Tests (22)

String Increment Tests (14)

Integration

Compatibility

Cross-Language Consistency

API Version

Files Changed

New Files

Modified Files

Statistics

Breaking Changes

Future Work

Checklist

References

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

1amageek commented Nov 8, 2025

pack/unpack Naming Decision

Current State Analysis

Cross-Language Consistency

Why `pack/unpack` Instead of `encode/decode`

1. Established FDB Terminology

2. Avoids Confusion with Swift's Codable

3. Semantic Clarity

4. Internal vs Public API

Implementation

Benefits

Uh oh!

1amageek commented Nov 8, 2025

Changes Implemented

Ready for Re-review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!