Proposal to re-architect Arrow Swift

[willtemperley]

Describe the enhancement requested

Arrow Swift can be made significantly more usable and performance can be significantly improved through introducing some major changes.

The problems

Buffers can only be a single type

Every ArrowArray holds an ArrowData instance, which holds an array of ArrowBuffers. An ArrowBuffer allocates its own memory. This means that IPC cannot be zero-copy, because the buffer cannot be any other type, such as a view over another binary object like Data.

This is compounded by the fact that the ArrowBuffer exposes its memory via:



public let rawPointer: UnsafeMutableRawPointer



Pointer arithmetic is then being done within the subscript of arrays. This is actually duplicated in each array. 
Second, having a public raw pointer like that is dangerous, e.g. it could easily cause use-after-free errors.


Allowing multiple buffer types, e.g. variable length type buffers and fixed type buffers can improve performance and memory safety.

ArrowTypes

Currently two reference types are used to describe an ArrowType. There is no reason I can think of for An arrow type to be mutable. Arrow types can be represented as a single value type, using an enum. This is likely improve performance and make the library far more usable, with enums offering much better ergonomics and safety, with exhaustiveness checking and simpler equality checks, for example.

Feasibility

This is definitely feasible because I have already done most of this in Swift Arrow:
https://github.com/willtemperley/swift-arrow/

This version includes support for all types that Arrow Swift covers, plus schema metadata, map types, binary views, binary with 64 bit offsets and nested types with 64 bit offsets.

It also passes Arrow Gold integration tests for these types.

I have compeltely rewritten the buffer infrastructure. Zero-copy IPC is available using views over memory-mapped Data objects.

I have replaced ArrowType with an enum which was adapted from from arrow-rs.

The challenge

Changing ArrowType to an enum would completely change the public API.
Replacing the buffer infrastructure means an almost complete rewrite of arrays and array builder.

Caveats

My version does not include C import / export and Arrow Flight support is pending.

Suggested way forward

I think this would simply need to be a clean-break fork, with a major version bump and a migration guide.
I'm happy to donate any code I've written in Swift Arrow back to Arrow Swift.

[kou]

@abandy Do you have any opinion for this proposal?

[abandy]

Hi @kou and @willtemperley, I hope all is well. I disagree with the statement "Buffers can be of a single type". A buffer points to a location in memory that can contain any data. So the data it stores can be heterogeneous.

Once data is loaded into an ArrowArray then that array can be shared by Tables and RecordBatches and hence is Zero-Copy. The data within a buffer is immutable and views could be added to filter this data without a rewrite. The ArrowBuffer init method takes in an UnsafeMutableRawPointer and this pointer can point to an existing source in memory and that is why IPC (C import/export) works.

There have been many changes in Swift since the inception of this library (pre Swift 5) so updating with current concepts definitely makes sense. If we are going to make an API breaking change then maybe we should meet and discuss to go over the changes? That way we can discuss the different approaches based on Swift language changes to ensure we get to a stable API.

[willtemperley]

@abandy , thanks for the feedback! To clarify, when I mentioned IPC, I was specifically referring to the Arrow IPC Format. With the current approach, all data is copied into ArrowBuffers rather than allowing a zero-copy view over backing storage. Perhaps this can be done without a buffer redesign though. Maybe @kou could shed some light on how Arrow CPP does this, allowing buffers to be backed by Memory Mapped Files or GPU memory.

Regarding the pointer arithmetic in arrays, I think we really should be making use of bound memory, by doing something like:

    let values: UnsafePointer<T>

    public subscript(index: Int) -> T {
        return values[index]
    }

Instead of the the current setup, which is doing a lot of extra work:

return self.arrowData.buffers[1].rawPointer.advanced(by: byteOffset).load(as: T.self)

Perhaps the buffer work could be done without a breaking change.

Yes I'd be happy to meet up and discuss this.

[abandy]

Hi @willtemperley, gotcha, thank you for the clarification on IPC! Cool, we can definitely look into add a backing storage to the current implementation and will wait for @kou to add the requested clarification.

Yeah, I see your point in regards to the pointer arithmetic. That looks like a nice performance boost and simplification to minimize the call chain!!

I am busy this weekend and next week but am available after that. Let me know what time works for you. @kou would like to join as well?

[kou]

Arrow C++ uses IO interface for buffer from memory mapped file.

Arrow C++'s IO interface has supports_zero_copy() predicate: https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/interfaces.h#L226-L229

If supports_zero_copy() returns true, the IO can return a buffer that just refers memory mapped data (zero copy) by Read() (and its family):
https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/file.cc#L664-L680
https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/file.cc#L530-L539

Arrow C++ provides Device and MemoryManager for GPU (and CPU but, in general, they aren't used for CPU expiclity). MemoryManager can allocate a buffer from GPU memory and create an IO that read/write from/to GPU memory:
https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/gpu/cuda_context.h#L253-L260

All Buffers have a pointer, an offset and a size.

A Buffer can have parent Buffer for a sliced Buffer. It's for preventing freeing the parent Buffer while children Buffers are alive. Memory mapped IO has a parent Buffer and returns a sliced Buffer from the parent Buffer for Read().

A CudaBuffer has CudaMemoryManager. It's for preventing freeing the CudaMemoryManager while allocated Buffers are alive.

In Arrow Swift, we may want to add ArrowBuffer.slice() to create a sub buffer without copying.

(Does this explanation provide enough information...? If there are missing information, please let me know.)

@kou would like to join as well?

I can't join it. Sorry! I don't listen/speak English... I only read/write English. If you share summary or something here after the meeting, I may be able to provide additional information.

[willtemperley]

Thanks @kou that’s really useful background. This is probably a battle tested architecture that can map well to Swift. @abandy I’m available to chat during UTC+8 waking hours most days next week.

[abandy]

Thank @kou for the info! @willtemperley awesome, maybe we can do 04/20 @ 9pm ET (I think that is 9am 4/21 for you)?

[willtemperley]

@abandy actually I think I'd rather just discuss here thanks.