Skip to content

[RFC] [cross_file] New architecture. #7591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion packages/cross_file/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
## NEXT
## 0.3.5

* `XFile` is now a read-only `interface`.
* Added `XFileFactory` classes for `native` and `web` to create `XFile` instances.
* Deprecated the former `XFile` constructors (`XFile` and `XFile.fromData`)
* Updates minimum supported SDK version to Flutter 3.19/Dart 3.3.

## 0.3.4+2
Expand Down
39 changes: 29 additions & 10 deletions packages/cross_file/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,38 @@ An abstraction to allow working with files across multiple platforms.

## Usage

Import `package:cross_file/cross_file.dart`, instantiate a `XFile`
using a path or byte array and use its methods and properties to
access the file and its metadata.
Many packages use `XFile` as their return type. In order for your
application to consume those files, import
`package:cross_file/cross_file.dart`, and use its methods and properties
to access the file data and metadata.

Example:
In order to instantiate a new `XFile`, import the correct factory class,
either from `package:cross_file/native/factory.dart` (for native development) or
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small drive-by comment:

Since the two implementations now have platform-specific imports (dart:io or package:web)
users who take the imports at face value might run into compilation issues, because they lack a conditional import?

Can we provide guidance on how to avoid this? With the legacy API we had the File or Blob types as internals and not on the public interface, which allowed for the plugin to provide a conditional import shim.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@navaronbracke thanks for the comment! This is a very interesting problem, for which I don't have a great answer. The TL;DR/Guidance is:

  • Are you reading XFiles? Use the XFile interface only.
  • Are you creating XFiles?
    • In mobile? Use the native/factory.dart
    • In web? Use the web/factory.dart

Each factory exposes constructors that are platform-specific, and we really don't have a cross-platform file constructor.

What problems do you think people are going to have? Is this a documentation problem, or a design issue?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If people take the comment at face value and do import "package:cross_file/web/factory.dart"; directly, they will run into errors like library dart:js_interop is not available on this platform. Since they don't directly import dart:js_interop, this might be confusing to new users.

I think this is merely a documentation issue. We should put emphasis on:

  1. If this is in a platform-specific plugin implementation, you can probably import it directly (as the plugin system only uses the platform implementations of your plugin on the correct platforms)
  2. If this is in a Flutter app, you'll have to use a conditional import on if (dart.library.js_interop) ... and you probably need an extra abstraction, since the web-only types (i.e. Blob) cannot be imported in native implementations (for the same reason why the plugin cannot provide the conditional import here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, if it's a documentation issue, it's probably easier to fix (LOL famous last words :P)

Do you have any ideas with how this comment would be clearer? What would you like to read?

(I'll try to come up with something more specific as well!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the main point is that the compiler itself would want you to use a conditional import.
So perhaps something like:

"In order to instantiate a new XFile, import the correct factory class, through a conditional import.
Using a conditional import is required to use types that are only available on the current platform.

import 'package:cross_file/native/factory.dart'
 if (dart.library.js_interop) 'package:cross_file/web/factory.dart'

"

The wording is just a suggestion, so feel free to tweak it.

Copy link
Contributor

@navaronbracke navaronbracke Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, wait. Since the API for both factories is divergent, would a user that uses this conditional import know at compile time if they could access the fromFile() (both with a dart:io or package:web File) or the fromBlob()/fromObjectUrl() methods? I think that Dart will only provide the base type XFileFactory, but I think you don't get a guarantee for the available methods.

Perhaps we can fix that by having the interface like:

  • XFileFactory is the base type that is exported through a conditional export. It provides all the methods that do not include platform-specific types (so only fromPath/fromBytes/fromObjectUrl)
  • WebXFileFactory and NativeXFileFactory implement that interface
  • WebXFileFactory provides the fromBlob/fromFile implementations for package:web types
  • NativeXFileFactory provides the fromFile implementation for dart:io
  • unsupported methods on a platform just throw UnsupportedError
  • we direct users to use kIsWeb and/or a type check on the factory
  • since we do the conditional export, the comment above can stay unchanged, no need to adjust the wording when we provide the export ourselves. We'll have to add guidance for checking which factory is provided instead (simple type check should do)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification @navaronbracke! I see what you mean! You want the XFileFactory to be cross-platform as well!

IMO the only cross-platform class from this package is the XFile interface! 1

If your cross-platform App needs to "create XFiles", you'll end up with some platform-aware code (because as you said, you can't import web stuff in native code), like this:

// Your app's main.dart
import 'src/get_file/get_file.dart'; // Platform-aware, see below

// ...

final XFile myFile = await getFile(/* config? data? */);

With a directory like this:

src/
  get_file/
    get_file.dart
    get_file_web.dart
    get_file_native.dart

Your get_file.dart could conditionally export the right file for the platform; like:

// get_file.dart

export "get_file_native.dart"
  if (dart.library.js_interop) "get_file_web.dart";

So each implementation file can use the platform-specific factories:

get_file_native.dart get_file_web.dart
import 'package:cross_file/native/factory.dart';

XFile getFile(/* config? data? */) async {
  // Do the right thing for native here, maybe
  // we write the data to a File in a temp dir
  // and then:
  return XFileFactory.fromFile(tmpFile);
}
import 'package:cross_file/web/factory.dart';

XFile getFile(/* config? data? */) async {
  // Do the right thing for the web here,
  // maybe we grab bytes, put them into a
  // Blob and do:
  return XFileFactory.fromBlob(dataBlob);
}

Maybe the confusing part is calling the XFileFactory the same on both native and web? Should we just have NativeXFileFactory and WebXFileFactory to make the separation clearer, and signal that we never expect a shared interface across both classes?


  • It provides all the methods that do not include platform-specific types (so only fromPath/fromBytes/fromObjectUrl)

This is why I think having a base class for the XFileFactory is more trouble than what it's worth. Unless I'm missing something, for this approach to work, the XFileFactory needs to be an actual instance that must be replaced at run-time (similar to a federated plugin?).

Since we can't expose web-only types through this common API, users would still need to conditionally import the native/web types to cast the instance to the one that has the platform specific methods. And if you're conditionally importing, just import the right XFileFactory static class? 😛

Footnotes

  1. This is similar to the Client interface from package:http that has multiple platform-specific implementations coming from different packages

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we could just do this automatically from cross_file.dart with conditional imports. The consuming code wouldn't be cross-platform because the factories are specific, but it wouldn't need to do an extra import.

Copy link
Member Author

@ditman ditman Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with conditional exports/imports is that the analyzer freaks out when the APIs are different, which in this case they are (see this comment). Or do you have something else in mind?

`package:cross_file/web/factory.dart` (for web development), and use the factory
constructor more appropriate for the data that you need to handle.

The library currently supports factories for the
following source types:

|| **native** | **web** |
|-|------------|---------|
| `UInt8List`| `fromBytes` | `fromBytes` |
| `dart:io` [`File`][dart_file] | `fromFile` | ❌ |
| Filesystem path | `fromPath` | ❌ |
| Web [`File`][mdn_file] | ❌ | `fromFile` |
| Web [`Blob`][mdn_blob] | ❌ | `fromBlob` |
| `objectURL` | ❌ | `fromObjectUrl` |

[dart_file]: https://api.dart.dev/stable/3.5.2/dart-io/File-class.html
[mdn_file]: https://developer.mozilla.org/en-US/docs/Web/API/File
[mdn_blob]: https://developer.mozilla.org/en-US/docs/Web/API/Blob


### Example

<?code-excerpt "example/lib/readme_excerpts.dart (Instantiate)"?>
```dart
final XFile file = XFile('assets/hello.txt');
final XFile file = XFileFactory.fromPath('assets/hello.txt');

print('File information:');
print('- Path: ${file.path}');
Expand All @@ -27,16 +50,12 @@ You will find links to the API docs on the [pub page](https://pub.dev/packages/c

## Web Limitations

`XFile` on the web platform is backed by [Blob](https://api.dart.dev/be/180361/dart-html/Blob-class.html)
`XFile` on the web platform is backed by `Blob`
objects and their URLs.

It seems that Safari hangs when reading Blobs larger than 4GB (your app will stop
without returning any data, or throwing an exception).

This package will attempt to throw an `Exception` before a large file is accessed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be sure, was this comment removed, because we now refer to the browser's behavior? Or was this fixed in recent versions of Safari?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't explicitly throw the "4 Gigabyte" exception; the browser may fail in unexpected ways (depending on what we end up doing with the bytes of the file)

from Safari (if its size is known beforehand), so that case can be handled
programmatically.

### Browser compatibility

[![Data on Global support for Blob constructing](https://caniuse.bitsofco.de/image/blobbuilder.png)](https://caniuse.com/blobbuilder)
Expand Down
3 changes: 2 additions & 1 deletion packages/cross_file/example/lib/readme_excerpts.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@
// ignore_for_file: avoid_print

import 'package:cross_file/cross_file.dart';
import 'package:cross_file/native/factory.dart';

/// Demonstrate instantiating an XFile for the README.
Future<XFile> instantiateXFile() async {
// #docregion Instantiate
final XFile file = XFile('assets/hello.txt');
final XFile file = XFileFactory.fromPath('assets/hello.txt');

print('File information:');
print('- Path: ${file.path}');
Expand Down
3 changes: 3 additions & 0 deletions packages/cross_file/lib/cross_file.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,7 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

/// Exposes the [XFile] interface (to use XFiles on any platform).
library;

export 'src/x_file.dart';
57 changes: 57 additions & 0 deletions packages/cross_file/lib/native/factory.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:io';
import 'dart:typed_data';

import '../cross_file.dart';
import '../src/implementations/io_bytes_x_file.dart';
import '../src/implementations/io_x_file.dart';

/// Creates [XFile] objects from different native sources.
abstract interface class XFileFactory {
/// Creates an [XFile] from a `dart:io` [File].
///
/// Allows passing the [mimeType] attribute of the file, if needed.
static XFile fromFile(
File file, {
String? mimeType,
}) {
return IOXFile(
file,
mimeType: mimeType,
);
}

/// Creates an [XFile] from a `dart:io` [File]'s [path].
///
/// Allows passing the [mimeType] attribute of the file, if needed.
static XFile fromPath(
String path, {
String? mimeType,
}) {
return IOXFile.fromPath(
path,
mimeType: mimeType,
);
}

/// Creates an [XFile] from an array of [bytes].
///
/// Allows passing the [mimeType], [displayName] and [lastModified] attributes
/// of the file, if needed.
static XFile fromBytes(
Uint8List bytes, {
String? mimeType,
String? displayName,
DateTime? lastModified,
}) {
return BytesXFile(
bytes,
mimeType: mimeType,
displayName: displayName,
lastModified: lastModified,
);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:convert';
import 'dart:typed_data';

import '../x_file.dart';

/// The shared behavior for all byte-backed [XFile] implementations.
///
/// This is an almost complete XFile implementation, except for the `saveTo`
/// method, which is platform-dependent.
abstract class BaseBytesXFile implements XFile {
/// Construct an [XFile] from its data [bytes].
BaseBytesXFile(
this.bytes, {
String? mimeType,
String? displayName,
DateTime? lastModified,
}) : _mimeType = mimeType,
_displayName = displayName,
_lastModified = lastModified;

/// The binary contents of this [XFile].
final Uint8List bytes;
final String? _mimeType;
final String? _displayName;
final DateTime? _lastModified;

@override
Future<DateTime> lastModified() async {
return _lastModified ?? DateTime.now();
}

@override
String? get mimeType => _mimeType;

@override
String get path => '';

@override
String get name => _displayName ?? '';

@override
Future<int> length() async {
return bytes.length;
}

@override
Future<String> readAsString({Encoding encoding = utf8}) async {
return encoding.decode(bytes);
}

@override
Future<Uint8List> readAsBytes() async {
return bytes;
}

@override
Stream<Uint8List> openRead([int? start, int? end]) async* {
yield bytes.sublist(start ?? 0, end ?? bytes.length);
}
}
150 changes: 150 additions & 0 deletions packages/cross_file/lib/src/implementations/blob_x_file.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:async';
import 'dart:convert';
import 'dart:typed_data';

import 'package:web/web.dart';

import '../web_helpers/web_helpers.dart';
import '../x_file.dart';

/// The metadata and shared behavior of a [blob]-backed [XFile].
abstract class BaseBlobXFile implements XFile {
/// Store the metadata of the [blob]-backed [XFile].
BaseBlobXFile({
String? mimeType,
String? displayName,
DateTime? lastModified,
}) : _mimeType = mimeType,
_lastModified = lastModified ?? DateTime.fromMillisecondsSinceEpoch(0),
_name = displayName;

final String? _mimeType;
final String? _name;
final DateTime _lastModified;

/// Asynchronously retrieve the [Blob] backing this [XFile].
///
/// Subclasses must implement this getter. All the blob accesses on this file
/// must be implemented off of it.
Future<Blob> get blob;

@override
String? get mimeType => _mimeType;

@override
String get name => _name ?? '';

@override
String get path => '';

@override
Future<DateTime> lastModified() async => _lastModified;

@override
Future<Uint8List> readAsBytes() async {
return blobToBytes(await blob);
}

@override
Future<int> length() async => (await blob).size;

@override
Future<String> readAsString({Encoding encoding = utf8}) async {
return encoding.decode(await readAsBytes());
}

// TODO(dit): https://github.com/flutter/flutter/issues/91867 Implement openRead properly.
@override
Stream<Uint8List> openRead([int? start, int? end]) async* {
final Blob browserBlob = await blob;
final Blob slice = browserBlob.slice(
start ?? 0,
end ?? browserBlob.size,
browserBlob.type,
);
yield await blobToBytes(slice);
}
}

/// Construct an [XFile] backed by [blob].
///
/// `name` needs to be passed from the outside, since it's only available
/// while handling [html.File]s (when the ObjectUrl is created).
class BlobXFile extends BaseBlobXFile {
/// Construct an [XFile] backed by [blob].
///
/// `name` needs to be passed from the outside, since it's only available
/// while handling [html.File]s (when the ObjectUrl is created).
BlobXFile(
Blob blob, {
super.mimeType,
super.displayName,
super.lastModified,
}) : _blob = blob;

/// Creates a [XFile] from a web [File].
factory BlobXFile.fromFile(File file) {
return BlobXFile(
file,
mimeType: file.type,
displayName: file.name,
lastModified: DateTime.fromMillisecondsSinceEpoch(file.lastModified),
);
}

Blob _blob;

// The Blob backing the file.
@override
Future<Blob> get blob async => _blob;

/// Attempts to save the data of this [XFile], using the passed-in `blob`.
///
/// The [path] variable is ignored.
@override
Future<void> saveTo(String path) async {
// Save a Blob to file...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems a bit redundant.

await downloadBlob(_blob, name.isEmpty ? null : name);
}
}

/// Constructs an [XFile] from the [objectUrl] of a [Blob].
///
/// Important: the Object URL of a blob must have been created by the same JS
/// thread that is attempting to retrieve it. Otherwise, the blob will not be
/// accessible.
///
/// See: https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static
class ObjectUrlBlobXFile extends BaseBlobXFile {
/// Constructs an [XFile] from the [objectUrl] of a [Blob].
ObjectUrlBlobXFile(
String objectUrl, {
super.mimeType,
super.displayName,
super.lastModified,
}) : _objectUrl = objectUrl;

final String _objectUrl;

Blob? _cachedBlob;

// The Blob backing the file.
@override
Future<Blob> get blob async => _cachedBlob ??= await fetchBlob(_objectUrl);

/// Returns the [objectUrl] used to create this instance.
@override
String get path => _objectUrl;

/// Attempts to save the data of this [XFile], using the passed-in `objectUrl`.
///
/// The [path] variable is ignored.
@override
Future<void> saveTo(String path) async {
downloadObjectUrl(_objectUrl, name.isEmpty ? null : name);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:io';
import 'dart:typed_data';

import 'base_bytes_x_file.dart';

/// A CrossFile backed by an [Uint8List].
class BytesXFile extends BaseBytesXFile {
/// Construct an [XFile] from its data [bytes].
BytesXFile(
super.bytes, {
super.mimeType,
super.displayName,
super.lastModified,
});

@override
Future<void> saveTo(String path) async {
final File fileToSave = File(path);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is lifted from the old implementation but... should the File be created from path + this.name?

await fileToSave.writeAsBytes(bytes);
}
}
Loading