Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/upload chunk size #805

Merged
merged 19 commits into from
Feb 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
c1babe8
feat(upload-core): upgrade to tus-js-client v4
jordan-a-young Dec 2, 2024
6715f84
chore(@availity/upload-core): release version 7.0.0-alpha.0 [skip ci]
jordan-a-young Dec 2, 2024
e8f2b6c
feat(upload-core): convert to ts
jordan-a-young Dec 9, 2024
d342e30
chore(@availity/upload-core): release version 7.0.0-alpha.1 [skip ci]
jordan-a-young Dec 9, 2024
00b4aad
feat(upload-core): add logging
jordan-a-young Dec 12, 2024
7f3e393
chore(@availity/upload-core): release version 7.0.0-alpha.2 [skip ci]
jordan-a-young Dec 12, 2024
faafb19
feat(upload-core): add logging
jordan-a-young Dec 12, 2024
e3f7ec2
chore(@availity/upload-core): release version 7.0.0-alpha.3 [skip ci]
jordan-a-young Dec 12, 2024
5749a93
feat(upload-core): remove logging
jordan-a-young Dec 12, 2024
2ad173e
chore(@availity/upload-core): release version 7.0.0-alpha.4 [skip ci]
jordan-a-young Dec 12, 2024
f6f2b26
feat(upload-core): do not auto generate id
jordan-a-young Dec 13, 2024
c5283ac
chore(@availity/upload-core): release version 7.0.0-alpha.5 [skip ci]
jordan-a-young Dec 13, 2024
5d24cba
fix: add credentials to fetch
jordan-a-young Jan 7, 2025
8d5b7ae
chore(@availity/upload-core): release version 7.0.0-alpha.6 [skip ci]
jordan-a-young Jan 7, 2025
5da7fe6
docs: add upload docs
jordan-a-young Feb 7, 2025
6a19dfd
feat(upload-core): upgrade to tus-js-client v4
jordan-a-young Feb 7, 2025
be3d2d8
Merge branch 'master' into fix/upload-chunk-size
jordan-a-young Feb 7, 2025
e6bb673
build: add back husky
jordan-a-young Feb 10, 2025
c5699c6
Merge branch 'fix/upload-chunk-size' of https://github.com/Availity/s…
jordan-a-young Feb 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .adiorc.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
module.exports = {
packages: ['packages/*'],
ignore: {
src: ['form-data', 'nock', 'xhr-mock', 'ts-jest', 'http', 'https', 'url', 'stream'],
src: ['form-data', 'nock', 'xhr-mock', 'ts-jest'],
devDependencies: ['typescript', 'tsup', 'axios', '@types/tus-js-client'],
peerDependencies: ['axios'],
},
Expand Down
2 changes: 1 addition & 1 deletion .husky/commit-msg
Original file line number Diff line number Diff line change
@@ -1 +1 @@
yarn commitlint -e $1
yarn commitlint -e $1
2 changes: 1 addition & 1 deletion .husky/pre-commit
Original file line number Diff line number Diff line change
@@ -1 +1 @@
yarn lint:affected
yarn lint:affected
205 changes: 183 additions & 22 deletions docusaurus/docs/api/uploads.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ Wrapper for tus-js-client

[![Version](https://img.shields.io/npm/v/@availity/upload-core.svg?style=for-the-badge)](https://www.npmjs.com/package/@availity/upload-core)

`@availity/upload-core` is a powerful wrapper for the `tus-js-client` library, providing a high-level API for managing file uploads using the tus protocol. This package simplifies the process of initiating, monitoring, and handling file uploads in JavaScript applications. Key features include:

- Easy-to-use Upload class for managing file uploads
- Progress tracking and real-time status updates
- Robust error handling and retry mechanisms
- File validation (size, type, name)
- Support for encrypted file uploads
- Customizable upload options and callbacks

## Install

### NPM
Expand All @@ -22,22 +31,62 @@ yarn add @availity/upload-core

## Usage

### Required params
### Basic Usage

```js
import Upload from '@availity/upload-core';

const file = new File(['file content'], 'example.pdf', {
type: 'application/pdf',
});

const upload = new Upload(file, {
bucketId: 'YOUR_BUCKET_ID',
customerId: 'YOUR_CUSTOMER_ID',
clientId: 'YOUR_CLIENT_ID',
});

// Add event listeners
upload.onProgress.push(() => {
console.log(`Upload progress: ${upload.percentage}%`);
});

upload.onSuccess.push(() => {
console.log('Upload completed successfully!');
console.log('File references:', upload.references);
console.log('S3 references:', upload.s3References);
});

upload.onError.push((error) => {
console.error('Upload failed:', error.message);
});

// Generate unique ID and start upload
await upload.generateId();
upload.start();
```

- bucketId
- customerId
- clientId
#### Required params

### Optional params
| Parameter | Type | Description |
| ---------- | ------ | ----------------------------------- |
| bucketId | string | The target bucket |
| customerId | string | The customer id of the organization |
| clientId | string | The client id for api calls |

- fileTypes: string array of file extensions to allow (error thrown if file.name does not contain one of the types)
- maxSize: maximum size allowed per file
- metadata: object mapping metadata keys and values to add to the TUS upload
- allowedFileNameCharacters: restrict the file name characters to a regex set
- pollingTime: custom av scan polling time (default 5000ms)
- maxAvScanRetries: amount of times to poll for av scan result before error is returned (default 10)
#### Optional params

### Upload object variables that can be set before call to start().
| Parameter | Type | Default | Description |
| ------------------------- | ------- | ------- | ---------------------------------------------- |
| fileTypes | string | | Allowed file extensions (e.g., '.pdf', '.png') |
| maxSize | number | | Maximum file size in bytes |
| metadata | object | {} | Additional metadata for the upload |
| allowedFileNameCharacters | string | | Regex pattern for allowed filename characters |
| pollingTime | number | 5000 | Interval for virus scan polling (ms) |
| maxAvScanRetries | number | 10 | Maximum retries for virus scan check |
| stripFileNamePathSegments | boolean | true | Remove path segments from filename |

#### Upload event handlers

**Each one of these should be an array of functions**

Expand All @@ -46,19 +95,131 @@ yarn add @availity/upload-core
- **onSuccess**: each function is called once if there is a success.
- **onError**: each function is called once if there is an error.

```js
import Upload from '@availity/upload-core';
### Advanced Usage Examples

#### File Type Restrictions

```js
const upload = new Upload(file, {
bucketId: 'a',
customerId: 'b',
clientId: 'c',
fileTypes: ['.png', '.pdf'],
maxSize: 3e8,
metadata: { key: 'value' },
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
fileTypes: ['.pdf', '.doc', '.docx'],
maxSize: 10 * 1024 * 1024, // 10MB
allowedFileNameCharacters: '_a-zA-Z0-9 ', // alphanumeric, spaces, underscore
pollingTime: 1000,
});
```

upload.start();
#### Custom Metadata

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
metadata: {
'document-type': 'medical-record',
'patient-id': '12345',
department: 'cardiology',
},
});
```

#### Handling Encrypted Files

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
});

upload.onError.push((error) => {
if (upload.status === 'encrypted') {
// Prompt user for password
const password = promptUserForPassword();
upload.sendPassword(password);
} else {
console.error('Upload failed:', error.message);
}
});
```

#### Progress Tracking

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
});

upload.onProgress.push(() => {
const uploadedSize = upload.bytesSent;
const totalSize = upload.bytesTotal;
const scannedBytes = upload.bytesScanned;
const percentage = upload.percentage;

console.log(`Uploaded: ${uploadedSize}/${totalSize} bytes`);
console.log(`Scanned: ${scannedBytes} bytes`);
console.log(`Overall progress: ${percentage}%`);
});
```

#### Pre-start Validation

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
});

// Add custom validation before upload starts
upload.onPreStart.push((upload) => {
if (upload.file.size === 0) {
console.error('Cannot upload empty file');
return false;
}
return true;
});
```

### Upload Status Values

The `upload.status` can be one of the following:

- `pending`: Initial state
- `accepted`: Upload completed successfully
- `rejected`: Upload failed
- `encrypted`: File is encrypted and requires a password
- `decrypting`: File is being decrypted

### Error Handling

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
});

upload.onError.push((error) => {
console.error('Status:', upload.status);
console.error('Error Message:', upload.errorMessage);
console.error('Error Details:', error);
});
```

### Aborting an Upload

```js
const upload = new Upload(file, {
bucketId: 'bucket123',
customerId: 'customer123',
clientId: 'client123',
});

// Later in your code
upload.abort();
```
4 changes: 2 additions & 2 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ module.exports = {
},
jsdom: true,
},
coverageReporters: ['json'],
coverageReporters: ['json', 'html'],
moduleNameMapper: {
...pathsToModuleNameMapper(compilerOptions.paths, { prefix: '<rootDir>/../../' }),
},
setupFiles: ['../../jest.polyfills.js']
setupFiles: ['../../jest.polyfills.js'],
};
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
"jest-environment-jsdom": "^27.5.1",
"jest-environment-jsdom-global": "^3.1.2",
"lint-staged": "^15.2.10",
"nock": "^13.5.4",
"nock": "^13.5.6",
"nx": "19.5.3",
"plop": "^4.0.1",
"prettier": "^3.3.3",
Expand Down
9 changes: 5 additions & 4 deletions packages/api-axios/mocks/handlers.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
// src/mocks/handlers.js
import { http, HttpResponse } from 'msw';

const handlers = [
// Handles a POST /login request
http.put('/api/sdk/platform/v1/regions/:region', ({params}) => {
http.put('/api/sdk/platform/v1/regions/:region', ({ params }) => {

Check warning on line 5 in packages/api-axios/mocks/handlers.js

View check run for this annotation

Codecov / codecov/patch

packages/api-axios/mocks/handlers.js#L5

Added line #L5 was not covered by tests
const { region } = params;

return HttpResponse.json({
return HttpResponse.json(

Check warning on line 8 in packages/api-axios/mocks/handlers.js

View check run for this annotation

Codecov / codecov/patch

packages/api-axios/mocks/handlers.js#L8

Added line #L8 was not covered by tests
{
links: {
self: {
href: `https://localhost:3000/api/sdk/platform/v1/regions/${region}`,
Expand All @@ -15,7 +15,8 @@
id: region,
value: region,
currentlySelected: true,
}, { status: 200 }
},
{ status: 200 }
);
}),

Expand Down
4 changes: 2 additions & 2 deletions packages/api-axios/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@
"dependencies": {
"@availity/resolve-url": "workspace:*",
"lodash": "^4.17.21",
"qs": "^6.13.0"
"qs": "^6.13.1"
},
"devDependencies": {
"axios": "^1.7.7",
"msw": "^2.3.4",
"msw": "2.3.5",
"tsup": "^7.2.0",
"typescript": "^5.5.4"
},
Expand Down
2 changes: 1 addition & 1 deletion packages/api-axios/src/resources/tests/regions.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ describe('AvRegionsApi', () => {
const region = 'FL';

api.setPageBust = jest.fn();
api.http = jest.fn().mockResolvedValue({data: {id: region}});
api.http = jest.fn().mockResolvedValue({ data: { id: region } });

const resp = await api.put(region);

Expand Down
2 changes: 1 addition & 1 deletion packages/api-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"dependencies": {
"@availity/env-var": "workspace:*",
"@availity/resolve-url": "workspace:*",
"qs": "^6.13.0"
"qs": "^6.13.1"
},
"devDependencies": {
"tsup": "^7.2.0",
Expand Down
2 changes: 1 addition & 1 deletion packages/upload-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

This file was generated using [@jscutlery/semver](https://github.com/jscutlery/semver).


## [6.1.4](https://github.com/Availity/sdk-js/compare/@availity/[email protected]...@availity/[email protected]) (2024-11-21)


Expand All @@ -10,7 +11,6 @@ This file was generated using [@jscutlery/semver](https://github.com/jscutlery/s
* **upload-core:** default chunk size to 6MB ([1a3e718](https://github.com/Availity/sdk-js/commit/1a3e718ac6914b1f18a7d4547328f48d9fc9129c))



## [6.1.3](https://github.com/Availity/sdk-js/compare/@availity/[email protected]...@availity/[email protected]) (2024-11-12)


Expand Down
1 change: 1 addition & 0 deletions packages/upload-core/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ module.exports = {
...global,
displayName: 'upload-core',
coverageDirectory: '../../coverage/upload-core',
coveragePathIgnorePatterns: ['/mocks/*'],
};
Loading