Skip to content

Commit 0c45788

Browse files
Rasmus Oscar Welanderglpatcern
Rasmus Oscar Welander
authored andcommitted
Fixes from comments,license, CI and README.md
1 parent dd95593 commit 0c45788

File tree

8 files changed

+541
-695
lines changed

8 files changed

+541
-695
lines changed

.github/workflows/ci-tests.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,5 @@ jobs:
3535
3636
- name: Test with pytest
3737
run: |
38-
pytest
38+
if [ -f tests/requirements.txt ]; then pip install -r tests/requirements.txt; fi
39+
pytest --cov-report term --cov=src tests/

.github/workflows/publish.yml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Publish To PyPI
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v*'
7+
8+
jobs:
9+
build-publish:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- uses: actions/checkout@v2
14+
15+
- name: Set environnment package version from tag
16+
run: echo "PACKAGE_VERSION=${GITHUB_REF#refs/*/v}" >> $GITHUB_ENV # extract "1.2.3" from refs/tags/v1.2.3
17+
18+
- name: Set up Python 3.12
19+
uses: actions/setup-python@v3
20+
with:
21+
python-version: "3.12"
22+
23+
- name: Install dependencies
24+
run: pip install build twine
25+
26+
- name: Build wheel
27+
run: python -m build
28+
29+
- name: Publish distribution to PyPI
30+
env:
31+
TWINE_USERNAME: __token__
32+
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
33+
run: twine upload --repository pypi dist/*

LICENSE

Lines changed: 174 additions & 673 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
# CS3Client
2+
3+
`CS3Client` is a Python client for interacting with the CS3 (Cloud Storage Services for Science) API. It allows users to seamlessly communicate with cloud storage services that support CS3 protocols, enabling file management, data transfer, and other cloud-based operations.
4+
5+
## Table of Contents
6+
7+
- [Features](#features)
8+
- [Installation](#installation)
9+
- [Usage](#usage)
10+
- [Configuration](#configuration)
11+
- [Examples](#examples)
12+
- [Documentation](#documentation)
13+
- [License](#license)
14+
15+
16+
## Features
17+
18+
- Simple and easy-to-use API client for CS3 services.
19+
- Support for common file operations (read, write, delete, rename, ...).
20+
- Support for common lock operations (set lock, get lock, unlock, ...).
21+
- Support for common share operations (create share, update share, delete share, ...).
22+
- Support for common user operations (get user, find users, get user groups, ...).
23+
- Support for restoring files through checkpoints (restore file version, list checkpoints).
24+
- Support for applications (open in app, list app providers).
25+
- Authentication and authorization handling.
26+
- Cross-platform compatibility.
27+
- Detailed error handling and logging.
28+
29+
## Installation
30+
31+
To install `cs3client`, you need to have Python 3.7+ installed. You can install the package via `pip`:
32+
33+
```bash
34+
pip install cs3client
35+
```
36+
Alternatively, you can clone this repository and install manually:
37+
```bash
38+
git clone [email protected]:cs3org/cs3-python-client.git
39+
cd cs3-python-client
40+
pip install -e .
41+
export PYTHONPATH="path/to/cs3-python-client/:$PYTHONPATH"
42+
```
43+
44+
45+
## Configuration
46+
47+
`CS3Client` can be configured by passing specific parameters when initializing the client through a ConfigParser instance.
48+
49+
### Parameters:
50+
51+
#### Required
52+
- `host`
53+
54+
#### Optional (parameter - default)
55+
- `chunk_size` - 4194384
56+
- `grpc_timeout` - 10
57+
- `http_timeout` - 10
58+
- `tus_enabled` - False
59+
- `ssl_enabled` - False
60+
- `ssl_client_cert` - None
61+
- `ssl_client_key` - None
62+
- `ssl_ca_cert` - None
63+
- `auth_client_id` - None
64+
- `auth_login_type` - "basic"
65+
- `lock_by_setting_attr` - False
66+
- `lock_not_impl` - False
67+
- `lock_expiration` - 1800
68+
69+
#### Example configuration
70+
```yaml
71+
[cs3client]
72+
73+
# Required
74+
host = localhost:19000
75+
# Optional, defaults to 4194304
76+
chunk_size = 4194304
77+
# Optional, defaults to 10
78+
grpc_timeout = 10
79+
# Optional, defaults to 10
80+
http_timeout = 10
81+
82+
# Optional, defaults to True
83+
tus_enabled = False
84+
85+
# Optional, defaults to True
86+
ssl_enabled = False
87+
# Optional, defaults to True
88+
ssl_verify = False
89+
# Optional, defaults to an empty string
90+
ssl_client_cert = test_client_cert
91+
# Optional, defaults to an empty string
92+
ssl_client_key = test_client_key
93+
# Optional, defaults to an empty string
94+
ssl_ca_cert = test_ca_cert
95+
96+
# Optinal, defaults to an empty string
97+
auth_client_id = einstein
98+
# Optional, defaults to basic
99+
auth_login_type = basic
100+
101+
# Optional, defaults to False
102+
lock_by_setting_attr = False
103+
# Optional, defaults to False
104+
lock_not_impl = False
105+
# Optional, defaults to 1800
106+
lock_expiration = 1800
107+
108+
109+
```
110+
111+
## Usage
112+
113+
To use `cs3client`, you first need to import and configure it. Here's a simple example of how to set up and start using the client. For configuration see [Configuration](#configuration). For more in depth examples see `cs3-python-client/examples/`.
114+
115+
### Initilization
116+
```python
117+
import logging
118+
import configparser
119+
from cs3client import CS3Client
120+
from cs3resource import Resource
121+
122+
config = configparser.ConfigParser()
123+
with open("default.conf") as fdef:
124+
config.read_file(fdef)
125+
126+
log = logging.getLogger(__name__)
127+
128+
client = CS3Client(config, "cs3client", log)
129+
# client.auth.set_token("<your_token_here>")
130+
# OR
131+
client.auth.set_client_secret("<your_client_secret_here>")
132+
```
133+
134+
### File Example
135+
```python
136+
# mkdir
137+
directory_resource = Resource.from_file_ref_and_endpoint(f"/eos/user/r/rwelande/test_directory")
138+
res = client.file.make_dir(directory_resource)
139+
140+
# touchfile
141+
touch_resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande/touch_file.txt")
142+
res = client.file.touch_file(touch_resource)
143+
144+
# setxattr
145+
resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande/text_file.txt")
146+
res = client.file.set_xattr(resource, "iop.wopi.lastwritetime", str(1720696124))
147+
148+
# rmxattr
149+
res = client.file.remove_xattr(resource, "iop.wopi.lastwritetime")
150+
151+
# stat
152+
res = client.file.stat(resource)
153+
154+
# removefile
155+
res = client.file.remove_file(touch_resource)
156+
157+
# rename
158+
rename_resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande/rename_file.txt")
159+
res = client.file.rename_file(resource, rename_resource)
160+
161+
# writefile
162+
content = b"Hello World"
163+
size = len(content)
164+
res = client.file.write_file(rename_resource, content, size)
165+
166+
# listdir
167+
list_directory_resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande")
168+
res = client.file.list_dir(list_directory_resource)
169+
170+
171+
# readfile
172+
file_res = client.file.read_file(rename_resource)
173+
```
174+
175+
### Share Example
176+
```python
177+
# Create share #
178+
resource = Resource.from_file_ref_and_endpoint("/eos/user/r/<some_username>/text.txt")
179+
resource_info = client.file.stat(resource)
180+
user = client.user.get_user_by_claim("username", "<some_username>")
181+
res = client.share.create_share(resource_info, user.id.opaque_id, user.id.idp, "EDITOR", "USER")
182+
183+
# List existing shares #
184+
filter_list = []
185+
filter = client.share.create_share_filter(resource_id=resource_info.id, filter_type="TYPE_RESOURCE_ID")
186+
filter_list.append(filter)
187+
filter = client.share.create_share_filter(share_state="SHARE_STATE_PENDING", filter_type="TYPE_STATE")
188+
filter_list.append(filter)
189+
res, _ = client.share.list_existing_shares()
190+
191+
# Get share #
192+
share_id = "58"
193+
res = client.share.get_share(opaque_id=share_id)
194+
195+
# update share #
196+
res = client.share.update_share(opaque_id=share_id, role="VIEWER")
197+
198+
# remove share #
199+
res = client.share.remove_share(opaque_id=share_id)
200+
201+
# List existing received shares #
202+
filter_list = []
203+
filter = client.share.create_share_filter(share_state="SHARE_STATE_ACCEPTED", filter_type="TYPE_STATE")
204+
filter_list.append(filter)
205+
res, _ = client.share.list_received_existing_shares()
206+
207+
# get received share #
208+
received_share = client.share.get_received_share(opaque_id=share_id)
209+
210+
# update recieved share #
211+
res = client.share.update_received_share(received_share=received_share, state="SHARE_STATE_ACCEPTED")
212+
213+
# create public share #
214+
res = client.share.create_public_share(resource_info, role="VIEWER")
215+
216+
# list existing public shares #
217+
filter_list = []
218+
filter = client.share.create_public_share_filter(resource_id=resource_info.id, filter_type="TYPE_RESOURCE_ID")
219+
filter_list.append(filter)
220+
res, _ = client.share.list_existing_public_shares(filter_list=filter_list)
221+
222+
res = client.share.get_public_share(opaque_id=share_id, sign=True)
223+
# OR token = "<token>"
224+
# res = client.share.get_public_share(token=token, sign=True)
225+
226+
# update public share #
227+
res = client.share.update_public_share(type="TYPE_PASSWORD", token=token, role="VIEWER", password="hello")
228+
229+
# remove public share #
230+
res = client.share.remove_public_share(token=token)
231+
232+
```
233+
234+
### User Example
235+
```python
236+
# find_user
237+
res = client.user.find_users("rwel")
238+
239+
# get_user
240+
res = client.user.get_user("https://auth.cern.ch/auth/realms/cern", "asdoiqwe")
241+
242+
# get_user_groups
243+
res = client.user.get_user_groups("https://auth.cern.ch/auth/realms/cern", "rwelande")
244+
245+
# get_user_by_claim (mail)
246+
res = client.user.get_user_by_claim("mail", "[email protected]")
247+
248+
# get_user_by_claim (username)
249+
res = client.user.get_user_by_claim("username", "rwelande")
250+
251+
```
252+
253+
### App Example
254+
```python
255+
# list_app_providers
256+
res = client.app.list_app_providers()
257+
258+
# open_in_app
259+
resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande/collabora.odt")
260+
res = client.app.open_in_app(resource)
261+
```
262+
263+
### Checkpoint Example
264+
```python
265+
# list file versions
266+
resource = Resource.from_file_ref_and_endpoint("/eos/user/r/rwelande/test.md")
267+
res = client.checkpoint.list_file_versions(resource)
268+
269+
# restore file version
270+
res = client.checkpoint.restore_file_version(resource, "1722936250.0569fa2f")
271+
```
272+
273+
## Documentation
274+
The documentation can be generated using sphinx
275+
276+
```bash
277+
pip install sphinx
278+
cd docs
279+
make html
280+
```
281+
282+
## Unit tests
283+
284+
```bash
285+
pytest --cov-report term --cov=serc tests/
286+
```
287+
288+
## License
289+
290+
This project is licensed under the Apache 2.0 License. See the LICENSE file for more details.

src/cs3resource.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,10 +195,12 @@ def get_file_ref_str(self):
195195
Generates a string from the file ref, '<type>="fileref">'
196196
197197
:return: str '<type>="fileref">'
198+
:raises: ValueError (Invalid Resource)
198199
"""
199200
if self._abs_path:
200201
return f'absolute_path="{self._abs_path}"'
201202
elif self._rel_path:
202203
return f'relative_path="{self._parent_id}/{self._rel_path}"'
203204
elif self._opaque_id:
204205
return f'opaque_id="{self._opaque_id}"'
206+
raise ValueError("Invalid Resource")

src/exceptions/exceptions.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,12 @@ class AlreadyExistsException(IOError):
6262

6363
def __init__(self, message: str = ""):
6464
super().__init__(message)
65+
66+
67+
class UnimplementedException(Exception):
68+
"""
69+
Standard error thrown when attempting to use a feature that is not implemented
70+
"""
71+
72+
def __init__(self, message: str = ""):
73+
super().__init__(message)

src/share.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -493,7 +493,7 @@ def remove_public_share(self, token: str = None, opaque_id: str = None) -> None:
493493

494494
@classmethod
495495
def _create_public_share_update(
496-
self,
496+
cls,
497497
type: str,
498498
role: str,
499499
password: str = None,

0 commit comments

Comments
 (0)