Skip to content

Commit e8e3881

Browse files
author
vyzyv
committed
Initial release
0 parents  commit e8e3881

File tree

148 files changed

+39544
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

148 files changed

+39544
-0
lines changed

‎.dockerignore

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
*
2+
3+
!README.md
4+
!LICENSE
5+
!setup.py
6+
!environments/requirements.txt
7+
!torchstar-docker
8+
!METADATA

‎.github/workflows/deploy.yml

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
name: deploy
3+
4+
on:
5+
release:
6+
types: [created]
7+
schedule:
8+
- cron: '0 7 * * *'
9+
10+
jobs:
11+
docker-gpu:
12+
name: "Base image nvidia/cuda:${{ matrix.tag }}"
13+
runs-on: ubuntu-latest
14+
strategy:
15+
matrix:
16+
tag:
17+
- "9.2-runtime-ubuntu18.04"
18+
- "9.2-cudnn7-runtime-ubuntu18.04"
19+
- "10.0-runtime-ubuntu18.04"
20+
- "10.0-cudnn7-runtime-ubuntu18.04"
21+
- "10.1-runtime-ubuntu18.04"
22+
- "10.1-cudnn7-runtime-ubuntu18.04"
23+
steps:
24+
- uses: actions/checkout@master
25+
- name: Clone repository
26+
run: git clone https://github.com/szymonmaszke/torchstar-docker
27+
- name: Login
28+
run: >
29+
docker login
30+
-u ${{ secrets.DOCKER_USERNAME }}
31+
-p ${{ secrets.DOCKER_PASSWORD }}
32+
33+
- name: Build and push the image
34+
run: >
35+
./torchstar-docker/src/entrypoint
36+
nvidia/cuda ${{ matrix.tag }}
37+
${GITHUB_REPOSITORY}
38+
${{ github.event_name }}
39+
40+
docker:
41+
name: "Base image ubuntu:18.04 (CPU deployment)"
42+
runs-on: ubuntu-latest
43+
steps:
44+
- uses: actions/checkout@master
45+
- name: Clone repository
46+
run: git clone https://github.com/szymonmaszke/torchstar-docker
47+
- name: Login
48+
run: >
49+
docker login
50+
-u ${{ secrets.DOCKER_USERNAME }}
51+
-p ${{ secrets.DOCKER_PASSWORD }}
52+
- name: Build and push the image
53+
run: >
54+
./torchstar-docker/src/entrypoint
55+
ubuntu
56+
"18.04"
57+
${GITHUB_REPOSITORY}
58+
${{ github.event_name }}
59+
python:
60+
name: Create and publish packages to PyPI
61+
runs-on: ubuntu-latest
62+
steps:
63+
- uses: actions/checkout@master
64+
- name: Change metadata if nightly
65+
if: github.event_name != 'release'
66+
run: ./nightly
67+
- uses: actions/setup-python@v1
68+
with:
69+
python-version: "3.7"
70+
- name: Install dependencies
71+
run: |
72+
python -m pip install --upgrade pip
73+
pip install setuptools wheel
74+
- name: Build package
75+
run: python setup.py sdist bdist_wheel
76+
- name: Publish package to PyPI
77+
uses: pypa/gh-action-pypi-publish@master
78+
with:
79+
password: ${{ secrets.PYPI_PASSWORD }}

‎.github/workflows/tests.yml

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
name: test
3+
4+
on: [push, pull_request]
5+
6+
jobs:
7+
test:
8+
name: Python ${{ matrix.python-version }}, OS ${{ matrix.os }}
9+
runs-on: ${{ matrix.os }}
10+
strategy:
11+
matrix:
12+
python-version: [3.7]
13+
os:
14+
- ubuntu-latest
15+
- ubuntu-18.04
16+
- ubuntu-16.04
17+
- macOS-latest
18+
- macOS-10.14
19+
20+
steps:
21+
- uses: actions/checkout@v1
22+
- name: Set up Python ${{ matrix.python-version }}
23+
uses: actions/setup-python@v1
24+
with:
25+
python-version: ${{ matrix.python-version }}
26+
- name: Install dependencies
27+
run: |
28+
python -m pip install --upgrade pip
29+
pip install -r environments/requirements.txt
30+
- name: Test
31+
run: |
32+
pip install pytest
33+
pytest

‎.gitignore

+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Ignore vim tags
2+
tags
3+
4+
# Byte-compiled / optimized / DLL files
5+
__pycache__/
6+
*.py[cod]
7+
*$py.class
8+
9+
# C extensions
10+
*.so
11+
12+
# Distribution / packaging
13+
.Python
14+
build/
15+
develop-eggs/
16+
dist/
17+
downloads/
18+
eggs/
19+
.eggs/
20+
lib/
21+
lib64/
22+
parts/
23+
sdist/
24+
var/
25+
wheels/
26+
*.egg-info/
27+
.installed.cfg
28+
*.egg
29+
MANIFEST
30+
31+
# PyInstaller
32+
# Usually these files are written by a python script from a template
33+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
34+
*.manifest
35+
*.spec
36+
37+
# Installer logs
38+
pip-log.txt
39+
pip-delete-this-directory.txt
40+
41+
# Unit test / coverage reports
42+
htmlcov/
43+
.tox/
44+
.coverage
45+
.coverage.*
46+
.cache
47+
nosetests.xml
48+
coverage.xml
49+
*.cover
50+
.hypothesis/
51+
.pytest_cache/
52+
53+
# Translations
54+
*.mo
55+
*.pot
56+
57+
# Django stuff:
58+
*.log
59+
local_settings.py
60+
db.sqlite3
61+
62+
# Flask stuff:
63+
instance/
64+
.webassets-cache
65+
66+
# Scrapy stuff:
67+
.scrapy
68+
69+
# Sphinx documentation
70+
docs/_build/
71+
docs/code/src
72+
docs/.buildinfo
73+
74+
# PyBuilder
75+
target/
76+
77+
# Jupyter Notebook
78+
.ipynb_checkpoints
79+
80+
# pyenv
81+
.python-version
82+
83+
# celery beat schedule file
84+
celerybeat-schedule
85+
86+
# SageMath parsed files
87+
*.sage.py
88+
89+
# Environments
90+
.env
91+
.venv
92+
env/
93+
venv/
94+
ENV/
95+
env.bak/
96+
venv.bak/
97+
98+
# Spyder project settings
99+
.spyderproject
100+
.spyproject
101+
102+
# Rope project settings
103+
.ropeproject
104+
105+
# mkdocs documentation
106+
/site
107+
108+
# mypy
109+
.mypy_cache/

‎LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2019 Szymon Maszke
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎METADATA

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
torchfunc
2+
0.1.0

‎README.md

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
![torchfunc Logo](https://github.com/szymonmaszke/torchfunc/blob/master/assets/banner.png)
2+
3+
--------------------------------------------------------------------------------
4+
5+
| Version | Docs | Tests | Coverage | Style | PyPI | Python | PyTorch | Docker | Roadmap |
6+
|---------------|-------|----------|-------|------|--------|---------|--------|---------|---------|
7+
| [![Version](https://img.shields.io/static/v1?label=&message=0.1.0&color=377EF0&style=for-the-badge)](https://github.com/szymonmaszke/torchfunc/releases) | [![Documentation](https://img.shields.io/static/v1?label=&message=docs&color=EE4C2C&style=for-the-badge)](https://szymonmaszke.github.io/torchfunc/) | ![Tests](https://github.com/szymonmaszke/torchfunc/workflows/test/badge.svg) | | | [![PyPI](https://img.shields.io/static/v1?label=&message=PyPI&color=377EF0&style=for-the-badge)](https://pypi.org/project/torchfunc/) | [![Python](https://img.shields.io/static/v1?label=&message=3.7&color=377EF0&style=for-the-badge&logo=python&logoColor=F8C63D)](https://www.python.org/) | [![PyTorch](https://img.shields.io/static/v1?label=&message=1.2.0&color=EE4C2C&style=for-the-badge)](https://pytorch.org/) | [![Docker](https://img.shields.io/static/v1?label=&message=docker&color=309cef&style=for-the-badge)](https://cloud.docker.com/u/szymonmaszke/repository/docker/szymonmaszke/torchfunc) | [![Roadmap](https://img.shields.io/static/v1?label=&message=roadmap&color=009688&style=for-the-badge)](https://github.com/szymonmaszke/torchfunc/blob/master/ROADMAP.md) |
8+
9+
[**torchfunc**](https://szymonmaszke.github.io/torchfunc/) is library revolving around [PyTorch](https://pytorch.org/) with a goal to help you with:
10+
11+
* Improving and analysing performance of your neural network
12+
* Daily neural network duties (model size, seeding, performance measurements etc.)
13+
* Plotting and visualizing modules
14+
* Record neuron activity and tailor it to your specific task or target
15+
* Get information about your host operating system, CUDA devices and others
16+
17+
# Quick examples
18+
19+
- Seed globaly, Freeze weights, check inference time and model size
20+
21+
```python
22+
# Inb4 MNIST, you can use any module with those functions
23+
model = torch.nn.Linear(784, 10)
24+
frozen = torchfunc.module.freeze(model, bias=False)
25+
26+
with torchfunc.Timer() as timer:
27+
frozen(torch.randn(32, 784)
28+
print(timer.checkpoint()) # Time since the beginning
29+
frozen(torch.randn(128, 784)
30+
print(timer.checkpoint()) # Since last checkpoint
31+
32+
print(f"Overall time {timer}; Model size: {torchfunc.sizeof(frozen)}")
33+
```
34+
35+
- Recorder and sum per-layer activation statistics as data passes through network:
36+
37+
```python
38+
# MNIST classifier
39+
model = torch.nn.Sequential(
40+
torch.nn.Linear(784, 100),
41+
torch.nn.ReLU(),
42+
torch.nn.Linear(100, 50),
43+
torch.nn.ReLU(),
44+
torch.nn.Linear(50, 10),
45+
)
46+
# Recorder which sums layer inputs from consecutive forward calls
47+
recorder = torchfunc.record.ForwardPreRecorder(reduction=lambda x, y: x+y)
48+
# Record inputs going into Linear(100, 50) and Linear(50, 10)
49+
recorder.children(model, indices=(2, 3))
50+
# Train your network normally (or pass data through it)
51+
...
52+
# Save tensors (of shape 100 and 50) in folder, each named 1.pt and 2.pt respectively
53+
recorder.save(pathlib.Path("./analysis"))
54+
```
55+
56+
For performance tips, plotting and other check [**torchfunc documentation**](https://szymonmaszke.github.io/torchfunc/).
57+
58+
# Installation
59+
60+
## [pip](<https://pypi.org/project/torchfunc/>)
61+
62+
### Latest release:
63+
64+
```shell
65+
pip install --user torchfunc
66+
```
67+
68+
### Nightly:
69+
70+
```shell
71+
pip install --user torchfunc-nightly
72+
```
73+
74+
## [Docker](https://cloud.docker.com/repository/docker/szymonmaszke/torchfunc)
75+
76+
__CPU standalone__ and various versions of __GPU enabled__ images are available
77+
at [dockerhub](https://cloud.docker.com/repository/docker/szymonmaszke/torchfunc).
78+
79+
For CPU quickstart, issue:
80+
81+
```shell
82+
docker pull szymonmaszke/torchfunc:18.04
83+
```
84+
85+
Nightly builds are also available, just prefix tag with `nightly_`. If you are going for `GPU` image make sure you have
86+
[nvidia/docker](https://github.com/NVIDIA/nvidia-docker) installed and it's runtime set.
87+
88+
# Contributing
89+
90+
If you find any issue or you think some functionality may be useful to others and fits this library, please [open new Issue](https://help.github.com/en/articles/creating-an-issue) or [create Pull Request](https://help.github.com/en/articles/creating-a-pull-request-from-a-fork).
91+
92+
To get an overview of something which one can done to help this project, see [Roadmap](https://github.com/szymonmaszke/torchfunc/blob/master/ROADMAP.md)

‎ROADMAP.md

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Roadmap
2+
3+
This document highlights possible improvements and additional implementations
4+
driving this library towards next release (version `0.2.0`).
5+
6+
You can check whether someone started work on this task in issues under the label
7+
`roadmap` and specific `improvement` or `implementation`.
8+
9+
Any help with tasks described below will be highly appreciated, thanks!
10+
11+
## Improvements
12+
13+
Following tasks might be considered improvements over what is already done.
14+
Please use label `roadmap` and `improvement` and describe the task you are willing to take
15+
(or describe new task you would consider an improvement).
16+
17+
- Increase test coverage (especially `torchfunc.performance` and `torchfunc.record` packages)
18+
- Testing with [Hypothesis](https://github.com/HypothesisWorks/hypothesis)
19+
- Documentation descriptions
20+
- Tutorials creation
21+
- Conda release
22+
23+
## Implementations
24+
25+
- `torchfunc.performance.technology` extensions, namely:
26+
- Approximation of [Arithmetic Intensity](https://docs.nvidia.com/deeplearning/sdk/dl-performance-guide/index.html#math-mem) for module
27+
- Approximation of good batch sizes for specific modules to avoid wave quantization (see [here](https://docs.nvidia.com/deeplearning/sdk/dl-performance-guide/index.html#gpu-execution))
28+
- Configurable invasive automatic improvement of module (e.g. changing data types to be tensor cores compatible), could be easily divided into multiple tasks.
29+
Open a PR if you are willing to discuss and/or talk about this functionality.
30+
31+
- `torchfunc.plot` extensions, namely:
32+
- Readable Visualization of Convolutional layers activation
33+
- Visualization of RNN activations
34+
- Basic visualization of Linear layer connections and their values

‎assets/banner.png

30 KB
Loading

‎assets/logos/large.png

22.7 KB
Loading

‎docs/.nojekyll

Whitespace-only changes.

0 commit comments

Comments
 (0)