Update

jmwang0117 · Sep 27, 2024 · 552cf94 · 552cf94
1 parent f63627f
commit 552cf94
Show file tree

Hide file tree

Showing 10 changed files with 911 additions and 146 deletions.
diff --git a/README.md b/README.md
@@ -1,21 +1,52 @@
-# OccRWKV
+<div align="center">   
+# 🤖 OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity
+*We will open source the complete code after the paper is accepted*
+</div>
 
-## OccRWKV: Rethinking Sparse Latent Representation for 3D Semantic Occupancy Prediction
 
+## 📢 News
 
-## Preperation
+- [2024/09]: OccRWKV's logs are available for download:
+<div align="center">
+
+| OccRWKV Results | Experiment Log |
+|:------------------------------------------------------------------:|:----------:|
+|OccRWKV on the SemanticKITTI hidden official test dataset | [link](https://connecthkuhk-my.sharepoint.com/:t:/g/personal/u3009632_connect_hku_hk/EYqFDMD6xexCqXwfZ_nPxEUB0akfqePg4TwuGiuf4fQK0Q?e=PFM1ma) |
+|OccRWKV train log | [link](https://connecthkuhk-my.sharepoint.com/:u:/g/personal/u3009632_connect_hku_hk/EcKG5MgDCTJJuu8DJ7VoS9sB0euzAEaMkpLjlY9LvRJ0GA?e=lwddX3) |
+
+</div>
+
+- [2024/08]: The pre-trained model can be downloaded at  [OneDrive](https://connecthkuhk-my.sharepoint.com/:u:/g/personal/u3009632_connect_hku_hk/ETCUIJ7rPnFJniQYMsDsPyIBHkzirRP4c3n-eU9fcBZTaA?e=P8AkQ2).
+- [2024/07]: 🔥 We released the code of OccRWKV. *The First Receptance Weighted Key Value (RWKV)-based 3D Semantic Occupancy Network*
+
+</br>
+
+```
+@article{wang2024omega,
+title={OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity},
+author={Wang, Junming and Yin, Wei and Long, Xiaoxiao and Zhang, Xinyu and Xing, Zebing and Guo, Xiaoyang and Qian, Zhang},
+year={2024}
+      } 
+```
+
+Please kindly star ⭐️ this project if it helps you. We take great efforts to develop and maintain it 😁.
+
+
+## 🛠️ Installation
 
-### Prerequisites
 ```
 conda create -n occ_rwkv python=3.10 -y
 conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y
 pip install spconv-cu120
 pip install tensorboardX
 pip install dropblock
 pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.1.0+cu121.html
+pip install -U openmim
+mim install mmcv-full
+pip install mmcls==0.25.0
 ```
 
-### Dataset
+## 💽 Dataset
 
 Please download the Semantic Scene Completion dataset (v1.1) from the [SemanticKITTI website](http://www.semantic-kitti.org/dataset.html) and extract it.
 
@@ -35,7 +66,7 @@ SemanticKITTI
 │   │  ├── ... ...
 ```
 
-## Getting Start
+## 🤗 Getting Start
 Clone the repository:
 ```
 https://github.com/jmwang0117/OccRWKV.git
@@ -49,15 +80,15 @@ We provide training routine examples in the `cfgs` folder. Make sure to change t
 
 ```
 $ cd <root dir of this repo>
-$ python train.py --cfg cfgs/DSC-Base.yaml --dset_root <path/dataset/root>
+$ bash scripts/run_train.sh
 ```
 ### Validation
 
 Validation passes are done during training routine. Additional pass in the validation set with saved model can be done by using the `validate.py` file. You need to provide the path to the saved model and the dataset root directory.
 
 ```
 $ cd <root dir of this repo>
-$ python validate.py --weights </path/to/model.pth> --dset_root <path/dataset/root>
+$ bash scripts/run_val.sh
 ```
 ### Test
 
@@ -69,21 +100,15 @@ You can provide which checkpoints you want to use for testing. We used the ones
 
 ```
 $ cd <root dir of this repo>
-$ python test.py --weights </path/to/model.pth> --dset_root <path/dataset/root> --out_path <predictions/output/path>
+$ bash scripts/run_test.sh
 ```
-### Pretrained Model
-
-You can download the models with the scores below from this [Google drive link](https://drive.google.com/file/d/1-b3O7QS6hBQIGFTO-7qSG7Zb9kbQuxdO/view?usp=sharing),
-
-| Model  | Segmentation | Completion |
-|--|--|--|
-| SSC-RS | 24.2 | 59.7 |
 
-<sup>*</sup> Results reported to SemanticKITTI: Semantic Scene Completion leaderboard ([link](https://codalab.lisn.upsaclay.fr/competitions/7170\#results)).
 
-## Acknowledgement
-This project is not possible without multiple great opensourced codebases.
-* [spconv](https://github.com/traveller59/spconv)
-* [LMSCNet](https://github.com/cv-rits/LMSCNet)
-* [SSA-SC](https://github.com/jokester-zzz/SSA-SC)
-* [GASN](https://github.com/ItIsFriday/PcdSeg)
+## 🏆 Acknowledgement
+Many thanks to these excellent open source projects:
+- [AGRNav](https://github.com/jmwang0117/AGRNav)
+- [Prometheus](https://github.com/amov-lab/Prometheus)
+- [SSC-RS](https://github.com/Jieqianyu/SSC-RS)
+- [semantic-kitti-api](https://github.com/PRBonn/semantic-kitti-api)
+- [Terrestrial-Aerial-Navigation](https://github.com/ZJU-FAST-Lab/Terrestrial-Aerial-Navigation)
+- [EGO-Planner](https://github.com/ZJU-FAST-Lab/ego-planner-swarm)
diff --git a/environment.yaml b/environment.yaml
@@ -0,0 +1,251 @@
+name: occ_rwkv
+channels:
+  - pytorch
+  - nvidia
+  - conda-forge
+dependencies:
+  - _libgcc_mutex=0.1=conda_forge
+  - _openmp_mutex=4.5=2_kmp_llvm
+  - aom=3.9.0=hac33072_0
+  - blas=2.116=mkl
+  - blas-devel=3.9.0=16_linux64_mkl
+  - brotli-python=1.1.0=py310hc6cd4ac_1
+  - bzip2=1.0.8=hd590300_5
+  - ca-certificates=2024.6.2=hbcca054_0
+  - cairo=1.18.0=h3faef2a_0
+  - certifi=2024.6.2=pyhd8ed1ab_0
+  - charset-normalizer=3.3.2=pyhd8ed1ab_0
+  - cuda-cudart=12.1.105=0
+  - cuda-cupti=12.1.105=0
+  - cuda-libraries=12.1.0=0
+  - cuda-nvrtc=12.1.105=0
+  - cuda-nvtx=12.1.105=0
+  - cuda-opencl=12.5.39=0
+  - cuda-runtime=12.1.0=0
+  - cuda-version=12.5=3
+  - dav1d=1.2.1=hd590300_0
+  - expat=2.6.2=h59595ed_0
+  - ffmpeg=7.0.1=gpl_hb399a10_100
+  - filelock=3.14.0=pyhd8ed1ab_0
+  - font-ttf-dejavu-sans-mono=2.37=hab24e00_0
+  - font-ttf-inconsolata=3.000=h77eed37_0
+  - font-ttf-source-code-pro=2.038=h77eed37_0
+  - font-ttf-ubuntu=0.83=h77eed37_2
+  - fontconfig=2.14.2=h14ed4e7_0
+  - fonts-conda-ecosystem=1=0
+  - fonts-conda-forge=1=0
+  - freetype=2.12.1=h267a509_2
+  - fribidi=1.0.10=h36c2ea0_0
+  - gettext=0.22.5=h59595ed_2
+  - gettext-tools=0.22.5=h59595ed_2
+  - gmp=6.3.0=h59595ed_1
+  - gmpy2=2.1.5=py310hc7909c9_1
+  - gnutls=3.7.9=hb077bed_0
+  - graphite2=1.3.13=h59595ed_1003
+  - harfbuzz=8.5.0=hfac3d4d_0
+  - icu=73.2=h59595ed_0
+  - idna=3.7=pyhd8ed1ab_0
+  - jinja2=3.1.4=pyhd8ed1ab_0
+  - lame=3.100=h166bdaf_1003
+  - lcms2=2.16=hb7c19ff_0
+  - ld_impl_linux-64=2.40=hf3520f5_3
+  - lerc=4.0.0=h27087fc_0
+  - libabseil=20240116.2=cxx17_h59595ed_0
+  - libasprintf=0.22.5=h661eb56_2
+  - libasprintf-devel=0.22.5=h661eb56_2
+  - libass=0.17.1=h8fe9dca_1
+  - libblas=3.9.0=16_linux64_mkl
+  - libcblas=3.9.0=16_linux64_mkl
+  - libcublas=12.1.0.26=0
+  - libcufft=11.0.2.4=0
+  - libcufile=1.10.0.4=0
+  - libcurand=10.3.6.39=0
+  - libcusolver=11.4.4.55=0
+  - libcusparse=12.0.2.55=0
+  - libdeflate=1.20=hd590300_0
+  - libdrm=2.4.120=hd590300_0
+  - libexpat=2.6.2=h59595ed_0
+  - libffi=3.4.2=h7f98852_5
+  - libgcc-ng=13.2.0=h77fa898_7
+  - libgettextpo=0.22.5=h59595ed_2
+  - libgettextpo-devel=0.22.5=h59595ed_2
+  - libgfortran-ng=13.2.0=h69a702a_7
+  - libgfortran5=13.2.0=hca663fb_7
+  - libglib=2.80.2=hf974151_0
+  - libgomp=13.2.0=h77fa898_7
+  - libhwloc=2.10.0=default_h5622ce7_1001
+  - libiconv=1.17=hd590300_2
+  - libidn2=2.3.7=hd590300_0
+  - libjpeg-turbo=3.0.0=hd590300_1
+  - liblapack=3.9.0=16_linux64_mkl
+  - liblapacke=3.9.0=16_linux64_mkl
+  - libnpp=12.0.2.50=0
+  - libnsl=2.0.1=hd590300_0
+  - libnvjitlink=12.1.105=0
+  - libnvjpeg=12.1.1.14=0
+  - libopenvino=2024.1.0=h2da1b83_7
+  - libopenvino-auto-batch-plugin=2024.1.0=hb045406_7
+  - libopenvino-auto-plugin=2024.1.0=hb045406_7
+  - libopenvino-hetero-plugin=2024.1.0=h5c03a75_7
+  - libopenvino-intel-cpu-plugin=2024.1.0=h2da1b83_7
+  - libopenvino-intel-gpu-plugin=2024.1.0=h2da1b83_7
+  - libopenvino-intel-npu-plugin=2024.1.0=he02047a_7
+  - libopenvino-ir-frontend=2024.1.0=h5c03a75_7
+  - libopenvino-onnx-frontend=2024.1.0=h07e8aee_7
+  - libopenvino-paddle-frontend=2024.1.0=h07e8aee_7
+  - libopenvino-pytorch-frontend=2024.1.0=he02047a_7
+  - libopenvino-tensorflow-frontend=2024.1.0=h39126c6_7
+  - libopenvino-tensorflow-lite-frontend=2024.1.0=he02047a_7
+  - libopus=1.3.1=h7f98852_1
+  - libpciaccess=0.18=hd590300_0
+  - libpng=1.6.43=h2797004_0
+  - libprotobuf=4.25.3=h08a7969_0
+  - libsqlite=3.46.0=hde9e2c9_0
+  - libstdcxx-ng=13.2.0=hc0a3c3a_7
+  - libtasn1=4.19.0=h166bdaf_0
+  - libtiff=4.6.0=h1dd3fc0_3
+  - libunistring=0.9.10=h7f98852_0
+  - libuuid=2.38.1=h0b41bf4_0
+  - libva=2.21.0=h4ab18f5_2
+  - libvpx=1.14.1=hac33072_0
+  - libwebp-base=1.4.0=hd590300_0
+  - libxcb=1.15=h0b41bf4_0
+  - libxcrypt=4.4.36=hd590300_1
+  - libxml2=2.12.7=hc051c1a_1
+  - libzlib=1.3.1=h4ab18f5_1
+  - llvm-openmp=15.0.7=h0cdce71_0
+  - markupsafe=2.1.5=py310h2372a71_0
+  - mkl=2022.1.0=h84fe81f_915
+  - mkl-devel=2022.1.0=ha770c72_916
+  - mkl-include=2022.1.0=h84fe81f_915
+  - mpc=1.3.1=hfe3b2da_0
+  - mpfr=4.2.1=h9458935_1
+  - mpmath=1.3.0=pyhd8ed1ab_0
+  - ncurses=6.5=h59595ed_0
+  - nettle=3.9.1=h7ab15ed_0
+  - networkx=3.3=pyhd8ed1ab_1
+  - numpy=1.26.4=py310hb13e2d6_0
+  - ocl-icd=2.3.2=hd590300_1
+  - openh264=2.4.1=h59595ed_0
+  - openjpeg=2.5.2=h488ebb8_0
+  - openssl=3.3.1=h4ab18f5_0
+  - p11-kit=0.24.1=hc5aa10d_0
+  - pcre2=10.43=hcad00b1_0
+  - pillow=10.3.0=py310hf73ecf8_0
+  - pip=24.0=pyhd8ed1ab_0
+  - pixman=0.43.2=h59595ed_0
+  - pthread-stubs=0.4=h36c2ea0_1001
+  - pugixml=1.14=h59595ed_0
+  - pysocks=1.7.1=pyha2e5f31_6
+  - python=3.10.14=hd12c33a_0_cpython
+  - python_abi=3.10=4_cp310
+  - pytorch=2.1.0=py3.10_cuda12.1_cudnn8.9.2_0
+  - pytorch-cuda=12.1=ha16c6d3_5
+  - pytorch-mutex=1.0=cuda
+  - pyyaml=6.0.1=py310h2372a71_1
+  - readline=8.2=h8228510_1
+  - snappy=1.2.0=hdb0a2a9_1
+  - svt-av1=2.1.0=hac33072_0
+  - sympy=1.12=pypyh9d50eac_103
+  - tbb=2021.12.0=h297d8ca_1
+  - tk=8.6.13=noxft_h4845f30_101
+  - torchaudio=2.1.0=py310_cu121
+  - torchtriton=2.1.0=py310
+  - torchvision=0.16.0=py310_cu121
+  - typing_extensions=4.12.2=pyha770c72_0
+  - wheel=0.43.0=pyhd8ed1ab_1
+  - x264=1!164.3095=h166bdaf_2
+  - x265=3.5=h924138e_3
+  - xorg-fixesproto=5.0=h7f98852_1002
+  - xorg-kbproto=1.0.7=h7f98852_1002
+  - xorg-libice=1.1.1=hd590300_0
+  - xorg-libsm=1.2.4=h7391055_0
+  - xorg-libx11=1.8.9=h8ee46fc_0
+  - xorg-libxau=1.0.11=hd590300_0
+  - xorg-libxdmcp=1.1.3=h7f98852_0
+  - xorg-libxext=1.3.4=h0b41bf4_2
+  - xorg-libxfixes=5.0.3=h7f98852_1004
+  - xorg-libxrender=0.9.11=hd590300_0
+  - xorg-renderproto=0.11.1=h7f98852_1002
+  - xorg-xextproto=7.3.0=h0b41bf4_1003
+  - xorg-xproto=7.0.31=h7f98852_1007
+  - xz=5.2.6=h166bdaf_0
+  - yaml=0.2.5=h7f98852_2
+  - zlib=1.3.1=h4ab18f5_1
+  - zstd=1.5.6=ha6fb4c9_0
+  - pip:
+      - absl-py==2.1.0
+      - addict==2.4.0
+      - aliyun-python-sdk-core==2.15.1
+      - aliyun-python-sdk-kms==2.16.3
+      - ccimport==0.4.2
+      - cffi==1.16.0
+      - click==8.1.7
+      - colorama==0.4.6
+      - contourpy==1.2.1
+      - crcmod==1.7
+      - cryptography==42.0.8
+      - cumm-cu120==0.4.11
+      - cycler==0.12.1
+      - dropblock==0.3.0
+      - fire==0.6.0
+      - fonttools==4.53.0
+      - fsspec==2024.6.0
+      - grpcio==1.64.1
+      - importlib-metadata==7.1.0
+      - jmespath==0.10.0
+      - kiwisolver==1.4.5
+      - lark==1.1.9
+      - markdown==3.6
+      - markdown-it-py==3.0.0
+      - matplotlib==3.9.0
+      - mdurl==0.1.2
+      - mmcls==0.25.0
+      - mmcv-full==1.7.2
+      - model-index==0.1.11
+      - ninja==1.11.1.1
+      - opencv-python==4.10.0.82
+      - opendatalab==0.0.10
+      - openmim==0.3.9
+      - openxlab==0.1.0
+      - ordered-set==4.1.0
+      - oss2==2.17.0
+      - packaging==24.1
+      - pandas==2.2.2
+      - pccm==0.4.11
+      - platformdirs==4.2.2
+      - portalocker==2.8.2
+      - protobuf==4.25.3
+      - pybind11==2.12.0
+      - pycparser==2.22
+      - pycryptodome==3.20.0
+      - pyg-lib==0.4.0+pt21cu121
+      - pygments==2.18.0
+      - pyparsing==3.1.2
+      - python-dateutil==2.9.0.post0
+      - pytz==2023.4
+      - requests==2.28.2
+      - rich==13.4.2
+      - scipy==1.13.1
+      - setuptools==60.2.0
+      - six==1.16.0
+      - spconv-cu120==2.3.6
+      - tabulate==0.9.0
+      - tensorboard==2.17.0
+      - tensorboard-data-server==0.7.2
+      - tensorboardx==2.6.2.2
+      - termcolor==2.4.0
+      - thop==0.1.1-2209072238
+      - tomli==2.0.1
+      - torch-cluster==1.6.3+pt21cu121
+      - torch-scatter==2.1.2+pt21cu121
+      - torch-sparse==0.6.18+pt21cu121
+      - torch-spline-conv==1.2.2+pt21cu121
+      - torchprofile==0.0.4
+      - tqdm==4.65.2
+      - tzdata==2024.1
+      - urllib3==1.26.18
+      - werkzeug==3.0.3
+      - yapf==0.40.2
+      - zipp==3.19.2
+prefix: /home/jmwang/miniforge3/envs/occ_rwkv