Initial commit

Author: rraileanu

CODE_OF_CONDUCT.md

@@ -0,0 +1,5 @@
+# Code of Conduct
+
+Facebook has adopted a Code of Conduct that we expect project participants to adhere to.
+Please read the [full text](https://code.fb.com/codeofconduct/)
+so that you can understand what actions will and will not be tolerated.

CONTRIBUTING.md

@@ -0,0 +1,29 @@
+# Contributing to Impact-Driven-Exploration
+We want to make contributing to this project as easy and transparent as
+possible.
+
+## Pull Requests
+We actively welcome your pull requests.
+
+1. Fork the repo and create your branch from `master`.
+2. If you've added code that should be tested, add tests.
+3. If you've changed APIs, update the documentation.
+4. If you haven't already, complete the Contributor License Agreement ("CLA").
+
+## Contributor License Agreement ("CLA")
+In order to accept your pull request, we need you to submit a CLA. You only need
+to do this once to work on any of Facebook's open source projects.
+
+Complete your CLA here: <https://code.facebook.com/cla>
+
+## Issues
+We use GitHub issues to track public bugs. Please ensure your description is
+clear and has sufficient instructions to be able to reproduce the issue.
+
+Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
+disclosure of security bugs. In those cases, please go through the process
+outlined on that page and do not file a public issue.
+
+## License
+By contributing to Impact-Driven-Exploration, you agree that your contributions will be licensed
+under the LICENSE file in the root directory of this source tree.

LICENSE

@@ -0,0 +1,42 @@
+# RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
+
+This is an implementation of the method proposed in <a href="https://openreview.net/pdf?id=rkg-TJBFPB">RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments</a>, which was published at ICLR 2020. The code includes all the baselines and ablations used in the paper. 
+
+We propose a novel type of intrinsic reward which encourges the agent to take actions that result in significant changes to its representation of the environment state.
+
+## Installation
+
+```
+# create a new conda environment
+conda create -n ride-env python=3.6.8
+conda activate ride-env 
+
+# install PyTorch 
+conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=9.0 -c pytorch
+
+# install other requirements
+git clone [email protected]:fairinternal/impact-driven-exploration.git
+cd impact-driven-exploration
+pip install -r requirements.txt
+
+# install the MiniGrid environments 
+cd impact-driven-exploration
+git clone https://github.com/maximecb/gym-minigrid.git
+cd gym-minigrid
+pip install -e .
+```
+
+## Train RIDE on MiniGrid
+```
+cd impact-driven-exploration
+
+OMP_NUM_THREADS=1 python main.py --model ride --env MiniGrid-MultiRoom-N12-S10-v0 
+
+OMP_NUM_THREADS=1 python main.py --model ride --env MiniGrid-MultiRoom-N10-S4-v0 --intrinsic_reward_coef 0.1 --entropy_cost 0.0005
+```
+
+## Acknowledgements
+Our vanilla RL algorithm is based on [Torchbesat](https://github.com/facebookresearch/torchbeast), which is an open source implementation of IMPALA.
+
+## License
+This code is under the CC-BY-NC 4.0 (Attribution-NonCommercial 4.0 International) license.

README.md

@@ -0,0 +1,5 @@
+# Copyright (c) Facebook, Inc. and its affiliates.
+# All rights reserved.
+
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.

__init__.py

@@ -0,0 +1,39 @@
+# Copyright (c) Facebook, Inc. and its affiliates.
+# All rights reserved.
+
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+from src.arguments import parser 
+
+from src.algos.torchbeast import train as train_vanilla 
+from src.algos.count import train as train_count
+from src.algos.curiosity import train as train_curiosity 
+from src.algos.rnd import train as train_rnd
+from src.algos.ride import train as train_ride
+from src.algos.no_episodic_counts import train as train_no_episodic_counts
+from src.algos.only_episodic_counts import train as train_only_episodic_counts
+
+def main(flags):
+    if flags.model == 'vanilla':
+        train_vanilla(flags)
+    elif flags.model == 'count':
+        train_count(flags)
+    elif flags.model == 'curiosity':
+        train_curiosity(flags)
+    elif flags.model == 'rnd':
+        train_rnd(flags)
+    elif flags.model == 'ride':
+        train_ride(flags)
+    elif flags.model == 'no-episodic-counts':
+        train_no_episodic_counts(flags)
+    elif flags.model == 'only-episodic-counts':
+        train_only_episodic_counts(flags)
+    else:
+        raise NotImplementedError("This model has not been implemented. "\
+        "The available options are: vanilla, count, curiosity, rnd, ride, \
+        no-episodic-counts, and only-episodic-count.")
+
+if __name__ == '__main__':
+    flags = parser.parse_args()
+    main(flags)

main.py

@@ -0,0 +1,27 @@
+certifi==2019.11.28
+cffi==1.13.2
+cloudpickle==1.2.2
+future==0.18.2
+gitdb2==2.0.6
+GitPython==3.0.5
+gym==0.15.4
+gym-super-mario-bros==7.3.0
+mkl-fft==1.0.15
+mkl-random==1.1.0
+mkl-service==2.3.0
+nes-py==8.1.1
+numpy==1.17.4
+olefile==0.46
+opencv-python==4.1.2.30
+Pillow==6.2.1
+pycparser==2.19
+pyglet==1.3.2
+PyQt5==5.13.2
+PyQt5-sip==12.7.0
+scipy==1.3.3
+six==1.13.0
+smmap2==2.0.5
+torch==1.1.0
+torchvision==0.3.0
+tqdm==4.40.2
+vizdoom==1.1.7

requirements.txt

@@ -0,0 +1,5 @@
+# Copyright (c) Facebook, Inc. and its affiliates.
+# All rights reserved.
+
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.

src/__init__.py

@@ -0,0 +1,5 @@
+# Copyright (c) Facebook, Inc. and its affiliates.
+# All rights reserved.
+
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.