Upgraded pyspark to 3.0.1 also set up full requirements

Author: wtsimple

requirements.txt

@@ -1,5 +1,65 @@
-pytest
-pyspark
-numpy
-pandas
-plotnine
+absl-py==0.9.0
+astunparse==1.6.3
+attrs==19.3.0
+cachetools==4.0.0
+certifi==2019.11.28
+chardet==3.0.4
+cycler==0.10.0
+descartes==1.1.0
+gast==0.3.3
+google-auth==1.11.3
+google-auth-oauthlib==0.4.1
+google-pasta==0.2.0
+grpcio==1.27.2
+h5py==2.10.0
+idna==2.9
+joblib==0.14.1
+Keras==2.3.1
+Keras-Applications==1.0.8
+Keras-Preprocessing==1.1.0
+kiwisolver==1.1.0
+Markdown==3.2.1
+matplotlib==3.2.1
+mizani==0.7.2
+more-itertools==8.2.0
+numpy==1.18.2
+oauthlib==3.1.0
+opt-einsum==3.2.0
+packaging==20.3
+palettable==3.3.0
+pandas==1.1.4
+patsy==0.5.1
+plotnine==0.7.1
+pluggy==0.13.1
+protobuf==3.11.3
+psycopg2-binary==2.8.4
+py==1.8.1
+py4j==0.10.7
+pyasn1==0.4.8
+pyasn1-modules==0.2.8
+pyparsing==2.4.6
+pyspark==3.0.1
+pytest==5.4.1
+python-crfsuite==0.9.7
+python-dateutil==2.8.1
+pytz==2019.3
+PyYAML==5.3.1
+requests==2.23.0
+requests-oauthlib==1.3.0
+rsa==4.0
+scikit-learn==0.22.2.post1
+scipy==1.4.1
+seaborn==0.10.0
+six==1.14.0
+sklearn-crfsuite==0.3.6
+statsmodels==0.11.1
+tabulate==0.8.6
+tensorboard==2.1.1
+tensorflow==2.2.0rc1
+tensorflow-estimator==2.2.0rc0
+termcolor==1.1.0
+tqdm==4.43.0
+urllib3==1.25.8
+wcwidth==0.1.8
+Werkzeug==1.0.0
+wrapt==1.12.1

todo_list.md

@@ -18,7 +18,7 @@
     - [x] create an object container for the models
     - [x] initialize the models with default hyperparameters
     - [x] fit and compare the results with the evaluator
-- [ ] fit and compare several classification models with tuning and crossvalidation
+- [ ] **fit and compare several classification models with tuning and crossvalidation**
     - [ ] be able to pass a list of hyperparameters values for each hyperparameter
     - [ ] tune and obtain the best hyperparam set per model
     - [ ] compare the tuned models with the evaluator