Training detector howto

Generate smooth normalized data from raw data

At first we have raw data here in the Sensors directory

https://yadi.sk/d/NJSeeOk5bYm7V

we need to normalize it first using Normalizer. Compile it and run this:

./Normalizer -output ~/work/ml/BlindMotionProject/Sensors/2015-01-25_SensorDatafile_norm.csv ~/work/ml/BlindMotionProject/Sensors/2015-01-25_SensorDatafile.csv

After that the data is smoothed with modifier

./modifier.coffee --input ~/work/ml/BlindMotionProject/Sensors/2014-12-16_SensorDatafile_norm.csv --output ~/work/ml/BlindMotionProject/Sensors/2014-12-16_SensorDatafile_smooth.csv --modifier ~/work/ml/BlindMotionProject/code/dashboard/modifiers/smooth.coffee

Prepaire data for learning

Generate csv files from raw data and events json file.

./prepaire_all_parallel.py

At the end several files will be generated, one for each date. The structure of the file is described in prepaire.py

Postprocess generated files to generate one train.csv that will be used to train the model.

./post_prepaire_all_parallel.py

At the end the outTrain.mat file will be created that can be used to train model.

Train model

octave -q main.m ~/work/ml/BlindMotionProject/workspace/outTrain.mat /tmp/myresult01

Trains model using outTrain.mat and saves trained model to /tmp/myresult01

Generate events from sensors

Generate chunks of sensors data from a sensors file

./generate_events.py -i ~/work/ml/BlindMotionProject/Sensors/2014-11-26_SensorDatafile_smooth.csv -o ~/work/ml/BlindMotionProject/workspace/events_gen/data.csv -t ~/work/ml/BlindMotionProject/workspace/events_gen/time.csv

Convert data csv to data.mat

octave -q ~/work/ml/BlindMotionProject/code/detector/core/prepaire_events.m ~/work/ml/BlindMotionProject/workspace/events_gen/data.csv ~/work/ml/BlindMotionProject/workspace/events_gen/data.mat

Predict events

octave -q predict_events.m result.mat ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/data.mat ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/time.csv ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_y.csv ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_time.csv

Predict events multiple nets

octave -q predict_events_multi_nets.m ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/data.mat ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/time.csv ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_y.csv ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_time.csv ~/temp/delme/results/tmpswPfk2 ~/temp/delme/results/tmpVDDhI4 ~/temp/delme/results/tmpuesuiJ

Convert prediction to json

./post_process_events.py -d ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_y.csv -t ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/out_time.csv -o ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/events.json

Compaire actual events with predicted

./compaire_events.py -a ~/work/ml/BlindMotionProject/Events/20140918/all.json -p ~/work/ml/BlindMotionProject/workspace/events_gen/results/2014-09-18/events.json

Do learning and prediction with scikit instead of neural networks in octave

will be soon here