This repo contains a demo for the following technical report (arXiv):
@misc
{
nets-and-trees,
author = {Nenad Marku\v{s} and Ivan Gogi\'c and Igor S. Pand\v{z}i\'c and J\"{o}rgen Ahlberg},
title = {{Memory-Efficient Global Refinement of Decision-Tree Ensembles and its Application to Face Alignment}},
year = {2017},
eprint = {arXiv:1702.08481}
}
Basically, the report discusses some modifications of the method introduced by Ren et al. (CVPR2014), "Face Alignment at 3000 FPS via Regressing Local Binary Features". However, instead to face alignment, these ideas are here applied to MNIST classification. This approach can illustrate the basic idea of the report in a relatively few lines of code.
The code for this experiment can be found in the folder basic/.
Here we use the trees in the usual way:
the features for each tree are sampled from the whole 32x32 patch.
After compiling tree.c (use the provided makefile), learn the trees (256 trees of depth 12):
th lrntrees.lua
Next, learn a neural network on top of the tree outputs:
th lrnnn.lua --type rrr
You can try other architectures by specifying --type nn or --type full.
Note that all architectures easily fit the training data. However, the generalization performance is not spectacular in these experiments (~3% test error).
The code for this experiment can be found in the folder treecnv/.
Here we apply the trees in a convolutional way.
This leads to significant gains in accuracy.
Compile the treecnv library:
cc treecnv.c -o treecnv.so -I ~/torch/install/include -O3 -fopenmp -lm -fpic -shared
Next, run the demo:
th lrnnn.lua
After several training epochs (30 or so), the test error should fall well below 1%. This shows that exploiting spatial symmetry improves results (it serves as a strong regularizer?).
MIT.
Also note that this repo contains code modified from here.
For any additional information contact me at [email protected].