Included some exercises

doc/orca-tutorial.md

@@ -32,7 +32,6 @@ From Matlab consoles, assuming you are on the `src` folder, the set of experimen
 ```MATLAB
 Utilities.runExperiments('../doc/tutorial/config-files/pom.ini')
 ```
-
 The syntax of these files will be explained in the [next subsection](orca-tutorial.md#syntax-of-ini-files). This should produce an output like this:
 ```MATLAB
 >> Utilities.runExperiments('../doc/tutorial/config-files/pom.ini')
@@ -105,6 +104,12 @@ title('AMAE performance (smaller is better)')
 
 ![AMAE performance of several methods](tutorial/images/pom-vs-svorim-vs-svc1v1.png)
 
+*** Exercise *** : you should repeat this barplots but considering:
+- One `global` (i.e. a metric where the class a priori probability is not considered) **nominal** metric.
+- One `global` **ordinal** metric. 
+- One **nominal** metric specifically designed for imbalanced datasets.
+- One **ordinal** metric specifically designed for imbalanced datasets.
+
 ### Syntax of `ini` files
 
 ORCA experiments are specified in configuration `ini` files, which run an algorithm for a collections of datasets (each dataset with a given number of partitions). The folder [src/config-files](src/config-files) contains example configuration files for running all the algorithms included in ORCA for all the algorithms and datasets of the [review paper](http://www.uco.es/grupos/ayrna/orreview). The following code is an example for running the Proportion Odds Model (POM), a.k.a. Ordinal Logistic Regression. Note that the execution of this `ini` file can take several hours:
@@ -460,7 +465,7 @@ ans =
      5     7     3    12     1
 >> targets = ERAData(:,end);
 >> k=10;
->> CVO = cvpartition(targets,'k',k);
+>> CVO = cvpartition(targets,'KFold',k);
 >> nameDataset = 'era';
 >> rootDir = fullfile('..', '..', 'exampledata', '10-fold', nameDataset);
 >> mkdir(rootDir);
@@ -524,7 +529,11 @@ for ff = 1:h
     dlmwrite(fullfile(rootDir,sprintf('test-%s.%d',nameDataset,ff-1)),ERAData(teIdx,:),' ');
 end
 ```
+The source code of this example is in [exampleERAHHoldout.m](../src/code-examples/exampleERAHHoldout.m). As can be checked, the `cvpartition` function performs the partitions, receiving the target vector. The targets are used in order to obtain a stratified partition.
+
+*** Exercise *** : you should prepare a `30holdout` set of partitions for the dataset `ESL`, which is included in the [exampledata](/exampledata). Try to find the differences between this dataset and ERA.
 
+*** Exercise *** : compare the results obtained for `ERA` and `ESL` datasets using the same experimental design you used in the [experiment section](orca-tutorial.md#launch-experiments-through-ini-files). Generate bar plots for comparing accuracy and AMAE.
 
 ### Warning about highly imbalanced datasets