We summarise the paper Coordinate Decent Algorithms and implement the algorithms in the repository. Coordinate descent algorithms are iterative methods in which each iterate is obtained by fixing most components of the variable vector x at their values from the current iteration, and approximately minimizing the objective with respect to the remaining components. They have been used in applications for many years, and their popularity continues to grow because of their usefulness in data analysis, machine learning, and other areas of current interest.
Accelerated_RCD.R: Accelerated Randomized Coordinate Descent (Nesterov 2012)
Pathwise_CD.R: Pathwise Coordinate descent for the lasso
RCD.R: Coordinate Descent method with randomized / cyclic rules and fixed step size for solving quadratic form objective function and Gradient Descent as baseline model.
Separable_RCD.R: Separable Coordinate Descent Algorithm for solving quadratic form objective function with L1 penalty(LASSO)
STA243_final.pdf Final report
Experiment.R Experiment file
Some figs used in the report.