Skip to content

Commit

Permalink
fixed types in test.bat and added link in readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Carldeboer committed Feb 27, 2018
1 parent 1d094fe commit 5a96829
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The outputs of the model are located in the `example/` folder. Model parameters
Motifs are represented as Kd matrices, which are essentially a negated log-odds PWM with a uniform background. See `example/PKdMs/` for examples.

# Tips for training models
Here, specific examples come from the test.bat file included in the repository.
Here, specific examples come from the [test.bat](https://github.com/Carldeboer/CisRegModels/blob/master/test.bat) file included in the repository.
1. Watch the MSE: When training a model, it is important that the model converge. In some cases, the default initial parameters are so far from the biological parameters that by the time the model has centred on the data, the parameters are so extreme that the gradients become 0 (and so the parameters become fixed at this extreme value). Thus, it is sometimes necessary to initialize parameters to either a small value (e.g. -ia 0.01 and -ip 0.01, for activities and potentiations, respectively), or a prior (e.g. -ic example/allTF_minKds_polyA_and_FZF1.txt ). On GPRA data (with a scale of 0-17), getting stuck at an MSE of ~18 indicates that the parameters have gone to irrecoverable extremes.
2. Play with the learning rate: The learning rate depends partly on the scale of your data and so picking the best learning rate is non-trivial (but see [here](https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10) and [here](https://hackernoon.com/8-deep-learning-best-practices-i-learned-about-in-2017-700f32409512). Included are ways to cycle or decay the learning rate (-clr and -lred options). I have so far not found these to be useful when using the Adam Optimizer, but they may be better with other optimizers.
3. Progressive learning of parameters: If you aim to have your parameters correspond to specific TFs (i.e. you load the motifs of known factors and desire that at the end the parameters learned represent those same factors), it is best to learn the parameters of the model progressively. The idea here is that changing the motifs before you know what those motifs are doing may result in the motif changing into a motif for a related factor (to better reflect the (perhaps randomly initialized) activity/potentiation parameters, e.g.). That said, there are certain parameter combinations that should be learned together (e.g. activities and potentiations), where, if learned separately, the models may converge to a solution that is suboptimal, but very difficult to break out of when a new parameter is introduced.
Expand Down
3 changes: 2 additions & 1 deletion test.bat
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ echo "############################################"
echo "Creating One-Hot representation of sequences"
echo "############################################"
seqsToOHC.py -i example/HighQuality.pTpA.Glu.test.txt.gz -m 110 -o example/HighQuality.pTpA.Glu.test.OHC.gz

echo "############################################"
echo "learning model from known motifs, holding concentrations and motifs static"
echo "############################################"
Expand All @@ -29,5 +30,5 @@ echo "############################################"
echo "testing ACPM model on training data and outputing predicted binding"
echo "############################################"
predictThermodynamicEnhancosomeModel.py -i example/HighQuality.pTpA.Glu.test.OHC.gz -sl 110 -b 128 -M ./example/test.model.APCM.Model -o example/test.model.APCM.pred.gz -ob
echo "See example/test.model.APCM.pos.pred.gz for predictions"
echo "See example/test.model.APCM.pred.gz for predictions"
echo "ALL COMMANDS SUCCEEDED"

0 comments on commit 5a96829

Please sign in to comment.