Therbot aka therapy bot
DialoGPT for therapy
Theory of gpt 2
- Huggingface book chapter 8
- https://www.youtube.com/watch?v=XynJ-gM6aD0&list=PLH0lCpFdVeJuqQLyYNBlEfRydl2L-CBb_&index=9&ab_channel=Co-learningLounge
- http://jalammar.github.io/illustrated-gpt2/
Dry running GPT-2
- run as a black box
- undertsanding the loader,train and evalute function of gpt-2
- look out for videos and tutorial if any before jumping into the code
Data prep GPT-2
- make a csv with X candidates for the context using counsel-chat data
- Loading the json or csv to the loader
Fine tuning GPT-2
- Load GPT-2 with microsoft dialoGPT weights
Useful links
- https://docs.google.com/spreadsheets/d/1eagdebHdRzB99nxmKilmnyaoeDA9A0i92FdO6zHkcvA/edit?usp=sharing data
- http://www.nbertagnolli.com/jekyll/update/2020/05/11/counsel_chat.html json data
- microsoft/DialoGPT#36
- https://simpletransformers.ai/docs/convAI-data-formats/
- https://arxiv.org/pdf/1801.07243.pdf
- https://www.freecodecamp.org/news/discord-ai-chatbot/
- https://towardsdatascience.com/make-your-own-rick-sanchez-bot-with-transformers-and-dialogpt-fine-tuning-f85e6d1f4e30
- microsoft/DialoGPT#45 (comment)
- https://github.com/juan-csv/Chatbot-with-personality-DialogGPT
- https://github.com/rushic24/DialoGPT-Finetune/blob/main/DialoGPT_finetuning_Chatbot_on_Covid_Dataset.ipynb
- https://discuss.huggingface.co/t/trouble-with-fine-tuning-dialogpt-large/8161
- https://huggingface.co/transformers/v2.11.0/model_doc/gpt2.html
- https://arxiv.org/pdf/1911.00536.pdf