In this video I will show you how to fine-tune the Alpaca model for any language. And it only costs $3! How did I figure this out? Watch the whole video to understand. I'll show you how to translate the cleaned Alpaca dataset. We will then use the translated dataset to fine-tune the Alpaca model (not the LLaMA model) for our desired language. For this, we will either use DeepL or ChatGPT. Also, I will show you how to evaluate the quality of your fine-tuned model. Last but not least you will learn how you can interact with your fine-tuned model in a UI. As always, if you have any questions, don't hesitate to reach out. Enjoy! :-)
My Medium Article for This Video:
https://medium.com/p/370f63753f94
Medium Article Showing the Evaluation Results:
https://medium.com/p/8e363a0a99ca
GitHub Repository for This Video:
https://github.com/thisserand/alpaca-lora-finetune-language
00:00:00 Intro
00:01:52 Calculating Estimated Costs
00:09:03 Decision Making
00:10:03 Creating A Subset Dataset
00:12:00 Dataset Translation
00:17:36 Fine-Tuning the Alpaca Model
00:26:15 Model Inference
00:27:18 How Much Training Data Do We Need?
00:31:24 Evaluation
00:37:32 Outro
References:
Self-Instruct Repository: https://github.com/yizhongw/self-instruct
Alpaca Blog Post: https://crfm.stanford.edu/2023/03/13/...
Alpaca Repository: https://github.com/tatsu-lab/stanford_alpaca
Alpaca-LoRA: https://github.com/tloen/alpaca-lora
AlpacaDataCleaned Repository: https://github.com/gururise/AlpacaDataCleaned
GPT-3.5 Documentation: https://platform.openai.com/docs/models/gpt-3-5
OpenAI API Introduction: https://platform.openai.com/docs/guides/chat/introduction
DeepL Pricing: https://www.deepl.com/pro-api
LLaMA Paper: https://arxiv.org/pdf/2302.13971.pdf
Is ChatGPT A Good Translator? Paper: https://arxiv.org/pdf/2301.08745v2.pdf
OpenAI Pricing: https://openai.com/pricing
vast.ai: https://vast.ai/
Stay in Touch
Medium
https://medium.com/@martin-thissen
LinkedIn
https://linkedin.com/in/mthissen135/
YouTube
Of course, feel free to subscribe to my channel! :-)
Of course, financial support is completely voluntary, but I was asked for it:
https://patreon.com/MartinThissen
https://ko-fi.com/martinthissen
