Fine tuning LLMs for Memorization

Dataset Generation and Fine-tuning

➡️ Generate an expanded synthetic dataset for fine-tuning.

➡️ Engrain specific knowledge in a large language model.

I introduce some tips and tricks around dataset creation and hyper parameter (batch size, learning rate, epochs to train for)… drop me a comment below with any questions.

Cheers, Ronan

Links:

➡️ ADVANCED-fine-tuning Repo

➡️ ADVANCED-inference Repo

➡️ Trelis Function-calling Models

➡️ One-click Fine-tuning & Inference Templates

➡️ Tip Jar