Data Preparation and Visualisation for Fine-tuning LLMs

Advanced Practices

I go deep into synthetic data preparation techniques for fine-tuning LLMs, including:

• What library to use for converting documents to markdown?

• How best to create chunks from document text?

• How to create high quality comprehensive question answer pairs?

• How to visualise your synthetic dataset via embeddings or tags?

• How to create a high quality evaluation dataset.

Cheers, Ronan

P.S. The scripts are in the ADVANCED-fine-tuning repo:

Video Links:

• Slides

---

Trelis Links:

🤝 Are you a talented developer? Work for Trelis

💡 Need Technical or Market Assistance? Book a Consult Here

💸 Starting a New Project/Venture? Apply for a Trelis Grant

---

Fine-Tuning Data Preparation: High-Quality Synthetic Datasets

Creating high-quality training data is essential for fine-tuning large language models. This article outlines a systematic approach to preparing synthetic question-answer datasets, covering document ingestion, chunking, question generation, and evaluation set creation.

Key Goals for Data Quality

1. Coverage: Questions must comprehensively cover document content across topics, formats, and difficulty levels

2. Contextualization: Questions need proper context to be unambiguous and answerable

3. Representative Evaluation: Eval datasets must reflect the distribution of topics in training data

4. Consistent Grading: Clear rubrics for evaluating answer correctness

Document Ingestion Methods

Three main approaches for converting PDFs to text:

1. Marker PDF: Most accurate, uses OCR and layout detection models. ~19s processing time (local on Mac M4 for 24 pages), $3/1000 pages

2. Microsoft's MarkItDown: Fast CPU-based conversion, ~free but lower quality

3. Gemini Vision: ~20s processing time, $0.50/1000 pages, handles embedded text in images

Chunking Strategy

Text is split into chunks using:

1. Sentence detection via regex or NLTK

2. Table extraction for structured data

3. Chunk size parameters:

   1. Minimum length (configurable)

   2. Maximum length (default 5000 tokens)

   3. Tables preserved as single chunks

Question Generation Pipeline

The process involves:

1. Generate document summary for context

2. For each chunk:

3. Pass chunk + summary to LLM

4. Generate multiple QA pairs iteratively until coverage is complete

5. Include evaluation criteria and difficulty rating

Visualization and Quality Assessment

Two methods for analyzing question distribution:

1. Embedding-based:

   1. Plot questions in 2D using dimensionality reduction

   2. Compare coverage across different models

   3. Identify potential gaps in coverage

2. Tag-based:

   1. Generate consistent tags across datasets

   2. Create histograms showing topic distribution

   3. Compare tag coverage between models

Evaluation Dataset Creation

Three approaches for creating evaluation sets:

1. Random Split: Simple but may create gaps in training coverage

2. Balanced Clone: Maintains distribution but measures verbatim learning

3. Balanced Clone with Rephrasing: Best practice - maintains distribution while testing generalization

The recommended approach uses:

1. 20% split or 32 examples (whichever is lower)

2. Clustering via elbow method for balanced sampling

3. Optional mirrored set for measuring overfitting

4. Question rephrasing to test generalization

This systematic approach to data preparation enables comprehensive coverage while maintaining high quality standards for fine-tuning. The resulting datasets can be evaluated objectively through embedding visualizations and tag distributions.