Document-Reading Agents

and read-write memory

Document-Writing Agents

Giving LLMs memory lets them remember who they are talking to.

Read-write memory also allows LLMs to "take notes" and read a document or book one chapter at a time - while remembering things from the past.

I walk through how to:

1. Build a document data base

2. Let LLMs read chapters from it one by one.

3. Let LLMs use Cntl+F !

4. Let LLMs store notes on their reading in local memory (i.e. in context).

Cheers, Ronan

🛠 Explore Fine-tuning, Inference, Vision, Audio, and Evaluation Tools

💡 Consulting (Technical Assistance OR Market Insights)

🤝 Join the Trelis Team

💸 Grants Program

---

Building Document-Reading Agents with Read-Write Memory

A technical overview of implementing document reading capabilities for LLMs by combining read-write memory with basic search and retrieval functions.

Core Memory Components

The system uses three distinct memory types:

1. Local context memory (recent conversation turns)

2. Read-only disk memory (conversation history database)

3. Read-write memory (new addition for persistent notes)

The read-write memory allows the LLM to:

1. Store user preferences and details

2. Take notes while reading documents

3. Maintain information between conversations

4. Use first-in-first-out (FIFO) memory management

Document Database Implementation

Key features:

1. Documents stored as markdown files

2. Simple keyword search (Ctrl+F style)

3. Line-by-line reading capability

4. JSON-based document metadata storage

5. Document conversion using marker-pdf library

Technical Details

Memory allocation:

1. Read-write memory: 4,000 tokens

2. Read-only memory: 16,000 tokens

3. Total context: 32,000 tokens

4. User messages: 250 tokens

5. Assistant responses: 500 tokens

Command structure:

1. XML-style tags for operations

2. Search syntax: query

3. Read syntax: filename:start_line:end_line

4. Write syntax: content

Practical Limitations

Current constraints:

1. Basic keyword search may require multiple iterations

2. Models struggle with comprehensive document review

3. Context length limits document chunk size

4. Memory management needed for long conversations

5. Human oversight recommended for thorough analysis

System Architecture

Core components:

1. Memory manager for FIFO operations

2. Document database interface

3. Tag-based command processor

4. Context window management

5. Stop token handling for commands

The system demonstrates how basic tools like keyword search and line-by-line reading can enable document comprehension tasks, though with some limitations requiring human oversight.