Context Compression is an optimization technique that sits between the retrieval step and the generation step in a RAG pipeline. It reduces the size of retrieved documents before they are fed into the LLM, keeping only the most relevant information while discarding redundant or irrelevant content.
Imagine you’re a busy executive preparing for a meeting. Your assistant brings you 50 pages of reports (retrieved documents). Instead of reading all 50 pages, your assistant first skims through them, highlights the 5 pages that are actually relevant to today’s agenda, and gives you only those. That’s Context Compression — a “smart assistant” that filters and condenses information before it reaches the LLM.
Context Compression techniques can be classified into several categories：
By Compression Approach：
Extractive Compression（抽取式压缩） Selects and keeps only the most important parts of the text, discarding the rest. Think of it as “highlighting” key sentences. 选择并只保留文本中最重要的部分，丢弃其余部分。可以理解为“高亮”关键句子。
Abstractive Compression（抽象式/生成式压缩） Generates a new, shorter summary that captures the essence of the original text. Think of it as “summarization. 生成一个新的、更短的摘要，捕捉原文的精华。可以理解为“摘要生成
Soft/Hard Compression（软/硬压缩）
Key Takeaways
要点
EN
CN
Context Compression定义
Optimization technique that reduces retrieved context size before LLM generation
在LLM生成之前减少检索上下文大小的优化技术
核心动机
Reduce token cost, latency, and context window issues
减少token成本、延迟和上下文窗口问题
LLMLingua方法
Uses a small model to calculate perplexity and remove non-essential tokens
使用小模型计算困惑度，删除非必要token
LongLLMLingua改进
Query-aware compression with contrastive perplexity
使用对比困惑度的查询感知压缩
ACC-RAG特点
Dynamically adjusts compression rate based on input complexity
根据输入复杂度动态调整压缩率
SARA方法
Combines text snippets with semantic compression vectors
Advanced RAG (Advanced Retrieval-Augmented Generation) is an evolutionary upgrade over the basic “Naive RAG” pipeline. It adds a suite of optimization techniques at every stage of the RAG workflow — pre-retrieval, retrieval, post-retrieval, and evaluation — to systematically improve retrieval precision, recall, and generation quality.
Naive RAG is like sending a junior intern to the library with a single, vague question, grabbing the first 5 books they find, and copying paragraphs directly. Advanced RAG is like sending a team of expert researchers who:
Rephrase the question in 5 different ways (Query Rewrite)
Write a fake “perfect answer” first to know exactly what to look for (HyDE)
Search both the card catalog AND the full-text index (Hybrid Search)
Have a senior editor re-rank the top results (Reranker)
Summarize and compress the findings before presenting them (Context Compression)
And finally, run a quality check on the entire process (Evaluation)
RAG (Retrieval-Augmented Generation) is an AI architecture that retrieves relevant information from external knowledge sources and provides it to an LLM, enabling the model to generate accurate, up-to-date, and context-aware responses.
LLM = Answer from Training while RAG = Search First + Then Answer
What is RAG 2.0? There is no official industry standard that defines “RAG 1.0”, “RAG 2.0”, or “RAG 3.0”.
The concept of “RAG 2.0”, known as Agentic RAG, represents a shift from a “simple data retrieval plugin” to a “deeply integrated, agentic system with reasoning capabilities.” The industry has indeed evolved from 1.0 and is now progressing toward 3.0.
Simplest Way to Remember
Version
Meaning
RAG 1.0
Search Once
RAG 2.0
Search + Reason
RAG 3.0 (coming ? on the way?)
Agent + Search + Tools + Workflow
RAG Foundation: Embedding
Embedding is a fundamental technology in machine learning and natural language processing that transforms discrete or complex objects (such as words, sentences, or images) into numerical vector representations of a fixed dimension
A simple RAG pipeline has two phases and five stages
Phase 1: Indexing (Offline / 离线阶段) Build the searchable knowledge base before users ask questions. 离线阶段（索引）：先把所有文档加载进来，切分成小块，调用 Embedding 模型转成向量，最后存进 FAISS 等向量数据库里。
Phase 2: Retrieval & Generation (Online / 在线阶段) Execute at query time for each user question. 在线阶段（检索+生成）：用户提问时，先把问题也转成向量，去库里找最相似的 Top-K 个块，把这几个块作为“参考资料”连同问题一起扔给 GPT，让它写出最终回答。
In AI, a chunking strategy is the method used to break down large pieces of information—like long documents, complex prompts, or even audio and video—into smaller, more manageable segments called “chunks”. These chunks are the fundamental units that AI systems, especially Large Language Models (LLMs), work with to understand, process, and retrieve information efficiently.
Common Chunking Strategies
The best strategy depends on the type of data and the specific task. Here are the most common approaches:
Strategy
Description
Best For
Fixed-Size Chunking
Splits text into chunks of a predetermined size (e.g., a specific number of words, characters, or tokens). Often uses a “sliding window” with overlap to preserve context across boundaries.
Simple implementation; works well when document structure is uniform and not critical.
Semantic Chunking
Groups text based on meaning, using algorithms to find natural topic boundaries and keep related ideas together.
Maintaining the coherence of ideas within each chunk for better understanding.
Structure-Aware Chunking
Respects the natural format of the document, splitting at points like paragraphs, sentences, or headers.
Documents with clear structure (e.g., articles, reports, code) where breaking mid-section would lose meaning.
Recursive Chunking
Starts with large chunks and recursively splits them into smaller ones until they meet a target size, trying to respect natural boundaries like paragraphs or sentences.
A balanced approach that aims to create the largest possible meaningful chunks.
Multimodal Chunking
Extends the concept to non-text data, such as segmenting audio by silences, video by scene changes, or identifying objects within images.
AI systems that process images, audio, and video, not just text
Chunking Strategy: Recursive Chunking
Recursive Chunking is a hierarchical text splitting strategy that uses a priority list of separators (e.g., ["\n\n", "\n", " ", """]), starting with the highest-level (semantically strongest) separator. If a chunk still exceeds the chunk_size limit after splitting, it recursively applies the next-level separator to that chunk, continuing until all chunks meet the size requirement.
Semantic Chunking is a strategy for splitting documents into smaller pieces (chunks) based on meaning, rather than on fixed character counts or simple separators. It tries to keep sentences or paragraphs that are about the same topic together, and split where the topic changes. Think of it as “a smart editor who knows where one idea ends and the next begins
Hybrid Search refers to a search technique that combines multiple search algorithms simultaneously to retrieve the most relevant results. It most commonly merges Lexical (Keyword) Search with Semantic (Vector/Dense) Search. 基于关键词的搜索（词汇搜索）与基于语义的搜索（向量/稠密搜索）
Lexical Search (Sparse Retrieval): Typically powered by algorithms like BM25 or TF-IDF. It relies on exact keyword matching and statistical term frequency. It excels at finding specific proper nouns, IDs, or rare terminology (e.g., “error code 404”). 词汇搜索（稀疏检索）： 通常由 BM25 或 TF-IDF 等算法驱动。它依赖于精确的关键词匹配和统计词频。它在查找特定的专有名词、ID 或罕见术语时表现出色（例如：“错误代码 404”）。
Semantic Search (Dense Retrieval): Powered by embedding models (e.g., Sentence-BERT). It converts text into high-dimensional vectors and retrieves documents based on “meaning” rather than exact words. It excels at understanding synonyms, context, and natural language queries (e.g., “How to fix a broken internet connection”). 语义搜索（稠密检索）： 由嵌入模型（如 Sentence-BERT）驱动。它将文本转换为高维向量，并根据“含义”而非精确词汇来检索文档。它在理解同义词、上下文和自然语言查询（例如：“如何修复断开的网络连接”）方面表现出色。
BM25 (keyword-based search)
BM25 (Best Matching 25) is a keyword-based ranking algorithm used in information retrieval to score and rank documents based on their relevance to a search query. It’s called “Best Matching 25” because it was the 25th variant in a series of scoring functions proposed by its creators. BM25 is the default ranking algorithm in Elasticsearch and most production search engines
RRF stands for Reciprocal Rank Fusion. It is an algorithm that merges multiple ranked result lists from different search systems into a single unified ranking. 全称是 Reciprocal Rank Fusion（倒数排名融合）。它是一种将来自不同检索系统的多个排名结果列表合并成一个统一排名的算法.
Compare normal DB, such as SQL DB, MySQL, PostgreSQL, They fundamental difference is what they search for and how they find it. A traditional SQL database (like MySQL, PostgreSQL without pgvector) is built for exact matching and structured queries. It answers questions like: “Find the customer with ID = 12345” or “Give me all orders over $100.” A Vector Database is built for semantic similarity and unstructured data. It answers questions like: “Find all documents that talk about the same topic as this paragraph” or “Show me products that look visually similar to this image.”
Normal DB vs Vector DB
要点
EN
CN
核心：精确匹配 vs. 语义相似度
Core: Exact matching vs. Semantic similarity
核心：精确匹配 vs. 语义相似度
SQL存结构化数据（数字/字符串），Vector存浮点数数组（含义）
SQL stores structured data (numbers/strings); Vector stores float arrays (meaning)
SQL存结构化数据，Vector存浮点数数组（含义）
SQL查询用WHERE精确条件；Vector查询用ORDER BY距离
SQL queries use WHERE exact conditions; Vector queries use ORDER BY distance
SQL results are binary (match/no match); Vector results are ranked (similarity scores)
SQL结果是二元的；Vector结果是排序的
SQL适合事务、财务报表；Vector适合RAG、推荐、AI
SQL suits transactions, ledgers; Vector suits RAG, recommendations, AI
SQL适合事务、报表；Vector适合RAG、推荐、AI
专用向量库不能做JOIN和ACID；但pgvector可以在PostgreSQL中兼得
Dedicated vector DBs can’t do JOINs/ACID; pgvector lets you have both in PostgreSQL
专用向量库不能做JOIN和ACID；pgvector可以让两者兼得
在实际RAG中，两者是互补的，不是替代关系
In real-world RAG, they are complementary, not replacements
在实际RAG中，两者是互补的，不是替代关系
Vector database selection is the process of choosing the right vector database technology for your AI application from dozens of available options — Milvus, Qdrant, Weaviate, Pinecone, pgvector, Chroma, and more.
Mainstream Vector Database Landscape
Category
Examples
CN
Fully Managed (PaaS)
Pinecone, Zilliz Cloud, Weaviate Cloud
全托管云服务
Self-Hosted Open Source
Qdrant, Milvus, Weaviate, Chroma
自托管开源
Database Extensions
pgvector, MongoDB Vector Search, Elasticsearch
数据库扩展
Cloud Provider Services
Azure AI Search, AWS S3 Vectors, Tencent Cloud VDB
云厂商服务
Embedded/Specialized
SQLite (vector), LanceDB
嵌入式/专用
Detailed Comparison
Database
Avg Query Time
Cost (1M @ 1536-dim)
Best For
CN 最适合
Milvus/Zilliz
50.7ms
$115/mo
Fastest queries + good flexibility
最快查询+灵活性好
Weaviate
51.7ms
$160/mo
Native datetime/geo + hybrid search
原生时间/地理+混合检索
Qdrant
73.1ms
$103/mo
Best balance (speed + flexibility + cost)
最佳平衡（速度+灵活性+成本）
Pinecone
106.3ms
$30/mo
Cheapest (⚠️ poor schema flexibility)
最便宜（⚠️ Schema灵活性差）
Chroma
275.4ms
$139/mo
Easiest setup + prototyping
最简单设置+原型开发
7 Types of Data Stored in VectorDB in AI Projects
类型
EN
CN
更新频率
过滤器
主要用途
RAG 文档块
RAG Document Chunks
RAG 文档块
低
source, page
问答
用户记忆 (Mem0)
User Memory (Mem0)
用户记忆 (Mem0)
高
user_id
个性化
工具 Schema
Tool Schemas
工具 Schema
中
tool_category
工具选择
Agent 轨迹
Agent Trajectories
Agent 轨迹
中
user_id, outcome
经验复用
黄金数据集
Golden Dataset
黄金数据集
低
category
评估
语义缓存
Semantic Cache
语义缓存
中
无
降本提速
代码索引
Code Index
代码索引
低
language, path
代码生成
Reranker is a sophisticated machine learning model designed to refine and reorder a list of candidate items—such as search results, document passages—to maximize their relevance to a specific query or context.
Advanced RAG (Advanced Retrieval-Augmented Generation) is an evolutionary upgrade over the basic “Naive RAG” pipeline. It adds a suite of optimization techniques at every stage of the RAG workflow — pre-retrieval, retrieval, post-retrieval, and evaluation — to systematically improve retrieval precision, recall, and generation quality.
