RAG in Practice: Integrating Lark Docs into an Enterprise Knowledge Base

Background

Our team built an LLM-based AI data platform at TikTok — providing AI chat, knowledge-base Q&A, and data generation capabilities, with APIs consumed by upstream systems like LabelGPT. The knowledge-base Q&A module originally supported .txt, .docx, .pdf, and .csv files. To better serve internal teams, we needed to integrate Lark (Feishu) documents into the Q&A pipeline. This post documents the full engineering evolution.

What is RAG

RAG (Retrieval-Augmented Generation) works by retrieving relevant passages from external documents before generating an answer — rather than relying purely on model weights. Put simply: RAG = Search + LLM Q&A.

A standard RAG pipeline: document loading → parsing → chunking → embedding → storage, then at query time: embed the question → retrieve top-K chunks → pass as context to the LLM.

V1: Naive vector retrieval

Document preprocessing

V1 called the Lark server API to pull plain text, flattened the entire document into one string, then applied fixed-size chunking: 500 characters per chunk with 20% (100-character) overlap between adjacent chunks.

Vector search

We used the open-source bge-large-zh-v1.5 model to embed chunks, stored vectors in PostgreSQL via the pgvector extension with an HNSW index, and measured similarity with cosine distance.

Problems that emerged

Preprocessing failures: Plain-text extraction discarded heading hierarchy and structural context. Fixed-size chunking split semantic units mid-sentence, directly degrading embedding quality.

Vector retrieval failures: Embeddings capture semantic similarity but struggle with exact strings. Queries like "8XLARGE64" or "99.9%" returned completely irrelevant chunks — a single vector can't faithfully represent every precise detail in a passage.

Vector search is excellent at semantic similarity and poor at exact keyword lookup — but in enterprise knowledge bases, exact lookup is the dominant query type.

V2: Markdown parsing + hybrid retrieval + reranking

Better preprocessing: plain text → Markdown

We switched from flat text extraction to Markdown-format export. Markdown preserves heading levels, lists, and code blocks, enabling semantic unit-based chunking (sections, paragraphs) rather than raw character counts.

Additional improvements: each chunk inherits its full ancestor heading path for context completeness even in deeply nested wikis; Markdown markup is preserved so the frontend can render retrieved chunks as rich text.

This single change had the largest measurable impact on retrieval quality.

Hybrid retrieval: vector + Chinese full-text search

Dimension	Vector search	Full-text search
Strengths	Semantic similarity, synonyms, cross-lingual	Exact keywords, IDs, codes
Weaknesses	Exact character matching	Semantic understanding, paraphrases

Our data lives in MongoDB. MongoDB's built-in full-text search tokenises on whitespace — useless for Chinese. We integrated jieba (via nodejieba) for Chinese word segmentation on both documents and queries, then queried via MongoDB's $text index ranked by $meta: textScore.

const nodejieba = require('nodejieba')
nodejieba.cut('vector database technology')
// → ['vector', 'database', 'technology']

jieba's search-engine mode gave the best recall in practice across all three tokenisation modes.

Reranking

Both retrieval paths return candidates. To merge and re-order them, we added a cross-encoder reranker: it scores each candidate against the query independently (much higher precision than bi-encoder), then returns results sorted by score. Our primary endpoint was an internal reranker; the algorithm team's ranking interface served as failover.

Other experiments

• Embedding model comparison: bge-large-zh-v1.5 vs m3e-large — negligible difference on our data distribution.
• RRF fusion: Reciprocal Rank Fusion to merge the two retrieval streams — didn't improve results on our dataset.
• QA pair decomposition: Pre-splitting documents into Q&A pairs for indexing — limited gains.

Lessons learned

1. Preprocessing quality sets the ceiling. Chunking strategy outweighs model selection by a wide margin. Structural integrity and semantic completeness come first.
2. Naive vector retrieval isn't enough. Exact-match queries dominate enterprise knowledge bases. Hybrid retrieval is a production requirement, not a nice-to-have.
3. Build your evaluation dataset before optimising. Without quantified metrics, you can't know whether a change helped. A golden Q&A dataset should be the first pipeline module you build.

RAG optimisation is a continuous process. Future directions worth exploring: context window management, multimodal retrieval (text + images), and domain-specific embedding fine-tuning — all with substantial headroom.