LectuLibre Builds Sub-500ms In-Book Word Translation Using Glossary and LLM Caching

LectuLibre, a platform that translates entire books using large language models, developed a feature called instant translation help, allowing readers to click any word in a translated text and receive an immediate contextual explanation. The core challenge was speed: full LLM calls averaged 3–5 seconds of latency, making them unsuitable for a seamless reading experience. The team solved this by preprocessing each translated book with a background Celery task to build a bilingual glossary of key noun phrases, using spaCy for phrase extraction and SentenceTransformers for cross-language alignment. At query time, the system first checks the local PostgreSQL glossary for a match and returns it instantly; if no match is found, it falls back to DeepSeek and caches the result for future lookups. This hybrid approach keeps most responses under 500ms while controlling costs by minimizing live LLM calls.