Streamlining content ops with LLMs: Wagtail user guide
A case study in LLM-powered content operations
After three months contributing to the Wagtail user guide, here is what we achieved and my thoughts on getting involved with open source.
Streamlined content operations and translations
The guide website had serious issues: broken versioning that cost editors hours of work per release, and translations that had not progressed since launch. The codebase wasn't complex, and there were a lot of opportunities to thoughtfully adopt AI models to streamline the work while maintaining or increasing quality. Here is what we achieved:
- 5x faster monthly content operations thanks to a big content versioning architecture overhaul
- 52 languages translations in sight, with an actually-viable path, via fully automated draft AI translations
- Flawless RTL ("Right-to-Left") languages support - for Arabic, Farsi, Hebrew, and Urdu.
- Vector indexing and related pages: to facilitate cross-linking across hundreds of pages.
- Project tooling improvements: adoption of AGENTS.md and AI code reviews.
You can view all of my work in GitHub for the duration of the project, here is how it happened in more detail.
Versioning
The biggest piece of work was replacing the versioning system. The old one required site editors to spend hours copying the entire site content. With a new Wagtail release every three months, that didn't scale. And because every release published entirely new pages, translations had to be copied over and re-edited for each language; it was too much work to keep up with, so many pages never got translated. The state before the project, with a very overwhelming picker:
After prototyping, we replaced this with special blocks carrying a version number and a "type-of-change" setting (added/changed/removed), so each update only touches the blocks tied to it, and readers switch between versions with a UI filter. This kind of block configuration is dramatically simpler with Wagtail’s block settings / StructBlock layout feature, added in version 7.3. Here is what it looks like on the site’s pages:
Translations
Testing RTL and translations properly ran into an unexpected blocker: the build fixtures had limited content, so I needed something closer to the deployed database. As a fast workaround, I pulled a copy of the deployed database and ran it locally by adding Docker Compose to the project, which also gave me a good excuse to widen my database knowledge. My mentor Thibaud recommended DBeaver, which I used to look at the actual schema, and I picked up commands and topics like EXPLAIN ANALYZE that I hadn't worked with before.
AI translations
The guide website was published in 2022, before LLMs went mainstream, so translating it into a new language required a lot of manual work. Which is why almost none of the languages were fully translated. Automated translation was the solution.
The next question was which engine to use. We initially planned to work with DeepL, but it did not seem viable for the scale of the site. Although it’s currently small, we planned to extend the content to 100+ pages across 10+ languages: a one-time bulk run came to roughly $1000, so we started looking at other options. That search turned into the richest part of my AI integration learning. I started with wagtail-localize-ai, but reviewing the Arabic translations I noticed output problems that needed prompt refining. Since the package doesn't allow changing its system prompt, I forked the repo to test different versions directly. It took a couple of iterations until I was satisfied. Since we couldn't depend on my fork, we moved to adding a machine translator to the repo itself, with a more advanced system prompt that reflects the needs of the site:
default_system_prompt = """\ You are a professional translator translating text from {source_language} to {target_language}. Translate only the text and keep its structure intact. Reply with just the translated text and no wrapper, explanation, or code fence of any kind. - Only standard HTML inline tags are allowed: a, abbr, acronym, b, code, em, i, strong, br. - <a> tags may keep only their id attribute; other tags must have no attributes. - Preserve any inline tags, whitespace, and punctuation exactly.""" default_glossary_prompt = """\ - Text inside <b> or <i> tags is usually a label from the Wagtail admin interface. When a term appears in the glossary below, use the official translation exactly as given; otherwise translate it naturally and consistently. Glossary (official Wagtail admin translations): {glossary}"""
With several translation models to choose from, picking the right one needed more than a quick read-through. I started with a Python script that calculates page cost from the translation logs wagtail-localize-ai already tracks:
But cost alone couldn't tell me which model actually translated well, so I built a full eval suite to benchmark quality too, in three stages:
- Translation: translate real content segments through each candidate model via API.
- Deterministic checks: no reasoning output, preserves tags
- LLM-as-judge: scores each candidate A/B/C pairwise on accuracy/fluency/rules (0–5) and assigns ranks.
We focused on open weight models like other Wagtail projects. DeepSeek V4 Flash came out on top with better quality. Gemma 4 was very competitive too but had trouble with some of the deterministic checks.
Right-To-Left languages support
RTL support was messy, and as an Arabic speaker, I know the frustration of finding a translated site with broken RTL or no RTL support at all. Working on this was one of the most fun parts of my work. I went in thinking it was just about reading content from the other side and it turned out almost touching every layer of theme: icons, handling the mixing of English words within Arabic text, and accounting for typographic differences between English and Arabic text, like line height and line breaks. Here is what the site now looks like in RTL and LTR:
The changes I made fell into four groups: making icons and text read their values from the page's direction; using dir="auto" so the browser picks the direction per string, which allows English strings on RTL pages (necessary in tech documentation); flipping directional icons like the prev/next arrows with a shared CSS variable that mirrors based on direction instead of maintaining separate SVGs; and RTL-aware positioning for the logo and menu button, so the layout didn't just mirror text while leaving buttons stuck on the wrong side.
Vector indexing and related pages
During the community bonding period, I attended the 7.4 release "What's New in Wagtail" session, where I was first introduced to vector indexing, where content is stored in vector databases to allow semantic comparisons between pages. The mechanism caught my attention. Three months later, I was using it in my own project. I built a "Related Pages" section that lets editors choose up to five related pages, shown as a card grid on public pages. With the goal of scaling the website up, editors picking pages by hand could become error-prone, so I added AI-suggested options on top, backed by a pgvector index of the site's content. This is based on the Wagtail AI package, with customizations for better multilingual content support.
How we worked
I didn’t know Wagtail before taking part in Google Summer of Code, and was drawn by the scope of the project and contributor experience. Maintainers are supportive and provide excellent documentation. There are guidelines around using AI, that helped prevent the messy slop PRs. I thought this would be a good starting point for a career in web development, discussed the options with mentors and made proposals for both the guide website and Wagtail’s official demo.
In our first meeting, my mentor Thibaud and I decided we'd use it as a space to introduce me to as many different areas as reasonably fit, rather than spending the whole summer going deep on just one, and each week meant picking up something I hadn't touched before, CSP one week and vector search the next.
We started with the developer experience. The project lacked basic tooling: no Docker Compose, no AGENTS.md, and Django wasn't on its latest version. Following Wagtail's push toward automation and agent readiness, I fixed all three, including migrating and testing the new built-in CSP support that shipped with that release (#444).
Planning and documentation were a big part of what kept the project on track. We worked in an agile structure, split into six two-week sprints, tracked through a GitHub Project. It helped me in tracking issues and PRs, deciding what to prioritize, and keeping a record of ideas that weren't ready to become issues yet.
If I had to pick one moment from the summer that stood out, it'd be my midterm evaluation. Before it, I'd been worried about my role in the project's direction, whether I was actually contributing the way I should be. Reading my mentor's feedback made that worry disappear.
“It's a pleasure to work with you! I'm glad we're making this project happen. I particularly appreciate your openness and ability to reflect in our discussions. Clear communication is essential to our work and you have done excellently on that, as well as quality of your contributions. It feels like you "get it" and found your way in our project very naturally.”
What's next
With the major goals of the project completed (better scale, better translations, better UX), we can look at writing new content for the site and delivering translations across all of Wagtail's supported languages. There's also room to revisit some of the site's design choices now that the underlying scale has changed.
Personally, having been through GSoC, I learned a lot about backend development and came out with real full-stack experience. I am continuing to study operating systems and databases, and I'll be looking for backend/full-stack internships and other open-source programs to build up my professional portfolio.
Besides this, I plan to stay involved with the guide website beyond the program; there are UX improvements and more AI automation I'd like to keep pushing on. I'm also hoping to talk about this project at Wagtail Space 2026 in November.
It was fun and valuable spending the summer working on this project with my mentor, Thibaud, who helped me a lot throughout and was always keen to share his expertise and make the project as fun as it was valuable.