Introduction to Mobile Summarization

Following the rollout of Firefox’s Shake to Summarize feature to Android devices this May, Mozilla took a closer look at the modeling work behind the feature. In a previous blog post, the team discussed its model selection process. Now that Shake to Summarize has been available on both iOS and Android for a few months, developers outlined their approach to prompt development, highlighting a specific use case that required extra prompt tuning: online recipes.

Developing a useful prompt begins with clearly describing what the Large Language Model should do. Because LLMs thrive on specificity, defining tasks sharply yields better results. This principle is especially critical when deploying smaller LLMs, which lack the advanced capability of larger models to read between the lines and intuit unstated intentions.

Categorizing Web Content for LLMs

Summarization tasks initially appear straightforward, but iteration reveals that what constitutes a good summary depends heavily on the content type. For instance, a novel summary requires a quick plot overview without verbatim text or deep specific details. Conversely, a recipe summary must include the recipe essentially as written, retaining precise measurements like four cups of vegetable broth or specific spices rather than vague descriptions.

To address these varying needs, the product team compiled a targeted list of article types, assigning a description of an ideal summary for each category. Recipes require ingredients, key steps, time required, and author tips. News summaries focus on important details such as events, timelines, and consequences. How-to guides start with required materials, tools, and warnings, while reviews highlight ratings, pros, cons, pricing, and target audiences. Research summaries target key findings, confidence levels, and real-world impact, and opinion pieces capture main arguments and supporting evidence.

Refining Prompts and Addressing Recipe Shortfalls

Mozilla wrapped these categories into general instructions to ground the model. Developers then foxfooded the initial prompt and found it performed well overall, producing concise, accurate, and informative summaries for many article types.

However, when applied to recipes, the model tended to over-summarize, often omitting key ingredients or the recipe entirely. For example, when testing a lentil soup recipe, the model generated a descriptive overview of the soup and its ingredients but left out the actual cooking instructions, failing to serve as a quick way to access the recipe without reading the narrative preamble.

Implementing Routing and Specialized Prompts

To solve the recipe summarization issue without causing the model to over-index on recipe formatting for other article types, developers created a separate prompt dedicated solely to recipes. They used structured data embedded in each webpage to route requests to the correct prompt. Relying on existing webpage metadata allowed for quick, deterministic categorization without adding inference overhead.

After deploying this change and running tests across a curated set of recipe websites, Mozilla found that the new system was more than twice as likely to return complete and accurate summaries compared to the previous iteration. The experience demonstrated that models yield the best results when given explicit instructions, and that treating summarization as a routing problem with category-specific prompts successfully improves output quality.

Sources

• Mozilla BlogUnder the Hood: Prompt tuning Shake to Summarize for recipes