Retail Content Moderation at uniopen

uniopen serves as a digital communication and membership platform developed by Taiwan’s Uni-President Enterprises Group, linking customers to ecommerce, membership perks, and varied retail experiences spanning web, tablet, and mobile environments. Within these channels, the platform enforces a specific moderation policy designed to evaluate interactions across two distinct axes: the underlying behavior, divided into nine specific categories, and the subject of that behavior, classified as brand, other, or forbidden. For a moderation action to prove useful, both axes require accurate classification, a domain-specific requirement that general-purpose models cannot inherently master right out of the box.

To address these custom needs, the engineering team adapted Amazon Nova 2 Lite to align precisely with business rules. The entire approach utilized supervised fine-tuning via Amazon SageMaker AI service page alongside prompt-level output optimization to keep correction data, training management, evaluations, and deployment controls inside a single, repeatable workflow.

Architecture and Human-in-the-Loop Feedback

The underlying architecture cleanly divides the production moderation path from correction handling, model training, evaluation, and eventual deployment. While Amazon Nova 2 Lite handles primary production requests, Amazon Nova 2 Pro assists by generating candidate corrections for any reported errors. Crucially, a human reviewer must explicitly verify each generated correction before it is allowed to enter the training dataset, ensuring that generated labels are never blindly trusted as ground truth.

Approved corrections and training inputs find a home in Amazon Simple Storage Service ( Amazon S3 ), while Amazon DynamoDB tracks both active and candidate model configurations. Operational coordination relies on Argo Workflows running on Amazon Elastic Kubernetes Service (Amazon EKS) to orchestrate evaluations, prompt optimizations, and deployments, while Argo CD applies approved configuration states directly into the production environment.

In addition, Amazon Simple Notification Service (Amazon SNS) and Amazon CloudWatch provide necessary alerts whenever hard release gates fail or when candidate models require closer administrative inspection.

Gate Controls and Responsible AI Measures

Promotion of model updates is carefully managed via two tiers of checks: hard gates and soft gates. Hard gates operate as mandatory regression tests; any failure here immediately halts the deployment pipeline and triggers operational notifications. Soft gates act as warning indicators, identifying scenarios like low confidence classifications or performance dips within specific evaluation classes.

Candidates clearing the hard-gate criteria without triggering soft-gate warnings may proceed to automated promotion. However, if a soft gate registers a warning, the candidate remains in a pending state until an administrator reviews and approves the deployment manually. Additional responsible AI safeguards complement these gates, including Amazon Bedrock Guardrails content filters applied to both model inputs and outputs, error monitoring across individual behavior and subject categories, and strict minimization of retained customer data.

Performance Gains via Supervised Fine-Tuning

To gauge improvements, the team evaluated three separate configurations against a consistent held-out test set comprising 737 conversation windows, utilizing a fine-tuning dataset built from 3,391 training windows. Metrics tracked included Per Behavior Macro F1, which tests classification consistency across all nine moderation behaviors evenly, and Subject Type Macro F1, which measures uniform accuracy across brand, other, and forbidden subject types.

Without customization, the baseline Amazon Nova 2 Lite model achieved a Per Behavior Macro F1 score of 0.5852 alongside a Subject Type Macro F1 of 0.4162, highlighting a gap in handling uniopen's proprietary policy taxonomy natively. Applying supervised fine-tuning via Amazon SageMaker AI delivered substantial performance enhancements. Following customization, the Per Behavior Macro F1 score surged to 0.8364, while the Subject Type Macro F1 jumped to 0.8302, successfully exceeding production targets for subject identification and moving behavior classification close to its final benchmark.

Conclusion and Continued Optimization

By combining Low-Rank Adaptation techniques within Amazon SageMaker AI with automated workflow orchestration and structured human review loops, uniopen successfully tailored Amazon Nova 2 Lite for production-grade retail moderation. Further details regarding specific foundational model architectures and regional availability can be explored via resources like the AWS Machine Learning Blog or documentation on supported models by AWS Region in Amazon Bedrock.

Sources

• AWS Machine Learning BlogHow uniopen customized Amazon Nova to their retail moderation policies for production deployment