Meta Releases Muse Spark Open Multimodal MoE Model
Meta has expanded its AI lineup with the release of Muse Spark Open, an advanced open-weight frontier multimodal mixture-of-experts model designed for complex agentic workflows.
Radwan AbdulhadiEditor and technology writer
Architecture and Scale
Meta has officially launched its latest open-weight model, continuing its efforts in open science and artificial intelligence development. Detailed on the <a href="https://huggingface.co/blog/meta-muse-spark-open">Hugging Face Blog</a>, the new architecture handles multiple input modalities including text, images, audio, and video within a unified framework. Further details are available from Hugging Face Blog in the original source material.
The model contains approximately 816 billion parameters in total, utilizing a mixture-of-experts design where about 42 billion parameters are active per token. This configuration allows deployment to leverage the memory capacity of an 816-billion-parameter system while running at the speed of a smaller 42-billion-parameter model.
Optimized for Agentic Coding
According to Meta, Muse Spark Open is heavily optimized for agentic coding workflows. It is trained to read codebases, write and debug code, call tools, and handle complex developer tasks like multi-file refactors and extended debugging sessions.
Because the model accepts images, audio, and video natively, developers can use it to read screenshots, listen to voice notes, or examine screen recordings while debugging code or navigating software projects.
Model Checkpoints and Deployment
Developers can access the released weights through specific checkpoint repositories. The available options include the BF16 checkpoint—which spans approximately 1.5 terabytes in safetensors format—and an NVFP4 checkpoint designed to store parameters in 4-bit precision for NVIDIA Blackwell GPUs with minimal accuracy loss.
Further technical discussions and community engagement around the release can be followed by exploring profiles such as <a href="https://huggingface.co/pcuenq">Pedro Cuenca pcuenq Follow</a> on the platform, alongside other contributors working with the ecosystem.
For deployment, the BF16 checkpoint typically runs across four nodes featuring eight H200 GPUs using tensor parallelism, while the NVFP4 version is structured to operate on a single B200 node.
Multimodal Input Processing
The framework utilizes specialized encoders to translate non-text inputs into a format the language model can interpret. A dedicated perception encoder processes images and video frames into patch tokens, while an audio encoder processes audio input through a convolutional pyramid structure.
These modalities all feed into an extensive context window that supports up to one million tokens at inference time. Grouped-query attention mechanisms help maintain a manageable key-value cache size despite the substantial scale of the overall context window.
Community Resources and Integration
Developers looking to integrate or fine-tune the model can find comprehensive guides and code snippets provided across the open-source community. Additional code updates and documentation can also be tracked via the repository <a href="https://github.com/huggingface/blog/blob/main/meta-muse-spark-open.md">Update on GitHub</a> for those implementing custom pipelines.
With built-in reasoning controls and processors designed to handle multi-image prompts and synchronized audio-video streams, Meta's new release offers a robust foundation for advanced multimodal research and enterprise application deployment.
Sources
- Hugging Face BlogWelcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta
Continue chronologically
Related entity coverage
Related stories
Meta Releases Muse Spark Open, an 800B-Parameter AI Model
Meta has introduced Muse Spark Open, an 800-billion parameter open-weight AI model optimized for Apple's hardware, requiring 1TB of unified RAM.
Google Joins Amazon in Challenge to Nvidia's AI Lead
Google has released its open-source TorchTPU tool, allowing PyTorch developers to easily move their applications off Nvidia GPUs and onto Google Cloud.
Meta Unveils Next-Gen Andromeda Ads Retrieval Engine
Meta details its next-generation Andromeda personalized ads retrieval engine, achieving billions of ad scores per second through model-system co-design.
Meta Unveils Prometheus Gigawatt-Scale AI Supercluster
Meta completes the first phase of its Prometheus gigawatt-scale supercluster in Ohio, powering frontier models with hundreds of thousands of GPUs.
