Meta Unveils New AI Models: Llama 4 Series

Meta has introduced a suite of new artificial intelligence models under the Llama 4 branding, showcasing advancements in AI capabilities. The launch features two primary models, Llama 4 Scout and Llama 4 Maverick, both of which are now available as open models. Additionally, a larger model named Behemoth is expected to be introduced shortly, along with a specialized reasoning model.

Mark Zuckerberg, the CEO of Meta, highlighted these developments in a recent video on Instagram, reiterating his belief that open-source AI models will become increasingly important and beneficial for a wider audience. The Llama 4 models are integrated into Meta AI and are designed to provide significant upgrades to existing AI assistants.

Llama 4 Scout and Llama 4 Maverick are described as natively multimodal models that utilize a Mixture-of-Experts architecture and feature open weights. This multimodal approach allows the models to process both text and images simultaneously. Each model comprises various expert components that activate based on the specific type of inquiry, enhancing efficiency and performance.

The Scout model operates with 16 experts, each featuring 17 billion parameters, resulting in a total of 109 billion parameters. Its context length accommodates up to 10 million tokens, which Meta claims is a leading capability in the field. Conversely, the Maverick model utilizes 128 experts, also with 17 billion parameters each, totaling 400 billion parameters, and supports a context length of one million tokens. Scout is designed to function on a single H100 graphics processor, while Maverick requires an H100 host.

Behemoth, the model yet to be released, is anticipated to be an extensive training model with 288 billion parameters distributed across 16 experts. Zuckerberg describes Behemoth as an ideal trainer for model distillation, a process that transfers knowledge from a large model to a smaller, more cost-effective version. Both Scout and Maverick will benefit from this knowledge transfer. Meta aims for Behemoth to surpass current benchmarks set by other advanced models, including GPT-4.5 and Claude Sonnet 3.7, though such claims should be approached with caution due to the inherent variability in model training and performance evaluation.

Improvements in pre-training and post-training methods for Behemoth are also noteworthy. Meta has enhanced the quality of synthetic data used in training, introduced a new vision encoder, and refined the fine-tuning and reinforcement learning processes to focus on challenging tasks. As a result, Llama 4 Maverick is expected to be more cost-efficient than its predecessor, Llama 3.3 70B. The models are also equipped to handle multiple images simultaneously, which significantly enhances their visual reasoning capabilities, including understanding temporal sequences.

In a blog post, Meta assured that rigorous safety standards were upheld during the development of these models. Addressing potential biases, the company acknowledged that all leading large language models (LLMs) face challenges related to bias, particularly in politically charged discussions. These biases often arise from the nature of training data sourced from the internet. Meta claims to have implemented measures to achieve more balanced responses with Llama 4, noting improvements over Llama 3 and comparable performance to other models.