Yuan launched the next generation AI models Llama 3 8B and 70B on Thursday. Llama 3, short for Large Language Model Meta-AI, offers improvements over its predecessor. The company has also adopted new training methods to optimize the model’s efficiency. Interestingly, Lama 2’s largest model was 70B, but this time the company says its larger model will have more than 400 billion parameters. Notably, a report last week suggested that Meta will launch a smaller AI model in April and a larger model later in the summer.
Those interested in trying out the new AI model are in luck, as Meta is taking a community-first approach with Llama 3. The new base model will be open source like the previous model. Yuan Zhai Qi blog post”, “Llama 3 model will soon be available on AWS, Databricks, Google Cloud, Hugging Face, Kaggle, IBM WatsonX, Microsoft Azure, NVIDIA NIM and Snowflake, AMD , supported by hardware platforms from AWS, Dell, Intel, Nvidia and Qualcomm. “
The list includes all major cloud, hosting and hardware platforms, making it easy for hobbyists to master AI models. In addition, Meta has integrated Llama 3 with its own Meta AI, which can be accessed via Facebook Messenger, Instagram and WhatsApp in supported countries.
During the show, social media giant Llama shared benchmark scores for 3 pre-trained and instructed models. For reference, pre-trained is a simple conversational AI, while the guidance model is designed to accomplish a specific task. The Llama 3 70B pre-trained model scores higher than Google 1.0 Pro Performance on the Gemini MMLU (79.5 vs. 71.8), Big-Bench Hard (81.3 vs. 75.0) and DROP (79.7 vs. 74.1) benchmarks, while the 70B Instruct model outperforms Gemini on MMLU, HumanEval and GSM. -Outperforms the 8K based model. On information shared by the company.
Meta opted for a decoder-only transformer architecture for the new AI model, but made some improvements over its predecessor. Llama 3 now uses a tokenizer with a vocabulary of 128K tokens, and the company has adopted Grouped Query Attention (GQA) to improve inference efficiency. GQA helps improve the AI’s focus so that it doesn’t stray from the specific context when answering questions. The social media giant has pre-trained the model using more than 15T tokens, which it claims are derived from public data.
Affiliate links may be generated automatically – see our ethics statement to learn more.
