Author(s): Towards AI Editorial Team Originally published on Towards AI. Microsoft’s Aurora, Codestral, MoRA, XAi raise & more. What happened this week in AI by Louie While there was plenty of newsflow in the LLM world again this week, we are also interested in how the LLM-fueled boom in AI research and AI compute capacity can accelerate other AI models. For several months, we have been seeing a lot of progress in machine learning-based weather models, and this week, Microsoft released Aurora — a 1.3 billion parameter foundation model for the atmosphere. This model can be fine-tuned using techniques familiar to the LLM world, such as Low-Rank Adaptation (LoRA) for specific forecasting scenarios. The model was trained on over a million hours of weather and climate simulations, and Microsoft estimates a 5,000x computational speed-up vs. the state-of-the-art numerical forecasting system “Integrated Forecasting System.” We have been watching many other exciting developments in AI weather models in the past six months — including this week from Windborne, who is also building a huge constellation of small-scale weather balloons to gather more data than is currently available. In March, Nvidia announced its Earth Climate Digital Twin cloud platform. Its Earth-2’s APIs include a new generative AI model called CorrDiff — a diffusion model that generates 12.5x higher resolution images than current numerical models 1,000x faster. Huang discussed aiming for prediction at 3-kilometer weather resolution. Google Deepmind is also working on AI weather models, including graphcast based on graph neural networks last year. Why should you care? The weather forecast may be taken for granted, but better weather prediction can still hugely impact people’s lives and the economy. More accurate hurricane prediction can make all the difference in saving lives, better longer-term weather forecasting can boost crop production, and better local forecasting can help you plan your day out! We are pleased that AI computes rollout and progress on models and techniques from the Generative AI boom (such as diffusion models and LoRA finetuning) are beginning to make a difference to weather forecasting and, more broadly, leading to progress and new foundation models across different industries. — Louie Peters — Towards AI Co-founder and CEO This issue is brought to you thanks to Ai4: Ai4, the world’s largest gathering of artificial intelligence leaders in business, is coming to Las Vegas — August 12–14, 2024. Join 4500+ attendees, 350+ speakers, and 150+ AI exhibitors from 75+ countries at the epicenter of AI innovation. Don’t wait — prices increase on May 31st. Apply today for a complimentary pass. Register now for 41% off final prices! Hottest News 1. Gemini 1.5 Pro and Advanced Ranks Second on the LMSYS Leaderboard The latest LMSYS leaderboard shows that Gemini 1.5 Pro/Advanced ranks second, right behind GPT-4o, while Gemini 1.5 Flash holds the ninth position, surpassing Llama-3–70b and closely competing with GPT-4–0120. 2. Mistral Introduces Codestral, an Open-Weight Code Model Codestral is Mistral AI’s new generative AI model, focused on coding. It has a large 32k context window and boasts proficiency in over 80 programming languages. It helps developers write and interact with code through a shared instruction and completion API endpoint. 3. xAI Raised a $6 Billion Series B Funding Round xAI has raised $6 billion in a Series B round to expand AI tech deployment, including their Grok-1 series, and to innovate new products, building on a year of significant AI advancements and the Grok-1 open-source release. 4. China Invests $47 Billion in Largest Ever Chip Fund China allocated $47.48 billion to a new chip fund to advance domestic semiconductor production, a critical step toward self-sufficiency and competitiveness in technology sectors, including AI. 5. ElevenLabs Moves Beyond Speech With AI-Generated Sound Effects ElevenLabs introduced their newest AI Audio model, which can generate sound effects. It can generate short instrumental tracks, soundscapes, and character voices, all from a text prompt. It is now available to all users. Five 5-minute reads/videos to keep you learning 1. Training and Finetuning Embedding Models with Sentence Transformers v3 The article discusses the release of Sentence Transformers v3.0, highlighting enhanced capabilities for training and finetuning embedding models to boost task-specific performance, and showcases the updated components, including datasets, loss functions, evaluators, and an improved trainer. 2. Accelerating Transformers with NVIDIA cuDNN 9 The NVIDIA cuDNN is a GPU-accelerated library for accelerating deep learning primitives with state-of-the-art performance. This article details the achievable performance of cuDNN SDPA, walks through how to use it, and briefly summarizes some other notable new features of cuDNN 9. 3. Exploring Linear Regression for Spatial Analysis Linear regression provides insightful information about spatial relationships, patterns, and trends and is a flexible and essential tool in Geographic Information Systems (GIS). This article explains linear regression in the context of spatial analysis and shows a practical example of its use in GIS. 4. LLMs Aren’t Just “Trained On the Internet” Anymore This essay examines LLMs’ capabilities, particularly the extent to which future improvement is expected. It primarily discusses the shift in trends regarding training data and how using custom data in LLM training can lead to better output. 5. LlamaIndex Launches a Framework for Building Knowledge Graphs with LLMs LlamaIndex recently launched Property Graphs. They can categorize nodes and relationships into types with associated metadata, treat your graph as a superset of a vector database for hybrid search, and express complex queries using the Cypher graph query language. This article introduces its capabilities in more detail. Repositories & Tools Tarsier provides webpage perception for a minimalistic GPT-4 LangChain web agent. FlashRAG is a Python toolkit for the reproduction and development of RAG research. LaVague is an open-source large action model framework for developing AI web agents. Llmware is a unified framework for building enterprise RAG pipelines with small, specialized models. The llama3-from-scratch repository is an implementation of llama3, one matrix multiplication at a time. Top Papers of The Week 1. Transformers Can Do Arithmetic with the Right Embeddings The paper highlights that the addition of positional encodings to transformer models […]