1.
우연히 일본 자료를 읽다가 재미있는 주제의 책을 찾았습니다.
어떤 회사인지 찾아보니까 Fixstars AIBooster라는 서비스를 제공하고 있더군요.
책을 읽어보지 않아서 어떤 내용인지 궁금합니다. 그래서 홈페이지에 올라온 White Paper를 받았습니다.
NVIDIA H200: AI Acceleration and Performance Engineering in Practice
2.
제목은 조금 다르지만 또다른 영어책이 있네요. 2025년 12월에 출간한 책입니다.
AI Systems Performance Engineering
대략 천쪽이 넘는 책입니다. 저자는 책과 관련한 github 레포지토리도 운영하고 있습니다.
솔직히 책 내용이 궁금합니다. 왜 이런 책을 썼을까 하는 이유가 있을 듯 했습니다. DeepSeek의 기술력이 배경이 아닐까 하는 예상을 했습니다. 원문을 제공하는 홈페이지를 보니까 1장이 Introduction and AI System Overview입니다. 여기서 Deepseek를 다루고 있습니다.
In late 2024, a small startup in China called DeepSeek.AI stunned the AI community by training a frontier large language model (LLM) without access to the latest, state-of-the-art NVIDIA GPUs at the time. Due to export restrictions, DeepSeek’s engineers could not obtain top-tier NVIDIA Blackwell (B200, B300, etc.) or Hopper (H100, H200, etc.) GPUs, so they resorted to locally available, export-compliant alternatives at the time,including the NVIDIA H800 GPU. They used custom kernels and advanced optimization techniques such as model distillation to squeeze out maximum performance from these less capable GPUs.
Despite these limitations, DeepSeek.AI trained their DeepSeek-R1 model and achieved reasoning capabilities near the performance of leading frontier models that were trained on the most capable NVIDIA chips at the time.
This case underscores that practitioners and researchers skilled in AI systems performance engineering can get the most out of their available hardware—no matter the constraints.
For example, DeepSeek’s engineers treated communication bandwidth as a scarce resource, optimizing every byte over the wire to achieve what many thought impossible on that infrastructure. They scaled out to thousands of these constrained GPUs—connected with limited-bandwidth interconnects—using novel software and algorithmic optimizations to overcome these limitations.
Contrast DeepSeek’s approach with the “brute force” path taken by the largest AI frontier labs in the United States and Europe. These labs continue to pursue larger compute clusters and larger models. Model sizes have exploded from millions to billions and now to trillions of parameters. And while each 10× increase in scale has unlocked qualitatively new capabilities, they require tremendous cost and resources.For instance, training OpenAI’s GPT-4 (2023) reportedly cost an estimated ~$100 million, while training Google’s Gemini Ultra (late 2023) is estimated at a staggering ~$191 million. This demonstrates the need for resource efficiency going forward as these models scale up in size and cost. DeepSeek claims that their DeepSeek-R1 model was trained for less than $6 million in compute—an order of magnitude lower than models like GPT-4 and Gemini Ultra. At the same time, DeepSeek-R1 matches the performance of rival models that cost orders of magnitude more money.
And while there is some doubt as to the validity of the $6 million claim—and what exactly it includes (e.g., just a single training run) or excludes (e.g., experimentation and the model-development pipeline)—the announcement briefly shocked the US financial markets, including NVIDIA’s stock, which dropped ~17% in a single day based on this news.This was caused by concerns that DeepSeek’s efficiency innovations would somehow require less NVIDIA hardware in the future. While this market reaction was a bit overblown—and NVIDIA stock recovered in subsequent trading sessions—it demonstrates the significant financial impact that breakthroughs in AI efficiency can have on global financial markets.Beyond model training, DeepSeek boasts significant inference efficiency gains through novel hardware-aware algorithmic improvements to the transformer architecture that powers most modern, frontier LLMs.
DeepSeek has clearly demonstrated that clever AI systems performance engineering optimizations can upend the economics of ultrascale AI model training and inference. These optimizations are covered throughout the rest of the book.The takeaway is a profound realization that, at these scales, every bit of performance squeezed out of our systems could translate to millions, or even billions, of dollars saved. Every bottleneck eliminated can have an outsized impact on training throughput and inference latency. This, in turn, reduces cost and increases overall end-user happiness. In short, AI systems performance engineering isn’t just about speed—it’s about making the previously impossible both possible and affordable.
위 본문은 아래에서 가져왔습니다. 중국어와 영어를 병행해서 제공하고 있습니다. 또한 구글링을 잘 하면 pdf를 구할 수도 있습니다.
AI Systems Performance Engineering
하루중 가장 많은 시간을 투자하는 영역이 Low Latency 혹은 Performance이기 때문에 저도 관심이 많은 주제입니다. 짬을 내려고 합니다.
