Nvidia GB 72NVL, RTX 5090 and Digits

Trelis News

A new series for the start of 2025 - Trelis News!

Cheers, Ronan

NVIDIA's CES 2024 Hardware Announcements

Grace Blackwell NVLink 72 Specifications

The Grace Blackwell NVLink 72 system comprises:

• - 72 Blackwell B200 GPUs (144 dies)

• - 576 memory chips totaling 14TB memory

• - 1.4 exa-flops computing capability

Each B200 GPU features:

• - 18 TOPS (tera operations per second) in 4-bit precision

• - 180GB memory per GPU

• - 8TB/s memory bandwidth

Compared to the previous Hopper generation:

• - 5x increase in TOPS

• - Modest increase in per-GPU memory

• - 2x increase in memory bandwidth

• - New support for 4-bit operations (vs 8-bit minimum previously)

Current Model Context

The system's 14TB memory capacity exceeds current model requirements:

• - Latest LLaMA model: 405 billion parameters

• - No current models utilize full 4-bit precision capability

• - Only DeepSeek model currently trains in 8-bit

• - Most open source models still use 16-bit precision

RTX 50 Series Gaming GPUs

The new RTX 5090 specifications:

• - 92 billion transistors

• - 4,000 TOPS AI performance

• - 1.8 TB/s memory bandwidth (G7 memory from Micron)

Compared to RTX 4090:

• - 3x higher AI TOPS (largely due to 4-bit vs 8-bit operations)

• - 70% higher memory bandwidth (1.7 TB/s vs 1.0 TB/s)

• - Similar pricing ($1,999 vs $1,499 MSRP, although on Amazon the 4090 is closer to $2,000)

Project Digits Home AI Computer

Specifications:

• - 4TB storage

• - 128GB VRAM

• - 1 petaflop FP4 AI compute

• - Estimated $3,000 price point

• - Capable of running 70B parameter models

• - Memory bandwidth specifications still unclear (estimates range from 256GB/s to 825GB/s)

• - Uses LPDDR5X memory modules

The system aims to enable home users to run large language models like LLaMA 70B, which typically require significant VRAM capacity unavailable in consumer hardware.