Nvidia says it can shrink LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory ...
The Next Platform

Nvidia Pushes Hopper HBM Memory, And That Lifts GPU Performance

For very sound technical and economic reasons, processors of all kinds have been overprovisioned on compute and underprovisioned on memory bandwidth – and sometimes memory capacity depending on the ...
TweakTown

Innosilicon Fantasy One GPU: up to 32GB of ultra-fast GDDR6X memory

Innosilicon has just held its "Fantasy One GPU Product Press Conference" where it unveiled the new Fantasy One GPU family, and a few interesting new graphics cards. Starting with the Innosilicon ...