Quantization of large language models with an overdetermined basis

Daniil Merkulov^ab, Daria Cherniuk^a, Alexander Rudikov^ac, Ivan Oseledets^acd, Ekaterina Muravleva^ae, Aleksandr Mikhalev^a, Boris Kashin<sup>cf

a</sup> Skolkovo Institute of Science and Technology, Moscow, Russia
^b Moscow Institute of Physics and Technology, Moscow, Russia
^c Steklov Mathematical Institute of Russian Academy of Sciences, Moscow, Russia
^d Artificial Intelligence Research Institute, Moscow, Russia
^e Sberbank PJSC, Vavilova st., 19, 117312, Moscow, Russia
^f M. V. Lomonosov Moscow State University, Moscow, Russia

Language: English

ArXiv: 2404.09737