Training LLMs on Mixed GPUs: My Experiments and What I Learnt

«In the last few months, I have been very interested in large language models. At the same time, the GPU world is also changing. Nvidia is still the market leader, but AMD, Intel, and even Chinese companies are making che...»