As always, we take up topic suggestions from our team. This month's comes from our managing director Caro and looks at the current hype around DeepSeek-R1.
Anyone examining the Chinese language model DeepSeek-R1 will immediately notice one property above all: the immense price advantage over other models – at competitive performance. That keen price is made possible by an extremely efficient training method and model architecture. DeepSeek-R1 has caused a genuine hype and is portrayed in the media almost as a quantum leap. But what is actually behind it?
Much ado about nothing?
In fact DeepSeek-R1 is not a new technology but merely an impressive combination of already known technologies, such as the mixture-of-experts approach.
This innovative combination makes it possible to run the model on comparatively inexpensive hardware. And DeepSeek-R1 has perfected reinforcement learning. Put simply, the results produced by DeepSeek-R1 are carried over into the next model version, achieving an enormous improvement in the “intelligence” of that next generation. OpenAI, for example, currently needs far more effort to improve its own models substantially – which is why the company is coming under increasing pressure from DeepSeek-R1 and the publication of its technical details.
Amid all the excitement, it should not be overlooked that there have been innovative developments in language models almost weekly in recent years – from companies, universities or state-driven actors. One example is the Teuken7B model. Neither the open-source approach nor the basis of DeepSeek-R1 is genuinely novel.
Rather, the development of DeepSeek-R1 is a reaction to US sanctions – China lacks high-performance hardware.
DeepSeek-R1 is by no means the first open-source language model to reach the top of the comparison tables for a short period.
DeepSeek-R1 is usable by anyone – including without running it on your own hardware. However, as with other models, the data you enter is passed on to the maker. You should also be aware that the knowledge it holds is in parts politically coloured – a circumstance that has, in part, been observed with other models too. So DeepSeek-R1 has achieved one thing above all: finding a low-cost, highly efficient way to train and improve complex language models. And it has shown that way to the public. It is therefore fair to assume that development in this area will pick up additional speed following DeepSeek-R1's release. It will probably not be enough for a lasting advantage over the competition, though.
