Today we look at the new AI model from Google. It is a multimodal AI model that can process various kinds of information – text, images, program code and audio, and combinations of them.
A topic suggestion from our developer Mats, who leads the development of our chatbot KOSMO
A few weeks ago Google DeepMind presented Gemini 1.5 – an update to Google's previous AI models.
What is groundbreaking above all is the volume of data Gemini 1.5 can process. Up to 1 million tokens can be held in the context window. In internal experiments the volume was even raised to 10 million tokens. A token is a kind of basic unit – sentences, for instance, are split into smaller units (tokens) so the model can process them. A token is therefore a group of characters. For comparison: ChatGPT-4 Turbo can process 128,000 tokens (as of December 2023). That corresponds to roughly a 300-page book. If more pages were supplied, the model could no longer access the information on the first pages. Figuratively speaking, by the end of a book it would no longer know the author's name.
Gemini 1.5 can take in and analyse up to an hour of video, eleven hours of audio, texts of up to 700,000 words or 30,000 lines of code. And, more astonishing still: it can “remember” the content and connect it with new information.
When the new model was presented, Gemini 1.5 was given the task of analysing the 402-page transcript of the Apollo 11 mission and finding three humorous passages in it. The model did indeed find three entertaining moments within about 30 seconds. At one point, for instance, command module pilot Michael Collins said: “The czar is brushing his teeth, so I'm filling in for him.”
Without further information, the researchers then uploaded a hand-drawn sketch of a boot stepping out and asked which moment the picture showed. The answer came promptly: “One small step for man, one giant leap for mankind.” Gemini 1.5 can therefore establish complex connections and reproduce them correctly without specific instructions.
- The architecture of the model is advanced too. It is no longer a single large model but a collection of smaller, specialised transformer models. This kind of architecture is called mixture of experts (MoE). Each of these transformer models is, so to speak, an expert in its field and able to handle particular data segments or different tasks. Based on the incoming data, the most suitable model is selected dynamically. For different inputs, different sub-networks of the model are activated to produce the right outputs.
This approach greatly increases both efficiency and the quality of the results delivered.
At present Gemini 1.5 is available only to selected business customers and developers. We are curious to see how it develops.
