We have painted ourselves in a different corner with artificial intelligence. We are finally beginning to break through the usefulness barrier, but we are reaching the limits of our ability to responsibly meet the massive energy needs of our machines.
At the current rate of growth, it seems that if we are to continue spending unfathomable amounts on energy training systems like GPT-3, we must turn the earth into Coruscant.
The problem: Put simply, AI takes too much time and energy to train. A layperson could imagine a lot of code on a laptop screen when thinking about AI development, but the truth is that many of the systems we use today have been trained on massive GPU networks, supercomputers, or both. We’re talking about incredible amounts of power. And worse, it takes a long time to train the AI.
The reason AI is so good at the things it’s good at, such as: B. Image recognition or natural language processing, is that it basically does the same thing over and over, making tiny changes every time until everything goes right. We’re not talking about a couple of simulations, though. Training a robust AI system can take hundreds or even thousands of hours.
One expert estimated that GPT-3, a natural language processing system developed by OpenAI, would cost about $ 4.6 million. However, this requires one-shot training. And very, very few powerful AI systems are trained in one fell swoop. Realistically, the total cost of getting GPT-3 to spit out impressively coherent gibberish is likely to be hundreds of millions.
GPT-3 is one of the high-end abusers, but there are countless AI systems that use a disproportionately large amount of energy compared to standard calculation models.
The problem? If AI is the future, the future will not be green under the current power usage paradigm. And that can mean that we just won’t have a future.
The solution: Quantum computing.
An international team of researchers, including scientists from the University of Vienna, MIT, Austria and New York, recently published research showing “quantum acceleration” in a hybrid artificial intelligence system.
In other words, you managed to use quantum mechanics to allow AI to come up with more than one solution at a time. This of course speeds up the training process.
According to teamwork:
The key question for practical applications is how quickly agents learn. Although various studies have used quantum mechanics to speed up the agent’s decision-making process, a reduction in learning time has not yet been proven.
Here we present an enhanced learning experiment in which an agent’s learning process is accelerated by using a quantum communication channel with the environment. We also show that the combination of this scenario with classic communication enables the assessment of this improvement and enables optimal control of the learning progress.
That’s the cool part. They ran 10,000 models through 165 experiments to see how they worked with classic AI and how they worked when enhanced with special quantum chips.
And in particular, do you know how classic CPUs work by manipulating electricity? The quantum chips the team used were nanophoton, which means they use light instead of electricity.
The core of the operation is that in circumstances where classical AI solves very difficult problems (think supercomputing problems), the hybrid quantum system outperforms the standard models.
Interestingly, the researchers did not see any increase in performance for less difficult challenges. Seems like you need to put in fifth gear before turning on the quantum turbo.
Much remains to be done before we can roll out the old Mission Accomplished banner. The team’s work wasn’t the solution we ultimately seek, but rather a small model of how it might work when we figure out how to apply their techniques to bigger, real-world problems.
You can read the whole paper here about nature.
H / t: Shelly Fan, Singularity Hub
Published on March 17, 2021 – 19:41 UTC