The ability to solve complex problems
DeepMind created AlphaCode
as a powerful code generator and problem-solving tool. It was created to assist
programmers by generating code, while also solving more complicated problems. AlphaCode
was created to try to solve issues that arose in other AI coding systems,
specifically having the ability to solve difficult problems that require analysis
and logic on a deeper level.
In order to achieve good performance for coding, DeepMind discovered
three key points that needed to be incorporated — an extensive and clean
competitive programming dataset for training and evaluation, large and
efficient-to-sample transformer-based architectures, and large-scale model sampling
to explore the search space, followed by filtering based on program behavior to
a small set of submissions, according to the company website.
The results from the research were published on Dec. 8 in Science.
Programming competition results
According to the study, after AlphaCode was trained, it solved about 34.2% of the problems
it was given. In order to further test the skill and expertise of the AI
system, DeepMind entered AlphaCode into online coding competitions. In competitions
with 5000 programmers or more, AlphaCode
ranked in the top 54.3%.
AlphaCode has the ability to generate millions of programs
using trained transformer-based networks — a neural network used for processing
sequential data — and then filters the programs into a maximum of 10 submissions,
comparable to the number of human submissions. “We filter, cluster, and rerank
those solutions to a small set of 10 candidate programs that we submit for
external assessment. This automated system replaces competitors’
trial-and-error process of debugging, compiling, passing tests, and eventually
submitting,” the researchers said on their website.