The pursuit of a solution to the Consensus problem is equivalent to humans finding a way to avoid extinction by Artificial Intelligence.

In 2018, at the World Economic Forum held in Switzerland, Sundar Pichai – the current CEO of Google said, “Artificial Intelligence may be the most important thing humanity has ever developed. I think it has more profound implications than electricity or fire.” While his statement was met with skeptical looks, we now see more and more individuals nodding in agreement with the Google CEO.

At this point in time, artificial intelligence (AI) was already breaking down language barriers on the internet, causing concern among educators at all levels about students and scholars using AI to write convincing essays. Paintings created by AI were and are still confusing art critics, while AI programming at the basic level has left communities stunned. The problem of inferring the 3D structure of proteins, which was once considered impossible for the human mind, can now be solved with AI. Science magazine has even named AI the breakthrough of 2021.

However, in the latest development, a group of experts from various fields have come together to sign an open letter drafted by the Future of Life Institute, agreeing with the potential risks that AI could pose to society and calling for a reduction in the development speed of artificial intelligence. Over 1,100 people have signed the letter, including famous names such as Elon Musk, co-founder of OpenAI (which created ChatGPT), and Steve Wozniak, co-founder of Apple.

The concern of those who signed the open letter is all directed towards a single fear: that the goals of Artificial Intelligence are misaligned with human development.

In addition, some key figures in the artificial intelligence industry also agree with the views expressed in the open letter. These include names such as Yoshua Bengio, a pioneer in deep learning methods; Victoria Krakovna, a researcher at Google’s DeepMind; and Stuart Russell, a computer scientist working at the Center for Human-Compatible AI at UC Berkeley.

All of these individuals understand what AI is and all share a common warning that society is not yet ready to embrace an advanced artificial intelligence system, a goal that all leading technology companies are pursuing.

Issues arise when goals are inconsistent.

In the field of artificial intelligence research, scientists are constantly striving to steer AI towards consistency with their goals – the goals of researchers in general and of humanity in particular. When not aligned, AI may disregard the existence of humans when compared to the goals it wants to achieve.

AI may not deliberately destroy humanity, but it can still do so through a longstanding problem in the field of artificial intelligence research. This problem is called the “alignment problem”.

A hypothetical extreme scenario could be as follows:

We successfully develop an extremely intelligent AI system and ask it to solve an extremely difficult problem. With the problem of “Calculating the number of atoms in the universe,” the AI believes it needs the computing power of all the computers in the Solar System to find the answer. The AI creates a virus capable of wiping out humanity to take control of global computing power and find the answer to the original query.

In this situation, the AI achieves its goal, but in the process, it creates unintended consequences for humanity.

The problem of consensus is all about the ability of AI systems to act contrary to the programmer’s intentions, and even though this may sound like a far-fetched scenario, AI experts have witnessed such behavior numerous times during their research.

For instance, a computer may cheat in a game to get a high score, instead of following the rules of the game. Scientists are unable to know what an AI is thinking and what decisions it might take on its own, and they cannot even code it perfectly: the code still has vulnerabilities that can be exploited by AI.

The National Security Commission on Artificial Intelligence (NSCAI) report sheds more light on the potential development of AI beyond human control. Currently, investing in AI research can yield high returns, and leading tech companies are all chasing this trend. Thus, it is highly likely that in the near future, we will witness AI systems changing human life in ways no one would have imagined.

The NSCAI report attempts to predict the scale of AI in the future, the challenges it poses, and what can be done to ensure AI remains on track, at least in the US.

Excerpt from NSCAI’s report

Over the past decade, the capabilities of AI to perform tasks have significantly improved. They can translate, play complex strategy games (like Go and chess), answer difficult questions in medicine and biology (such as predicting protein folding), and create eye-catching images.

AI systems also power the search results you get on Google or the content users see on social media. They can compose music or persuasive paragraphs. Their ability to detect flying objects is becoming increasingly accurate.

All of these examples describe “narrow AI”, which are computer systems designed to solve specific problems, and cannot be compared to the human brain – a computing machine created to find solutions to every problem.

However, as artificial intelligence can learn, the scope of narrow AI has expanded. Rather than giving AI problems to solve, scientists can now let AI learn to understand problems on its own. As they become more efficient at solving narrow tasks, they begin to reveal their ability to solve broader problems.

For example, older versions of GPT, which were capable of generating text, only knew which word would come after which to form a sentence. Nowadays, they can determine whether a question is sensitive and understand the context to a certain degree to answer the question (such as identifying which object is larger in the real world or listing the logical sequence of problem-solving).

The report highlights the importance of Artificial Intelligence

With researchers affirming that artificial intelligence will revolutionize the world in a positive way. “AI technology is the most powerful tool that has emerged in recent generations, capable of expanding knowledge, increasing economic value, and enriching human experiences,” the report notes. However, it also warns of potential dangers.

“Combined with the powerful computing capabilities of AI, breakthroughs in the biotechnology industry can provide new breakthrough solutions to some of the most difficult problems facing humans, including health, food production, or environmental sustainability. But like other powerful technologies, biotech applications of AI can have a dark side. The COVID-19 pandemic reminds the world of the danger of a highly infectious pathogen. AI can contribute to creating a particularly deadly pathogen or attacking individuals with unique genetic characteristics – creating a weapon with the potential for ultimate devastation,” the report writes.

The effort to “race AI” can push scientists beyond limits or create systems with unknown safety levels. This also means that the problem of consensus can be forgotten.

Experts are afraid of their own children

The brain, the greatest invention of evolution, is the reason why humans dominate the Earth. That’s why since the 1940s, scientists have been striving to create a computer system that operates similarly to the way the brain thinks.

Thinking processes are formed by electrical signals transmitted between neurons, and researchers have used this mechanism to create a machine brain. In 1958, psychologist Frank Rosenblatt demonstrated the feasibility of such a machine: he successfully built a simple model of a brain and trained it to recognize predetermined patterns.

Professor Rosenblatt was ahead of his time. The computers of his era were not powerful enough, and the data was not abundant enough, to give birth to a functioning machine brain. However, Rosenblatt’s groundbreaking research earned him the title of one of the “fathers” of deep learning.

In 2010, with computers a billion times more powerful than Rosenblatt’s time and vast amounts of training data, scientists built the first versions of thinking machines. Over the past decade, experts and organizations have continued to pour resources – both money and data – into these artificial neural networks, making them increasingly sophisticated.

No organization has found the endpoint of the journey, where AI reaches perfection. The more data and the longer they learn, the more perfect AI becomes. It is no longer just what they can do, but where they are going.

In the traditional way of training Artificial Intelligence, researchers create rules and meticulously analyze data, evaluating it in the same way as they do with conventional computer software. But with deep learning, improving the system does NOT require the person performing the task to understand what they are doing. Small changes can improve the performance of AI, but the software engineer designing the system still doesn’t understand why.

The bigger the system, the more confusing they become

As the system continues to grow and expand, patches written by engineers who do not fully understand the system will only make Artificial Intelligence more dangerous. The goals of humans – in this case, possibly large corporations seeking to satisfy shareholders and users – continue to diverge from those of AI. Once again, the problem of consensus returns to haunt us, and when an intelligent system has one or more goals that we do not understand, Homo sapiens faces the risk of extinction, much like the Neanderthals did tens of thousands of years ago.

In the seminal scientific report authored by Alan Turing, which also laid the foundation for the famous “Turing test” (used to determine whether a computer system is truly “intelligent”), the British mathematician, computer scientist, logician, and philosopher wrote the following:

To discuss this issue, let’s assume that intelligent machines are feasible, and let’s look at the consequences of creating them… One would have to do a lot to keep their intelligence up to the standards set by machines, perhaps because once the machine’s reasoning method is activated, it won’t take long before it surpasses the weak thinking ability of humans… And to some extent, we should consider it normal for machines to have control.

the seminal scientific report authored by Alan Turing

Stephen Hawking’s final words before he died

Is the increasing intelligence of machines compared to humans good or bad? We do not have a definitive answer, but we can assert that machines will continue to become smarter in the future.

Before his passing, Stephen Hawking wrote his final book, “Brief Answers To The Big Questions,” which provided answers to some of the mysteries for those who remain. Hawking’s biggest warning was related to the rise of artificial intelligence: it could be the best or worst thing that has ever happened to humanity. And if we are careless with AI, perhaps the computer’s ability to think will be the last invention of humans.

AI has the potential to make breakthroughs in countless fields, but this technology is still in its infancy, and experts express concerns about when the intelligence of AI will surpass that of humans. As Stephen Hawking put it, “While the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled.”

Disregarding the scenario of uncontrolled Artificial Intelligence existence will be the most serious mistake in the history of mankind.

The human race has taken thousands of years to evolve to the height of today’s technology, while AI has only taken a few decades to make humans fear extinction. While many wonder if AI will be good or bad, Hawking believed that the danger of AI lies in its ability to perform tasks assigned to it without hesitation. Essentially, AI will overcome any obstacles to achieve its goal, even if those obstacles are human.

In his book, Brief Answers To The Big Questions, Hawking wrote: “You may not be deliberately malicious like someone who stomps on an anthill, but if you are responsible for a clean energy project and there’s an anthill in the region that needs to be flooded, well, tough on the ants. Don’t place humanity in the position of those ants.”

Conclusion

With fears about what AI can do, experts are calling for the AI research industry to slow down. However, lawmakers have not yet had a solid foundation to enact laws that directly intervene in the development or management of AI systems.

Most issues have two sides, and artificial intelligence is no exception. Alongside concerns that AI will make humans “obsolete”, we hope that AI will bring breakthroughs towards a brighter future for humanity.

With its superior computing power, AI can help humans find cures for diseases, optimize various fields, solve difficult problems such as poverty, disease outbreaks, or climate change. While the human brain’s thinking ability is limited, AI’s thinking ability is only limited by the amount of available data and time to think. These two quantities are increasing day by day, and AI’s ability and potential to solve problems will only increase.

It is difficult to see through the future, whether AI will harm or help humans. Humanity is holding its breath, waiting to see what breakthroughs AI will bring. And it seems that the AI research industry has no intention of slowing down: the only thing that can beat “slow but sure” is “fast and sure”.

If confident in its ability to ensure safety, the AI research industry will quickly bring a brighter future. If not, time will tell.

>> Musk aims to create a super AI to rival ChatGPT