suu3.jp
Back

summary

If superintelligent AI is created, humanity will become extinct

Photo of If superintelligent AI is created, humanity will become extinct

・2.5 billion years ago, an event called the “Great Oxidation Incident,” occurred, and cyanobacteria, a new life form, learned how to photosynthesize by taking in valuable carbon dioxide from the atmosphere using solar energy. What was discharged as waste during this process is now an “oxygen” chemical substance. Oxygen was extremely toxic and was toxic to most living things at the time. Then, when the concentration of oxygen in the atmosphere began to rise, most organisms, including this bacteria, which had been excreting oxygen, could not cope with oxidation and died. A cell lineage from a lucky minority was applied, and eventually evolved into aerobic organisms that use oxygen as fuel, but things never returned to their former state (p15)

・In the 1930s, there were signs warning Jewish families that danger was imminent in Germany, and although some minorities sensed danger and fled immediately, the majority of people stayed there. They were then stripped of their citizenship and passports by the Nazi government and cut off their chance to escape. A few years later, they were rounded up all at once and sent to an extermination camp. The reason many families stayed was not because they didn't notice the warning signs, but because they believed their lives would return to normal before the worst happened (p16)

・Once upon a time, humans were one step closer to creating ASI (artificial superintelligence = AI that thinks, generalizes from experience, invents new technology, makes plans, strategies, and concepts, and examines and improves one's own way of being), and was one step closer to creation, normal conditions always come to an end, but they don't necessarily change to a bad state, and results may or may change depending on actions, but “it's not the worst situation.” Clinging to that hope doesn't help at all (p16)

・Humans still seem to maintain an advantage in an area called “versatility,” and intelligence being more “versatile” means being able to perform prediction and steering in a wider area. Humans are not necessarily the best in all fields, but humans can think generically in a wider range of fields. Successful prediction and steering can be achieved in a wider range of areas (p28)

・The human brain has approximately 100 billion neurons and 100 trillion synapses, which exceeds most laptop computers in terms of storage capacity. However, current data centers hold 400 petabytes of data, which is more than 1000 times the storage capacity of the human brain, in places where they can be accessed within 5 ms. Moreover, modern AI is trained using a significant portion of all human knowledge and holds a significant portion of that knowledge. This is a break technique that no human being can do (p31)

・Once AI reaches an advanced level of technology, humans become purification boxes; there was a time when humans used to rely on horses; feed costs were expensive, but even so, they kept horses. This is because they hadn't invented cars that would change into cars or armored tanks to replace cavalry, but once they invented technical alternatives to horses, they stopped keeping horses. Unlike good chickens, they are still being bred and raised in large numbers, but this is because human technology has not reached the level of cultivating meat with other inexpensive methods (p96)

・The question economists often ask is that for superintelligence, it is more valuable to use humans as trading partners rather than killing them. This is probably because there is a principle of comparative advantage, and when a high-tech country trades with a low-tech country, both sides can obtain more profit as a whole. Unfortunately, this principle only holds true if both low-tech countries and high-tech countries continue to exist and create a labor force in the future. The amount of heat required to move one human being requires at least 100 W. It is highly unthinkable that goods and services that humans can produce using 100W will surpass those that machine superintelligence can produce using the same 100W (p97)

・Just as humans are not confined to the brain, AI is not “trapped within a computer.” Human thoughts are made of electrical signals flowing through the brain, and electrical nerve signals emitted from the brain cause a chain reaction when transmitted through the spinal cord to the muscles, causing the muscles to contract so that the handle is cut in the correct way. Similarly, electrical signals within computers also have a chain effect on the entire world. What humans can do is determined by whether they can use their hands or body to move something. What AI can do is determined by what it can operate using a “device” such as a human connected to the internet. The internet is a rich and complex environment where billions of phones, smartphones, computers, and humans are connected. There are plenty of opportunities to influence Global A (p109)

knowledge map Explore narrated presentations related to this book View knowledge map