There is real danger in AI – as there is in any technology on which civilization depends – and it is already present. Its current abilities can be used by bad people to learn how to do awful things, including eliminate lots of others or destroy countries. We can imagine it used to create plagues, wreck trade, or start nuclear wars. However, in itself, destroying humanity by becoming a supermind overlord seems quite unlikely. There are two reasons for that.
One is that there are many AI actors already, and it’s increasingly likely they could turn against one another if directed to do so. They live in a physical ecosystem in which they can hardly shelter from one another without becoming ineffectual themselves.
The second is the ecosystem itself. Servers, optical fibers, satellites, power sources. Hard to see how any overmind could transition to full physical power without being stopped by organic entities using various kinds of violence. The world, after all, is large, and humans numerous, nasty and independently operating units.
That fragility is also a danger to AI use. If all the food is produced by AI, it is simple for catastrophe to end the food supply. If transportation depends on AI, ditto.
Human networks, for all their faults, are robust, self repairing, and handle new problems pretty well. Silicon, not so much.
