Great piece! I think along similar lines. Most serious work is certainly moving molecules around (although some things have relatively low barriers to entry, like making computer viruses, or getting a lab to fabricate some biological virus; things that smart humans might already do). So for me the bigger worry is synthetic cells, nanobots, human upgrades, even cyborgs — all of which are "AI" in a loose sense but not the smart algorithm type. They're scary because their talents will be moving molecules rather than crunching data.
These definitely scare me as well. It's strange to imagine a world where the advantages of humans aren't our brains, but our hands and... the fact that we can be powered by food? Anyway, my theory is that for AI to make the jump to manufacturing nanobots etc. will require lots of human molecule moving at least at the beginning. (Though very hard to say how fast that might spin out of control...)
Great piece! I think along similar lines. Most serious work is certainly moving molecules around (although some things have relatively low barriers to entry, like making computer viruses, or getting a lab to fabricate some biological virus; things that smart humans might already do). So for me the bigger worry is synthetic cells, nanobots, human upgrades, even cyborgs — all of which are "AI" in a loose sense but not the smart algorithm type. They're scary because their talents will be moving molecules rather than crunching data.
These definitely scare me as well. It's strange to imagine a world where the advantages of humans aren't our brains, but our hands and... the fact that we can be powered by food? Anyway, my theory is that for AI to make the jump to manufacturing nanobots etc. will require lots of human molecule moving at least at the beginning. (Though very hard to say how fast that might spin out of control...)