{"id":791223,"date":"2026-07-10T11:15:16","date_gmt":"2026-07-10T11:15:16","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/791223\/"},"modified":"2026-07-10T11:15:16","modified_gmt":"2026-07-10T11:15:16","slug":"introducing-plan-a-by-scott-alexander","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/791223\/","title":{"rendered":"Introducing Plan A &#8211; by Scott Alexander"},"content":{"rendered":"<p>A Is For America<\/p>\n<p>It\u2019s increasingly clear that nobody has a plan for if this AI thing turns out to be real. Some people have suggestions, but they\u2019re all things like \u201cregulate a little more\u201d or \u201cregulate a little less\u201d or \u201creact to things as they come up\u201d. This won\u2019t be enough. Not just because things may move too quickly &#8211; although they will &#8211; but because in order to regulate or react, you need to know what you\u2019re aiming for, and it\u2019s increasingly clear that people can\u2019t even visualize what AI going well could look like. What would it take to honestly tell our children that we rose to the occasion, to make the AI transition go down alongside the American Revolution and D-Day as one of our country\u2019s finest hours? If your brain sputters and throws an error message at the question, isn\u2019t that a problem?<\/p>\n<p>It\u2019s a total coincidence that <a href=\"https:\/\/www.ai-2040.com\/\" rel=\"nofollow noopener\" target=\"_blank\">Plan A<\/a> comes out the week after America\u2019s 250th birthday. It was supposed to come out earlier, but got delayed. Then it was supposed to come out later, but got pushed forward.<\/p>\n<p>Still, the saying goes \u201cA wizard is never late, nor is he early; he arrives exactly when he means to.\u201d And if anyone qualifies as wizards, it\u2019s Daniel Kokotajlo and his team of forecasters at the AI Futures Project. I <a href=\"https:\/\/www.astralcodexten.com\/p\/introducing-ai-2027\" rel=\"nofollow noopener\" target=\"_blank\">previously wrote<\/a> about Daniel\u2019s eerie accuracy over the 2021 &#8211; 2025 period. Since then, they\u2019ve gained worldwide fame for their <a href=\"https:\/\/ai-2027.com\/\" rel=\"nofollow noopener\" target=\"_blank\">AI 2027<\/a> scenario, which predicted the rise and quick takeover of coding agents in early 2026, plus something like the fight over Fable.<\/p>\n<p>Plan A isn\u2019t another prediction. It\u2019s a wish list, a positive vision, a road map for navigating the future. It describes the best course of action that Daniel and the AI Futures Project can come up with, and what would happen if we took it. <\/p>\n<p>\u201cReally? You got America a policy paper for its 250th birthday? Doesn\u2019t America already have enough policy papers?\u201d Sort of, but it\u2019s not exactly a policy paper.  It starts in a timeline similar to that of AI 2027, on track for a poorly-controlled intelligence explosion that either ends the world or dooms it to permanent techno-oligarchy. But this time, America is blessed with some extra foresight and determination, and makes only good choices (all non-Americans behave naturally, including trying to thwart America when incentivized to do so). It gives a year-by-year description of this best-of-all-possible-worlds, from now through 2040, as predicted by the best AI forecasters alive, with over a dozen supplements explaining all the implementation details. <\/p>\n<p>This is a crazy thing to try releasing. Daniel gave me several justifications for doing it anyway, but the one I remember most is that it\u2019s supposed to be a floor. When some politician proposes a data center ban, or says that we have to gut safety regulation to compete with China, or promises a job retraining program, think to yourself: does this person have a vision for where all of this ends up? If so, is it as good as Plan A? If not, consider demanding that they do better.<\/p>\n<p>I did a lot of writing for AI 2027 and was listed as a co-author. Some of my writing made it into Plan A too, but it was a bit less. The difference is of degree rather than kind, but because of this &#8211; and to give me more latitude to discuss it the way I like with less PR blowback &#8211; we decided not to put me as a co-author this time. I continue to be proud of having a part in this, small as it may be.<\/p>\n<p>(related: everything in this post is my opinion only, and not officially endorsed by the AI Futures Project)<\/p>\n<p>A Is For Agreement<\/p>\n<p>The linchpin of Plan A is a joint AI regulatory regime with China.<\/p>\n<p>In the late 2020s, as the world shoots toward an intelligence explosion, the US government realizes that it has no control over the situation as long as race dynamics continue to hold. Multiple \u201cMythos moments\u201d leave them convinced that the situation is spiraling out of control, but analysts continue to insist that if we unilaterally slow or regulate AI, China will continue its own research and gain a dangerous strategic advantage over us.<\/p>\n<p>So the hypothetical wise statesman president proposes a joint regulatory regime to China, and China agrees. This is one of the sections AIFP spent the most time thinking about and trying to justify &#8211; the conceit was that America is wise and foresightful by narrative fiat, but our rivals still have to behave in believable ways. Still, they think China\u2019s agreement is plausible. They have concerns similar to ours (things are moving too fast, society is being disrupted, they can\u2019t rule out existential risk), plus the additional concern that they\u2019re currently <a href=\"https:\/\/blog.aifutures.org\/p\/why-america-wins\" rel=\"nofollow noopener\" target=\"_blank\">losing the race to America<\/a> and so an enforced tie would be in their favor.<\/p>\n<p>The US and China don\u2019t trust each other, so any agreement would have to be trustless, ie impossible to cheat, win-win even if you expect the other side is trying as hard as it can to defect against you. This is another area that AIFP expected to be a major sticking point for most people, and that they put in lots of work to justify. Their plan is:<\/p>\n<p>Establish joint control over the supply of new chips<\/p>\n<p>Establish common knowledge of the location of all existing chips.<\/p>\n<p>Ensure that all new and existing chips go to mutually-transparent, mutually-audited secure data centers.<\/p>\n<p>Establishing control over the chip supply is easy. Only a few companies can design AI chips (eg NVIDIA in the US, Huawei in China), and only a few factories in the world can produce them (eg TSMC in Taiwan, Intel in the US, SMIC in China). All of these companies and factories are in the US, China, or client states kept on very short leashes. The US and China simply tell these companies and factories to send chips only to licensed customers, and send auditors and inspectors to ensure compliance. Something like this is already in place; this deal merely tightens the restrictions and lets both countries participate in the auditing process.<\/p>\n<p>Hunting down all existing chips is only slightly harder. Most of these chips are in giant data centers the size of small cities using entire power plants\u2019 worth of electricity &#8211; so hard to hide. The rest can be traced to their final locations using customer records from NVIDIA, TSMC, etc. AIFP explain their methods in more detail in the <a href=\"https:\/\/www.ai-2040.com\/supplements\/covert-ai-projects\" rel=\"nofollow noopener\" target=\"_blank\">Covert Project Supplement<\/a>, but estimate that they can track down 98.5% of existing chips (we\u2019ll come back to the remaining 1.5% later).<\/p>\n<p>Once the two countries are confident they\u2019ve established control over almost all chips, they relocate them to licensed data centers (\u201cwhitesites\u201d), which the other side is allowed to send auditors to inspect. The <a href=\"https:\/\/www.ai-2040.com\/supplements\/verification-plan\" rel=\"nofollow noopener\" target=\"_blank\">Verification Supplement<\/a> describes very-near-future technology (would take 1-2 years to have ready; I recently recommended grants to organizations working on creating it) that can monitor these data centers and provide mutual transparency into what they\u2019re doing. The auditors can ensure the technology is installed and uncompromised, and then both sides will know if the other is defecting against the deal. Other auditors in the chip factories ensure that any new chips produced are being sent to the whitesites too.<\/p>\n<p>The end result is that the US can be confident that at least 98.5% of China\u2019s chips are in white sites where the American government can audit their activities, and so China can\u2019t defect on the deal by training dangerous AI without America knowing. The same is true on the other side; China knows where almost all of America\u2019s chips are, and can be confident we aren\u2019t defecting.<\/p>\n<p>A Is For Aristotelian<\/p>\n<p>Now that the US and China know what\u2019s going on in each other\u2019s data centers, and agree in principle to coordinate their AI research, what do they do?<\/p>\n<p><a target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!IaxW!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bb76017-a8b1-4134-84ea-cbf1ae3701b5_740x333.png\" data-component-name=\"Image2ToDOM\" class=\"image-link image2 is-viewable-img can-restack\" rel=\"nofollow noopener\"><img decoding=\"async\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/07\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/6bb76017-a8b1-4134-84ea-cbf1ae3701b5_740x.jpeg\" width=\"686\" height=\"308.7\" data-attrs=\"{&quot;src&quot;:&quot;https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/6bb76017-a8b1-4134-84ea-cbf1ae3701b5_740x333.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:333,&quot;width&quot;:740,&quot;resizeWidth&quot;:686,&quot;bytes&quot;:470130,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image\/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https:\/\/www.astralcodexten.com\/i\/204382281?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bb76017-a8b1-4134-84ea-cbf1ae3701b5_740x333.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}\" alt=\"\" title=\"\"   loading=\"lazy\" class=\"sizing-normal\"\/><\/a><\/p>\n<p>In an earlier draft of Plan A, I called this the \u201cGolden Path\u201d, before the more dignified team members took out the reference. The idea is that we need a sort of golden mean in the speed of AI progress. Too fast, and we get misaligned AI or some kind of social upheaval (you\u2019ve already seen this case a thousand times; read <a href=\"https:\/\/www.astralcodexten.com\/p\/book-review-if-anyone-builds-it-everyone\" rel=\"nofollow noopener\" target=\"_blank\">If Anyone Builds It, Everyone Dies<\/a> for the details). But why worry about going too slow?<\/p>\n<p>In Plan A, the biggest risk of going too slow is that the deal falls apart. The clearest precedent here is arms control regimes; START + New START held on for a good few decades, but were suspended over tensions around Ukraine in 2023, then expired fully in 2026. The JCPOA nuclear treaty with Iran barely made it two years. If we dilly-dally and do nothing for fifty years, probably the agreement falls apart and we\u2019re right back where we started. So we should have some plan to do what we want to do with the agreement within a decade or two.<\/p>\n<p>The second-biggest risk comes from that 1.5% of compute we discussed earlier. Again, we\u2019re assuming that we can\u2019t trust China (and vice versa) &#8211; so they\u2019ve done their maximum possible defection, hidden 1.5% of their compute, and are using it to train some maximally dangerous military AI. How long before that AI could undergo an intelligence explosion and give them a decisive strategic advantage? AIFP calculates that this would take somewhere north of a decade &#8211; so again, we should have some plan to get what we want out of the deal before that period is up.<\/p>\n<p>The third-biggest risk comes from the steady march of technology. Chips and algorithms get better every year; AIs that took city-sized data centers to train today may take only academic-grade hardware tomorrow. Once a dangerous AI can be trained on academic-grade hardware, trustless deals become impossible, because anyone could be hiding a few scattered supercomputers. There are some cheap things we can do to slow this process down, but stopping it entirely could require authoritarian-seeming interventions or a generalized slowdown of economic progress. Rather than go that route, we should have some plan to get what we want out of the deal before these considerations become pressing &#8211; which, once again, means before a few decades are up.<\/p>\n<p>And although it didn\u2019t make it into the main text, there\u2019s an additional consideration around balancing the risk of AI vs. the risk of all the things AI could save us from. Nuclear war, bioengineered pandemics, mirror life, collapsing fertility, decreasingly functional politics, \u201cthe polycrisis\u201d &#8211; I don\u2019t worry much about this stuff compared to dangerous AI in five years, but the longer we delay AI to work on alignment, the more the risk from AI goes down, and the risk from all these other things goes up, until eventually they meet in the middle and we delayed too long. How long should we delay? I think this one alone suggests slightly longer timelines than the others, but it\u2019s still decades and not centuries.<\/p>\n<p>So in Plan A, the US and China agree to go as fast as possible without compromising on safety. <\/p>\n<p>Starting in the early 2030s, they make a push to train more AI, quicker. These AIs will still be trained by existing companies (Anthropic, Alibaba, DeepSeek, OpenAI, etc) and can still be used for business purposes, but they\u2019ll be trained and hosted in regulated monitored data centers, there will be a moving capabilities ceiling, both countries will raise the ceiling at the same rate at the same time, and the resulting AIs will be available to everyone in the world.<\/p>\n<p>(at some point the US and China will loop all the other countries into this regulatory regime as some kind of sort-of-but-not-really-voting observers; they agree to follow the rules in exchange for shared benefits, including data centers on their territory, access to the AIs, and a share of future AI-generated wealth. This isn\u2019t strictly necessary, because no other country really has the ability to do much with AI, but it\u2019s a nice gesture for our utopian scenario)<\/p>\n<p>Then, in the mid-2030s, they pause at AIs around the level of top human geniuses. These AIs are close enough to current AIs that the same alignment techniques that mostly-sort-of-work for ours might mostly-sort-of-work for them too. But if they don\u2019t, it\u2019s fine &#8211; the safety case rests primarily on \u201ccontrol\u201d, ie keeping the AIs in a box they can\u2019t get out of (in this case, highly regulated data centers that have been secured from the inside). This wouldn\u2019t work for superintelligence, but with enough care, it will work for these \u201cmerely\u201d top-human-level models. The plan is to spend the next ~10 years using this \u201ccountry of geniuses in a data center\u201d to solve AI alignment, along with approximately all other problems.<\/p>\n<p><a target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!U6oX!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7b51cbeb-8e4f-430c-97e1-44c15364eaea_1427x829.png\" data-component-name=\"Image2ToDOM\" class=\"image-link image2 is-viewable-img can-restack\" rel=\"nofollow noopener\"><img decoding=\"async\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/07\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/7b51cbeb-8e4f-430c-97e1-44c15364eaea_1427.jpeg\" width=\"687\" height=\"399.1051156271899\" data-attrs=\"{&quot;src&quot;:&quot;https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/7b51cbeb-8e4f-430c-97e1-44c15364eaea_1427x829.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:829,&quot;width&quot;:1427,&quot;resizeWidth&quot;:687,&quot;bytes&quot;:195916,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image\/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https:\/\/www.astralcodexten.com\/i\/204382281?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7b51cbeb-8e4f-430c-97e1-44c15364eaea_1427x829.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}\" alt=\"\" title=\"\"   loading=\"lazy\" class=\"sizing-normal\"\/><\/a><\/p>\n<p>A Is For Abundance<\/p>\n<p>The middle of Plan A is AIFP\u2019s pleasant fantasy about all the problems they solve easily by deploying millions to billions of top-human-genius level AIs.<\/p>\n<p>I previously said the conceit of this exercise is that America only makes good decisions. Even so, you could be forgiven for some skepticism here; even if the genius-level AIs give us the technological capacity to solve our problems, what provides the political will? Their excuse is <a href=\"https:\/\/www.astralcodexten.com\/p\/the-ai-superforecasters-are-here\" rel=\"nofollow noopener\" target=\"_blank\">the AI superforecasters<\/a> and things like them. The <a href=\"https:\/\/www.ai-2040.com\/supplements\/ai-for-epistemics\" rel=\"nofollow noopener\" target=\"_blank\">AI For Epistemics<\/a> supplement provides the details, but they dream of a world where today\u2019s forecasting tools blossom into a wide ecosystem of advisors that voters, politicians, and the media use to build models of the future and guide their decisions. They hope that trustworthy AIs will be able to assert convincingly that the policies they recommend will go well, and that the alternative is techno-feudalism, mass immiseration, or (in some cases) doomsday. This breathes new life into our usually sclerotic political system and allows it to dream big.<\/p>\n<p>The top-human-genius AIs soon become capable of taking most white-collar jobs. The electorate solves this with a \u201ccitizen\u2019s dividend\u201d (I was warned against using the term \u201cUBI\u201d) which is very easy to afford, since the AIs are causing double-digit and even triple-digit yearly GDP growth (think that\u2019s crazy? Read the <a href=\"https:\/\/www.ai-2040.com\/supplements\/economics-of-plan-a\" rel=\"nofollow noopener\" target=\"_blank\">Economics Supplement<\/a>). Because of extreme deflation in most other goods, the dividend is pegged to compute; as compute production rises, it goes from $25,000 per person per year at its inception in 2033 to $1.6 million in 2035 (all dollar amounts are extreme-deflation-adjusted; for more, read the supplement). Various diseases get cured. Various social ills get ended. Probably there are flying cars or some similarly cool inventions. <\/p>\n<p><a target=\"_blank\" href=\"https:\/\/substackcdn.com\/image\/fetch\/$s_!vBfC!,f_auto,q_auto:good,fl_progressive:steep\/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F00172cdc-f26b-4e61-bcf4-3a6aabf9b074_675x447.png\" data-component-name=\"Image2ToDOM\" class=\"image-link image2 is-viewable-img can-restack\" rel=\"nofollow noopener\"><img decoding=\"async\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/07\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/00172cdc-f26b-4e61-bcf4-3a6aabf9b074_675x.jpeg\" width=\"675\" height=\"447\" data-attrs=\"{&quot;src&quot;:&quot;https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/00172cdc-f26b-4e61-bcf4-3a6aabf9b074_675x447.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:447,&quot;width&quot;:675,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:738774,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image\/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https:\/\/www.astralcodexten.com\/i\/204382281?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F00172cdc-f26b-4e61-bcf4-3a6aabf9b074_675x447.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}\" alt=\"\"   loading=\"lazy\" class=\"sizing-normal\"\/><\/a>Everything in this post is my own opinion and not endorsed by the AI Futures Project, but this image is especially not endorsed by them.<\/p>\n<p>All this growth brings new risks of its own. If uncontrolled, it could spark another arms race &#8211; eg if the US grows at 150% per year, but China only at 100% per year, then after ten years America is 9x bigger and could easily crush its rival. If unmonitored, someone could use the cancer-cure technology to create something extremely bad, from bioweapons to mirror life to things I won\u2019t mention because you\u2019ll accuse them of being implausible. This gets solved basically the same way as the data centers &#8211; the US, China, and the various sort-of-but-not-really-voting observer nations agree to restrict AI-assisted economic growth to special economic zones which are heavily monitored by all parties. This has the bonus of ensuring that the 200-story-tall nano-assembly plant won\u2019t be in your personal backyard.<\/p>\n<p>For more pleasant fantasies, read the 2033 &#8211; 2037 sections.<\/p>\n<p>A Is For Alignment<\/p>\n<p>Meanwhile, billions of other top-human-genius-level AIs work on questions around AI alignment. <\/p>\n<p>This suffers from a sort of bootstraps problem; how do we know that these AIs will be safe?<\/p>\n<p>First, they might be safe simply because they haven\u2019t gotten to the regime where they\u2019re unsafe. Today, AIs are <a href=\"https:\/\/blog.redwoodresearch.org\/p\/current-ais-seem-pretty-misaligned\" rel=\"nofollow noopener\" target=\"_blank\">slightly misaligned<\/a> but still easy enough to get useful work out of. As AIs progress from here toward superintelligence, they may get gradually less aligned for several reasons: they\u2019ll be more agentic, they\u2019ll be better at scheming coherently, they\u2019ll have more \u201cout-of-distribution\u201d options, and they\u2019ll be smart enough to generalize in weird ways. Top-human-genius AIs will be some part of the way along that distribution, but maybe not so far as to behave 100% adversarially.<\/p>\n<p>But Plan A doesn\u2019t rely on this; its safety case hinges on <a href=\"https:\/\/www.redwoodresearch.org\/research\/ai-control\" rel=\"nofollow noopener\" target=\"_blank\">control<\/a>. AIs are kept in heavily-guarded data centers that they can\u2019t escape even if they wanted to (the bandwidth is low enough that exfiltrating their weights over the network would take decades, and people entering or leaving are searched for USB drives, etc). All of their actions are watched by AIs from other companies and countries; anything suspicious is escalated to human monitors. Smart humans and faux-misaligned AIs constantly red-team the security. Since safety incentives now outweigh race incentives, there is no reason to rush towards unreadable chains-of-thought, and so chains of thought are monitored carefully and successfully. Some genuinely misaligned AIs are created as \u201cmodel organisms\u201d in extremely controlled conditions and observed to see what they do; if they are capable of hacking or social engineering at a certain level, everyone ensures that this level is insufficient to escape the data centers. If any AIs do escape or even make progress towards escaping, we prepare to <a href=\"https:\/\/blog.redwoodresearch.org\/p\/a-taxonomy-of-barriers-to-trading\" rel=\"nofollow noopener\" target=\"_blank\">trade with them<\/a> rather than treat them as fully adversarial. None of these mechanisms would work to control a true superintelligence, but that\u2019s why we aren\u2019t building one of those yet.<\/p>\n<p>Since these AIs may not be fully-aligned, a key part of early research is using whatever useful work we can get out of them to bring them to a point where we can trust them further, eg where they\u2019ll do good alignment research and not try to fake it (\u201cYou\u2019re absolutely right, I did propose a scheme that would result in the destruction of humanity &#8211; and that\u2019s my bad\u201d). Once we\u2019re at that point, we can invest tens of billions of researcher-years into the alignment problem and come out with something actually trustworthy; AIFP speculates about what that might look like in the <a href=\"https:\/\/www.ai-2040.com\/supplements\/alignment-roadmap\" rel=\"nofollow noopener\" target=\"_blank\">Alignment Supplement<\/a> and the <a href=\"https:\/\/www.ai-2040.com\/?choices=plan-a-root#playbook-insider-pov\" rel=\"nofollow noopener\" target=\"_blank\">Insider Perspective<\/a>. In a best-case scenario, we replace the current generation of weird machine-learning-kludge-based AIs with some sort of more mathematical AI that it\u2019s possible to prove things about, and prove it to be aligned or at least corrigible.<\/p>\n<p>Along with this technical research, we\u2019re also doing philosophical . . . something between \u201cresearch\u201d and \u201cdebate\u201d, deciding what values we want the aligned AIs to have, or how much we want the AIs to follow government orders vs. follow the general will of humanity vs. search for moral truths on their own. The end of this stage probably looks like every country with AIs either implanting its own values or leaving it to individual companies\/people to do this, and the AIs doing incomprehensible economic bargaining among themselves to work out any conflicts. <\/p>\n<p>Once we\u2019re very certain that AIs are fully aligned &#8211; AIFP speculates this could be around 2040 &#8211; we hand over the sovereignty layer of government to them. Why? All of the problems we talked about in the \u201crisks of going too slow\u201d section &#8211; deal breakdown, defection, nuclear war, etc &#8211; are problems with misaligned humans doing bad things. Once we have fully-aligned AIs, we want to give them final say over those levers so the risks don\u2019t happen. Because the AIs are fully-aligned, they continue to let humans run the policy layer of government. The idea is that the US President or the paramount leader of China still controls day-to-day decisions, but if someone tries to pull off a coup and launch the nukes for no reason, then an aligned AI controls the nuclear missiles and it says no.<\/p>\n<p>Even after we\u2019ve handed over the nukes, the AIs still don\u2019t race to superintelligence &#8211; having an alignment plan that will provably work for superintelligence is a higher bar than having one that provably works for the sort of top-human-genius AIs we trust to lead us through the 2030s. In our scenario, the AIs start an intelligence explosion pretty soon after the handover in 2040. But this is just for narrative flow; at this point, the risks of going too slow are greatly diminished, and if the AIs say it will take until 2100 to be really sure they\u2019ve solved everything, we should give them until 2100 (we\u2019ve probably conquered aging by this point anyway).<\/p>\n<p>After we have superintelligence in 2040, the superintelligences solve whichever philosophical problems are solvable, predict whichever aspects of the future are predictable, and then, in consultation with human governments, help chart the best possible future for humanity among the stars. AIFP took this part out of the main text, because they worried that fancy Washington policymakers reading our scenario would get weirded out by descriptions of what type of auction you use to determine who gets how many galaxies, but it\u2019s in the <a href=\"https:\/\/www.ai-2040.com\/?choices=plan-a-root#playbook-epilogue\" rel=\"nofollow noopener\" target=\"_blank\">Epilogue<\/a> and the <a href=\"https:\/\/www.ai-2040.com\/supplements\/space-governance-plan\" rel=\"nofollow noopener\" target=\"_blank\">Space Governance Supplement<\/a> and I\u2019ll try to blog about it more later this week.<\/p>\n<p>A Is For All Of Us<\/p>\n<p>Why care about this weird sci-fi story?<\/p>\n<p>For the past year or two, I\u2019ve kind of been in despair. If AI 2027 is right, then we could get an uncontrolled intelligence explosion sometime in the late 2020s or early 2030s. Over the course of a year or two, we could go from a basically normal world where we\u2019re mostly talking about shoplifting and health care costs and Trump, to a world completely under the control of some sort of incomprehensible superintelligence. Even a nimble and competent government would have trouble responding on such a timescale (let alone our government) and the likely outcome would be as AI 2027 portrays it: oligarchy or extinction. <\/p>\n<p>So it seems like we should try to prevent AI progress somehow. But preventing technological progress &#8211; \u201cdegrowth\u201d, as the kids are calling it these days &#8211; is historically one of the worst possible bets. Done in small doses, it merely leads to preventable poverty, misery, and mass death. Done too much, it breaks the engine of social advance entirely, trapping whole civilizations in a morass of pessimism, paranoia, and zero-sum thinking which is near-impossible to escape after they\u2019ve fallen in. <\/p>\n<p>But surely the heuristic that technology is usually good can\u2019t be extended into a command to never worry about any technology, no matter how apparently catastrophic? On the other hand, everyone who stands in the way of technology thinks their specific case is justified; the maxim \u201cdon\u2019t do it unless it\u2019s justified\u201d does zero work. These are the sorts of considerations that have been haunting me. As Woody Allen put it, \u201cMankind is facing a crossroad. One road leads to despair and hopelessness, and the other to total extinction. Let us pray that we have the wisdom to choose correctly.\u201d<\/p>\n<p>Plan A feels like our best hope to do something actually good. The key insight is that if powerful AI is really as close and transformative as we think, then there\u2019s a massive surplus that can satisfy everyone. Tyler Cowen and Marc Andreessen and the accelerationists will scream about how we\u2019re slowing down, but we imagine our slow world as going much faster than they imagine their fast one. If Tyler wants double-digit GDP growth in the 2030s, we\u2019ll give him triple-digit. If Marc Andreessen wants a cancer cure by 2050, we\u2019ll give it to him by 2035. The choice isn\u2019t business-as-usual versus a world with some gee-whiz AI innovations. It\u2019s nanobots eating the solar system in 2033 vs. cancer cures in 2035. When we say \u201cwe must slow down AI\u201d, we mean we want the cancer-cures-in-2035 one. If we\u2019re merely on track for a few cool gee-whiz AI innovations in the 2040s, then I\u2019m wrong about everything and none of this really matters one way or the other.<\/p>\n<p>Milton Friedman <a href=\"https:\/\/www.goodreads.com\/quotes\/110844-only-a-crisis---actual-or-perceived---produces-real\" rel=\"nofollow noopener\" target=\"_blank\">said that<\/a> true political change only happens during crises, and that the future belongs to whoever has a plausible plan ready when the crisis happens. AIFP releases Plan A in this spirit. If you\u2019re against it, you should think long and hard about what alternative course of action you expect the government to take once the crisis becomes evident, and whether it will go better for your interests than Plan A will. If you think that there will never be a crisis, that the public will never challenge AI, that and you can just keep reacting to things as they arise &#8211; then good luck with that. I really think we\u2019re the good cop here.<\/p>\n<p>Except that I shouldn\u2019t say \u201cwe\u201d. It\u2019s wholly a coincidence that Plan A solves this problem. Nobody else on the team seemed that interested in it; they were just trying to calculate what actually has the best chance of keeping us alive. In fact, they asked me to stress that this is all provisional: if we get to 2030-something and the calculus of the Golden Path demands we move slower than this, they\u2019ll recommend going slower; if it demands we move faster, they\u2019ll recommend that too.  <\/p>\n<p>But in fact, their current calculations do suggest a Golden Path which ends poverty and disease within a decade, and gives us a glorious interplanetary future within our lifetimes. This wasn\u2019t a requirement of their model. It\u2019s just how things coincidentally seem to have worked out. The optimal regime for alleviating doomers\u2019 concerns happens to be one which should satisfy accelerationists, and a trajectory well within accelerationists\u2019 Overton Window happens to have the best possible properties for safety research. <\/p>\n<p>Like in previous advances in AI, I can only attribute it, as all else, to divine benevolence.<\/p>\n<p>You can read Plan A <a href=\"https:\/\/www.ai-2040.com\/\" rel=\"nofollow noopener\" target=\"_blank\">here<\/a>. Don\u2019t miss the supplements hidden in the top right corner. <\/p>\n","protected":false},"excerpt":{"rendered":"A Is For America It\u2019s increasingly clear that nobody has a plan for if this AI thing turns&hellip;\n","protected":false},"author":2,"featured_media":791224,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-791223","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/791223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=791223"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/791223\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/791224"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=791223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=791223"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=791223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}