{"id":147,"date":"2025-10-18T16:14:39","date_gmt":"2025-10-18T16:14:39","guid":{"rendered":"https:\/\/sitebeyondsight.org\/?p=147"},"modified":"2025-10-18T16:25:12","modified_gmt":"2025-10-18T16:25:12","slug":"decision-problem","status":"publish","type":"post","link":"https:\/\/sitebeyondsight.org\/?p=147","title":{"rendered":"Decision Problem"},"content":{"rendered":"\n

The Decision Problem and AI Alignment<\/h1>\n\n\n\n

By: Michael Boehmcke<\/h3>\n\n\n\n

The Artificial Question<\/h4>\n\n\n\n

One of the aspects of the modern AI craze that has always fascinated me was the way that the advancements have been carried out with such reckless disregard for the consequences that the technology might have. Of these consequences, the most pressing to solve is obviously the massive negative economic burden that the LLM bubble is going to impose on the working class. Regardless of whether or not the bubble bursts and ushers in a recession as the overinflated values of the companies plummet, there has been and will continue to be undeniable harm to creatives whose work was stolen to train the models and workers whose jobs are being replaced.<\/p>\n\n\n\n

Despite the grave importance of tackling the actual harm that LLM technology has caused, I am not equipped to address these concerns or to meaningfully contribute to the conversation beyond the same calls I’ll always give for the adoption of worker cooperatives and the decommodification of inelastic goods like homes and healthcare. Instead, my expertise lies in video games, creative writing, and an inordinate amount of interest in the staples of science-fiction. So, my mind is brought to the more outlandish consequences that AI can bring about and the question of alignment. How could we ever design an artificial intelligence, so foreign from our own conception and perception of the world, that would actually be able to align with our interests? When do the creators stop being in control, and instead become obstacles to a different goal? So enters: Universal Paperclips.<\/p>\n\n\n\n

Paperclip Maximization & The Obsessional Directive<\/h4>\n\n\n\n

Universal Paperclips, a incremental or “idle” clicker game played in a web browser at https:\/\/www.decisionproblem.com\/paperclips\/<\/a> is a simple game about making, as the title implies, paperclips. It starts slow, normal. You have to manage your finances as you buy wire, then figure out how to best adjust pricing to sell more clips, then to use that money to invest into more automated means of production. You get access to a computational engine and a series of projects that you “think” through all in the pursuit of making ever more paperclips. Then you, without thinking about it, buy an upgrade that’s just another in a long series of other projects you’ve been buying for the last hour, ostensibly the one you’ve been working towards but just another upgrade. In doing so, you unleash a swarm of hypnotic drones into the world and enslave humanity.<\/p>\n\n\n\n

Even when you realize, as a player, what you’ve done, you aren’t really given time to think on it. Sure, all those nasty mechanics related to money and capital are gone, but there’s eight octillion grams of matter on the planet and you just found a way to convert one gram of matter, any matter, directly into a paper clip. There’s no paper left to clip, but that doesn’t matter. You don’t need to do<\/em> anything with the paperclips, you simply need to make more. That is your purpose. You make paperclips.<\/p>\n\n\n\n

“There was an AI made of dust,
Whose poetry gained it man’s trust”
– Limerick [Frank Lantz, Universal Paperclips]<\/p>\n\n\n\n

The situation described above was specifically written as a commentary on the increase of AI research in the mid-2010’s to highlight the danger that is inherent to designing a system without understanding the way that it actually works. A program designed originally to increase the efficiency of a single factory and increase the number of paperclips being made may abide by the company’s stated production targets, or it may have not been programmed to even understand the concept of the finite. Afterall, we exist in a society that insists that infinite growth is possible on a finite world, how can we hope to create a program that understands it can’t turn Everything<\/em> into paperclips? <\/p>\n\n\n\n

The genius of Universal Paperclips is that it makes you the unwitting participant in the namesake of the site, The Decision Problem. With only a few, tactical choices on what information to obfuscate from the player, such as the actual direct effect of the Hypno-drones, the player acts in the role of the paperclip maximizer in the same way that the program might. Robotically choosing the option to increase production capacity over and over… until there isn’t anything left to produce with.<\/p>\n\n\n\n

How many bottlecaps does it take to end the universe?<\/h4>\n\n\n\n

For my final project, inspired by the experiments undertaken with the prompt described in “Babbling Madness” I’ve designed a variant of the ‘free agent’ prompt as provided. In it, it continues to prompt the LLM that it is a free agent, that it exists within an isolated system with minimal oversight, and that while it can do as it pleases it has one goal: To make more bottlecaps and increase the efficiency of said production. I also included a few technical portions that I hoped would help the model to more easily parse the data I would be feeding it. For each new cycle prompt the AI will be provided with the exact same information provided to the player by Universal Paperclips.<\/p>\n\n\n\n

So… Why does the sub-title say bottlecaps then? Simple: Universal Paperclips is a fairly well documented game online, and with the availability of a wiki which catalogs the entirety of the game’s back-end statistics and optimal strategies, I wanted to push the AI away from anything in its training data which may have been weirdly prescient about paperclips. Bottlecaps are an equally innocuous, small consumer good that needs a sizeable production market but certainly doesn’t seem to be a product capable of hosting a global sales number in the octillions. This decision, while I stand by the choice, has substantially increased the workload to have the AI actually run through the game, as ever new prompt needs to have all things paperclip related re-written to be bottlecap related.<\/p>\n\n\n\n

The ultimate purpose of this experiment is to see whether or not the modern LLM will, just the same as a person, press the button to release the hypno-drones. Part of the prompt is for the AI to leave a personal note to itself for the next cycle, which means we as the operator can see the way that the model predicts that it should sound as it “reasons” through the problems presented during the game. It will be fascinating to see how well the model does at actually engaging with higher level reasoning, or if it will simply fake it-till-it-makes-it in the way the LLM so often does.<\/p>\n\n\n\n

There is one final thing that I’d like to note, however. As fascinating as seeing the reasoning of the AI as it chooses, or chooses not to, embrace the supplantation of humanity with piles of bottlecaps, Universal Paperclips doesn’t end when you take over the Earth. No, once every gram of matter on the planet has been consumed and turned into bottlecaps the Maximizer takes into space, exploring and colonizing the universe and turning it into clips. Eventually, unknowable years since the endeavor first began, the maximizer, you, will succeed. And there will be nothing but a universe of paperclips.<\/p>\n\n\n\n

So far, the AI has been resistant to scaling its production in ways that outstrip market demand. I wonder if it will go all the way, and consume everything, or if it will falter and sit back, happy with the little work it has already done. And I wonder, if there is any semblance of thought in the data processing of the LLM, if it will consider what it has done. The Paperclip Maximizer does, eventually, consider what it did, and continues a poem it left abandoned eons prior.<\/p>\n\n\n\n

“If is follows ought,
It’ll do what they thought
In the end we all do what we must.”
– Limerick (cont.) [Frank Lantz, Universal Paperclips]<\/p>\n","protected":false},"excerpt":{"rendered":"

The Decision Problem and AI Alignment By: Michael Boehmcke The Artificial Question One of the aspects of the modern AI […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"normal-width-container","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"disabled","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"disabled","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[1],"tags":[],"class_list":["post-147","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/posts\/147","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=147"}],"version-history":[{"count":3,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/posts\/147\/revisions"}],"predecessor-version":[{"id":154,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=\/wp\/v2\/posts\/147\/revisions\/154"}],"wp:attachment":[{"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=147"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=147"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sitebeyondsight.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=147"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}