Alexander Macris on Model Collapse

Model Collapse
Image: RylanSchaeffer

It’s the final 2 days for our Lord of Fate campaign. Don’t miss your last chance to claim exclsusive rewards and unlock the Burned Book in audio. Back it on Kickstarter now.


A concept that’s proven illuminating for those just learning about it is the idea of model collapse. The term originated in the field of machine learning to describe the information degradation that happens when A.I. algorithms start learning from each other too much and fall into incestuous feedback loops. But it can happen in any system with a heavy reliacne on models.

That was one of the more fascinating topics I discussed last night with top roleplaying game designer Alexander Macris. As he pointed out, the decline we see across the board in places like Hollywood and oldpub fits the pattern of model collapse. Most writers of Current Year entertainment product are so far removed from real-world experience that their collective output is now just a degrading series of Xeroxes.

Copy of a Copy - Fight Club
Screencap: 20th Century Fox

Even more interesting is Alexander’s theory that tabletop roleplaying games may offer a way out of the model collapse death spiral–at least in entertainment. TTRPG adaptations were once considered low-quality product. That stigma arose in the 80s and 90s when more degrees of separation stood between the state of gaming and writers like Tolkien and Hemingway who actually had real-life adventures. Time and cultural drift have now left most writers more distant from IRL experience than gamers, who at least simulate high-stakes adventures.

Alexander and I get into all of that–plus many other subjects, including his endorsement of neopatronage, on the stream. Catch the replay here:

And be advised: The crowdfunding campaign for my new dark fantasy novel Lord of Fate has just 2 days left. Don’t miss your last chance to get exclusive rewards and unlock the audio version of my previous thrilling novel The Burned Book.

BB Audio Mockup Shadow

Time is running out, so don’t wait. Back it on Kickstarter now.

15 Comments

  1. bayoubomber

    We didn’t need AI or technology to prove that point that having a closed loop data driven system was a bad idea.

    When your references are too old or too stale to use as inspiration, then reconnecting with real world is the best solution. I hear it all the time from my art professors and I think this is the key to why modern creativity is garbage and our works are made of a never ending chain of cult references (if I see one more motorcycle slide, I’m going to lose it).

    TTRPGs are a more obvious choice to escape stale entertainment because of how random they can be. No model can predict the exact roll of a dice or how a player may react. The randomness reflects real life and keeps things fresh.

    However, if you’re someone like me who is a cave dweller, doesn’t like to “experience life” all that often, I keep my reference folder fresh by deep diving into history. Not just topics of the past, but recent things too. My favorite is hearing about forgotten history or little known history. There’s no excuse for creatives to lack fresh irl-based influence in their works. The information is out there, you just need to look for it.

  2. Eoin Moloney

    Miyazaki complained about anime falling into this same self-referential rut, but I think in that case he’s being a little unfair. He’s an elder statesman of his art who has time to go out and experience the vibrancy of life, while most of the people in the field are forced into work schedules that would make an assembly line blush. It’s hardly a wonder that slaves who barely have the right to sleep have an impoverished experience of life.

    • bayoubomber

      This is another point I was thinking of when I was writing my main post. Even in America, we work too hard for too little. If people do go out to “experience life”, it’s usually going to the club, the bar, or just vegging out to watch Netflix.

      Brian talks infrequently about the value of leisure. I think if/when we reform our labor practices in this country, we will see another creative Renaissance. Less energy spent slaving away means more energy toward creative pursuits.

  3. Eoin Moloney

    I concur with BayouBomber’s opinion on history. As an autistic person, I’m more obsessed with regularity and routine than most people, but learning about other people and cultures has been a tremendously useful source of inspiration (as well as showing that ridiculous nonsense and stupid mistakes happen way more often in reality than fiction)

  4. Andrew Phillips

    As it happens, I’ve got an academic paper on the subject from May of last year sitting on my desk right now. A model collapse indicates there’s no real learning happening. In the case of an LLM, that’s to be expected. AI is 100% artifice and 0% intelligence, so of course the “AI” can’t notice its inputs, or outputs, are gibberish. The phenomenon illustrates the “GIGO” principle very well. However, since people can learn, noticing they are in the downward spiral of a model collapse raises the question “why not?” As I think about it, the IP Death Cycle and the RLM observation “consume product and get excited for next product” merry-go-round may be two sides of franchise model collapse in action.

    Have deeply woke (TT)RPG companies like White Wolf and Sorcerers by the Sea begun to demonstrate that sort of model collapse?

    On the other side of things, when repetitive processes, which in a sense take their outputs as inputs, do not experience model collapse, why keeps them vital?

    • LLMs and the Entertainment-Financial Complex are like the Bourbons: Forgetting nothing and learning nothing.

  5. b3k

    Baudrillard sees the same things for all media. LLMs trained on LLM output are, in his terms, “third order simulacra” that are “hyperreal” and near no necessary relation to the real world. Modern D&D is similarly hyperreal, being based on prior D&D rather than on real world experience or on fiction by authors with such experience.

    • Matthew Martin

      “Modern D&D is similarly hyperreal, being based on prior D&D rather than on real world experience or on fiction by authors with such experience.”

      This has been a design goal for a quarter-century:

      “In 2nd Ed, the rules referred to history and to historical legends to describe the game, such as referring to Merlin to explain what a wizard was or to Hiawatha as an archetype for a fighter. But by the time we were working on 3rd Ed, D&D had had such a big impact on fantasy that we basically used D&D as its own source. For example, 2E took monks out of the Player’s Handbook, in part because martial artist monks have no real place in medieval fantasy. We put them back in because monks sure have a place in D&D fantasy. The same goes for gnomes. The 3E gnome is there because the gnome was well-established in D&D lore, not in order to represent real-world mythology.”–Jonathan Tweet, Lead Designer of 3rd Edition D&D and fervent supporter of Planned Parenthood, https://www.enworld.org/threads/3e-and-the-feel-of-d-d.667269/

      • Rudolph Harrier

        Also clearly established in D&D before 3rd edition:

        -Players can rule kingdoms and lead vast armies.
        -Multiple parties can coexist in the same campaign world, and will probably be in competition with each other, maybe even outright coming to blows.
        -Things like spell research and magical item creation are time consuming processes which must be done at the exclusion of adventuring (though the rest of your party will probably still adventure without you.)
        -In order to accommodate all of the above, strict time keeping is essential.

        But 3rd edition provides no support for any of this. The absence of any discussion of timekeeping beyond the combat round is particularly notable, since even BECMI went into that and Gygax certainly thought it was of paramount importance.

        This is a phenomenon that happens a lot in clone degeneration. Through the process of copying the core of the original is lost. Look closely at 3rd edition: when you consider the actual mechanics, focus in character development, the assumed flow of the campaign, etc., it’s basically a completely separate game from 1st edition AD&D. When this happens people start focusing on cosmetic features of the minor and acting like they were essential, so that they don’t have to admit that the copy of the copy is not a true successor. Hence “but we’ve got monks and gnomes!” rather than worrying about the game at large.

        • Eoin Moloney

          Wow, “3e sucks and ruined D&D”? You, Sir, have attained grognard levels that I thought purely the stuff of myth.

          • Rudolph Harrier

            What I am claiming is this:

            -3rd edition is essentially a completely different game than 1st edition. In contrast, 1st edition is natural evolution of the original D&D rules. For completeness 2nd edition maintains the appearance of the D&D game, but gives the bare minimum of support to make it work and 2nd edition books are generally full of terrible advice.
            -In order to pretend that 3rd edition is not a new game, it has to double down on an identity of “D&D”. I.e. “we will throw in this thing because it was in a previous edition,” but only when it comes to the superficial. In contrast 1st edition is explicitly based on fiction of the time, primarily that of Vance, Howard, Leiber and Anderson. (For completeness, 2nd edition tried to replace that fiction with references to real world history and mythology, but it never really took.)

            All of this is obvious if you just read the 1st edition core books and then read the 3rd edition core books.

            Whether 3rd edition is a good game is another question; since it has a completely different aim than 1st edition it has to be evaluated differently. It is a different game though, as much as Tunnels and Trolls, Palladium Fantasy, etc. are. But since D&D is primarily understood these days as a brand, not as a game, this is difficult to see unless you really do go back and compare the sourcebooks.

          • Matthew Martin

            Myth? That reaction against 3rd Edition, especially with the 3.5 revision and the growing experience that the game didn’t play the same way when people turned their minds to taking full advantage of the mechanics, is what created the Old School Renaissance. 4E accelerated it, but the seeds were already there and starting to sprout in the mid-2000s.

  6. Rudolph Harrier

    I am reminded of 20th Century Boys, which at its core is about nostalgia and imitation.

    Several characters say a variation of the following:

    “No one cares about the original, the copy is lucky to break even. The one that makes money is the copy of the copy.”

    If we look at RPGs, we have the following for D&D:

    The original: OD&D and 1st edition AD&D. Moderately successfully, but now ignored even by the majority of RPG players.
    The copy: 2nd edition AD&D: Did well but not well enough to save TSR, somewhat remember but mainly as an “also ran.”
    The copy of the copy: 3rd edition D&D: Became a money printing machine for the Sorcerers by the Sea, and defines what people think of as “RPGs” to this day.

    20th Century Boys also explores what comes when everything before the copy of the copy is wiped away: total collapse and lack of meaning.

    What I find most interesting is that while the series is heavily centered on nostalgia (for 1960’s Japan), every good character disapproves of the younger generation mindlessly copying the 60’s (or worse, engaging with copies of copies of the 60’s) and instead encourages them to appreciate was good about that time, but create their own new meaning.

    Getting back to D&D again, note that the part of the OSR that amounts to “just copy what B/X did” or something similar is constantly falling apart. The part that is using Gygax’s philosophy to power player driven games (including in games outside of D&D) is where the real growth is.

Leave a Reply

Your email address will not be published. Required fields are marked *