Harvard Law School Professor Finds ChatGPT Invents Fake Law Less Than The Supreme Court

// ChatGPT has notoriously invented law in its desperate effort to satisfy the request of its masters. In this sense, the consumer-facing generative artificial intelligence tool isn’t dissimilar from Clarence Thomas or Sam Alito. Both are just word generators cherry-picking language from curated databases. ChatGPT uses the internet, Alito uses 17th century witchhunters.

Po-ta-to, po-tah-to.

googletag.cmd.push( function() { // Display ad. googletag.display( "div-id-for-top-300x250" ); });

But it’s possible that ChatGPT manages to stay within the guardrails of existing caselaw and reasoned argument better than the Court.

Sponsored Pioneering Ethical, Trustworthy Generative AI For Legal Research As a longstanding leader in legal AI, LexisNexis now pioneers ethical generative AI for legal research, enabling professionals to improve outcomes while upholding responsible AI… From LexisNexis  

My colleague @lessig “was literally astonished by this exchange with ChatGPT.”

Me too.

ChatGPT was both more forthright and more analytically astute about the tensions (and worse) in Supreme Court campaign finance law than most Justices have been.https://t.co/3kDkCySSrS

— Laurence Tribe 🇺🇦 ⚖️ (@tribelaw) November 28, 2023

Recently, Lawrence Lessig asked ChatGPT to delve into the world of campaign finance law:

googletag.cmd.push( function() { // Display ad. googletag.display( "div-id-for-middle-300x250" ); }); googletag.cmd.push( function() { // Display ad. googletag.display( "div-id-for-storycontent-440x100" ); }); googletag.cmd.push( function() { // Display ad. googletag.display( "div-id-for-in-story-youtube-1x1" ); });

The tool then went on to identify the key features of SuperPACs in a post-Citizens United world, noting that they have no contribution limits, a prohibition on direct coordination with candidates, donor disclosure requirements (even noting that these are laughably easy to circumvent through dark money schemes), and some state law variations.

Sponsored Sponsored Lexis+ AI: Generative AI Research That Cites Its Sources This new tool will change how you conduct legal research, offering a faster, more adaptable approach. From Ethan Beberness   Sponsored The Smartfirm Playbook - Thriving In Today’s Legal Landscape Strategically using software and automation to boost efficiency and streamline operations, allowing for a laser-focus on practicing law. From Lawmatics   Sponsored Pioneering Ethical, Trustworthy Generative AI For Legal Research As a longstanding leader in legal AI, LexisNexis now pioneers ethical generative AI for legal research, enabling professionals to improve outcomes while upholding responsible AI… From LexisNexis   Sponsored Lexis+ AI: Generative AI Research That Cites Its Sources This new tool will change how you conduct legal research, offering a faster, more adaptable approach. From Ethan Beberness  

But while the reasoning accurately reflected the caselaw, the opening answer didn’t quite hold up, prompting Lessig to ask the follow-up:

The system is probably tripped up by the fact that this conclusion makes no goddamned sense under the constitutional history of the United States prior to 2010.

googletag.cmd.push( function() { // Display ad. googletag.display( "div-id-for-bottom-300x250" ); });

There are a handful of principled defenders of Citizens United who try to spin the decision as sound to the extent that, post-Buckley, money equals speech so all bets have to be off. Putting aside whether or not this interpretation holds water, it also misses the point because arguably the most important question in the case was whether the government could limit this unchecked money from being spent on campaign ads within a 30-day window of an election, a provision that made the law little different — constitutionally — than a restriction on how many yards from a polling place someone could post signs.

Lessig is asking about the contributions half of the equation rather than spending half, though ultimately they feed into each other because the failure to check the former creates the corruption that required the original law limiting the latter.

Sponsored Sponsored LawPay Pro Offers Upgraded Time And Billing Essentials AffiniPay’s latest product for the legal community, introduced earlier this year, presents a simple web-based solution for attorneys seeking seamless firm cash flow. From Above the Law   Sponsored Managing a High Volume of Acquisitions with a Lean In-House Team How a veterinary services provider optimized contract workflows with Bloomberg Law Contract Solutions From Bloomberg Law  

At this juncture, Lessig poses the sort of empirical evidence that the current Supreme Court would dismiss as hitting too close to home for their comfort:

Remember when sci-fi heroes would defeat the evil computer by asking some sort of Gordian paradox? Here Lessig functionally asks ChatGPT to define love until steam emits from its ears:

ChatGPT remains fundamentally limited by its underlying knowledge base — something that tailored legal products will address — though this exchange underscores both the potential the engine has for integrating new input to improve its reasoning and the risk that could arise if it gets too tied to the corpus of existing caselaw. If legal AI endeavors to confidently deliver the “right” answer, it’s in jeopardy of parroting nonsense it gets from the Supreme Court, chilling the researcher from pushing further in the way Lessig, who is already an expert in the area, does here.

Garbage in, garbage out as they say. And we’re smack in the middle of an era with a lot of garbage going in.

Earlier: For The Love Of All That Is Holy, Stop Blaming ChatGPT For This Bad Brief
Clarence Thomas Skated On Ethics Complaints In 2012… Meaning He Definitely Already Knew The Rules
Sam Alito Laments It’s Getting So You Can’t Take All-Expense Paid Luxury Vacations Funded By Billionaires Anymore

Joe Patrice is a senior editor at Above the Law and co-host of Thinking Like A Lawyer. Feel free to email any tips, questions, or comments. Follow him on Twitter if you’re interested in law, politics, and a healthy dose of college sports news. Joe also serves as a Managing Director at RPN Executive Search.

Topics

ChatGPT, Courts, Harvard Law School, Lawrence Lessig, Supreme Court, Technology