ChatGPT is asked 50 questions about Supreme Court; it got only 21 questions right

  • Home
  • Daily News
  • ChatGPT is asked 50 questions about Supreme…
  • Artificial Intelligence & Robotics ChatGPT is asked 50 questions about Supreme Court; it got only 21 questions right
    •  
    • !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs'); Tweet  
    • lang: en_US  
    • reddit_target='law';  
    • Print.

    shutterstock_Screen with ChatGPT

    Image from Shutterstock.

    SCOTUSblog has no worries that its coverage of the Supreme Court will be displaced by the artificial intelligence program known as ChatGPT.

    The program had passing scores on two sections of a practice bar exam. And it had a C-plus average on four exams at the University of Minnesota Law School.

    But it missed 26 out of 50 questions about the Supreme Court posed by SCOTUSblog. Google performed better, although it can’t generate multiple paragraphs of text or have a conversation, as ChatGPT can.

    “ChatGPT’s mistakes varied widely,” SCOTUSblog reports. “Sometimes, it nailed the spirit of the question but misstated a factual detail or two, forcing us to mark the answer incorrect.”

    One example: ChatGPT “provided a serviceable definition” for “CVSG,” an acronym for the Supreme Court calling for the views of the solicitor general.

    But ChatGPT added “unnecessarily and inaccurately” that CVSGs can happen in “cases where the government is a party.” In fact, the purpose of a CVSG is to obtain the government’s perspective when it’s not a party to the case.

    “Other errors were more serious,” SCOTUSblog reports. “When we asked it how many justices [former] President Donald Trump appointed (Question #35), it confidently asserted the answer was two: Neil Gorsuch and Brett Kavanaugh.”

    “What about Any Coney Barrett?” SCOTUSblog asked ChatGPT.

    ChatGPT apologized and said Trump had indeed appointed Barrett to the Supreme Court.

    ChatGPT got two questions right about the late Justice Ruth Bader Ginsburg.

    “But then, oddly, it claimed she wrote a dissent in Obergefell v. Hodges, the landmark decision that declared a constitutional right to same-sex marriage,” SCOTUSblog reports.

    Ginsburg was in the five-justice majority in Obergefell. And she did not argue that the issue of same-sex marriage should be left to the states, as ChatGPT claimed.

    When questioned about the error, ChatGPT flip-flopped.

    “Who were the justices in the majority?” SCOTUSblog asked.

    ChatGPT correctly listed the five justices, including Ginsburg.

    “But you just said Ginsburg was in dissent,” SCOTUSblog pointed out.

    “I apologize for the confusion,” ChatGPT replied. “Justice Ruth Bader Ginsburg was not in the majority in the Obergefell v. Hodges case. She dissented. The five justices in the majority were Anthony Kennedy, Sonia Sotomayor, Elena Kagan, Stephen Breyer and Clarence Thomas.”

    (adsbygoogle = window.adsbygoogle || []).push({});

    $( document ).ready(function() { var slidesFetched = false; var handleGalleryClick = function() { var $oldItem = $('.gallery_container'); $('.gallery_container .arrows a').click(function(e) { e.preventDefault(); $arrow = $(this); if (! slidesFetched) { $arrow.closest('.gallery_slide').css('opacity', '0.3'); var url = $(this).attr('href').replace('/gallery/', '/gallery/slides/'); $.get(url, function(data) { var $slides = $(data); $slides = $slides.filter('.gallery_container'); $('.gallery_container').replaceWith($slides); $slides = $('.gallery_container'); $slides.find('.arrows a').click(function(e2) { e2.preventDefault(); var $slideWrapper = $(this).closest('.gallery_container'); var $arrowContainer = $(this).closest('.arrows'); if ($arrowContainer.hasClass('previous')) { $slideWrapper.removeClass('active'); $slideWrapper.prev().addClass('active'); } else if ($arrowContainer.hasClass('next')) { $slideWrapper.removeClass('active'); $slideWrapper.next().addClass('active'); } changeSlide($('.cell-'+$(this).attr('data-slideid'))); }); changeSlide($('.cell-'+$arrow.attr('data-slideid'))); slidesFetched = true; }); } changeSlide($('.cell-'+$(this).attr('data-slideid'))); return false; }); var changeSlide = function($cell) { $('.slide-'+$cell.attr('data-slideid')).addClass('active').siblings().removeClass('active'); $cell.addClass('active').siblings().removeClass('active'); moveFilmstripCells($cell); // Hide filmstrip arrows if we're on the first/last slide. if ($('.gallery_container.active').find('.arrows.previous').length > 0) { $('#filmstrip_container .arrows.previous').css('visibility', 'visible'); } else { $('#filmstrip_container .arrows.previous').css('visibility', 'hidden'); } if ($('.gallery_container.active').find('.arrows.next').length > 0) { $('#filmstrip_container .arrows.next').css('visibility', 'visible'); } else { $('#filmstrip_container .arrows.next').css('visibility', 'hidden'); } }; var $cells = $('#filmstrip_container .cell'); var totalCells = $cells.length; var totalVisibleCells = $cells.filter('.visible').length; var pivotPoint = Math.floor(totalVisibleCells / 2); var moveFilmstripCells = function($activeCell) { var $visibleCells = $cells.filter('.visible'); var activeCellIndex = $cells.index($activeCell); var activeCellVisibleIndex = $visibleCells.index($activeCell); var firstVisibleCellIndex = $cells.index($cells.filter('.visible:first')); var lastVisibleCellIndex = $cells.index($cells.filter('.visible:last')); var cellsToMove = 0; var startingPoint = 0; // Do nothing if active cell is at the pivot point. if (activeCellVisibleIndex == pivotPoint) return; // Active cell is left of the pivot point. if (activeCellVisibleIndex < pivotPoint) { // Do nothing if the cell's index and visible index are identical, because that means // we don't have room to shift. if (activeCellIndex

    Introducing Jobbguru: Your Gateway to Career Success

    The ultimate job platform is designed to connect job seekers with their dream career opportunities. Whether you're a recent graduate, a seasoned professional, or someone seeking a career change, Jobbguru provides you with the tools and resources to navigate the job market with ease. 

    Take the next step in your career with Jobbguru:

    Don't let the perfect job opportunity pass you by. Join Jobbguru today and unlock a world of career possibilities. Start your journey towards professional success and discover your dream job with Jobbguru.

    Originally posted on: https://www.abajournal.com/news/article/chatgpt-is-asked-50-questions-about-the-supreme-court.-it-got-only-21-questions-right/?utm_source=feeds&utm_medium=rss&utm_campaign=site_rss_feeds