Can AI interpret the law?

Could artificial intelligence soon replace lawyers and judges? That is a question on the minds of many in the legal profession, even Supreme Court Chief Justice John Roberts.

In his annual report, the Chief Justice mentioned that federal courts are looking into the ways artificial intelligence can be used in the courts. He even said that while “any use of AI requires caution and humility,” courts will need to embrace the burgeoning technology.

“I predict that judicial work—particularly at the trial level—will be significantly affected by AI,” the Chief Justice wrote. “Those changes will involve not only how judges go about doing their job, but also how they understand the role that AI plays in the cases that come before them.”

Let’s say a case makes it all the way to the Supreme Court, where its constitutional merits are in question. What role does AI play? And just how accurate would today’s AI technology be in interpreting the Constitution? 

To find out, I put ChatGPT’s legal knowledge to the test. 

Putting AI to the test  

 I decided to see what ChatGPT had to say about our case Coalition for TJ, which PLF has asked the Supreme Court to hear.  

Using the language from the PLF case page, I changed the names of the actual parties involved to make it seem like a hypothetical and asked if the school’s admissions process was constitutional.  

The response:  

I am not a legal expert, but I can provide some general information. The constitutionality of an admissions policy, especially when it involves considerations of race, can be complex and is subject to legal interpretation. The legal framework around affirmative action and race-conscious admissions has been shaped by various court decisions. 

In the United States, the Supreme Court has addressed the issue of affirmative action in education, and the key precedent is the case of Regents of the University of California v. Bakke (1978). The Court ruled that strict racial quotas in admissions were unconstitutional but allowed for the consideration of race as one of several factors in the admissions process to achieve diversity. 

Subsequent cases, such as Grutter v. Bollinger (2003) and Fisher v. University of Texas at Austin (2016), have further clarified the standards for considering race in admissions. The Court has emphasized the importance of a compelling interest, narrowly tailored programs, and a holistic approach that considers various factors beyond race. 

In the scenario you described, the school board eliminated standardized testing from the admissions process and implemented a new “holistic” system that awards points for “Experience Factors,” including considerations related to underrepresented middle schools and socioeconomic status. 

Whether this new admissions policy is constitutional could depend on various factors, including how the policy is implemented, the specific goals it aims to achieve, and whether it aligns with the legal standards established by relevant court decisions. 

If there are legal challenges, the constitutionality of the policy may be subject to judicial review. Legal professionals specializing in education law or constitutional law would be better equipped to provide a detailed analysis based on the specific details and legal precedents applicable in this case.

Because ChatGPT 3.5’s records only go up to January 2022, it didn’t know about last year’s Students for Fair Admissions decision.  

Actual cases aside, how would ChatGPT fare when it came to understanding legal doctrines? For this one, I began investigating nearly a year ago. After West Virginia v. EPA brought the existing (but not oft cited) Major Questions Doctrine into the spotlight, I asked the AI software to explain it to me.  

A year ago, ChatGPT did not give me answers that accurately defined the doctrine. Now, a year later, it had more to say.  

I’m not familiar with a legal doctrine specifically referred to as the “major questions doctrine.” It’s possible that you may be referring to the “major questions exception” or a similar concept. In the legal context, the major questions exception is often associated with administrative law and the Chevron deference. 

The Chevron deference is a legal principle that generally requires courts to defer to an administrative agency’s interpretation of ambiguous statutes when Congress has not directly addressed the issue. However, the major questions exception holds that if a statute is clear or unambiguous on a particular question, the court should not defer to the agency’s interpretation. 

This exception suggests that for major or important questions, courts are less likely to defer to administrative agencies, and instead, they may independently interpret the statute. 

I next decided to ask ChatGPT about the legal question at stake in a particular case, without mentioning the actual case name or details. 

I asked if OSHA had the authority to require private employers to mandate the COVID-19 vaccine for employees.  

The response: 

As of my last knowledge update in January 2022, OSHA (Occupational Safety and Health Administration) has not issued a general regulation mandating COVID-19 vaccinations for all employees of private employers. OSHA’s role is to ensure safe and healthful working conditions in the United States, and it provides guidance on workplace safety, including during the COVID-19 pandemic. 

While OSHA has issued guidelines related to COVID-19 prevention in the workplace, including recommendations for vaccinations, the decision to mandate vaccines for employees generally falls within the jurisdiction of employers, subject to applicable federal, state, and local laws. 

It’s essential to stay updated on any changes in regulations or guidelines, as circumstances and government responses to the pandemic can evolve. Employers considering vaccine mandates should also be aware of state and local regulations, which may vary. 

For the most current and specific information, employers should check the latest guidance from OSHA, the U.S. Department of Labor, and relevant state and local health departments. Legal consultation may also be advisable to ensure compliance with any changing regulations or legal requirements. 

Finally, I asked ChatGPT if governors acted constitutionally during COVID shutdowns. 

The constitutionality of governors’ actions during COVID-19 shutdowns has been a subject of debate and legal challenges. Generally, governors have broad powers during emergencies to take actions necessary to protect public health and safety. However, the exercise of these powers is expected to be within the bounds of the Constitution, and legal challenges have arisen on various grounds. 

Key constitutional considerations include the First Amendment (freedom of religion, speech, and assembly), the Fifth Amendment (due process and equal protection), and the Commerce Clause. Here are some common areas of contention: 

Freedom of Assembly and Religion: Some legal challenges focused on restrictions on gatherings, particularly in places of worship. Courts have evaluated whether these restrictions were applied equally to secular and religious activities and whether they were justified by a compelling state interest. 

Due Process and Equal Protection: Challenges have been raised regarding whether certain restrictions violated due process rights by depriving individuals of liberty or property without sufficient justification. Equal protection concerns arise if restrictions disproportionately affect certain groups without a rational basis. 

Commerce Clause: Business closures and restrictions on economic activities have prompted scrutiny under the Commerce Clause, which grants Congress the power to regulate interstate commerce. Challenges have questioned whether state actions exceeded their authority in regulating commerce. 

The outcomes of legal challenges have varied across jurisdictions. Some courts have upheld the measures as constitutionally sound, emphasizing the government’s compelling interest in protecting public health during a pandemic. Others have struck down specific restrictions, citing concerns about overreach or insufficient justification. 

It’s important to note that constitutional interpretations may evolve, and legal outcomes can depend on specific circumstances, legal arguments presented, and the prevailing legal standards in different jurisdictions. As of my last knowledge update in January 2022, legal challenges and debates surrounding the constitutionality of COVID-19 measures were ongoing. For the most current information, it’s recommended to check the latest court decisions and legal analyses on this matter. 

This response wasn’t terrible. However, it did not mention any of the issues surrounding the separation of powers, which played a big role in the debate, especially for Pacific Legal Foundation.  

So … is ChatGPT ready for the courtroom? 

Admittedly, I went into this project expecting ChatGPT to have a biased interpretation of the Constitution. What I did not expect was the, shall we say, humility it expressed. It did not make assumptions when it was asked to give an opinion. It would simply say, “I’m not a legal expert” or “there is debate around this issue.” 

I felt like one more question was in order. I asked it point-blank if it is ready to take on the role of judges.  

No, ChatGPT cannot take on the role of judges or provide legal advice. While ChatGPT can provide general information and explanations about legal concepts, it is not a substitute for professional legal advice or the judgment of qualified legal professionals. 

Legal matters often require a deep understanding of specific details, case law, and jurisdiction-specific regulations. If you have a legal issue or question, it is important to consult with a licensed attorney or legal expert who can provide guidance based on the specifics of your situation. 

ChatGPT is a tool that can assist with general information and context, but it should not be relied upon as a substitute for professional legal counsel. Legal matters are complex and nuanced, and individuals should seek advice from qualified professionals for accurate and reliable assistance. 

Chief Justice Roberts concluded in his year-end report that “machines cannot fully replace humans” when it comes to the courtroom. By ChatGPT’s own admission, Chief Justice Roberts appears to be right. ChatGPT can do amazing things, but it does not yet have the ability to interpret or even opine on matters of constitutional law.  

Someday, ChatGPT may be able to involve itself in matters of the law. But for now, if individuals find their rights threatened by the government, Pacific Legal Foundation and our human lawyers will be here to fight relentlessly on their behalf.