Potential Applications and Implications of GPT-4 in Legal Inference Using Korean Legal Aptitude Test (LEET)

In November 2022, OpenAI released ChatGPT, a conversational AI chatbot, which caused a global sensation. The most recent model, GPT-4 is expected to replace experts in various legal tasks, including interpreting legal documents and drafting contracts, due to its impressive performance enhancements, such as achieving top 12\% on LSAT. This study aims to objectively prove the concerns surrounding GPT-4 through experiments. To achieve this, the study confirmed the results of the LSAT experiment announced by OpenAI and evaluated the potential impact of GPT-4 on the legal field by analyzing the problem-solving of LEET, a legal aptitude test similar to LSAT, in Korea. The study conducted various prompt design experiments to ensure that GPT-4's problem-solving is logical, valid, and able to draw accurate inferences. The study analyzed the problems that arose during GPT-4's inference process through this experimental method. The research results showed that GPT-4 can only perform superficial analysis when solving LEET test questions, has difficulty applying new facts, returns inconsistent answers to the same question, and often ignores implicit facts. Therefore, it is currently difficult to recognize GPT-4 as a rational inference model. However, it has potential to aid human inference in law by pre-training with legal data, incorporating expert feedback, and developing specialized legal technology.