Schools and universities in particular face a challenge when dealing with ChatGPT. In the USA, professors from two universities have now examined how well the chatbot performs in their exams – and what opportunities can be derived from it.
Since “ChatGPT” was published at the end of November, in addition to the fascination, uncertainty has also grown among many. With the chatbot developed by the company “Open AI”, all kinds of questions can be answered and even complex tasks can be solved. At first glance, the resulting texts can hardly be distinguished from those of a real person.
This poses a number of challenges, especially in the field of education. Many schools and universities are unsure about how to use the technology and what impact it could have on teaching. In New York City, for example, the Ministry of Education decided just a few weeks ago that the chatbot can no longer be used within the school network and on school devices.
ChatGPT passes exams at US universities – but it also has its limits
In the USA, professors at the University of Minnesota and the University of Pennsylvania have examined how ChatGPT performs on their exams. At the University of Minnesota, four law professors wanted to find out whether and how well the chatbot can pass law exams. To do this, they had him write four regular exams and graded them blindly along with those of their students. On Wednesday they published a short research report titled “ChatGPT goes to law school”. The result: ChatGPT passed the exams, in which 95 multiple-choice questions and 12 essay questions had to be answered, with an average grade of C. On a grading scale where A is considered the best grade and F is considered a fail, this is a lower middle grade.
At the Wharton Business School of the University of Pennsylvania, which as a so-called “Ivy League University” is one of the eight best private universities in the USA, ChatGPT achieved even better results in the grade range B to B-. According to the published report, the program was able to answer simpler management and process analysis questions very well. When it comes to more complicated questions in this area, however, it reaches its limits. Surprisingly, the AI would have struggled with calculations at sixth grade math level.
ChatGPT could support teaching
With regard to the consequences for teaching, the conclusions of the professors at both universities are similar: While the framework conditions for exams may have to be adjusted in future to make cheating more difficult, universities should also recognize the opportunities offered by AI. For example, they should think about their students’ collaborations with ChatGPT in certain areas and incorporate them into their teaching. According to Jon Choi, one of the law professors at the University of Minnesota, it is only a matter of time before AI assistants for lawyers would be used by default and universities should prepare their students for this.
Microsoft only announced on Monday that it wanted to invest ten billion dollars in the company “Open AI”. It also plans to make the chatbot available in its own cloud service soon.
Sources: CNN, CBS News, Vice
