ChatGPT goes to University of Minnesota law school and passes final exams

[Eric]

We've known for a while that this technology has been growing by leaps and bounds, while not perfect it was close enough to pass law school final exams and it sounds like institutions are concerned about students taking advantage of it.

ChatGPT goes to University of Minnesota law school and passes final exams

"What it did was pretty remarkable and well above and beyond what I would’ve anticipated four or five months ago," said University of Minnesota Law School professor Daniel Schwarcz.

www.fox9.com

ST. PAUL, Minn. (FOX 9) - The artificial intelligence application ChatGPT went to law school and passed the final exams.

ChatGPT attended law school at the University of Minnesota and passed all four final exams it took, but it didn’t ace them. The chatbot finished behind all or most of the humans who took the same tests, but it did well enough for professors to wonder whether an improved version might displace some future lawyers.

The ChatGPT impressed its four professors at the University of Minnesota Law School even while earning just a C+ average on its final exams.

"What it did was pretty remarkable and well above and beyond what I would’ve anticipated four or five months ago," said University of Minnesota Law School professor Daniel Schwarcz, one of the four professor experimenters along with Jonathan Choi, Kristin Hickman, and Amy Monahan.

 

[lizkat]

We've known for a while that this technology has been growing by leaps and bounds, while not perfect it was close enough to pass law school final exams and it sounds like institutions are concerned about students taking advantage of it.

I dunno. Like all developing technologies, ChatGPT et al software from contending groups will have their ups and downs. OpenAI, the outfit that has brought us ChatGPT, has a new tool that can be used to analyze a sampling of text and assess whether it was written by humans or not. Apparently though, it took a look at a bit of the Bible's Book of Genesis and concluded that it was written by a bot. Heh. Lightning bolts might still be in the offing...

​

And God was mightily offended by the insults lain at the feet of His talented human scribes by their descendants​

And God said: next time I reboot this thing I'm gonna stop before I even create the amphibians. ​

For those interested, the bit about the OpenAI tool reference was in a piece in The Atlantic

ChatGPT Is About to Dump More Work on Everyone

Artificial intelligence could spare you some effort. Even if it does, it will create a lot more work in the process.

www.theatlantic.com

 

[Chew Toy McCoy]

Considering what some idiots google search before committing their crime, this technology could lead to some interesting outcomes.

 

[lizkat]

Considering what some idiots google search before committing their crime, this technology could lead to some interesting outcomes.

I've always wondered about how trainers decide to let specific AI applications "learn" stuff off the net. Letting them look at old google search caches would be one of the "lessons" that I'd put a lot of guardrails around in advance. Every so often some media outlet does a roundup and prints up a comic relief piece of its findings. But god forbid some sophomoric AI bot is allowed to read stuff like that and just figure oh ok I think I get it.

 

[Eric]

Considering what some idiots google search before committing their crime, this technology could lead to some interesting outcomes.

Threw this question at it just to see how it would respond, it took its time in writing it but seemed to nail it.

Here's the link for reference

https://chat.openai.com/chat

 

[Chew Toy McCoy]

Threw this question at it just to see how it would respond, it took its time in writing it but seemed to nail it.

View attachment 21653

Here's the link for reference

https://chat.openai.com/chat

I was thinking more along the lines of "how to dispose of a human body" but I'll go ahead and let you ask that one.

 

[lizkat]

I was thinking more along the lines of "how to dispose of a human body" but I'll go ahead and let you ask that one.

Yeah those are the ones I would not like to let an AI creature be learning without some guardrails. Not even sure I'd want to clutter an AI's grasp of human culture with one of those funny collections of dumb searches like "why isn't 11 pronounced onety-one?"

It's hard to tell what use an AI might make of this or that material gleaned from the net... unless we ensure that adequate context is also absorbed, so that it learns more of the whole range of human intent behind any utterance one of us has made.

 

[Herdfan]

We've known for a while that this technology has been growing by leaps and bounds, while not perfect it was close enough to pass law school final exams and it sounds like institutions are concerned about students taking advantage of it.

In 5 years will a student ever write a term paper on their own?

 

[lizkat]

In 5 years will a student ever write a term paper on their own?

The people who wrote students' papers for $$$ before the days of chatty bots are now doing what, I wonder. Working for Door Dash? or writing code for AI critters?

And as for the kids who bought those term papers... one hopes at least that they only bought papers for courses that were not germane to whatever occupation they landed in. You know.. the scientist whose paper on Iris Murdoch's fiction was pretty good but it's even better, all things considered, that knowledge of Murdoch didn't even remotely figure into his latest equations for calibrating some mirror on the next great space telescope. All his shortcut in English Lit 101 did was make him a really boring guy at parties...

But what if he became a teacher? A philosopher? At some point then those shortcuts will turn up, even if only in the next generation.

 

[Eric]

And we've come full circle.

 

[Chew Toy McCoy]

We know what the eventual ultimate outcome is. Black market Boston Dynamics robots that require less maintenance illegally crossing the border and replacing American Amazon distribution center bots.

 

[Alli]

In 5 years will a student ever write a term paper on their own?

Of course they will. Locked browsers, test monitoring, it will all work out.

 

[Eric]

Of course they will. Locked browsers, test monitoring, it will all work out.

You're describing all of my IT certification tests and most recently my drone pilot (Part 107) test as well. You are at their testing centers on locked down machines (some which you have to travel a long way to get to), you start by checking in all of your electronic devices, they make you show your empty pockets and a proctor is walking around the room the entire time watching.

 

[Cmaier]

You're describing all of my IT certification tests and most recently my drone pilot (Part 107) test as well. You are at their testing centers on locked down machines (some which you have to travel a long way to get to), you start by checking in all of your electronic devices, they make you show your empty pockets and a proctor is walking around the room the entire time watching.

Every law school uses Examsoft, which essentially makes you boot your laptop into a special OS that monitors everything you do, deactivates network connections, encrypts and time stamps your work, and produces a file you can upload once you boot back into your own OS. We used it for every final exam (and, in law school, the only exams are final exams) and for the bar exam.

I took the patent bar exam at one of those testing centers on their shitty computer, and at least with examsoft you can use your own machine. Of course, my machine froze in the middle of day 1 of the bar exam, but at least when I re-booted all my work was still there other than the last sentence I typed.

 

[Herdfan]

We used it for every final exam (and, in law school, the only exams are final exams) and for the bar exam.

Have they dumbed down the bar exam like they have the CPA exam?

Way back when I took it you had to go into an auditorium with a couple hundred other people, take all 4 sections over 2 days, get a 75% to pass a section, but also had to get 50% on all sections for the ones you did pass to count. Governmental killed me twice. It was held twice a year.

Now, you go to an exam center, you can take 1 section at a time and I think you can take the same section once a month. It would be so much easier to study for/pass one section at a time.

 

[Eric]

Have they dumbed down the bar exam like they have the CPA exam?

Way back when I took it you had to go into an auditorium with a couple hundred other people, take all 4 sections over 2 days, get a 75% to pass a section, but also had to get 50% on all sections for the ones you did pass to count. Governmental killed me twice. It was held twice a year.

Now, you go to an exam center, you can take 1 section at a time and I think you can take the same section once a month. It would be so much easier to study for/pass one section at a time.

I'll let @Cmaier comment here as he's the resident expert. However, I have a friend who is one of the smartest people I've ever had the pleasure of knowing and he failed it after studying his ass off and prepping for a long time. From what I've seen it's extremely challenging, long, drawn out and comprehensive.

 

[Cmaier]

Have they dumbed down the bar exam like they have the CPA exam?

Way back when I took it you had to go into an auditorium with a couple hundred other people, take all 4 sections over 2 days, get a 75% to pass a section, but also had to get 50% on all sections for the ones you did pass to count. Governmental killed me twice. It was held twice a year.

Now, you go to an exam center, you can take 1 section at a time and I think you can take the same section once a month. It would be so much easier to study for/pass one section at a time.

Every state does their own thing. In California, it looks like the most recent pass rate was 50%, which is a little higher than it was in the past. I know they loosened a few things up due to covid.

A bunch of states use the same bar exam (not california), though some also augment with their own additional stuff. Typically it takes 2 or 3 days, and you can try again every six months. Pass rate goes way down for people trying for the second or third time.

 

[Herdfan]

Every state does their own thing. In California, it looks like the most recent pass rate was 50%, which is a little higher than it was in the past. I know they loosened a few things up due to covid.

A bunch of states use the same bar exam (not california), though some also augment with their own additional stuff. Typically it takes 2 or 3 days, and you can try again every six months. Pass rate goes way down for people trying for the second or third time.

I was one year too late to get Diploma Privilege from WVU or I might have gone. But to be honest, 1L (Scott Turow) did a great job of scaring me away from going.

 

[lizkat]

Considering what some idiots google search before committing their crime, this technology could lead to some interesting outcomes.

At this point, a lot of corporations are more worried about what their employees are CONTRIBUTING via ChatGPT to what can be searched for about their operations.

Companies are looking for ways to keep their corporate secrets out of ChatGPT

ChatGPT is lacking in transparency over user data, where it goes or how it could be used.

www.axios.com

 

[Eric]

At this point, a lot of corporations are more worried about what their employees are CONTRIBUTING via ChatGPT to what can be searched for about their operations.

Companies are looking for ways to keep their corporate secrets out of ChatGPT

ChatGPT is lacking in transparency over user data, where it goes or how it could be used.

www.axios.com

This has to be a huge security issue around Intellectual Property, many of the companies I've consulted with have entire departments dedicated to protecting it, consisting of experts in their respective fields and attorneys.

At the very least we'll likely see it blocked at a firewall level for corporations but now that a majority of workers are in a home office there's only so much they can do to stop it. I see education and policies as key here, they'll really need to work with users so they fully understand the consequences.