this post was submitted on 13 Aug 2023
783 points (97.7% liked)

Technology

58123 readers
4476 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

College professors are going back to paper exams and handwritten essays to fight students using ChatGPT::The growing number of students using the AI program ChatGPT as a shortcut in their coursework has led some college professors to reconsider their lesson plans for the upcoming fall semester.

you are viewing a single comment's thread
view the rest of the comments
[–] HexesofVexes@lemmy.world 105 points 1 year ago (56 children)

Prof here - take a look at it from our side.

Our job is to evaluate YOUR ability; and AI is a great way to mask poor ability. We have no way to determine if you did the work, or if an AI did, and if called into a court to certify your expertise we could not do so beyond a reasonable doubt.

I am not arguing exams are perfect mind, but I'd rather doubt a few student's inability (maybe it was just a bad exam for them) than always doubt their ability (is any of this their own work).

Case in point, ALL students on my course with low (<60%) attendance this year scored 70s and 80s on the coursework and 10s and 20s in the OPEN BOOK exam. I doubt those 70s and 80s are real reflections of the ability of the students, but do suggest they can obfuscate AI work well.

[–] maegul@lemmy.ml 7 points 1 year ago (32 children)

Here's a somewhat tangential counter, which I think some of the other replies are trying to touch on ... why, exactly, continue valuing our ability to do something a computer can so easily do for us (to some extent obviously)?

In a world where something like AI can come up and change the landscape in a matter of a year or two ... how much value is left in the idea of assessing people's value through exams (and to be clear, I'm saying this as someone who's done very well in exams in the past)?

This isn't to say that knowing things is bad or making sure people meet standards is bad etc. But rather, to question whether exams are fit for purpose as means of measuring what matters in a world where what's relevant, valuable or even accurate can change pretty quickly compared to the timelines of ones life or education. Not long ago we were told that we won't have calculators with us everywhere, and now we could have calculators embedded in our ears if wanted to. Analogously, learning and examination is probably being premised on the notion that we won't be able to look things up all the time ... when, as current AI, amongst other things, suggests, that won't be true either.

An exam assessment structure naturally leans toward memorisation and being drilled in a relatively narrow band of problem solving techniques,^1^ which are, IME, often crammed prior to the exam and often forgotten quite severely pretty soon afterward. So even presuming that things that students know during the exam are valuable, it is questionable whether the measurement of value provided by the exam is actually valuable. And once the value of that information is brought into question ... you have to ask ... what are we doing here?

Which isn't to say that there's no value created in doing coursework and cramming for exams. Instead, given that a computer can now so easily augment our ability to do this assessment, you have to ask what education is for and whether it can become something better than what it is given what are supposed to be the generally lofty goals of education.

In reality, I suspect (as many others do) that the core value of the assessment system is to simply provide a filter. It's not so much what you're being assessed on as much as your ability to pass the assessment that matters, in order to filter for a base level of ability for whatever professional activity the degree will lead to. Maybe there are better ways of doing this that aren't so masked by other somewhat disingenuous goals?

Beyond that there's a raft of things the education system could emphasise more than exam based assessment. Long form problem solving and learning. Understanding things or concepts as deeply as possible and creatively exploring the problem space and its applications. Actually learn the actual scientific method in practice. Core and deep concepts, both in theory and application, rather than specific facts. Breadth over depth, in general. Actual civics and knowledge required to be a functioning member of the electorate.

All of which are hard to assess, of course, which is really the main point of pushing back against your comment ... maybe we're approaching the point where the cost-benefit equation for practicable assessment is being tipped.


  1. In my experience, the best means of preparing for exams, as is universally advised, is to take previous or practice exams ... which I think tells you pretty clearly what kind of task an exam actually is ... a practiced routine in something that narrowly ranges between regurgitation and pretty short-form, practiced and shallow problem solving.
[–] Spike@feddit.de 2 points 1 year ago* (last edited 1 year ago) (1 children)

In my experience, the best means of preparing for exams, as is universally advised, is to take previous or practice exams … which I think tells you pretty clearly what kind of task an exam actually is … a practiced routine in something that narrowly ranges between regurgitation and pretty short-form, practiced and shallow problem solving.

You are getting some flak, but imho you are right. The only thing an exam really tests is how well you do in exams. Of course, educators dont want to hear that. But if you take a deep dive into (scientific) literature on the topic, the question "What are we actually measuring here?" is raised rightfully so.

[–] maegul@lemmy.ml 0 points 1 year ago* (last edited 1 year ago) (1 children)

Getting flak on social media, through downvotes, can often (though not always!) be a good thing ... means you're touching a nerve or something.

On this point, I don't think I've got any particularly valuable or novel insights, or even any good solutions ... I'm mostly looking for a decent conversation around this issue. Unfortunately, I suspect, when you get everyone to work hard on something and give them prestigious certifications for succeeding at that something, and then do this for generations, it can be pretty hard to convince people to not assign some of their self-worth to the quality/value/meaning of that something and to then dismiss it as less valuable than previously thought. Possibly a factor in this conversation, which I say with empathy.


Any links to some literature?

[–] Spike@feddit.de 2 points 1 year ago (1 children)

Used only papers in german so far, sadly.

Here is something I found interesting in english:

Testing the test: Are exams measuring understanding? Brian K. Sato, Cynthia F. C. Hill, S. Lo Biochemistry and Molecular Biology Education

in general: elicit.org

really good site.

[–] maegul@lemmy.ml 0 points 1 year ago (1 children)

Hadn't heard of that elicit cite ... thanks! How have you found it? It makes sense that it exists already, but I hadn't really thought about it (haven't looked up papers recently but may soon).

Also thanks for the paper!!

[–] Spike@feddit.de 2 points 1 year ago (1 children)

Have found it relatively early after it was created, using it for getting a quick overview over papers when writing my own. It is sooo good for that.

[–] maegul@lemmy.ml 1 points 1 year ago

Yea cool. Thanks for this!

load more comments (30 replies)
load more comments (53 replies)