Seems like proctored, handwritten essay exams are the best way to deal with it all.
Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
So the problem here is degree inflation. 20 years ago when I was getting my first master's degree it was a 5 years of in person classes. In the meantime degrees became required for lots of positions. Teachers need to get additional degrees to get promotions, you need a degree to apply for a job in the public sector and so on. So what used to be a proof of your qualifications became just a certificate you need to get. Now I'm getting my second master's degree and it's a year of online classes, without any in person evaluation. We will either start distinguishing between bullshit degrees from online universities and real ones or all of them will lose any value.
I saw rants very much like yours 30 years ago - before you first masters...
A masters has always to my knowledge been one additional year after you have a 4 year degree. So that they only require 1 year for the second degree is expected - you already have a 4 year degree, a masters is one additional year of advance study on top of it.
Yes, degree inflation is nothing new. I remember my uncle telling me how difficult his high school graduation exams were and how easy they got later.
We already had private school giving diplomas to politicians without attending any classes (I know about such instances in Spain and Poland). With AI getting some type of a degree will be just a formality for most people. No idea if it's an issue or not. If employers want to use degree as a proof for anything they will have to verify which ones mean something. If they don't care then I guess no one will.
The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
Doesn't scale.
It works great. That's basically how it's done in aviation, flight instruction is inherently 1 on 1, so I'd assign my students to do some reading and then it was pretty obvious if they'd done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say "Yeah I read it" and I have to do the clarifying they didn't read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It's almost certainly the first flying job you'll get after earning a commercial certificate, and thus spend at least some time doing that job. You're not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Agreed 100%.
Every exam should have some oral component. Doesn't need to be the whole of one's grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can't otherwise pass.
I can think of a couple of rules that could prevent a lot of the more blatant cheating:
-
All computer generated coursework must have some form of change tracking enabled, thus the assessor can see the way the essay was written.
-
All computer generated coursework must come with an accompanying oral assessment where the student has to discuss their work and research methodology.
The former would make it difficult to copy directly from ChatGPT, as the lack of any editing would leave an obvious fingerprint
... And passing the latter would require a level of understanding that would be difficult to replicate if you hadn't done the research yourself.
Passing the latter is basically the same step as a tradesperson needs to complete to show they know what they are doing. It is completely reasonable and would also catch plagiarism and paying some other person to do the work in addition to ai.
remember these?
Yeah, it’s a made up problem. In my experience, generated by two factors: universities wanting to increase the number of tests for each subject, and wanting students to be experienced with using computers. Thus many tests become at home, with the computer, where cheating is soooooo easy.
Yeah, cheating with electronics is nothing new. Laptop and phones and internet has existed long before AI, and people could get outside help through messages to have people feed them answers.
Yet, schools dealt with it by either doing in class paper and pencil exams, or digital in class exams on locked down computers with no internet access and no desktop access with software like examsoft.
I've had even math teachers ban graphing calculators or provide use of in class graphing calculators well aware of notes that could be typed into them and programs made to solve answers before hand.
So all these current issues seem to be the result of either schools moving to take home, or letting people use personal electronics with full internet and file access during exams. Its like teachers lost the ability to say no to what is and isn't allowed during exams.
As someone who is terrible at project work but great at tests, maybe now is finally my time to get a degree. Because tests seem the only thing you can't cheat on now.
Reap what you've sown, assholes.
What are you on about? This is about how universities have a hard time managing AI use and misuse by students. Have they sown those AI tools?
The essence of an academic paper is that a student shows that they have learned all the goals that were set for that course or courses. If a student lets tgere paper be generated by AI, they can not prove that they have learned those goals. That is why universitues have (or should have) strict AI guidelines, so that students know which kinds of use are allowed and which are not.
This is a hard nut to crack. Of course you will see overfitting and underfitting until someone somewhere fimds a universal approach. I personally think AI-detection tools are shit, but you cannot fault universities for trying it. My university handles AI fraud manually, with a committe. That is very labor intensive. We only have a couple hundred students, but if you have thousands, it quickly becomes impossible.
A lot of unis give students free AI access. Companies like Microsoft bundle Copilot into their education packages, so that students will graduate without learning to write essays on their own. Then they'll be working, paying professionals who can't go without AI because they didn't actually earn their degrees. It's in the business model.
I thought that they were trying to get at the hypocrisy of using ai tools to detect ai writing, which has a dismal success rate and ends up with a crazy amount of false positives.
I see that the growth of computer/digital testing directly led to student abuse of ai. Once students were no longer called on to write a blue book or even just generate responses to short answer questions, we failed our students. I understand that term papers and larger writing assignments have their place, but there really is nothing like being able to sit down and show your knowledge in an on the spot essay (at least for a lot of humanities courses where I see ai being abused).
A systemic issue in this space comes from universities and licensing entities giving testing companies carte blanche to race to the bottom in terms of effective testing tools. For licensing, apprenticeship/mentorship and observation are always better indicators of on the job performance, but that doesn't have the ease and profit margins of big bullshit testing, so companies would never go for it unless states mandated such pathways. With regard to a lot of current digital university testing, a lot of that was Frankensteined from existing licensing exam tools/strategies and similarly does not effectively assess student/candidate aptitude.
Even if this was not the intent of the commentor, my argument is that carelessness and disingenuous/gatekeeping testing practices from universities and licensing institutions certainly bred a significant level of disengagement in students which leads to them phoning it in with ai slop. If we wanted rigorous work from our students, we should have kept the same standards for ourselves, collectively.
You pretty much got it. I'm a little surprised more didn't tbh.
Can you elaborate?
Universities started pushing AI as a learning tool. They pay for subscription AI services for enrolled students. So I'm not sure this isn't them doing this to themselves.
These places that went all in on hyped up "AI" and dystopian surveillance can take their lumps. They're "realizing" what they've been told from day one.
Happy cake day!