this post was submitted on 24 Feb 2024
762 points (98.8% liked)
Technology
59402 readers
3048 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The students should get together and jack the machine away into their hacking club and do some reverse engineering, so that we get more information on how the data collection worked as opposed to just trusting the company's statements. If a hacking group like the German Chaos Computer Club got behind this, they could release their findings while keeping the perpetrators anonymous. However, I’m pretty sure the machine is just a frontend to a server, which got shut down as soon as the students complained, with no GDPR-like checkout being available in the jurisdiction.
When you start tinkering with a machine learning model of any kind, you’re probably going to find some interesting edge cases the model can’t handle correctly. Maybe there’s a specific face that has an unexpected effect on the device. What if you could find a way to cheese a discount out of it or something?
Imagine a racist vending machine. The face recognition system think this customer is black with 81% confidence. Let's increase the price of grape soda! Oh look, a 32 year old white woman (79% confidence). Better raise the price of diet coke!
In Japan they had some kind of facial recognition on vending machines selling cigarettes that would determine the age of the person in attempt to prevent kids from buying cigarettes. But it only worked for Japanese people.
Stupid racist vending machine wouldn't sell me smokes!
shame. id like to send you a carton.
It's cool, I quit years ago.
Also I was in a diverse group of people and we were able to do some science. Fortunately we had a Japanese person in the group which allowed me to purchase the smokes. But yeah, it failed on everyone that wasn't Japanese.
When you use a generated face with a mixture of white and black features, that’s when it gets interesting. Maybe you can even cause an integer overflow.
Imagine AI bsod when seeing you
Yo mama so ugly...
Vending Machine Phreaking
I firmly believe that every system has exploits. The more complex the system, the harder it can be cheesed.
I don't think they're doing dynamic pricing on an individual basis, that would be too obvious. But checking the demographics of each location or individuals' shopping habits, and potentially adjusting the prices or offerings? Definitely.
So, if you show it 100 faces from group A and 4 faces from group B, that could start gradually shifting the prices in a specific direction. If you keep going, you might be able to make it do something funny like charging 0.1 € for a Pepsi and 1000 € for a Coke or something like that. If the devs saw that coming, they might have set some limits so that the price can's spiral totally out of control.
I am sure the profit margin is taken into account, so you won't get an ultracheap Pepsi unless it expires soon. Similarly, I expect it to consider economic viability, so it won't keep raising prices unless people are willing to pay them. Of course, you never know what the model actually does or what goals it follows (maximizing profit is a good guess, though), or how bad the coding is. The program might be very versatile and robust, or it may break when you show it a QR code - how can I know? Probably something in between.