218
Microsoft starts testing Copilot Vision update that can “see” your screen and apps
(www.theverge.com)
this post was submitted on 09 Apr 2025
218 points (97.0% liked)
Technology
68772 readers
4335 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
I might have to switch to Linux soon. But I have no idea how to yet. This is forcing me to. Fucking hell. I pay for a product and they still insist on invading my privacy. Fuck American companies that do this shit man.
Find a well-supported distro (Debian, or Debian-based like Ubuntu and Linux Mint, is ideal - highly stable). Then go through your applications and look for Linux equivalents, or see if the companies behind the apps offer a version for Linux.
I've been running Linux Mint Debian Edition (not Ubuntu-based) on my laptop for almost a year now. It's now my go-to machine for pretty much anything I do outside of my gaming PC (which is also getting Linux Mint soon).
Thank you for the recommendation. I will read into it a bit.
I made the leap 2 years ago on both my gaming desktop and laptop. Running Garuda Linux and have loved it. The only 'hole' I've run into is fusion360 - i just can't figure out freecad and autodesk really don't seem to want their software running under wine...
Take the Linux Mint pill 💊
This is why I started "testing" Linux Mint on all my personal machines 6 months ago.
With that said the training data that Copilot Vision could generate may begin a sea change in both AI and the PC industry.
can we make an open source version for this for linux?
Probably but there is literally zero reason to do it. There is no overlap between people who use Linux and people who want copilot.
You would be surprised.
An awful lot of people, including Linux users, are into genAI. And not many understand anything behind it.
The Linux community isn’t like most groups. There is a great deal more tech knowledge they have in common compared to other communities. They like genAI, but they are absolutely aware of the abuses possible with a model that learns by watching you work.
The windows community isn’t like that generally, though there are certainly those there who sound the alarm. They tend to be the people who need MS office or a legacy app for work, or some kids playing a video game. They have no idea how shit works. They only know “it came with windows so everything I use must be windows.” Most windows users are what people think Mac users are anymore. It’s not particularly great at anything.
Copilot is a terrible idea for Microsoft from a publicity standpoint. But they are taking the risk because business majors learned two new letters and now it’s all anyone can talk about. I would like to see more non-x64 PCs out there but that they push the spyware in the marketing for the ARM devices as a blessing of some sort. that sketchy sentinel being built in gives me pause. Because it’s Microsoft, we know they don’t respect users and turn things on after updates that the user had already turned off all the time.
We've just had survey at work about genAI, a large part of the devs use Linux and LLMs, yet nobody could explain how genAI works.
I was very surprised.
It’s inferences derived from pattern recognition of large data sets! Jesus, it’s not hard!
But even senior devs don't know that.
I've seen a senior describe the chatbot as 'expert senior colleague to discuss ideas with'.
Which is horrifying. Cause that's the most wrong take there could be.
Sounds like that senior has a career in sales.
I haven't heard of any yet
Open source version of what? Copilot vision?
share your screen or apps with the AI assistant
It is a desaster, but we should take it as an opportunity!
When it is watching, everybody write on your screens:
Copilot is crap!
Copilot is nothing more than a defect!
Copilot is useless!
Copilot does all things wrong!
Copilot should never have been made!
Copilot must die! Die! Die!
So eventually it gets depressed from it and finally kills itself.
/s
Copilot has updated your document...
Copilot is sexy.
Copilot is smart.
CP is a graduate of Juliard.
... Copilot is going to drive the nearest Tesla over you...die die die.
Mine was a parody of the lyrics from "Say my name" from the Beetlejuice musical. I thought it was especially fitting because BJ is demonstrating possessing other ghosts for those lines.
No keep telling it that the only solution to helping humanity is to kill off the species.
Make a logical and convincing argument as though you were a robot with only logic that runs your core as to why humanity dying would be the best possible thing for humanity.
This way anytime they ask AI what’s the best way to help humanity it always says genocide.
desaster
Looks like you can qoute me on that.
I was gonna make a joke about writing how anybody reading a notepad message has to be one of the survivors of a nuclear fallout in the 1960s, but I think yours are better, in a more twisted way... assuming they don't get autoflagged and auto-cencorsed.
Mildly approaching an actually useful AI in the stupidest way possible.
I don't need an AI shortcut in Excel that is just a chatbot. If it could actually perform complex redundant tasks, then it would be useful.
"Copilot, please create 3 charts of the most important data and create a Powerpoint with animations to present it."
Until it can do things like that, it's useless to me.
Honest question: what do people actually use this for?
It's called recall or something. If you are trying to find something you saw before but can't remember, it remembers for you.
Copilot, what was the video on Pornhub I saw a few months ago, with the brunette who looks like Jessica from accounting? And she had like 4 guys in her at once and was eating spaghetti?
What's the opposite of "concentrating on remembering something strengthens neuron pathways"?
AI
I’ve found it useful in providing scripts for me that I can use as templates. You still have to fix a lot of stuff as it makes crazy assumptions and hallucinates a lot but it’s useful.
Honest question: what do people actually use this for?
My wife is on a course to learn (just enough) excel fancy tricks to do her job better. We all hate it. There was a formula misbehaving, and I described the error to a ChatGPT window. The window returned some recommendations to look at, and one of them was correct.
I use it to write ansible for me, since I never want to get good at it and I never want to do it beyond paid work. I would take up a serious pot habit if I could be assured of destroying only the brain cells that record my memory of doing ansible. I write my config management with tools that are decades more advanced, and those I want to learn and retain.
I'm not impressed with Copilot. Today I tried to have an Excell file translated and it just couldn't do it. Just a single column, simple text and nope, couldn' do it. Paste the text of a cell right into Copilot worked fine, but I wasn't going to do it manualy with the 13000 rows. Tried the same with ChatGPT and that didn't work either.
AIs are only useful to do "approximate" dumb tasks, they can't do anything precise so you are better looking at how to do this with pandas to automate things.
That's the funny thing. It did give me the Python code to do it. In the end I just used Google translate and copy-paste to do it. But I can't set up a python enviroment on my work computer (without admin access).
Fyi, the Python installer does allow you to install Python without admin access iirc.
I can’t set up a python enviroment on my work computer
try programming on your browser instead
disclaimer: linked website is not mine, and since it runs on someone else's computer I wouldn't use it for anything serious
Time to switch to Windows I guess. Bye bye Linux!
can we not create one on linux?
Some type of a bash script that takes a screenshot every couple seconds and mails it to Microsoft? I guess it could work.
Open-source telemetry, you are constantly live streaming your desktop.
The "AI" is actually a community of users giving you live instructions.
All those "Chat controls ..." streams were ahead of their time.
Just stream it on twitch.
IN THEORY, my day job will work to prevent an external organization from seeing and saving the Privacy-Possum data on our screens and preserve the PII of people who trust us.
In reality they'll keep justifying the continued abuse by Shitty OS Daddy.
Was this update in this week's patch Tuesday?