Technology

That's pretty much what the article says. The model needs to be trained on the target keyboard first, so you won't just have people hacking you through a random zoom call

I think you might have misunderstood the article. In one case they used the sound input from a Zoom meeting and as a reference they used the chat messenges from set zoom meetings. No keyloggers required.

I haven't read the paper yet, but the article doesn't go into detail about possible flaws. Like, how would the software differentiate between double assigned symbols on the numpad and the main rows? Does it use spell check to predict words that are not 100% conclusive? What about external keyboards? What if the distance to the microphone changes? What about backspace? People make a lot of mistakes while typing. How would the program determine if something was deleted if it doesn't show up in the text? Etc.

I have no doubt that under lab conditions a recognition rate of 93% is realistic, but I doubt that this is applicable in the real world. Noboby sits in a video conference quietly typing away at their keyboard. A single uttered word can throw of your whole training data. Most importantly, all video or audio call apps or programs have an activation threshold for the microphone enabled by default to save on bandwith. Typing is mostly below that threshold. Any other means of collecting the data will require you to have access to the device to a point where installing a keylogger is easier.

It's bad now, but where we're at with AI... It's like complaining that MS paint in 1992 couldn't make photorealistic fake images. This will only get better, never worse. Improvements will come quickly.

I'm skeptical too, it sounds very hard to do with the sound alone, but lets assume that part works.

The keylogger part could be done with a malicious website that activates the microphone and asks the user to input whatever. The site would know what you typed and how it sounded. Then that information could be used against you even when you are not in the malicious website.

Well to train ai you need to known what the correct answer is.

it doesn't need a keylogger. Just needs a Videocall meeting, a Discord call meanwhile you type to a public call, a recording of you on youtube streaming and demoing something.. etc.

Thanks, I hate it.

Reconstruct sound using smartphone accelerators? What do you mean? That accelerometers can act as speakers and produce sound? Or they can act as microphones and record sound as numerical data of vibrations etc? Can you point me to any articles or sources?

SpyApp is spying in background

User thinks "why is battery draining so fast?"

Opens battery setting

Oh, this app shouldnt work right now

Restricts SpyApp's battery permissions

are you saying that a cellphone accelerometer can be used as a microphone? That sounds... interesting. Do you have a source?

My wife types with her fists when I’m trying to have Zoom meetings.

Mx blue and unicomp here.. HELP

Never knew my mutant blue switch keeb would come in handy one day. I've lubed the blue switches and added foam and tapes so now it sounds like a clicky-thocky blue-brown switches keeb.

I was just thinking, streamers might have to be careful actually — you can often both see and hear when they're typing, so if you correlated the two you could train a key audio → key press mapping model. And then if they type a password for something, even if it's off-screen from their stream, the audio might clue you in on what they're typing.

I have to assume it could be modified to work on any language. You just have to know the keyboard layout for the language in question do you know what to listen for. Languages with a lot of accents like French maybe could be slightly more complicated but I seriously doubt that it couldn't be done. I'm honestly not sure how the keyboard is set up for something like Chinese with so very many characters but again if this can be done, that can be done with some dedication and know how.

I would think that would be a safe assumption most of the time. Less than 1% of typists use Dvorak, for example.

It uses the sounds it records and compares again the messages you send. So in theory it's layout agnostic.

It says it's acoustic, not electromagnetic...

There are specific bags for phones, so I assume so.