Machine Learning

Well we already have roombas and robots that serve tables at restaurants, but I'm guessing you mean more like a robot butler from Fallout, capable of receiving verbal instructions and performing arbitrary tasks. As far as I can tell, there are two main reasons.

The first is that robots are dangerous. If you make a robot strong enough to perform tasks that humans do, like lifting heavy objects, climbing stairs, and opening doors, it's gonna have to be pretty heavy. There are tons of scenarios where the robot could fall on someone, drop something on someone, or just hit them with something really fast. The best way to overcome this would probably be one of those baymax style squishy robots, but it still seems like a challenging engineering problem.

The second is more the ML side of things. Even with "a limited range of tasks", you correctly pointed out you'd need RL, because every environment is different. If you're curious about why RL hasn't seen many real world applications yet, I'd recommend you try to train an RL agent yourself. Even with the state of the art, it's incredibly mercurial, and often falls into a failure mode unless the hyperparameters are just right and the observations and rewards are designed perfectly. The people at nvidia, meta, and openai are desperately trying to monetize RL with all sorts of tricks and techniques, but it's just not there yet.

So ultimately we'll probably have robot butlers around the time we solve both of these problems, which depending on who you ask, will either be in the next couple decades or never.