Your observations are spot-on. PINNs are indeed mostly a terrible way to solve a differential equation, and I would definitely recommend doing a PhD on something else :)
However, the broader field of sciML itself is booming right now -- this is an excellent choice to work in! Let me give you some examples.
- On the bio side there is a lot of work happening in applying LLMs/diffusions to modelling proteins, for drug discovery, ...
- Neural differential equations are a great way of improving existing diffeq models of many phenomena (chemical kinetics, population dynamics, weather modelling, ...).
- I've seen a lot of use-cases for ML in astrophysics recently -- describing priors for black hole imaging, a fair bit of symbolic regression, ...
- On the computational side there have been great recent strides in building an "autodifferentiable everything" software stack, e.g. a lot of my recent work has been about building "scipy in JAX".
- This has actually brought with it some new numerical methods too; e.g. I've been involved in developing some new numerical SDE solvers, and some new quasi-Newton optimisers.
And so on and so on. You'll find a similar story across most of science these days; I'm throwing out some of the examples above that I'm most familiar with.
You also touch on the difficulty of coming up with ideas right now. First of all, don't stress about that -- this is totally normal in your current position. I strongly believe ideation stems primarily from a technical depth of knowledge, and that's something you only really develop a couple of years into your PhD. (And after that, trust me, you'll have the opposite problem: too many ideas and not enough time to pursue them all.)
Find a good advisor and they'll be more than happy to hand you off some of the ideas they haven't had time to tackle themselves; this is the fairly standard arrangement for most PhD students and their supervisors.
You might like Just Know Stuff, which a post I've written on topics worth knowing in ML, and How to handle a hands-off supervisor, which also includes a lot of advice on tackling a PhD more generally.
For context, my own background was mathematics originally. I've since wandered through CS, machine learning, and now focus primarily on problems in numerics and in computational biology.
I do scientific machine learning, with a particular focus on numerical methods and computational biology.
The other big piece of fundamental mathematics needed is differential equations -- ODEs at least, but ideally also SDEs+PDEs+numerics. (Soapbox moment: I find it kind of unusual how poorly this is taught outside of math/engineering courses, given that it's easily the single most successful modelling paradigm of the past few century.)
Just Know Stuff also has a short list of things I consider worth knowing.