Hello Everyone, Little Background: I am a graduated Math and Computer Science Major, currently working with a professor on PINNs, more on the error analysis and convergence side.
I am currently in the process of applying to PhD in Applied Math, my research supervisor for undergraduate research will hopefully be my supervisor for the PhD and we have talked about going into SciML as focus for my PhD along with structure preserving methods. The problem is while working for him it is very overwhelming to comb through the research due to high number of publications, and I once I was able to grasp everything coming up with anything new was difficult because all the possible ideas in my grasp were already done. This is also due to my limited knowledge as an undergraduate, there is only so much math and comp sci I could’ve learnt during my undergraduate to help me come up with ideas. Also this is not to mention how terrible PINNs are as a solver. But even after this I came up with an idea and did my due diligence of looking it up thoroughly in the literature. I was starting to compile the results and found a paper with the same ideas as me, this really demotivated me and now I am questioning my decision to do a PhD in this area because if this keeps happening I would just not be graduating. There is also almost no one doing error bounds on SciML methods even though they are trying to compete with traditional methods, and their numerical simulations often seem naive at best. Usually they would do a 3-4 dimensional system as a high dimensional benchmark and say that should be enough proof of the method working. Coming from a math background I simply don’t buy that. So even though I came up with an idea and worked on it, some person who has a half-assed paper on arxiv is the reason I cannot write on a topic.
I wanted to know your thoughts on this, especially who joined PhD since 2019.
This is my first post on this subreddit please let me know if I violated any rules.
Your observations are spot-on. PINNs are indeed mostly a terrible way to solve a differential equation, and I would definitely recommend doing a PhD on something else :)
However, the broader field of sciML itself is booming right now -- this is an excellent choice to work in! Let me give you some examples.
And so on and so on. You'll find a similar story across most of science these days; I'm throwing out some of the examples above that I'm most familiar with.
You also touch on the difficulty of coming up with ideas right now. First of all, don't stress about that -- this is totally normal in your current position. I strongly believe ideation stems primarily from a technical depth of knowledge, and that's something you only really develop a couple of years into your PhD. (And after that, trust me, you'll have the opposite problem: too many ideas and not enough time to pursue them all.)
Find a good advisor and they'll be more than happy to hand you off some of the ideas they haven't had time to tackle themselves; this is the fairly standard arrangement for most PhD students and their supervisors.
You might like Just Know Stuff, which a post I've written on topics worth knowing in ML, and How to handle a hands-off supervisor, which also includes a lot of advice on tackling a PhD more generally.
For context, my own background was mathematics originally. I've since wandered through CS, machine learning, and now focus primarily on problems in numerics and in computational biology.
Do you have any use cases on using the neural networks for differential equations for chemical reactions?