I have a df64 and a 1zpresso J Max, which IIRC is 48mm conical. The product tastes pretty similar in an aeropress, although I would say df64 is slightly better (could be due to many things, like conical v.s. flat, geometry etc.), but not $200 better.
The speed difference, even considering hand grinding is negligible unless you are making coffee for more than 4 people.
In general I find the diminishing return might hit you real hard if you decide to upgrade; but it is a hobby anyway: if it makes you happy then it makes you happy; who am I to judge :)