It won't damage it. If it has high output impedance (/u/Bingturong says 330Ω) it will tend to boost the bass a bit on the HD650 as it has an impedance curve that peaks in the bass. This is not necessarily a bad thing, and the HD650 is itself high impedance and one of the headphones likely to sound best out of a high impedance source.
You can see the effect here (scroll to/search for "Amplifier output resistance effect"):
That's at 120Ω and it is very subtle. It will be a bit more at 330Ω but maybe still reasonable.