While your HE6se gets loud enough, it might sound compressed if it's under powered. I found that my HE6se V2 had a tighter bass and better stage depth when given a lot of power. I remember seeing an interview with a Hifiman employee that said the v2 needs 10W per channel.
To see if you need more power you could just buy a cheap $100 speaker amp like a Fosi Audio v3 or a S.M.S.L DA-6, and pair it with Hifiman's HE-Adapter, or get Another cable . If it doesn't sound better then just return the amp.