I wouldn't use the PAM2804. The datasheet recommends a PWM frequency of 500 Hz, and if you look at the "PWM Dimming: ILED vs. Duty Cycle" chart on page 5 you can see why - at 1 kHz there is already a significant reduction in the average LED current. The enable response just isn't fast enough to modulate at 38 kHz - by the time you want to turn it off, it will barely be on.
This isn't quite enough for your 38 kHz either, but it's closer (and waaaay more complicated): MAX16834. The datasheet says it can do up to 20 kHz, but at a glance I did not see a particular spec that seems to actually limit that.
Here's an article about fast switching LEDs. All the linked products unfortunately have supply voltage ratings above yours, so they won't work. But it discusses the concept of shunt dimming, where you'd end up simply bypassing the current around the LED rather than actually turning off the current regulator. Inefficient, yes.