I always disliked this feature on Spotify. I can swear that it messed up dynamic range in the songs somehow, because with this setting on, parts of the song just sounded... wrong. That said, the last time I used it was probably 2 years ago.
Replay gain is great when implemented properly. But at least then, Normalize was kind of borked on Spotify.