The rate of growth does matter yea. If an instance gets worried, they can lock signups. Slow growth means the software has time to improve as they notice issues.
Lemmy had many issues scaling before, except Lemmy had huge surges with the Reddit API blackouts.
If people start recommending PieFed now, it's on their own terms instead of a massive wave. They can backoff if they get too many users.
https://piefed.fediverse.observer/list