+1 on the fact that I have infrequently seen my own merge experiments spit out "### Reponse:" while using the standard Alpaca format. It so far hasn't happened enough times to be a concern. I've seen more benefits than deficits when it comes to merging models overall however. I'm coming at this primarily from a creative writing aspect, so I'll usually find models that have characteristics I enjoy, merge them together, and then continue merging models down the line to experiment with output quality.
+1 on the fact that I have infrequently seen my own merge experiments spit out "### Reponse:" while using the standard Alpaca format. It so far hasn't happened enough times to be a concern. I've seen more benefits than deficits when it comes to merging models overall however. I'm coming at this primarily from a creative writing aspect, so I'll usually find models that have characteristics I enjoy, merge them together, and then continue merging models down the line to experiment with output quality.