You're overthinking it. Do you apply same logic for the car you own? Or a part of the city you live in?
You might, but why... You'd never be able to control it all and people will judge either positively or not, there's no an universal rule.
To be sure have a neutral watch or the one you mentioned, for the specific meetings when seeing those clients in person.
In a wold of people wearing Apple watches in every occasion only the extreme minority will notice what kind of watch you wear and for rarity of it nowadays appreciate that you're a watch person instead of apple notifications addict :)
So I'd relax and wear what watch I feel like in that moment.