Well I find that I'm very sensitive to my white stone colors matching.
I love love my warmer CZs. But absolutely hate when they are warmer (or likewise whiter) than the diamonds/Czs or moissanites around them. So that'd be a dealbreaker to me too. While others really like the contrast.
It's all personal preference!