Right association in itself can never be the cause of suffering.
Right association leads to a re-alignment of your associations. It is not as you break your existing relationships. Rather your existing relationships get aligned in the right direction.
It is not as if you give up your family or work or something. It’s just that now you align all of them with the Truth.
But this re-alignment will be resisted. Be prepared. One way to ease the whole process is to make it inclusive. Include more and more people in it. Bring them together, so that when the change comes, it is a change welcomed by all.
Otherwise, you will be a solitary crusader, and you will see needless resistance. That which can be done easily must be done easily because anyway there are a lot of battles to face.
Why then spend your energy in needless battles? Conserve your energy, it will be needed when the real battle arrives.