Interesting. I have never had echo but disconnected ANC right away without using phone and ANC at the same time. Certainly disconnecting ANC should not make phone audio worse as there are three microphones in the roof with at least one separate from the ANC (as far as I can tell).
I am not aware of any cons of disconnecting ANC unless the hum from variable cylinder management bothers you. I didn't really notice any significant road noise being removed beyond the low frequency engine noise when VCM is on, which isn't often.