When people use "mentally ill" in a negative way

One thing that's always bothered me the most about transphobes is how they hurl the phrase "mentally ill" at us.

I can understand that perspective coming from an ignorant cis person. Bc your average person doesn't understand the definition of a mental illness: a state that causes mental discomfort and suffering.

Being trans doesn't cause mental discomfort on its own, but if transition is delayed, it can cause gender dysphoria. We have gender dysphoria in the DSM-V in order to make sure that transition care is considered medically necessary, which makes being trans a unique thing that kinda stands on its own in the healthcare world. Being transgender itself is no longer considered a mental illness, bc we've made progress in researchers understanding us better.

I understand cis people having trouble grasping that gender dysphoria is not the defining characteristic of being trans, and that not every trans person experiences it.

What I cannot understand is using "that person is mentally ill" as an excuse to justify their hate speech. Even if they think we are mentally ill, the caring way to view us would be that we should get the treatments we deserve. Meaning, the treatment found most effective by research - transitioning.

I'm aware that transphobes don't care about logic or empathy. I just find it particularly nasty how they have zero compassion for people that they think are "sick" and need treatment. They know conversion therapy is torture and they don't care, they want us to be tortured. It's so twisted.