I see this assumption all over the place, in so many areas, ableism, homophobia, transphobia, racism, etc. the argument that somehow there are a finite number of rights and by giving a marginalized group some we are taking from someone else. BS. Truth is when you build an inclusive society, everyone benefits. Say you have a hospital trying to hire 4 surgeons. In the top 4 candidates, one of happens to be trans. In one society a hospital would reject the trans candidate and hire a less skilled and qualified candidate, in an inclusive one the better doctor would be hired, even if they happen to be trans. I would prefer the better doctor if I was under the knife. I
What if prestigious universities had a rule that excluded people in wheel chairs because they didn’t want to “take away others rights” by installing wheel chair ramps? They would argue the financial resources going towards making ramps would take away from other programs. Imagine such a university turning down Stephen Hawking from its faculty.

When you build a society that includes more people, you build a stronger society because you allow more people to contribute their talents to the community. Excluding anyone, including trans people, sets you up not only for elitism, but untapped resources and potential.