Yeah I have heard that argument too and have long thought of it. It is an interesting point, and a good one to talk about. That allowing trans women to use the women's bathroom will allow cis men to pretend they are trans and thus use that to rape. That would be a terrible for a cis man to do, and he should be held accountable.
However, I don't think trans women should be held accountable for crimes cis men may commit. Forcing a trans woman to use a men's bathroom is just as unsafe as a cis woman using a men's bathroom. An ex-girlfriend of mine who is a trans woman used the men's bathroom until this very thing happened, a man tried to rape her. I beleive she and other trans women should be able to use the bathroom without getting rapped, assaulted, beaten up, or harassed.
it reminds me of an argument I heard once that women should watch what we wear in public, because if men are turned on by us, they may want to rape us. Here is another example of how as women we are suppose to change our behavior to compensate for the actions of men. We are supposed to suffer because of what men might chose to do. Only there's not a valid alternative offered for trans women. They don't have another place to pee safety in this whole debate. It's not safe for them in the men's room anymore than it would be for a cis woman.
There is a hudge difference between a trans woman and a man pretending to be a trans woman. Putting women in the way of danger and risk, because of perceived threat of something a cis man may chose to do is very unfair and scape-goating.
JKR has had many times where she publicly says that she doesn't consider trans women to be women, and alludes that they are men in her tweets. We can't put an entire demographic of people at risk because of a threat somebody else might do. Men need to be held accountable for rape, and trans women need to be able to pee without getting assaulted.