Yes, labels can be toxic and unnecessary, leading to problems. I do think you have a point to see beyond the labels and gender as a whole. But labels can be good and even helpful in the right times.

For example, I know a trans woman who dates men. This can often be dangerous, as some men don't want to date trans women, and there are numerous incidents and violence and hate crimes around men who realized they were attracted to a trans woman. Infact, in some places it's even legal to kill a trans person if you don't know she is trans (gay panic defense). So, on her dating app profile, it says quite clearly, "I am a trans woman" and before she goes on a date she makes it clear the man knows she is trans. How would she be able to do that without the label of being trans? Truth is, many of her future dates may care a lot if she is trans or not, and aren't interested in looking beyond her gender and want to know before going on a date, and she makes it clear to avoid being murdered.

My girlfriend happens to be trans, and she sees an endrocrinologist to regulate her hormones. Specifically, she sees an endocrinologist who has experience in carring for trans women and the complex cocktail of HRT. Here, her having the label of being trans, makes her able to receive quality healthcare. Not all endocrinologist know how to regulate a trans woman the way hers currently does. Without the label of trans, and even trans healthcare, she would receive less quality healthcare.

Having a label can also help a trans person understand their feelings and experiences.

For example, many trans people experience gender dysphoria. Gender dysphoria can be very painful and very distressing for a person. However, if a trans person tries to discuss this with a cis person, often the cis person will have no idea what it is and totally invalidate the tran's person experience. This can lead to gas-lighting of the trans person's experience and cause further psychological harm. One a person understands they are trans, and that their dysphoria is a common and normal part of their trans experience, it can be validating and helpful for them to understand their pain and experiences. Having that label of being trans here can help them navigate what they've been experiencing.

I do see the point, to many things are toxically labeled, and overall we'd do a lot better to loosen the tight constraints of gender rolls and the labels they hold over us. Let's start seeing people as they are and not just their genders. However, gender still exists in our society, we can lessen it's toxic roles, but it's not going away, and we still need some sort of gender labels at some point.

Here's another example, if we totally dismiss gender, than countless people's sexualities will be dismissed. A man saying he is straight and only attracted to women won't be respected, because the label of women is gone. He will then be expected to be attracted to everyone regardless of gender. There is still some biological attracted around biological sex and and gender roles present. Sometimes labels are necessary and needed.

It kind of reminds me of the arguments back in the day that if we stopped seeing different people's skin color, and everyone would become color-blind we could end racism. This was found to be rather toxic to black communities because it totally erased and invalidated their experiences as black people. But in truth, it was a good idea to stop making assumptions about people based on their skin color. That was the intention, but erasing skin color went to far.