That's not the problem. It's deeper than that. Society makes the assumption that for females, unless you're beautiful, you don't deserve to be listened to, or to be talked to. It doesn't have to be at a sexual context. Whereas men can get away with being less than perfect on the outside as long as they're? funny, have a good personality, or some other trait (women will talk to them and hear them out, outside of a sexual context), women don't get that chance.
This isn't about dating or sex, this is about life. When men are unattractive they are afraid they won't get the girl. When women are unattractive they aren't perceived as having any? worth at all - as another commenter said earlier, their right to exist is put into question. Just look at the way people chastised Marion Bartoli, the women's Wimbledon champion, for not being attractive. Do you think the same would ever happen if Wayne Rooney made such an achievement?
a lot of women TALK to men no matter what they look like, forget what you see on television, and we are not talking about dating. many men will only bother to TALK to women they find attractive.
Men are judged by their worth in society, women are judged by society on their appearance. The first implies action, the latter is a static quality. Men are active agents who create change, women are passive objects to be? desired. Always has been, always will be, because why would men surrender their dominance over the world?