Two sexes... only?

Male or female, that's the question... But what if reality is actually more complex? For if sex boils down to the appearance of genitalia, how can we "classify" people who are born with intermediate genital shapes, and what about gender identity?