He’s never given me any reason as to why he thought I was a man.

Neither did the GenderAnalyzer when I plugged in my blog to analyze my gender.

Curious to see how they’d arrived at this conclusion, I contacted the team at GenderAnalyzer and asked. Jon Kågström, the brains behind the operation, wrote me back to fill me in on the details of this wondrous machine:

It all started back in 2004 when I was doing my master thesis on machine learning for spam filtering. I was fascinated by how well they worked on spam (often better than humans) and started to wonder what more than spam text classifiers could be used for. I did a big number of hobby research projects testing classifiers on different domains (e.g., sentiment, happy/sad, web page categorization).

Doing this while improving the classifier technology derived from my master thesis, I came up with the idea to let everyone have access to classifiers. So with help from two friends, Roger Karlsson and Emil Kågström, we built uclassify.com. The idea is to share advanced classifier technology that is easy to use (don’t even have to be a programmer) for free in a Web 2.0 format.

After we had finished with uclassify.com we decided to test if it’s possible to have a computer differentiate between males and females by looking at their text. We found the idea really interesting so we collected 2,000 blogs written by males and females and used the uClassify API to train a classifier. We decided to put it into test and created GenderAnalyzer.

The accuracy is lower than we expected and we believe a major reason for that is that the training data is biased (only collected from blogspot). We think we can get better accuracy by using the URLs that users test as training data (when they vote if it worked or not, we train it accordingly). In this way we would have a classifier that adapts to real world gender data. Just as machine learning spam filters do.

Today on the uClassify blog, Kågström elaborated on the Analyzer’s current low accuracy (53%):

Our training data of 2,000 blogs is automatically collected from blogspot. Running internal tests (10 fold cross validation) on this data gives us an accuracy of 75%. This effectively means “Given that the corpus is a perfect representation of real world data, the classifier is able to give any real world data the correct label by a chance of 75%”. So our training data is probably not very representative, as a matter of fact it’s very stereotypical.

When someone is testing a blog we are not crawling through posts on the blog to get a good amount of text. We are only hitting the given URL and using the text (and html) that appear there as test data. So a page with mostly images or frames will give bad test data.

We are trying to encode test data to utf-8 which is the format of the training data – it could be that we are missing some encodings.

It’s a worthwhile experiment and the technology is there to make the process of gender analysis possible. If there are any issues with GenderAnalyzer, further development of the tested blog samples will enable the Analyzer to increase its accuracy levels, and as more people make use of it and the machine gets more training, it will become better equipped to answer the question “man or woman—who is writing that blog?”

Though Kågström brings up a good question in the uClassify’s blog post, maybe “the difference between male and female writing is not significant?”

Only one way to find out. Best of luck to the team at GenderAnalyzer.

As for the college e-fling, we still talk form time to time. We’re both married now—to other people. I think he finally believes that I’m a woman. It only took him five years.