Something is missing

Looking at my RSS aggregrator, I came to a realization, it looks an awful like what a newspaper or magazine would: lots of catchy headlines for topics that interest me. The difference is, of course, I choose where the headlines come from, whereas news go through editors. So, there's an certain inherent bias to what is presented to me via RSS, and there's a different bias to what news make the final cut on a newspaper or magazine.

What is common about both is that this bias converges into a culture of sorts. There's a certain level of expectations from things that appear on News.YC, and there's a certain level of expectations from things on the New York Times. The great thing about the Internet is that there's enough choices out there to make a pretty varied selection; the sucky thing is that we're stuck with them once we make them. (Well, not really, but let me explain what I mean).

As Malcolm Gladwell says in his TED talk, you can't really ask people what they want. We don't know what we want, we only know if we like what we have or not. Out of sight, out of mind.

The thing about content (at least for me), is that "how-much-I-like-content-from-any-given-outlet" is a bell curve: there are some great things and a lot of useless junk. The catch is that this applies to stuff outside of my preferred cultures as well. Meme sites usually irk the hell out of me and I'd not want one in my RSS aggregator, but I can't deny that they do have some pretty nice content every once in a while (considering what the intentions of meme sites are, anyways).

Do you see where I'm going? We are stuck with the choices we make in our readers because we're asking the wrong question: what is the perfect culture for me? There is no perfect culture. Only many noisy ones with some perfect content (and there may or may not be a correlation between the culture's voting patterns and what would actually please me). My netvibes page wouldn't know that though. All it can do is map one culture to one portion of the screen.

One could say that content aggregators would solve the problem, since they grab the best of all worlds, but, again, look at News.YC. It obviously has a very strong culture. You'd never see a clever lolcat picture with a great geek culture reference in News.YC. Similarly, you'd never get a paper about JVM internals on StumbleUpon. It seems every site develops a culture by virtue of its own history of posts, even when they encourage randomness.

How does one develop a fitness function that takes Internet content as an input and that evolves into a filter that gives me what I like (as opposed to giving me stuff that is roughly in the direction I expect from a given culture)? Is that even possible?

Sometimes I wish my aggregator had a magic "Surprise me with something nice" button. But I have no idea if the fitness function to power it can even be done. I must be missing something.