RSS Contact

Wednesday, February 3, 2010

Don’t Stereotype Me

Recommendation System: 
Collaboration Filtering vs. Profile-Based Personalization
By:Sol Tzvi

Personalization is not about a process or a concept but experience, the experience of personalization by real flesh-and-blood people; it cannot be reduced to a statistical model of future behavior based on probabilities and prediction alone. It must be lived, experienced, felt. It is an existential realm, complex and verging on the chaotic, not a forumla that can be reduced to an algorithm.

Personalized experience is part of the subjective field of everybody's life; it might include reflection or random thoughts, but it is not about thinking per se, it is about  feeling, emotions, the flow of sensations.

Because it increases revenues and enhances profits, businesses endeavor to know how to better serve customers. To that end, one new “killer feature” on the internet today is a quick match-up of available services to an online person as manifest through a pattern of clicks.

How can we force this digital box to think out of the box itself and discover a different domain of digital life, a different way of thinking?

In order to implement the mechanism of personalization, a system should predict what will capture the user's attention at any given time. But is the prediction based on past and future, or is prediction in itself a creative act? In the movie The Matrix,  the Oracle told Neo not to worry about the vase, and as Neo turns to look for a vase, he knocks one over, a vase of flowers shatters on the floor. Then the Oracle asks. "Would you have broken it if I hadn't said anything?"

Did the Oracle predict what was going to happen or did she manipulate Neo and cause him to break the vase? And does it really matter? Either way, Neo experienced her words as prophecy, a prophecy he fulfilled. But did the prophecy follow the action or forecast it?

Our individuality is unique
A system intended to make recommendations that are followed by a user must predict a person's choice based on a pattern of interests but also make a person choose – even unconsciously - to create a new behavior. Personalization happens when the new content is based on the person's profile, but does provides something genuinely new, different, and possibly unexpected. Is this a paradox?

There are currently two major methods for providing a working recommendation system:

-    Collaboration Filtering (CF);  recommendation agents employ prediction algorithms to provide users with options that match interests based on membership in a virtual community.

-    Profile-Based Recommendation; user history indicates a user's current or future desires.

Both methods have pros and cons. Combining both provides reasonably good predictions.

Collaboration filtering predicts a person’s interest by analyzing the behavior of the virtual tribe to which he belongs at any given moment. By being identified as one of many. by feeling as if we are known, we feel special ... but do we all really act the same, based on tribal membership, or do we remain unpredictable individuals?

Behavioral patterns

Can past behavior predict future behavior?
If you ask a person how he will react in a situation, although he can't really know, he will base his answer on his past, based on his intentions so far as they can be understood and a relationship between past behavior and future behavior.

Using previous experience is the best way to respond to a question that allows a user to predict his own future actions - we can imagine it, we can envision our self making a decision based on our beliefs, faith and hopes.

But even for ourselves, is a prior behavior really the best predictor of future interest? To some extant yes, probably, but because our intention is dynamically changing and ultimately beyond our full understanding, a personalization system based only on past behavior will not predict the genuinely novel, the really creative, it will not predict, that is, the next click, the click that says “BUY.”

We end as we began: personalized experience is subjective and always, always integrates feelings and emotions. So ... no, I do not like it when a big news portal stereotypes me based on my alleged virtual tribe. And because I don’t like it, I will do what I can to make an unpredictable decision ... to remain free, that is, to remain fully human.