For those of you following this new blog, I just noticed that CRITEO now allows you to register for their FREE API online. It takes about 2 business days to setup your database ID, but once it is set up, you can move forward!

I’m also running a separate blog that will be focused on integrating CRITEO with blogs, hopefully working on building plug-ins or code modifications to use their free API with your low-traffic site. This is really exciting stuff for even the smallest blogger or online retailer: by allowing your users to log-in and rate your posts/products/services in terms of what they like and dislike, you will be able to give them very relevant links to other items you offer. This will increase the chance of them staying on your site, which means more products/services sold or more ads displayed. This is a win-win situation for anyone of any business model or size.

Get going and register for their free API engine access today!

Here’s some amazing news on the collaborative filter front — relevancy prediction engine developer CRITEO has made it to the Top 30 leaderboard for the NetFlex Prize, a competition between 13,610 teams to try to beat the current NetFlix prediction engine. What is even more interesting is that the CRITEO engine performs the analyses for this competition in real time, versus most of the competitors that are churning information that can take minutes, hours or even days!

On top of all that, CRITEO offers this predictive engine as a public API right now, and you can try it on your site for free. Check out CRITEO’s DVD recommendation test site to see how quick and relevant their engine is — and realize how this public API can help your site no matter what products, services or articles you cover.

The CRITEO API offers huge opportunities for both online companies as well as brick and mortar service and product providers. The NetFlix system is one that is constantly being beaten up by all of their users — and with a system like CRITEO, the off-site maintenance and management makes great sense since it relegates many responsibilities to a third party, so you can focus on providing your users with the service they want. The API isn’t just for huge companies like NetFlix, though, it also will work great with smaller online merchants and publishers. You can allow your users to find other items or services based on what they’ve used or like, or even allow them to show relevant articles to what they come to your site or blog to read about.

Why just base your relevancy comparisons to keywords and visit popularity when you can actually allow your users to quickly rate their visited pages/items to instantly give them relevant topics? This means you’ll retain them on your site longer, increase the opportunity to offer them what they want, and produce a site that is more competitive than others that not offering relevant advice. Also, an API like the CRITEO engine allows you to offer relevant in-links that are not biased by your views — the relevancy results are only biased by what that particular user has chosen to rate, giving them a unique view of your site and services/products.

Check out the API today — it is free to try, and easy to work with.

I’ve always loved the news — I can remember scanning the newspaper even before I could really read. Now that news is available in such a wide variety, covering opinions from many different angles and qualities of newscasting, the sheer amount of choices out there makes it hard to find all the relevant news out there. What is relevant to me may not be relevant to you — we might agree on topics, but our regions might be far apart. The wonderful news sites out there are still “one size fits all” in many ways, even if they have different sections or categories. I’m a regular at slashdot, NewsVine and digg, and I love their interfaces, but they’re still very one-dimensional. Sure, digg lets you vote up or down a particular article to hit the home page, but if an article hits a homepage, everyone sees it. We’re all thrown into one pool. Slashdot has the moderation system, a system that is a little more adaptive since you can modify moderations using an individually-managed preference (giving more “mod points” to your friends and taking away from your foes), but even slashdot isn’t very collaborative. Newsvine is gaining a lot of speed, but it is still a “Yay” or “Nay” system — not much collaborating going on.

Who wants to be the first news site that truly allows collaborative filtering, such as the API engine provided by CRITEO? Imagine a news site that lets individuals submit articles, comment on them in a blog-like fashion, and rate them using a collaborative filter? I might love a particular news story — you might not. Over time, as thousands or millions of others rate articles submitted, a good filtering engine will give us the stores that WE specifically are interested in reading — with great accuracy! Instead of trying to wade through thousands of articles a day, you can just do your normal scanning of the sites you like and submit links to those articles just like you do at newsvine. You can comment on the article, just like you can at slashdot. You can rate the article, just like you can at digg. But with an API such as CRITEO, that site would weigh the articles differently for each individual — some might love news on puppies, others might want news on the latest lawn mowing technology. Some might love news from major trade corporations, others might like news from small bloggers. It doesn’t matter, because the collaborative filtering engine will handle figuring out what you’d like to read, and what you should pass on.

Who wants to step up?

Jason Calacanis, ex-GM of AOL’s Netscape division, told an audience in a Chicago symposium today that he predicts that Wikipedia will be the top website, out-trafficing the second runner-up by 1000%. Wikipedia is probably the end-all be-all collaboration site — allowing anyone to discuss, edit and propose new information that is quickly fact-checked, corrected or deleted for irrelevancy. Yet does a collaboration site like Wikipedia have the opportunity to take advantage of a collaboration filter engine, such as the one provided by CRITEO? While it is hard to imagine how one could intergrate an external API into Wikipedia (which as far as I know doesn’t accept any external applications), there are thousands of other Wikis on the web using the Wikipedia engine. These secondary Wikis could definitely utilize a collaboration filtering system to prioritize interests for the casual and determined reader.

Wikipedia itself is mammoth — easily one of the fastest growing sites on the web. This is due to millions of amateur writers passionately adding and editing information with no financial compensation — the so-called long tail of the writing industry. While the top writers are dedicated to writing, they are limited and so are the funds available for those positions. This leaves an incredibly powerful long-tail of amateurs with close to professional writing or fact-checking skills. Yet the immensity of Wikipedia leaves many users in a very basic 2D linking process — they read something they’re interested in, and then they’re left with basic links to other topics that are only one-hop away in terms of relevancy. Surely this is a good system, but it can be made better, especially with a collaboration filter backing it up.

Imagine visiting a Wiki and reading about a topic you’re interested in. Normally, you could then hop to another topic closely related to that one. While interesting, I can see a future where you’d also get some top picks from others with your interests in driving towards information that might seem non-sequitir or irrelevant to the topic you’ve just read, but in reality these non-obvious topics might be of huge importance to you as a reader or researcher. For example, say you are interested in the metal gold — the Wikipedia entry on gold is big, with many links and pertinent information. Yet a collaboration filter might return to you other Wikipedia articles that are seemingly incongruent with the topic of gold metal, but after clicking to these other topics, you might find information that is very valuable to your research. You might find articles on the Great Depression (which some blame on the fall from a true gold standard); you might also find articles on cyanide leaching — a process used to extract gold from rock and stone.

The current collaboration engines are not ready to provide this specifically pertinent information — they’re better at providing an overall look at what others with your preferences are interested in. You might be interested in gold metal and bicycles, so a collaboration engine might tell you that others who are interested in both are also interested in the ocean. For Wikipedia, this might not be an attractive feature. But it DOES open the doors to what the collaboration engines might need as a step in the next direction: not just a filter that tells you what you’d like or won’t like, but also a filter that offers more than just a rating, but even a “distance” from another rated item. You might like a drama movie, and you might like a comedy movie, but they may not really be “close” in terms of relevancy to one-another. They’re both movies, they both are available on DVD, but they’re far enough apart that relevancy would need to be sorted by another factor: such as topic or category. The CRITEO engine does allow for these secondary sorts, so the opportunity to develop an interesting Wiki-plugin exists.

Who’ll be the first to try it?

An API by itself can add great value for both a site visitor and a site operator. An API by itself can offer a huge return-on-investment for both user and operator, too. Yet integrating one API with another API, a so-called API mash-up, can offer an order of magnitude more for both user and operator.

Picture, if you will, a website dedicated to restaurants. Some of these websites cover the world, some a nation, some a specific region such as a city or a county. While other users can rate specific restaurants, those ratings are very limited. This is where a predictive relevancy engine such as CRITEO can help add value to those ratings — it allows various users to see what else they might like or dislike, and allow the to cultivate a better relevancy by continuously adding to their ratings. But for some, that might not be enough. A “I like it” or “I don’t like it” rating is great, but what more could you offer?

There are incredible APIs out there for user-interactivity: Google Earth’s API is awesome since it can allow a site operator to limit a user’s view of their site to a specific region, maybe a 10 mile radius to where the user lives. The relevancy predictor could return a list of dozens of restaurants the user might like (or dislike, or whatever the user filters by), but a connection to an external API such as Google Earth or Google Maps can allow the user to limit their list from a huge international list of restaurants to one that lets them focus on what is important to them. A network of sites dedicated to various regions can co-exist within the same relevancy prediction, allowing users to travel to other areas before a vacation to see what they might like.

I might run a review site on my region (Lake County, Illinois), but I might want to work with hundreds of other sites that cover specific counties, too. If we share the same prediction engine through a cooperative network, my regular users could allow their past decisions to reflect what they might like elsewhere. Say a Chicago resident wants to go to Las Vegas, and wants a good review of Italian restaurants. Right now they might go to another site than mine, but their information there might be irrelevant to their likes and dislikes. By combining a variety of APIs real-time, and allowing a co-op of individual sites, a relevancy engine could really flourish by taking into account the current user’s ratings, and diagnose other users within the network and their likes and dislikes. A user can make a decision based on the multiple-API predictions, and after actually visiting the restaurant, their review can increase the relevancy for thousands or even millions of other users who normally have no connection to one-another.

A prediction engine API is useless without a reason for existing. The most basic reason for utilizing a prediction engine is to entice users with more that you offer. Many people think that prediction engines are good for only one thing: selling items online. Yet this is short-sighted, and that mindset must change. A prediction engine can have MANY uses for anything a consumer might want — sometimes even more information.

One area I see the prediction engine being the most powerful in is visitor retention, and not just for consumer items such as buying DVDs or toys. Imagine a blog that covers a niche topic such as programming techniques or guides. A blog like this would have a variety of tags and categories, allowing users to traverse the site based on finding articles of common topic. Yet that is a very uninteractive experience — users don’t really have much input into what they like and dislike about articles. Even the sites that ask for user input do almost nothing for the user while they’re on the site. Asking for a rating is helpful for long term changes, but it is completely a waste for the user RIGHT NOW. Why should a user bother rating a site or a page if they don’t get anything out of it (I call this an information profit). How about tossing that user’s ratings into the prediction database, and allowing that engine to generate a list of articles the user would like? By giving real-time feedback based on a user’s preference, the engine gives the user instant reason for taking 5 seconds to judge an article’s value to them. The value by itself is useless for the short term, but the aggregate of the user’s opinion combined with all the users on a site offers huge value for the user now and the site operator in the future.

The power for an individual blogger or web operator is big, but it is also big for blog networks or any network of information provided. By integrating the decision making process across a network, even a loosely-organized network that is not really integrated under one topic or idea, can reap huge dividends for the site operator in reader retention, while also providing the user with an interesting path of topics to read. The user’s investment is almost nothing — 5 seconds to click a rating. With a Web 2.0 or AJAX interface, the user won’t even be taken off the page they’re on, but they could receive an instant update to interesting articles. The user’s profit is just that — more information for the information-hungry visitor. The web operator’s investment is taking the little bit of time to learn the prediction engine API, and possibly a small financial investment to provide for the server-time needed to process the information. Yet with a system such as CRITEO, the web operator’s costs are scaled — few users, little cost. Many users, higher cost. More users = more advertising dollars. More users sticking around = much more advertising dollars. And the best part: once the web operator takes the time to connect to the predictive engine, the rest is simple. There is almost no upkeep, maintenance or modification needed for the web operator to keep using the product, but the product will continually adapt to both the web operator’s and the visitor’s needs.

The user can also instantly let the web operator know that an article is useless or useful, and the web operator can run reports to diagnose what they SHOULD write about next. This is incredible power for almost no real price time-wise or money-wise.

Buying gifts for people revolve around 3 processes: knowing what they want (wish lists or gift registries), knowing what they might want (based on hints from them or friends) and taking risks to think they may want an item you’ve come across. Wouldn’t it be interesting if we could use a relevancy engine such as CRITEO to come up with gifts for friends and relatives at Christmas-time or birthdays and anniversaries? Imagine taking two married users and letting the prediction engine show what BOTH might want or need.

Right now we have individual wish lists available at many online retailers as well as big retail chains. The Target Corporation allows anyone to create a wishlist that anyone else can print and fulfill at any location. But these are very one-dimensional, only based on what the actual wish-list creator produced. What keeps these retailers from entering into an agreement to use a prediction engine? For now, it is definitely a chicken-and-egg situation!

A retailer won’t use the engine until they can enter user data. Users won’t know about the prediction ability until the retailer utilizes the product. Yet most retailers, even small-sized ones, can easily produce a history of past purchases. Retailers can track purchases based on credit card numbers used (last 4 digits plus date), or on acquired phone numbers. Retailers can even create group-users based on zip codes or other acquired information. By providing this information to a prediction engine (through an API), the retailer can immediately start offering predictions on relevancy for others to use to make gift-buying decisions. Buying gifts are one of the hardest things to do — we never know if someone wants something we’ll buy them. Yet gift receipts are commonplace now, for this very reason. If a retailer can use returned gifts as a resource for provider better relevancy for future predictions, the gift-giving market could blossom and the retailer could get a real return-on-investment: better predictions mean less returns AND happier customers. A retailer can put themselves ahead of the competition by offering this sagely advice.

One shortcoming of most relevancy predictor engines is the inopportunity to help a user decide if they need something or they might merely want something. What is the opportunity to change this? Right now, all prediction engines are based on a common idea: a thumbs up/down rating, or a rated score (say, 1-10 or 1-5). I forsee the chance to introduce a 2D rating system that can allow users to rate items based on real need (1-10) and desire (1-10). This engine would allow for people to create wishlists, too!

Imagine if I buy eggs and I say I rate them a 9 in terms of need, but a 6 in terms of desire. A 2D engine is incredibly more complicated to manage, but report generation should be as simple as the other functions. By telling users what else they might need, but also what they might want, it can really make life easier for both the user as well as family and friends of the user, especially at Christmas time. If you know that brother Joe wants a new bike, but needs pants, you can prioritize your gift-buying. You might also get insight into what Joe MIGHT want or need, without Joe even realizing it. Talk about a surprise gift!

When you go shopping for a retail product — be it clothing, music, movies, food or any consumer item — you usually go to pick a product and leave. While you are shopping, though, you might purchase items above and beyond what you hoped to get. This is part of the retailer’s aim — to get you to buy more than you originally wanted, with the hope that you buy something you need. Yet even the smallest retail store is overwhelming for any shopper with a limited schedule or with limited funds. The items that are marketed the best are usually those on endcaps or in the middle row of items in an aisle. This is extremely limiting, though, because it removes the chance to find obscure items that might actually be relevant for the consumer’s needs or wants.

I believe that a relevancy engine such as CRITEO could be utilized outside of the web, directly by retailers. I can imagine two functions in the retail environment — one that would require almost no real initial or long term investment, and one that would require an initial investment but not much longterm commitment. Both ideas would be wonderful for consumers, distributors (the store) and producers (the manufacturers).

The first idea would be the receipt ad. Right now, if you go to a grocery store in the U.S., you are usually offered coupons from competitors of the products you actually bought. When I purchase cat food from Fancy Feast, I usually get coupons to try a competitor’s cat food. I usually toss these coupons since I rarely find reason to switch products — even if I can save financially. The time investment to try the product really precludes me taking the risk on a new product. This is common for most consumers, the great majority who are not enticed by coupons or sales. While we may not be interested in purchasing competitive products, I do believe many consumers would be interested in a form of direct-marketing based on their selected items. If the retailer can input your purchases into a relevancy engine such as CRITEO’s, they could offer you real-time items that might complement what you’re buying. If I am buying cat food and cat litter, I might be interested in some support products such as cat toys or cat shampoo — items that are not promimently displayed and might be passed by my consumer eyes. If I am buying eggs and bacon, such a relevancy system might remind me that I need toast or milk. If I am a regular visitor to a particular chain of stores, the relevancy system might even go so far as look at my recent purchases and notify me if I haven’t bought a particularly relevant item as I had in the past. The CRITEO API should allow for not just comparison with other shoppers like me, but with my own shopping history! Also, the store could allow for the ability to produce a “reminder list” not just on my receipt, but also when I enter the store. A quick swipe of my preferred customer card should allow the store to print a list of items I might need, items I might not know about, or items that others like me purchased and liked.

Should I return an item I tried based on a suggestion of the relevancy predictor, I should be able to rate that item (either in the store, or possibly online). The store should give me a reward for returning the item AND rating it. These ratings would allow the prediction engine to remove other items that I probably won’t like. This is using a conglomeration of many individuals to provide for very unique relevant items for each individual separately.

The second item that would be amazing would be a real-time analyzer. The days of electronics being integrated in odd ways is coming ahead. The shopping cart hasn’t changed in decades. Can you imagine a shopping cart with a integrated LCD display? When you pick up an item, you can quickly barcode scan that item (or do it instantly with RFID) with the display to show you other items that can integrate with the item I am purchasing, or give me a heads up to what I might actually like. If I put eggs and milk in my cart but not cereal or bacon, the cart can easily diagnose my past performance and find items I normally buy (again, integrating with purchase history to not remind me about things I bought within the last week or whatever time frame). The cart can offer me intriguing purchase ideas — maybe the cart can even let me know of recipes I might like based on the items I’ve bought — recipes that could use just one or two additional purchases to finalize.

Surely such a shopping cart would be an expense to purchase, but a retailer would not have to make a significant number of them available. If a consumer is interested in new items, they can use the interactive cart. The return-on-investment should be huge since the cart is a one-time purchase, whereas the relevancy engine is a set cost with a return of some kind.

The relevancy predictor market is one that is focused mostly on the web, but it is short-sighted to think that it must stick ONLY with the web. By providing off-the-shelf turnkey solutions for even small retailers, the relevancy engine could increase its reach by orders of magnitude. Retailers want more information for their users to use, but most are closed off from the emerging technology scene. Could a relevancy predictor engine be integrated to the retail market? Absolutely!

It was by chance searching that I came across CRITEO, a French company focused on personalized relevancy sorting through any kind of data. The company is a recent startup in the last few years, focusing on a new pre-emergent market product that is utilized through an API, allowing websites of all sorts to enter user preferences and come up with a real-time match for each user. Currently they are demonstrating their API for YouTube videos and Amazon movies — their system operates much like a “thumbs up, thumbs down” style interface, where users can rate a variety of products and be shown relevant products that should interest them.

The CRITEO example is limited, though, only because their current drive is to show website operators the power of their API. The limitation is a requirement for their early business, since they are not concerned with specific markets — they’re concerned with markets finding relevancy in this product. For most Web 2.0-style applications, the market is relatively unknown. Instead, the hive of users that can use a product will be the ones that develop a new market, regardless of what the developers of an interface desire. This is good for the end-users, good for the product suppliers, and good for the company acting like a middleman.

CRITEO is offering their entry-level API free of charge for website operators to use and test. Playing with their Amazon relevancy site is incredible — in less than 5 minutes of rating movies, I was really impressed with how quickly their engine offered me insight into what I’d like. 5 more minutes gave me an incredibly accurate diagnosis of what I’d like and what I won’t like.

This blog is an attempt to discuss the idea of relevancy from a perspective outside of CRITEO and the current players in the consumer-relevancy market. I hope to discuss ideas on what these engines can do outside of the norm, outside of the box, and outside of common thought. Without taking risks, website operators will not realize rewards. I believe entering into a relationship with a company such as CRITEO can give the common and complex website operators significant retention in terms of users, and therefore customers.