Credence, or how to differentiate spam from legit content

CommentaryZDNet mentions an interesting Cornell University research project called Credence. Credence aims to differentiate spam from legit content in peer-to-peer networks. Basically, it involves a distributed voting scheme where users can contribute positive and negative evaluations of files, and users. The algorithm then collects random votes and weighs sources by trustworthiness. My interest in this kind of voting schemes is long lasting, and I believe Credence’s approach is flawed.

I mentioned a distributed voting scheme indirectly in a recent post on the future of trackbacks and pingbacks. Feature problems set aside, trackbacks and pingbacks both greatly suffer from potential spamming schemes. Thus, I find myself thinking about and designing, from time to time, a protocol to replace both; ideally, this protocol would integrate anti-spam features.

Now, straightforward pattern-recognition techniques aside, one of the key aspects to solving the trackback and pingback spam problem will involve some kind of user voting scheme. Moreover, this voting scheme will be similar to the one that is necessary to filter peer-to-peer network content. And, while interesting, Credence’s approach is wrong in that it involves an absolute measurement to determine a source’s trustworthiness.

As explained in the overview of credence:

Each vote collected from the network is not used directly, however, since some peers in the network may accidentally vote incorrectly, or even lie intentionally about the file’s authenticity. Therefore we assign to each peer a correlation coefficient, or weight, reflecting the historical usefulness of the peer’s votes. In effect, this helps remove the incentive for an attacker to lie about the authenticity of files. A consistent liar is, after all, just as useful as an honest peer when it comes to distinguishing authentic files and pollutions. And an inconsistent voter will come to be be ignored by others in the network.

Thus, if you introduce a sufficiently big number of bots who consistently lie about files’ authenticity, and who consistently report they trust each other, credence will suddenly sprout an unwanted side effect: Trustworthy files will be rated as untrustworthy and vice-versa on the one side, and trustworthy users will be rated as untrustworthy and vice-versa on the other.

A solution exists, of course: It involves revisiting the psychology of groups, so as to introduce a relative criteria. Specifically, and opposite to a wide spread assumption and belief (e.g. James Surowiecki on the Unwisdom of Crowds), you and I do not simply imitate other people; you and I are much more likely to imitate people that we trust.

This shapes a huge problem: Trust is extremely complex to model because you and I don’t just trust people. We trust people based on context and past experience.

As a side note, if you fully understand the implications of this, feel free to contact me: We should be designing next generation computational linguistic technologies together.