Departmental Papers (CIS)

Date of this Version

8-28-2012

Document Type

Conference Paper

Comments

Eighth International Symposium on Wikis and Open Collaboration, Linz, Austria, August 2012.

Abstract

As evidenced by SourceForge and GitHub, code repositories now integrate Web 2.0 functionality that enables global participation with minimal barriers-to-entry. To prevent detrimental contributions enabled by crowdsourcing, reputation is one proposed solution. Fortunately this is an issue that has been addressed in analogous version control systems such as the *wiki* for natural language content. The WikiTrust algorithm ("content-driven reputation"), while developed and evaluated in wiki environments operates under a possibly shared collaborative assumption: actions that "survive" subsequent edits are reflective of good authorship.

In this paper we examine WikiTrust's ability to measure author quality in collaborative code development. We first define a mapping from repositories to wiki environments and use it to evaluate a production SVN repository with 92,000 updates. Analysis is particularly attentive to reputation loss events and attempts to establish ground truth using commit comments and bug tracking. A proof-of-concept evaluation suggests the technique is promising (about two-thirds of reputation loss is justified) with false positives identifying areas for future refinement. Equally as important, these false positives exemplify differences in content evolution and the cooperative process between wikis and code repositories.

Subject Area

CPS Internet of Things

Publication Source

8th International Symposium on Wikis and Open Collaboration (WikiSym '12)

Start Page

13.1

Last Page

13.4

DOI

10.1145/2462932.2462950

Copyright/Permission Statement

© ACM 2012. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 8th International Symposium on Wikis and Open Collaboration (WikiSym '12), http://dx.doi.org/10.1145/2462932.2462950.

Keywords

WikiTrust, wikis, code repository, SVN, reputation, trust management, content persistence, code quality, software engineering

Share

COinS
 

Date Posted: 04 June 2013

This document has been peer reviewed.