An Architecture for Justified Assessments of Service Provider Reputation

In a service-oriented system, an accurate assessment of reputation is essential for selecting between alternative providers. In many cases, providers have differing characteristics that must be considered alongside reliability, including their cost, experience, quality, and use of sub-providers, etc. Existing methods for reputation assessment are limited in terms of the extent to which the full interaction history and context is considered. While factors such as cost and quality might be considered, the assessment of reputation is typically based only on a combination of direct experience and recommendations from third parties, without considering the wider context. Furthermore, reputation is typically expressed as a simple numerical score or probability estimate with no rationale for the reasoning behind it, and there is no opportunity for the user to interrogate the assessment. Existing approaches exclude from consideration a wide range of information, about the context of providers’ previous actions, that could give useful information to a user in selecting a service provider. For example, there may have been mitigating circumstances for past failures, or a provider may have changed their organisational affiliation. In this paper we argue that provenance records are a rich source of information on which a more nuanced reputation mechanism can be based. Specifically, the paper makes two main contributions: (i) we provide an analysis of the challenges and open research questions that must be addressed in achieving a rich provenance-based reputation mechanism, and (ii) we define an architecture in which the results of these challenges fit together with existing technologies to enable provenance-based reputation.