How many cases? Assessing the comparability of EU Judicial datasets

The paper discusses the consistency, reliability, and compatibility of the data set and the appropriateness of indicators used by the EU Justice Scoreboard to assess the efficiency of European justice systems. The EU Commission has introduced the Justice Scoreboard as a tool providing objective, reliable, and comparable data on the functioning of the justice systems of all Member States (EU Commission 2013, p.3). This instrument is formally directed toward strengthening the rule of law, the functioning of national justice systems, and upholding more effective justice. The tool identifies the comparison of the efficiency of Member States justice systems and of other variables as specific means to reach such objectives. This paper analyses conceptual flaws and data quality problems, which undermine the use of the tool. From a conceptual perspective, the study makes clear that caseload data are used to measure workload. Furthermore, the Scoreboard uses the numbers of cases (incoming, pending and resolved) and indicators of effectiveness like clearance rate and length of proceedings to measure the efficiency of justice systems. The assessment of how Member States count apparently simple categories of cases as administrative and litigious or non-litigious civil and commercial cases shows cross-country inconsistencies that make a cross-country comparison at least problematic. While suggesting some clear actions to improve the flaws, the paper advice to take an extremely cautious approach in using such indicators for comparative purposes in the academic debate and policy-making.