Not so Open Government Data

New frameworks and technologies are often documented with small textbook examples. Naturally, these pet shops and the like always work flawlessly and are "up and running" in five minutes. The trouble is, they have nothing to do with real-world projects. That's why we prefer to evaluate new components through incubator projects. Our team scrutinises ideas brought in by our staff, then builds them as prototype applications using exactly those new technologies and frameworks.

One such project in 2014/2015 was "ju§earch!" When it comes to publishing legislation and court rulings, there are hardly any tools that link the data from the various sources together. The tools that do exist are expensive and rather dated. So we found the field interesting, especially as the data is public. Both legislation and the rulings of the various courts are available online and fall under the umbrella of "Open Government Data".

Our prototype was one of our first Scala projects. The use case was ideal for making the most of Scala together with the Play! web framework and the MongoDB NoSQL database. And so it proved. The prototype quickly took shape. At first, we pulled the data straight from the websites of the Federal Supreme Court and the Classified Compilation of Federal Legislation and parsed the information.

So we had a proof of concept. And we had interested users. What didn't convince us was the way we had to collect the data, and the data quality that came with it. On top of that, the web publications left out a lot of metadata and structural information that clearly existed. What "ju§earch!" needed was machine-readable data. XML or any other structured format.

We told the Federal Chancellery and the Federal Supreme Court that we needed structured data. The Federal Chancellery understood our request and pointed us to an ongoing project that would make the structured data available once it was finished – at some point, anyway. At the Federal Supreme Court, our request fell on deaf ears. We were referred to a paid platform that offers this kind of structured data. Paying CHF 5,000 a year or more for data that should really be public was too much for us. After all, our search platform was meant to stay free and openly accessible in the future too.

Under these conditions, we stopped developing our prototype, and in September 2017 we also shut down the technology preview. In those two years, nothing had changed in terms of data availability. The Federal Supreme Court is sticking with the status quo, and in the meantime opendata.swiss has published the Official Compilation of Federal Legislation up to 1998 – even in Akoma Ntoso format. More recent and current legislation, however, is still only available online or as PDF.

If public data is to be put to good, efficient use for new ideas, several federal agencies need to rethink how they publish it. When that shift will happen is anyone's guess.