Standards enable publication of new data

“Which datasets should I publish?” and “How should I publish them?” are frequent questions from data publishers who are looking to publish open or shared data.

Open standards for data can:

• help publishers identify datasets others are sharing

• act as a guide to help publishers share data in consistent ways

• make it easier for data users to reuse shared data

• encourage publishers to share data that was previously closed

• build publishers’ confidence that they are sharing useful data

• encourage the development of new tools for data users and publishers

Publishing data in standard ways means existing standard tools can be used by the publisher and their stakeholders, as the following case study shows.

Case Study: UK Department for International Development (DfID)

In 2010, DfID launched the UK Aid Transparency Guarantee (UKATG), committing the UK government to greater transparency and clarity on international aid spending.

Partner organisations receiving grants or contracts from DfID must publish tracking information using IATI (International Aid Transparency Initiative), unless specifically exempted.

As part of a data use pilot with Publish What You Fund and Development Gateway, DfID reported that new information was published by their partner organisation which helped them understand activity and aided making decisions in countries receiving aid.

Standards produce better quality data

In her book ‘Executing Data Quality Projects’, Danette McGilvray defines data quality as “the degree to which data can be trusted for any required use”.

Open standards encourage the development of tools and services to help data publishers produce good quality data, including tools to validate, preview and compare data.

Open standards can advise data publishers how often data should be published. Some standards include ways to share publication schedules, publication dates, location and methods of accessing data. Sharing this information makes it easier to trust published data.

A data publisher can publish good quality data without standards. However, open standards reduce barriers to data publication by allowing publishers to focus on sharing consistent and quality data.

Case Study: US Environmental Protection Agency

As a federal agency protecting human health and the environment, the US Environmental Protection Agency (EPA) collects environmental data from a number of organisations.

The EPA developed data standards and the Central Data Exchange (CDX) platform to address issues of data quality and cost. The EPA case study describes the benefits of using open standards for environmental data, including improved consistency and reduction of redundant data.

Standards encourage creation of new tools and services

When data is published consistently, the time, cost and processes involved in using it are reduced.

For example, 353 councils in England are required to publish information about grants. As each dataset is different, combining data about grants becomes a bespoke activity for each council.

Consistent publication encourages the creation of new tools and services that are designed to take advantage of data that conforms to a standard.

Tools and services can be developed to:

• support data publishers by making it easier to validate and publish data that conforms to the standard – for example, the Local Government Association data validator for validating data published to the Brownfield Site Register Open Data Standard

• support data users by making it easier to analyse and use data that conforms to the standard – for example, Development Gateway created procurement analytics tools to support using the Open Contracting Data Standard

• support developers by making it easier to access and use consistently formatted data that conforms to the standard – for example, Google developed tools for testing, merging or viewing files that conform to the General Transit Feed Specification (GTFS)

Case Study: Building & Land Development Specification

The Building and Land Development Specification (BLDS) standard focuses on publishing building and construction permit data consistently in the US, where each state collects permit data differently. The standard was developed collectively by technology and building services companies in the private sector, and US state governments.

Tools developed using BLDS:

• Boulder County developed a pilot open data platform to extend access to land use information using the open standard for the public and developers

• Accela built a year in review app so data users can compare permits across regions

• Socrata support data publishers sharing BLDS permit data on their platform