Data Interoperability and the IoTUK Test Beds
This blog forms part of a series that looks at the technical experiences of three IoTUK projects that ran between 2016 and 2018.
Each of the three projects implemented their own IoT technology platform and in this blog we’ll explore how they addressed data interoperability between the multiple layers of the solution. You will learn about the schemas that were used and how Hypercat was used to aid discovery of new data sources.
CityVerve is the UK’s smart cities demonstrator, based in Manchester. CityVerve explored how IoT technologies and the forging of new relationships between the public and private sector can make a city truly smart. The resulting solution was a Platform of Platforms connected by a central portal that exposed data from different elements of the city. The data was open to 3rd party developers allowing them to create new and innovative apps.
Diabetes Digital Coach (DDC) is an NHS IoT Test Bed project and has been developed by consortium of 10 technology and evaluation partners, led by the West of England Academic Health Science Network (AHSN). DDC offers an online service to help people with Type 1 and Type 2 diabetes manage their condition and cut the risk of complications. It is available via computer, smart phone and tablet and brings together a number of digital self-management tools, which provide personalised support. When people sign up for an account with the Diabetes Digital Coach, they provide information about themselves, their health, lifestyle, and how they currently manage their diabetes. The Coach then suggests the most appropriate tools to suit their individual needs. The Coach features five tools which have been carefully selected by both healthcare professionals and people with diabetes; this ‘menu’ covers self-management education, dietitian support, optimising physical activity, insulin and glucose management, and a personal health record.
Technology Integrated Health Management (TIHM) is an NHS IoT test bed aiming to transform support for people with dementia and their carers. The collaboration involves partners from the health, voluntary and technology sectors. Each family in the trial was provided with a home technology pack that was suited to their particular needs. Like the DDC portal, TIHM connects different technology partners but here there is real-time correlation between different data points allowing insights to be gleaned. Any unusual signs are flagged to clinicians who will decide on the appropriate action to take.
The data collected in the TIHM project originates from disparate sources such as environmental, medical, wearable, and interactive applications. The data is published by multiple providers, meaning that the original data may have different formats (e.g. CSV, XML, or JSON). Having such a wide range of data presented a challenge in accessing, processing, integrating, and interpreting the data. It was important to define a common data model to provide the interoperability and allow for the integration of multi-modal data.
TIHM adapted Health Level-7 (HL7) which refers to a set of international standards for the transfer of clinical and administrative data between software applications used by various healthcare providers. HL7 specifies a number of flexible standards, guidelines, and methodologies by which various healthcare systems can communicate with each other. In particular, the Fast Healthcare Interoperability Resources (FHIR) is a new standard derived from HL7 designed to be easier to implement, more open, and more extensible than other versions of HL7. FHIR makes use of an HL7-defined set of “resources” to support information sharing by a variety of means, including documents, messages, etc. This made it suitable for use in a wide variety of contexts, such as mobile phone apps, cloud communications, and server communication in large institutional healthcare providers.
Since the collected data in TIHM covers a wide range of measurements, and not just clinical, it was necessary to adapt the FHIR model so that it complies with TIHM requirements – this led to the development of FHIR4TIHM3 data model. The main FHIR resources used in this project are:
- Questionnaire Response
- Detected Issue
Additional resources were also created to facilitate the needs of this project. All resources are reusable structures where each resource has logical table, UML definition, block diagram, and XML or JSON template. Fig 1 shows the UML and structure for the ‘Observation’ resource.
With data consistency in mind across data publishers, validation of the incoming data must be strict. A JSON validator was setup on the central TIHM backend to validate the incoming data. If the data received on the TIHM server is invalid, the corresponding publisher would be notified about the failed validation and the reasons of invalidity. Otherwise, the valid data would be stored in a database for analysis.
Each technical partner within DDC made use of their own existing platforms and data schemas. However, data that was sent to the central portal had to conform to a common schema. DXC is a large enterprise partner that already had a widely used healthcare schema. DXC’s Clinical Data Repository (CDR) was therefore adopted as the common schema for the project.
As with the DDC and TIHM, CityVerve also mandated data schemas to standardise the format information was published in. Rather than a single schema covering all aspects of city data, they opted for well-established schemas for each data type to reduce the effort for independent developers. There were essentially 4 types of data:
- Sensor data – The most common type
- Event data with context
- CAP – Common Alerting Protocol for Planned events (football match) and Unplanned events (traffic accident)
Geo-Spatial data – Points, lines, polygons & attributes for information type, eg Bus Stop
- Journey data – Series of timestamps
- DATEX II – used for traffic data
To further simplify the app development, CityVerve APIs were designed to allow developers to request data in three different formats to reduce the number of transformations tasks. The available formats were:
Within the CityVerve API, Hypercat was used as the cataloguing technology. HyperCat is an existing specification which is designed to provide automatic resource discovery for the Internet of Things. Hypercat itself does not store any IoT data, applications must retrieve the data from publishers described in the resource description. The key design goal of HyperCat is to “provide interoperability over entities and/or services for discoverability”. Hence it is very aligned to the goals of the CityVerve API.
Applications connect to a known Hypercat using HTTPS/REST and resource descriptions of IoT assets are returned in a JSON format.
Technology partners involved in the TIHM project provided their own Hypercat catalogues containing a full list of devices used in the project, along with their description and a link to their latest observations.
CityVerve is a Platform of Platforms and three of the main hubs within the project each have their own HyperCat catalogue. Some of the hubs did inherit CKAN to manage the publication of open data. Existing CKAN functionality is now being migrated to Hypercat for consistency across the project.
The Hypercat Catalogue for CityVerve is available for anyone to view here.
Common patterns across the IoTUK projects
All the projects made use of existing data schemas to accelerate data interoperability. TIHM adapted the FHIR standards whereas DDC used a proprietary schema developed by their major enterprise partner, DXC.
Hypercat was used by all projects to expose new sources of data in a standardised way. This was particularly useful at CityVerve where new applications continue to be built by 3rd party developers.
In the next blog in this series, we’ll take a closer look at the data analytics performed by the test beds.