Building a local intelligence service - LIS integration and interoperability
Sandwellâ€™s LACI project What we did Automated data retrieval into the system
Pilot LIS launched Dec 09. Full launch August 10 (www.sandwelltrends.info). Developed two new approaches to increasing LIS data holdings, to cover national contextual data and local partner and service data. The two approaches were the development of the NeSS data monitor and a Data Brokerage software installation.
Why we did it NeSS data monitor Collect once, use numerous times (COUNT) Consistent approach
Piloting an approach
Data brokerage Local data sharing
Reducing the demand for ad hoc sharing Disclosure considerations
In order to reduce officer time spent downloading contextual datasets from national websites, e.g. for needs assessments, a tool was developed for bulk retrieving of datasets from ONS neighbourhood statistics database, using the protocols of the neighbourhood data exchange (NDE). This enables a visualisation of such data with the online tools in www.sandwelltrends.info, and enables the bringing together of both local and contextual data within a single trusted environment. The NeSS database was chosen for this as the NDE was well established, but the approach, whilst not entirely generic, would serve as a pilot for developing a similar approach for other national database resources, such as NOMIS and data.gov.uk. The more local aspect of this project, and potentially the more rewarding is the development of an installable piece of software to enable two organisations to share data in a secure, consistent, automated fashion. Much of a research officerâ€™s time can be consumed in supplying data on demand, often the same dataset many times. The data brokerage approach enables the researcher to do this once for such datasets. Even more officer time is consumed if the raw data has to be amended (e.g. through aggregation) before external supply. Furthermore, data governance may require that a sharing protocol is prepared and agreed between the parties before this can happen. The data brokerage solution offers the prospect of agreeing a single sharing protocol, particularly if the data is disseminated via a single portal (e.g. Sandwell Trends), although it does not preclude the option to share with multiple organisations. The software can be configured for each dataset shared, such that it receives the raw data, but carries out necessary aggregation, such that the data supplied conforms to data protection rules. An important aspect of this project was to demonstrate how it could enable partner organisations to share data. For this reason the pilot approach chosen was the sharing of data (birth data in this case) between the PCT and the local authority. Whilst the health context was chosen to pilot this approach, it was to be followed up with further development to make the product configurable to be used in any context.
What worked NeSS data monitor
The NDE query protocols have been successfully developed (Coop Web) and trialled with a number of NeSS datasets. An interface has been developed for choosing datasets for bulk download. For optimum performance, the actual data transactions take place overnight. The status of all requested datasets is viewable on-screen as part of this interface design. Once the data is delivered into the Sandwell Trends database (MySQL), further processes are required to make these available through the LISâ€™s query screen for viewing within the online visualisation tools. A simple sequence of repeatable standard processes has been developed to enable this to happen (in-house resources).
Both phases of development work (pilot software and configurable software) have been completed and delivered (Coop Web). Testing of the passing and aggregating of dummy data has been undertaken to demonstrate the effectiveness of the solution in principle, but further work is required to achieve a practical
implementation (see ‘Lessons learnt’). A web-based interface has been developed to handle the programming and configuring of datasharing transactions.
Lessons learnt Currency Transferability
Selectability Higher geographies
Many of the datasets in the NeSS database are from the census, making them of limited current interest. The techniques developed for this tool cannot simply be applied to other such national data stores (e.g. NOMIS and data.gov.uk) – some specific development is required for each, requiring a further cost. The semantic web is not yet as transparent as it is sometimes claimed to be. It is useful to have a selective approach (rather than a wholesale download), to avoid clogging up servers with unnecessary noise. Greater value may be derived from the NeSS from borough-level data, regional and national comparator figures. Limited work has been done on this aspect as yet, but see Sandwell Trends’ existing benchmark tools for NIs and CIPFA stats, as a potential application. (E.g. http://www.sandwelltrends.info/lisv2/navigation/fin_benchmark.asp)
Data brokerage Data governance issues
The choice of pilot dataset threw up some interesting data governance problems. The ONS owns the data, and specifically forbids the sharing of the monthly Public Health Birth Files (PHBF) with local authority colleagues. For this reason, the pilot was limited to the annual dataset in the first instance. The ONS is apparently currently reviewing these constraints.
It was relatively straightforward to achieve agreement with the principles behind the infrastructural technology requirements for installing data brokerage within the NHS secure network. However, this led to a level of optimism that ultimately proved misplaced. As it turned out, the organisational barriers to the necessary technology (and consequential financial barriers) proved to be insuperable within the time and budget constraints of the project.
Choice of pilot too ambitious?
With hindsight, it may have been too ambitious to attempt to achieve a practical implementation with the NHS in the first instance, despite the excellent collaboration and commitment from the Public Health Intelligence team colleagues.
It is hoped that the opportunities afforded by the GCSx-N3 Interconnect Project may prove fruitful for the successful deployment of this approach within the health context in the near future (see ‘What next?’).
What next? NeSS data monitor
Further data to be downloaded and visualised, including a facility to easily find the Sandwell figure for an indicator, to include regional and national comparators in Sandwell maps and charts, and to develop Sandwell Trends’ benchmarking tools to include ONS contextual data. Protocols developed to be applied and further developed for other similar databases, e.g. NOMIS and data.gov.uk.
Installations with other partners and/or other council databases will be pursued. Work will continue on sharing health data, e.g. via N3 connection, or using the GCSx-N3 interconnect option. Discussions are being pursued sub-regionally (i.e. Black Country) about a collaborative approach to this.
Contacts Sandwell MBC Sandwell PCT Co-operative Web Ltd
Alex Hawley, email@example.com. 0121 5693008 Ralph Smith, firstname.lastname@example.org. 0121 6121416 Ed Russell, email@example.com. 0845 4786322 www.web.coop
NeSS data monitor
Published on Sep 11, 2011