A supervised learning process to validate online disease reports for use in predictive modelsReport as inadecuate




A supervised learning process to validate online disease reports for use in predictive models - Download this document for free, or read online. Document in PDF available to download.

Reference: Patching, HMM, Hudson, LM, Cooke, W et al., (2016). A supervised learning process to validate online disease reports for use in predictive models. Big Data, 3 (4), 230-237.Citable link to this page:

 

A supervised learning process to validate online disease reports for use in predictive models

Abstract: Pathogen distribution models that predict spatial variation in disease occurrence require data from a large number of geographic locations to generate disease risk maps. Traditionally, this process has used data from public health reporting systems; however, using online reports of new infections could speed up the process dramatically. Data from both public health systems and online sources must be validated before they can be used, but no mechanisms exist to validate data from online media reports. We have developed a supervised learning process to validate geolocated disease outbreak data in a timely manner. The process uses three input features, the data source and two metrics derived from the location of each disease occurrence. The location of disease occurrence provides information on the probability of disease occurrence at that location based on environmental and socioeconomic factors and the distance within or outside the current known disease extent. The process also uses validation scores, generated by disease experts who review a subset of the data, to build a training data set. The aim of the supervised learning process is to generate validation scores that can be used as weights going into the pathogen distribution model. After analyzing the three input features and testing the performance of alternative processes, we selected a cascade of ensembles comprising logistic regressors. Parameter values for the training data subset size, number of predictors, and number of layers in the cascade were tested before the process was deployed. The final configuration was tested using data for two contrasting diseases (dengue and cholera), and 66%–79% of data points were assigned a validation score. The remaining data points are scored by the experts, and the results inform the training data set for the next set of predictors, as well as going to the pathogen distribution model. The new supervised learning process has been implemented within our live site and is being used to validate the data that our system uses to produce updated predictive disease maps on a weekly basis.

Peer Review status:Peer reviewedPublication status:PublishedVersion:Publisher's version Funder: Bill and Melinda Gates Foundation   Funder: Wellcome Trust   Notes:Copyright © Helena M.M. Patching et al. 2016; Published by Mary Ann Liebert, Inc. This Open Access article is distributed under the terms of the Creative CommonsLicense (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided theoriginal work is properly credited.

Bibliographic Details

Publisher: Mary Ann Liebert

Publisher Website: http://www.liebertpub.com/

Journal: Big Datasee more from them

Publication Website: http://www.liebertpub.com/big

Issue Date: 2016

pages:230-237Identifiers

Urn: uuid:7d8d63cc-762d-4f0c-8441-3598aedc4bd1

Source identifier: 591030

Eissn: 2167-647X

Doi: https://doi.org/10.1089/big.2015.0019

Issn: 2167-6461 Item Description

Type: Journal article;

Version: Publisher's versionKeywords: big data analytics data acquisition and cleaning machine learning structured data Tiny URL: pubs:591030

Relationships





Author: Patching, HMM - - - Hudson, LM - - - Cooke, W - - - Garcia, AJ - - - Hay, SI - institutionUniversity of Oxford Oxford, MSD, NDM,

Source: https://ora.ox.ac.uk/objects/uuid:7d8d63cc-762d-4f0c-8441-3598aedc4bd1



DOWNLOAD PDF




Related documents