"By working closely with our collaborators in Thailand, we have developed a framework that can bring advanced modeling techniques to the inevitable messiness of real-time data.”
In the current issue of PLOS Neglected Tropical Diseases, they describe a unique collaboration between public health officials in Thailand and a U.S.-based research team that focuses on merging real-time data management with advanced analytics. They developed a model that improves several key practical features of prediction techniques, offering strategies that account for reporting delays in data, for example, and comparing model-based predictions to simple predictions such as a seasonal average. This helps to ensure that the new models can be shown to add value above and beyond current practices.
Dengue is a mosquito-borne virus that infects over 400 million people worldwide each year and places “an immense public health and economic burden upon countries around the world, especially in tropical areas,” the researchers point out. They developed a real-time forecasting model for dengue hemorrhagic fever in the 77 provinces of Thailand, creating a practical and highly computational infrastructure that generated multi-step predictions of incidence every two weeks for 2014.
Reich says, “We set out to highlight the challenges of working with real data in real-time. The disease surveillance system in Thailand has been around for decades and is a treasure trove of data that can help us understand the dynamics of many diseases. By working closely with our collaborators in Thailand, we have developed a framework that can bring advanced modeling techniques to the inevitable messiness of real-time data.”
The researchers’ new “prediction pipeline,” which includes rigorously evaluated real-time predictions and a disease transmission model that takes into account spatial and temporal trends, should “enable existing prediction methods to reach their full potential in making an impact on public health decision-making and planning,” say Reich and colleagues at Bangkok’s Office of Disease Prevention, Thailand’s Ministry of Public Health, Johns Hopkins University and the University of Florida.
Ongoing results from this collaboration are being integrated into public health decision-making by the Ministry of Public Health in Thailand. Findings are presented to public health officials to inform decisions about public health resource allocation and where to implement targeted interventions, such as to prevent excessive mosquito breeding.
As Reich explains, the predictions presented in this work represent the team’s first attempt at predicting dengue in Thailand and show mixed performance across provinces. In three quarters of the provinces, their model makes better predictions than a seasonal baseline predicting two weeks into the future. In over half of the provinces, their model performs better at predicting disease incidence up to six weeks into the future.
The researchers also wanted to assess the degree to which delays in case reporting make long-range prediction a challenge, so they compared the performance of their real-time predictions with predictions made with fully reported data. “Basically, the fact that it takes a long time for some cases to be reported is one of the major challenges in this research. This is much different than influenza surveillance by the Centers for Disease Control and Prevention in the U.S. where after one to two weeks we know very precisely the current levels of flu. It sometimes takes us three to six months to have a good sense of what was going on in a Thai province at a particular time. This makes our job of predicting that much more difficult,” says Reich.
They learned “valuable lessons,” the researchers say, for implementing real-time predictions in the context of public health decision making. “Comparing our ‘fancy’ statistical models to conventional wisdom and making sure that they are adding value has been a key component of this work,” Reich says. “Everyone in Thailand knows that dengue tends to have strong seasonal patterns, just like influenza in the U.S. We wanted to very carefully assess the extent to which our models could add value above and beyond predicting just a seasonal average. We worked really hard to hold ourselves to a high standard for success, both highlighting the times when we were able to beat that baseline but also acknowledging when we didn’t.”
Overall, Reich and colleagues compared predictions from several different statistical models, identifying locations and times where their predictions were accurate.
“Broadly speaking, improving real-time predictions can enable more targeted, timely interventions and risk communication, both of which have a measurable impact on disease spread in epidemic and pandemic scenarios. It is vital that we continue to build knowledge about the best ways to make these forecasts and integrate them into public health decision-making,” they say.
UMass Amherst News Office
Banner image: Hardy research group