“Before COVID-19, we didn’t have a need to do analysis on real-time location data. The crisis proved there is an application for using real-time data to aid decision-making, and now that we have it, we’re seeing a lot of applications where we can use it.” Bernd Vindevogel, Chief Analytics Officer

Business Goals

True Digital Group, the digital arm of True Corporation PCL, Thailand’s leading telecommunication company, wants to get ahead of the problem by providing real-time monitoring that helps to prevent new viral outbreaks. They have built a system named Tracepulse (https://www.truedigital.com/true-analytics) which provides a heatmap of people gathering in an effort to prevent viral hotspots and assists the Thai government to prevent the spread of COVID-19 by:

  • Monitoring real-time statistics on the percentage changes by geofenced boundaries and jurisdictional boundaries
  • Alerting on population densities that exist based on set thresholds anywhere in the country
  • Determining what medical deliveries can be made to help Thai authorities predict what is needed and obviate the need for testing
  • Combining current data with historical data to enable test-and-trace at scale and in a timely fashion
  • Helping health authorities proactively plan their resource strategy according to population density

Technology Requirements

But, to achieve this True Digital Group needs to build a real-time visualization and alerting system that automatically determines where large gatherings of people are occurring via geographic density of mobile phone locations. Given the urgency of the pandemic, the solution needed to be built in a two-week time frame.

This requires powerful technology that can perform event stream processing on 500,000 anonymous mobile phone location events every second for 30+ million mobile phones in Thailand. This visualization must respond to ad hoc, interactive analysis by users. The solution must support a moving time window of undetermined duration to support queries against real-time and recent data for contact tracing. Additionally, two petabytes of reference and historical data must be available in some situations for further analysis.

Lastly, the real-time visualization must support geoanalytic queries with ultra-low latency on the order of milliseconds.

Why SingleStore? Fast and Real-Time

Prior to the pandemic, True Digital Group developed a prototype of the web app that could do real-time, interactive analysis on large data sets. Given the size of the dataset and the complexity of the queries, they could only achieve satisfactory query response times and interactivity by pre-aggregating results. This took many hours to perform.

For COVID-19 the data has to be low-latency, real-time data—with MemSQL supporting more than a trillion rows per second on raw data, it comfortably handles the ingestion of 500,000 events and being the application database for True Digital Group’s customer-facing PWA.

Business Outcomes

One week to develop the first operational version of True Digital Group’s Tracepulse

The Tracepulse application was designed and built with MemSQL showing that rapid development and quick, agile iterative DevOps cycles can be done with MemSQL. Since it was developed, the functionality has been iteratively developed and extended in order to demonstrate new possibilities for analysis to the Thai Government’s authorities.

In-the-moment view of mobile phone density

The SingleStore database serves highly concurrent queries invoked by an interactive geo-analytic dashboard, and this drillable dashboard provides an in-the-moment view of the density of mobile phones in any given area from the country-level down to the city block.

Power wall-sized dashboards in Thailand’s crisis operations control center

Streaming raw, anonymized location data from the core telco network into SingleStore via Google Cloud Storage (GCS), allows dashboard users to zoom in, real-time and see where large gatherings may be forming and helps to determine the optimal placement of resources, like medical supplies, to enable dynamic real location of human and physical resources to make disaster relief efforts more proactive.

Granular visibility of mobile phone density data

Users of Tracepulse can visually explore this geographical information and zoom in for a granular, detailed view of clusters of anonymized mobile phones to spot potential emerging problems. Each click progressively queries SingleStore to calculate the new statistics for each geographical region as users drill down.