A City, If You Can Keep It: We are surrounded by data, but starved for insights

File Photo/Winters Express

Support Local Journalism


Yolo County has a data problem that is making it hard to have confidence in their coronavirus efforts. Before I go further, some background info on me: I do predictive and prescriptive medical case data analytics for the largest Independent Physicians Association in Northern California. Before that, I was Active Duty with the National Guard as an emergency response planner and operations manager working hand in hand with all of the State Agencies that are involved in emergency response (primarily Cal OES, DGS, and CalFire). This ongoing COVID response is a strange situation that hits on elements of my current and former job.

There have been two recent significant revelations about how Yolo County is managing and reporting data related to COVID-19. The first was cases have been aggregated by report date (the date the County was notified about them) rather than test date (the date the test was administered). The second stems from the first: the County Director of Public Health reported that he has little confidence in our trailing case numbers due to lag in reporting. This is a predictable but manageable issue if you know what you are doing.

Both of these are Day One fundamental errors in medical data analysis that should have been recognized months ago. Lag in reporting is a fact of life when you’re running analysis on medical case or claims data. Ideally, we know about the relevant data points as soon as a procedure is done. In reality, it takes time for a test to be processed and more time for the results to be communicated. This is called claims lag in the insurance industry. Accounting for this lag is routine when conducting analysis. This claims lag means you will not know the true case total on a given day until the lag is statistically complete.

The date field by which data is aggregated determines what is being measured over short time periods. If we want to measure the daily new positive COVID cases, but we aggregate by how many cases we were informed about per day rather than test date, we are measuring the efficiency of our test processing and reporting network more than the rate the virus is spreading. Over a long period of time, we get the same cumulative result, but our time series graph has greater variance day to day, week to week, month to month. We’ll have higher highs and lower lows and an ill-defined trend curve. Best case, our results are simply shifted by the days it takes to get results back. Worst case, we are reacting to spikes in reported results thinking they’re daily spikes in new infections.

Fortunately, the aggregation date mistake was corrected as of the other week. But for Yolo County to be making these kinds of mistakes in month four does not give me a warm fuzzy feeling about how they are handling our collective situation. It also gives me little faith that they are doing any meaningful data analysis. Which in turn means the decisions being made that are having tangible effects on lives and businesses may be based on bad data or faulty analysis.

I requested the County publish an open dataset, but so far they are unwilling to do so because the GIS vendor that runs the dashboard would have to “develop the appropriate functionality and … run it daily” which would be an additional cost to the County for a “functionality” they do not think they need. I do not believe running a table export function from the database counts as developing functionality, but that is the answer I received. The response phrased as it was, furthers my belief that the type of expertise and technical know-how to manage and analyze the COVID data is not something organic within the County staff and our elected officials.

A bright spot, however, is that they are reportedly working with the UC Davis School of Public Health, Health Informatics class for analysis. But, as of publication, we are day 146ish of 14 days to flatten the curve and that partnership seems to be just getting going.

I would still like to see the County publish their daily data roll ups for download. Currently, we the people have no way to verify the County’s math for each geopolitical sublocation in the County. We can cross check total county numbers with the State’s open dataset, but we still have to take their word that they are now aggregating on the correct dates when submitting results to CHHS and CDPH. Downloadable data would also open up the possibility that one or more of the many people in the health industry residing in Yolo County may discover a useful insight.

Questions I would personally like to examine are: Do we have any direct or lagging correlations between cities or is each city essentially an island in Yolo County? Is rate of infection, hospitalization, and death by age group uniform between all cities or is that specific risk different depending on where you are in Yolo? What does each city look like when you remove the cases from the Long Term Care Facilities (LTCF)? Are Cases in LTCFs leading, lagging, or unrelated indicators to their city? The dashboard looks good but it is ill suited for this type of analysis.

There may be no useful information gleaned from the answers to those questions, but not treating each city, each age group, each industry like one monolithic risk category would be a huge step in the right direction. The County, when armed with granular analysis, will be better positioned to push back on blanket State restrictions that may not apply to our specific situation. And the Board of Supervisors could actually represent our County rather than acting as a radio repeater for State decisions.

Early in my Army career I was told the following: a leader does not pass the blame for unpopular orders received from higher up, he (it was an all male unit) issues the orders as if they are his own. It would be nice to feel like our elected County officials are making sound decisions of their own based on our data rather than simply echoing State orders.

Leave a Reply

Your email address will not be published.

Previous Article

A Winters Tale: Lights, Camera, Winters

Next Article

Winters Public Safety Report: Aug.12, 2020

Related Posts