Triangulating Evidence on Outbreaks in Kid Settings

The school reopen debate continues. A number of large school districts have made decisions — mostly to move to a fully remote schedule — but some are still deciding (like New York) and smaller districts, as well as many private schools, have yet to make decisions. I have maintained since the start of this, back in March and April, that summer camps and others child care settings are a source of information on what to expect school reopens. In the service of this, I (with some COVID-Explained team help) have been trying to aggregate information. Today, I want to talk through what I think we are seeing, triangulating from several sources. Before getting into this, I want to be clear on what we are looking for in the data. From a decision-making standpoint, the central question is whether these settings are sources of outbreaks. That is: if we open schools will they contribute to spread, make it worse, raise the “R0” value we hear so much about? This is different from the question of whether people in schools will have COVID-19. As I’ve written about recently elsewhere, when we open schools we should expect some cases. The rates in the school will mirror the community even if schools do not contribute to spread at all. This is a reason, in my view, to limit opening to places where the virus levels are lower (no, I do not have an exact cutoff, but here’s a link to some people who do). But it’s also a reason to focus in the data on looking for outbreaks, not single cases. When we go to look for these outbreaks, we can start by asking about the ideal evidence. What would be ideal, from a data standpoint, is if every child care setting — camp, child care center, etc — were required to report their enrollment and any COVID-19 cases regularly (say, once a week). Perhaps they could also indicate any precautions they are taking. This could be coordinated by states or, better yet, the CDC. It would give us a complete picture of what is going on. But there is nothing like this. So today I’m going to talk through three imperfect pieces of information and see how we can triangulate. Specifically, we’ll talk about: Information I’ve crowd-sourced from camps and child care settings Media reports on outbreaks State-reported data I’ll sum up at the end, at least what I take from this. No TL;DR here since I think we should all draw our own conclusions. Crowd-Sourced Information A couple of months ago, fueled by coffee and frustration, I started an effort to collect information from open child care settings on outbreaks. A piece of this effort has involved tracking locations over time, looking at their enrollment and cases by week. (Big thanks to — among others — people at YMCA, the American Camp Association and Winnie.com for their help on recruitment). Effectively, I’m attempting to generate a voluntary, crowd-sourced sample which would do what I describe as the “ideal” above. However, such an approach is far from the ideal. The problem is not so much that it is not universal — that’s an inevitable part of sampling — but that it’s not random. Places with COVID-19 outbreaks may be less likely to participate. More generally, the places that want to engage on an effort like this are likely those who are taking other precautions about COVID-19. For all of these reasons, the numbers generated by this approach are likely to understate what we’d see in a random sample. You may ask (people have) what is the point of seeing these data given those issues. I can see a few reasons. The first is to match anecdote with anecdote. The media coverage (more on this below) can give one the impression that it is impossible to operate a child care setting without outbreaks. These data contradict this. Second, by tracking the same place over time, we can see how common it is for a single case to turn into multiple cases. We can see whether the cases we observe turn into outbreaks. And, finally, this is an opportunity to see what methods these settings are using as prevention. What are we seeing? First, if you want to access all the (de-identified) raw data, you can see it here. Enjoy! For a quick topline, the case numbers are limited. Among over 11,500 students and 3,800 staff in the most recent week, there were 2 student and 1 staff case. The camps — at least some of which are large overnight camps — showed similarly low numbers. In the Google sheet, we can separate out places in hotspots and the rates are still low. Again, I am not saying these are the average rates for all organizations. What we do see is that it is possible to run (for example) a large overnight camp in Texas without COVID-19. What we can also see is that there are places which have 1 or 2 COVID-19 cases in a week and then do not have cases the following week. Effectively, we see containment. In the most recent week I asked locations what they are doing to prevent COVID-19. I asked about just a few specific things: are you doing temperature checks, symptom tracking or routine testing. Nearly all of the locations in these data — 95% — are doing at least one of these. 90% of them are doing routine temperature checks (note that many states require this) and 73% are doing symptom tracking (again, often required). More notable, about 15% of the sample is doing some routine testing of either staff or students. These high numbers reflect good compliance, and may be part of what has kept cases down. Media Outbreak Reports In a sense, the polar opposite of the crowd-source data is media reports on outbreaks. I say this because these focus only on settings with cases (stories of camps with no cases are not interesting). If the crowd-source data selects for places with few cases, media reports on outbreaks selects for places with lots. Still, this provides a different kind of evidence. It tells us something about what can happen in the other direction, the bad direction. It’s difficult to be comprehensive about outbreaks reported in the media. We sourced from Twitter, from online searches, from various threads. We did our best! (See a way to weigh in below if you have things to add). To make it easy to see, we put it in a table, with some key information added. (The table with sources is here). An important note here is that we found additional media stories which report on a camp or child care closing due to one case among staff, or reporting the presence (or suspected presence) of a case. We did not include these in the table for the simple reason that this isn’t an outbreak. See longer discussion above. We focused this on cases where there were more than two cases noted. Some of the outbreaks listed here are very small. Three or four cases in a setting, with limited details provided. There are, however, several notable, larger outbreaks. Some of these seem inevitable. The first location listed — a camp at Allaso Ranch, in Texas — posted Instagram photos showing large, indoor gatherings with singing and no masks. Given the overall COVID-19 levels in Texas and what we know about indoor unmasked singing events, this is not surprising. But there are other examples — for example, the Kanatuck camp or the Lake Grove KinderCare — where the outbreak seems less explicable. In the last case, there were a large number of cases in both staff and children and (based on some digging we did) it doesn’t seem like the location did anything objectively “wrong”. It would be helpful if we knew more about these situations, although I think it’s understandable that privacy concerns limit how much locations want to reveal. State-Level Data The last thing I’ve been looking at is the data that states have been putting out there. We’re doing this over at COVID-Explained, and you can see the whole messy disaster here. These data mostly focus on child care facilities, not camps, but it’s a bit of a hodge-podge. A challenge with the data provided from many states it is that it provides no ability to look at outbreaks as distinct from cases. For example, California reports out cases linked to child care centers, but this includes all counts of children, staff and parents who are affected. They do not provide evidence on clustering of cases. You’d expect some base rate of cases in these settings, and without knowing if the cases are clustered, there’s not sufficient information. Texas data has a similar problem, as I discussed in an earlier newsletter (let it not be said that I do not repeat myself). There are a few states which specifically focus on outbreak data. Colorado reports all clusters of two or more cases within 14 days as outbreaks; their data has 7 child care facility outbreaks since the start of the pandemic, mostly with very small case numbers. North Carolina reports ongoing clusters here and their most recent update includes 14 clusters, with case counts ranging from 5 to 14. Oregon does a similar type of reporting and has two clusters identified, one of which is that larger Lake Grove KinderCare case we talked about above. The random nature of this reporting, the fact that much of it must be dug out of newspaper articles, is not great. But more data is better, even if we cringe a bit at the quality. Pulling it Together Ultimately, the idea is to pull these data together and triangulate — at least a bit —what we might see in schools. I think there is both bad news and good news here. On the bad news side, the fact that we see some outbreaks here tells me that it is not realistic to expect no school outbreaks at all. In some cases, the large outbreaks cited above seem clearly linked to behaviors we’d hope schools would avoid, but it is simply not realistic to expect no clusters of cases to emerge. Some of these could be large. But on the positive side: many child care centers and camps, even large ones and even in high prevalence areas, are operating without significant outbreaks. They are dealing with cases without having them turn into clusters. We see this in the crowd-sourced data. We can also infer it, to some extent, from the media reports and state-level data. The number of clusters observed are simply very limited relative to the overall population. It’s easy to forget the denominator, but estimates suggest in the range of 5000 summer camps are operating this summer, including perhaps 1500 overnight camps. The number of child care centers in the US is in the hundreds of thousands. Yes, it is concerning that there are 14 child care locations in North Carolina with clusters, but this is out of a total of about 6500 locations. Again, this doesn’t mean that there aren’t individual cases at other centers — we know there are — but the cluster rate simply seems limited. I will stop short of drawing any overall conclusions on schools here, since I think people may read this differently. Some will say it is not responsible to open schools if there is going to even one cluster. If we take that approach, this is not good news, and I think it would realistically say we need to stay closed. A different approach would be to say that these data suggest the risk of a cluster is fairly small, and focus on opening with a key set of procedures in place for limiting spread when individual cases do crop up. And everyone, cautious or not, should leave this conversation with the sinking feeling that our data isn’t good enough, and that it would be great if I were not the one collecting it.

This Post Is for Paid Readers Only