PregnantData: a weekly guide for a more informed and empowered pregnancy. Now updated for 2025! Sign up for a free trial.

Emily Oster

11 minute read Emily Oster

Emily Oster

Researching the Importance of Paid Leave

A look into how studies are conducted

Emily Oster

11 minute read

The United States is one of the only countries in the world that doesn’t guarantee paid parental leave. We point out this fact a lot, but it’s useful to think about what that means. What does it mean for a family when parents don’t have paid leave and don’t have the ability to take time off when a baby is born?

It means a lot of things. It means moms going back to work while still recovering from childbirth, it means parents struggling to figure out child care for their baby, and it often means babies going to group child care settings, which may be wonderful but do expose them to germs —  germs that are more dangerous when babies are small than when they’re bigger. There are a lot of reasons that we talk about the importance of paid leave. We can talk about these reasons and we can see why they might matter for kids’ outcomes, for families’ outcomes, but to figure out how much they matter and in what ways, that’s what research is for.

And this past summer, Dr. Katherine Ahrens and Dr. Jennifer Hutcheon, who are both epidemiologists and professors, wrote a paper titled “Paid Family Leave and Prevention of Acute Respiratory Infections in Young Infants.” The paper is an analysis of paid leave in New York State, and the authors are looking at the impacts of that paid leave on hospitalizations for infants, mostly for RSV. 

Bottom line from the paper: paid family leave keeps babies healthier and keeps them out of the hospital, and now we have the data to prove it and to show that the effects are large in terms of numbers. 

Putting together a research paper like this is surprisingly tricky. You need to know what questions you’re asking, and you need to think about how you’re going to determine causality rather than just correlation. So we’re going to take their research from idea to final peer-reviewed paper, and we’re going to talk about everything you always wanted to know about how research is conducted. 

Here are three highlights from the conversation:

Why is it important for research to have funding?

Jenn Hutcheon:

Research costs money. To get the data, to answer these questions, you need to buy it. You need to hire analyst time, you need to pay for faculty time to focus on the project. So it’s possible to do research without having funding. But certainly in health care, I think it’s almost necessary that you have some sort of grant to help pay for it. The grant we got for this was actually a fairly small grant that really just paid for our data and Kate’s salary and a little bit of help towards students, but that was really it. Most of the time, people would want a much bigger grant.

Emily Oster:

And so Kate, when you look out at the world of grants, how much of what you’re going to study is like “I want to study this and I’m looking for money to finance it,” versus “I’m looking for money in my general space and it’s going to push a little bit in one direction or the other”? I’m just curious about that balance.

Kate Ahrens:

Well, definitely money is important. I’m in an entirely “soft-funded” money position, so if I don’t have grant money or contract money, I won’t actually have a job. I can’t really dip below 100% funding. So it’s really important. But I’m not perseverating about funding too much. I’m kind of an opportunist researcher. So I work on topics I’m directly interested in, but I’m also happy to contribute my data analysis skills and writing skills to topics that are slightly tangential to what I’m interested in as a co-investigator.

Definitely NICHD, the National Institute of Child Health and Human Development, shares a lot of interest with what Jenn and I do because it’s focused on families and maternal health and infant health and child health. That’s a good institute for us to apply for funding from.

What is the difference between a natural experiment and a regular experiment?

Jenn Hutcheon:

A regular experiment would be where the researchers are manipulating what’s happening — so, deciding who’s going to get a treatment and who’s not going to get a treatment. In this case, that would be randomly assigning paid family leave to some people and not to others, which would be a very challenging thing to do. Instead what we did was looked for states that had recently introduced paid family leave, which we sort of took as what we would call a “natural experiment,” because it’s not anything we did, it’s just something that happened and what can we learn from it.

What is the path between writing a paper and having it published?

Kate Ahrens:

Well, first Jenn and I wrote the first draft and then we sent it to co-authors, who gave a lot of feedback, and we revised it based on all the co-author comments. And then we agreed upon the final draft as a team and then we submitted it to a journal: JAMA — the big JAMA, not JAMA Pediatrics but the big one. It got reviewed, but it got rejected.

Emily Oster:

That’s actually meaningful. So when you submit to a journal, you send it in and sometimes they just tell you to go away right away and sometimes they send it to reviewers.

Kate Ahrens:

Yup. So the “go away right away” is called a desk rejection. And this one, they actually sent it out to reviewers and we got comments back probably within a month. They’re pretty quick at JAMA. The reviewer comments were actually not so bad, I don’t think. They were mostly concerned. I think it was sent to health economists, and we’re mostly epidemiologists. We have actually two health economists as co-authors, but Jenn and I are more epidemiologists. And so I think they were interested in that we had a different method than is typically used in health economics research and a “controlled diff-in-diff” analysis or “event study diff-in-diff” analysis.

And so we kind of felt like these comments from these reviewers, though the paper was rejected, they were actually pretty addressable. We had thought a lot about our model and a lot about why not to use this typical method that’s used often by health economists. So we actually asked JAMA if we could respond to the reviewer comments and at the same time submit the paper to JAMA Pediatrics, which is another JAMA-family journal but it specializes in pediatrics. And they said okay. So we did that. We responded to the comments of the original reviewers at the same time we submitted to JAMA Pediatrics. That process went very smoothly. It wasn’t actually sent out for review again. They used the original reviewers. It was a very efficient process.

Full transcript

This transcript was automatically generated and may contain small errors.

Emily Oster:

Kate Ahrens and Jenn Hutcheon, I am delighted to welcome you both to ParentData. I would love if we could start by just having you introduce yourself before we get into paid leave and illness. Kate, maybe we can start with you.

Kate Ahrens:

Sure. Great to meet you. I am an epidemiologist and associate research professor at the University of Southern Maine, which is part of the University of Maine system.

Emily Oster:

And Jenn?

Jenn Hutcheon:

Thanks. Nice to meet you as well. My name is Jenn Hutcheon and I’m an associate professor in the Department of Obstetrics and Gynecology and a perinatal epidemiologist as well at the University of British Columbia, which is in Vancouver in Canada.

Emily Oster:

Our goal today is to talk about this very nice paper, which I highlighted for everybody at the top of the episode. So I’m not going to review the findings again. But the primary argument in the paper, the primary finding is that paid leave reduces childhood illness. And so I guess what I would love to start by asking you guys is about the very beginning here. How did you get interested in this topic? You can answer that question either how did you get interested in writing this paper or why do you care about this general area of the world? Either one.

Jenn Hutcheon:

I can go ahead and take that. We actually kind of migrated into this. Kate and I had originally done a lot of work looking at research in birth spacing and how we can try to prevent rapid repeat pregnancies. And then the NICHD put out a notice of special interest, sort of a call for research to try to prevent rapid repeat pregnancies that specifically was interested in how different policies or practices could help prevent rapid repeat pregnancies. They actually specifically listed paid family leave as one of the things that they were interested in evaluating. So we actually put in a proposal to look at paid family leave and prevention of rapid repeat pregnancy, which is kind of what got us into this topic.

That grant did not get funded. And as we were kind of rethinking what to do with it for resubmitting and things like that, one of the considerations that we had is that when you’re evaluating these policies, because not everybody in the population is affected by a policy. So some people will have already had leave or will already be taking leave, some people still won’t take leave, even if it’s available. You need to have a fairly strong signal in the group of people who are affected by the paid family leave policy to be able to detect it overall.

I think we were a little bit worried that because there’s a couple steps involved in prevention of rapid repeat pregnancy, it has to be that you are more likely to go to your postpartum visit and get contraception to be able to actually prevent a rapid repeat pregnancy that maybe we couldn’t detect signal overall. And then I happened to be working on another project that was looking at prevention of lower respiratory tract infection in infants and going through all the literature on the risk factors of which the main one that stands out is day care and older siblings. And so that kind of was the connection between saying, “Hey, if we want a strong direct effect, why aren’t we looking at the effective paid family leave on newborn respiratory tract infections? That’s kind of how we got to where we were.

Emily Oster:

Okay. I love this. I have many follow-up questions because I understood everything you said, but I suspect that for many people on this, they didn’t. Let’s step back to the first part of this, which is when you are thinking about what to research, the first thing you said was we saw a grant submission, we saw a way to have financing. That’s something I think people have a hard time understanding, how important financing is in what we study. Can you say a little bit just for someone who isn’t in this space, what does it mean when you say we saw a grant and that was important? Why would you care about there being money? Why don’t you just do whatever you want?

Jenn Hutcheon:

Because research costs money, so get the data-

Emily Oster:

Research costs money.

Jenn Hutcheon:

To do anything. To get the data, to answer these questions, you need to buy it. You need to hire analyst time, you need to pay for faculty time to focus on the project. So it’s possible to do research without having funding. But certainly in healthcare, I think it’s almost necessary that you have some sort of grant to help pay for it. The grant we got for this was actually a fairly small grant that really just paid for our data and Kate’s salary and a little bit of help towards students, but that was really it. Most of the time, people would want a much bigger grant.

Emily Oster:

And so Kate, when you look out at the world of grants, how much of what you’re going to study is like, I want to study this and I’m looking for money to finance it versus I’m looking for money in my general space and it’s going to push a little bit in one direction or the other. I’m just curious about that balance.

Kate Ahrens:

Well, definitely money is important. So I’m an entirely soft funded money position, so if I don’t have grant money or contract money, I won’t actually have a job. I can’t really dip below 100% funding. So it’s really important. But I’m not perseverating about funding too much. I’m kind of an opportunist, honestly, researcher. So I work on topics I’m directly interested in, but I’m also happy to contribute my data analysis skills and writing skills to topics that are slightly tangential to what I’m interested in as a co-investigator.

Definitely NICHD, National Institute of Child Health and Human Development shares a lot of interest with what Jenn and I do because it’s focused on families and maternal health and infant health and child health. That’s a good institute for us to apply for funding from.

Emily Oster:

Often people ask why does some things get funded, some things get studied and some things don’t. I think that a piece of that is that a lot of our funding comes from either companies, pharmaceutical companies or from the government. The government has a sort of set of things that they’ve said, “We’re interested in studying this.” And then that pushes research a little bit in various directions. And then there’s sort of pharma funding, which I think would not be appropriate for this kind of project because there’s nothing to patent. If you learn that paid leave is good for families, you can’t then sell that in a shot. We’re working on it, but it’s proved to be very challenging.

Okay. You sort of get into this space, but it is not initially the thing you’re most interested in. Can you just say a little bit about this idea of rapid repeat pregnancies and actually what that is and where you started, what were you trying to understand there?

Kate Ahrens:

Well, Jenn and I have worked a lot on this topic. We probably published maybe 10 or 20 papers on this topic. We actually led a workshop. When I worked for the Office of Population Affairs in 2017, we led a workshop where we had experts come in and talk about the methodological challenges of looking at this issue of rapid repeat pregnancy, which is also called Studying Interpregnancy Intervals. A lot of literature for decades has found that there are poor infant outcomes if an infant is born shortly after their sibling is born. And so there actually have been new types of studies that have come out recently doing different types of methods with these data that you can get at a population level. Some of those studies were challenging the findings of previous studies and there was inconsistency. So we were looking a lot on the methodology of how to study interpregnancy intervals on infant health and maternal health.

Emily Oster:

Why is that hard?

Kate Ahrens:

Well, it’s actually, the concern is confounding. So the type of people who have children close together have different characteristics than the type of people who have children farther apart. And so to control for that confounding has been challenging and there’s always been a sense of doubt that it’s not actually the spacing that matters and might be the characteristics that we haven’t properly controlled for. So they started doing these within family analysis. So within a mother looking at the spacing between her first and second baby and her second and third baby. But those also have challenges, analyzing those types of studies. So we are just doing a deep dive into these study designs.

Emily Oster:

And then how’d you come to paid leave then?

Kate Ahrens:

I can’t remember. I mean, part of this is that Jenn’s from Canada and I’m from the U.S. I’ve never had a job that has had paid family leave. In fact, my current job-

Emily Oster:

I’ve never had a job. We don’t work here. It’s not like Canada.

Kate Ahrens:

So Jenn always finds it ridiculous that we don’t have paid family leave. She thinks it’s sort of preposterous. I’ve never had one of all three kids, no paid family leave. Part of it was part of just our interest in this topic and why it’s so different between Canada and the U.S. It does seem like there are so many things we think about for maternal and child health, what can we do, especially in rural health and telehealth and stuff like that. But it does seem like one of the obvious things we might be able to do that has been shown to have many health benefits is to offer some type of paid family leave.

There are only 13 states enacted paid family leave with Maine being the 13th state. So it’s not the majority of states in the U.S. that have paid family leave, which is an outlier among many high income countries. I think we’re the only-

Emily Oster:

Literally all of them, all of the high income. And actually most of the low income countries too, we’re holding our own. So Jenn, you find our lack of paid family leave disturbing and you wanted to study that, is that basically… You came judgmentally from the north to show us we’re wrong?

Jenn Hutcheon:

I’m not going to say that. It’s definitely a difference. And my husband has been doing a lot of work in Boston right around the time we were thinking about this study and I was on mat leave for part of the time and had just come off mat leave. And so it was almost just kind of mind-boggling to be like, how do people make this work? How do their kids not get sick all the time if you put these very young kids into day care? I hadn’t really thought about it much, to be honest, until spending time in the U.S. when you realize just how challenging it is and all the carry on effects of what happens if parents go back to work right away, what happens to their kids? Do those kids get sick? All these kinds of things.

Emily Oster:

Yeah, I think what’s interesting then as a researcher is you sort of have this sense of like, and I think a lot of our ideas often come from our own lives or the experiences we’re having. So you’re like, “Oh, here it is, paid family leave. It seems like it’s important.” And then you’re connecting it very directly as I would too to, okay, well what would you do with your kid? Well, I remember when I send my kids to day care for the first time, they got really sick. Okay, well if that’s happening earlier, you think about infant health. Do you sort of come up with this basically a theory of how these things might relate?

And then there’s this challenge which is, and you talked about confounding, but just to be clear on the challenge, you couldn’t really just write a paper where you compare people who have leave to people who don’t, because those groups are really, really different. Maybe they’re in different countries, but even if you looked within the U.S., paid leave is more likely from an employer for people who are white collar who have more resources and other ways. So you wouldn’t really be able to do that. So your challenge there is to say, “Okay, I’m interested in this question of how paid leave might impact infant illness, but I can’t just compare people.” So what is your next step in trying to figure out how you can do a better job figuring out a causal relationship?

Jenn Hutcheon:

I think here our next step was to try to look for the natural experiment type situation that helps do a better job of controlling for confounding. So doing a better job of helping you isolate if there actually is an effect of the policy or if it’s just that people who have paid family leave also have better health outcomes for many other reasons, but that introducing paid family leave won’t actually make any difference there. And so we started looking around for cases where there could be this type of natural experiment. When we were originally-

Emily Oster:

What is the difference between a natural experiment and a regular experiment?

Jenn Hutcheon:

A regular experiment, I guess in our world, would be where the researchers are manipulating what’s happening. So our deciding who’s going to get a treatment and who’s not going to get a treatment. In this case, that would be randomly assigning paid family leave to some people and not to others, which would be a very challenging thing to do. I’m sure it’s possible with a lot of money, but-

Emily Oster:

You would need a much, much larger grant for that.

Jenn Hutcheon:

Yes, exactly. But instead what we did was looked for states that had recently introduced paid family leave, which we sort of took as this what we would call a natural experiment because it’s not anything we did, it’s just something that happened and what can we learn from it.

Emily Oster:

So you find a state, your state is New York, you say something about what happened. Kate, do you want to tell us what happened in New York? What is the policy change?

Kate Ahrens:

Yes. New York is a large state, which is also why it’s a nice state to look at. Has maybe 250,000 births per year, I think. It was the fourth state in the U.S. to implement paid family leave. It implemented it in January 2018, and it started out with eight weeks of paid family leave. You could take it for various medical circumstances, but one of them was if you had a newborn baby. If you gave birth to a baby, you could take eight weeks of paid family leave within the first 12 months of the baby being born. That was a generous policy. It had generous eligibility and the benefits were good. The percentage of your wage that was replaced was good. We had seen reports about New York’s paid family leave where roughly a third of newborns had parents who took family leave benefits that first year was implemented. So there was really high uptake, which is something you want to look at when you’re evaluating a policy. It’s much easier if there’s robust uptake of the policy.

Emily Oster:

Yeah. Because your idea is you’re going to compare before and after the policy, not almost independent of whether people take it up. You’re just going to look before and after. And if nine people take it, then of course you’re not going to see an effect. But that’s just because nobody took it. So this issue of uptake is really important because these policies don’t have, they definitely don’t have 100% uptake. They’re hard. People don’t know about them. Other problems occur.

Kate Ahrens:

Yes, yes. So that was great about New York. I think there was a paper that compared New York to the other states at the time, and it had the most robust uptake among the states that had paid family leave at the time.

Emily Oster:

Okay. So you have your policy, it changes over time. There’s a natural experiment. So you’re confident, you’re more confident about your causality and your confounding, but now your problem is that you need to have data and because what you want to study is illness, you actually need some data on whether kids get sick. And so if you were in Europe, it would be great because in Europe they know everything about everybody beginning at birth and going until they’re dead, including all of the things that they had for lunch on Tuesday and all kinds of other great stuff. But in the U.S., we don’t really have that. So where do you go looking for data of that type?

Kate Ahrens:

Jenn and I looked for data that was population-based, so it covered the whole state of New York. There are these administrative databases that include all hospital discharges for the state of New York and all emergency department visits data. And so these are emergency department visits that didn’t result in hospitalization, so they’re mutually exclusive. So you have these data sets for most of the states in the United States.

Emily Oster:

Who collects this data?

Kate Ahrens:

The state agencies, different state agencies collect the data. In New York, it’s the Department of Health and Human Services in New York, I think. And they’re differently collected. Sometimes they’re sent on in a package form to HCUP, which I’m forgetting, it’s AHRQ, I’m forgetting who collects the data. But they collect the data.

Emily Oster:

There are so many acronyms, you very rapidly dive in the acronym hole. But yes, some other locations.

Kate Ahrens:

Some other entity in the government collects the data and harmonizes it and you can actually purchase it. But we couldn’t use those packaged data sets because we wanted two pieces of information that aren’t always available in those packaged data sets. We wanted the infant’s age in days or weeks or at least months so we could get the zero and one-month-old because we wanted under eight weeks of age. That wasn’t available in all these packaged data sets. And then we also needed zip code of residence for a secondary analysis that we’re doing right now, and those aren’t always available.

So we had to actually go to each state and apply to use the data directly from each state’s Department of Health. We had to go write an application and go before their data governance board and ask for the data and say our hypotheses and methods and everything. So we did that for New York, Maine, Vermont, New Hampshire, and Massachusetts, New York. And then the control states did not have any paid family leave at the time of January 2018.

Emily Oster:

Yeah, so this a very important part of a study like this. It’s not enough to just get the data from New York, which would let you look before and after. You actually want some comparison places. So you can rule out the idea that there was just a change over time by comparing to other comparable states. How’d you choose your comparable states.

Jenn Hutcheon:

Well, here, I mean our choice of control states was actually influenced to some degree by data availability. So we were looking at hospital visits, which all states collect, but not all states collect information on emergency department visits. We had hoped to include Pennsylvania, but they don’t include information on emergency department visits. So we couldn’t really use them as a state. I think we had also looked into using Maryland, but they don’t collect information on, I think it was age in weeks or days. They only had it in years. So you could only know if someone was a year old or younger, but we couldn’t identify the infants that were less than two months. And those are the ones who would benefit from paid family leave.

So the control states that we picked was sort of this balancing act of do they have data, do they have the data we want, and are they a reasonable control state and can we afford them? Because there were some states I think that were very, very expensive to get the data that we couldn’t.

Emily Oster:

I think when people read, certainly if you read media coverage or a paper like this, but even if you read the paper, so much of this stuff is hidden. The paper is just like we have these five control states and you list them, you’re like, “Okay, sure.” But then to dive into, well, every one of those was a decision about, does this match what I need? How much does it cost? Can I show up in front of these people and convince them that my project is good? And then you get the data and then you have to do… It’s not like they just deliver a result for you. Like you get the data, somebody has to clean it up and make it nice before you do anything.

It’s so frustrating because it’s so… Maybe not frustrating, but it’s such a huge part of the work of a project like this. And then it gets three lines in the method section. We use these control states, you’re like, “But let me explain to you about that meeting I had with the people in Massachusetts and I can’t believe what they asked me to do.” It’s like, no, nobody gets to hear about that problem.

Jenn Hutcheon:

No. And the one line that I’d hoped we could add into our methods was getting our data from New York State because we had applied to New York State first to try to get some pilot data to see if we could, is there even a signal there that makes this worth pursuing in a grant? So we had applied to them first, and to get the data, you needed an ink signature on the approvals for the data. Our data request made it to them in March 2020 when COVID was at its peak and New York Department of Public Health had many, many, many more important things going on. But this person who worked at the Department of Public Health made a trip from her quarantined home to the house of whoever it was who needed to make this ink signature so that we could get our data in a timely manner. That blew me away that they did that. So I was very impressed with that and I wish that could have gone into the methods section.

Emily Oster:

I totally agree. That definitely deserves at least an appendix of some sort.

More ParentData, including what it feels like to actually be reviewed by your peers, what do you do as a researcher when your research doesn’t match your hypothesis and when it does, and where this research might take public policy, after the break.

Emily Oster:

Okay, so you get all of your data and then you run your analyses. I’m curious if there is a moment when you have this theory, the regressions you run are in some ways relatively simple. I mean, once you have all the data and it’s cleaned up and it’s organized and you have your policy thing, it’s kind of like a couple lines of code to get the answer. Is there that moment when you first see that the answer is a thing? Was that fun?

Jenn Hutcheon:

I would say it was more a big exhale rather than fun. I think one of the challenges with doing research in an area where you care quite a bit about what the answer is, is that it’s very challenging if the answer doesn’t support your theory and you’re sort of torn between. Of course, you want to do what’s most rigorous from a scientific perspective, but when your emotions get in and it’s not the answer you want, it’s tempting to say, “Well, maybe we should just check to make sure we modeled seasonality right. Maybe we should just do some other checks.”

I found that a very stressful part about working on this project. A lot of the other work I’m working on right now is things related to drug safety, where if you find a signal of concern, that’s worth knowing and reporting on. But if you find something safe, that’s also important and worth reporting on, whereas something like paid family leave is a bit harder because of the feelings invested in it. So I think when we got our answer, I exhaled. And when we did every single sensitivity analysis, I exhaled a little bit more. And then when the last of our sensitivity analysis that we did was the one where we looked to see if we saw a similar effect in one-year-olds who were infants that we hypothesized would not be affected by paid family leave because their parents weren’t eligible. And we saw that there was basically no change at the same time in those infants. And so I think that’s when I finally truly exhaled.

Emily Oster:

Kate, what about you? Are you an exhaler or are you a celebrator?

Kate Ahrens:

Well, I think the best thing about research actually is uncovering things and finding things and finding things that are consistent with your hypothesis or inconsistent. Both of those are very interesting about the job of a researcher. I think we did it together. In this project, we didn’t hire a data analyst. I am the data analyst. Once the data were in aggregated format and harmonized, Jenn and I did coding together on aggregated data. And so we looked at the results in Stata together when it popped up. And so that was kind of exciting. We didn’t do it on our own. We waited for a regular meeting to do it together.

And then also with this, to point out, this is something that you have maybe some opinion about how the data should go. So we had an elaborate analytic plan that we ran by our co-authors and we had a meeting before because they were all on the same page because I find that’s the best way to do things where you don’t get down into a, “Well, what if we look at this way? What if we look at it that way?” We had a good pre-analysis plan.

Emily Oster:

Yeah, and always people ask, “What is the best thing about being an academic?” And I’m like, “It’s that moment when you know something that no one else does and it’s like all of this work has gone. It’s just like, that’s so magical and very difficult to, I don’t know, it’s difficult to describe.” There are other less good parts, which I want to get in now.

Okay, so then you write the paper, you have your results, you have your beautiful sensitivity analysis. What is the path between writing a paper like this and having it published? I’m not sure if people have any sense of what happens. What do you do with your paper?

Kate Ahrens:

Well first, Jenn and I wrote the first draft and then we send it to co-authors who are excellent and gave a lot of feedback and we revised it based on all the co-author comments. And then we agreed upon the final draft as a team and then we submitted it to a journal. Are we allowed to say Jenn, the journal? So we submitted to JAMA, the big JAMA, not JAMA Pediatrics, but the big one. It got reviewed, but it got rejected.

Emily Oster:

That’s actually meaningful. So when you submit to a journal, you send it in and sometimes they just tell you to go away right away and sometimes they send it to reviewers.

Kate Ahrens:

Yup. So the go away right away is called a desk rejection. And this one, they actually sent it out to reviewers and we got comments back probably within a month. They’re pretty quick at JAMA. The reviewer comments were actually not so bad, I don’t think. They were mostly concerned. I think it was sent to health economists and we’re mostly epidemiologists. We have actually two health economists as co-authors, but Jenn and I are more epidemiologists. And so I think they were interested in that we had a different method than is typically used in health economics research and a controlled diff-in-diff analysis or event study diff-in-diff analysis.

And so we kind of felt like these comments from these reviewers, though the paper was rejected, they were actually pretty addressable. We had thought a lot about our model and a lot about why not to use this typical method that’s used often by health economists. So we actually asked JAMA if we could respond to the reviewer comments and at the same time submit the paper to JAMA Pediatrics, which is another JAMA Family Journal, but it specializes in pediatrics. And they said, “Okay.” So we did that. We responded to the comments of the original reviewers at the same time we submitted to JAMA Pediatrics. That process went very smoothly. It wasn’t actually sent out for review again. They used the original reviewers. It was a very efficient process.

Emily Oster:

One of the reasons I wanted to talk to you guys about this paper as opposed to one of the many other papers I’ve talked about in the newsletter is I think your paper is really good. Sometimes I’ll write about things where I think that the reviewers should have made the paper better. Do you think it is the job? It sounds like you thought that your reviewers at least maybe they didn’t make the paper better, but maybe they were not problematic. Do you generally think this process works at both sorting out what’s good research and improving the research, or is this just a weird waste of everyone’s time?

Kate Ahrens:

I’m curious what Jenn thinks, but I actually do think that most of the time, it does improve your paper. There are some times where you send a paper to a journal and you get two or three sentence review and that’s just so not helpful because someone’s clearly busy and it’s fine, they don’t really think it’s that interesting. They don’t really provide many comments. But I think for the most part, either between the editors’ comments or at least one of the reviewers’ comments, it does really point out things that could be wrong or confusing or inconsistent with other papers or the literature. I find it’s a very helpful process for me, but I’m also a reviewer all the time. I’m always reviewing a paper, so I’m also part of this process, so I have a stake in the game. What do you think, Jenn? How do you feel about peer review?

Jenn Hutcheon:

I think it’s really hit-and-miss. I mean, I think there’s many, many papers that get published that you just read and are like, “How did this make it through the peer review process? There are so many problems with this paper. Where did things go wrong?” And that happens often, I would say. On the other hand, I think the peer review process probably does a reasonable job of making sure that studies that have major, major flaws in them don’t make it into a top journal. They’re still going to get published somewhere because that’s the nature of publication that if you pay money, you can get it published anywhere. But I think the top half is a big mix of ones that should be there and ones that definitely shouldn’t be there.

There’s definitely a lot of room for improvement in peer review. It’s hard to think of a time where I’ve gotten comments from reviewers where my thought was, “Oh, I had never thought of that before. That’s such a great point.” Most of the time, it’s comments that are worth discussing a little bit more. But because you have a limit on the number of words you can put in your manuscript, you couldn’t put it in the first time. And so maybe you get to discuss something in a little bit more detail later. But it’s hard to think of times where I’ve really felt like they made a major improvement to my work.

Emily Oster:

Yeah. I mean, there’s an interesting philosophical question about how we… There’s a lot of different fields of… Academic fields do this differently. In economics, the review process is actually way more involved. For a top outlet, it would be five or could be five people writing 15, 5 pages each of detailed comments. It takes a year to revise. It’s very, very long. And then you have fields like physics where the kind of review process is you upload things to the internet and other people look at them. And then if enough people look at your… It’s basically some kind of Reddit voting thing where if enough people upvote your thing, you get to put it. It’s not quite like that, but it’s basically a totally crowdsourced structure in physics or math.

It’s suggesting me, the process can’t be perfect because every field is doing it differently. So if we had a really great process, we’d probably all have the same process. And it must be that it’s kind of in some way problematic. But your paper got published and then when it gets published, then what happens? What do you do? If paper comes out, does anything happen?

Kate Ahrens:

Well, this is I think where I do not do well. Once a paper is accepted, then it goes through this whole copy editing process and stuff. And then it finally gets published maybe a couple of months later. I think I have colleagues who really promote their work on LinkedIn or ResearchGate or I guess Twitter or X or just do a lot of stuff.

Emily Oster:

Podcast. Do they come on podcasts?

Kate Ahrens:

They come on podcasts or their university issues or press release, which actually my university did. Or they kind of have these… Oh, we had a one-pager, Jenn, for our previous paper that we issued that was kind of in layman’s language or something. But I kind of feel like at that stage that I’ve burst this thing and it’s out there to be consumed. And I don’t do a lot of work promoting my papers at all. So this is very unusual that I’m doing this podcast. But yeah, Jenn, what do you think? I feel like I get the impression I should be doing more, but I don’t.

Jenn Hutcheon:

No, my feelings exactly. I’m an introvert by nature. And so going out there and sort of promoting my research, promoting findings is not something that comes naturally to me. I appreciate the importance of making sure either research doesn’t just sit on a shelf and collect dust, but I think in general, the people who are drawn to doing science might not necessarily also be people who are drawn to public speaking and kind of engaging with the public and about their findings and that kind of thing. So I think I agree I should also do more to make people hear about these findings, I guess.

Emily Oster:

I think it’s a pretty different skill, but it does also strike me that the relationship between the quality of the work and how much attention it gets in the media is not, I would’ve said that correlation was about zero, maybe negative. And so it almost feels like there should be another layer. It’s not really your job to… I’m not sure that it should be the job of the researchers to push their research on the public, but it would be good, especially in a case like this, I think people should know about this. I mean, this is yet another reason we should have more paid leave. It really matters for people’s lives. I guess I feel like somebody should be helping you do that. But as Kate eloquently says, once you’ve given birth, why do you also have to be a tiger mom? Can’t someone else take over now? I did all the hard work.

Kate Ahrens:

And you have so many other projects in the pipeline to work on. So it’s kind of like I don’t have a lot of time to devote to promoting previously published papers, but yeah.

Emily Oster:

Yeah. And the reward, I guess the other thing to say is the reward inside academia for doing that is quite limited. From the standpoint of the things that your job cares about, they will care that you have a job at pediatrics paper for sure. But if that paper is covered in New York Times or is covered… That matters less from a promotion prospect. It’s not a core part of how we’re evaluated as academics.

Kate Ahrens:

When we do reports to NIH on our grant every year, they are RPPR, I forget what it stands for. Some kind of report that you do each year. I think they do like to actually hear about, in addition to academic publications, they like to hear about presentations or podcasts or some kind of dissemination on YouTube or something. They like to hear about alternative ways of disseminating or study findings so it’s not just academic publications.

Emily Oster:

All right, so TikTok, you guys should be on TikTok-

Kate Ahrens:

Exactly.

Emily Oster:

…is what I’m saying.

Kate Ahrens:

That would be perfect for Jenn.

Emily Oster:

Yeah. Jenn, I think Jenn, you would be perfect on TikTok. A little dance, some [inaudible 00:35:52]. It seems like we could get something good here. All right, before I let you guys go, tell me what’s next. Is there a new project that we should be covering on this in a few months when it comes out?

Jenn Hutcheon:

We are working right now at looking paid family leave and to see how it affects different disadvantaged groups in the population. So looking to see if there’s different effects of the policy in different-

Kate Ahrens:

Yeah, so looking at disparities. So one of the biggest concerns about paid family leave or any kind of policy effect is it can have unintended consequences. We actually wrote a piece about this for another paper that came out in JAMA Pediatrics recently. It can exacerbate or reduce disparities. What we’re doing now is we’re using the same data, but we’re looking to see if the effect of the policy was the same across different race groups, across Medicaid versus commercial payer. And then by childhood opportunity index, which is kind of like a socioeconomic index that’s been developed for kids to see if there was the equal effect of the policy across these groups. So that’s the paper right now.

Emily Oster:

That’s great. Thank you guys. This was absolutely a delight. I really appreciate you being here and promoting your work on this podcast, which I hope will go in your RPPR report for NIH.

Kate Ahrens:

Yeah, I definitely will.

Jenn Hutcheon:

Thank you for having us.

Kate Ahrens:

Thank you.

Emily Oster:

ParentData is produced by Tamar Avishai with support from the ParentData team and PRX. If you have thoughts on this episode, please join the conversation on my Instagram, @ProfEmilyOster. If you want to support the show, become a subscriber to the ParentData Newsletter at ParentData.org, where I write weekly posts on everything to do with parents and data to help you make better, more informed parenting decisions.

For example, earlier this fall, I wrote an article that went deeply into Jen and Kate’s paper’s findings, and explain where this paper fits in the world of public policy, which I think we can all agree is in desperate need of data like this. Read it at parentdata.org.

There are a lot of ways you can help people find out about us. Leave a rating or a review on Apple Podcasts. Text your friend about something you learned from this episode. Debate your mother-in-Law about the merits of something parents do now that is totally different from what she did. Post a story to your Instagram, debunking a panic headline of your own. Just remember to mention the podcast, too. Right, Penelope?

Penelope:

Right, mom.

Emily Oster:

We’ll see you next time.

Community Guidelines
0 Comments
Inline Feedbacks
View all comments