Building the Real-Time Learning Network: Creating a leading medical resource for the COVID-19 pandemic

Mati Hlatshwayo Davis: [00:00:13] Hello everyone, and welcome to this edition of the Let's Talk ID podcast. My name is Dr. Mati Hlatshwayo Davis. I'm the director of health for the City of Saint Louis. I'm also an associate editor for the Real Time Learning Network for IDSA. So I know you guys know think that I say this every time that it's going to be a favorite. But this one is kind of personal because this is a team that I've worked with for the last three years or more? Time has gotten away from me. But we have been a part of something truly magical, which is the establishment and the leadership over a really exciting project through a grant with the CDC called the Covid 19 Real Time Learning Network. That grant is coming to an end, sadly, in a few weeks. And so this is our last hurrah. And I wanted to introduce you to these fearless leaders, this amazing team, and to just talk to you about what we built. I think it's such an amazing testament to work that was done during Covid in general, but this project specifically that you all have benefited from. So let's jump right in. And first of all, let me introduce these amazing people. First is Doctor William, or Bill Warble. He is the assistant professor of medicine for the Division of Infectious Disease, Section of Transplant and Oncology Infectious Diseases at Johns Hopkins University School of Medicine, and the associate director of epidemiology and Quantitative sciences, Johns Hopkins Transplant Research Center. Welcome, Bill.

William Werbel: [00:01:58] Hello. Thanks for having me.

Mati Hlatshwayo Davis: [00:01:59] Next is Dr. Maria Sundaram. She is the associate research scientist for the center for Clinical Epidemiology and Population Health at Marshfield Clinic Research Institute. Welcome, Maria.

Maria Sundaram: [00:02:15] Thanks so much.

Mati Hlatshwayo Davis: [00:02:16] Next is Dr. Ethel Weld. She is the assistant professor of medicine at Johns Hopkins in the Division of Infectious Diseases and Clinical Pharmacology. Welcome, Ethel.

Ethel Weld: [00:02:26] Thanks, Mati. Great to be here.

Mati Hlatshwayo Davis: [00:02:28] Next is Dr. Payal Patel. She is the system wide director of antimicrobial stewardship and associate professor of infectious diseases at Intermountain Health. Welcome, Payal.

Payal Patel: [00:02:40] Hey great to be here.

Mati Hlatshwayo Davis: [00:02:41] Last but certainly not least is Gayle Levy. She is the senior director for digital and content strategy at IDSA. Welcome, Gayle.

Gayle Levy: [00:02:50] Hi, Mati. Thanks for having me.

Mati Hlatshwayo Davis: [00:02:52] It's a big crew, but rock with us. We got this, and I just want to kick it off with you, Gayle. Could you talk to us a little bit about how we got started, what was the impetus and the background for the creation of the Real Time Learning Network?

Gayle Levy: [00:03:09] So first, I'd like to acknowledge CDC for their support for this important resource over the last four and a half years. And thank all of you and our managing editors who are not on the call today. Our partners and our staff, everyone's hard work for their dedication over these last four years. The beginning especially, we're working really long hours to get resources out to the clinicians who needed them most. When we set out, the goal of the Real Time Learning Network was to synthesize the latest Covid 19 information for busy, frontline clinicians who might not have time to read all of the journals and keep up with all of the latest information. In the beginning, there were not many resources widely available. There were a few protocols from large university hospitals that were generously shared, as well as some journal resources and government websites like ClinicalTrials.gov, Of CDC and NIH were also good sources of information, but guidance was really scarce, and our goal was to tease out the important information from the noise that was prevalent at the time. The Real Time Learning Network initially consisted of a lot of different components, and over time, as information became clearer, more widely available, we phased out some of those components, keeping what was only necessary for our goal, which is helping clinicians navigate the Covid 19 landscape.

Mati Hlatshwayo Davis: [00:04:25] Well that's phenomenal, Gayle, I really appreciate the background. It really sets us up for success. I think I'm going to go to the associate editors next, and then I'll come back to you specifically with some of the successes that you've seen to my wonderful team of associate editors. Can you talk a little bit about what it was like getting started? How did you become a part of this incredible initiative, and what did you see your role as? Because we each have different topic areas. So I'll start with you, Payal. How did you get involved and what was your role?

Payal Patel: [00:04:58] We'll shout out the founding editor in chief, Natasha Chida, here, who is really, you know, had a personal connection to a number of the associate editors and kind of invited us on one by one. And she was kind of doing a one man show at the beginning. This was a really important resource. You know, I think of like early on, we were all trying to do our own institutional guidelines, and we had like a poll at some point and asked a couple of stewardship programs, like, how many times have you done your Covid 19 treatment guidelines? And it was somewhere between 20 to 50 times, you know, people had been changing things, you know, as we had new resources. So we really saw this as a way to kind of connect everyone who is fighting the same fight and get resources out to the community.

Mati Hlatshwayo Davis: [00:05:47] Love that. And I remember those times, like sometimes, I know as ID docs we kind of want to put that behind us. But these were early times where there wasn't a one stop shop. And to be quite honest, people were even looking through Twitter and ID Twitter, which was still Twitter back then, if you can remember that wonderful, glorious time. I think what excited me most about being invited to join on, and I was invited on as the Health Equity Associate Editor, was number one that I love working for an association that continues to be a leader in caring about health equity. So it was exciting that that role was made possible. But the fact that it was all in one place was just truly unprecedented. And the fact that we were able to be on a website, on socials and make these resources available was just really exciting to me. Bill, how did you get involved in what's your role?

William Werbel: [00:06:35] Kind of similar to Payal's origin story. So I worked with Dr. Chida at Johns Hopkins. I was a trainee of hers, actually. And I was looking at my email in December of 2020, she reached out to me because she had been putting a lot of work editing, maybe like eight out of the 12 sections that were at that time present on the website. And that was just too much. You know, it's like drinking from a fire hydrant as the kind of data and evolving science grew. So I'd been somebody who was trying my best to keep on top of things, particularly in my little subfield of immunocompromised persons who were suffering a lot of disease burden. And myself and other care providers were not sure what we could do to keep them healthy and safe. And so Natasha reached out to me to kind of like, see if I could synthesize some of the stuff that I was trying to digest into these sort of bite sized but comprehensive morsels for the clinician to, you know, update that on a rolling basis. Keep people in the loop. And so I kind of honed in on that stuff like disease, you know, manifestations and sort of special populations, quote unquote, people with HIV, cancer transplants. Et cetera. And I got integrated that way and then expanded out into other things as we might discuss.

Mati Hlatshwayo Davis: [00:07:53] I love that you put a time stamp on it because December 2020. It was a wild time, right? We're rolling out vaccines for the entire nation again. Everybody is looking towards the CDC, but as ID docs who are really leading the charge, it was incredible to get from our colleagues, from experts in the field, these bite sized pieces of information that you can have in clinics, on the wards, as you're thinking about how to lead your own divisions through it. Most of us were part of our own incident commands and our own hospitals. Right? So it couldn't have been more timely. So, Ethel, could you start sort of shifting to what your role is and maybe some examples of how the information you curated was helpful, maybe not just to your colleagues, but even to family members, because for me, I was literally using this website both within my division at the time. This was before I was the director of health, and I've used it in a public health department since, but also for family and friends. But what's your role been and how have you been utilizing it?

Ethel Weld: [00:08:50] As we all just tried in that era and when the pandemic started to fly the plane while putting the plane together in mid-air, i think it was this brain trust of the the group of of associate editors for TLN was an incredible resource for me personally, as I kind of assisted with my own institutional guidance. And while taking care of patients and answering all the friends and family curbsides that came across my chat daily, which probably numbered in the teens to 20s and I'm sure everyone listening can relate. Having the expertise among the editorship in diagnostics and infection control and all these other things that weren't my exact bailiwick was really important. So my niche is therapeutics. I have a background in clinical pharmacology and infectious disease. I was really interested in both repurposed drugs and the bespoke drugs that were being created for this indication, and was really reading every word of every EUA that came across and trying to digest that. And so it was helpful for me personally just to try to synthesize that not just for others, but for myself. So I would get family and patient questions about, for example, one is myocarditis. Like, is my teenage son's heart going to fall out if I get this vaccine for him? This seems like a really hard decision. How do I think about this?

Ethel Weld: [00:10:16] And so just being able to pull up that great infographic  that people worked on of just pointing out, okay, risk of myocarditis after vaccination, extremely low, less than 1 in 200,000, about the time we made the graphic, a little bit higher risk in young males 12 to 29 years, but still very low, 1 in 14,000, much lower in young females. And then if you're unvaccinated, a 16 times higher risk of myocarditis with Covid in all ages and and ten times higher risk of hospitalization at that time. So being able to just pull those out and present those to people and say, just this is a way that you can weigh this decision that might be reassuring and helpful. And all of us are trying to weigh these decisions at the same time as making this guidance and providing care. So I felt like that was one helpful infographic. And the infographics in general, I found very, very useful visualizations to be able to point to and post in our clinics.

Mati Hlatshwayo Davis: [00:11:18] Couldn't agree more. I believe that we heard from so many people that those infographics specifically were downloaded. They were in clinics, they were used for rounds. I mean, people really sort of flocked to them. And what I loved about what you built, Ethel, that is going to be just legacy defining work, quite frankly, friend, is that you made it so accessible for folks and really helped move the conversation along. So can't thank you more for your leadership. Maria, bring us home. Talk a bit about your role. And like Ethel, any examples that come to mind for you as your role has evolved from that first invitation?

Maria Sundaram: [00:11:56] Yeah, so I'm the editor for the Vaccines and Immunity pages. I'm a little bit of the odd one out among this group because I'm not a clinician. I'm an academic public health researcher. So I have a PhD and not an MD. And a lot of what we run into in my field is doing the research and then having it sort of sit there on PubMed or, or med archive and just sort of like fester away and never be implemented. And that's, that's one of the most deeply challenging and frustrating things about doing the work, is that you don't want to be screaming into the abyss. I was so gratified to be a part of this group because it really was epidemiology for me. It was epidemiology in action. And just like Ethel said, it was also such a wonderful group to be a part of this kind of brain trust. That's probably true for a lot of the folks here that we were all trying to do our best sort of in our own silos, in our own way, at our own time, or in our own institution, and being part of a group of people that all had that goal, and being able to bounce ideas off each other and being able to support each other with helping writing a paragraph or suggesting a structure for an infographic was so deeply meaningful and special, and it made a lot of the very hard stuff during Covid a lot easier.

Mati Hlatshwayo Davis: [00:13:16] That is such a great way to sum it up, because I think what people may not know is just how much of a team, this is family. We're in a hilarious group chat which could be published on its own. We meet weekly, we collaborate on every single piece of information, and one thing I loved was that we didn't put our names to authorship because it was truly such a team centered approach, even though, as you heard, we all had our distinct roles. So I'll bring up the rear with health equity again. It was rare to see people without being forced to care about health equity. I'll be honest that my role was really challenging. I was kind of separate from you guys from a while before I was brought into meetings, because I was kind of cross-sectional across all of these. And then it didn't make sense that I wasn't a part of these incredible conversations. And so that has been wonderful. I'm used to health equity not being what drives metrics. And Gayle's going to talk to us a little bit about what our impact was and how we measured that. But it was beautiful. Ethel and the rest of his team, who reminded me that the numbers aren't what drives the work of health equity, but that we do it because we know that it is foundational and important and critical.

Mati Hlatshwayo Davis: [00:14:23] So to consistently be seen as valuable within this work was something important not just for me, but my community, that throughout these this almost four-year journey, we constantly centered the work of public health and community engaged spaces and minoritized communities. And so the justice part of what we did will always stay with me. And I think that we built something that is a role model for grants to come, even though I'm very clearly biased. So then, Gayle, to lead a group like ours because the jokes don't stop joking during a meeting, and you and Jessica and a lot of the leaders that came before Jessica have done a marvelous job of reeling us in. But measuring impact is always difficult. Can you talk a little bit about how we thought about impact? I believe we had a truly QI informed approach to this, and you and the IDSA team were just so great about this, and there was a lot that went into it. But what were your thoughts about how we measured impacts and can can you share some of the things that we saw over the last couple of years?

Gayle Levy: [00:15:28] Sure. Yeah. You know, we take a really analytical approach to looking at metrics here at IDSA. And for the CDC, they, you know, we had a lot of reporting out that we did initially. We were reporting out to them weekly. So we were looking at our metrics weekly. And when you look at your metrics that granularly, you're able to really kind of pivot as you're seeing what's you know, popular what's not as needed anymore and and really go from there. We then expanded out to looking at them monthly, which is still really helpful and also incorporated looking at, you know, Google search trend terms and kind of trying to look at what information people are looking for to really respond to that and give our audience what they need. And in turn, our traffic has just soared as we've done that with the website. So the website's had over 5 million page views since we've started in 2020, which is pretty amazing. We have over 100 pages on the website. And then, you know, there were a number of other parts of the grant as well that were included in the Real Time Learning Network. So there are five rapid guidelines on Covid 19 that were developed as part of the grant.

Gayle Levy: [00:16:42] We held 84 clinician calls, 81 media briefings. We convened an advisory group with 12 partner organizations that all were like minded and wanted to assist in getting information out around Covid 19. We ran a Twitter account for a number of years. We also let non-members sign up for the IDSA Covid 19 newsletter, which we had never done before. So there was a really big reach that we had as part of getting all of this information out to clinicians, but also to the public. The page that actually has gotten the most traffic overall on the website was we used to have more granular, specific pages that we would update over time, now we have more of a blog style format. But in the past when we had our Moderna page, when we first came out, we were the first ones to come out with information on that vaccine. It was a late nights weekend work, you know, but we got it out really quickly. And that page went viral and had several million page views itself, or several hundred thousand, not million.

Mati Hlatshwayo Davis: [00:17:44] I mean, the impact was impacting, as the young people would say. We really did touch so much. And what I'm so proud of is it wasn't just monolithic. Early on, when we were at the height of Covid, I had a dedicated podcast series. I started my career as a seasoned podcaster for the health equity portion of the Real Time Learning Network. Right. And what I'm also proud of is Gayle's team had to make really difficult decisions that I think actually helped us because just like we saw the pandemic grew and changed over time, we had to change with it. So we kind of moved from that model to the website being updated. That was a really big time for us, was updating the website and we each had such a big voice. So in thinking about the impact that you're hearing from Gayle, do some of you want to talk about where you see the landscape going now as the grant is over, and what advice you would have for people who still have to lead in this space of Covid and or may get similar grants or continue to just to need to advise and lead, whether it be in the clinical, the research, the public health space. What do you guys think? And I'll come to you, Maria first.

Maria Sundaram: [00:18:52] Public health and infectious disease medicine is lucky in that it has just no shortage of incredibly intelligent, very capable, very hard working people. What's really special about this group, in addition to the people in it, are the ways that we have gotten to work together. I've seen that in the past, actually in, in sort of like lessons learned from other pandemics that, you know, you've got to build that social capital if you want it to be sustainable, if you want it to be long term, if you don't want people to burn out. I was so challenged as someone, as someone on Twitter, when it was Twitter, and trying to get information out there to deal with the sort of like, you know, on the one hand you could say, well, no one really knows. And on the other hand, you could be extremely certain because 140 characters really gives you the space to be like, well, this is what it is. I have a PhD or I have a doctoral degree in this. You know, everyone should listen to me. And I think, uh, neither of those really feel right. And so I was really so grateful to have that that space for the middle ground, that space for discussion and that space for support. That's probably the biggest thing that I would want to tell folks gathering for the next pandemic or writing the next grant. What do you need? And I think the biggest thing that gets overlooked is social capital. And so I'm so glad to have been a part of this, this group that valued that so highly.

Mati Hlatshwayo Davis: [00:20:13] Oh, I love that so much. Ethel, what are you thinking about when you think about what's next and how to advise folks?

Ethel Weld: [00:20:19] I guess I have one just tiny granular thing for anyone trying to lead in this space, which is really try to collaborate with an excellent graphic designer, because high impact words are one thing, but high impact words linked with images that are indelible and stay in people's minds. Or it just ups ups the ante and kind of the impact of what you're trying to say. So and I think there's some really great CDC graphics that have been produced in several spaces. That needs to be prioritized in public health communications, because it's a way of cutting through some of the chatter and getting to the meat of the thing, which is what most people want to know about. Um, so that's just one thing I'm that I'm thinking about.

Mati Hlatshwayo Davis: [00:21:08] Oh, I love that so much. Especially as someone leading in the public health space. This is something that we're charged to do all the time, right? Increase health literacy. Make it accessible as simple as possible. And when I came on, I only had two public information officers, and I built out a whole bureau because of how crucial this part is. And I'll tell you that a dedicated social media person, dedicated digital media person on staff has changed what we've been able to do in this space. So I'll tag on mine to yours, is that I hope that we understand that even though places and platforms like the Real Time Learning Network may be coming to an end, you have local health departments and I sometimes we don't do our due diligence as academic centers and making sure that we're partnering with them when they have dedicated communications bureaus, dedicated community health workers, boots on the ground. Right. And so in this next phase, we're currently in a Covid 19 surge here in Saint Louis. Right. And we have language around that and packaging it around respiratory virus illness season as well. There are ways to really move into this phase with folks who do this all the time. And that's my hope. My hope is that we were able to provide a resource, and we'll talk a little bit about it at the end about how some of that will live on. But in the meantime, please make sure that that collaborative spirit continues with our local public health partners. Bill, what are you thinking about as far as advice and next steps?

William Werbel: [00:22:30] I thought about through the pandemic and then looking even further backwards, how nuanced and public health messaging is very challenging, but is to a degree essential. So, you know, the message that's made needs to be clear, and ideally straightforward. So there's some bottom line takeaway. But it does require some contextualization or nuance which can be through just like, targeting. Like you can say something that's pretty clear and then just say it applies to this group of people. And then you're usually safe, like it's something that tends to actually fit the data that you know that we're that was behind the statement you're making and maybe helps get away from some of this like oh the what abouts. You know. It's like oh what about somebody with this sort of profile or somebody with this history or whatever. Like, because once you start getting to those nuances, it's very difficult. It's not one size fits all. If you're being very specific about some sort of recommendation from a public health standpoint. But for a select group, it can be very specific. So this is a very difficult balance, like having an overarching. Kind of message that is true to many people is based on data. And then being able to drill down with the bottom line that can target a group. Which, like in my world and also your world, Mati, is about like someone who's very high risk, who you don't want to mince words. And so you're like, you need to get to that person with a direct message that is backed with some sort of data that you can link to, whatever it may be, you know, but maintain that kind of like welcoming sort of banter that's like, here's this message we're trying to deliver, you know, like that's sort of trying to navigate this. How can you be nuanced but not be wishy washy? I feel like that's a really difficult but important thing to keep in mind.

Mati Hlatshwayo Davis: [00:24:15] And I think the fact that you had to speak to it, Bill, means that it's not for everyone. And I'm not shy to say that. Like, I love, Maria brought this up a little bit in the social media space, there was a time where everybody was a glorified specialist and it created a lot of mistrust and distrust, some of which was very valid actually. So I love that we created a place that had validity and expertise and that led with that unapologetically, and that we can say to you that it's actually not easy. It's much easier to waffle on and on and on. Or write five pages about something. Sometimes when we were charged with a half a paragraph to a page, that's hard to do and give everyone what they need. Which is why, again, Ethel's point around infographics and visuals is key. Kyle, I'm going to come to you last. Any final thoughts you have about next steps and any advice?

Payal Patel: [00:25:02] I would really be thinking about this as from the federal funding perspective, you know, we learned that misinformation was such a huge, you know, barrier in this pandemic. And so this was really a way to use federal funding to flexibly involve experts. You know, they included EPI experts like Maria, infection prevention experts like myself, bring it all together, have some independence, but then come up with a weapon to fight misinformation. And so I would say that's my takeaway is that's really we're going to need that again. And we need to be flexible in how we think about that. And so this is a really a good way to partner with federal resources, but then also have that independence and flexibility. And that's what I think of when I think of the Real Time Learning Network.

Mati Hlatshwayo Davis: [00:25:51] I love that so much. So as we wrap up, where do we go from here? And, Gayle, if you could lead off by just telling folks what's going to happen to this website and all this information we curated? Do folks have access to it? And then would just love to hear from you all. What is the what is tomorrow, next month, next year look like as we move on from this incredible endeavor together?

Gayle Levy: [00:26:14] IDSA is redoing its own website that will launch toward the end of the first quarter of next year, 2025. A lot of the content will come over into that new website at that time. So the Real Time Learning Network website, Covid19learning.org will be available until that happens. And then once that happens, the content will be redirected to the new IDSociety.org website.

Mati Hlatshwayo Davis: [00:26:40] That's so important folks, because as someone who serves on the board, I can tell you that the new strategic plan, the new rebranding, everything coming out between the end of this year and first quarter of next year is such an exciting time for infectious diseases. It's really, really going to coalesce around our priorities, around workforce recruitment and retention and everything that we hold dear. And so this is the time to really get on the website and take what you need. But like Gail said, you will see it in different forms and formats, maybe spread a little bit in an archived fashion across the website, but please visit us now if you didn't know a lot about us, this is a great time to visit the website. If you have known about us and really depended on us, this is the time to kind of get what you need so that your clinics can have it ongoing. There are so many materials that are currently up that are timeless, quite frankly, and that have served all of us really well through the pandemic. Anyone with final words? I mean, I want to just say a big thank you to all of you. But as we wrap up, any final words from folks?

Gayle Levy: [00:27:42] I would also like to say a big thank you to all of you. You know, when we were talking about how to make, you know, some this work again for some other, you know, some other issue that comes up that we need to really rapidly respond to. And none of the work that could have happened across the website, the podcast, the media briefings, the roundtables that Marti did, all of the work that was done would never have happened without the generous volunteering of time by all of you and all of our other experts that volunteered on some of the press briefings and those kinds of things. It takes people that want to go above and beyond their day job, so to speak, to really give of their time and of themselves to do something like this. So I just want to thank all of you for doing that.

Mati Hlatshwayo Davis: [00:28:25] Thank you, Gayle. I want to make a plug to anyone in national leadership that this work is crucial. We're not done, even though we understand that funding is winding down for a lot of these initiatives. We know that pandemic preparedness is key, and that Covid only was a warning shot for everything that's to come. And we've already seen some of that. Mpox and avian flu. So many things that we are contending with now and so much more that could come. So I'm making my plea for prioritization of this work and funding for this work. Resource allocation for this work, it is crucial. Final words from anyone else.

William Werbel: [00:29:02] My thought would be see if we can remember lessons learned. You know we learned we but you know society burn some public health trust along the way. You know, in no small part because there was mis and disinformation battling in the same sort of field. But some things worked, some things didn't. Some there are some scars and there are some opportunities. So I think like being mindful of what's gone through and then being wiser as we go through the inevitability of the next threat to health.

Mati Hlatshwayo Davis: [00:29:37] Oh, I love that so much, Bill. And honestly, I'm going to say ten toes down. There were examples of things that worked and things that didn't. This worked. We worked. We have a living, breathing platform for how this worked, for tools that have been validated and used time and time again, the impact is undeniable. So I just want to say congratulations. Thank you for the honor of serving alongside you. I love that we will have a group chat till the end of time and be able to continue celebrating the births and construction and fashions within that group chat. I love you all dearly and thank you, thank you, thank you. This is the Covid 19 Real Time Learning Network leadership team saying sayonara and it has been our honor to serve. Thank you so much.