Josh (00:00:00):
Welcome to Friday Forum. What is this? The third 1 of January, Friday the 13th this time. Let’s jump into it. So as a reminder, the Friday forum is a weekly sync time just to celebrate accomplishments, achievements, what’s going on across functions. As a fully remote Async team, it’s often, as we’re focusing in our deep work, kind of difficult to stay up to speed with everything that’s going on across the company. And so that’s what this is all about, is sharing culture, getting on the same level. And then we do Async updates that go deeper into function progress. We have ad hoc meetings. And, of course, we have other forays for social and more business metrics. So just want to make sure we’re all aligned on the purpose.
(00:00:41):
All right. Recent achievements this week. So on the product side, Levels Levels rolled out to members as metabolic program. I think that went out today or yesterday, which is super exciting and very excited to see more of our members getting in there and getting that continuous feedback, the positive reinforcement of building points and getting rewards and just generally checking off the healthy little habits that you can get digital receipts from with the CGM. So that’s huge.
(00:01:11):
An updated member portal is live now where members can control their account, control, their subscription, et cetera. We had this great Labs 2.0 announcement video, which Rob graciously put together with us. It’s really nice and it explains the details of the panel that we’re putting together for Labs 2.0. Excited about that. In work, we’ve got an updated product strategy coming together. We’ve got Labs 2.1, trends. And then generally on the inside, a lot of bugs and tech debt that we’re catching up on right now, which has just built up over the past few months as we focus on shipping big lifts. So, good opportunity as product strategy comes together for the team to sort of catch up on some of that backlog.
(00:01:53):
Then in 2023, and I think Nicole has a deeper update on this, but we are focusing on manager development and just making sure that our performance reviews process, which we rolled out last year as a response to company strategy, or actually it might’ve been… Yeah, last year 2022. Just as a response to what we were hearing across the team, we’re now going to revisit, audit and calibrate our processes. So anyone that’s managing at the company is going to be connecting with Casey and with Nicole as we seek to understand how all the managers are individually calibrated and make sure that we are constantly updating that. So look for more of that in the next few weeks.
(00:02:33):
And then we had a really big guides experiment, which was quite scrappy. It was built on Instagram. I know Casey is going to do a deep dive on this, but Casey and Stacy both launched one of these. The objective was to see how someone who is guiding by sharing their own expert-level decisions and practices, daily practices around health, how that might help other people understand the value of both CGM, but also metabolic health. And it was really fascinating.
(00:03:00):
We had like 83% adherence, people were seeing the stories every day on the survey on the backend, something like 90 plus percent interest in multi-year engagement on something like this. 80% interested in trying CGM if they had not, and almost 100% said that they learned something valuable. So these are big numbers. Definitely exceeded what I would expect from something as scrappy and as quick as this was, but can’t wait to see the deep dive on that.
(00:03:27):
Support. As we know, the renewal season hit beginning of this month and there’s been a huge backlog of additional tickets. Did an amazing job catching up, getting to inbox zero midweek. So huge shout out to that team. And then Chris has put together a couple of new dashboards to help us stay on top of the metrics and new categories of member that we’re building. So a couple of new dashboards and yeah, that’s always helpful for us to be able to get a deep dive into the data. The Nurture series shipped. So the email series that Jen was working on is now out, it’s a six-email series to help people sort of understand what Levels is and nurture through the funnel to conversion.
(00:04:10):
The Demand Gen team is passing off website ownership to demand capture. So Karen’s going to be taking the lead on that with a big focus of course on improving our product messaging, product marketing and signup flows, et cetera. We’ve also got a couple really cool projects like this product teaser video that you can see here that’s just about ready to wrap and very exciting. I watched it and it got me hyped. I think it’s going to hit everyone the same way.
(00:04:36):
On the renewals front, for certain cohorts, we hit 70% retention for members who had an active CGM subscription. So this is a really interesting data point. Numbers were very different between those who had active subscriptions and did not of course, but to see numbers in this category is really exciting and it reinforces that for those people that find that value in the persistent CGM data and feedback, the membership also clearly is connected in their minds to that value, which is great.
(00:05:04):
And then the Try It experiment hit. We’ve now wrapped, Sys is going to do a deep dive on this, but we hit over I think three and a half times target conversions. So well over 90, I think the target was 25. So huge result there, really exciting. Editorial team came out with a bunch of new stuff and how to cut sugar. Introduction to Metabolic Health and bone broth are all very exciting as well as starting the foray into product marketing. So the theme that we’re seeing right now is really across all of our functions, pulling together the various skill sets and approaches to developing information and education about the product.
(00:05:43):
So you’re seeing product marketing across four functions right now. As we really lean into that new muscle that we’re building, everyone, I highly recommend checking out those new articles that the product or the editorial team put together and super excited to continue to see us all flexing that muscle. Then lastly on the demand capture side, tons of work on growth analytics, just getting our entire flow and funnel instrumented so that we can better track the steps of leading someone from landing from an organic search on our website all the way through to a potential conversion and even afterwards for retargeting.
(00:06:19):
Lastly, we kicked off sign up 3.1. 3.0 went out to 100% of traffic. So now with 3.1, we’re focusing on faster payments. We’re focusing on improving pricing messaging, which is a really important element right now. It’s by far the most confusing part about landing on the Levels checkout is it’s not clear that you’re paying for two different things if you are and sort of why. So really focusing on making that crystal clear and then adding in user referrals. And finally, performance ads are continuing to hit our CAC targets. So customer acquisition cost targets.
(00:06:53):
We got a 2022 mission patch coming together that was finalized. We have a couple of great memos out about some performance campaigns, venture debt drawdown, et cetera, tons of great UGC. And then lastly, sent out an email that was explaining to our members the split between our different podcast channels and that caused a huge highest single-day playback for all three feeds, I believe, which is great. So awesome to draw that attention there. I think that’s pretty much it.
(00:07:25):
With that, I want to welcome Cheryl. Cheryl Medina is a member and investor in Levels. She’s a project manager, loves many of the things we all love, including individualized health and nutrition. She happens to reside here in Texas like myself. Cheryl, thank you so much for setting aside some time this Friday to come hang out with the team. I’d love to just hear from your own experience what you are loving and excited about in the world of health, your experience with Levels. Just generally, we’d love to hear some thoughts.
Cheryl (00:07:53):
Okay. Yeah, thank you so much for having me. I’m honored to be invited here. I’m really nervous right now, so you may need to pop in with questions if I lose my train of thought here. I’ll try not to go off on too big a tangent. But before I go into my experience directly with Levels, I just want to give a little brief background about myself in terms of where I’m coming at it from. I’m not coming at it from an elite athlete standpoint. I see myself more of sort of the average American. I got get caught up in the standard American diet, so to speak, over the years and up and down, up and down with things.
(00:08:35):
And I’ve always wanted to have better insight into why can’t maintain certain things. And so I’ve always been big into tracking devices. I’m going to H myself a little here. All the way back to the analog pedometer, to the little… Remember those little Nike things that went on your sneaker [inaudible 00:08:58] your steps? Fitbit, now it’s the watch or ring, even have the eight sleep bed to try to help me maximize my sleep.
(00:09:10):
So coming into that, that’s where I’m at, is I always want to have insight, measure, and see where I can improve. And just before I joined Levels, I was starting on that intermittent fasting and time restricted eating. And I was using the Keto Mojo to track my ketone level and glucose and just have a snapshot. So when I heard about Levels on one of the biohacking podcasts that I listened to, I just wanted to jump on that to see. It’s one thing to see a snapshot, but it’s nice to see the actual curve and see the differences in how certain foods or activities affect the spike, how long it is, how high it goes, but also how the duration and so on and so forth.
(00:10:00):
And I’ve always been a little bit of a science nerd. Going back to aging myself. After university, I dove right into DNA sequencing back when it was really manual. So as a lab tech on the bench and worked my way up into operational management in the DNA sequencing world. And so I was on the project that sequenced the human genome first in the academic side. And then I moved to, you may have heard of [inaudible 00:10:32] Genomics. I worked with them and helped build the sequencing lab there and we completed the sequencing of the human genome.
(00:10:39):
So I’ve always been interested in really creating just whatever small contribution that I can make and be a part of is really something that drives me. And so when I saw the mission of Levels and what you guys are really after, it really hits me and I see it as just the beginning. That’s why I chose to be at a small level granted, but an investor in the company. I really believe in how you guys are building this and what it’s all about. And I am one of the people that as soon as I saw that email with the three different breakout podcasts, downloaded those so I could get… Because I really love listening to not only the podcast that Casey mainly will do with the science and the doctors, scientists and so on. That’s fascinating.
(00:11:32):
But I’m also being an operational manager and a project manager now, love hearing about company, the building of the company, and all the thought and process that you guys put into. Just I dig it, I love it. So thank you guys for being transparent, putting that out there for us to hear, for people to hear.
Josh (00:11:51):
Tons of synergy it sounds like.
Cheryl (00:11:53):
Yeah. Yeah. I love it.
Josh (00:11:53):
As a fellow eight sleep owner, I couldn’t do without anymore. So I would love to hear, as you’ve gone through your experience, and obviously, you’ve been in wildly fun fact that you were there at the beginning of the human genome sequencing, by the way, we should have had that on the slide. But yeah, you’ve obviously been in this world of health technology or at least paying attention to the way that the human body works far beyond what most people have.
(00:12:22):
And so I’d love to hear from your perspective where you want this industry to go in order to maximize for your own experience for people that in your life who could benefit from better health. Where does the technology need to go and specifically what could Levels do better to get there?
Cheryl (00:12:39):
Wow. Okay. So as I mentioned earlier with the Human Genome Project and everything, I really feel like personal, what got me into that wanting to really support it is personalized medicine. And now 20 years later, it’s the functional medicine. And I’m a big believer Dr. Hyman’s food is medicine kind of thing. And so what I’m seeing it as used for me is to see what… To bring everything together into what do I need to do to keep myself healthy, and then my family members and my community, how can this platform help? It’s funny because you guys had an email this morning that came out about a new, I forgot what you called it, but it’s going to give you insight into the different components that you put together to earn points or something. I forget, sorry, what it was called.
Josh (00:13:40):
Levels Levels. Yeah.
Cheryl (00:13:43):
It’s funny because as I was thinking about this call, I was like, “It’d be really cool if they could bring all this together and put a score on there rather than just my spike time or my metabolic time.” What else could I personally do based on my data? And I think you can incorporate the sleep data, but to be honest, I haven’t really dove deep into that yet. So I think it’s there with Google Fit or something like that. So for me personally, to keep me going, I like to see those things. To me, it’s motivational.
(00:14:20):
When I was asked to put my little slide together, I asked my husband, I’m like, “What’s my hobby?” He is like, “Duolingo.” I’m like, “What are you talking about?” I didn’t put it on there, but I’ll tell you. But just like the streak going, even if it’s only five or 10 or 15 minutes in a day, to me it’s just something that keeps me going. And I know back in the early days with Levels, there was a streak, but then it was like, “No, we don’t really want to keep people going on a streak just for the sake of doing that,” because I could have a glass of wine with dinner if I wanted to maintain my low level streak.
(00:14:55):
If you learn how to game the system and I don’t really want it for that reason, but at the same time, I want something to motivate me to come back and see how I did and earn points towards it. I don’t know, for some reason that appeals to me, one of those people. So thank you, guys, to whoever had that idea, the person implementing that. I’m going to jump on that. I logged into the app this morning, I’m like, “Oh, it’s not there yet.” She did save the next few times.
Josh (00:15:22):
It’s exciting to hear that that’s positive reinforcement. It sounds like it’s going to land with you. And obviously, it’s a big lift to get to that point. I’m really curious to follow-up and hear how that new layer on top is exactly what you’re saying, is this is about additive behaviors that are healthy as opposed to restrictive behaviors to try to hit some flat streak, right? It’s learning and continually reinforcing. Love to hear that.
Cheryl (00:15:48):
For the right reasons. And that way too, bringing in… So my sister is also a Levels member. I dragged her in after my first two months. I was showing her the data and, of course, being siblings, I’m like, “I wonder if you have the same reaction I have to such and such.” And thankfully, she was able to get in and she lives in San Diego, I’m in Fort Worth, so in Texas. So we’re not close, but we’ve had times where we’re on vacation together or something and we’re actually eating the same meal the same time and have very much the same… And it’s interesting to see how different sometimes we really are.
(00:16:28):
She has huge reactions to things like sugar substitutes and I don’t. And her metabolic rate is somewhat slower too than mine, it takes her longer to clear. And we do have diabetes in our family, so that’s another reason why I wanted to be on top. My end goal is to make it into old age and not have to have a drawer full of pharmaceuticals, quite honestly. I don’t want that for my life.
Josh (00:17:04):
You’re in good company. Well, that is the mission here is to make sure that people have the tools and technology to get well ahead of those sorts of things, and be shooting for an optimal target as opposed to trying to avoid a negative outcome. And ideally, the avoidance is part and parcel with the optimization.
(00:17:21):
Cheryl, it was really amazing to hear from you. Thank you again for spending time with us this morning. If you’d like to stick around, we have a full meeting and would love to have you the whole duration. If you’ve got other stuff to do though, totally understand. And again, thank you for sharing directly with us. I know it might be stressful, but it is super valuable for our team to hear directly from you.
Cheryl (00:17:40):
Yeah, absolutely. Thank you again. And again, I want to just say thank you to everybody from all the department areas, everything that you’re doing. It’s making a huge difference and impact on me and my life personally, and I’m a huge fan. So keep up the great work, guys. Thank you. Sorry, I’m going to hop off because I’ve got to go get on a cruise ship today. I’m actually calling you from Long Beach right now.
Josh (00:18:06):
Go enjoy it. That sounds amazing.
Cheryl (00:18:08):
Thank you.
Josh (00:18:09):
Bye-bye. Thank you.
Cheryl (00:18:09):
Bye, guys. Thanks so much. Bye-Bye.
Josh (00:18:12):
All right. That was awesome. Okay, culture and kudos. So I’m going to go ahead and refresh this. Yeah, okay, there we go. So first off, at the top, culture survey Q4 is ongoing, so please make sure that you get those responses in, we’re shooting for 100%. So please do respond to that, it’s really important for us. Happy one year to Ian and Sonny both on January 10th. Super huge that year flew. Riley and Charu here in Kenmore, Alberta. Got a chance to hang out and ratchet up those bingo points for 2023. Don’t forget.
(00:18:44):
And then lastly, some kudos to Priya. There was a difficult series of tickets that popped up in Help Scout this week, and Priya just did a really great job of being proactive, pulling in the right people, and then finally closing the loop on those conversations and just want to appreciate the attention to the details and to making sure that everyone was on the same page about how the cases were closed out. This sort of stuff really matters. Sometimes it’s unclear whether or not to go that extra step and share an additional message in comms. And I think in general, especially fitting with the confidence is earned thesis, more information is always better and it’s so helpful to have those loops closed. So thanks, Priya, for setting a great example there. Over to Miz.
Michael (00:19:28):
This is your monthly reminder not to buy gift cards for Sam or Josh. Worst case scenario here is that we buy some gift cards and waste a few hundred dollars. It’s actually not a bad outcome. These are relatively benign fishers and hackers, but it could be worse. And so this is just really good monthly training for us and I thank these people for doing this on a regular basis. So we got flooded with these last week emails, texts, just a reminder that if you get a text or an email that looks like this from Sam, Josh or anyone, don’t take the bait, don’t buy gift cards and visit Levels links/security to brush up on your security basics and skills. And that’s it.
Josh (00:20:10):
Thank you, Miz. And thanks, team, for continuing to pass these tests. All right. So this week it’s Sunny Async with our culture value. Don’t ask how you can help, just help. Go ahead and kick that off.
Sunny (00:20:27):
Hey, team. So excited to be presenting the cultural slide this week in particular because this is the week of my one year Levelsversary. And while the mission is very, very near and dear to my heart, it was in fact the culture and how we do things that ultimately led me to apply to Levels. So with that, today I’m presenting on how the axiom of don’t ask how you can help, just help really impacted me at the close of last year.
(00:20:53):
So the first thing I want to share is how hearing a year’s worth of Levels culture and seeing how this axiom in practice ultimately led me to take action in my personal life. Just after returning home from holiday and closing up the Thanksgiving festivities, I received notice that both of my grandparents were taken to the RR. And while a trip to the ER is never pleasant when you’re in your mid-90s, it’s a pretty big deal. Due to the nature of their medical concerns, one was taken to a facility at many hours away, the other state in the local facility.
(00:21:25):
And this is a location in the United States that is very remote and you can’t even hire, you can’t pay people to come down to do in-home or healthcare. So having seen this again and again in practice and also hearing, “Take care of what’s important, take care of you, take care of your family, we’ve got this.” Having built that confidence, I didn’t ask on the phone of my uncle, “Oh, my gosh, this is horrible. How can I help?” I didn’t ask my supervisor, “Hey, can I take some time and take care of some family stuff?” That weekend I booked the flight and I just got down there and I just said, “I need to be there to do anything I can do to help.” So seeing this in practice led me to take action on a personal level.
(00:22:04):
And then I really wanted to share more than that once I was there how my colleagues on the support team lived and breathed with this. And I know I’m going to miss calling someone out in particular, my sincere apologies for that, but some examples of this in action that really meant a lot to me. It helped me focus on family. Rebecca, for just stepping up to do some of the daily processing I do, including processing at the in-app replacement dash. She just did it.
(00:22:32):
There wasn’t an ask, it was a, “Hey, just so you know, I’m covering this for you.” Brittany did the same thing. Thank you, Brittany, for covering my week ahead for support posts where we kind of rally everyone on the team to make sure everyone knows what’s going on and where. This is something that she had taken on for the first time the week prior when I was on vacation.
(00:22:47):
And instead just said, “I’m going to go ahead and take care of this for this week as well.” Thank you to Lynette who covered, we had just onboarded her onto onboarding parties. So Lynette had just hosted her first solo onboarding party and that week said, “Hey, I’ve got this covered for you, don’t worry about it.” That also means we didn’t have to cancel and disappoint any of our members. And everyone on the team, but especially when I put out or call out Taylor, who at the time, we were really working through e-commerce migration, crushing lots of bugs.
(00:23:17):
At the time there were close to 400 orders in our state and I was working on these things between 11:00 PM and 4:00 AM. Taylor was exceptionally helpful in don’t ask how you can help, just help by helping me triage and say, “These are the things that I need your attention on. These other things, I want you to be situationally aware, but otherwise, no action needed at this moment.”
(00:23:40):
So this axiom of don’t ask how you can help, just help was really embodied by everyone on the support team. And I didn’t even, I’d be completely remiss to not mention Chris who embodies all things Levels culture. But in particular we have a weekly one-on-one and Chris just so ahead and cleared my schedule and said, “Nope, you don’t need to be doing this.” Although they are the highlight of my week, we still did our one-on-one.
(00:24:02):
So thank you to everyone support team for supporting me and for picking up, not asking how you could help, but just taking care of business to ensure our members are taken care of, and also to allow our internal customers, our teammates, to be able to focus on the other important things in their life. So thank you team. I hope sharing this makes you proud of the work that we do here.
Josh (00:24:25):
So good. I love that super personal example. Thank you, Sunny. And yeah, I think that that just crystallizes exactly what this value’s about asking how you can help is a nice step, but it places the burden on the other person. And in this case, if you can see what needs to be done and just take it on, we see examples of this all the time at Levels and I love it. So thank you, Sunny. All right, I’ll try not to replay that. People Ops update to Nicole.
Nicole (00:24:55):
All right. Well, shifting gears a little bit, that was also beautiful, but I have a few people ops updates to share. Also wanted to mention too, there will be a weekly people ops update in the Async update the video that goes out every week. So make sure you’re catching those. And I’m going to include a lot of broad things that are happening, but also week-to-week projects and things like that. So first off, we are closing in on about one year, as Josh mentioned about with having performance reviews.
(00:25:26):
And we know we need to iterate on this process to make it as valuable as possible for everyone. And specifically, we really want everyone at Levels to know where they stand, how they’re doing, and to get the regular feedback that everyone needs to grow and succeed in their roles. And currently, we have a lot of room for improvement and especially around consistency and more thorough documentation.
(00:25:48):
So we’re working on several projects within this realm to get closer to this goal. And the first of that is a project with Sonia and Casey and myself auditing how the current system is being used. So everything from our 30 and 60-day onboarding check-ins to six-month performance reviews, to the star rating system, to monthly check-ins that we have and how these are being used area to area. We want more consistency because we know not having consistency makes it a little bit more difficult.
(00:26:21):
So we want to know the gaps and understand where we can improve upon these systems. And we’re doing manager check-ins currently and I’m going to do more of those more frequently to help guide and support. And, of course, this project ties into a lot of different things within Levels, everything from role leveling and promotion and that process to giving feedback and improving performance.
(00:26:41):
And then also especially around the star rating system and making sure that that’s consistently applied. We’re reviewing the existing performance trackers and reviews and our plan is to share our findings and takeaways by the end of this first quarter, so March-ish. And we’ll have specific improvements that we share at that time. And then a related project, because it’s very intertwined, is around manager support. We want to make sure that our managers are well supported and that they’re continuing to learn and grow because that is amplified throughout the rest of the company.
(00:27:17):
And so Casey and I did some check-ins with people managers to talk about the performance review audit, but also how things are going in general. And then I’ve also done some surveying with the managers to hear how things are feeling, where they want more support. And then we’ll have a lot more weekly discussions and prompts within the people manager group.
(00:27:37):
There’s a lot of benefit that comes from that internal community building and support structure. And then, of course, we’ll bring in additional speakers or training is needed throughout the rest of the year, but there’s a lot to be gained just from internally auditing ourselves, really looking at how we’re doing, give ourselves that tough and critical feedback and try to improve our systems.
(00:27:57):
And then my final note is around the culture survey. Please fill that out today if you can. We really want 100% participation and right now we only have about 30 responses. I’ll share the link in the meeting chat in just a minute. But this survey is very helpful for us, it gives us a really consistent trend in a lot of different ways. And this is also what really informs us around how we improve systems like the performance review process and things like that. So thank you so much for those who have filled it out. And for the rest of you, please go ahead and fill that out and if not, you’ll get a reminder from me at some point. So thank you so much everyone.
Josh (00:28:37):
Awesome. Thank you, Nicole. Important stuff there. And with that we’re going to run into company objectives. Level shows you how food affects your health. No changes here. Product stop priority with that, Maz.
Maziar (00:28:52):
Morning, welcome to product update, the first one of 2023. All right, we wanted to give you another preview of what we went up to and what we’re thinking. Next slide, please. Starting from the company objective and what is product really here to do? Going back to the basics, we’re really here to improve our members’ health and do it in a way that we grow the company so we can get to our mission and vision, which is improving the health of a billion people.
(00:29:22):
And if we can do this, we believe that we can achieve the three company objectives. We can obviously improve people’s health, that’s in the line and we can grow and retain people. So this is what we’re really focused on and to really stay focused for 2023, we’re going to focus on our Maureen persona. She is our largest cohort and she’s our most engaged cohort no matter how you cut it. And we are really, really happy to focus this year on Maureen. Next slide, please.
(00:29:54):
But what does it really take for Maureen to achieve her metabolic goals? And we’ve invested in all of these areas throughout ’21, ’22 and we’ll continue to invest in these areas really to help Maureen improve her health. We need guidance, a guidance that’s trusted, personalized and engaging. These are the core values of Levels, the great work that we’ve been doing, blog, the stuff that we started putting in the app, really focus on guidance to really drive actions in a way that is simple, rewarding and sustainable really based on the behavior change principles that we really have been investing in and developing.
(00:30:33):
And then finally, keeping people accountable. I think we’ve made some progress in all of these areas, but really what we’re trying to do for 2023 is double down on these three to improve the metabolic health of Maureen. Additionally, we have heard from our members that they love the accountability of the CGN, but it’s simply not affordable to sustain over a long period of time, especially if you really want to go after large numbers and growth, given that not everybody can afford a CGM for 12 months. Next slide, please.
(00:31:09):
All right. So what are we doing? First of all, today is all about telling you about an experiment with it. But before that, one of the big things we’re thinking about is making the CGM optional. And the way we do that is not to get rid of biological feedback. Biological feedback is our core, is our DNA. And I think it’s what really differentiates us from all the competition out there. But really instead, use the CGM data from the community to help people that either cannot afford, don’t want to wear CGMs because it’s invasive, or whatever it is, it doesn’t matter.
(00:31:40):
And really leverage the community data to provide them the biological feedback and really make CGMs a shared resource. So that’s a big, big thesis for us for 2023, and we’re going to put a lot of resources in that and make CGMs optional, not CGM-free, there’s a difference. But CGM optional, meaning if somebody wants more personalization, somebody wants more data, we will make that available too. But for a lot of people, they may not be in the position, especially when they start to wear a CGM.
(00:32:05):
So we wanted to do an experiment, the experiment based on a thesis that was actually in place in 2021, which is can people learn better from other people? We continue to actually do this experiment 2022, we did an experiment called Personality Driven Insights and we used a few of our influencers like Austin to do it and we really were encouraged with results.
(00:32:27):
Austin’s videos were once some of the highest engagements that we saw for content in our app and on our social, which gives it a little bit more confidence to say the thesis or the hypothesis of do people learn better from people is true. And it turns out, spoiler, actually I won’t give you the spoiler. Casey will talk about that. People have been learning from people since the beginning of human civilization. So this hypothesis should not be surprising.
(00:32:55):
This is why we have schools, this is why we have universities, and this is why we have YouTube during COVID so people can learn from people. And our hypothesis was, can actually being able to connect with somebody that you resonate with trust, can that person help drive the guidance, drive the action, and drive the accountability. And with that, I’ll turn it over to Casey to talk through the beautiful experiment that Casey and Stacy did to run us to read Hypothesis. Hypothesis in a true level style of having the hypothesis, testing it, building this skateboard.
Casey (00:33:34):
Awesome. Thank you so much, Maz. Can you guys hear me okay? Thumbs up. Awesome. Okay. Next slide, Josh. Okay. So I’m super excited about this experiment, but also because this is my first time ever presenting on behalf of the product team, I’m super excited because the outcomes of the project were, they were surprising, they were exciting, and they are very relevant to our software optional CGM future trajectory.
(00:34:05):
So I want to start with drilling into Maureen a little bit and what we’re here to solve for her. We have surveyed her several times and learned a lot about why it is challenging for her to be healthy. And this is a summary of the barriers that Maureen have consistently expressed that are barriers to achieving her health goals. So there’s too much information. Who do I trust? The information that I’m getting from these podcasts and blog posts, it’s not actional, it’s not for me, it’s not engaging.
(00:34:38):
Healthy living is hard. It feels unsustainable. I have to do this every day. I lack support and encouragement. We often hear that the environment is challenging, the family environment, having kids and husbands and whatnot who are part of the equation too. I struggle with accountability. I already have so much to do and being healthy is expensive and this phase of life is expensive, there’s kids, et cetera. So that’s what Maureen is up against.
(00:35:10):
And our opportunity is to help her meaningfully solve these pain points in a way that doesn’t necessarily require a CGM and may leverage data from other individuals or communities wearing CGM to give her some semi-personal insights. And this all serves as a foundation for our guides experiment. And just to summarize the hypothesis, the hypothesis was that a software-only product with components of guidance, action, accountability and affordability can solve Maureen’s pain points and that she would want to use this product. Next slide.
(00:35:46):
So briefly we’ll discuss how we designed and ran the experiment. There were three cohorts as you can see here, and I’ll walk through them. In the first week of December, David, Maz and I started shaping and designing the experiment and kicked off the first of three cohorts two days after Christmas on December 27th. And the final one actually ends today. Initially we recruited Marines who were Levels members, ages 40 to 60 in their first month of using Levels. They had joined Levels in the last month. They were pretty new.
(00:36:16):
We sent out a signup email to 200 of these people and 65 of the 200 signed up, which in and of itself is pretty amazing. There was a 32% signup rate with just one email, and I think that happened in a day. We slotted 10 of these 65 women into our first three-day pilot, which took place in an Instagram private group chat and was very high touch, very interactive guides experiment where 10 of these women and myself as the guide were having daily personal interactions in the chat as well as with each other.
(00:36:44):
I was guiding them through a semi-structured, light metabolic health curriculum through the chat, sharing recipes, photos of my logs and food, explaining my logs, recording videos for them in the chat, showing them how I implemented metabolic health in my day-to-day life, prompting them with questions and challenges, and inserting little polls into the chat, and answering their questions very directly. Then in cohort two we wanted to iterate on a more scalable version utilizing the close friends feature of Instagram stories.
(00:37:13):
So this is where the guide would be producing content and projecting it out to many people in a one to many interaction. So just sort of sharing these stories out into the void of all these followers. And the followers, they don’t know who the other people are, they’re not engaging with each other and they’re not directly interacting with the guide, there’s a barrier. And they can only interact through structured places like AMA boxes in the chat.
(00:37:40):
So this is a much more scalable one to many type of interaction. And we wanted to see whether the feedback we got from that high touch version versus this much more scalable version, how that would differ or be similar. So we brought in 35 people from that first recruit of that first group of emails we sent out, so women in their first month of Levels, ages 40 to 60. We took 35 of those from that group.
(00:38:05):
And then we had a control group. So these were people who I recruited through my followers through an Instagram story, sign up link, a jot form link. And I said, “Hey, if you’re not wearing CGM, you’re in this age group, you want to learn about metabolic health this week in a private chat, come on and join.” And we got capped in minutes.
(00:38:24):
We were trying to sign up 35 people. By the time I refreshed the link, there were 65 people who would sign up. So we closed it down within the hour. And so now we had 35 people of CGM wears, 65 people who are non-CGM wears in a private chat who are going to follow this content for five days, all added to a close friends group. So then we did cohort three, which was what Stacy ran, which is finishing today, which is amazing. And this is 10 times in size.
(00:38:54):
So this included 1200 women in terms of demand. So Stacey put up a jot form link, said, “Hey, I’m going to be…” And important to remember also my group is a software-aware group of people. Stacey’s is fairly software and [inaudible 00:39:10]. This is more of a lifestyle, fashion and travel Instagram account. And so it was a question mark, would these people want to sign up for this?
(00:39:18):
And Stacy put up a story that 11,600 people saw and out of those 11,600 people, 1200 people signed up. So 10% sign up rate within 24 hours, which is incredible. And so all those women were added to Stacy’s close friend group for five days. And so all the people in that group did not have a CGM on. So we tested basically CGM only high touch CGM and software-only, low touch cohort two. And then software-only people, cohort three.
(00:39:49):
So this is a brief snapshot of the type of content that you might see. And if you’re not on Instagram, this is kind of just give you a sort of idea of what Instagram stories look like. So this is now looking at cohort two, my group on the left and Stacy’s on the right. So stories are basically… These are screen grabs, they can be videos that are up to one minute… Oh, I’m sorry, Josh. Yeah, next slide.
(00:40:12):
These are screen grabs of the stories. They can be up to one-minute long videos, you can post as many as you want per day. I produce about a hundred pieces of content in that five days, so about 20 per day. And they can include screenshots, polls, question boxes, links, screen screen recordings, et cetera. There’s a lot of functionality. We both shared pretty consistent types of content, including a basic metabolic health curriculum, a peek into our daily living and how we implement metabolic health principles, recipes, articles, links, product recommendations, screenshots of our glucose data challenges for the group Ask Me Anything, and polls.
(00:40:48):
And again, in these groups, the followers were not interacting with each other. So there was no expectation of direct contact with anyone, it was really just more content consumption. This is just… Oh, sorry, next slide, Josh. Sorry, I’m clicking through on my mind and I’m realizing it’s not showing up for you guys. If you just want to press, this is just a brief screen record of the archives of these stories.
(00:41:11):
They disappear after 24 hours, but you can imagine each of these are like a one-minute video talking through me taking my morning sunshine walk, why sunshine is valuable for metabolic health, a list of all my favorite metabolically healthy cookbooks, posting the AMA responses from people, answering people’s AMA questions, talking through my glucose responses, talking about coal plunging, et cetera. Next slide.
(00:41:40):
Okay. So let’s talk about what we learned. So we sent a post-survey to all the Maureens. So I’m just going to talk about the learnings from cohort two and three and simply put Maureens found it very, very helpful. So in cohort two, which is my cohort, 70% of… We also got over 50% adherence on the post-survey. So this is a pretty good data. 70% of people found the close friends chat very helpful. And in cohort three, 91% of people rated the close friends group experience with Stacy very helpful. Next slide, Josh.
(00:42:18):
Maureen’s almost universally learned something and made diet or lifestyle changes during this five-day experiment. So people actually took action. 93% of cohort two said they learned something and 99% of cohort three learned something. Remarkably, 67 and 76% of people actually made healthy changes during the five-day experiment and 20% more in each group actually said they hadn’t yet but planned to make changes. So there was some action that was taken. Next slide.
(00:42:49):
So I’m going to now run through, on the top right, you can see our different pillars of our product experience. So guidance, action, accountability and affordability. So I’m going to run through each of those and what we learned. So let’s start with guidance. And under guidance we have these principles of is it trusted, is it personalized and is it engaging? So we’ll talk through whether each of those criteria were met.
(00:43:14):
So was it engaging? The data points to yes, 75% of the participants in the groups viewed the stories daily and there was minimal drop off throughout the day or over the course of five days. So basically just steady viewing, people were clearly waiting for these and watching the whole thing on normal Instagram stories just for context. Usually if I post 10 stories, there’ll be a dip after each story, like 100 people fall off with each story. With this, it was just basically consistent throughout the day.
(00:43:42):
For the polls, when we posted a very simple passive poll like this, there’s a picture here on this slide, 53% of viewers responded to the polls. When we posted question box, 34% of viewers responded. And when we did Ask Me Anything, 64% of people responded and submitted a question. 70% of people surveyed felt that their Levels journey felt more fun. There was 24 to 35% click-through rate on food recipe links that were shared. And cohort three, we don’t have that exact data right now, but we drove 3,200 people to Levels content, blogs and videos.
(00:44:19):
And in the surveys, 87% of people were most interested in engaging with the experience to see how to put healthy principles into practice. And 67% of cohort two said that they would actually engage with the guide feature for years if they had the opportunity to. And 89% of cohort three said they would engage with the experience for years, which is pretty cool. Next slide.
(00:44:47):
So the next question is, was it personalized? That’s our second pillar of guidance. So 65% of people agreed that their Levels journey felt more personalized, which is fascinating because there was no direct interaction with a person in this. So that’s kind of interesting. But these were some of the quotes, “It felt very personal, like getting advice from a friend.” “To learn what works for others, not only a theory behind it?” “I had a real person, someone I follow.” “So relatable.” And these are quotes between Stacy and my feedback.
(00:45:16):
So very relatable came up many times. “Ability to have my specific questions answered in the AMA.” So the only time they could actually interact in a removed way was through the AMA box, the Ask Me Anything. They never actually were interacting with the guide directly and they still felt like it was very personal and getting advice from a friend. Next slide.
(00:45:38):
So next pillar of guidance, did it engender trust? And we didn’t have a direct question for this on the survey, but we got a lot of quotes that mentioned trusting elements. So they were happy to be able to chat with an expert. Again, they weren’t chatting actually with the expert. They wanted to hear from an expert. The doctor sharing real glucose data was so thorough. Science-backed principles explained in an actionable format. Next slide.
(00:46:05):
So now moving on to action. So our action pillars are that it’s simple and sustainable. So we’ll go through whether we fit those. So in terms of just action generally, 67% of participants made healthy changes to their diet inspired by the content. And 22.4% planned to make healthy changes, but hadn’t yet. In Stacey’s group, 72% agreed that they took more action that week and felt… I’m sorry, in my group cohort two, these are people somewhere, the section of my group wearing Levels who were surveyed said that 72% said they took more action and felt more accountable than if they were using Levels alone.
(00:46:45):
So they’d had a few weeks to use Levels, then had this experience and they took more action. And in Stacy’s group who were non-CGM wears, 96.6 agreed that they felt empowered to make small changes. Next slide. So now in terms of action, was it simple? Was it sustainable? Here’s some quotes that speak to these. So simple people said, “Seeing how this person does this makes it doable.” “It’s aspirational and achievable.” “It’s approachable and real. Small steps to implement.”
(00:47:13):
Easy to implement, encouraging, practical, easy to implement. This came up over and over again. And was it sustainable? People feel like they got to view what is possible and how it could be maintained. It didn’t feel restrictive, it didn’t feel judgmental, it was approachable. So we think we really met this simple and sustainable through the personality-driven content. Next slide.
(00:47:34):
Next is accountability. And our accountability principles are was it supportive and objective? So in terms of accountability, 72% agreed that they took more action and it felt more accountable than if they had been using Levels alone. And then in Stacy’s group, which was all non-CGM wearers, they were asking whether they felt more accountable compared to their normal health journey, and 76.9% felt that they did. Next slide.
(00:48:00):
Was it supportive? All signs pointed to yes, many comments on this. They felt connected, they felt accountable. The reminders helped hold them accountable. It felt like a small community. They felt like they were part of a special like-minded group. They love the community, Stacy’s building here. Again, there was no community. It was actually just passively consuming content. So this is sort of fascinating. And then was it objective? So we don’t have actually a question from the survey to answer this, but we did these check-in several polls in the stories asking people. We challenged them like, “We’re all going to not eat refined grains and sugar tomorrow. Are you in?” And then survey them on whether they actually did it.
(00:48:40):
And so, people really loved those surveys to show whether they actually did it or they didn’t or what was the problem. And so that was our objective data for that. Next slide. We’re almost done here. So in terms of learning, the last principle is CGM optional, which means basically is it affordable? Can we do something valuable in an affordable, sticky and non-invasive way? And so we questioned two on their willingness to pay, and 60.8% of people said they would value this experience of just interacting with a guide or just following a guide for $5 or more per month. And 43% said they would pay greater than or equal to $10 per month just for this experience, which is pretty cool. Next slide.
(00:49:30):
We got some comments about affordability for this, “This type of thing. And Levels Kitchen helps me when I’m strapped for cash.” “For this knowledge base, I’d pay up to $10 per month.” And one healthcare provider said that she has a lot of uninsured patients and simple suggestions like this that are free and showing how to do them is something she’s going to share with her patients. So really love that this helped people who felt like there’s a barrier to entry for Levels. Next slide.
(00:49:57):
So the next question we asked them was, does this make you want to use a CGM? And so remember in my group, 30% were CGM wearers, but 65% were not. And Stacey’s 1200 people not wearing CGM. And my group was A CGM-aware group, Stacey’s was GM naive. In my group, 96% of participants were interested in trying a CGM. And in Stacey’s group who, again, this might’ve been their first exposure to CGM at 76.3% of participants who are interested in trying a CGM. Next slide.
(00:50:29):
So just to wrap up, this was a really interesting useful experiment. We learned a ton and it was just a really great, I think, example of the Levels ethos of rapid experimentation. We learned from nearly 1500 women in our target demographic and in analyzing their engagement, their survey data, we found that the software-only experience allowed them to successfully receive guidance that they felt to be trusted, personalized and engaging. They took action in meaningful ways and found the strategies to be both simple and sustainable and they found ways to feel accountable in a group that was supportive. They also showed a very high willingness to pay and they were inspired to use CGM.
(00:51:09):
So a lot of these you’ll see brought up in our product strategy coming in the coming weeks. But just want to thank Maz, David, Stacy, sissy and Nina for the huge help and work on this and it was really such a great team effort. And if you’d like to see more, you can go to the next slide, Josh. There’s a retro with tons more data and you’ll see more of this in the product strategy coming up. Thanks so much, everyone.
Josh (00:51:34):
Well, can’t really add anything to that. That was an awesome deep dive and what a cool experiment. I can’t wait to read the retro. Thank you, Casey. Thanks, product team. Awesome. And Stacy. Okay. We’ve got Cissy with demand capture update.
Cissy (00:51:53):
Amazing recap, Casey. Super excited about everything that’s coming out of the guides experiments. So I’m going to share a couple of quick highlights from our $99 try pricing experiment, which wrapped last Friday. Next slide. So for those who need a refresher, this was an offering that was open for three weeks. It was access to a one-time, 14-day sensor as well as access to the Levels app. And there was no commitment to the annual membership. So it was a one-time you try Levels for two weeks.
(00:52:27):
This had originally been scoped as gifting, which was a $100 gift card that could be redeemed for the same kit, one CGM and access to Levels. The ultimate thing we were looking to learn here is does a softer entry point into Levels $99 generate more leads and conversions that we’re not seeing at the 398 price point, which is our one kit with two CGMs and the annual membership.
(00:52:56):
So in other words, do we see higher Levels of people converting to the annual membership after they use Try It versus directly signing up for the annual membership? Next slide. Digging a bit more into this experiment here on the left-hand side, you can see the initial email that we sent out for this campaign. Pretty simple. It went out to about 2,600 people who were non-members. So all of these folks had signed up for the mailing list within the last six months.
(00:53:26):
And the engineering requirements here were primarily a dedicated signup flow. We wanted to ensure that Try It was outside of IRBS and not muddle that data. And then we needed to make sure that we could ship just one CGM with no commitment to the annual membership. And from the op side, we needed to work with True Pill to ensure that they could replicate our CGM replacement flow for Try It orders. Next slide.
(00:53:59):
For targets and results. So how did this all turn out? On the left-hand side you’ll see a little chart that has targets and actuals for orders and membership conversion. So we ultimately benchmarked conversion for orders to around 1%, given that two weeks prior, the team had run a Black Friday campaign at a much larger scale and that drove 0.5% conversion. So we targeted around 25 orders coming in and what we actually saw was 3.5 times that.
(00:54:32):
So, you can see 90 orders on that top right-hand chart is the total amount of purchases that came in from this. And you’ll see actually that huge spike at the very end or right around January 7th is when the reminder email came out. And so we actually hit our initial order target in just that one day with 23 orders. Now on the second piece, membership conversion. So this is the key metric we’re tracking. So this all remains to be seen because we only have a couple of folks who have gone through the full 14 days.
(00:55:09):
And so after they go through the first 14 days, are they converting to full membership? So we targeted around 30%. So we assumed of the 25 people who buy A Levels kit, eight of them would convert to a full membership. As of yesterday, only 50% of the orders have been activated. So we have around 45 people who’ve started Try It. And you can see on this bottom right-hand chart that there’s only five people who’ve actually made it to the end of Try It and are actually eligible to even join Levels as an annual member.
(00:55:46):
And so two of the five people who have finished the program have converted so far, but we’re tracking this metric closely because it’ll ultimately tell us if the unit economics on this type of offering will be viable over the long term, which is the whole point of this experiment. Next slide. So to recap this, we had three big learnings come out of this, a Levels learning, an experiment learning, and a pricing learning. The first learning from A Levels perspective is getting crystal clear around what you’re actually looking to learn.
(00:56:19):
So early on, we had approached this as a gifting experiment, but realized that there were a lot of areas of gray that we weren’t solving. For example, if it’s a gifting experiment, that means we need to rely on people forwarding the gift to their members or to their family and friends. What if their family and friends don’t open that email? So there was a lot of things that we couldn’t control for. So what was the most scoped-down version of this? And so what we ended up doing was deciding to pivot to Try It, which ultimately allowed us to build the rails to streamline this type of pricing experiment versus building a gifting functionality that we were only really planning to use once a year.
(00:57:01):
So that’s the first learning. The second learning was experiment learning. You could see from that chart in the last slide that our day of reminder email drove 25% of orders and we hit our target on that one day. And so that just shows how important communication with members is when you’re reminding them, “Hey, this is the last day to do something.” A lot of people will wait until that last day to do it. And so if we can send reminder emails, that drives a huge amount of conversion and people taking action.
(00:57:32):
And the final learning is pricing experiment learning, which was we saw clear demand for this one-time experience at $99. So we saw 3.5% conversion versus 0.5% conversion for Black Friday. But remember, this takes into account that this was just a one-time thing, whereas for Black Friday you’re committing to an annual membership.
(00:57:53):
And so again, the more important metric that we’re looking to track here is how many people of this cohort will actually convert to full membership, which remains to be seen at this point. And then the next steps in areas of exploration. The big thing to underscore here is the pricing experiment was run in the context of our current product offering. So we’re focused on selling CGMs in this current product offering. So it doesn’t really factor in the fact that we’re going to be taking the software approach. So there’s a couple of things that impact that when we think about software approach.
(00:58:31):
At the more tactical level, we’re focusing on understanding what the conversion from Try It to full membership is. And if we find that to be viable, we’ll start exploring whether or not having collaborations with some of our partners makes sense, so potentially having them offer Try It to their audiences. And then more strategically, the two areas that require more of these pricing experiments are annual membership. So testing appetite for monthly versus annual, what does that look like, and then CGM.
(00:59:03):
So software first versus CGM, reliant. We’ll need to define this and what the pricing experiment looks like as we continue to flush out our product strategy discussions. And then just to wrap, I wanted to say huge thanks to Galit and Raphael and Hao for enabling Try It from an engineering perspective, Chris for helping wrangle True Pill to ensure that the one kit CGMs could happen and then Taylor for serving as a liaison from the support team. That’s it for Try It. If you have any questions, feel free to reach out.
Josh (00:59:36):
Awesome. Thank you, Cissy. Looking forward to that membership conversion data once that gets here. Really awesome experiments today. I’m just going to quickly jump through to wrap up because we’re at time, but this was an awesome meeting. Subhiksha starts January 20th down in Austin. Looking forward to that. We’ve got an RD engineering position open and that’s all we’ve got posted right now. Otherwise, if you or someone you know is interested, check out levels.link/careers. And that is it for this week. Huge meeting. Thanks, team. I don’t know that we’re doing… I don’t think we’re doing cafes after this one. So with that, have an awesome weekend. Thank you, everybody. This was great. Looking forward to next week.