Josh (00:00):
Everybody, welcome to the first Formal Friday with Josh in a collared shirt. Also, the middle Friday of May 2023. Reminder on Friday Forum, this is weekly sync time, so we’re just synchronizing across what is a remote, async team about recent achievements on the different functions, hearing directly from members and partners, and catching up on culture, that sort of thing. This is not our deep business analytics meeting or our primary social connection channel. So just always reminding on that. And on the achievements this week. So, all right, let’s dive in here.
(00:34):
So the main thing is preparations for soft GA, which you’ll notice we’ve changed the language from beta 3 to soft GA for the Digital First program. So this is a lot of work on engineering, on app performance, the program mechanisms that hybridize and allow people to flex in and out of CGM. There’s a lot of work to do there primarily on the tech side. Then continuous onboarding multiple programs, hybridizing more screens, and then adding features like community feed and much improved AI logging.
(01:04):
So a lot of work that has happened. You may see the new logging function that shipped I believe this morning, which features some AI recommendations on foods like sandwiches. It can boost opportunities for adding more ingredients to improve the quality of the logging. So a lot of work there. Definitely need to continue iterating, and our team has to continue providing feedback before we ship these to our members.
(01:28):
We’re also synthesizing lessons from our alpha, beta, smoke tests into the plan towards soft GA. So there’s just a lot of work going on here, and you’ll notice the plan shifting in real time. And so, just bear with us as we’re doing so.
(01:41):
Important findings are that we need to probably add guidance on top of video only. So I think Maz is going to give us some more updates on this, more specifics. We also have found that of the members that are landing on our homepage, or the potential members, there’s a huge demand for CGM still. So the vast majority of people that are coming to our website are still coming to our website to try to find a way to get a CGM. Of course, costs, we know, is a major driver in being able to offer that to them in an accessible way. So navigating positioning and pricing is an ongoing experiment, and our next smoke tests are going to continue to do so.
(02:16):
General availability activation plan is ready. So big effort from Ben and the growth team for awareness, consideration, conversion plan there, all shipped and ready to start executing on. So focusing on emails, partnerships, owned media, and then performance marketing. So you can read the memo that was shipped on that.
(02:35):
Then our support team has done a great job. You’ll know last week we were highlighting just how hard they were getting hit with the queue and keeping up with our member questions. They’ve gotten the queue back to normal, so there’s just a colossal effort that is always ongoing. So shout out to them.
(02:51):
Then the contact rate has been the lowest in months. This is a good thing. Our replacement rate is below 8% for going on, I think, 12 weeks now. So all good stuff on the support side.
(03:00):
Our conversion rates on the homepage. So we’ve been running smoke tests which give people an A/B experience. So you may actually land on our homepage, want to buy Levels, and currently not have an option to because you land in a smoke test. That’s deliberate. Well, we’ve wrapped our smoke test one, but we know that over the past few weeks this has been ongoing. So although conversion rates are up, overall numbers are down due to the fact that we’ve been running these smoke tests. We’ve got good results on smoke test number one, and by good results I mean important results which Tom handed off to the team this week. We’ll be influencing our smoke test number two, which is going live on Tuesday.
(03:34):
We are trying to position with that smoke test better position our digital first offering. So rather than just dropping it on people who may not be familiar with that as an option in the checkout flow, we’re going to do a different job like basically build an entire landing page to educate and inform what that offering really is. Then the goal there is to finalize pricing and packaging for our packages for the soft general availability. Again, June 5th is the target there.
(03:58):
Dr. Chatterjee, Rangan Chatterjee, offered a newsletter note about Levels which led to a 10% increase in UK waitlist signups, which is a huge boost. I think over a thousand, we’re now over 10,000 on the UK waitlist. Tons of guides and R&D video produced this week. You can see screenshots of this. This is all in comms and, of course, in our app for our members. So the content machine is just running full force.
(04:22):
Then Rob and Rick Johnson, Rob’s doing a couple episodes that are upcoming. But Rob and Rick Johnson talked about Alzheimer’s and uric acid metabolism and fructose this week on a whole new level. Really, really exciting to see more of these expert interviews hosted by our very own Rob. So we really love that.
(04:39):
We’ve got lots of protein content. So you can see actually Kelly, who’s with us this morning, Kelly’s protein here has surfaced. It’s an amazing product. But we’ve just been talking about protein on the blog a lot more since we are so heavy on a protein-based diet.
(04:53):
The blog has been updated with a new top toolbar to allow people to jump into the Levels product flow and checkout. This has not typically been connected in the past, so this is something that we’re testing to see how people find their way from blog articles over to the product pages.
(05:09):
Then, finally, we’ve got big memos on market analysis and business model. So lots to dig into there. Huge effort. Big shout out to Lauren and Stadi and many others who have been working on filling out those memos and the strategy associated.
(05:24):
So you can see some of the smoke test positioning stuff here, screenshots from the Whole New Level episodes. But, yeah, Jesse and Ben did a Whole New Level on diabetes management and some love from the social channels.
(05:36):
Right. I think that’s the main stuff. With that, I want to re-welcome Kelly LeVeque, one of our very earliest and really best partners here for Levels over the years, an amazing channel of education that Kelly has … Well, multiple channels that Kelly has built that are really pushing metabolic health content for people who otherwise would not know how to take control of their diet and lifestyle. Kelly has really been amazing in surfacing blood sugar control as an important lever.
(06:03):
Love having you back on, Kelly. Would love to … Thanks for joining, first of all, in a very important time and would love to hear some of your updated thinking and excitement around metabolic health.
Kelly LeVeque (06:12):
Absolutely. Well, first and foremost, thank you for having me and thank you for letting me weasel my way into Levels. I’m a huge fan. I’ve been a nutritionist and author preaching blood sugar science for about a decade and never did I ever have a tool that I could link up with clients that allowed for them to really see the benefits.
(06:36):
The problem with having clients who have metabolic goals but mostly vanity goals is that they want to see the scale change. They want to see their measurements change. With Levels, I can show them metabolic changes that then result in the vanity and vanity changes that they’re looking for at the end of the day. Then I can speak to all the benefits that we all know about metabolic health and the benefits of preventing long, chronic lifestyle diseases.
(07:04):
And so, for me personally, not only has it been a great tool … I’m 37 weeks pregnant, actually. So this is my last day, my last day before maternity leave with my third … I’m leaving to go on maternity leave with my third son. I was joking before you guys all got here. I guess we should just check it because it feels like on brand, right? There we go. Still at 83. So when your blood sugar is 83 and you’ve already had your breakfast and your coffee and you’re sitting here and a little bit excited about presenting, you’re like, “Oh, it’s pretty good,” right?
(07:41):
I can’t thank you guys enough. Just being able to access this information for clients is what changes how they eat every single day, and that makes a big impact for me. So when I can’t be there 24/7 texting clients back, I really can help so many more people with their blood sugar control. I’ve talked for years about how the combination of protein, fat, fiber, and greens together supports the balance of blood sugar and really creates an elongated blood sugar curve versus a high spike and a hard crash.
(08:15):
To see it time and time again and see whether it’s someone in my community who’s drinking one of my smoothies or preparing a Fab Four meal, post it on social media and get a score of an 8, 9, or a 10, that’s just validation for me that what I’m talking about, it doesn’t have to be rocket science for a lot of people. They’re just looking to serve themselves and their families healthy, balanced meals, and to have a short checklist and to see that you can replicate that in your own kitchen and see it show up on your glucometer in a healthy way is really, really, really rewarding. So I’m super excited about that.
(08:52):
I’m also … In talking to your team, your tech team in the updates that could potentially happen of getting teams together, we talked about being able to do Levels with your friends. I know that there … Think about Apple Rings where people are sharing how many steps they’re taking and whether they’re sharing their MyFitnessPal. There’s a lot of communities that can be created around glucose, and it really does motivate people to change.
(09:23):
And so, for me, whether it’s my clients sharing their Levels results with me or sharing them on social media with me, it’s really rewarding to see that. But I’m excited about the future of Levels and how they can create communities of people that really are focused on metabolic health, because we all know it’s definitely the number one marker in preventing these diseases that are crippling our healthcare system and creating disease that is preventable.
(09:54):
So I would say my most recent example would be Halle Berry. I don’t know much … If you guys know, but she was diagnosed with type two diabetes at a young age. She was in her 30s. She luckily found a holistic physician who told her that she didn’t need to take medication, she could prevent this with lifestyle, and has been a huge advocate of balancing her blood sugar and talking to her community about that. So you might not know that, but that has a lot to do with her physique, the way that she eats and takes care of herself. And so, it motivates her community to look at their blood sugar balance and balancing their meals in the same exact way.
(10:39):
So I think we just need to keep pounding the drum, and Levels is such a tool that is priceless, in my experience, for my community. It’s been the best tool that I’ve been able to give any of my clients in the past decade. So thank you so much for all that you guys do and how you continue to iterate the app and make it more user-friendly and all of the tips that you guys are giving the users. It really makes a difference.
(11:05):
I saw in the chat, “What’s a sandwich?” That was funny, but it is true. It’s focusing on that protein first, fiber, fats, leafy greens, whole foods. It takes sometimes for people to biohack or to see their own numbers to make change, and you guys are making it happen. So thank you for all that you do and continuing to make my job extra easy.
Josh (11:30):
Love it. Yeah, you give such an awesome narrative on how this can be a tool that not only is useful just at the individual level, which can be really overwhelming for certain people who are just not quite familiar with the material, but that can reinforce the techniques and tools and education that professionals like yourself have been putting out there for a long time, but that doesn’t necessarily just resonate until people see it for themselves.
(11:55):
So, Kelly, you touched on some of this stuff along the way, but you’ve been using and following and distributing Levels for a long time to your community. I would love to just hear what’s the number one thing that you recommend Levels fix, improve, build from scratch going forward that you … Whether you see us doing it now or don’t, we’d love to just hear what should we do differently?
Kelly LeVeque (12:15):
No, I think you guys have continued to iterate … I mean you’re ahead of my recommendations where I would see someone have a spike and a crash that’s potentially not food related. Maybe it’s an infrared sauna, maybe it’s a really hard workout, maybe it’s caffeine. Just notes. I guess it’s when you allow for push notifications like, “Hey, just letting you know your blood sugar is rising.” I got one of those the other day and I was like, “Oh, this is great,” to get it ahead of time instead of having to jump into the app for those is really amazing.
(12:52):
There’s so much room for education. I think people can feel overwhelmed and put on Levels and not understand what they’re doing wrong. I had a client. She is the CEO of Full Picture, which is a very well … She’s a very well-connected female in Los Angeles. She had always thought that quinoa was a protein and healthy, and she was making scrambles with veggies and quinoa. She’s like, “I don’t understand what the problem is. I’m adding all this protein.”
(13:24):
And so, the only thing that is a little bit difficult for clients sometimes is when they have an idea that a food is a specific thing. I realize that people say that quinoa is the highest in protein of grains, but let’s be honest, it’s a major glucose driver. I had to explain to her it’s actually mostly carbohydrates. We jumped into the Cronometer app. Depending on a client, they might use MyFitnessPal or Cronometer, and I always have them double-check there.
(13:54):
So I think when I think about the iterations of the future is being able to differentiate between the ingredients that someone’s uploading as their meal and saying, “This serving size of quinoa is delivering a certain number of carbohydrates and a certain amount of fiber,” which may be the driver of your glucose.
(14:16):
Just getting people those insights even deeper than you are right now. But I mean I’m still so impressed with Levels, so please don’t take this as anything but major praise. But little things like that have come up along the way where she was getting frustrated and had used it for a certain number of days and was … As we all know, if you’re starting your day with a high carbohydrate-based meal, even if it’s sometimes a whole grain, it can really just regulate blood sugar the rest of the day. She’s seeing her crash, she’s having cravings for more sugar, she’s asking for a piece of fruit, she’s trying to get herself back up to a place of feeling really energetic.
(14:57):
That’s just an example of someone who is very well-educated and has all of the support, whether it’s a functional MD and a nutritionist and a trainer. But I think maybe there was an article, or a friend who said quinoa is high in protein and really nutrient-dense and she went, “Great, I’ll start my day with it.” So that happens to people and they can be totally derailed and not know why.
(15:23):
So just that information becomes priceless for clients. And so, I think, as you continue to iterate, being able to look at a meal and say, “This is probably the driver of your glucose,” would be interesting.
Josh (15:38):
Yeah, definitely. Yeah, it’s a really important reminder and I think the key to being able to surface those insights is better logging. So as we continue to focus on making logging low overhead and effortless, but also more detailed without necessarily requiring minutes and minutes of logging and measuring and estimating. I think we ultimately can get something very similar to that, especially with some very new and very fast-moving AI tools that are coming to life.
(16:03):
So it’s a great reminder. Also very good examples as usual, Kelly. So thanks again for being such an awesome partner and spending your Friday morning with us, especially as you head into maternity leave. I’m sure life is extremely chaotic and busy. We just really appreciate everything that you’ve done to work with us over the past few years and all the best. If you want to hang out for the rest of the meeting, please do. But, of course, we understand if you have to jump. The whole team really appreciates hearing directly from you.
Kelly LeVeque (16:28):
Hey, I really appreciate being here, you guys. Thanks for everything. I’m going to go actually get fetal monitoring right now, so I have to go. But I appreciate you all and thank you so much for all of your hard work in providing such an epic product. It really is a game changer.
Josh (16:45):
Thanks, Kelly. Good luck with your leave and can’t wait to catch up in a few months.
Kelly LeVeque (16:49):
Okay. Thanks, Josh.
Josh (16:50):
Bye-bye.
Kelly LeVeque (16:50):
Bye.
Josh (16:52):
All right, awesome. Those are always great. Over to culture and kudos. So quick reminder, first off, it’s review season and as much as we’ve got going on, we need to wrap up self and manager reviews. Got to get this whole process done. It’s multistep, so it’s not just about submitting that type of form. It’s also then managers have to synthesize, do a synchronous catch up, and then we do a calibration process. So try to get those reviews in this week. Nicole, if you have anything to add, please let me know. Otherwise, I’m going to jump in and give a big shout out to John.
(17:27):
John has taken on logging improvements and took complete ownership over the end-to-end. I think the most important things are the speed and productivity and also the de-scoping. We were able to pull in a phase two scope into this most recent release thanks to John’s relentless work on taking smaller bites, pushing out things that we can rationalize pushing out, and shipping an experience that we can test internally even more quickly and validate our assumptions. It’s super important and just love to see it. So thank you, John. Then I think … What’s going on here? I need to refresh here. Yes. Okay, there we go.
(18:04):
Then, finally, today is Charu’s last day here at Levels. So I just want to thank Charu for all the work she’s done on the data science and the backend engineering over the past … It’s got to be over a year now. I just genuinely appreciate the work and also the understanding as the team has evolved and as our understanding of our data science needs have evolved, just being open to having, first of all, adaptability to the role and the needs of the team in real time.
(18:29):
Then, finally, just as you go on to the next phase, I would love to just keep in touch and just genuinely appreciate you contributing to getting Levels to where we are today. It’s been really important and, of course, everyone’s sending off a warm best of luck to Charu in your next journey. So please reach out to Charu as she departs and grab the contact info and stay in touch. All right, I believe this is Jackie.
Jackie (18:56):
Yeah, that’s me. Okay, so closing the loop. So I have a recent personal story for this one. I connected a friend of mine to someone I thought could help them from a work perspective. So I sent a note to connect them and then was moved to BCC and then never heard anything. I found myself wondering next week like, “Oh, I wonder if they ever connected.”
(19:19):
Then I was like, “Oh, well, maybe it was a really not good interaction or something that she didn’t want to tell me.” Then I was like, “Or maybe it was really positive.” Then I was like, “Well, wait. Why wouldn’t you want to tell me that?” So I was like, okay, I’m spending way too much time thinking about this at this point. So I was like, “They’ll just let me know how it went eventually.” Then they never did.
(19:39):
So I followed up to see how it went. I’m sure that we all have a bunch of examples in life like that where you helped someone with something, and then you got invested in it and you wanted to know the outcome. It got me thinking of the other side of the coin in terms of all the interactions we have daily where we say we’ll do something or we’ll look into something, or we connect someone with someone else, and we do that thing and we know it’s top of mind for us and that we’re working on it.
(20:06):
But I think I don’t always think about the importance of closing the loop with others involved and the cognitive load that they might be bearing thinking about that thing that we told them that we would do, or the outcome of the connection they made for us, or whatever it is.
(20:21):
So my goal is always to avoid anyone ever spending time thinking about, “I wonder if Jackie ever did that thing that she said she would do or how it went.” I’m not perfect at this by any means. It’s definitely something I’ve been working on and will continue to work on. But I’ve been thinking a lot about how much cognitive load that we can all save each other by overcommunicating generally, especially in this remote, async world that we’re living in, working in. Overcommunicating for me definitely feels uncomfortable at first. In practice for me, this looks like just setting a bunch of reminders and sending messages without expecting a response, which feels weird.
(20:57):
But I’ve sent a ton of messages on top of each other in some cases, like five messages saying, “Don’t respond to this,” “No need to respond to this,” which feels weird. But I’ve learned from watching people like Ben and Tom who do this really well, just sending a quick note that says, “No response needed, FYI,” to let people know you’re thinking about something, just that they don’t have to.
(21:21):
Yeah, so it feels weird at first, but to send a message and not expect to receive a reply. But I found it’s a great way to build trust with the team and also to hold yourself accountable, too.
Josh (21:34):
This is such an important one. It’s hard to overemphasize in a remote, async environment. People cannot see and naturally organically absorb what’s happening, and maintaining that confidence level is super important and just hearing exactly what you’re saying. That’s basically a much more efficient synthesis of the confidence is earned memo. So I should have just asked you to summarize. It would’ve been a lot shorter. But at the end of the day, that’s really what it comes down to is just volunteering information. So it’s easily discoverable for people who depend on you and they don’t have to go searching for it, which can be very inefficient.
(22:09):
So thanks, Jackie, and thank you for being an awesome example of this constantly. I do see those updates very frequently. I think that one point of saying no need to respond is such an important one, because if someone reads a message, they may feel, “Oh, I have to respond to this.” But by simply opting them out of needing to, it just lowers that burden even more. Okay. Oops, what’s going on here?
(22:31):
All right, main thing. Levels shows you how food affects your health. We’re doing a lot more than this obviously, so I’m just going to put that out there. But that is the main thing right now. I’m going to let Sam take the data number slide.
Sam (22:43):
Cool. So we’ve had some movement around beta 3. I think a lot of you are aware that we’re shifting more towards a soft general availability. It should be a percentage rollout. So some of these numbers are going to be changing in how we keep track of this within the next couple of weeks. But our engagement numbers right now are quite high, but they do tend to decline over time. And so, we’ll hopefully have better ways of measuring and demonstrating that as well. So no specific actions here, just hopefully as we get closer to soft general availability, we’ll be able to get better data and update the slide.
Josh (23:28):
All right. With that, handing over to Maz.
Maz (23:36):
All right, thank you and welcome to product update, May 12th. Next slide, please. All right, so big thank you to the team. I think everybody’s been executing, and execution is like a fun hike. You’re walking down the trail and you’re enjoying it. But once in a while you pick up your head in the forest and say, “Well, where am I?” You ask yourself, “What have I learned or what have I seen and where do I want to go?” Next slide, please.
(24:06):
So I think it’s that moment that we have been executing since March, and now it’s time to rewind and say where are we, what have we learned, and where do we want to go next? So we’re going to talk about that today. Next slide, please.
(24:23):
Okay, rewinding back to the strategy. What do we want to do? Ultimately we wanted to build a product that people love, buy, and use for a long time. We want to do that by helping them improve their health and reach their health goals.
(24:34):
The three pillars that we set out to do, as you guys all know by now, is guidance, the what, why, and how to let people really take the actions that improve their health. The second pillar was action, being able to actually track the actions and see the progress. The last one was accountability, accountability to themselves and being rewarded for the actions they take. All of it together really helping members improve their health. We want to talk about what have we invested in and where are we now and where are we going next around these three pillars of product. Next slide, please.
(25:10):
All right. So as you guys all know, we started with two offers, and then we set up three betas. At each step, we had a set of constraints. I think constraint number one is we wanted to move really fast so we really optimized for speed and for learning.
(25:25):
Second, we wanted to de-risk the guidance, because guidance was a new experience that we didn’t have before, and it was probably the biggest unknown. So we spent a lot of time upfront to try to de-risk guidance, knowing that it’s not going to be the only piece we invest in, but really a thing that we need to de-risk.
(25:41):
Last one is we wanted to stay focused, so really focused on Maureen, even though we knew we’re going to get some non-Maureens in our cohort. So if you look at that, in alpha 1 and 2, we pulled in schedules to try our programs earlier. In beta 1 and beta 2, we’ve been executing … Beta 1 was really focused on creating a guidance experience without the CGM, but we knew that wasn’t going to be the end goal.
(26:05):
Beta 2 was introducing the option to get a CGM. Really, in beta 2, we decided to pull in beta 3 and do general availability first. In that experience, what we really want to do is combine the guidance focused on CGM and add accountability as a layer, really bringing full circle the product strategy and start investing in really integrating the CGM ball and building the accountability layer next. Next slide.
(26:38):
All right. So along the way, we learned a bunch of stuff. Number one, which is probably the most painful one, is whatever you do, you have to reduce friction, whether it’s converting people, whether it’s allowing people to renew, whether it’s the data that you have to actually analyze your results. I think a big learning for us is if you want to make decisions based on numbers, it’s worthwhile investing in getting those numbers in a way that’s less confounded. I think we found a lot of confounders, for example, for retention, and investing in that upfront would’ve probably paid off in being able to see those numbers a lot clearly. It’s a trade-off we made, but it’s definitely a learning experience for us to maybe invest in areas that will help you get clear data.
(27:22):
Effectively what we did was we used, for example, type form to get people in, because that’s the easiest way to convert. But what that led to was it made it very difficult for people, for example, to renew. So that’s an example of friction, that if we have thought about it a lot more and anticipated downstream impacts, we would probably have had less compounded numbers. But, again, hindsight is 20/20 and I think the big lesson for us is really reduce the friction across the board, both for ourselves and for members. Next slide please.
(28:00):
The next lesson that we learned is that in order to provide effective guidance, video is really important, but it really needs to be paired with a lot more in different formats. People really learn from video and from other human beings. But also having a more cohesive experience for providing guidance is really valuable. People, for example, really want to know where they are and where they’re trying to go and what steps they need to take next. That really needs to be provided by software versus video.
(28:30):
So in the next experience, for example Community v1, really investing in bringing a lot richer ways to be able to provide guidance to people and provide a path for them to follow. So we’re going to be investing in that in the next iteration. Next slide, please.
(28:50):
All right, the next one is one of the things we saw both from all the alphas, betas, and also the smoke test is our current audience, the people that come to us, really value a CGM experience. And so, the learning for us is the entry point into the experience is probably for our current members, the most valuable is by using the CGM. So we’re really designing our programs and experience to be centered around CGM entry and be able to deliver a product that meets the cost target, which is most people want to checkout, or get out or checkout, for less than $100 and be able to buy the CGM first. We’re going to invest in that in our program, in our product offering in the next iteration. Next slide, please.
(29:37):
We also learned that personalized guidance is really important and it really matters. It needs to be personalized because people’s skill levels are different, and also their lifestyle and constraints are quite difficult. So really providing a guidance that’s specific to them so they know what to do and how to do it, but also is flexible to fit in their lifestyles and constraints. It’s accessible meaning it really meets them where they are from a skill level, and they can do it and feel successful and learn to go through the journey.
(30:08):
The next one is really thinking about what type of coaches do we actually want to experience? The three pillars that are really important is, first of all, the coach has to be engaging. We have seen a lot of great feedback that our coaches today, Sonja, Lynette, and Mike are really engaging. People love actually listening to them and watching their content.
(30:28):
The second thing is they need to be credible in the area that they’re talking about. For example, we’ve heard a lot from our members that when people talk about something, it’s really important that they feel that they’ve had that life experience and they’ve mastered that. For example, if a person in menopause is interested in content around menopause, they would love to hear that from somebody that’s experienced that and has knowledge. And so, we’re seeing that and taking action on figuring out how do we actually provide credible guidance in areas that members care about and really round out our guides?
(31:05):
The last one is uphold Levels’ value. We want to be proud of the people that really talk and provide guidance on behalf of Levels. And, so really investing in that. I think we’re doing really well there. It’s definitely an important aspect as we diversify into other guides and potentially external and guest guides.
(31:20):
Then, finally, while we wanted to really focus on Maureen to start because we wanted to focus our resources, the reality is our channels when we go to channel availability are going to be 45% male. And so, we really need to create an MVP experience that works for them when we don’t turn all of our members that are male. If you want to hit that 60% monthly retention or month one retention and 45% of our members are male, it’s really important that we think about that and be able to provide an experience that doesn’t turn them out.
(31:50):
Then the other piece that we’ve heard consistently is that because our members are also older, have content that’s relevant to them. So the two pillars that we’re going to invest in for soft GA in terms of content is really create some experience for men and also focus on creating content for older cohorts. Next slide, please.
(32:16):
So this is probably the favorite of support. Because we wanted to build iterative skateboards, we’ve taken on a lot of jank. So we know that because we’re moving fast, a lot of experiences aren’t refined. So we’re going to actually … When we go to general availability, we focus on the things that we’ve heard the most from members. For example, performance is a big issue. As we create experiences, first we want to see whether they work and then we have to optimize them or focus on optimizing.
(32:47):
We’re at a point that we need to create some performance improvements and we’re investing in that. We hear our members and support line clear. So thank you, Taylor and Chris. We are working and allocating engineers to work on performance issues.
(33:01):
There’s also stuff that we built to iterate and now we want to make sure that we don’t build a bloated app that has a lot of services. So we’re going to invest in actually reducing some of these services and create a more cohesive experience, and remove some of the pain points.
(33:17):
Then, finally, one of the other things we’ve heard is people love the content but they can’t find it, and that’s just a matter of where we are. And so, we’re going to invest in being able to make sure people can find the content, fund their data, and see it in one place. So that’s another learning that we’ve heard consistently from members. Next slide, please.
(33:36):
All right, and the last one, one of the things that we’ve heard consistently, that human connection matters. We’ve heard a lot of really fantastic feedback from people that they feel like their coaches are their friends. There was a quote on the bottom right which I personally heard from one of the members in our community which said, “Lynette and Sonja feel like friends, but you don’t leave your friends for other apps.”
(34:02):
We’ve consistently heard this from members where even though it’s a one-to-many relationship, they actually feel like Lynette, Sonja, Mike are their friends, and this is really powerful. It’s powerful because the experience is engaging and it’s powerful because it’s actually retented, so that’s really positive. There’s tons and tons of good feedback that we’ve gotten, and Adra’s created something called The Wall of Love, which has a lot of quotes. It was really interesting to actually see what members care about and resonate with and really double down and invest in those.
(34:29):
We saw in beta 1, even though beta 1 was really focused on guidance and it wasn’t the full experience, we saw that for the people that retain, which was about … For Sonja’s core, it was 45% of people were still engaged with the app at the end of four weeks, and the engagement was about three minutes. And so, that was basically up from zero minutes.
(34:45):
So I think it’s a good trajectory. Obviously it’s not enough because we have to keep people, and the jury’s out on that. We’re investing a lot in accountability to move forward. But I think the trajectory of the first beta is positive, but we still have a lot of work to do. We’re not out of the woods. Next slide, please.
(35:02):
All right, so what are we going to focus on next? Where are we going on that compass in the forest? We’re going to focus on two things, retention and conversion obviously, because those are our two business objectives. Next slide, please.
(35:16):
All right, so conversion, we really … Both for conversion and retention, we want to solve this equation, which is perceived value up, cost and effort down. Everything we do in product is trying to optimize for that. So I think, first and foremost, we want the product to support the growth team because it is going to have to be a collaborative effort to make sure that everything ticks and ties and is a cohesive approach. So we are here to support growth.
(35:38):
As we move into the growth phase, second is … One of the things we did is pull GA so we can actually get higher quality data for growth. Then last one is trying to create this experience or this journey where people can check out with a $100 price and have a CGM board experience out of the gate. So these are the three priorities that the product is going to do to help with conversion, even though conversion is primarily a growth team’s effort and initiative. Next slide, please.
(36:08):
And retention. This is primarily the product, Orum’s responsibility. Here, again, we’re optimizing for perceived value, cost, and effort. And so, the areas that we’re going to invest in is, number one, we’re going to invest in accountability through this idea of goal setting, tracking, and really making sure that people stay accountable and do the actions, and those actions lead to their improved health, because ultimately I think what’s going to retain people over the long-term on the health product is if they see improvement and getting closer to their goals.
(36:38):
Then second one, we’re going to continue on the community feed, which is really both going to help with accountability and guidance, accountability from the community and the guide, and guidance obviously clearer and more ways to actually communicate to members in different types of content formats.
(36:54):
We’re going to invest in content, continue to invest in content, really develop more for the older demographic, as we talked about, and for men and make the content more refined. We’re going to invest in UX improvements, really make the stuff that’s resonating with people better and deprecate the stuff that’s not working.
(37:11):
We’re going to invest a lot more in data. I think one of the key things about our approach and our company culture and values is this iterative scaper approach. I think that this next phase is really important. So we’re working with engineering and across the board, signing DRIs to be able to learn from the data, be able to collect the data, analyze the data, and learn from the data. Then last but not least, really being proactive on non-engineering fronts to make sure that our members stay retained and continue with the product so we can help improve their health and their lives. I think that’s it. Thanks very much.
Josh (37:45):
Thanks a lot, Maz. I appreciate the deep dive on the learnings and next steps. I’m going to hand it right off to you, Chris Jones.
Chris Jones (37:55):
Thanks, Josh. Covering operations with a little bit of voice of the member sprinkled in. So going to be a little bit of a two-for-one. Next slide, please. First off, I know we’ve been talking about this for a while. Just want to officially welcome some of our new team members from Autonomy. They’re starting this week, so two weeks of onboarding training.
(38:18):
Barbara, who’s going to be the team lead, is going to be joining us in a couple of weeks. So we’re going from a team of two to six. So we’re going to have a lot more horsepower on Autonomy as this team gets ramped up. A lot of that in terms of us helping to get some of our SLA numbers up also frees up the existing team. I know that with the beta program, there’s been a lot of asks of support and we have a very talented team that can do a lot of areas besides just provide excellent support. So being able to free that up for product or for content or for videos is super important why we’re bringing Autonomy on. So excited for this ramp up. Next slide.
(38:59):
All right. A little bit of feedback on the beta and beta 2. So what I looked at is I tried looking at multiple inputs. So imagine type form, Health Scout, surveys, Facebook. So I looked across five or six different sources trying to figure out what feedback are we getting across different services? A lot of this is very much in line with what Maz already covered in terms of the feedback we’re getting around the guides, the video, the program, people loving … Feeling like Sonya is their dear friend.
(39:33):
The comment that really jumped out to me was, one, that upper right of my Levels with this program versus without are unreal. It talks around in terms of it’s just really helping people that are getting stuck. Some people start Levels, they figure it out, “Just give me the data and leave me alone.” But there’s a lot of people that need a nudge, a guidance of, “Show me how to do it.”
(39:56):
So us, I think to Maz’s point of having multiple ways that people learn, of some people are like, “I want to watch a video. I want to get a recipe. Just give me access to the data,” and us meeting people where they’re at is really resonating with people. Also, the foods and recipes. Don’t worry, Mike, I even saw a couple shoutouts for the exercises. So your love wasn’t lost. Next slide.
(40:23):
But feeding on exactly what Maz has already covered, there is a lot of negative feedback, and it’s not the … Really, I don’t get it, but because the app is doing so much more, loading videos, loading content, getting a lot more data from Apple Health. It’s just things are timing out which causes people going, “Hey, how come my exercise isn’t showing up? How come this food isn’t getting? How come I’m not getting credit?”
(40:53):
The comment that really resonated to me is that it used to be so easy. So people that used to use Levels have an expectation in the terms of how it performed and would work, and now as we’ve been adding a lot to it, it’s just not to the same level that they expect.
(41:08):
One of the comments from Maz’s community call, I think someone said, “Performance is going to be the number one reason why I’m going to stop using Levels in terms of if I try to do something, I’m in there, I’m trying to log, and it’s just not working. My patience is eventually going to run out.”
(41:25):
Now I know there’s are a lot of work that the team’s planning to do around this, which is great. I also want to call out Marillo and John for work that’s been done to date. I know that they’re included on a lot of tickets from people like me and Taylor, and they are taking lots of shots of goals to improve it to … So there’s been a lot of work done already and I want to acknowledge and thank the team around the work done, and we have a lot more to go.
(41:49):
There’s a couple of people in terms of the price value, and this resonates towards the CGM or the software only. Without the CGM, I think people are struggling more around is there enough value to justify the price point? I think that’s definitely something we’re seeing where the CGM is an essential part even if it’s not all the time. Next slide, please.
(42:13):
And then the novel. So those of us who remember Vero who came to the Friday Forum, I was just blown away around this piece of feedback. The reason is … Why I wanted to call this out is this is the level of engagement. Some of our members who are cheering for us and want us to be successful, they are taking so much time to think about what’s going well, what’s in it for them, how to improve. It’s not just like, “I like your app,” or, “I don’t like your app.” They are want to be part of the conversation because they know how important this is and it’s this level of people giving us feedback and helping us because they want us to succeed. So this really just blows me away when we see this people being this verbose and trying to help us in our journey. Next slide.
(43:05):
Now to get to the data part of it, because it’s not going to be a Chris update without some Snowflake dashboard. There’s been … In one of Maz’s slides that talked about doing more cohort analysis. So the top three graphs, what you’re looking at, and I have a link to it, this is a new dashboard I’ve been working on this week, engagement by starting cohort. So I’m looking at members that start in a given month, and then I follow day-by-day what percentage of them are logging food, logging exercise, getting sleep, so you can start seeing the decay or even in some cases the increase over time of by day seven, what percentage of people are logging sleep? By 14? By day 21? It follows these people and I have different slides on it.
(43:51):
But I wanted to call out the red line, which is what I call beta 2+, meaning this isn’t just beta 2 members, it’s people that really started in that same month. So I’m looking at people that started last month and I’m tracking their engagement day-by-day.
(44:06):
What you can see is of people that started last month, a higher percentage of them are logging food, a higher percentage are logging exercise, a higher percentage are logging sleep, which is exactly the behaviors we want to see. So I’m going to be doing a longer update in data analytics in general to walk through this dashboard because I have different views and there’s also some caveats in the data, so more to come around this. But definitely going in the right direction. So this is great signals to see.
(44:35):
Then on the bottom of it was also … Which we talked about a little bit last week, is it also drove more support cases. So in that first week when we launched beta 2, because of some of the things we did, we saw an increase. On that Monday when we got buried, we had 100% more cases. Then it definitely, that first week, we were digging ourselves out for a while and we cleared through it. So things are pretty calm right now relatively, but that first week was rough. That’s the update I’ve got. So thanks.
Josh (45:06):
Awesome. Thanks a lot. These are always just so clear, concise, transparent, directly into what’s going on. Just really appreciate the presentation and some data to back it up. Thanks, Chris and support team.
(45:20):
All right, we’ve got a quick hiring update here. No change from last week. Backend software engineering role is open and live. So check out levels.link/careers. Forward this to someone who might be a great technical and culture fit, or if you’re interested in other potential opportunities, we have a general application there on the website.
(45:41):
Okay. We’ve got some time here. We burned through the main event, so I’m going to stop the share here and we can do some of that individual contributions. So just hit the … Let’s see, the reaction button, raise your hand. Definitely recommend jumping in here.
(45:59):
Yeah, I mean I think I’m just going to say on the professional front, I just continue to really appreciate that the transparency of the team as we’re learning things, some hard truths, some known knowns that we’re just further validating. There’s a lot of work going on, and it doesn’t always go exactly as we expect. How the team adapts and reacts and pulls in dates to move faster, to validate sooner is just a … That’s how we have to operate here. I just appreciate everyone on a team as large and as many functions involved as we have. It’s hard to do, but I just want to appreciate everyone for that.
(46:37):
Then on the personal side, I’m going to be traveling a bunch over the next month, bouncing back and forth to California, bouncing out to Philly for a wedding, some other stuff. I think I have a camping trip somewhere in there on a river somewhere, hopefully. So, yeah, I haven’t traveled much in the past few months, so I’m looking forward to breaking my rituals. Caitlin?
Caitlin (47:03):
Okay. So I have a personal, profesh moment to share from this morning. Waking up feeling pretty tired from life, and I was taking my baby to daycare and I saw my neighbor who also as a baby and we’re being like, “Baby, baby.” I was wearing my Levels hat and she was like, “Oh, have you been able to get Levels?” I was like, “I work for Levels.” She gave me a very like, “Oh, man. That’s so cool.” She actually said, “That’s epic,” and I was like, “All right.”
(47:36):
So personally that just made me feel cool. It’s always very cool to feel like you work at a cool place. Then professionally it makes me realize there’s still people in these past couple of years built up this interest and there’s still people that don’t realize they can get access and have been wanting access and now we’re going to have an even better product to have access to. So I was like, “June will be a great time for you to join. You should join up then. We should talk about it.” But, yeah, that was a cool moment and just wanted to share.
Josh (48:12):
Love that. Yeah, we had a point in time when Levels was very scarce, very hard to get. That moment has passed, so we should get her on the website, see whether she picks A or B on the landing page. All right, folks, we’ve got a quiet audience here. Chris?
Chris Jones (48:38):
First off, on the professional standpoint, obviously I’m super excited about beta and all the changes coming and getting into GA and really getting out there live to more people to see it. It’s super exciting. So just a huge hat tip to everyone involved, which pretty much is the entire company. It’s just great to see everyone working towards this and taking the feedback and iterating.
(48:56):
I also want to do a little bit of shout out to our leadership, people like Sam and Maz and Josh. When I see them doing community calls, it is so refreshing when … When you’re trying to advocate for members and you’re not just like, “Okay, I can just give you some comments,” but for people like Maz to hear from people’s mouths around what they like and what they love, it’s super powerful.
(49:26):
So I know it takes a lot of time to do those calls. I just can’t thank the leadership team enough for making time in their busy schedules to meet directly and hear from them, because that’s so much more powerful than anything that support can highlight up. So I just want to say thanks.
(49:43):
Then on a personal standpoint, my lawnmower is fixed. I’m super excited. It’s been rainy, now it’s sunny. So I’ve got a good six hours of lawn mowing in front of me this weekend. I’m actually looking forward to it and can’t wait. Farm life is in full effect.
Josh (50:01):
Beautiful. From my inside intel, it seems like the weather is really hitting its prime right about now. So enjoy. Maz?
Maz (50:11):
On the professional front, I don’t know, I think people know this. But to Chris’s point, just to reiterate this, support is a big driver for retention. If you guys watched the community call, two or three people out of the four people basically said one of the reasons they don’t want to change from Levels is because how fantastic support is.
(50:36):
I don’t think you ever hear that. I’ve never said that about any support at any company I’ve ever used or worked at, including Apple. And so, just big thanks to you guys. To top that off, they were like, “Wait a minute. Is Lynette the person in the video the same person at support?” I was, “Yeah, obviously,” and it just made me so proud that not only you guys are so fantastic at your work, but also so multitalented. It is just really phenomenal, strong, passionate, creative, and this is a great team.
(51:13):
So big thank you to support. You guys take things in strides. Also, another thing that I really appreciate from my seat, I know we cause all the pain for you guys, so I deeply apologize for all the beta jank and all the stuff that we have in the product. But you guys handle it so well and have been really, really fantastic partners. So just want to say a big thank you, Chris, team, Taylor, everybody, Lynette, everybody. So thanks. That’s on the professional front.
(51:38):
On the personal front, we are on the Island of Skye, which is what my daughter is named after. And so, we’re seeing it for the first time. This place is magical. So the weather sucks, but the natural beauty is just magical. So highly recommend if you guys are ever UK side.
Josh (51:55):
Wow. Enjoy. Yeah, big plus one, by the way, on the support. I think the multi-talented, multifaceted nature of that team, being so deep in the product to the extent that they’re like Lynette is a guide in the product, no support team goes to those lengths, and I think that is the explanation for why our support team is so capable of giving people an awesome experience, because you really do understand what the member is going through. So that’s something that … Yeah, it’s organic, but it’s really awesome. So I do want to boost that. Hui?
Hui (52:28):
Yes. So on work side, yeah, I’m just day-by-day, week-by-week, and continuously feeling proud of the team, feeling excited about the execution, and also appreciate the transparency, visibility into everything. Among all these, I especially want to call the smoke test results that … Seeing people talking about that, especially Tom and the leadership team, and even the numbers and not shedding great light to lead us out of the woods, there’s still mist in front of us, but just seeing people talking about that in a transparent and direct way. I think that’s a very encouraging and also a learning experience for myself. Yeah.
(53:12):
So I guess personal side, well, I guess Happy Mothers Day to every mom on the call and also to people who support moms around you. Yeah, thank you.
Josh (53:25):
Yeah, big shout out to the moms. All right. Anyone else? We got time for a couple more. Going once, twice. All right. Yeah. I also want to say I tried the Walk-In … This is just a sidebar, but I did the Walk-In Labs experience yesterday, and it was really easy.
(53:51):
So we have a lot of raw materials still that we have not really leveraged to the extent that we can, and the pieces of a really important health feedback loop or flywheel, which we haven’t really activated just yet. So there are a lot of really interesting projects that we’ve put a ton of effort in to get to a certain point and a certain quality level. I think I’m excited about tapping more of that and helping people understand why we’ve built those features and making them more accessible to people, like the walk-in experience, which cuts the price in half, is a really huge step forward. So we can iterate a bunch there. So, anyway, just wanted to say that was a good experience.
(54:33):
All right, unless anyone else wants to jump in here at the last minute. All right, you get three and a half minutes back of your Friday, of your Formal Friday. All right, have a great weekend. Good hanging out, and thanks for all the hard work.