Josh Clemente (00:00):
Let’s do it. Welcome to Friday Forum, August 19th, 2022. This week, action packed as always. Some really great news from the support team. We’re starting to get back on top of the SLAs and even quite close, I think 75% were within the three hour SLA this week. Inbox zero several times. Happiness is back up to 95% and the stuck orders in fulfillment queue for various reasons are down by 80%. So this is huge. Obviously with Rebecca and Lynette getting fully onboarded and into the queue, that’s been just a major bandwidth relief for the team. So shout out to the sport team for sticking it out and getting much closer. I think we’re going to be, if these trends hold, we’ll be back in the SLAs this coming week. And then lastly, NPS surveys are now going out to our streaming CGM members, so that is an upgrade and we’ll be getting obviously new data from the NPS experience on that side.
(01:01):
On content, we had a new ultimate guide published relating to the leading causes of death. I think this uses 2020 numbers because I believe in 2022, it’s more like nine of 10 leading causes of death are related to metabolic dysfunction or glucose dysregulation. But the ultimate guide takes a very rigorous dive into that and I highly recommend everyone read into it. We’re also reinvesting in improving older posts. Our blog has now been around for going on two and a half years, and some of these posts are still extremely relevant, but we do not have all the tools on the editorial side that we do now. So we’re reinvesting in improving those old posts, updating with poll quotes, relinking to new blog posts inside of them such that SEO improvement happens. And you can find your way through our blog even with those older posts, which is really great. Love seeing that.
(01:53):
Casey recorded round three on Doctor’s Pharmacy with Mark Hyman. We’re now trying out a bonus content offer. I think using Mark’s link you can unlock some bonus content between Casey and, oh, sorry, Casey’s hidden behind his blog cover there. But anyway, so you can unlock Casey and Mark special content. It’s going to be an interesting. I think the first bonus content offer we’ve done, so that’ll be an experiment. And then we have ads live on Tim Ferris’s show, Max Luca’s Art of or Genius Life, and then Art of Being Well with Dr. Will Cole. And then we locked in one of the two sponsorship slots on Peter Peter Diamandis’s new podcast, which is very exciting. And Matt Walker is also using Levels and we’re in conversation about a potential sponsorship with an AMA series. They’re doing all very exciting stuff.
(02:45):
on Pinterest, we’re experimenting with some videos on that channel. We have not yet done much over there and I’m so excited to see how that goes. We’ve got a video member story, which is in the early draft phase with a really amazing member, Dave Phillips, who we’ve done a story on our blog with, and that video content is coming together. I saw a little sneak peek and it’s pretty unbelievable. Shout out to Dave and also the whole team who’s bringing these member stories to the forefront. And through various channels, we’ve got new episodes from Austin McGuffey on Insulin and Insulin Resistance. We have a conversation on whole new level as well as YouTube on erectile dysfunction and cardiovascular disease risk with, I cannot pronounce his last name, I’m sorry, but the story between Casey and Dr. Mark, I believe his name is. And then Sarah Bemer and her story about IVF as a former fitness competitor also live.
(03:46):
And then we’ve got the UK founding member cohort was onboarded this past week. Really great engagement there. It seems in Telegram as well as on that initial conversation. We’ve got the website branch for the UK is now live, so that’ll be, I think it’s Levels health.com/uk currently up and tracking to the close beta, including all the backend work by end of the month. Still very exciting. Dr. Gottfried’s research, which we were able to participate in as a sponsor slash gift. It’s focused on using glucose methods to identify early the onset of prediabetes. So this was a meta-analysis review. It’s been accepted for publication. Super exciting to see that continuing to proceed through. And then our research team is digging into the types of glucose excursions and how they correlate with variability.
(04:31):
I think Azure’s done some really great work. I haven’t been able to review it deeply, but the goal there is to look at how with our new scoring approach focus primarily on stability and reducing spikes, understanding what sorts of glucose excursions will drive better and or worse variability. On the labs front, so the metabolic health panel, we’ve had some progress this past week on Labs 2.0. The objectives are a simpler menu of tests and a focus on cost and comprehension. So making this more accessible and more understandable. Obviously the tenets of everything we do at Levels, this is going to be, it’s a bit of a balance to make sure that we’re getting sufficient data from the blood panel with a reduced number of tests.
(05:12):
So we’re going to probably introduce some new ones. And also we had a cool press release go out, which it sounds like the support team really benefited from on the new scoring initiative. So the press release approach, which I think is kind of made in the image of Amazon. The goal there is to release a forward-looking, kind of like what we would put out into the world on releasing a major new product along with FAQ. And we did that internally such that we could prepare for and sort of anticipate what the sorts of concerns troubleshooting questions might be as our members try it. Engineering is wrapping up in-app demographics scoring V2 and UK infrastructure e-commerce on track for end of August. Data science is improving processes. I think they push out their first engineering spec this past week, so just working on making things more scalable on that side.
(06:03):
There’s a focus on the data science side on better tag recognition and suggestions for the new logging and tagging. And then retros are out this week or in work on the outage that we had. So we had an outage this past week, which we’ll touch on in just a minute. And then Now V2. So retros are continuing to make their way into our work culture. Let’s see, some exciting stuff there. You see the liftoff patch in the middle. We had a bunch of shoots this past few weeks with Levels team members. We were able to meet with the Texas Innovation Center here, or currently in Jersey, but in UT Austin, and talk about some potential pipelines for hiring. We’ve got a couple great, really awesome, I think this week, testimonials that were very organic. Sam received one that just was quite simple, but just said that Levels has changed for life. Thank you.
(06:53):
And then I received a message yesterday, just a candid interaction between someone I know and someone that they ran into at Starbucks. And this person said that she used Levels for eight months and she attributes her ability to get pregnant to that process and what she learned there. And that’s an amazing product and company. So just these are very exciting. And there were actually several others that were mentioned to me. I saw Levels in the wild, so very exciting. And then we got some good coverage from built in our advisor, Ben Bickman, on MINDBODY Green Health, and I believe that that’s that. With that, I want to welcome Elena Schuber, member of Levels certified holistic health coach and very, very excited to have you, Elena, here on the Friday forum. Thank you for making the time on a Friday morning. The team really appreciates hearing from members like yourself who clearly make this such a front and center part of your life. So I would love to hear about your experience with Levels and what you’re excited for in Metabolic Health.
Elena Schuber (07:48):
All right. I don’t know why my video’s not on, it says it’s on my behalf, but
Josh Clemente (07:53):
We can see you.
Elena Schuber (07:54):
Oh, you can? Okay.
Josh Clemente (07:54):
Yes.
Elena Schuber (07:57):
Oh, great. Hi guys. How’s it going? Good to see you all. Thanks for all those exciting updates. Levels is amazing, and I promote it heavily to people I encounter on the daily, and it’s kind of fun when we wear that patch. It’s like people recognize you on the street. It’s like, yeah, Levels. It’s like you’re at the farmer’s market and you see it’s like the seal of approval. It’s like you have a lot of like-minded people. I have a family history of diabetes type two from my mom and from my aunt, so it’s very, very important for me to keep my sugars in check and to keep everything low and flat. And it’s been really eyeopening, especially some of the things that we thought were not spikers, real spikers. And it is, it’s totally individual, so it’s like what works for me doesn’t work for everybody else. So it’s really been lovely just to just see the data right in front of you and just be unemotionally connected to just the data and just be like, it’s just the facts.
Josh Clemente (09:20):
I love that. Yeah, just the facts. What we’re working on right now is that tool set just for the first time, see information about what’s happening in your body without any real external opinion. But I am curious in your experience where you see the opportunities for improvement in the product where we can offer not just those sort of plan statements of fact, just data and insight and where we can point in a direction that you feel would be beneficial whether to your experience or to other people that you’ve seen use Levels.
Elena Schuber (09:57):
Well, just thinking globally, it’s just making it more affordable for people would be the number one top of mind thing that comes to, because it’s an elevated product, but just of course the more people that know about it and the less expensive it can become, the more people that use it. So just putting it out there for financially, just speaking, it’s like my aunt is not in that place where she can just afford the membership, but I believe that will come and hopefully become more affordable in the future. And really, as far as ease of use, it’s a piece of cake. You just scan and you put it in and that’s the facts. It’s like no ifs, ands or buts. And the crazy thing is how stress is such a component of it, even if you eat clean, but if you’re highly stressed out or if you’re having a heated argument, I know one of my best friends, she and I compare, we don’t compare what we’ve discussed, and she was like, oh my gosh, it was so crazy.
(11:08):
She’s like, I was in a heated fight with my husband and it’s like I looked at my thing and it was spiking sky-high. It was even higher than eating cookies or eating a baguette. She’s like, I couldn’t believe it. It was off the charts. And it’s just holistically thinking how important that is to pay attention to not just the food, but it just gets a little tricky sometimes because with the spikes with the exercise, but the more you know about it. And I really love the articles you guys produce and how easy you just list things out. The recent one I looked at was just the best foods that don’t raise your blood sugar simply spelled out. Take it to the grocery market, just check them off your list or put it on your fridge and just look, okay, what can I make with these things? What do I already have? And just get rid of all the other stuff.
Josh Clemente (12:02):
Love it. Yeah. Well, I want to address the pricing thing. That’s a primary focus of the company is to increase accessibility. And so we’re a hundred percent aligned there. The objective is to make this available to everyone that can benefit from it, which we believe is a large majority of people today. From there, you mentioned a few things, some friendly comparisons between you and your friend. I’m curious, is that something that you would enjoy or benefit from in the app in the product, more of that sort of social interaction around the data directly in the app?
Elena Schuber (12:37):
Especially from new people. It’s like when they have questions and it’s just like, yeah, a community-based plus, it’s a community. I see it all the time. I live in a highly Levels thick community, so I live in Brentwood, so it’s like the Brentwood’s farmer’s market. It’s like, yo, what’s up Levels? It’s like, but you see it on the back of your arm. So it is a community and yes, I’m lucky because some of my best friends just do it. So they’re the ones actually who introduced it to me, which is great. And they gave it to me as birthday present. Isn’t that fun?
Josh Clemente (13:13):
I love that.
Elena Schuber (13:15):
That’s why I was like, oh my gosh, how fun. So yeah, definitely if you feel lawfully or alone or don’t understand the data, just put it out. I just finished a program and we have a big WhatsApp chat going and it’s helpful. It’s like a lot of people connect and communicate. It’s a little daunting when I see the 900 messages, but that’s okay.
Josh Clemente (13:44):
Well, community is definitely one of the main pillars of what we intend to build in going forward. And Cissy has been taking lead on a new human-centric initiative internally to make connecting with specific individuals, whether on the team or within the membership community or the affiliate and partnerships community to make that more accessible. And so really love hearing these sorts of stories and just the tails on the ground. It sounds like the chat’s blown up right now. We got to make a team trip to the Brentwood farmer’s market. It seems we all go crazy when we see those in the wild, but it sounds like it’s not so infrequent over there. Well, Elena, thanks so much for sharing and obviously we always want to communicate directly with our members who are really living with the product every day, and we can’t get better feedback than this. So thank you for coming on live and sharing your morning with us team. Really appreciate it.
Elena Schuber (14:38):
You’re so welcome. I’m happy to. Thank you.
Josh Clemente (14:40):
Amazing. All right, well feel free to stick around. I know you’re likely very busy this morning, but if you’d like to stay, we’ve got another 45 minutes of content here and we’d love to have you.
Elena Schuber (14:50):
Awesome, thank you guys.
Josh Clemente (14:51):
All right, great. Quick culture and kudos slide. I want to say happy one year to Steph and Helena who both hit 365 days, which it actually feels quite a bit longer on this case. David, Karen and Stacy hanging out for a pint and Thai food in London, which I’m very jealous of. Seems like this is also nearby to Cosima’s hometown or maybe even her house, which is awesome. And then again, shout out to Lynette and Rebecca for getting through onboarding, sticking to the onboarding flow that’s so important to our culture, but then diving in headfirst and just crushing the queue. Rebecca took lead on the queue as help scout number one within three days, so that’s huge. And then Lynette is really doing an amazing job closing the loop between what she’s seeing in the various support queues and how we might be able to improve for the members to get ahead of those problems and really love to see that.
(15:49):
So thank you both. And also David and Cosima are doing an unbelievable job of managing all the specs and launch docs. Impact can’t be overstated. Thank you both. And lastly, I want to touch on the outage that happened this past weekend. So first of all, shout out JM. I think Matt also raised it, but just kind of relentlessly paying attention to the experience and to the systems that our customers, our members are interfacing with. JM uncovered and really stayed focused on getting attention on this outage that happened. And then the ENG team really took over, took the time and ran this thing quickly to a close. We now fully understand the problem and Dave put out a retro video, which was really helpful both for us, I think on the ENG team, but then also for the wider team to just see how a retro can be structured.
(16:41):
And the Loom was a really nice accompaniment to the document just to get a quick insight into what did we learn who was involved and how we’re improving based on it. So I doubt we will have an outage just like this one again. And if we do, there are sort of some soft landing features that they’ve implemented such that the member or the potential member does not have this blaring red alert going off in their face when they’re trying to check out. So thanks Eng team. Thanks JM. Thanks Matt.
(17:10):
All right, company objectives, main thing has not changed. We show you how food affects your health. We should all be working towards this priority in one way or another. If you aren’t, please raise that concern. And the objectives that fall out of that are member retention right now, member health improvement and new member acquisition. So all of the major initiatives that we’re investing resources on fall through the objectives and key results program that we’re putting together with the help of Lauren Sonia into one of these three categories. So all of us, I think by this point inside of the functions have worked on identifying our functional OKRs which stack up through these top level company OKRs. If you haven’t done that, please do or coordinate with Lauren if you think that would be helpful. All right, over to product.
Maziar Brumand (17:57):
Hi everyone. Welcome to the product update. This is a live one this week. Just wanted to show you, reiterate what we’re working on. The two big features that the team are working on shaping actively is healthy food choices, which David and Alan will give an update today in the live sync live forum and then Labs 2.0, which Josh mentioned. There’s a lot of thinking going into this, really focusing on three things, comprehension, people understand what’s going on in their body. Second is it’s actionable, which is really important to us so that the information is compounded and people can actually take action. And then third one, bringing the cost down so people can do it more often and it becomes really a part of your experience where you effectively check your metabolic health status every three months if you want to, and you can then change your behavior within the app to drive to the numbers that you want and the wellness outcomes that you want.
(18:49):
So it’s really a lot of thinking going in this, I’m really excited about it and it’ll be a core part of the app. And there’s UXR that’s out right now that Taylor and Cosima worked really hard on. We created videos, we created the mock lab and it’s gone to our members to get feedback. So really excited about both of these efforts. These are effectively I think will become the keystone of our features. And then there’s a number of stuff that we’re working on that will make these features a lot more powerful. Obviously the in-app content will become the lifeblood of our app, the information we’ll give to people, how we give it to people, when we give it to people, it’s going to be really important. There’s a lot of thinking going on, there’s a lot of experimentation with different formats which the whole team is really contributing to.
(19:31):
But big thanks to Mike D for really spearheading the event-based insights lead for building the tools that we can put into app quickly, Stacy for working on the personality driven insights or content and then obviously David and Alan really taking the lead here for creating the architecture for in-app content and thinking through a lot of nuances that hopefully will make the app a lot more powerful. And we’re also kicked off the first personalization feature, which is bringing demographics into the app so we get information so we know what people are interested in, we know how to present information to them in a way that’s motivating and is within their ability. So really excited for personalization. It could be a cornerstone to make our app feel magical. Levels is really helping you work towards a goal, which really fits in with the rest of the stuff.
(20:22):
So you have this feeling of progress as you do things in the app. So that’s another exciting thing that will tie into everything else. And then reward is great with that. Brett shared last week of how do we actually make the app look fun and magical and then’s this community piece, which we’re thinking about as well, I think Josh mentioned it earlier, how do we actually bring the power of our community, which is a very active and passionate community into the app? And so this is something that we’ll be thinking about once we get through the other priorities on this list. Can you go to the next slide Josh. So it’s a little small here, but what we’re thinking about is visualizing the roadmap. And what you see here is everything basically ties to the three company objectives. And I just showed two here because we’re prioritizing these two, which is efficacy effectively improving people’s health and behavior change.
(21:16):
And the other one is retention. And all the features that we’re thinking about on talking about are effectively beating one of these two things. And there are some growth stuff, but we left it off just for simplicity today. And this basically the way to read this chart is from left to right is priority. When are we going to start working on stuff? The coloring kind of shows to you whether it’s complete, whether it’s being built by engineering, which is the border of green or blue, which is something that design and product that working on to shape. But effectively every row belongs to one feature category. And as we rev them and improve them, you’ll see the progress. Another thing you’ll notice here is that some features feed to other features. For example, if you have rewards here, rewards will feed into healthier food choices 2.0.
(22:03):
So as we build the supporting features or the supporting cast for our major features, they will feed them so that they become more magical and more powerful, but yet simple at least to use. So anyways, this is a representation of that. This is working in progress, but just want to preview you guys. We’ve gone through a number of iteration, but really the goal here is to help people understand what features that we’re working on, how are we revving them and what company objectives they tie to. With that, I’ll hand it over to David and who will talk through more detail on a number of these things.
David Flinner (22:33):
Josh, can you do a quick refresh? Nice, I got that down to a pet. All right, I wanted to quickly highlight that we’re starting to really double down on retrospectives, which are pretty much the most important thing we can do out of all the experiments that we’re running at the company. This is the thing where we take a moment after we’ve launched something and reflect on what we learned and what we’re taking away from the experiment that we’ve ran, highlight the action items and the lessons learned so that the whole company can learn from that and we can build on top of each other as we go onto the next step. So I wanted to let everyone know that the retrospective for the Now V2 project is now available. You can find it at Levels.link/nowV2-retro. I’d encourage you if you’re interested in what we did there, what we learned, go ahead and read through that.
(23:18):
And as a quick summary, the takeaways that we want to pull out from this one, there’s a whole bunch more, but the top level ones were when we go out to general availability, we want to make sure we’re shipping complete skateboards full end-to-end experiences. In this project we uncovered that there’s a really big need internally for process around pre-launch testing as well as post-launch analytics. And so we spun up a whole bunch of initiatives around those. You can find the complimentary memos and things that we did from that and those links there. And then there’s also a need for proactive comms.
(23:49):
So before we do a big project with a big member facing release, we’re going to be communicating much more thoroughly so that the member success team is equipped with FAQs, talking points so that the whole team knows what we’re launching, like Casey is equipped to talk about it when she goes on podcasts, all sorts of things like that. Just to quickly recap Now V2 was the precursor to what we’re doing now with the stability ring. It was laying down the rails, the foundation, it had the graph at the top and the pinned insights and activities list and our members have been using that for a couple months. Retrospective now available. So go ahead and take a look. Next slide.
(24:26):
And our very first release comms doc is also now available. So in the spirit of learning from Now V2 when we’re releasing scoring, you can go ahead and see everything you’d want to know about the scoring V2 release, which is the stability ring and the glucose state descriptions and spike detection with spike tracking and spike streaks. So there’s a document for that. But the bigger picture here is that we’re starting to do this for all of our launches. Cosima is actually going to be spearheading both our retrospectives and maintain the product release database. So huge thanks to her for that. She put together a link on the left here, which you can find a notion, any feature that we’re working on, you’ll find a row in there. You’ll see what the release date is, the document that we have that’s preparing you to understand what that is. So go ahead and take a look at that link. Next slide.
(25:16):
And the first thing that you’ll find there is scoring V2. So I just wanted to touch base on a couple flight media projects we have in product land. The first one is scoring V2, which is the stability ring spike tracking. This is really the core rails of our behavior change program that is laying the foundation for the game that motivates people to know where they’re at, understand if their day is good or bad, and set the stage for what to strive for. The external beta has been underway. We have 35 people out of our goal, 50 enabled right now. We invited several people. This is an opportunity to improve, so we actually hit higher opt-in rates on that, but that’s been going for a week. I’m sending a survey today, so I’ll send the results of that as they roll in. General availability is going to start on Tuesday.
(25:59):
So this is when we start rolling out to the broader audience and it’s going to be a phase rollout because there’s a lot of technical complexity to this release on the backend and the end team’s been doing a stellar job preparing us to support the load, but we need to take it slow to make sure that our server can scale up to handle everyone with all the spike detection and data analysis that this entails. So we’re starting with 1% and Dave and I will be looking every day to see, hey, can the system handle a 5% rollout, 10%, 20%, 50%, 100%, and hopefully over the course of a week we’ll ramp up and things will be smooth.
(26:30):
So you can see the full plan in the release document in every milestone that we want to hit. But this has been an incredible effort across all the different teams from data and analytics, backend engineering, design, everyone at the company has been involved in this, the member success team and you all testing this out and providing feedback and really pushing us to get that release doc done and highlighting that members will be missing the metabolic score and want some information about this. So we’re pretty excited about this one. Do let us know, keep the feedback coming, but this one will be launching starting on this coming Tuesday. Next slide.
(27:10):
And then the other big piece of our foundational table stakes product pieces is logging and tagging. So John has been underway doing great work laying the foundation for the UI pieces for this. We’re focused on making logging very fast and very seamless. So you can test this one out now. It’s underway. Helena has been working on the backend ranking algorithm for the tag suggestion. So the tags, we want to make it very fast to input your food and auto complete the words that you’re typing. So Helene has been working on that. That’s being handed off to the backend engineering team for implementation. Now we still have a few more weeks on this one, but we’re targeting the code complete to August 29th, which is another cycle from now. We’ll probably need a bit more time to validate and learn from that at the end, but that’s where we’re at. It’s progressing quite nicely. Next slide.
(28:02):
Okay, so far I just highlighted a lot of the things that we’ve been doing to kind of shore up the table stakes product. And I wanted to highlight where we’re moving and bring it back up to in the past Levels to date has been about metabolic awareness. We’ve really taken you from a broken network connection and for the first time showing you the map your landscape, where are you on the map, what are the things around you and how far away are you from the goal that you want to get to?
(28:24):
Next slide. And where we’re going is really improvement. So never before have we really highlighted the path on where to get there. We’ve done isolated things, but we’ve really only given you tools and it’s been self-service. So the next few things that we’ll talk about are incrementally us showing you the routes on how to get there, which is our first steps in helping you improve. Next slide.
(28:46):
And where we really want to get to is layering on all of both our step-by-step proactive guidance hand in hand with the work that Brett is doing on rewards, which is really key and critical towards people aren’t going to just naturally walk the path that you provide for them. We have to be their guide, we have to be their Uber chauffeur, we have to be driving them there. And so this is going to be a combination of the direction that you’re going, the motivational stuff that Brett’s working on, the Levels levels that we’re working on to hook into status and motivation and ranking.
(29:18):
All these things are going to come together in the behavior change model and be the things that let us as Levels be your Uber chauffeur along the way to that navigation. The things like Cosima is working on Victor with personalization. It’s not going to be one size fits all, but as we kind of tackle this layer by layer bit by bit, it’s going to piece together where people are just going to have less and less friction and more and more motivation to meet their health goals. So next slide.
(29:44):
Oh sorry, that was back relating to the [inaudible 00:29:47] that was all for member health improvement. So we want to prove efficacy, we want to actually help you achieve your goals. So next slide. The other big thing that we’re focused on to the three at the company is member retention. And so a big part of what we’ll be doing is focused on that member health, but if we nail that and if we provide some other routes, we’re going to also nail retention. A couple of the efforts aligned with member health improvement are also about what happens when you don’t have a CGM or is Levels only and all about CGM.
(30:18):
And one of the things we’re exploring is bringing proactive content from a variety of sources at a high level, you can think about personalities, experts like your nutritionists or people like Kelly, the community, all the great work that Cissy’s doing experiment in. There’s ways for us to layer in journeys that don’t necessarily always rely on the data or do rely on the data and augment it in certain ways. So right now we’re experimenting with a couple of things in here. Event-based insights in-app personalities and healthier food choices. Next slide. And I wanted to pass it over to Mike to briefly talk about what we’ve done so far on the event-based insights side, helping people understand what to do next or instead.
Mike DiDonato (31:02):
Cool. Thank you David. So yeah, with event-based insights, goal is to surface the right content at the appropriate time. And as we develop these insights, we’re using our behavioral design framework and process, which includes identifying a trigger or an anchor for our members, an action or better as to say, increasing the ability or the accessibility of the desired behavior or outcome that we want. And then we want it to be motivational and rewarding. So it shows here right now we do have a oatmeal swap live in the app for members. So if somebody tags oatmeal and receives a zone score of six or less, they will receive this insight, which the first page just gives a quick overview of oatmeal and then it talks about chia and also pushes them to our chia recipe on the blog. In a future world, I could imagine this being instead of going to the blog, it could be like a video of Casey.
(32:05):
I think I’d like that as well. And then the other thing I wanted to update on, so shout out to Galeet who’s working on a tool to help us increase velocity. So for those of us that don’t know, we have something called the user feed template editor in retool, which has historically allowed us to load content in app to create and or edit insight cards with event-based insights. Once this was done, Galeet or someone on edge would have to wire that up on the backend. But with this new tool we’ll be able to basically do just about any insight including the triggers without any ENG work, which should greatly, greatly increase philosophy.
(32:48):
And then the last thing, just kind of like next steps as we go to develop the next set and future insights, as David mentioned with everything else, we’re going to do a retro because even though we’ve been working on this for a few weeks, we already have a lot of learnings. And then the other thing we’re going to focus on, we have so much rich information with all the amazing work that Haney’s done with the blog, Casey and Sonia and all the stuff that we have compiled in Notion and Cissy with getting to the green library. So we want to compile that in one place so that we can effectively use that for event-based insights and the other in-app content that we’ll do. And I think that’s it.
David Flinner (33:30):
Yeah, one of the things that Mike was mentioning, well two things he mentioned ability and motivation, and that’s really the theme of this. So you can think about there’s this meta level where we want to really help our members across all these different tasks we can take because people are motivated by different ways and if we just give them one choice, it may not be for them. So we need to meet them where they’re at, give them a variety of things to meet them ability and then also motivate them in different ways. Next slide.
(33:53):
The primary way we do this is showing you how your food affects your health through your own biofeedback. But then we also are going to do it by meeting you in different moments where event-based insights is retroactive where we’re meeting you afterwards. And Alan is going to talk about right after this, how we’ll proactively show you healthier food choice options. But one other totally different route that we’re experimenting with is that motivation layer and can we pair you with someone who you relate to that you can see yourself in or you’re just interested in their journey. And so that’s the idea behind in-app personalities.
(34:24):
We’ve seen a lot on social that our best performing content is really related to two things. One is if it’s related to recipes and food, but then moreover, anything that has a real person behind it does super well. So people tend to gravitate towards figures that they can relate to. We’ve seen this in other apps like, I don’t know Peloton people following their trainers or other metabolic health influencers in this space. And so Stacy’s been working on getting us set up with a few people that we work with on Instagram. You can see Janet Mallory here and she’s been working with them to pilot the first in-app personality content that is similar to the video that I made eating bread.
(35:03):
The idea here is that they’re going to try something that is unique to them, it’s true in authentic and see how their body responds to a trial where it’s something, one of their favorite dishes and then they try it again with a Level strategy to see if they can improve it. And you can see here Mallory’s eating some keto waffles with regular syrup and she tried it again I think with some sort of keto syrup and she improved her zone score from a four to an eight. So that’s the sort of thing that we want to highlight and show people that there is a path forward. Janet was trying her favorite toast with jam and butter versus I think a variant of that toast and butter. Stacy’s going to send out an update on this later today. We have some draft videos of them ready to share and I think that’s going to be really interesting to see how it performs in the app. Next slide. So I’ll pass it off to Alan who we will talk about healthier food choices among other things.
Alan (35:53):
Hey folks. All right, so design updates August 19th. Next slide please. So every week I’m going to share some progress in design. Today we’re just going to focus on one project because we’ve got not that much time, so there’s lots of stuff going on. We’ve got rewards. So Brett’s doing tons of great animation work there. We’ve got blood V2 kicking off that was red and I changed it to black and white because it was kind of gross. Today we’re going to focus on healthy food. Next slide please.
(36:23):
So as David was mentioning, we’re going to start bringing more content into the app. We know that users care about this, they want it, and we’re going to start with giving people a bit more of a headstart on what to eat on any given day. So we’re going to be running an experiment, it should be roughly seven days, and we’re going to start integrating things like recipes and our content about thought leadership. I think we’re going to focus on recipes primarily so we can help people make better healthy choices. And so this is the initial screen on the homepage. You’ve got a little link there at the bottom that you’ll see every day in the morning and those will be hints on what to eat.
(37:00):
So you can go to the next slide. You’ll get a selection of recipe videos and content that you can get started with. And the goal here is to give you something stable, something compelling, something easy to make so you can get your days started off right. Next slide. And then there’s your recipe video so you can load up a little video. This is primarily meant to be a small test and we’re going to see what works with people, what resonates. Of course, this is a pretty relative to what we want to do. We want to be able to help people close the loop on making healthy choices and we don’t think just serving content is going to do that, but this is going to give us some infrastructure to start serving more of this content into people’s feed and help them make better choices. Next slide.
(37:47):
Oh yeah, more details. Yep. So the usual stuff, call it the typical score for people video when it’s available and we’ll tell them a little bit about why we think it’s so great. And of course there’ll be instructions on shopping lists on how to make it. Next slide. And as I alluded to, we’re getting get more content in the wait that’s not glucose goddess, that’s Casey. We’re going to start bringing more content into the app, so this is a great opportunity to start personalizing for people. We haven’t really done a lot of that in the past, but we’re pretty excited about it in the future and this is a great chance to be able to serve to people what they’re interested in, what they’re Googling already.
(38:26):
We have tons of users as we heard before, dealing with PCOS and pregnancy and so on. We should be able to get that to them or give them a hint on what to eat in the morning or like Scott, tell them how to get those gains on a plant-based diet. You should all be able to benefit from this content and so I think personalization is kind of the path there. Next slide. That’s it. Thanks.
Josh Clemente (38:52):
Awesome updates. Thanks everybody on product and design looks amazing. Real quick, Alan, are we going to give a mechanism for people to save those recipes for later if it pops up in real time? Do we have an option for them to just store those somewhere?
Alan (39:08):
We’ve heard that request a couple times, so I think so. Yeah, it’s definitely an issue.
Josh Clemente (39:12):
To have a queue, that’d be awesome.
Alan (39:14):
I think for the initial experiment, no, but it’s definitely something on the list of follow-ups.
Josh Clemente (39:19):
Sweet. It’s going to be awesome. Thank you. All right. Experimentation learning. I believe this is Cissy.
Cissy Hu (39:27):
Yep. Hey Dean, I’m going to share a recap on the second getting started cohort that just wrapped up this past week. And so for people who are unfamiliar with these cohorts, you can see a outline here of what these cohorts are made up of. They’re structured month-long programs where we’ve experimented with in partnership with the wearable challenge team and they’re specifically designed to help new Levels members get started alongside other new members. The first time we kicked off this cohort was back in June of 2021. And so you can see here in addition to the standard set of CGMs that you get when you purchase Levels as well as our annual membership, we also offer a kickoff call with the wearable challenge team and Levels as well as access to a private telegram chat and a 30-day metabolic guide. And so the things that are highlighted here are the variables that we’ve since upgraded since the first cohort.
(40:32):
For the first cohort we did a relatively wide-reaching outreach and so we emailed anyone on the wait list who might be interested in getting started with Levels for this cohort, we focused on folks who specifically selected that they wanted to improve their diet when they were signing up for the wait list. The second piece is we added this kickoff call component with a health coach from wearable challenge as well as with our team. And then the third piece was we had a telegram in the last, or they used the Circle platform, but we used Telegram this time around and we actually decided to amp up the number of prompts. So we had a daily piece of content that was ripped out, whether that be a question that we asked members to reflect on, a poll, some sort of experimentation they should do. So we kept that consistent to see whether or not the content we were pushing out would drive engagement. Next slide.
(41:30):
The two primary goals that we were focused on was first the focus on shared interests. So when the team ran the initial cohort last year, the realization was that because they had reached out to a relatively wide range of folks, the thought was there wasn’t as much engagement because it was just everyone who was interested in Levels not really segmented out by people who were interested in a specific goal. So this time around, the specific goal was to improve diet. The second goal was on the internal side to build a playbook so that way we could leverage it for a future US cohorts across specific niches as well as alpha cohorts in international expansion, which I’ll talk a bit about in the next few slides. Next slide. For a snapshot of this cohort, we emailed 2,200 people on our wait list, about 45 members registered, so about 2% of those folks registered.
(42:30):
Of the 45 people who actually purchased the CGM and their annual membership, 27 members joined the telegram, so 60%. The hypothesis here is that we were doing this outreach in parallel to when the wait list was starting to get outreach and we were starting to open up for general availability. And so the thought is 40% of folks probably just wanted access to Levels and wasn’t actually interested in the community aspect, but there were 27 folks, 60% who joined, and then when all was said and done, five of those participants continued on with sensors, 19% conversion. And interestingly enough, about 60% of them, the people who continued on were relatively engaged in the telegram over the four weeks and then six participants who ordered the blood panel, which was a 22% conversion of all the folks in the telegram, a hundred percent of those people were engaged in the telegram.
(43:24):
So it’s pretty interesting to see no one who wasn’t engaged in the telegram ended up doing a blood panel, but if you were engaged in the telegram, it might’ve helped with encouraging you to move forward with using our blood panel. The chart below shows the wide range of dietary preferences that the members coming in had. And so this is just to signify that although we had a more narrow program this time around, which is improving diet, there’s still opportunities for us to even cut through and focus on a low-carb cohort, a carnivore cohort, a keto cohort. So we have opportunities to niche down even further if and when we decide to run more cohorts. Next slide.
(44:10):
In terms of cohort engagement and feedback, on the top left, you can see our weekly active members. So it actually stayed relatively sustained over four weeks, which even in the last week we had 17 folks engaging in the telegram, which was meaningfully better than the engagement stats in our first cohort. The second chart on the top right is the prompt engagement versus discussion. So you can see about 25% of the conversation in the telegram was driven by prompts that we were dripping out, and then the remaining 75% of conversations were just members talking to one another.
(44:50):
We excluded any stats of our own messages, so none of the Levels or wearable challenge messages are included in these stats. The bottom left, you can see the breakout of the primary questions that the groups were focused on asking. The biggest chunk here is spikes. A lot of folks had questions about why they were spiking, how they were spiking, what does this mean for them? And then sensors, we had a huge amount of questions about sensors the first week, which was to be expected. And the second piece was when they were changing over at the 14-day mark. So a lot of questions there, but we think that having the kickoff call initially helped actually mitigate any additional questions we would’ve gotten. And so that was a pro to have the kickoff call.
(45:42):
And then from an engagement perspective, you can see that by prompts we got the most amount of engagement from folks who were answering polls. And so some of this experimentation will help inform how we want to scope out community in the app. So because there’s so much engagement with polls, potentially that’s a way we engage folks in forum if we would decide to do some sort of forum chat feature in the app. Sorry. Yeah, thanks for that. Thanks for that, Chris.
(46:18):
So we had a mix of seven folks who actually answered the survey after the cohort ended and we had an average NPS of 98 and the themes that we saw were seeing the difference from making little changes was really helpful for folks. And then having the wearable challenge team and the Levels team in Telegram being there to answer these ongoing questions and the content drips was also really appreciated. Almost everyone cited that in their feedback. And then it was interesting to see that content drips drove 25% of the conversations and so we’d have 25% less conversation, and so we’ll experiment with maybe pulling that back next time if we do a more specific cohort, even more niche than just improving diet. Next slide.
(47:09):
The key takeaways are the first one is there’s an opportunity to hand off some of these well-defined micro communities with new members to wearable challenge. The goal here is to continue to do some experimentation around what our community product might look like and so what are the engagement tactics that we should be focused on? What content should we be prioritizing? So you saw on the last slide we did a huge breakout of what all the topics were and so trying to figure out where should we be surfacing more content in the app in order to minimize some of the questions that come up.
(47:45):
The second piece is for the getting started cohorts that Levels is more heavily involved in, we should be focused on aligning those with key personas and markets that we’re building for. And so a good example here is for the UK alpha cohort that we just kicked off last week, Karin and I are moderating the Telegram and we’re getting really good real-time insight from this group. And so that makes sense for us to be spending a lot of time focused on managing that cohort because we’re building into the UK, it’s a new cohort of people that we haven’t learned from before, whereas for US new members, it makes sense to actually figure out what continues to engage them and so we might not have to be as involved in those cohorts. Another example is extending these cohort offerings to existing members who fall into some of these key personas that we’re trying to build for.
(48:35):
So Alan mentioned PCOS and Women’s Health reaching out to existing members and saying, hey, you already know how to use the app. This isn’t so much about how to use the app. It’s actually more about how can we better service you on Levels for your specific need. And so leaning into these as one month long member programs that we can actually do learning alongside them and help support our members. That is it. If you have any questions, feel free to ping me and I will share a more in-depth debrief in threads in the next week. Thanks and shout out to Chris for helping pull some data.
Josh Clemente (49:14):
Awesome, nice dive. I love seeing how this is continuing to evolve. The wearable challenge is far different than it was on day one and we’re continuing to learn a ton it seems. So that’s awesome. Thank you cissy. All right, hiring updates. Nothing changed here. So Taylor is joining us in, well, September 6th in a few weeks. So more on the support side and on the open roles, continuing to have software and both backend and mobile openings. And then we’ve got an R&D engineering position open and in general always would like aligned people to send us information, reach out through our link on Levels.link/careers.
(49:54):
All right, we’ve got about 10 minutes here. I’m going to stop the share and we can transition into the contributions. We will go through the hand raise approach here. So if you’d like to share something this week, just hit your reactions button and raise your hand like this. So I’ll kick off. This week professionally, I’m really excited about retros, I’m excited about all of that amazing product, just inertia that seems to be building and on the inside just refinement and speed and retros. So all very exciting stuff. And then on the personal side, had a really good week, saw Levels in the wild, got a bunch of testimonials that made my heart warm and I’m here at the shore for the weekend and life is good. Chris.
Chris Jones (50:48):
On the level side one, just a huge shout-out to the support team now to kind of come through the other side, turn in the corner. So I just want to appreciate two months of a bit of pain. So just thanks for hanging in there. Then also on the Levels front, super excited watching today’s update and just seeing the product roadmap come together. It just feels like a well-oiled machine. I know we’ve been making lots of great products, but I know I was always like, well, what’s coming on the roadmap and when’s that going to be released?
(51:19):
So it just feels like that team’s firing on all fronts. So I just want huge shout out to the team, both in terms of things like PR, updates around when things are coming, how many people it’s going to hit, so I just can’t thank you enough. It means a lot to support to know when things are being changed in the app and how many people see it. On a personal side, easy one, I’m excited about an HVAC guy coming to fix my AC because it’s been dripping condensation on my bed, so I haven’t been able to use AC in 90 degree heat. So I’m looking forward to getting that fixed and learning some new Montana man skills.
Josh Clemente (51:56):
Love it. Get a dry bed there, Chris. Or put that water through an H sleep pad. Brett.
Brett Redinger (52:07):
Sorry. Yeah, one. So proud to hear everybody speak so seamlessly about rewards and behavior change and you build that doc for a while and we did it as a team and now that it’s just absorbed into everything else, it’s fun to see it running around and doing its thing. Very proud. Also, people keep mentioning rewards, even though I’m the only person really listed there, there’s been tons of feedback. Everybody has been lurking in the different updates and given great actionable feedback that is all present in the current iteration and looking forward to that. So it’s been super fun. And then, yeah, personally getting ready for Burning Man, I’ve been packing up. It’s my first vacation since I started, so it’s going to be nice to get a little break and then come back and even go even harder. So really excited.
Josh Clemente (53:04):
Amazing. I think we’re going to have a couple Levelers there. Riley.
Riley Walker (53:08):
I just wanted to comment yesterday, really good cafe around delegation. So I’m pumped to take some of the feedbacks and lessons and stuff that people were sharing and supercharge my delegation skills a little bit. So a good refresh. And then really excited about the OKR process that Lauren and Sonia are leading and that they’ve just been doing such a fantastic job. So from my perspective, really excited about the common language around goals and stuff coming together and excited about how finance fits into that.
Josh Clemente (53:43):
Nice, Mercy.
Mercy Clemente (53:47):
Professionally, I guess I had my first Levels in the wild sighting, so that was very, very exciting. And then personally, I have a new niece this week, so excited to have met her and see her for the week.
Josh Clemente (54:01):
Yes, very exciting, Sam.
Same Corcos (54:05):
Yeah, for me personally, I decided last minute to also go to Burning Man, so I haven’t been in about a decade, so that’s going to be good. Brett, I’m on Esplanade in 10 if you want to drop by at some point. On the work side, I’m really excited to see the product velocity pick up. I think we have a lot of things in flight. We’re getting some things delivered. It’s really encouraging. So that’s got me really excited for what’s to come.
Josh Clemente (54:42):
Likewise, Ben.
Ben Grynol (54:46):
Yeah, work wise super stoked on all the support stuff. Love seeing Rebecca and Lynette jump into the queue and just start hammering. So hat tip to them. It’s always awesome to see new team members wrap up and just see [inaudible 00:55:00]. Personally, I’m here, so is this young man and that’s what I’m stoked about. So hat tip to Justin and that’s it from the [inaudible 00:55:11].
Josh Clemente (55:11):
Nice. All right, we’re out of hand raisers, so I will put out a last call. Anyone want to share? Maz.
Maziar Brumand (55:21):
Why not? We have time, right? Work wise, I’m really excited about the product Velocity too. I think the team’s just been really, really knocking out the park. Big shout out to Cosima. She’s just come online and added so much capacity, energy, and just good attitude. I’ve been thoroughly impressed. Big shout out to Cosima, I know she goes on the shout-out as well. So I’m really excited about the team. On the personal front, the nieces and nephews are visiting from England and so it’s just really fun spending time with them and it doesn’t happen all the time. Took them to a baseball game. I didn’t go, but they were taken to a baseball game yesterday and they had a blast. So we’re just throwing around ball. They’ve got the gloves and the shirts and caps and stuff like that, so they’re super excited. It’s been good.
Josh Clemente (56:09):
Sounds like fun. Enjoy the weekend, Sonja.
Sonja Manning (56:13):
Hey, team. Levels wise, there’s almost too many things to count to be excited about, but I had my first think week last week and really got to dive into digital marketing and a bunch of advisor strategy and just feeling like the sky is the limit on some of the things that we can do in both of those categories. I’m so excited to continue to put things together and share with you all very soon. And then personally, I’m in Minneapolis for the next two weeks, spending some time with my family, which is really nice. And next week we’re going to the Minnesota State Fair, which is the largest state fair in the country and it’s going to be a very bad blood sugar day, but it’ll be a very interesting day. I might make a reel of all of the crazy things that I try at the state fair.
Josh Clemente (56:55):
I don’t remember if it’s fried Oreos or fried butter, but the state fair has probably the least Levels friendly foods in the world.
Sonja Manning (57:04):
Actually. Maybe I’ll try to go to hunt for Levels friendly foods at the state fair and show people that it could be possible to enjoy the state fair.
Josh Clemente (57:15):
There you go.
Sonja Manning (57:15):
Stay tuned.
Josh Clemente (57:15):
That’s going to be, that’s TikTok viral. Casey?
Casey Means (57:18):
Yes. Hey everyone. So personally, I took a vacation week and was backpacking in Wyoming for eight days and it was awesome. Totally off the grid, really refreshing. And the views and vistas were just so spectacular. I’ll post some photos in the vacation channel soon. Professionally, it is just so crazy to go away for a week and come back and just see everything that’s happened. Like this forum, as many people have said, it kind of just blows you away with how much progress is happening, how much velocity and the support stuff, the product stuff, the design stuff. It’s really amazing. And I’m so excited about the ways in which we’re going to help people figure out what to eat and how to do it and how to make it simple and delightful.
(58:05):
So super exciting. I also had my first one-on-one with Charu this week, and it was really amazing to see how many similar passions we have and also just how much exciting stuff is happening in data science. And then the last thing I’ll mention is just a shout-out to Sonja who made coming back from vacation so much better than it has been in the past with a very organized list of what I had to look at. And I’m just super, super grateful for how amazingly organized and how much systems expertise that she has. So thanks all,
Josh Clemente (58:38):
Love all that. Well, welcome back, Casey. Hope that was an amazing trip. It sounds like it was. And yeah, everyone have a great weekend. We’ve got office hours with Sam after this. If you’d like to ask Sam questions that you have not had the chance to ask him or pin them down on, definitely take advantage of that and that’s an experiment. Looking forward to hearing how it goes. And otherwise, have a great weekend everybody. Enjoy. Thanks for all the help.