Josh (00:00):
Welcome to Friday Forum, March 24th, 2023. Get straight into it. So just a reminder, Friday Forum is weekly sync time. The goal here is to celebrate achievements across the company. Look at what’s going on in different functions that are… it’s not always in sight when we’re fully remote, hear from members and partners to share the culture. This isn’t the deep dive business analytics meeting, this is not where we do strategy. This is just where we share what’s happened over the past week. All right, on that front. So, this was our biggest eng week in terms of poll requests, releases and fastest deploy time on record for engineering. At least, since we’ve been measuring. So, pretty big numbers there. Awesome. We built out for Beta 1, polls, advanced video player, a new feed algorithm version three, which has better content management tabs and Trends 1.1, all this week.
(00:58):
On the support side, we’ve actually raised the bar on three of our SLAs due to just outperforming for the past few months. And so, we’re going to be bringing a number of these in and we’ll continue to do so. So, this is a big improvement over where we were a few months ago when we were rolling out to general availability and really struggling to keep up. So, huge shout out to the support team, Autonomy, which is the group that we’ve been using to scale support. We’re going to do a phase two with them, basically amplifying the amount of support we’ve got from Autonomy, especially over weekends to alleviate our team to be able to rest, recover, et cetera. So that’s going to be great.
(01:36):
Then we’ve got all hands on deck from the support team helping out with product strategy across the board. So, I’m going to have another shout out on that in just a second. My emojis got all screwed up here, which is annoying. We also doubled the Beta 1 size. So originally we had I think 330 people for Beta 1, which starts April 3rd. We’ve now doubled that and we’re going to split it into two plans. We’re in the process of doubling it with new signups. We’re going to split this into two separate plans, which will give us a really nice insight into kind of an A/B strategy in terms of what sorts of plans people are really resonating with. So, did a fast print on expanding the number of or the signup flow for more people. Sending out new email campaigns, including A/B tested versions, as you can see over here.
(02:21):
And we have an onboarding survey ready. So once we take those signups, they’re going to go through the onboarding survey, which will allow them to pick the plan, which of the two plans they want to go into. We also have a retro, which Ben produced for the first wave of Beta 1 signup. So, the 330 people that had signed up. And we ended up with a conversion rate of about 9%. Our target was 10%, so ballpark good. This is the first kind of shot on goal for this sort of thing. So that’s a nice first swing. We’ll be learning a lot more as we go through Beta 1, Beta 2, et cetera. And then Lauren’s kicking off a business model project, which is diving into the market research and positioning strategy for the CGM optional product. We’re digging into Q2 growth priorities, which Tom is taking the lead on.
(03:05):
And then kind of parallel to that, Maz is taking the lead on a bring your own device exploratory pilot. So, we’re going to look at additional options for expanding the availability of CGM driving price down a couple of things that we haven’t revisited in several years, including Amazon Pharmacy potentially. So no major changes happening right now, but just as a heads-up, we do have a pilot that we’re developing right now for the, sort of as we enter the CGM optional space, what can we do to alleviate overhead, move faster and potentially bring prices down. On the Guides v5, I think we touched on this last week. We had our Guides v5 Retro come out based on Sonja’s cohort. We’ve had a number of updates go live since that Alpha 1.
(03:48):
So, multiple stories are now live links, likes, guides profile, kitchen tab. Essentially a whole new suite of features that guides can leverage when engaging with their audiences. We’ve also got guides content linking directly into the food guide, which you can see down here, this is a huge press that happened this past week. And then a new Alpha 2 metrics dashboard, which is essentially looking at retention engagement, the major metrics of product market fit that we’re going to be tracking in each of these cohorts as we roll them out.
(04:17):
And then the team is in Miami, some of the product team cranking on guides content. Which you can see some of right here, Stacy and Mike. This week we had a homepage conversion rate record, we’re up in the low 5% range, which is a big improvement over the past few months. Fixed a major Apple Pay bug, which may be contributing here. And then we’re starting to build the Beta 2, CGM optional web-based signup flow. So, we’ve been doing janky signup flows through Typeforms and things like this. We’re now building a web-based signup flow, including the CGM optional experience where you can sign up for just Levels, kind of see it right here, or you can opt into trying a CGM.
(04:57):
So, we’re starting to take all that signup magic that Karen and the team have been building on the growth side and applying it to this new product strategy. We’ll obviously be testing this and iterating through a number of strategies here as we learn what converts best. But yeah, getting started on it now. We’ve also launched the sensor education page, which we got to be cautious about how much we share on this, but it allows people to actually see the difference between the different sensor systems, which we have not done yet. And then we’re running user experience research. Lynette’s taking the lead on this for a future referral strategy. So our goal is to be able to launch a referral program sometime soon. And then we’re continuing to experiment on prospecting and YouTube ads are really actually moving quite a bit faster than we’d expected relative to other platforms like Meta.
(05:47):
Finally, huge content and product lift. So the Levels food guide, this is 250 plus foods, which Azure and Haney and Caitlin have all been really building out at an unbelievable rate. I think also leveraging some external resources. So we’re building out this huge food guide, which will be part of the guides experience. We’ve got a whole new level with Molly Chester from Apricot Lane that went live with Casey. Content pieces on school lunches, fiber, new metrics dashboard for the content production that we’re doing. And some content growth updates, which will update the blog and include better email capture and some other elements.
(06:19):
So, it’s on there. We were also featured in Men’s Health 2023 fit list. Really great piece on Medgadget this week, kind of the leftovers of a PR press from last year, which is pretty awesome to see. All right, I think that’s everything. So, I see it’s Joey. Did Joey join us? Okay, awesome. Yeah, so Joey Roselli, welcome to the Friday Forum.
Joey Roselli (06:43):
Thank you.
Josh (06:44):
Joey’s a partner of Levels, I think we’ve been working together since October last year. Joey’s a content creator, focuses on what I like this tagline of, “Focusing on building healthy lifestyle, not just a diet.” Joey, would love to first of all, appreciate you joining us on a Friday morning. We’d love to hear a few thoughts, what you’re excited about in the future of metabolic health.
Joey Roselli (07:05):
Yeah, thank you. Thanks to Jackie for having me, and to you as well. My experience was incredible. Like I told Jackie, I think it was super easy to understand. However, the thing I would raise and a lot of messages that I got from my audience is that just because I understood it doesn’t mean that everyone else understood it. They didn’t understand the whole link and affiliation between measuring your blood glucose and how that plays a role in your health, and how it plays a role in your metabolism and how your metabolism works in general. I think that’s also a factor because my audience is not exposed to all that information from me just yet.
(07:43):
But I think the overall experience was easier to understand, again for me, but now it’s like can we, “dumb it down” or make it easy for even a 15-year-old to understand, if that makes sense. I like for example, when it showed there was a 53… Thank you, Lynette. Yes, easily digestible. Thank you. For example, when there was, let’s say, I think your first 24 hours, once the first 24-hour experience is over, there’s a article that appears for a suggestion, for recommendation, for a tip, for a little more in-depth guidance. And I think if we can make those articles a lot easier and not as lengthy for the user to understand, I think that would help a lot more as well, right?
(08:26):
Like, “Why did my glucose spike by 53 milligrams?” For example. Or, “What are the top 10 foods that can help when eating pasta, for example, to help balance blood sugar levels?” Those are the top two things that I’d raise awareness to from my experience.
Josh (08:43):
Yeah, I’m curious with your… So, you’re doing a one to many content production, which is one of the interesting opportunities and challenges here is that I think people learn trends from seeing someone else’s experience, their data, et cetera. But then they want to know how it applies directly to them. So, I’m curious your perspective on crossing that divide for people that don’t have a sensor on, for example. How to create that same level of engagement and interest without being able to be explicit with why that individual, or how that individual in real time might be responding to the food. Does that resonate? Do you follow what I’m diving into there?
Joey Roselli (09:21):
Yeah, a little bit. I mean, I think what you’re saying is not everyone understands the, I guess, the relationship of the correlation between the food that they eat and the results, right?
Josh (09:36):
Yeah. It’s a question of as we’re moving into this new product strategy where we have guides who are leading people through and showing their real time data from a continuous glucose monitor. And essentially teaching people the themes that sleep, exercise, nutrition content and even food order, those sorts of things. A lot of people can easily grasp the underlying concepts and then others want to know, “Well, what’s happening in my body right now?” And so, I’m curious, do you see themes where people are learning most… they most want to understand the foundational principles or whether they most want to see their own reactions in real time?
Joey Roselli (10:16):
I think definitely their own reactions because everybody responds differently to, for example, a plate of pasta. If you’re having just spaghetti ravioli, for example or just plain white pasta with not many fats or proteins surrounded in or accompanying it, you’re going to notice probably a surge in energy, maybe a crash later on or a midday crash. So I think really by measuring it through their moods and their own reactions and their own results, that’ll help the user better understand what foods I’m responding to, what foods I’m not. And how my body is even working with these certain foods.
Josh (10:54):
I think what’s interesting is of the people that engage with your content on these, the continuous glucose monitoring biosensing trend, what’s your sense on how many people are coming into the content understanding what a continuous glucose monitor is versus they start following the content because it’s unique and interesting and then they get interested in trying a sensor themselves?
Joey Roselli (11:18):
For my audience in specific, I’d say it’s small portion because I’d say the base of my content is recipes. I think I’d say a good three quarters of it, healthy recipes. So they think one would assume that a healthy banana and ice cream is a healthier option compared to traditional cookie dough ice cream, just for example. Which it could be, but your body might respond and react differently to it than a cookie dough ice cream. There might be more sugar because we’re adding three bananas versus one banana.
(11:49):
And so, I think because they’re coming from that end from audience, they’re not fully understanding what metabolism is, how our body takes certain foods and transfers it into energy and what their stay or blood sugar readings are, to begin with. So, that’s something that my audience needs to be exposed more to and to understand and learn more about.
Josh (12:16):
Yeah, it’s interesting because the arc that we’re building toward, and I think I look forward to exposing the new product strategy to your audience. Because I think a lot of people ultimately will first experience or see what A CGM does and is capable of doing through other people. And obviously people aren’t out there looking for continuous glucose monitors until they see someone else’s information and recognize why that might be useful for them. So yeah, I’m excited to sort of build that top of funnel.
(12:46):
I think you touched on it on the right off the bat, but what’s one area that you think Levels really needs to improve, whether it’s in the product or in the content, or where would you recommend that we focus some attention?
Joey Roselli (13:01):
I’m going to say this response based off the messages I’ve received, it was just the affordability component. And I don’t think that’s something that you guys can control. I mean, you could control ultimately, but you can’t control the reactions from others or from potential customers. Because not everyone comes from the same space or the same state of mind when it comes to investing in your health and improving your health. So for someone, they might think X amount of money is a reasonable price and the other person might think that it’s not.
(13:32):
So that’s a lot of reaction that I got saying, like, “Oh, why is it so expensive?” But it’s like, in my opinion, it’s not because it’s like, let’s just say for $200 you’re going to go spend that at a restaurant tomorrow night, aren’t you? But you won’t spend that for a yearly round thing, a yearly round membership.
Josh (13:50):
Well, it’s great insight and it’s definitely something that right now we’re particularly focused on is making it more accessible. Still actionable product, and then obviously driving the cost of sensor use down continuously. So it’s a great reminder. Joey, I really appreciate you joining, I really appreciate you being a partner to Levels and helping us get the message out. And if you’d like to hang out for the rest of the meeting, please do. But otherwise, appreciate you taking the time, the team really finds this valuable.
Joey Roselli (14:18):
Awesome, thank you to all.
Josh (14:20):
All right, thank you. Jumping into quick culture and kudos. So first off, I want to think Hao. Hao is going to be… His last day is on Tuesday, Hao’s been with us for almost three years and he’s been really just a pleasure to get to know and to work together with. Really graceful, talented person and taught me a lot about coffee, taught me a lot about backend engineering and really excited to follow along as Hao moves into his next chapter. And just appreciate everything you’ve done for us from the earliest days, Hao.
(14:51):
And then I want to just highlight the support and product teams. So this week, as I mentioned, we’ve both moved SLAs in a positive direction. So we’re just getting better on the support side with time and that’s a huge effort in and of itself. We also have Sunny, Lynette, Rebecca, Priya, all jumping in to help out on the guides, the product development process. From meal planning to guide submissions, just a ton of work back there and a lot of support to help us find product market fit, which is amazing.
(15:19):
And then down in Miami we’ve got a group of people, I think maybe it just wrapped up. But essentially we got a little onsite to film more guides content and make some big strategic decisions about the next cohorts that we’re working towards. So, I love seeing that progress and you can open the app this morning and you’ll see some of that content that was filmed just a few hours ago. All right, over to Nicole.
Nicole (15:47):
All right, thank you. So, just a couple of reminders, it’s been a very busy season for all of us. And so, we wanted to touch a little bit about some of the best norms and in general good practices around when we do have sync meetings. Our value around async work is as async as possible but not more. And in certain phases and certain times it might mean that that shifts a little bit. And so, we wanted to do some reminders about some documents that we do have out there in the realms of notion and then just touch on some of these norms. So, if you check out the levels.link/sync, there’s a whole document about this. But some quick reminders around sync meeting norms. So, try to plan ahead as much as possible.
(16:38):
And there’s also to a difference between one-to-one meetings and other decision making meetings. So knowing what sort of sync meeting you’re going to have will also dictate some of these best practices and norms. But whenever possible, try to plan as far ahead as you can and give people at least a day warning before you throw, or before you put something on their calendar, if not even further. Just in the nature of having flexible work or deep work, maybe that day, you want to be able to give people as much of a heads-up as possible.
(17:11):
If there’s a meeting and you think that there needs to be more than three, really consider if that’s actually what needs to happen. Once you get beyond three, it’s harder to make decisions and there’s also the best practice of a recording. And so, if someone might need that information but doesn’t need to be there to make a decision, think about if you can just record the meeting and distribute that recording for additional input afterward.
(17:37):
Start and end on time. Always try to join at least a minute if not more ahead of time so that you’re there and you can work through any AV issues. And then also really try to end the meeting on time and be respectful of everyone’s time and calendar. In the case of a brainstorming meeting or if there’s sort of an expectation that this might go over, that’s totally fine and just make sure everyone’s aware that they can jump off if needed. And that actually jumps ahead to another one, which is the permission to exit. So if you do find yourself in a meeting and you don’t necessarily need to be there, you are totally welcome to just slip away and that is encouraged and fine. And again, you can always catch that recording later too for additional context.
(18:24):
Make sure you have that clear focus and understand what the decision or the action item is to come away with. And that will also really help with making sure that things feel like they’re moving forward versus being static. And then, avoid recurring meetings as much as possible. There’s some companies that’ll do a recurring meeting, wipe off the calendar and restart or go bankrupt on them and start from scratch every quarter or once a year. But always question whether a recurring meeting is necessary. And if you do have recurring meetings on the calendar and nothing super pressing to talk about, let that person know and maybe drop that off of the calendar and go async if you can and do a Loom exchange or just a written update.
(19:12):
And then finally, always record and distribute the Loom of that recording. Recording meetings has been a really valuable thing from my perspective coming in and that not being a norm for me. I’ve found them so helpful so many times. So, make sure that that is the default and distribute them to anyone else that might need context. And then a few just best practices. If you can join and your background isn’t super noisy, go unmuted if you can. And that just allows for more casual conversation back and forth. And then, if you’ve got a meeting of more than five people, use the raise hand feature on Zoom. It’s very helpful. It helps the moderator be able to direct conversation. It’s helpful for those who maybe aren’t as… maybe their personality isn’t more likely to jump in. So you can use the raise hand and create that norm.
(20:07):
And then if you’re on a walk or if you’re just tuning in and you don’t have the best internet, you can put the read only or maybe an emoji behind your name in Zoom, and that will let us know that you probably shouldn’t be called on if that comes up. And then also finally, audio only, especially for one-to-ones is more than okay. Video can be taxing, and especially with all of us right now having a few more meetings on our calendar, it’s totally great to go just with audio or just over the phone or whatever feels best for you to meter your energy. So, check out that link and let me know if there’s any other questions or feel free to add more comments or suggestions to the document itself.
Josh (20:52):
Right. Thank you, Nicole. Yeah, highly recommend the walking meeting. I’ve been doing a bunch of those lately. All right, company objectives. So just a reminder, main thing we’re focused on is showing people how food affects your health. Everyone should be working towards this priority. If you aren’t, definitely raise that. And with that jump over to growth.
Ben (21:14):
All right, here is an update on Beta 1 conversion tests. There’s a retro super long Loom. You don’t have to check it out, but we’ll go over everything pretty high level. So next slide please. This is familiar to Riley Walker, a little hat tip. This is something we had at Skip, “More orders, no new work.” And so, this was a value that we had internally where whenever we worked with restaurant owners, that was our positioning, we’ll get you a bunch of orders and there’s no new work. So going into conversion tests, had to think about this value of how do we actually convert people? How do we create orders without it actually creating additional design and engine product work?
(21:53):
Next slide please. We needed a starting point. And so, it went to first principles thinking what would it take to get my mom or a friend or someone into the beta? And it’s pretty simple. You kind of just need their name, their email, and you need to be able to collect payment because we’re not handing over 10 bucks. Next slide, please. So, thinking about payment infrastructure, what do we have as an MVP? How can we start to collect payment for Sonja’s v5 cohort and Stacy’s as well, we are using Typeform with a Stripe integration to collect payment. And we’re doing a bunch of demographic data collection upfront. So we had this really long questionnaire where people would go through and it very much looked like a survey.
(22:35):
With this specific survey, there were 12 questions. And so we need to think about it like, “Do we actually need those questions to collect payment?” And the answer is no. You can see the average time to completion on this was 10 and a half minutes and the completion rate was 34%, which is still really high. That’s a very, very high completion rate. But the question becomes like, can we increase that number and can we reduce the amount of time? Clearly the answer is always yes, there’s always some way to do something. Next slide, please. So what we did is we had this hypothesis that we needed a landing page to drive traffic. Anybody that we were reaching out to for the beta conversions. We needed a landing page where they could click through to a checkout.
(23:17):
And so, after stepping back and thinking, “Hey, what if we just turn Typeform into a lander? Does that work?” And the answer is yes, we can do it. So this is just Typeform, but it looks very much like a landing page and a website. And part of it is building this trust with our brand. Anytime somebody lands on something, it’s like opening a door to somewhere that they don’t know where it’s going to take them and all of a sudden there’s this reveal. And so, when they land on something that looks a little bit janky, it might erode trust a bit. We’ve got a pretty well curated brand if you want to call it that. And we take a lot of time to think about all these touch points.
(23:54):
And so having somewhere that people land and they go, “This feels like Levels, it feels cohesive.” That was important. So you can see that by minimizing it down to name, email and payment, we were able to increase the completion rate to 70%, so doubled the completion rate. And one of our goals was to reduce the time to completion by a factor of 10. So we reduce it by a factor of 10, which is awesome. Next slide, please. So how do we start? Got to start small of course. So cold emails, personal outreach. Rather than going through Drip and just saying, “Let’s reach out to a thousand people.” Picked nine names from the nine most recent people on, I think that was February 22nd that signed up for an email list and reached out to them personally from my inbox.
(24:41):
And the goal was to test messaging like, what questions are they going to have about this thing? So that we could figure out whether or not our value prop or positioning made sense. And there were a couple of trigger words in there that we changed because questions did come in. So the first one was guides. There were questions around like, “Am I getting a PDF? Is this a person? I don’t understand this.” So that was a pretty good data point. The next was around four week journey. Some people ask, “Am I paying 10 bucks per week over the course of four weeks or is it 10 bucks for a month? I don’t understand.” So that was a pretty easy change where it sounds semantic, but changing to one month versus four weeks makes a big difference in framing.
(25:21):
And then even though it says, “You don’t need to wear a CGM.” People still ask like, “Hey, do I need to wear a glucose monitor?” The takeaway was we got to reiterate that a few different times and we got to make sure that people understand what it is when it starts and set expectations. So we changed guides to arbitrarily just as a placeholder to metabolic rebalance, so it sounded like a program. We started referring to guides as coaches, Levels coaches. And the four-week journey became a one-month journey or program I think it was. Next slide, please.
(25:52):
We got four conversions out of those nine. So that was early data point, but then scale up slowly. So you’ve got to find these breaking points when segmenting leads. So going into Drip, we’ve got 336,000 people in Drip and unfortunately we don’t have country attribution for a ton of people, specifically people in the US. So the best way to put a whole bunch of work, when we said no new work, whole bunch of work onto supports plate is to just reach out to people and they go, “Wait, this is international, I can’t get in the door.” So, thought through, what if we segment by Pacific Mountain and Central Time Zone? So that limits, we know we’ve got a pretty tight slice if we segment that way. And then how do we ensure that we’re not getting any of those sneaky Canadians in there?
(26:38):
So we did some geofencing on the short link, so that even if people landed on it, they would be redirected to the website. We’re still missing people in Eastern Time Zone, that’s not for the sake of beta. There’s a lot of people in New York who use our product. But this at least gave us some assurance that we’re not going to be creating a whole bunch of churn and work and people that are a little bit disappointed. So reach out to a hundred people, did this and it worked pretty well. Next slide, please.
(27:08):
Then you start A/B testing. We do this again with 100 and then 200, you keep scaling up. So first with subject lines, we wanted to see, it doesn’t matter what is in the content, it’s kind of like who cares what’s behind the door if people don’t even open it? So you want to figure out what’s going to make people actually open this door here. And so, we ran five subject lines and we figured that the exclusive invite one, we figured that out that that was the one that got the highest open rate. It ended up having an 80% open rate. And I think the click-through rate ended up being 20% on it, which is super high versus other ones like the $10 offer, didn’t have any click-through and it didn’t even have as an open rate. So there’s certain things of figuring out which one’s going to work. So, we figured out our subject line, then we went to content length.
(27:55):
And so, the hypothesis is, “Hey, I get this really long email like, ‘Abort mission.’ I’m not going through that thing because it’s way too much. The shorter one’s going to convert better.” Naturally, that’s what you think. But the long one gave enough context with the program, a product marketing video, and then some social proof as far as those little blue squares or testimonials of what people said about the previous programs. That was a winning email, so we ran with that for all the campaigns. Then the next was pricing, we wanted to see what was the willingness to pay of 10 bucks versus 20 bucks. We only ran pricing tests for very small numbers of people. But what we found is that $10 for bottom of funnel leads, so this means people who open the email converted at 8.3%, which is super high and 20 bucks was 6.1%.
(28:41):
So you go, “Okay, well that’s great.” Maybe, but $20 actually has a higher leverage point on revenue. Because even though it’s got a lower conversion rate, you can find that 10 bucks has a 45% lower impact on overall revenue. And the caveat to all this, we’ve got to take these data points in isolation because they’re small sample sets, they’re not statistically significant. We don’t know how that extrapolates into the future for retention and all these things. But again, it’s data points and so we’ve got some now. Next slide, please.
(29:13):
So results and outlook. So after five campaigns, we filled the beta with 336 people. As of last Tuesday, 268 people were new leads, 68 people were retained from Sonja’s v5 cohort and our target was 250 people. So everyone’s going like, “Is this good? Is this bad?” The takeaway, the main thing is that the conversion test gave us a good early signal, but not a significant gauge to extrapolate growth moving into general availability.
(29:41):
So even if it was super high, we don’t want to say, “Hey look, this is going to have the same impact as we reach out to more and more people.” We have to look at conversion, with retention, with engagement, with all of these other metrics as quantitative input, plus qualitative feedback that we’re getting as things evolve and get tighter. So yeah, our target was 10% for a strong signal, and so being 1.7% below, we didn’t hit that target for a strong signal. But this is based over, we’ll just call it 4,800 leads and pretty good signal. So, happy with that.
(30:17):
Next slide, please. Post-acquisition. So gardening emails super important, and why? It’s to mitigate the doorbell effect. So if somebody signs up three weeks ago and they’re ringing the doorbell, they’re going, “I know someone’s there, but no one’s answering.” And we don’t want people to start to either lose trust or build resentment towards us if we’re not setting expectations and keeping them warm. And so, every Friday we’ve been sending these emails that are basically metabolic health primers or what to expect with the experience. And these are the emails that have been getting… I think the first one is the one that had a 92% open rate, which is mind-blowing. It just shows that people are very hungry for this information.
(30:57):
The second is doing proper onboarding. So, doing a handoff between growth and product to make sure that these leads are now being onboarded and their expectations are being set as far as downloading the app and what to do next. But the reason to do this is, within growth, we could do all this onboarding and take this on. There’s still a lot of shared responsibility between both. But if we take on this work, then we’re not focusing on our area of impact, which is bringing more people into the future betas and starting to think through that. So everything has to work in sequence. So, it’s important to have that very clear handoff and make sure that we’re doing everything from the Typeform surveys to the emails that go out. So, it’s been a great way of working together between product and growth.
(31:39):
Next slide, please. So the main takeaway, the silly analogies that always come up. Somehow, our whole family has grown up fishing. You can see Pam’s on the board, Theo, Penny, Zoe, they’re all there. Pam’s even pregnant in a bunch of those pictures. But one and done never does it. So you can’t send one email just like you can’t take one cast and expect that you just land a fish. Fishing is very similar to email where you can change the lure, you can change the structure of the water depth. There are all of these considerations and you have to keep tweaking and learning and tweaking and learning, and it takes a lot of patience. So, we’ve got lots of fishing to do and lots of casts to take. So that is it. That is Beta 1 conversions.
Josh (32:22):
Nice. Great dive, Ben. Thank you. Yeah, looking forward to seeing how this evolves, obviously as we’re still super early. But building these signals as early as we can is the move. All right. I think Maz is going to lead off your own product.
Maziar (32:41):
Thanks Ben, that was really great. Love the fishing picture at the end. Welcome to the product update of March 24th. Here we go. All right. So Beta 1, it’s coming together, it’s about 10 days away and the team has been cranking all across the board. Whether it’s engineering, whether it’s product design, support, everybody, editorial. And we’re getting ready to test two plans. And the reason for this, if you remember, is that we want to see what will resonate the most for members and where we can get the most traction and drive the impact that we desire. And from this test, then we can decide which direction to invest more in and go deeper versus wider. So really, Beta 1 is set up to give us signal to know where to invest our time to develop these plans.
(33:31):
And so, the plans are effectively the two side of the spectrum. Plan one is basically metabolic baseline, or 101 or reset. It’s basically to teach you the fundamentals and to encourage you to adopt some of these metabolic health principles, and to make small changes that will lead to big impact over time. But it is slower and it is less commitment and it is a lot of the basics. And it’s really towards people that want to avoid disease longevity and it’s a lifestyle type approach. But two, on the other hand is the other side of the spectrum is one that’s higher commitment. It’s actually targeting weight management. And the way we position this healthy and strong, it’s basically focused on losing body fat and building muscle and strength.
(34:20):
And that program is higher motivation and higher specificity. So we are going to test these two approaches. One is led by Stacy and the other one by Sonja. And we will see where we see more traction and how do we learn to make these programs better and more effective based on Beta 1. So, all that is coming together. There’s a huge effort going on to pull all this together for April 3rd. Next slide, please. All right. So, the team has been working very hard to create the different assets and I think some of the slides are missing.
(35:01):
Let’s see, I think you’re missing one slide. If you can go… Yeah, there we go. Yeah, that’s the one. Perfect. The team has been working really hard to create the assets that will really power our plan. So really big, big thanks to a cross-functional team, Azure, Caitlin, Haney, Mike D., Alan, David, Sunny, Cissy, and a lot more people that are really creating a lot more depth into what we create around our meal guidance or food guidance, which is the core of our experience. Really putting together these beautiful guides that will eventually end in the Apple, inform the app and create the assets for the app that will help people understand what they should enjoy, what they should pair and what they should avoid. That’s really the Levels food guide that will create the basis for how people should eat.
(35:51):
The second one is meal planner. So now that I know what ingredients are good, how do I actually put it together and how do I actually create a plan for the week in a way that it’s flexible, specific? Which means we don’t tell you what to eat specifically on this day, but these are some suggestions that you can understand the principles and you can pick from. So, it provides both the specificity but also the flexibility that fits in people’s lives. And then the food swaps, which is part of our core to tell people that, “Look, you can still enjoy really good food and here are some great swaps that you can employ in your diet so that you can eat the stuff that you like but have a stable glucose response, and the nutrition density that we want people to have.”
(36:34):
So, all this has come together and it’s been a huge amount of work and it’s looking great. And I encourage you to go check them out, they’re all in notion. Next slide, please. All right, another piece that we’ve been talking about for a while now is behavior change and developing the habits that will help us get healthier. And so, the team’s been working on putting some of these habits into the framework of, how do we make habits small and build on them? And there are a host of habits that now we have that the guides will actually encourage people to adopt. One or two, we can practice them based on what works for them and what they prefer to really help them improve how they eat and eventually their activity as well, and sleep.
(37:19):
Next slide, please. All right. The final piece. So metabolic health is how you eat and also how you move really matters. So we’re actually bringing a set of principles of how we should help people be active. And we’re putting it together based on some of the best research in the world to create a program that again, is specific but flexible, meaning I know what to do because I really want an intelligent human to put a plan together for me so I don’t have to take on that cognitive load to figure out what to do when I want to exercise. But also gives me the flexibility that if I miss a day, I can easily search around my schedule and focus on the things that I need to focus without being slaved to a specific schedule or set off directive. So, we’re creating a plan that is science-based, that is specific and that is flexible. And has been a lot of effort on that too.
(38:10):
So, basically if you sum it all up for the plans that we’re launching, we’re going to have clear guidance around food, clear guidance around activity, and also all the great metabolic health principles that we have put together over the past few years and do it all in a way that can be built with simple habits that can be sustainable to improve our health. So, all that work is going on and people are really excited to launch this to our customers. So stay tuned, at least 10 more days to go. Thanks, Josh. And over to David.
David (38:46):
Thanks Maz. All right. Can you do one more refresh, Josh? I’m not sure if this one came through. I think we’re good though. All right. So as Maz previewed on the beta testing, I just wanted to reiterate that for our beta tests, we want to really maximize learning for what that effective program looks like and then how we implement that. That’s what we’re going to be focused on as we look towards Beta 1. I wanted to give a quick Alpha 2 update. But in order to understand the alpha two update, it’s really helpful to understand what we’re doing in Beta 1 because Alpha 2 is all about teeing up Beta 1.
(39:19):
So in Beta 1, we want to test our hypothesis that a guides-driven experience will be engaging for users. And we also want to test another hypothesis that our systems approach to the core software only features is effective. So essentially what Maz was talking about tying in all the food guide recommendations that you’re given from the guide, easy tie-ins with the logging, that entire loop that we’re [inaudible 00:39:50]. Our alpha experiments are other hypotheses. The next slide, Josh. And so far what we’re seeing is good engagement based on where we expect it to be for Alpha 2. So, one of our main metrics for Beta 1 is tracking engagement as a proxy for retention.
(40:10):
And our goal for Beta 1 is going to be 50% of people open the app at least two times per week. And so far we’re two weeks into our Alpha 2 test, which is with Stacy’s cohort from Instagram. We were seeing that 70% of people so far in the last week have opened the app at least two times in the last week. And we’re seeing overall fairly good engagement on that end. With Alpha 2, we also have this hypothesis that we’re not going to see as much engagement as we will later on as we keep iterating and building in the pieces of the app that we think are really core components to having an engaging guidance-led experience that we saw in Instagram.
(40:46):
Our hypothesis was that the app experience and levels for Alpha 1 and 2 would feel lonely compared to what it did in Instagram, where they had a full suite of features already built out for Sonja and Stacy to engage with their cohort. And that’s sort of what we saw in the first couple of weeks. So, Stacy’s been posting daily videos and they’re being well received. But we’re seeing that some of the features that we have planned for Beta 1, we think are necessary to bring in to up the engagement with that guidance video in the app context versus Instagram.
(41:17):
So, we have… 64% of people are engaging with the videos at least two times per week in the app and the first week of Alpha 1. And in the second week of Alpha 1, we’re seeing a little bit of a drop for that one, down to 43% so far, that week is still ongoing. But I think this is as expected, we didn’t expect to have perfect parallel with what we saw on the Instagram engagement when we first launched this into Alpha. Knowing that our table stakes set of experiences is really coming for Beta 1, if that makes sense. So yeah, I think overall we’re seeing higher than expected engagement with app opens compared to where we want to be even next month. So, let’s keep that up. And I’m excited to get to some of the features that we’re piloting to bring in more of the community feel into the experience where it’ll be interactive polls, ability to send guides, feedback messages. So that the guide can then further reflect that back to the audience. Better feed, feed ranking and experimenting with other things like community comments. So that’s it for Alpha 2 update.
Josh (42:26):
All right, thanks David. And thanks, Maz. Yeah, cool to see this arc building. And in the background we’re continuing to iterate again on the CGM optional elements and looking forward to seeing that launching in parallel. So just a reminder that we’ve got a lot of exploratory stuff happening simultaneously and it’s sometimes difficult to keep tabs. But I definitely recommend having in mind these major launch goals and the objectives below them. So Alphas and then moving into Beta 1 where we first start our retention goals building towards two, three, and ultimately general availability. So lots going on, super excited to continue to see this paying off.
(43:09):
Okay. I’m going to jump into open roles. We do not currently have open roles at Levels, but if you’re interested in submitting a future application, checkout levels.link/careers or forward along to somebody that you think would be a good fit for what we’re doing here. And we got a forum survey here, which I think we send out in comms. So no action to take there. All right, we got some time for individual contributions. So I’m going to stop the share here, shift over to this mode. And yeah, I recommend you can use the emoji reaction down at the bottom, raise your hand and we can share a little something. Chris, you beat me to it. Go for it.
Chris (43:55):
So first off, I want to welcome Carrie to the team. Super excited to have you on board and hopefully you’re around when we’re having our offsite in Austin in early April. So, I’m looking forward to meeting all the team there. Second, want to do a big happy birthday shout out to Mr. Flanagan. I forgot to get that in the slide deck. So, I would ask everyone to sing him a happy birthday right now, but I don’t want to embarrass the guy. But make sure you wish him a happy birthday, which I think is this weekend, probably Sunday.
(44:25):
And then lastly, I would say it’s super exciting to watch the speed or any kind of velocity with the product and all the alpha, the betas. I think when I saw the initial plan of attack of moving towards the June 1st date, I looked at it going, “Wow, that’s really aggressive. There’s no chance we’re going to hit any of those dates.” Just in terms of just seeing a lot of teams put aggressive dates on there and then having to slip. And not only is the team hitting the dates, they’re pulling additional work forward. So multiple times of the, “Hey, we are going to do this in Beta 1, and we decided to test it in Alpha 2.” And that’s now becoming a trend, so they’re actually increasing even on top of their aggressive dates, and that is almost unheard of.
(45:11):
So, just a huge hat tip to everyone involved working quickly, fastly. I know it’s a lot of, “Is everyone on the same page?” But it’s just super exciting to watch. So, thank you everyone involved.
Josh (45:26):
Yeah, I’m going to sound like a broken record here on Chris. But yeah, extremely excited to have Carrie on the team and spent Tuesday and Wednesday down in Austin, got a chance to really jam with the team. That’s been awesome. And then to surface and catch up on everything that happened this week is blowing my mind. I was digging through for hours last night, hours this morning just to catch up on product and eng and guides growth. All the work that’s happened over the past few days, which is sometimes hard to believe. So, just…
(45:59):
I didn’t even know the Miami thing was going on until this morning and I saw the content pop up in my app. And so, it’s pretty amazing. I just want to celebrate everybody’s efforts. It’s great to see. Scott?
Scott (46:10):
We just go now?
Josh (46:18):
Yeah.
Scott (46:21):
Yeah. Nothing new to add. I just feel like there’s… last eight weeks, I feel like just a renewed sense of rigor and movement and it’s felt like everybody on the team is rallying around it. I think we’re seeing it in product or feeling in product maybe the most. But I feel like everyone’s responding really well, which has been really cool to watch. So, just call that out. I want to wish a happy birthday to Tom. Birthday’s tomorrow. Happy birthday, Tom. It from me.
Tom (46:48):
Thanks, Scott. Oh, Scott’s birthday too. I forgot we share a birthday, Scott.
Scott (46:57):
We did this last year and I’m pretty sure I’m going to do it to you every year now.
Tom (47:01):
That’s hilarious. I totally forgot. I was like, “Why is Scott wishing me a happy birthday?” And then like, “Because he wants a happy birthday, that’s why.”
Josh (47:08):
Happy birthday week for the team. Priya?
Priya (47:14):
Hi everybody. Okay, so personally I’m really excited because I’m going to try to sell my Tesla this weekend, I inherited it in my divorce. Most of you know that I’ve had so many issues with it, so I’m really excited about that. And then workwise, I did something really hard for me this week, which was ship something that didn’t work out. And I know Chris, in the beginning when I first started and the queue was like, “Priya, you got to be okay with making mistakes here, or things not working out and stuff like that.” And so, it was really, really hard for me to ship this guides audition that didn’t work out of a life event that happened this weekend. And so, I just really appreciate all the support that I got from everybody, which was… it was really hard to send out something like that that didn’t work out.
(48:00):
So, I just wanted to say that I’m still really excited about guides and that I’m super excited to support from behind the scenes for now. But thank you for encouraging me even though it didn’t work out the way I wanted it to. So, yeah.
Josh (48:14):
Appreciate that you still went for it, Priya. And sorry to hear about the Tesla. It’s a revolving door, I actually picked one up last week and I’m loving it so far.
Priya (48:24):
I know, I think I’m in the minority there. But we’ll see what happens.
Josh (48:32):
It’s all good. Hai?
Hai (48:32):
Yeah. First of all, I definitely share the sentiment with everyone else here. So excited about so many things happening. And welcome Carrie. Specifically, I also want to say I’m so proud of the engineering org and everyone on the team. There’s so much momentum, velocity, sense of urgency, sense of ownership. Yeah, I’m just so proud. And on personal side, I’m going to throw a birthday party on Sunday for my son, seems to be birthday season. Yeah, so happy birthday to the people who are having birthdays.
Josh (49:04):
Nice one. Tom?
Tom (49:07):
Yeah, I mostly just wanted to say that it feels great to be back. I haven’t been on a Friday Forum in a few weeks, I was in Africa for two and a half weeks. And appreciated a bunch of people stepping up while I was out. And man, do you really feel how fast this team works when you’re out of the country and on vacation. The people I was with just couldn’t believe how much work and documentation was piling up when I’d give them updates on what it felt like to catch up. And people were just sort of looking at the… friends who also work in startups, were looking at the Looms and the documents being produced and the work happening. And they were like, “Your team is out of their minds.” And I said, “Yes, they are, in the best way.” So yeah, it feels really good to be back and the speed, velocity has just been incredible.
(49:55):
Oh, also shout out to Mike D. I opened the app this morning and candidly, I’m not really a social media user. And so I was skipping through the videos and not taking them super seriously. But Mike D. was showing me how to make chia pudding and I stopped and literally made it two minutes after watching it. And all jokes aside, it was a really fascinating aha moment for me with the app where I’ve never… I’ve never been a recipe person. But just seeing the visuals of someone jumping right into making something in such a practical way, I couldn’t believe that I took action on it immediately. So that was definitely a product aha moment for me this morning.
Josh (50:38):
Mike’s making Tom a man of action. I never thought I’d see it. Just kidding, happy birthday, Tom. Ben?
Ben (50:46):
Yeah, stoked to have Carrie join the team, very excited about that. Super stoked this week on watching all the checkout work come together too. There’s so much collaboration across product and growth and all these things going on. So hat tip to Karen and Scott, and Dalit and Kosman. There’s just so many people contributing to it and it’s kind of fun to watch it all sort of happen in Figma or happen in whatever platform these iterations happen. And so super stoked on that. And then personally, Maz sent me probably two or three weeks ago this new app called Street FC. And it’s basically pick up games for soccer. So even though I said I’ve hung up the boots and I’m no longer playing, went on Nike and ordered a pair of boots.
(51:32):
And so, I played on Wednesday night and it was actually super fun to play with random people and it was quite competitive and enjoying that. So probably do that every couple weeks and keep it casual. But that’s it.
Josh (51:45):
For the non-Canadians. I think Ben’s referring to cleats when he talks about boots.
Ben (51:50):
It’s what they’re called in the UK too, Josh. Thank you.
Josh (51:53):
Well, sneaky Canadians, was the topic of this week.
Ben (51:57):
Karen’s got my back, it’s okay.
Josh (52:00):
All right. Any other… We’ve got a couple minutes left here, Rebecca?
Rebecca (52:05):
My win is that I made guides videos as a tryout and submitted those on Monday. And it was really fun, it was energizing. And after I was done, it was kind of like I was in this place of, it doesn’t matter if I show up in Levels like this or just in my own life. But showing up and creating that was just so… it was an opening for me. So I felt really good about that. Just really excited about the direction of product and the guides, and I think we’re doing really big things. Also, to echo what everyone said, just like everything’s moving at such a fast pace and I feel so in it and it’s really awesome.
Josh (52:45):
All right. Happy to end on that note if nobody else wants to jump in, but it’s as good as any. Cool. Well, yeah, thanks again everybody for crushing it this week. Looking forward to continuing to accelerate and have a great weekend.