Josh Clemente (00:00):
All right, let’s do it. Friday Forum, February 10th, 2023. Let me know if anybody’s having any trouble seeing the slides. I think they’re up. All right, cool. Friday Forum, just to reiterate, weekly sync time. We’re celebrating our achievements across the company. Everybody can hear and see what’s happening in different functions and celebrate the culture and the progress. This is not a deep business analytics dive. This is not where we do our primary meetings, looms, decisions. This is all about a retro on the week, just a consistent reminder there. To jump straight into it for this week, couple of really strong signals on the conversion front, on the growth side. Our conversion rate is starting to stabilize at around 4%, which represents almost a 2X improvement over a Signup 2.1, which was already an improvement over where we were just a few months ago.
(00:58):
Specifically in there, we’ve got mobile conversions doubling just in January. We’ve made some really great improvements. Big shout out to Karen and the growth team, growth eng team, but improvements like just Apple Pay and faster payments right in mobile and mobile optimizations really dramatically improving that flow. As you can imagine, if you’re checking something out on your mobile phone and then it’s not optimized or you can’t pay easily, you’re not likely to remember when you get back to your computer. This is a big improvement. New members are also showing a 20% week-over-week increase, which is a strong signal. Obviously, we don’t want to over index on week-over-week trends, but really good to see these sorts of things. Growth products, that’s kind what we’re calling the elements of growth, working with product to improve our funnel. Crystal clear pricing is shipping next week, so a lot of great work on that and how it works, which is a flow.
(01:51):
You can see some of this over here, the intention to show what the Levels product really is and how it works. That’s in work should be another couple of weeks before that goes live, but just continually improving the way that people can enter the funnel and understand what Levels is and what they’re getting and how the pricing plays out. Our remarketing ads, this program’s been going on for a while. We’ve tried a number of different things, user-generated content, as well as white labeled content. We’ve also tried some new prospecting ads, which is actually trying to find new members for the first time as opposed to remarketing to someone who’s already heard about Levels. The remarketing ads are doing super well. We’re well below our target, right around $100 for a cost acquisition, which is great, so great [inaudible 00:02:34], especially since we’ve never done this really before.
(02:36):
We’re updating our new product partnership strategy. In the context of the CGM optional, we are starting to improve our thinking around partnership strategies, how will we adapt into the really Maureen-focused world and this new product that we’re building. That’s great. Then the UK wait list is growing. I think we’re at about 7,000 people right now. Attribution analytics are really strong there, so paying close attention to the inflection points that happen around specific events like we’ve had a couple podcasts go live, we’ve had some great coverage this week, well really this month in the UK on some luxury brands covering us and a couple of different editorial moments, which we can see those attribution analytics and we’re starting to understand what’s working really well over there as we build out that wait list.
(03:21):
Peter Diamandes was on a whole new level this week. We recorded this a few weeks ago, but this is a really great episode. I super enjoyed having a chance to speak with Peter and definitely recommend checking that out. It’s awesome to have him in our orbit, no pun intended. He’s an aerospace guy. Then R&D internal update. Also, we did a bunch of work on this past week and recorded a couple pieces which we’ll be going out. I think we’ve already released one to the team, so definitely check that out. It’s a good dive into what we’re up to over on R&D, we had some huge numbers put up this January on the digital side. Big shout out to the editorial team podcast team. Really all the digital work that we’re doing, we had 291,000, so approaching 300,000 blog views in January. That’s an all time high. YouTube views were up 68% subscriptions were up 48% and both podcast feeds the whole new level, and Metabolic Insights had record plays 49% for whole new level, 138% increase for Metabolic Insights. All really awesome numbers.
(04:24):
Just to make sure I increase the volume on this one, Tom is going to get a tattoo of the Metabolic Insights blog once we hit 500,000 page views in a month. Everyone, hold him accountable for that. We published a strong response to the new childhood obesity guidelines, which I really loved. I just want to definitely highlight this article. The team did an amazing job taking something that really does not… It’s not grounded in the science that we understand and responding to it in a very articulate and cohesive way. I highly recommend reading that. It was really inspiring to see. Editorials also diving into the Guides Program Content support email series, just helping across functions to make sure that our cohesive and world-class content is making its way into the hands of our members.
(05:11):
We had a data outage this week which drove a big spike in support volume. I just want to extend some love to the support team who had to deal with a big increase, something that was entirely out of our control and did so with Grace. As always, the autonomy pilot is still going well, which is that pilot where we’re scaling with contractors, our support team. Then we’re exploring additional logistics opportunities to distribute and fulfill orders. There are some exciting new products coming down the pike, which we may be able to leverage with our product. I won’t go too deep into that, but I think everyone knows what’s going on in the world of CGM. Then 2023 financial plan is released, so shout out to Riley for that. We’ve got a finance a next week, which I’m going to add another note on in just a second.
(05:54):
Then we’ve also completed the process audit for the reviews and performance reviews. We’ve got a bunch of actions there, lots of stuff coming that way, so thank you to everyone that helped out there. A meetup’s 2023 memo. Check that out. Obviously, we’ve got a couple of big meetups coming over the next few weeks and I think I guess through April, so definitely check that out to see how we’re evolving our thinking on that. Pods are live, so we’re trying out this new concept of engineering Pods. These are live for growth engineering for the Guides project and for the new CGM optional product readiness. Awesome work to see this come together. I’m really excited to see what we learn. We’re also shaping a CGM optional Alpha with Stacy’s followers, which will be in app, so moving very fast towards being able to test the very basics of this.
(06:40):
Then Levels levels 1.6. You can see some of the visuals here where we are testing simplified dynamics, so going towards completing checklists, which is based on member feedback as opposed to, well, just evolving in that direction from the early game dynamics. I think that’s obviously some great content being posted. You can see some of the visuals here on all the record numbers we’re seeing. Overall, a couple of really great shots of Sonia as she starts her Levels Guide project. With that, I want to welcome Elissa Goodman, Elissa’s on the call with us. She’s a partner here at Levels. She’s an author, she’s a nutritionist, and she’s on a mission very, very parallel to hours. She’s been working with Levels for about five months now to spread the word, share her our story, our mission through her lens to her audience. Elissa, just can’t thank you enough for taking some time to join us this Friday morning. We’d love to hear a little more about your experience and what you’re excited about in the world of health and metabolic health specifically.
Elissa Goodman (07:44):
Thank you for having me. Yeah, I’m very excited about Levels, first and foremost. I mean, when I used it last fall, when I first got it, I learned a lot about my blood sugar issues. I always thought, “I don’t eat a lot of sugar. I’m very conscious about what I eat,” of course, being a nutritionist and I know it’s supposed to be protein and fiber, and just again, I’m not a big sugar girl, but it was fascinating to see some of the things that happened to me when I woke up in the morning and had my coffee and then went to my green juice, which I usually do with no sugar. My blood sugar shot up like crazy. No one could figure out why though, which was fascinating. That was, no one could figure out, it’s just celery, cucumber, lemon, ginger in that juice.
(08:44):
That was interesting, but I noticed also when I had a high-carb meal before bed, I would wake up at two o’clock, three o’clock in the morning with a blood sugar drop. That probably has been one of the biggest things for my clients to learn, because a lot of people are wondering why am I waking up at two and three and not being able to go back to sleep, but their blood sugar’s dropping and their cortisol’s kicking in and keeping them up. I was totally fascinated by that, mostly by that part. Now, clients are very conscious about the food that they’re eating before they go to bed, making sure that it’s not food that spikes their blood sugar and then they’re not waking up at two and three in the morning, which has been amazing. I just think it’s revolutionary to be able to really monitor your individual processes of what you eat and what’s happening with your blood sugar.
(09:55):
It’s been life-changing for my clients in terms of their blood sugar levels. Also, a lot of them have been taking a supplement while they’re using the Levels and it’s called berberine. You guys, I’m sure, know about berberine and that’s been really fascinating to also regulate their blood sugar. They’re doing 500 milligrams three times a day in between meals of the berberine, and that’s helped lower blood sugar levels as well as just noticing what foods are raising their blood sugar and not.
(10:34):
I would say that, I mean it’s just incredible to be able to have something that you can firsthand wear and see on your phone right away what is raising your blood sugar and what isn’t and keeping yourself… That is a huge issue for my clients when I look at their blood levels, their A1C and their glucose levels are way too high, most of them, all of them, and they really don’t know what to do to lower them. That has been huge, especially women in the menopause years. Now that they can really monitor their blood sugar, they’re also being able to lose weight more easily and it seems like they’re regulating also their hormones a little better. Even their thyroid and their adrenals, everything is falling into place a little bit better because the sugar levels are being more monitored and they are staying away from the stuff that is raising their blood sugar, or they’re eating it in different order as we know basically, eating carbs [inaudible 00:11:48].
Josh Clemente (11:50):
Yeah, it’s so interesting, because I think the group that you are working closest with and who you’re talking about a lot here is really the core persona that we’re focusing on in this current stage of product development. Just helping to provide additional just transparency into awareness is a great way to put it, metabolic awareness, how is food affecting health and in real time for people who are at that stage in life and are typically dealing with some level of confusion or contradictory information and just having that clear lens into what’s happening in real time. I know the juice thing reminds me of one of the early moments in Levels history where I grabbed a pressed juice from a juice cart and it was just apple and celery and I think carrot juice and my blood sugar was well into the 200s, and that was one of the big moments that was really shocking to the team that was around me. It’s like, “Wait, that’s objectively healthy.” That’s the really, I think, compelling thing about this product. I love hearing these examples from you and from your clients.
Elissa Goodman (12:50):
Well, I mean, I have a question about that, because my juice doesn’t have any apple, doesn’t have any carrots, doesn’t have any [inaudible 00:12:56]. I guess it’s just the natural sugar in those veggies, right, that’s probably doing that?
Josh Clemente (13:01):
Our working assumption is that when it’s stripped of the fiber, the digestion is so much faster that even though generally if you eat that whole vegetable, it’s going to have a completely different response. It’s just that processing element. That’s what we’re working with right now.
Elissa Goodman (13:15):
If I blended it, it’d probably be a whole different story, but since I juice it, even though it doesn’t have the sugar in it, so it probably has some sugar from some of these veggies, but very little. But that’s what was really shocking. I know because in LA we are juicing a lot, we are doing a lot of that. We are going to [inaudible 00:13:37] picking up juices and creation and all, and it’s a great way to get nutrients in to your body on an empty stomach. But yeah, that was very enlightening.
Josh Clemente (13:49):
Well, if there were one thing that from your experience thus far that you would hope that Levels would focus on build or fix, could you share product feature? This can be an 11 out of 10 experience that… Doesn’t need to be grounded in reality, just something you would hope to see.
Elissa Goodman (14:07):
I’m not sure if I can answer that, because I feel like when I used it was last November, so I used it for three months. It was September, October, November. I thought there was some glitches, but overall it was a great experience for me. I’m sure you guys are going to be really fine-tuning everything. I know I love the recipes that Levels is offering, just the health tips, all that stuff is so incredible, the easy things that people can grasp to help them. I wish I could answer that question better, but it feels like it was a little too long ago.
Josh Clemente (14:59):
We’ll have to add more information around the pressed juicing. Maybe that’s an option to… Yeah, that’s really where we’re focusing, honestly.
Elissa Goodman (15:07):
Juice without fiber
Josh Clemente (15:08):
Showing swaps and showing alternatives and relating it back to what other people in the community are doing. Yeah, excited to get you back in the product.
Elissa Goodman (15:16):
Well, even some of my clients that are doing healthier sugar options with their coffee, that too has been interesting. They’re putting monk fruit in instead of some other sugar and it’s still raising their blood sugar, even though it’s a low glycemic. That was fascinating too. Then also some of the really healthy nut milks, but they were raising their blood sugar. Have you seen that?
Josh Clemente (15:43):
I personally do see a blood sugar elevation from caffeine alone. I think there’s just tons of [inaudible 00:15:49].
Elissa Goodman (15:48):
Is that just individual?
Josh Clemente (15:51):
It’s unclear. I think that is likely individual, the enzymes that we have to break down these sorts of things similar to alcohol.
Elissa Goodman (15:58):
Does that mean if we’re a fast caffeine metabolizer or a slow metabolizer or we don’t really know yet?
Josh Clemente (16:04):
Not clear. This is why I think we’ve got the more people that are generating this data and sharing it amongst each other, I think the faster we’re going to start to be able to understand these things. I really genuinely appreciate you setting some time aside this morning. There’s so much more here. I can’t wait to actually have you jump into the next version of the product that we’re building out that’s going to be able to really incorporate this community stuff.
Elissa Goodman (16:27):
Okay, thank you.
Josh Clemente (16:28):
Yeah, thank you again for showing up. The team just absolutely loves hearing directly from people like yourself who are working with clients. We have a bit of a packed meeting, so would love to have you stick around as well for that if you’re interested. Otherwise, on behalf of the team, thank you so much for showing up, Elissa.
Elissa Goodman (16:43):
Thank you, Josh. Appreciate it.
Josh Clemente (16:44):
All right, thank you.
Elissa Goodman (16:44):
Bye
Josh Clemente (16:45):
Bye-Bye. All right. Coming ahead quick, culture and kudos aside here. Firstly, I want to extend a shout out on the Pods program to Scott and who are really taking to lead on two of the brand new Pods, spun these things up and are just charging ahead on these initiatives. This is an entirely new way of coordinating projects and efforts that we’re doing internally. Scott, Ian, I think Maxine is the lone wolf over on the growth end side right now, but also taking on a POD program there. Thanks to all of you folks for basically trying something brand new, and there’s going to be some symbols along the way, I’m sure, but right now things are looking fantastic it sounds like on the Pods Initiative.
(17:30):
Then we’ve got an upcoming finance ama and I just want to highlight this and make sure that especially team members who joined in late ’21 or early ’22, check this out, because there’s also going to be some info about option investing, exercise eligibility and tax implications. We’ve got some resources available. I would just recommend being in touch with Riley about this stuff, checking out the financial plan, just generally getting a feel for what’s happening there, especially if you have these exercise options or exercisable options coming available. There’s a lot of resources we can share and don’t want this stuff to fly under the radar.
(18:04):
I think that’s that company objectives. Levels shows you how food affects your health. No major changes here working towards this priority as priority number one, still in Q1. I think we are… There we go.
David Flinner (18:20):
I think, sorry, did you want to do the growth one first or the product one, either way?
Josh Clemente (18:26):
I think we’ve got growth first, so why don’t we just jump into that and then we’ll loop back on product?
David Flinner (18:31):
I just have my slides in another deck, so up to you.
Josh Clemente (18:33):
Okay, cool. Great. Let’s run through the growth one and then we’ll hand it over to David to share that one.
Tom Griffin (18:39):
Sweet. Okay, growth update this week. Next slide. All right, first, when thinking about what to present here, I feel like there are a few different types of team updates on forum. There’s the high level 30,000-foot view of how a function works, how to think about growth. There’s an update on priorities, there’s more of the tactical experiments and learnings and results, and then there’s metrics-focused updates, how are we making progress? How are we doing on our KPIs? This is going to be mostly number one and number two. Then there will certainly be more of three and four to come in the future. Next slide.
(19:18):
Some more context. A few things have happened since I’ve done one of these real high level updates on demand capture or growth as we’re now calling it. First, around December we aligned on a new company strategy and really just acknowledging that we’re not currently likely on a path to product market fit with the current product. As a result of that, we aligned on at least the need. Then eventually we built out a new product strategy focused on, as we all know, CGM optional software first, more affordable. As a result of that, we had a slight shift in our Q1 demand capture priorities, which I’ll talk briefly about next. Then the most recent update was just that we took demand gen and demand capture and put it under one roof called growth. Next slide.
(20:05):
In terms of these Q1 demand capture priorities and the shift that happened here, the TLDR is that we aligned on our first priority being to prepare to sell the new product, and our second priority, albeit still a priority but not quite as important, which is sell the current product. Next slide. Fortunately, there’s some overlap between these objectives. When thinking through this on the far left side here in terms of objective number one, there are things that clearly fall into the bucket of just preparing to sell this new product. Examples of this are building a new signup flow so that people can buy this new product, figuring out all of our product marketing and then a lot of the collaborations with growth around even just the preliminary testing of demand and what is landing with our audience with this new product. There are also a bunch of things that are in this middle circle that are both driving near-term conversions, but also are very much needed in order to prepare to scale growth for the new product.
(21:01):
Examples of this is all the work that Karen and Maxine are doing on just improving the signup flow right now. Certainly, all of the backend growth engine analytics infrastructure and even things like growth levers, like performance marketing fall into this bucket because yes, performance marketing is driving near term awareness and conversions, but also we really need to set up these levers and build the muscle so that we can do them very well once it really counts and we’re looking to scale growth. Then on the far right side, there are these things that we probably would be doing if our number one priority was just to scale growth right now, but we’re pausing on them. Examples of this would be a new year, new you marketing campaign or some new podcast sponsorship experiments that are pricey, but we’re taking swings on them to see how they perform. Both of those are examples of priorities that we pressed pause on. Next slide.
(21:52):
Then as mentioned, we took demand gen and demand capture, which were separate groups and we put them under one roof here and we’re calling it growth. Not a ton is changing here, just more so that we’re trying to think of this as one team and one funnel, which maybe we should make T-shirts that say one team and one funnel. Haney, I’m going to put you on that. Next slide. In terms of this funnel, this is the basic way that it works. This is very 101 here. But someone hears about Levels, they learn more about it, they buy the product, and hopefully they love the product and they tell their friends about it and it repeats all over again and it’s a nice little loop. Next slide.
(22:35):
On the right side here, these are the marketing speak stages of the funnel. There’s awareness, first finding out about it, consideration, conversion, and then again, ideally product-led growth through something like user referrals. Thinking through how this might happen in practice, these are just a couple of examples, but maybe you’re googling around for what should your glucose levels be, you land on our blog, you eventually get to our homepage, and then you enter the formal funnel learning more about it and then buying it. Or maybe it’s an ad on something like Huberman with Andrew Huberman endorsing us or a Facebook ad. Then you’re getting to our website that way. Next slide. Important to just call out. There are things that we can do at each stage of the funnel to improve the funnel, which leads to more growth.
(23:20):
Next slide. For example, these are some of the questions that we’re asking ourselves at each stage of the funnel. For awareness and a lot of the demand gen activities, there’s how do we reach more people? And then also how do we reach the right people, so that once they actually get into the funnel, they’re more likely to buy for consideration? How do we effectively position Levels as a solution to Maureen’s problems, who’s our target demo right now? This is happening a lot on our website and landing pages. Then in terms of the actual signup flow, what pricing makes the most sense? Then also how do we clarify and improve the purchase experience, which we’ve been focused on a lot lately. Then lastly, we just want to make word-of-mouth marketing and user referrals more of a thing. Next slide.
(24:06):
In terms of the team working on this, this will probably be mostly intuitive but maybe helpful to see. In terms of the top of funnel, we’ve got Haney, Tony, Caitlin, working on all things content and editorial and demand gen. This is the blog and YouTube and podcast. Then also I’ve slotted Jackie and Paul up here on the awareness section as well. Again, a lot of these stages are fluid, but you could think of a lot of our paid marketing efforts as well as top of funnel. This is partnerships efforts, podcast ads like that Huberman example, as well as some of the ramping up of things like Google search ads and even Facebook ads and soon-to-be YouTube ads, a lot of the performance marketing efforts. Then in terms of the actual funnel, once someone’s getting to our website, this is a lot of the work that Karen and Maxine are doing on growth engineering and growth product.
(24:53):
Then Lauren is now going to be taking on DRI for pricing and business model. Then Ben who is now on special projects, but very much with an emphasis on growth is going to be pitching in across the funnel. Next slide. All right. The next section is just going to be a few things that we’re thinking about these days across each stage of the funnel, especially as we prepare for the new product. This is not a comprehensive update of all of our strategies or the tactics that we’re working on, but these are just a few things of late that are top of mind. Next slide.
(25:23):
Okay. For awareness, one thing that we’ve been talking a lot about recently is just better understanding this connection between the hundreds of thousands of people that are hitting our blog, our YouTube, our podcast. As mentioned, we had a record month all time this past month in January, but trying to understand what is the relationship between these surfaces and actually getting people into our marketing funnel. This is a lot of analytics work that’s happening here. Then how can we continue to experiment to drive people from these surfaces to our marketing funnel? Inside Levels is an example of this. This is this more product marketing-focused area of our blog. I would caveat that by saying that also while maintaining true to our values and mission of building trust through education. These are conversations that I’m having ongoing, particularly with Haney over the last week.
(26:16):
Also more on the paid side of things. As mentioned, Paul is building this performance marketing lever. We just hired a contractor who’s working on this. Then on the partnerships front, Jackie’s starting to think through, Josh mentioned this too at the start, how do we think about our partnerships channels and individual creators in light of a slightly new target demo and also a slightly new product and that it will be more affordable and it’s going to be a little bit more of a holistic health app. Next slide.
(26:45):
In terms of consideration, we first need to align on some of our core product positioning and just our approach to product marketing generally with this new product, we’re doing a lot of work alongside the product team here at Levels to test demand and test some of this positioning with our audience. Then more practically, we’re going to need to work on updating our website, our landing pages, our email flows and surfaces to reflect this new product vision and the offerings that we’re going to be presenting in 2023.
(27:16):
Next slide. On the conversion and specifically the signup flow front, we are aligning on new pricing and the new business model which Lauren is working on. We need to build the new signup flow and checkout experience. This is very much Karen and Maxine’s world and then continue building the experimentation infrastructure in order to build this learning engine as we launch and continue to run these experiments. Then lastly, next slide. As mentioned, we need to make a great product and launch user referrals and of course having a great product and word of mouth marketing is ultimately how a consumer product wins over time and really the foundation of a growth engine. Next slide. All right. Just zooming back out and some practical near term, like order of operations and next steps, especially in light of these two objectives of focusing on this new product and preparing and then also continuing to drive sales.
(28:11):
Near term, we’re going to be supporting product experiments, so all of these experiments that are already underway with Guides, Ben’s helping out a lot with this. We need to align on the business model and then just our core positioning, even in this next set of experiments that we’re running, we need to prepare our marketing surfaces and signup flows. As mentioned, we’re going to revisit some of these top of funnel acquisition strategies in light of a new slightly new product and new target demographic. Then we’ll continue to invest in growth infrastructure as we’ve been doing, as well as build out some of these growth levers as mentioned like the paid ads and user referrals. Next slide.
(28:47):
Just a couple more here. These are just a few themes that are driving a lot of how we’re working and what we’re focused on lately. Product and growth collaboration, particularly around figuring out what experiments we need to run next and the sequencing of these. David has been DRI on this and this is really coming together, so a big shout out to David. As mentioned, taking a more holistic view of the funnel, again, I think symbolically this is like putting demand gen and demand capture under one roof, but just trying to better understand the connection between these activities. Then lastly, really emphasizing data-driven learning and creating these feedback loops both on a weekly and a monthly basis.
(29:30):
We’re now doing this on a weekly monthly basis. Basically, all of the individuals on the growth team are looking at all of our key metrics each week and then adding highlights and annotations and reflections to figure out how individual actions that we’re each doing at each stage of the funnel are actually impacting our numbers on a weekly and monthly basis. Last slide here. Just a few things to expect on upcoming forums, I’m going to be working on a new growth strategy that’s going to get into a little bit more detail on some of this that I talked about today. Definitely be doing more of a paid acquisition deep dive in the future, outlook on expansion. Some of these editorial experiments that have been mentioned recently, Jackie’s work on a new partnership strategy and definitely a lot of the learnings that are coming from the growth product experiments and collaborations. That is it.
Josh Clemente (30:23):
Great update. All right, folks. That was designed for growth for 2023. Over to David.
David Flinner (30:30):
All right, no, that was awesome, Tom. Thanks so much. Okay. Today, Stacy and I are going to be giving an update from the product side, and we’ll be talking about Stacy’s Guides v4 experiments. Let me move this Zoom thing. Yeah. What you’ll expect to hear today is recent learnings from our Guide experiments, and specifically the one for Guides v4, which is for Stacy. Then after that I’m going to go through a little bit about where we’re going next. As Tom was mentioning, getting into this software only experience, which is including Guides. We’ve been in an era so far of scrappy experiments outside the app and we’ll be shifting more towards bringing that in the app and also rounding out everything beyond Guides as well. Everything that’s required to make the app we’re building from Marine, the affordable one that’s software first a huge success.
(31:21):
Focusing back on Guides, we have a big Guides vision. Just I’m going to walk you through a few of the things that you’ll recall from Alan’s walkthrough on our 2023 product vision. But Guides are these relatable, knowledgeable, and engaging people that Maureen would relate to and they would help her navigate what to do and to help her achieve her health goals. As you can recall from Alan’s presentation, there’s a whole bunch of ways that we envision ultimately that we want to bring Guides into Maureen’s experience and helping her reach her goals. But we can’t start everywhere at once. But just to remind us where we’re going, tips on exercise, engaging beautiful, delicious recipes that are easy for people to also engage with and see how to bake and see community data responses on. Members helping each other and collaboratively together with the Guides, figuring out ways to improve their meal responses. Engaging back and forth with the Guides in a way that feels personal, but it’s really one to many. Engaging on polls and things like that.
(32:23):
Learning new things on exercise and of course getting to know all the favorite habits and what’s working for each of the Guides and in the future, each of the members. Things like ask me anything where you can go deeper with the Guides to understand, have your personal questions answered, and of course have affirmation from the Guides on things that you’re doing well in a way that is one to many that the Guides doing it for a lot of people all at once, but it feels really personal to you. How do we actually validate that all this is going to work? Because we can’t jump right in and just go build what we think is going to be a successful way to achieve this software-only vision and really help Marine reach our goals with Guides.
(33:04):
What we’re trying to do is iteratively validate our hypotheses about our product vision first. We are running a series of experiments across the company to de-risk this vision, to ultimately lead to building something people want. Then hand in hand with what Tom was talking about on the product marketing side and growth, together over time as we iterate, achieve that product market fit. Right now, we’re within this realm of experimentation. Zooming into this bubble, there’s a bit lower altitude that we’ll get into here. We’ve been running these Guides’ experiments, which have all been outside of the app. Soon we’ll be shifting into a period where we’ll be building out scrappy versions of the software-only experience, including Guides within the app. We’ll progressively iterate on that until we get to a point where we’re confident that we have something that we want to test with the increasingly large sets of people with the software-only experience.
(33:53):
Zooming further into the Guides v1-5 experiments, we’ve been running these scrappy experiments outside the app, starting in December and running through now. We started with Casey and she proved out that people would want to listen from someone who was like a guide and they could learn that way over a three-day and a seven-day experiment. Then Stacy ran one, an experiment with a slightly larger audience going from 100 to 1,200 people, people who didn’t really follow health influencers and also we saw that they would learn from that. Stacy then is running the one we’re doing now, which is the Guides v4 experiment. Parallel, Sonia has started running one iterating for Guides v5. Today, we’re going to send focus on what we have been learning from Stacy’s Guides v4 experiment. I’ll hand it over to Stacy.
Stacy (34:37):
All right, a bit of background on how it started. Early January, I was inspired by Casey’s experiment and started one of my own with my Instagram followers. I have about 75,000 followers, and invited, checked in said, who’s thinking about their health in January. 1,200 people ended up signing up for a close friends experiment. It was unpaid. But after one week we got incredible feedback, and so we invited people to do a month-long program and continue what was working well for the first week and incorporating the feedback that they gave in the survey. It’s all running through my Instagram. It’s a close friends group as you probably have heard previously. 1,200 different members are seeing a private collection of stories.
(35:22):
For this, we’re testing paid conversion. When we transitioned from the one-week to the full month experiment, we didn’t kick anyone out. We thought that wouldn’t be kind. But we did offer an opportunity for them to pay to see all of the information saved in a handbook. If they don’t catch a story on a given day, they can always refer back to the handbook where every day’s stories are captured, there are recipes, there are links out to research. It’s a great resource for busy people. We’re testing if people paid. Do we have other slides actually with the numbers on that?
David Flinner (35:57):
Yeah, I’ll get to that in a second. But yeah, I think just the general hypothesis that we’re testing here are, will people pay for this experience? Then since it’s a month long, we’re testing engagement and retention over a longer period of time. Can a guide actually drive engagement and drive action over a full month versus the original week? Then ultimately this is yet to be determined, but will we see those people who paid renew?
(36:22):
And so, getting into some of the details on paid conversion, what we saw was that we exceeded our goal. We thought 10% of people would be a really excellent number of those 1200 people to convert. We saw around 14% of the active members paid. Not only did they pay, but they paid more than the suggested amount. We gave it an open-ended, we said suggested $10, but on average we were seeing $11.87 cents for payments. We had a strong conversion on this initial test to see if people wanted to go premium here.
(36:55):
Then on the engagement and retention front, there were several things that we were testing on this, and it’s really about can the guide help these people understand what to do and actually take consistent action throughout the entire month-long journey? I’m going to walk through some of the stats real quick here that we learned. Then Stacy’s going to walk through some of the other aspects on what we were testing with engagement is also with that vision that I reminded us from Alan, which of those experiences are going to work and which ones do we need to iterate on? She’s been piloting those all in Instagram as well.
(37:33):
Have you learned anything from this experience so far? 100% of people have said yes from the midpoint survey that we sent out. Have you made any changes to your diet or lifestyle? Nearly 100%. We had one person say that they haven’t made changes, but everyone else has made a change from Stacy’s recommendations
Stacy (37:48):
Or they plan to.
David Flinner (37:49):
Or they plan to. This time, a very strong result on people taking action or feeling more motivated to reach their wellness goals. What this is showing the accounts reached versus date. Each of these dots on here is a piece of content that Stacy delivered on Instagram. What you’re seeing here is the total number of unique people reached, and it’s a pretty constant line. Here, I think the average is around 850 people a day. What we found is that there’s about 1,000 active people overall in general, and on average, 850 of them are coming back every single day to see the content. We wanted to walk you through a bit about how we’re scrappily prototyping all the different categories that you saw earlier and seeing which ones work and what’s resonating with the audience.
Stacy (38:42):
All right. For this section, we took Alan’s presentation and overlaid some of the things that we’re testing right now in Instagram. All right, tips on exercise for Stacy, oftentimes I’ll be putting up a tip, an actual workout that people can do and an alternate option. If people don’t have 20 minutes in their day, they can still stay active, for example, with doing jumping jacks, pushups a plank for a minute each. We’re trying to do the high commitment and low commitment options offered. Whatever your level is, you can engage and you can feel good and successful about what you’re accomplishing. All right, recipes. There’s a principle, start with a low-carb breakfast, then I explain what that could be. I show exactly what I ate and then I give the glucose response. Oftentimes, most people are learning about glucose for the first time from me.
(39:30):
And so, it’s really interesting to see people interact with and absorb this. Lots of examples of members sending me their own tips as they absorb these things. Then taking very specific action on whatever is recommended. Sharing some tips that if you’re going to indulge, this is a way that you can do it and help minimize the spike and help minimize the wear and tear on your body. That was specifically from a travel trip that we did. We don’t normally make pancakes at home. We’re also pulling people and trying to encourage specific actions and see if they take them. In this example, we had a question, did you get morning light after I shared a series on the importance of morning light? Of the 1200, I believe 950 saw this, and we had 346 people say, “Yes, I got morning light.” And others were like, “I’ll try again tomorrow.”
(40:25):
All right, giving tips to make sure people know how to get exercise, whatever their situation, you can say, “Oh, it’s difficult to do while you’re traveling,” but I’m like, “You can always access YouTube while you’re traveling and you can pull out some things from the mini bar and use those as weights.” Trying to give little fun insights to keep people motivated. All right. Okay. This has actually been something that has received the most engagement. It’s a sneaky spiker experiment with an oat milk latte. I actually hadn’t had an oat milk latte before. I knew it spikes a lot of people very high, and so I tried my first one for the benefit of followers, and you can see the progression here. At the beginning on the left, I pulled people, “What do you think this is going to do to my blood sugar?”
(41:12):
Was giving real time results as I shot to the moon and then shared that those of you who guessed Big glucose rise were correct. The engagement on this was wild. We, I think, had… I forget what it was. It was like 200 responses or something. It was a huge number of responses. Then I provide them with an alternative, a link to the Best Milk Alternatives blog post from our editorial team. That got 300 link clicks, which was our second highest link, because people were very motivated to change their behaviors. Just next one is a sampling of some of the feedback on this particular one. Thank you. “I’m so surprised, going to be changing that habit ASAP. Love this content.” Thank you. It’s really neat to see people absorbing things and actually changing their behaviors in real-time, even though they don’t have a cgm, just from observing my personal response to it.
(42:13):
I know Chris Jones deals with oat milk a little bit better than me, but there you go. All right. Other things are, I put up question boxes and invite people to ask specific questions and we’ll answer them in video format. Oftentimes linking out to great resources to dive deeper. Go ahead. Then always looking for opportunities to highlight what people are doing around the world. I have members from every continent, except Antarctica. It’s fun to see people incorporating these into their lives all around the world and celebrating them. I always ask for permission before sharing, but people are really thrilled to see that their meal has been featured. Lastly, we want a lot of these things to be evergreen resources that we have available in the app. Our solution for this Instagram pilot or test has been to save everything in my healthy handbook. This is the item that people are paying for. Everyone gets to watch the Instagram stories for free, but 14% of the people wanted to be able to have access to this to refer back to at any given point.
David Flinner (43:27):
And so, everything that you see here are ways that we’re prototyping and rapidly learning around what we want to test and de-risk for the 2023 vision around this. The items here, we’re prioritizing the riskiest things first, so will actually pay for this. Now we know that they actually will enjoy it and they’ll engage with it on a short-time period. Will they pay for it for a longer period? And then will they stay sustained and take action over that longer period? So far, the answer has been, yes. One of the things that we just did hitting the midpoint and Stacy’s month-long experience was ask them if they’d be interested in a more structured approach to hitting their health goals together in the next weeks as opposed to right now, lot of this is the guide relaying principles as they live their life as they normally would live their life.
(44:10):
But what we want to test to is meeting the members a bit more where they’re at and starting from exactly where they may be on their early metabolic journey and giving them a structured path to ramping up. We had a really strong response in this, so about 92% of people want a little bit more of a lightweight structure, even though they’re already getting a lot of value. The next two weeks we’re piloting out and Sonia is starting to pilot out as well a lightweight guided experience. We’re working with editorial and with Alan to figure out what that might look like. Over the rest of the month, we’re testing out what would that lightweight guided experience look like? What will the retention look like month over month? And we’re continuing to do the categorical experimentation. Everything that Stacy mentioned there, we’re getting really actionable feedback on, “Hey, this one worked better than that one.”
(44:53):
We got a really strong response on the sneaky spikers. Let’s make sure we add that in. Every day, we’re doing constant learning. Sissy’s helping us document and capture all of our learnings, feeding this back into each of the different Guides’ experiments that are coming upspeaking of the Guides’ experiments that are coming up, the next one, which launched a few days ago is, Sonia, if you don’t follow her already on Instagram, definitely follow along. I’m learning a lot about metabolic health, even though I’ve been working at Levels for a while. It’s really excellent content. She’s really crushing it. With her experiment, we’re trying to figure out can we ramp up someone who’s not a professional full-time influencer into being a guide? Can we work on our guide ramp-up materials, and can we also spur action over a longer period with that lightweight guided program?
(45:37):
Then bringing us back to the higher altitude that I started off with. The next step for us is getting this into the app. We’ve been piloting everything outside the app, and now we want to say, “Okay, we’ve learned that people do like to learn from other people. We’ve learned that they’ll do it for a long period of time, and we’re learning that they’re staying accountable and taking action through this current experience, but we don’t know if they’ll still have the same results in the app.” What we want to do next is have this alpha test where this isn’t the next week and a half to two weeks. We’re going to work to get Stacy’s existing followers into the Levels app in a scrappy way. Scott and his pod on the software-only pod are working to very quickly enable new Levels members to actually be software-only signup and get into the app and not have a CGM.
(46:25):
We want to just see, will they resonate with any of the value that we already have, like the daily checklist, the meal analysis, like the glucose spikers, things like that? Then in parallel, Ian and his pod, the Guides Pod is working on the Scrappiest way to get Guides content into the app so that we can have some of Stacy’s archive content in there and accessible. The real goal that we’re working towards here, the next big milestone is in March. At the end of March, the whole company’s rallying around having a beta release, which would be the minimum viable in-app Guides experience and an acceptable version of our software-only product where we can have a very first beta test where we take on the order of hundreds of new members. These would not be Stacy’s followers, it would be people outside of our friends and family and see how that lands.
(47:13):
Then we’re going to move into a period where we’re going to be iterating based on that, as necessary, based on what we’re learning. With the ultimate goal that we want to reach a minimum viable product based on our 2023 vision, sometime late Q2, early Q3, the exact dates are still TBD, but this is just the general arc that we’re shooting for. I think that’s all we had to discuss today. Stay tuned for more from the Guides experiments. Oh, I should just call out that there’s so much I didn’t mention here. This is really a huge cross-functional team effort. Huge thanks to Casey, Sonia, Haney, Caitlin Cherry from the Athena team who’s really doing quite a bit of help with us, sissy, all the engineers working on this is really a whole company effort to rally around this. Of course as Tom mentioned, Ben, Karen, Tom, and everyone who’s helping out on that side as well. I’ll hand it back off to you, Josh.
Josh Clemente (48:10):
All right, awesome breakdown. Thank you. Thank you, both. Hiring updates. We don’t have anything open right now. Check out Levels link, slash careers and send us something if you’re interested in being a part of Levels someday. Then unfortunately, we don’t have time for individual contributions, but this was a very packed meeting and a lot of great stuff. We’re going to jump to an AMA with Sam just after this. Everyone, have an awesome weekend. Thank you for contributing. Super Bowl weekend as Mike D reminds us. Enjoy.