Josh Clemente (00:00):
Welcome to Friday Forum, February 24th, 2023. I think this, yeah, this is the last one of February already. Quick reminder, so Friday forum is sync time. We’re here to talk about recent achievements, share what’s going on across various functions, hear from our members and partners directly, talk about culture, generally level set, but it’s not about deep dives into the business metrics. It’s not the social connection core element. And we have a lot of meetings and other such engagement that you can catch up on, of course, in our recordings. So this is all about the recent achievements primarily.
(00:36):
And on that topic, let’s dive in for this week. So on the growth and conversion side, we’re continuing to see compounding 27% month-over-month improvements on our conversion rate since December, and this is since we started the Sign Up 3.0 and on process. So a lot of really great work there, and it’s continuing to pay off. And we’re now in this most recent week seeing up to 5% conversion rates for the homepage, which is really nice, and a huge shout-out to the team for building this out slowly and consistently.
(01:12):
The next iteration of this is crystal clear pricing, which is going out today. A lot of work went into compiling all of the various elements of the Levels offering into a very transparent view that you can land on and understand in one fell swoop, which is going to be really nice to test against our previous iterations. We’re also shaping positioning for the new product and additional purchase options that will ultimately come down the pike. Things like bundling and various monthly versus annual membership options. You can see some of the shaping that’s going on here, but just thinking ahead to what’s coming very soon. And we’re also putting out a 1,500 member referral, we can think of it as a referral sentiment survey to people who are somewhere in the magic moment stage, like 15 to 30 days into their experience.
(01:59):
So we’re trying to understand how they would like to refer, who they would like to refer, that sort of stuff. So a lot of good testing coming on the referral side. Product side, so we’ve got Labs 2.0 sign up in internal testing. So the end-to-end sign-up processes is starting to get tested out. We’ll have some initial draws for the new Labs product experience starting I think next week we’re going to be able to get some of that stuff for the team. Trends 1.0, I believe it’s just about ready, and Cosima is going to have more details on this stuff today. We also have Guides Alpha v1, so the very first experience with guides in the app, so shipped to 50 participants on time, which is huge. You can see what people will be greeted with. This is Stacy’s, I think, initial 50 participants are jumping into the app now.
(02:45):
On A Whole New Level, we had Dr. Stephanie Estima on with Lauren to talk about women’s health as it relates to metabolic health. We’ve got the editorial priorities 2023 just about ready for release. The editorial team is really supporting daily guides with stories, content. We’ve got a four-week CGM guide. We’ve got additional, I think, an overall master guide on CGM experiences coming. So a lot of great work going on there that will support the Guides experience and the new product. We also have a new inside-the-company pod between Sam and I, which discusses the concepts of leadership inside of Levels. I’m curious, I haven’t listened to it yet. That was a wide-ranging topic and conversation for Sam and I, and I’m really interested to see how the team pulled together some magic to make that a coherent podcast.
(03:30):
On the demand testing and signup strategy side, so we’re thinking ahead to Guides Beta 1, which will be a more advanced version of this initial Alpha and thinking a lot about what would we have to see, what demand testing can we do, smoke test various signup verification strategies to just see where the interest lies and test it with very high-intent audiences that we already have access to. So Ben has done a bunch of work on a document there, obviously with a ton of input from others, and that’ll be starting very soon, just in the next few weeks. And then we did a great retro on… Oh, sorry, Sonny did the doc on the Guides Beta 1. I think Sonny did the onboarding parties retro, which was looking into the white glove experience for onboarding into the app. So we’ve tested this over the last few months, and essentially we are comparing people who do a managed onboarding experience versus people that don’t.
(04:27):
And the results are really interesting. Those that did the onboarding had 50% more glucose data, 47% more food logs, higher likelihood of additional CGM purchases. A lot of really great trends, so comparing against those who don’t. So it’s obvious that there’s something here, and we’re going to, from here, figure out what to do about this and how to build into our experience. Membership receipts were up 16% week over week. The team financials updates, everyone probably saw these, at least team function leads likely saw these posted into comms this week. So a deeper dive into the financial over the past few months on individual teams. And we’ve got a good metrics dashboard coming that will give us a glanceable overview of the main things for how the business is.operating.
(05:10):
On the sports side, so we’ve been at 90% happiness for three weeks in a row, which is another awesome streak. Replacement rate was less than 10% for seven weeks in a row, also amazing. And the data outage that we saw this week was really handled in a flawless way. So the team has really predicted how these should go down, and some really awesome work from the support team to see that outage had occurred and handled it with even before the operations team entirely understood what was going on. So really great work there. And we’ve also had the best support SLAs since liftoff. And finally, the first primary research, we’ve touched on this a few times, but the first primary research that the Levels team funded was published last week, and this is from Philip Prince, Andrew Koutnik, Dom D’Agostino, Jeff Volek from Virta, Tim Noakes, some really big names here. And it essentially showed that recreational athletes, these are all male athletes, so we can’t necessarily anticipate how this extrapolates to others, but essentially these athletes massively improve their fat metabolism and their glucose metrics without a performance loss.
(06:15):
So moving from a high-carb, low-fat diet to a low-carb, high-fat diet, they were able to match their performance with no loss while massively improving the amount of fat that they’re metabolizing during the exercise. I was able to record a podcast yesterday with Professor Noakes to dig into this, and really some amazing stuff. And I’m very proud of the team for being a part of this research, and I’m excited for more of our research to make its way to publication. Okay, I think that’s the high level. We’ve also got some great stuff from Dr. Becky. Her YouTube video had a very strong week for us on conversions front and some awesome stuff going on with the growth content for Sonia.
(06:53):
All right, with that, I want to welcome Dr. Anjali Dsouza. Dr. Dsouza is a partner here at Levels. She’s a Founder and Medical Director at the District Center for Integrative Medicine, very deeply aware of psychiatry, behavioral medicine, all of the elements that really drive the softer side of metabolic health and that we’re actually interfering, interfacing with, I should say, here at Levels as we try and help people find their way towards better wellness. So Dr. Dsouza, I really appreciate you joining us today. I really appreciate your support of Levels so far. I’d love to just hear from your perspective, you have a really unique perspective on this world, what you’re excited about in the world of health and wellness and metabolic health specifically.
Dr. Dsouza (07:35):
Yeah. I mean, I think that first of all, I’m excited that we are talking about this more. I came to this world with a background in nutrition, and so this was always front and center for me, the role of lifestyle and food and how that really can improve one’s day-to-day health but also prevent against the diseases that we’re so run over by in this country and in this world. So I just love that there’s more dialogue about this. I love that there are offerings like a CGM through Levels that you guys offer, and I think people are starting to get it.
(08:09):
It used to be even five years ago that I would have these conversations with patients and they’d give me the nod, and some of them would be really interested, but most of them didn’t understand how metabolic health was really at the center of almost everything. And now I think more people get that, and it’s thanks to companies like your own. And that’s really exciting to me, because I feel like it gives me as a clinician more leverage to support people and to get what I want for them, which is for them to have the most incredible optimal health that they can have in their lives.
Josh Clemente (08:41):
I love that perspective, because that’s exactly how we see this tool, is helping people have an awareness into what’s going on in their bodies, be able to be in the driver’s seat more in terms of understanding the reactions to the actions they’re taking, but then also to be a force multiplier for experts and practitioners like yourself who are working with the same people, so that you can just work together with objective data. I’d love to hear what you hope to see in the future from the Levels offering specifically. What can we do better? What would you hope to see that that would improve outcomes on the individual level, just from your perspective with your experience?
Dr. Dsouza (09:17):
Yeah, I mean, I think to be totally frank, I think the offering is quite excellent, and I was actually introduced to you guys because one of my very good friends is very good friends with Sam, and so I was using CGMs before, the old school version of them, the simple version of them. And once I started using on myself, your product, and with my patients, I was just blown away. The difference, the outcome difference was just enormous. And I think it’s the visuals are beautiful, the quick and rapid connection to more data or more education I think is really powerful. So I mean, honestly, I don’t have anything to add there. I think the only sticking point for some people is price, if I’m being totally frank. But on the other hand, what you’re offering is incredible. So I understand how it’s like you’re trying to balance all the things.
Josh Clemente (10:14):
Well, that is definitely an area that we’re looking to improve rapidly, is making this more accessible and then expanding the education that CGM can provide and scaling it. So using community data, for example, or perhaps benchmarking someone based on their use of a CGM, but then also being able to extrapolate that data into the future so you don’t necessarily have to be wearing a CGM on a continuous basis. So I’m curious… Yeah.
Dr. Dsouza (10:41):
Yeah, I think that even the education platform that you guys provide, and similar to the Glucose Goddess, for example, providing people with just ideas, well, this is what might happen if you consume oatmeal. Right now that may not be individual data, but at least it gets people thinking about what’s going in their bodies. So I think it’s an invaluable service.
Josh Clemente (11:02):
Amazing. Well, we’ve got a lot to build in the future, of course, to get there, and I’m excited to hear your first-person perspective having worked in this space for so long. And I thank you again for taking some time to hang out with the team. This sort of thing is really invaluable for us, and we just love to have you here with us live. So if you’d like to stick around for the rest of the meeting, we’re going to dive deep into some function sections and would love to have you do so.
Dr. Dsouza (11:27):
Yeah.
Josh Clemente (11:27):
But otherwise, I know you’re super busy. Feel free to jump out.
Dr. Dsouza (11:30):
I’d love to hang out. This is great. And just to say every person that I’ve interacted with on your team has been absolutely impeccable and delightful. So just to make a plug for you guys, it’s been really lovely to be part of your world.
Josh Clemente (11:44):
Well, I 100% agree, and I really appreciate that. Thank you for sharing. And the whole team, thanks for joining, and please stick around.
Dr. Dsouza (11:53):
Yeah, I will.
Josh Clemente (11:55):
All right, Culture and Kudos section. So first off, you may have seen we posted this morning a memo Sam posted in comms. This is at the highest level channel I believe, but on burn, runway, and team size. So essentially this dives into some of the prickly topics of where we are in terms of growth rates, in terms of the macroeconomic environment, in terms of runway, burn, and the important elements of team size as it factors into those numbers. As of right now, there’s a ton of detail in that memo, which I highly recommend everyone read. But then we also have an AMA this evening just to get ahead of the whole meetup where people will be traveling over the next few days. So recommend reading that, joining the AMA this evening, 6:00 PM Eastern. Sam is going to take lead on that. I’ll also join. And we can answer questions, and just generally the goal here, of course, is to continue to be as transparent in real time as we can about the current state of the business and how things are trending.
(12:54):
Okay, kudos. So Cissy and Paul have both just hit one year as of February 21st. So congratulations to both of you. Thank you for all you’re doing to continue to press product and growth forward. And you’ve touched many parts of the business already, and it’s amazing that a year has already gone by. I want to give a big shout-out to the working group on software only, so the MVP of Alpha 1. So again, this is taking an experience that did not exist a few weeks ago or, yeah, a few weeks ago, where someone comes into the app without having a CGM on and is able to engage with Guides content, and building that into our existing framework as quickly and as efficiently as possible without changing massive amounts or letting scope creep. And the team did a great job and stuck to the initial deadline and just nailed this one.
(13:41):
So really great teamwork, huge shout-out, love to see this. Very excited to see what we can learn from it. Then lastly, a big specific shout-out to Lynette who identified and responded to the data outage we had this week so fast that Miz didn’t even know that we had had an outage. It was just handled that efficiently. So this sort of thing is, I think, the reason that we get so much love for our sport team, is that they’re just all over it. And when Lynette, thank you for being at the tip of the spear on that. And with that, I’m going to jump into the main thing.
(14:12):
So Levels shows you how food affects your health. Everyone’s working towards this priority. If you don’t feel that you are, please raise that with us. And right now, top priority in Q1 is getting to product market fit, and of course, we’re still building and selling our existing CGM primary experience, which will increase that accessibility, lower the hurdle to get into the world of metabolic health, metabolic awareness, and trying new experiments like Guides v1, et cetera. With that, I’ll hand it over to the product team.
Cosima (14:43):
Cool. So if you can go to the next slide. Just to give a quick update, we’re moving along nicely with the skateboard that’s being built. So Alpha 1 is out as of yesterday, and seeing some nice positive early results, and we’ll share more with that next week. But if you go to the next slide, I wanted to talk about a few of the things that have either about to go live or in testing that are going on in the background of a lot of the MVP work that we’re focused on in the pods. So if you go to the next slide. On the notifications front, so we actually just got post hog events firing off as of yesterday. So everything’s fixed and ready to go in terms of doing evaluation and getting a readout by approximately the end of next week. So what is notifications and why did we go down this route?
(15:36):
This is something that a lot of our members have reached out about for a while, and I think it’s something that we’ve really leaned in the direction of not wanting to be another annoying app that’s sending you tons of notifications. And so I think we leaned in the direction of being very quiet, but members really want to, in the moment, make changes and do what they can to blunt a spike, to know when their meal score is ready. To know when there’s an action that they could be taking. And so our first set of notifications, we really focused on, based on member feedback, what would be the most impactful notifications in real time? And we try to always include something actionable of what a member could do in the moment so it feels like we’re interrupting them for a purpose. The other really important thing we wanted to do, even though we wanted this first version to be a skateboard, was to make it configurable.
(16:30):
So not only can you, of course, turn off notifications globally, but you can go in and specify which notifications you want. And we’ve gotten really great feedback and no one reaching out annoyed about notifications, because those who want to tune it down have that ability from day one. So I linked the release notes, so if people want to read more, understand more of what’s going on with that feature, you can always go there, and more to come on results readout next week. And if you go to the next slide, so Labs v2. This is a long time coming. We’ve been working on this for a super long time. What actually ended up being the long pole for us is that Truepill stopped being able to cover our panel offerings. So we had to integrate an entire new phlebotomist via Getlabs and Labcorp, and we’ve finally hit the end of that.
(17:25):
So we’re dev complete, and we’re in end-to-end testing. And just want to really shout out to Dr. Rob Lustig who I know is on the call. But thank you so, so much. It’s been so much going into not only behind showing all of the panel results and the details and explanatory text, but also the first thing that you get when you land on your lab results is this awesome overview that I think really helps the experience feel high touch. So in terms of what we were trying to do for members, some of the feedback that we got from Labs v1 was that for the price point, members really weren’t feeling like they were getting the value. They didn’t walk away feeling actionable. They didn’t have a sense of what am I supposed to do with these results? And the results felt very ancillary to the rest of the experience.
(18:14):
So what we’ve really tried to do with Labs v2 is make the results feel personal. So let’s look at your exact results and try to tell you something about what they mean in combination. Let’s also make the experience feel high touch and personal with a video and way more interactive information. So when you click into the biomarkers, there’s a lot more details. There’s also visualizations of the direction that your metrics are moving. Something we also wanted to focus on was the panel is so expensive in the past because we were looking at 11 different markers. Bringing the focus down to these five biomarkers allows for it to feel as exhaustive as possible at a price point that is repeatable. And so our goal as we continue to think about building these programs that allow members to not wear CGMs all the time, is looking for opportunities where maybe checking in on a panel every few months is something that you can do in addition to checking in with a CGM to see how you’re doing with accountability on a day-to-day basis.
(19:16):
So we’re really excited to build this to the program as we go. And yeah, this is just a little bit of a snapshot, but you guys will be able to test it out soon as well in production. We’re looking at launching around mid next week. And then on the next slide, so Trends v1. This is an exciting one as well. I’ll talk a little bit about what v1 is versus some of these 1.0, 1.1, 1.2. So Trends v1 is really introducing the ability to show relationships patterns and how members’ key metrics are moving over time. So we’ve got, of course, our core things like your stability score, your average glucose. We’ve introduced waking glucose as a concept that’s sort of a proxy for fasting glucose. And then we have all of these input metrics, like your steps and your exercise time and your sleep time.
(20:10):
And the goal is to not only help members visualize how those metrics are moving, but also start to see the relationships between those metrics. And we’re doing this at a phased approach, so 1.0, which is what we’ll release next week, is really focused on laying that foundation. So we’ve done a whole redesign of My Data that’s much more modular. So when you land in My Data, you basically have access to all of these different metrics and can click into see more. And this is going to be a really nice flattening down of My Data that allows us to show relationships much better. So 1.0 really lays the foundation. I wouldn’t expect much huge gains in terms of member capability, though we are going to be introducing more metrics as part of that. So there are some nice gains for members.
(20:58):
1.1, which is what we’re picking up right away starting this sprint is when we’re going to start to introduce a trend algorithm. So that’s what you can see on the right, which is where basically you can go into any particular metric and see whether there is a significant shift in the direction of that metric. Very similar to what you see in Apple Health and other health apps where you’re able to see if a metric is moving over any period of time that you want to look at. 1.2 and beyond is where we’re going to start exploring relationships. We’re going to look at the best way to approach this considering our focus on CGM optional, and trying to not overemphasize the need for only highlighting your personal trends, but see what we can do to start to highlight more community trends and insights and relationships between metrics.
(21:51):
And in terms of why are we even approaching trends as a problem area, I think a really important thing in motivating behavior change, even though we want to focus on the tiny habits, is zooming out and seeing how those tiny habits are compounding over time. And I think this also can really help the product feel sticky. This product is getting to know me over time. The more months I invest in it, the more it’s understanding what the relationships are in my body that matter. And yeah, really excited to build on this, and I think this new foundation for My Data will allow for us to plug in lots more cool insights and relationships. And those are just a couple updates of things going on outside of the pods.
Josh Clemente (22:37):
Awesome. Really exciting stuff. Love the near term of having both trends for the CGM data combined with the new Labs experience and being able to see how those things overlay on each other. I think that’s going to be really powerful. So thanks team. Thank you, Cosima. Okay, we got Haney with content.
Mike Haney (23:01):
Alrighty. Make sure, yep, looks like I’m hearable. All right, and I’m still recovering, so if I break into a coughing fit at some point I’ll try to hit the mute button beforehand. So why are we chatting today? Well, this is going to be a little bit more of a high-level overview, I would say, than some of the more tactical content updates we’ve done recently. And really what spurred this is we’ve talked about the growth consolidation that happened recently, where we brought all of growth together under Tom. And a downstream effect of that is that we decided to also consolidate all of content under me. And from a people ops perspective, this didn’t really change much, other than Tony is now reporting to me instead of Ben. But I think it’s a useful, meaningful change. What I’ve listed here is all of the services, and I may even be forgetting some that constitute the content that we make here at the company.
(23:59):
So it’s not just the blog, although that’s what tends to get a lot of attention. And then obviously all the video and audio that Tony does, but also the emails, the new Inside Levels channel, the content we do in the app. And these have just been scattered around the company for no particular reason, other than just how things tended to grow. So the move here is really just to bring these things together into one team so that we can start to work together a little better to find the synergies to make sure we’re all moving in the same direction and to start to see some of the connections between all of these different services and growth. So next slide.
(24:37):
So what this has led to is Tom and I have been having a lot of chats, a long async conversation over the past few weeks, just stepping way back and saying, all right, if we’re going to talk about content as a thing, as a whole department of people who make stuff, let’s reexamine the role of content at Levels, not really with the intent of making any significant changes. And you’ll see in this there aren’t really significant changes that we’re talking about. But really just to go back to some of our first principles and think about why do we make stuff here and what purpose does it serve and what’s the relationship between those buckets? So I think of content as really always having had three primary purposes here. So education, which is to say that the vast majority of people who come in contact with the content we make, whether it’s a podcast episode or a blog article or a social post, they’re going to take what they can from that post and then they’re going to move on with their lives and maybe never come back to us again.
(25:33):
And we’re okay with that, because putting this useful information out in the world and helping people improve their health is something we’ve decided from early on is a worthwhile part of what we do. But it isn’t the only thing that we do with content. So content has also always worked with product. David will remember, I think it was early ’21, he and I and Casey were working on writing specific little content articles for the app when we used to have a content tab in the app itself. Obviously in the current version we have insight cards which draw a lot from the content that we produce. And then in the new Guides version is going to be heavily content reliant as well. So that’s also always been a thing that content’s worked on. And then content has always had a growth aspect to it as well. We’ve always been the top of the demand gen funnel. We’ve been a huge traffic driver, particularly through the blog, but through other parts as well, that bring people into the Levels universe in some way or another.
(26:31):
And then we’ve also always done parts of content that are more growth-focused, like member stories for instance. Next slide. But this is probably more what it’s felt like in terms of what the content has been, and maybe somewhat of a more accurate version of where the effort’s actually gone as well. And I think that’s for a number of reasons. The reason I think we talk a lot about education as the component of the content that we do is because we definitely do it a little bit differently, that we have this purely education-focused part of our content arm in the blog is different than what a lot of companies do. I talk to a lot of other folks who have my role at pure companies, and their chart here would look like the inverse of this, where the vast majority of the content they’re creating is really growth-focused, it’s SEO optimized, it’s aimed at driving conversions. And then every now and again they’ll do a white paper that’s aimed at education or credibility or something like that.
(27:30):
Whereas we’ve really focused on the education component, and we’ve also just made a lot of education content. So I think that’s the other reason that it gets a lot of the attention, is Tony’s produced a ton of audio and video. We’re closing in on 500 blog articles now. So there’s just a lot of that, whereas some of the other aspects are starting to grow a little bit more now. And for a long part of the history that we’ve been working on content, because we started this content arm so early, it predates me, and I’m pretty early here, for a lot of it, we just weren’t in growth mode. I used to talk to folks who had my job at other companies, and they would say, “Well, what’s your conversion goal?” And I would say, “Well, we’re not trying to sell anything, so I don’t actually have a conversion goal.” And that’s obviously changed since last July. So that points us a little bit to where we’re headed. So next slide, one more nice Chiclet slide here.
(28:24):
So this is a little bit more where I think, not even aspirationally where we want to go, but I think where we’re already headed. So we’ve talked a lot about in the last month or so how Caitlin and I have been able to jump in, and I know Tony started to do some work as well, working with the product team much more closely on Guides and all of that content. And I think that’s been great, and we look forward to continuing to do a lot of that kind of work. And then Caitlin’s also talked about, I think presented here, the Inside Levels channel that she’s been working on growing, which is much more specifically growth-focused content, where we’re going to try doing much more traditional content marketing, writing articles about the app and how great it is or why CGMs can actually be useful and some of the Labs content, that kind of stuff. So this is, I think, a bit more of a reflection where we’re headed here. And again, none of this really reflects a change in what we’re doing, it’s really just about where the focuses are. Next slide.
(29:24):
Yeah, so as a way to think about what we focus on or what our priorities are, I thought it might be useful to think about the way we look at content KPI. So I break it into three buckets. There’s audience growth, which is just sheer eyeballs, just how many blog visitors do we have, how many total views do we have on our podcast? And the focus on those has been both organic and specific over the past couple of years. So with the blog, for instance, it’s primarily been organic growth. We’ve done some things, like we try to always optimize for SEO, we try to write articles we think people are going to find useful and that will bring visitors in, but it’s really just been make more content and more people will come. And over time, we’ve seen that audience line go up as opposed to a project like Ben and Tony did where they 10-xed the YouTube visitors.
(30:13):
That was a very specific project that had parameters to it with a real goal. And over several months, we watched that YouTube audience growth just grow like crazy. And Sonia did similar things on the social side. The next bucket is the audience engagement. So that’s just really the folks who have come, getting them to do something, whether that’s read more articles or read the full articles. The time on page is something we look at. How many of these people are coming back, are they finding what we make useful and being return visitors, are they opening emails, are they clicking on things? Are they liking posts? Are they watching more videos? And again, these are things that in some ways we’re always paying attention to just as part of keeping the lights on, so we’re always AB testing things within our Digest newsletter. Caitlin and I are always trying to make good related articles within our articles to get people to circulate.
(31:02):
But then there are also sometimes projects we do that are specifically trying to really grow this, so when we redesign the newsletter, for instance, or introduce a new newsletter, it’s designed to really bump up the engagement. And then the last bucket here is what Tom and I have called growth funnel engagement, which this is a category metric we have here because we are creating content from within the scope of a company that’s also trying to sell something. So this is related more to contents that one role of content as the top of the demand gen funnel as the way that people find us. And so we think of these metrics as basically anything that moves people further down the funnel to engage with the company. So that might be something like email signup, which is something we’ve always had on the homepage of the blog.
(31:49):
That’s great for the peer editorial. Sign up for that. They get the weekly Digest newsletter or the bi-Weekly Digest newsletter. But it’s also good for growth, because once they sign up for email, they get the welcome series, we start to tell them about Levels and give them opportunity to convert. For other spaces, like Inside Levels, which is much more growth-focused, the growth funnel engagement metric there might be something like conversions, because we’re putting direct join buttons on it. The point there is to tell people how awesome the product is and why it might be useful to them, and we want them to go directly from that to a conversion. So the point here is basically that all of our content services have some aspect of this included. They have some growth funnel purpose, but what that purpose is and what those metrics are are different by surface.
(32:33):
Every surface has its own principles and guardrails that we’ve established over time and that we’re continuing to explore and establish to make sure that we don’t ever lose those key aspects of all of our content that we’ve established, like trustworthiness, like utility, that we don’t sacrifice those to move this growth funnel to the forefront, but that we’re also recognizing that, yes, content is the top of the demand gen funnel and where we can appropriately move people down it, we should. The last thing I put at the bottom here is a framework that somebody at a16z gave me early on which I found useful, which is the idea of active metrics versus passive metrics. So active metrics being the things that you’re watching with an eye toward actually improving. So you’re trying to take actions consistently to make those metrics go up. Passive metrics being the things you just keep an eye on because they’re interesting and they’re good to learn from.
(33:24):
So as for instance here, if the active metric for metabolic insights for the blog was conversions, it would strongly incentivize us to put join buttons on the articles, even if we felt like that violated another principle that we had already established for what we want the parameters of that content surface to be. If that’s the thing that you’re incentivized to do, it drives the actions that you take. Instead, I think of conversions as something like a passive metric that I’ve always watched with the blog. We’ve always tracked, early on we tracked a metric in our weekly spreadsheet that was number of blog visitors that ultimately signed up for the waitlist.
(34:02):
And it was a little bit of a complicated Google Analytics path, but we always tracked it because I just wanted to know how many people come into the blog and then ultimately end up engaging with the company. And so that doesn’t necessarily mean we’re going to make significant changes to the blog based on that information, but it’s still useful for us to know, and it probably will inform downstream strategy and tactics that we do. So a lot of what we’re going to be working on over the next six months is just establishing these active and passive metrics for each of the content surfaces and really thinking about that growth funnel engagement bucket and how it relates to each one of these services. So next slide.
(34:43):
So finally, in terms of the priorities here and what all this means in terms of what we’re going to be focusing on, the first note here is just, I think as with everybody across the company, product is our number one priority. So anything we can do to support folks there, we will. And across the surfaces, we’re making adjustments like our target cadence of posting or publishing, we’re going to bring down a little bit to make space for being able to work on product stuff and some of these other things as well. But the second point here is really that of those buckets that I talked about, the work or the primary effort over the next few months is going to go more into that growth funnel engagement bucket and doing both experiments and learning those. So we’re going to test things like I threw up a comms post this week, and I don’t know that we’re going to do it, but as an idea that we threw out is should we put a short ad in our podcast for Levels?
(35:34):
Should we jump in at 15 minutes and tell people like, “Hey, by the way, this is brought to you by Levels, and we have a product and it’s pretty awesome. And if you want to check it out, go here”? Those are the kinds of ideas that we want to surface and we want to test the ones that seem appropriate to test and that seem like a good use of our time to test, and just to start to develop more learning about, again, the relationship between these surfaces and growth. And then the second part is really improving our analytics. Just taking a look at what are we tracking, what can we track, what don’t we know? And I think there’s a lot that we don’t know about the relationship between these surfaces and growth and what people’s journey through them are. And then in terms of those other buckets of audience growth and engagement, we’re not stopping anything.
(36:16):
We’re going to continue to publish and post and produce videos and social posts and blog posts and keep these surfaces running so that we continue to get that organic audience growth. We know that is a great asset for us as a company that we’ve got a 340,000-person email list right now. We got almost 300,000 visitors to the blog every month. We definitely want to keep that growing organically, just to say that you probably won’t see any big 10x projects in those buckets over the next few months as we focus on this other aspect of it. There’s a memo that is close to being final that will dive into all of this a little bit more deeply. That’ll probably be out in the next week or so. And I think that’s it.
Josh Clemente (36:58):
Awesome. Thank you, Haney, for sharing the evolution and history, and really excited for what’s to come. Already some great signs as editorial expands into something bigger. All right, we got Hui with an engineering pods update.
Hui Lu (37:17):
Hello. So can everyone hear me? Cool. Yeah. So I’m sure everyone here have been heard about engineering pods for a few times already, and today I’m going to talk about what is it and the why is behind it, and also in practice, what does it look? Yeah, next slide. So before we go into details, what is an engineering pod exactly? So it’s a smaller group of engineers led by tech lead, I will introduce more about that later, who share a common set of goals to allow them to build a continuous context, stay focused, have more ownership to be able to move faster. So this is something that we have been thinking about for engineer org and have been recently formalized and started to operate. And with this kind of organizational change, we all know there are some problems we want to solve with it. So what exactly are the problems?
(38:13):
Next slide. So what did we use to do? So previously, engineers were assigned to projects by availability mostly. So this is screenshot of last year’s eng allocation plan, and you can see it’s pretty random because of that. And one good example is Murillo worked on Levels, Levels 1.2 and 1.3. But later Rafael picked up 1.4, because Murillo needed to take time off during Christmas week. So this costs very high context-reaching cost, and it’s not just about the domain knowledge, the code written about the domain, it’s also about the people that you have been working with. It’s very difficult for you to build a continuous trust and close working relationship with the people, especially for product group. It’s around 20 people. If you just randomly ramp up and ramp down on project, it’s really difficult to build that relationship.
(39:11):
Next slide. And previously, engineers were only responsible for technical implementation. And when a feature spec and design mark were handed off to engineers, it’s considered to be finalized and final. The decision was made previously to help reduce engineering on have big feature specs and design marks, but in the end, it did lead to engineers feel lack of autonomy and ownership. It’s also more difficult to manage scope, because when product team and design team, they are working on feature scopes, it’s very difficult for them to know whether some decision they made is technically feasible or technically complicated.
(39:59):
Next slide. Previously, product team was responsible for both product decisions and also project management. And that includes linear task breakdown, linear task management platform engineering team has been using, including all the communication, coordination on comms and such. So this made product team more resource constrained than it is already be and it necessarily needs to be. Cool. Next slide. So all these lead to lower velocity, lower output, which is not something that we can afford and would like to see why we want to full speed sprinting on the product 2023 vision. So this is basically the callback to the definition, hopefully now you can see why these specific keywords are highlighted to help solve the problems. But what does it mean, and how does it look like in practice?
(40:59):
Next slide. So we have three engineering pods now, and each part works on a longer term goal and builds context continuously on their relative domain, sorry, respective domain. And now you can see from their current engineer allocation screenshot, each part is… Oh, cool. Yeah, so from the screenshot, you can see each pod is now allocated to their respect domain, and people are allocated to the work that all leads to their common goal for their pod. So the first two pods are their product pods. Their common goal is to execute on and ship that software only MVP. I think product update showed it. There are multiple alphas and betas those two pods are aiming to execute on and ship. And the last one, the growth pod, it’s a joint collaboration between product growth and engineering. And the blue highlighted person role, which is Ian Scott and Maxine, they are the tech leads for these pods.
(42:13):
Next slide. So yeah, what does that mean, tech lead? So in order to give engineers more responsibilities but also make sure their roles and expectations are clear, we formally introduce the tech lead role, which basically the DI for engineering deliverables. They are getting involved earlier during product scoping process to give their engineer opinions on whether something is technically feasible or complicated to help manager and provide a bit of pressure on how can we keep the future scope small and minimum. They are responsible for managing the engineering scope to make sure the engineering solutions solve their practical problem, not a theoretical problem in the future. They’re doing project management to take that off the product team’s plate. They also hold other knowledge and context to unblock and enable the IC engineers in the pod to be able to execute as fast as possible. So as a result, the product team who is a DI remains to be the DI for product decision, they can now become more focused on product, which is what hypothesis we want to validate.
(43:28):
What features do we need to validate? When do we want those features get onto user’s hand to validate? Yeah, next slide. So we have been running the engineering pods, mostly the two product pods, for the past three weeks. We are seeing increased velocity and morale, so we just shipped the Alpha 1 this week, and people are generally feeling good about how everything works, but obviously there will be continuous learning. For example, in the beginning, their communication cadence is really high, which is needed. But after we head into the execution phase, how can we make sure people are not always all the day on comms? And there are other improvements that we will continue to, I guess, observe what are the growth room for improvements and apply those. So maybe in a few weeks I will also be talking about their retro engineering pods and what we learned in this forum again. Yeah, I think that’s all.
Josh Clemente (44:33):
Awesome. Thank you, Hui, and definitely excited to see this sort of structural experimentation, and can’t wait for the retro sometime soon. And thanks team for obviously being flexible and working inside the new framework and helping us continuously increase efficiency. All right, quick hiring updates. No currently open roles. Definitely if you or someone you know is interested in Levels, check out Levels.link/careers and send us your info, and opportunities will be changing over time. With that, we are at the individual contribution section, so I’m going to stop the share here and we can do some shares. So you can raise your hand by hitting the little reactions button on your Zoom panel, and you should see a raise hand thing. We’ll do this. I’ll go ahead and go first. So professionally, yeah, I mean, the product updates today were really exciting.
(45:29):
I first of all am stoked about the Alpha v1 shipping already. Can’t wait to see what we’re learning there. Trends and Labs v2, just near and dear to my heart, cannot wait to see that stuff and be able to track that information against itself. Yeah, I mean, I think that the overall exciting learnings and the acceleration in the rate of learnings is just something to constantly be proud of. And then the sport team crushing it this week. On the personal front, I still have not gotten a car, I’m the wrong person to put on this task, so I’m over-optimizing, and this is now dragging into multiple weeks, but I’m having fun in some sick way about the whole thing. So hopefully that will come to a close soon, and I got a chance to hang out at Ian’s house last night. Thanks, Ian, for hosting. Lynette.
Lynette Diaz (46:24):
All right, so this week professionally, super psyched to have managed that outage. Thank you so much to Matt for providing a beautiful document. I was just able to follow that and do the things, and the response from the members was so overwhelmingly positive because we were able to respond quickly. And they were so grateful, they were like, “Take your time fixing it. No problem. Thank you so much for the quick response.” So that was really awesome. And then personally super psyched. I’ve been following the Guides experiments and sharing stuff on my own Instagram too, and it’s exciting to see other members of family and other people reaching out and sharing my metabolic health info with them, and seeing those light bulbs go off for people is really exciting. So yeah, that’s the update for me.
Josh Clemente (47:11):
Awesome, and thanks again to Lynette and Matt for crushing that. Ian.
Ian Schumann (47:16):
On the professional front, just getting the Alpha shipped on time and was great. It feels like a great just validation of the whole team and the whole arrangement, early success and so on. It really came together when I saw just my app showing that new tab and the app now feeling alive, because there’s a person talking to me every day with something new, and it makes the app feel alive. And it’s not even a skateboard at this point. It’s like a sled that we’re pulling along with the help of Athena, but it already feels more lively. On the personal front, we’re getting the baby room together, and that’s starting to feel more real too, so that’s exciting.
Josh Clemente (48:01):
Huge. Awesome. Scott.
Scott Klein (48:05):
On the work front, apologies for being quite behind on comms, but I’ve been wrapping up thinking about the New York offsite. Nicole and I are going to record a bit of a walkthrough just to pre-flight asynchronously a lot of information about the schedule and whatever so that we don’t have to spend a lot of time doing it synchronously. So please, if you’re coming, make time to watch that. We’ve hemmed and hawed about the group orientation and how to get things organized, but it should be a good offsite for everybody, and at the very least it’ll be great to spend some time and build a lot of trust battery there. So excited to see most of you in New York.
(48:39):
On the personal front, the CrossFit Open is going on right now. For those of you that don’t know, it’s where everybody that does CrossFit in the world does the same couple of workouts a couple of weeks in a row. And I got totally shellacked on the first week. I was doing a lot of strength training last year, but obviously not a lot of everything else training, and so this is week two. And see how it goes tomorrow. So that’s my weekend excitement.
Josh Clemente (49:04):
Good luck. Post results. I don’t know if we have more than one person doing the open this year, but it’d be cool to track. Rob. You’re muted, Rob.
Robert Lustig (49:20):
Thank you. Sorry about that. If any of you have time tomorrow, The People’s Pharmacy on NPR is going to be what you don’t know about weight loss with you know who. And I’m pretty sure I snuck Levels in there at the end, so a national shout out. We’ll see.
Josh Clemente (49:44):
Love having Dr. Rob Lustig sneaking us into national shout-outs. That’s a big benefit for the team. Sam.
Sam Corcos (49:58):
On the professional side, it’s been really cool. I’ve been able to step back on a lot of specific actions and seeing Hui really step up in a big way in engineering, seeing Scott and Ian really step up and take a lot of ownership on getting these things over the finish line. I feel just a lot more ownership over these major projects, and it’s been really cool to see that. On the personal front, I’ve set myself a goal of learning some small skill every week. I learned how to juggle, I learned how to do a card trick. I’m currently trying to learn how to do knots. And all of these things turn out to be super easy. It’s like you could watch a movie or you could learn how to juggle. That’s about the equivalent amount of time that it takes. I expected this would be a much harder thing, same with card tricks, but if you get through that first 15 minutes, it turns out it’s actually pretty simple. So I’m surprised at how easy so many of these things are that I’ve avoided for my entire life.
Josh Clemente (51:06):
One of the Rubik’s cubes. Ben.
Ben Grynol (51:13):
Yeah, a couple things. On the professional front, plus I went to Ian’s, that was super fun last night. I guess that was sort of personal, professional. It’s been super fun to do IC work again and really go deep and have these extended days where there’s just lots of first principles and foundational thinking where you can spend the time to get things done. So leave the music out please. Sorry for the background noise. But no, it’s been super fun to do that and to go back to this idea of building, so having so much fun doing that. And then very stoked on the white glove onboarding learnings that Sonny put forth, and there’s so much work that we can do moving forward with that. So there’s that.
(51:54):
And then on the personal front, since I have not been playing soccer since moving, made a goal of trying to be physically active and have been sticking to it as far as going to the gym, even if it means getting to the gym at 9:30 or just some amount of time to commit to that. And so this self-accountability and a little text group of Josh and Mike D to make sure that we’re ringing that bell. So anyway, that’s it. And yeah, that’s all for now.
Josh Clemente (52:24):
Nice. What happened to the background music? All right, we got a couple more minutes here. Got time for one or two more. Anybody? Miz.
Michael Mizrahi (52:37):
I wasn’t in there. Want to, I guess, say thanks to Sam and really just the whole company for living the transparency values. I think the burn, runway, team size memo is really a testament to how serious we are about that. And so while it’s a difficult topic, I’m happy that we can really live that value out and share that openly and have that conversation. So looking forward to that reception, just everyone being on that same page I think will make a lot of things much, much better and easier. On the personal side, it’s a big week for fitness, apparently. I am trying out Ladder, switching over from Future after six, seven months. So really interesting learnings, really interesting parallels with some of the concepts we’re playing with. I’m not hooked yet, but I’m excited to give it a try. My Future trainer is moving on, and so it felt like a pretty natural break point, but also really interesting to test that product and see the reliance on an individual trainer and how much that matters. So yeah, switching away and keep you posted.
Josh Clemente (53:36):
Nice. Yeah, shout out to the Ladder team too. I think Tom from their team had a chat with our guys on the guides-like thing that they have with fitness. Yeah, they have a very cool product. I recommend checking it out. All right. Well, I think we’ll call it here, unless anyone else wants to jump in. Okay. Well, awesome work this week. This is a great forum, and I appreciate all the contributions, all the work that’s going into the acceleration we’re seeing and all the experiments. Have a great weekend. Again, we have an AMA this afternoon, so we’ll see some of you there.