Prose's CX Manager on Turning AI into a CX Analyst

Why personalized beauty makes CX harder

Jessica (0:02) My name is Jessica. I work for Prose. We make custom beauty products, hair and skin primarily right now. I’ve been at Prose for seven and a half years – very long time. The company’s been around for about nine years, I believe. So, a good long chunk. I’ve been on the CX team the whole time. I’ve grown my role from being an agent in the inbox to being a manager working on various strategy initiatives like Siena and other internal tooling needs for our team.

Lisa (0:30) What do you think makes CX harder for this business model, and more important, when the whole product is personalized?

Jessica (0:38) I think that when a product is custom, there’s more investment from the customer. They’re being told that this product is for them, and at Prose it is. So I think that it’s inherently even more emotional. Buying beauty products already is such an emotional connection, especially people’s hair. Hair is super significant to everyone. It can be very cultural, very deeply personal. And then as that translates to the ops and agent side: because we make everything custom, everything is made to order. We have our own order management system, our own ops fulfillment processes, production processes, and a lot, a lot, a lot of ingredients to know. I remember when I was training as an agent, I was like, there’s no way I’ll ever know everything. And since then, we’ve launched even more products and new ingredients to spotlight. So I think from the service POV, it’s a lot to know. There’s not a bunch that can be applied from other places. Of course, things like policy, certain treatments of situations. But Prose is very unique and special, and it makes training agents a fun and challenging process.

Lisa (1:40) What did your support operations look like before Siena?

What support looked like before Siena

Jessica (1:44) Before Siena, we did not automate anything. We have our inbox split into three tiers or so. A tier one, a tier two, and a tier three. Tier two is really a half step up from tier one. Now we’ve combined things, changed things. I think that having AI has given us a lot of flexibility. We didn’t actually get rid of anyone when we onboarded Siena, but we’ve grown a lot as a business since we onboarded Siena, and we haven’t done a bunch of hiring since then. And we definitely would not be in an okay spot now without Siena. I think it’s given us a lot of freedom, especially when issues arise. The topics that we have Siena handling — let’s say that subscription breaks on site, users can’t cancel their subscription, which has happened very briefly before — we didn’t even notice, because Siena fully absorbed the tickets. And that’s something that would have previously put us into a hole that took days to get out of. We found out, of course, after the fact when we were looking at the numbers. But it makes it so we don’t even have to worry about certain things anymore. It allows us to have a lot more flexibility in staffing, scheduling, everything really.

Reducing noise, and why the custom integration was non-negotiable

Lisa (2:44) What other benefits do you think your team has gotten since you’ve adopted AI?

Jessica (2:49) When we onboarded Siena, our goal was always to automate what made sense for us to automate, and not automate things that we didn’t feel that we needed to or wanted to. We were really intentional and very strategic. As you know, we have a custom integration with you guys. We’re not on Shopify, so you guys built us our whole stack. So we had to be very specific about what we were choosing. And our mission with Siena has always been: reduce the noise for our agents. What are things that are kind of a waste of their time to handle, so that they can focus on things that are more important to the customer journey and help create those retention moments?

Lisa (3:24) Can you share a little bit about that process? We see that there are more and more requirements for custom work. There’s no one-size-fits-all in customer service or in e-commerce. Every single brand is completely different. Even if we were to talk to ten brands in the same space as Prose, everyone has a different setup. So I’m curious, what does it mean for Prose to have an AI partner, or just a vendor partner, that builds specifically for your stack? How important is that?

Jessica (3:52) It’s really important for us specifically. A lot of places won’t build custom integrations, and with Siena it was our first time doing an open API call at all. So it was a really big unlock for, I mean, yes, the customer service team, but an exciting project all around. I think that if we weren’t able to find a vendor that would make us a custom integration, AI for support tickets just wouldn’t make sense. There aren’t enough tickets for us, at least, that don’t require some level of intervention, in enough volume, for it to make sense to replace humans with an agent. So it was really important for us that we find someone that could — at the time it was very limited, no one was connecting to Kustomer. So you had to be able to integrate with Kustomer, and you had to be willing to make us a custom integration, because so much of our outreach is so operational. The volumes of just more FAQ questions are so low. We needed the insight.

Does emotional, identity-driven beauty raise the bar for AI?

Lisa (4:43) That’s amazing. As this category is very emotional and identity-driven, I feel like customers are trusting the brand with everything that they share and the problems that they have. Do you feel like that element has raised the bar for what AI can actually deliver, or has to get right?

Jessica (4:58) I think we’re in kind of a unique position, because I think AI is actually really great at recommending products and recommending routines. It’s not something that we’ve been able to apply to ourselves, because we haven’t unlocked Siena’s ability to read customer formulas, and so she can’t really speak to our formulas just yet, because she doesn’t know what they are. But I think that it is super important in these spaces where the product is super emotional. Again, I think AI is really great at giving recommendations when there is a path to an expert or a human that can really expand and validate and make customers feel heard. I think that’s what a lot of people want when they’re sharing their concerns, and Siena’s actually very good at that. We found sometimes she’s gotten a few tickets she hasn’t been supposed to get into, and not only were they correct answers, but she did an amazing job validating the customer’s experience. We’ve literally taken things that she has said to people and put them into our own macros.

Tiering AI: simple questions to Siena, nuance and troubleshooting to agents

Lisa (5:50) What do you think about AI sitting in that space? How are you approaching it?

Jessica (5:54) So we’ve broken it up into tiers, basically, and we have Siena handling our tier one level of product questions, which are super straightforward, very specific questions. So like: are your products vegan? Are your products color safe? Can I use them on my keratin-treated hair? Things like that. Things that are really specific, and we’ve given her various knowledge sources to connect and check. Things that are a little bit more nuanced we leave to agents, so that they can do additional research and really look at all the products being recommended to the customer and the exact ingredients, and really speak to that.

Lisa (6:27) Do you have any programs, like maybe white glove or loyalty, that you’re also running?

Jessica (6:35) We have a very extensive — we call it troubleshooting — process for customers that are unhappy with their products. That is our highest tier. Our most experienced agents handle these tickets. If a customer is not happy with their formulas — we’re in a very unique position as a company that makes custom products to be able to change that. A lot of companies can’t. When you don’t like their products, it’s a little bit just like, “oh well.” But they walk customers through the entire experience. They learn more about how they’re using each product, what step in their routine they’re using the product, any other products they’re using. They really get a full look at their routine, and then assess whether maybe this tip would work for them, or maybe we should just try a second round of their formula and use our Review and Refine feature. And they’ll hold our customers’ hands through that process and get them new adjusted formulas.

What the beauty industry gets wrong about CX and AI

Lisa (7:24) What do you feel like the beauty industry gets wrong about CX and AI?

Jessica (7:29) I think it’s many things. I think that CX for beauty has the potential to be so valuable, to really convert customers and also build retention. Again, you mentioned earlier, so many companies have so much data on their customers. We know so much about them. People should really be using that information and trying to keep customers, trying to convert them, really making them feel heard and supported. I think with AI, people assume that CX is easy. It’s operational. Why not just automate everything? And I don’t think that that’s the best path forward. I think that with AI and automation, you have to be really strategic about how you use it, because it is an incredible tool, but at the end of the day, you need humans in there at some point to balance the experience and to try to foster those relationships. We do a yearly evaluation of everything that we’re automating, whether or not it still makes sense, and any topics that we think we could still build on top of or add to Siena’s list of things to handle. So yeah, I would say that it’s not really a set-it-and-forget-it kind of thing. It’s something that requires a lot of active work, and it’s important to invest in both the AI side and the human side of the experience.

How buying hair care online will change

Lisa (8:34) How do you think it will look when someone buys hair care online in the future?

Jessica (8:37) I think it’ll look so different. I mean, I think we’re starting to see the beginning of it with ChatGPT scraping Reddit for reviews and things. I think customers will become a lot more knowledgeable, and I think reviews will matter even more than they currently do, because there’s no hiding bad reviews with Reddit or ChatGPT, things like that. Customers have so much more knowledge at their fingertips and in a way are becoming experts in their own right. And so I think it’ll be a lot harder, but there are a lot more opportunities as well. Even being able to compare two products is so easy now. For a long time I was my family’s product researcher, and it’s so easy for them to do it themselves now. You don’t have to have multiple tabs open and read reviews on five sites. You can just very simply ask one engine. And so I think a lot of people will be a lot more knowledgeable, because you don’t necessarily have to be interested in it to have that kind of knowledge anymore.

Surfacing insights in minutes, and catching problems before they surface

Lisa (9:26) What has your experience been like, being able to finally surface these types of insights super fast?

Jessica (9:33) I would say we’ve barely scratched the surface of what we can — not for lack of trying. I think it’s just a learning curve, to be able to ask these questions that I’ve had but always thought there’s no real way to find an answer. I’ve wanted a tool like Ask Siena for many years now. I’ve always found issue tracking in support platforms very difficult. Or it’s not that it’s difficult — you have to choose what’s most important. Messages aren’t static. They can contain multiple problems, and you have to choose: okay, well, what’s the most important problem? What do I track this as? And so for me it’s been great to disregard that. That’s still great directional information, but it’s been so nice to be able to think, okay, those things don’t matter, this is what I’m interested in — let me just ask Siena and she’ll know. I recently used it to pull a bunch of customer emails about a missing transactional email we don’t send. I have a customer desire for this transactional email, which previously I would find with keywords or by looking up specific language that we use in response. So it’s messy. It’s not super clean. But Siena is able to truly, in minutes, search thousands of inquiries and be able to put numbers behind something that previously I could really only speak to qualitatively. So it’s really nice.

Lisa (10:47) Do you have other examples where you might have learned something or caught something proactively — a pattern or product signal, or maybe a problem surfacing before it happens?

Jessica (10:55) We haven’t caught anything yet, knock on wood. I’m sure that there is a ton of stuff. I think really we’re still all learning what do we even ask. The world is truly our oyster with Ask Siena, and so it’s very open-ended, and it can be difficult to pin down. Our education manager has been using it recently to surface insights on a product that was receiving very specific feedback from a bunch of users. And it’s something that we noticed months ago, maybe even a year ago, I’m not sure. And we were able to surface it to our development team. But I have to imagine that with a tool like Ask Siena, we would have been able to catch and flag this specific small trend so much sooner. I think it’ll be really great as we launch more products and can just have a scheduled run frequently set up to be like: check for mentions of this product, accumulate all feedback, make it into a doc and share it. I’ve been using it primarily to improve our automations, which has been really nice, because some of our automations are several years old, and so they’ve accumulated some things that don’t need to be there anymore. It’s been so helpful at cleaning up the automations. And then also we’re monitoring how it impacts the handback rate from Siena to a human. We’re hoping to see a pretty big decrease in that. Very exciting.

Lisa (12:09) Do you have any projects or goals related to turning CX into more of a proactive function?

Turning CX into a proactive, insight-driven function

Jessica (12:14) I think that it will unlock our ability to have more true voice-of-the-customer reporting more regularly. We’ve long reported on outreach topics, but changing how we view how we report, and also how we market our report across the company, into something that’s really more insight-driven — less about numbers and more about sentiment. It’s so much easier to capture sentiment and put numbers to it with Siena. This many people felt this way. Versus before, it’s like: okay, well, I know this many people asked to cancel their order, but I don’t know if it was positive or negative unless I spent tons of time digging through conversations.

Lisa (12:50) Do you feel like it changes the type of conversations you can have in a leadership meeting?

Jessica (12:57) I mean, I think that it democratizes access to customers a little bit. In a leadership meeting, if we need an additional data point, it’d be so easy for us all to just pull up Ask Siena, ask her a quick question, and see the answers. And I think it gives us the space to, yes, still share quantitative things — numbers are important — but to focus more on the sentiment and have more qualitative data laced through for real impact. Because even if it’s a smaller amount, if it’s a really negative sentiment, it should probably be addressed. Versus, there are some things people might just always be unhappy about.

Lisa (13:28) Do you still see CX in your organization, or in our e-com space, treated as a cost center? Or do you feel like AI is finally starting to shift that perception?

Is CX still treated as a cost center?

Jessica (13:40) I think that it’s starting to shift, but I still think that there’s a lot of work to go. At the end of the day, I also think CX is something to invest in. Good customer service is great for your business. A good customer service moment can change someone’s perception about your whole brand, versus a bad one can really ruin it and even make them want to tell their friends not to touch your brand. And so I think it’s changing, because we have access to so much more outside of just static data points like tags and issue types. But I think that there’s still a lot of work to be done in that fight.

Lisa (14:08) When you adopted AI, I think CX was one of the first departments where you adopted it. You definitely led the way there. How did your team react to it? How did their roles change throughout the last couple of years?

How the team reacted to adopting AI, and the skills that matter now

Jessica (14:24) I think our team was pretty nervous at first, which I totally understand. I think a lot of companies have just onboarded AI and gotten rid of humans, and then had to rehire them. But that was never our intention. Our intention was always to automate what they don’t need to spend time on — super operational things — so that they can spend their time now really becoming masters of formulas, or developing skills in other places. Despite having a smaller team, I think that we’ve really been able to keep focus on the moments that we think are most important in the customer journey, whereas maybe we would not have been able to put so much emphasis and focus there before. We can do retrainings on topics. We have just extra space to function.

Lisa (15:02) What do you think are the most important skills now, if we take CX for example, that would make someone hirable?

Jessica (15:11) I think people that ask questions are really great. I’m someone with a lot of questions. I like to know how things work. And I think that’s a lot of what customer service is: understanding how things work, solving problems, which can be very fun. And then also people that are really adaptable. Things change all the time in customer service, and just in work and the world, everywhere, all the time. It’s kind of important to roll with the punches. I definitely was not good at rolling with the punches before, and it’s something I really had to work on, and it’s very freeing. I think once you accept that things change all the time, you can stop dwelling on it and just focus on solutions. Being a little scrappy, trying to find answers yourself and figure things out.

Lisa (15:47) I feel like these skills all apply very well in AI, in the new world.

Jessica (15:52) Constantly changing. A small model change can lead to a very different result in AI’s output. I think a while ago you guys changed models, and we were like, “Oh my gosh, Siena is following our instructions a little bit too well.” And we had to make changes because we were like, “Okay, thank you for following our instructions, but now you’re sounding a little robotic.” I mean, it only took a few small tweaks and it was super easy. It’s gotten easier and easier to work with AI as time has gone on, for me at least, and especially working with Siena. You guys are always making tools for your operators, which I really appreciate. It’s truly made my life so much easier. I think with AI, the landscape is still shifting, and we still don’t know how the cookies are going to crumble.

Lisa (16:29) Have you been adding other AI tools on top, or how are you thinking about it moving forward?

Jessica (16:32) As an organization, we are using Claude, and we’ve been using Claude on the CX org. Before we had Ask Siena, we had Claude connected to Kustomer through an MCP. Ask Siena is much better. And we still have our knowledge sources connected to Claude, which has been really useful. I use it all the time, rather than just searching in our knowledge tool. Being able to ask Claude and pull all the relevant information about something has been incredible. We haven’t onboarded any other major tools, but we’ve tried to create an ecosystem where all of our tools talk to each other. And we’re hoping to get more AI tools for our reps as well, so that when they get stuck on a ticket, there’s something to talk to and something to guide them, but that’s based on all of our processes and knowledge. I think that the more interconnected everything is, the better, for the most part. I’ve worked on our tooling for the team for so long, and my goal is always to reduce the number of tabs they have open. I know some people can function very well with many tabs. I’m not one of them. And so my mission primarily has been to reduce my tabs, which typically reduces tabs for others as well. So the more connected everything is, the better. I think that having siloed data doesn’t really help anyone.

Lisa (17:41) What are you working on, for example, in the next 6 to 12 months? What are some of your goals when it comes to automation or expansion of your AI services?

The roadmap ahead, and advice for brands choosing AI-native vs. add-on

Jessica (17:51) We plan out our roadmap with Siena for the year at the end of the previous year, so our roadmap has been planned for a while. We have a new integration that we’re launching with you guys soon. It’s basically done. It’s almost done. And we have plans for one more. We’ve really thought about building these in a way where the experience stacks on top of each other. So we did updating addresses this time. And what will be great about that especially is that humans can’t get to it fast enough, typically. Our operations team is incredible, and they move extremely quickly making customer formulas. So by the time a ticket’s been sitting in the queue, it’s often too late to update their address. It’ll be so great to have Siena be able to get in there and update them. We’ll very much so reduce mis-shipments, et cetera. And then after that, we’ve planned for her ability to replace an order, so to really help with lost packages and things like that.

Lisa (18:41) What’s one conversation you wish more brands would have about AI right now, or advice that you would give?

Jessica (18:49) I would say to be very strategic with how it’s being used. I think AI is very much a new shiny toy for a lot of people, and it has amazing applications, especially with data, as I’ve sung the praises of Ask Siena. It’s truly incredible. But I would think about AI in how it can enhance an experience, and not replace something, necessarily.

Lisa (19:09) How would you describe Ask Siena in one or two words?

Jessica (19:12) I don’t know. That’s really difficult, because it covers so much. I mean, until the demo we had a few weeks ago, I didn’t even realize half the things I could be asking Ask Siena. I guess just like: data, unlocked. Not even co-pilot, I don’t think, but like a true CX analyst. I mean, earlier in my career I would spend a lot of time trying to find very small trends to report on in customer outreach so they didn’t get lost. And I’d spend so much time looking through tickets to get a sense of volume, to rule tickets out, being like, “No, this is not what I’m looking for. This is what I’m looking for.” It takes something that would have taken days, weeks, months even, and boils it down into something that can happen so quickly. Like, immediately, basically. I think that it really will allow us to move a lot quicker, especially if we walk into an inbox that’s at 2,000. Hi, why is our backlog 2,000? To really be able to understand what’s happening so quickly, and not spend hours looking through tickets. The contact reason, which is great, is directional at best. We still don’t know exactly what has happened for such a spike. So it’s really hard to boil down Ask Siena into two words, but yeah, “CX analyst” I feel like works, is close.

Lisa (20:20) What would you advise someone that is on the fence between going with an AI-native solution like Siena versus maybe an AI add-on that their help desk or their existing ticket provider would offer?

Jessica (20:34) I think it’s really about the level of control that you want. We wanted a lot of control over the output, and we found that a tool like Siena was best at that. Being able to create how the customer responds, sure, but also provide so much knowledge that would otherwise be missing was non-negotiable for us, really. I can’t imagine having just a kind of add-on feature that doesn’t allow for me to go in and say, “Okay, never touch this, but always touch that. Handle it exactly this way,” and really provide such nuanced instructions based on various different possible outcomes and solutions. I think that we’re able to provide much more custom responses as a result, because we can take a single intent and say, okay, but if it’s this, respond like this, or if it’s that, do something else. I think that that’s what makes AI good, and not just something that your human support staff then have to pick up after later.

Lisa (21:28) Those are good criteria for helping other brands evaluate solutions. Thank you for sharing.