Any conversation with Bret is a deep learning experience. And it’s particularly interesting speaking with someone in the voice space with his marketing credentials.
Here’s the This winter it was as if CES was trying to prove to the holiday season that it could create the most buzz around voice assistants. So at least heading into 2018, it looks to be an exciting year for the #VoiceFirst movement. And Google recently announced winners of the Actions on Google Developer Challenge, with awards and cash going to what they judged were the best voicebots on the Google Assistant platform in 2017. It’s about time for someone to launch an awards show called the Bottys. At any rate, it’s worth looking into the winners and seeing what they’re doing. While we don’t have hard data to indicate if what they’re doing is working, at least they’re getting Google’s nod of approval. So I’m going to analyze the first, second and third place winners to see what we can learn from them. And without further ado, here they are. There are 4 areas we’ll assess to better understand how these apps work. The invocation is what you say to Google to get to the app. It’s the name. Now, voice apps aren’t that easy to discover. It’s not like people surf Alexa or Home like they might the web. Currently, users take a linear path through the medium, and typically have a clear idea of what they want to do or find out, if not specifically where they want to go. Categorically, audio information is consumed more linearly than graphic information. The invocation is no small part of an apps success. The easier to say and remember, the more likely people will be to actually use it. Here are the invocations for the three winners. 100 Years Ago “Talk to 100 Years Ago” Credit Card Helper “Ask credit card helper what the best credit card is” “Ask credit card helper what the risks of credit cards are” “Ask credit card helper what to do if my card is stolen” “Ask credit card helper what type of credit card i should get” “Talk to credit card helper” My Adventure Book “Talk to My Adventure Book” Credit card helper is using a range of invocations to touch on different user interests. Consider the difference between “what the best credit card is” and “what to do if my card is stolen”. The first one could drive users directly into a new customer funnel, and the second helping an existing customer’s make account changes. Both great features to offer, addressing people at different stages of the customer experience path. So developers and marketers will want to use terms that are easily remembered, easy to say, distinct from other voice apps, and offer directions to specific sections of the app, where possible. Once the user passes through to the app, it’s imperative to create a positive experience as quickly as possible. Like websites and mobile apps before them, voicebots will likely succeed or fail based on the first 30 seconds of the experience. And here’s how each app greets you after a successful invocation. These screens were grabbed while interfacing with Google Assistant on the phone, as it displays the script the Voicebot speaks for easy reference here. 100 Years Ago Credit Card Helper My Adventure Book You can see the relative simplicity of these apps. And of course that’s to be expected from first forays into new territory. However, even early web pages often provided link after link. This is an early indicator of what will likely be a challenge for the voice web for a while to come: The small amount of content that users can comfortably navigate and consume. One UX issue I noticed is, when going back through an app repeatedly, it’s great when you can skip through large sections of the script that you’ve already been through. Especially at the intro. Credit Card Helper did this very well, and it removed a sense of tedium from the process. You just have to say “Hey Google, skip” and you’re on to the next section. Here’s the main navigation structure for each app. 100 Years Ago Credit Card Helper My Adventure Book It makes sense the simplicity of the navigation reflects the simplicity of the overall app experience, although Credit Card Helper has extensive content on each of the credit cards in its database. A frequent discussion point in voicebot design is the optimal depth for a nav. I’ve heard more than one person say 3 is ideal. I think that might be where we are now, although I’m sure that will expand as people become more familiar with the technology. Additionally, the AI technology behind the speech interface is also going to improve dramatically, increasing the accuracy and quality of experience. Due to the invocation process for voicebots, one of the major marketing challenges is getting users to remember the exact invocation name. You might hear it on the radio, or see it in an ad, but when you go to visit the voicebot, via voice, you have to remember what’s likely to be a 3 word name. Not a simple task, especially when you consider marketers spend bazillions just to get consumers to remember their simple brand name. To help with finding apps, Google has created an app directory where each voicebot has its own page. This is an opportunity to prep the visitor for their upcoming user experience. Here are the Google Directory pages for each app. 100 Years Ago Credit Card Helper My Adventure Book Firstly, you can see how the many invocation phrases on the Credit Card Helper stands out, offering the user additional ways to invoke and discover the app. The description area offers brands a place to briefly (hopefully) summarize why users should visit the app. You can bet there will be a lot of discussions within marketing departments over what goes on this page. Suffice it to say that well written copy with a clear sense of the app’s mission is going to be critical. I’d also guess that Google will expand on the media available for presentation on this page. Given the recency of this industry, there’s not much in the way of best practices or industry standards. Brands are going to need to use this first wave of apps for as much design and user experience information as they can get. In addition to reviewing each app, I contacted the app creators to get additional insights about the app development process. So here are some of the highlights and challenges they reported. 100 Years Ago developer Jesse Vig used feedback from a previous app he developed to guide the direction for his winning voicebot. According to Jesse: Prior to building 100 Years Ago, I built another action called Time Machine that reads headlines from the past and plays a brief time travel sound effect. Surprisingly, most of the positive feedback I got was about the sound effect. Based on this experience, I wanted to create a richer audio experience with 100 Years Ago. Arun Rao, CEO of Starbutter, the company that developed Credit Card Helper, also has strategic advice for app developers: On the design side, state your Action’s key objectives early and try to design a “Wow” experience around them. If you do too much or don’t have a clear objective, your Action won’t be interesting. The first part of “Wow” is to not do anything really dumb – which takes a lot of user testing to figure out. I previously mentioned the challenges of designing information for a voice interface. Jesse Vig also addresses this, saying: I was able to create a much richer experience when the action had access to a visual display, as in the case of Google Assistant on mobile devices. Reading is faster than listening, and obviously images cannot be conveyed through an audio interface. Arun Rao has some good advice for approaching new apps: For use cases, think about where voice or chat interactivity adds much more value than a current experience. Test this out with prototypes or videos before you build anything (BotSociety is a good prototyping tool to start with). He also offers up a valuable technical recommendation for maximizing app performance: On the technical side, go serverless and use Google Cloud Functions or Amazon Lambda. These are efficient and more scalable and error-proof for webhooks than having a real or virtual server. Along a similar vein, Nick Laing, developer of My Adventure Book, suggested new developers use the existing tools. My advice to any beginners is to start with DialogFlow, there is a lot you can do with that console alone. Once you get familiar with the platform you can write your own code to expand the functionality. As the voice assistant industry enters it’s 4th year as a consumer gadget, there’s enormous potential on the horizon. These early examples are the tip of a large and growing iceberg. (And Earth needs more ice, right?) If one thing is certain, it’s that the devices, the apps, and the user base, are all going to evolve considerably over the next few years. It should be an exciting race. Thanks to Arun Rao, Jesse Vig and Nick Laing, not only for their work but their generosity in providing guidance for other app developers. I’ll be reviewing more voicebots in the coming weeks, so if you’d like to stay up on the latest creations, consider clicking the SUBSCRIBE VIA EMAIL button below. I’m not sure I’ve ever followed a story with the combination of awe and trepidation as AI for the past several years. The potential job displacement alone is staggering. In the past, as a marketing person who trades in creative thinking, I’ve clung to the idea that however far technology might progress, jobs based in aesthetic judgement were in the clear zone for the foreseeable future. Machine learning has been great on tasks with well-defined goals, like determining which ads in a campaign to give the most impressions to based on audience response. When the decision is highly subjective, though, it’s been a different situation. But that’s a barrier being broken through. Making the question not if creative jobs will be impacted, but how. The recent AI experiment by Google is an example. They turned an experimental deep learning system loose on their database of Street View images with the goal of artistic curation. Per their blog post. It mimics the workflow of a professional photographer, roaming landscape panoramas from Google Street View and searching for the best composition, then carrying out various postprocessing operations to create an aesthetically pleasing image. Here are two of hundreds of beautiful images the AI selected, cropped and processed. They had a team of professional photographers judge the results of the AI along side a range of other non-AI photos, with impressive results. There should be a lot of ways to use AI for creative development. What I find interesting in the Google example is the training of the AI system. Talk to any AI expert and they’ll emphasize the importance of having good training data. Garbage in, garbage out. Whatever you put into the system, that becomes the measuring stick. The Google team pulled the training database from several photography communities, using the human-generated rating system to define the aesthetic taste. (If you want the gory details, you can read the full research paper.) So the tastes of the photographic communities drove the training data which defined the results. But presumably you could use whatever data you want to train the system, and that’s the direction it will go in. Your new creative brief is the training data. Deployed on the full data set, off it goes generating concepts or creatives or whatever. In remarkable volume. And it never complains or causes HR nightmares. Sounds like a creative director’s dream. Or a chief marketing officer’s. Using AI to help creatives generate more ideas seems pretty obvious. You start a project with 100s of loosely-relevant-but-possibly-tangentially-interesting ideas already on the board. And AI may never be more than a powerful tool creatives have at their disposal. But like Photoshop and digital media in general, this is another way technology compresses deadlines and reduces budgets. It also gives agencies and creatives another learning curve to climb. With opportunities and pitfalls parallel to previous waves of technology. And of course you shouldn’t ignore the possibility of AI becoming more of a threat than a tool. The name of the Deep Learning system Google used to generate these images? Creatism. It’s stated purpose? Artistic content creation. You can almost hear a deep voice on a sizzle reel enthusing “Creatism, an artistic content creation agency.” If you work in digital marketing and haven’t read “Shipwreck Diary of a Content Marketer“ on McSweeney’s, put it at the top of your ToDo list. Actually, put it on your calendar to be sure it gets done. It’s a precise takedown of the marketing world’s latest growth area, content marketing. We join our castaway hero shortly after he washes ashore on a deserted island. The sole survivor of a content marketing misadventure. An apt analogy for any marketing effort gone awry. Despite his situation, there’s an immediate sense of confidence as he considers the various content marketing tools at his avail. And over the course of the coming days, our hero rifles through click bait, crowdsourcing, a viral effort, an infographic, newsletter, multimedia, a brand ambassadorship, a podcast (rightly sponsored by Squarespace), a VP of Disruption and Growth Hacking, and of course native advertising. Each of those shiny objects follows a similar trajectory, from enthused optimism to a fast crash and burn, with the latest solution hastily discarded for the next available tech. And that’s what I like most about this piece. That it pulls back the cover on the real problem with content marketing. The new technologies of scale and reach that have many thinking content marketing is fast, easy, and a matter of checking off a few boxes. That’s why it’s so ripe for ridicule. In fact, content marketing is anything but a latest trend. Each week content marketing gurus Pulizzi and Rose cover a historical example, sometimes dating back to the early 1900s. The key to those programs success obviously wasn’t any of the new technologies. It’s the quality of the content. Per Content Marketing Institute’s recent survey on content marketers, a full 86% of B2C companies use content marketing. However, an unfortunately high percentage of content marketing programs are operating without a documented strategy or mission statement. Sailing without a rudder, you could say. Want to avoid a shipwreck in your content program? Consider what might have happened in this story right before the shipwreck. No strategy? Lack of planning? Shortage of resources? New channels of distribution are great, and of course every marketer should use them where relevant. But they’re only the messenger. Great content marketing programs have great messages. There’s a lot more you can measure your Facebook content by than simply likes or engagements. In this article we’re going to look at why there’s a push for metrics beyond engagements, specific examples of metrics to use, and how to apply them. Social media introduced a metric type that hadn’t been used or discussed much prior. Engagement. Existing somewhere between an impression and a conversion action, engagements became a defining metric for evaluating the performance of social media content. Initially seen as a value-add for social media over static media, the idea was to provide a metric showing that while a lot of people may see something, there’s also a large number of them willing to actively engage with the content. Those engagements were potentially viewable by others on the social network, or at least those in the engager’s circle, and thus created an implicit endorsement for the brand. Sounds great, except those engagements don’t necessarily lead to the ultimate goal of a purchase or other conversion action. Or even the inclination to purchase. Perhaps you read “About Face”, the Facebook measurement report published late last year from the BBDO Comms team. It’s a well-supported report and outlines some alternate metrics to indicate your content is moving the audience closer to a purchasing mindset. Their setup is clear and straightforward, with several key points Taking this advice, points one and two mean you’re going to be paying for content impressions. And they won’t be targeted to your most core fans, as it’s actually the light buyers who, with a little prodding, can purchase more and increase sales. (Your core fans are already in max consumption mode, the theory goes.) Points 3 and 4 focus on our key interest. Using metrics to determine what content makes the best impact on our target audience. So taking a cue from the BBDO report, let’s take a look at metrics that go beyond basic engagement to represent a more successful brand impact. In “About Face”, the author’s cite a study showing the longer Facebook videos were viewed the more positive shift in brand recall and purchase intent. Pretty powerful stuff. If video view length is your KPI, Facebook offers some good data to determine which of your creative campaigns are performing best. One of the report’s key points is that marketers need to view social media content as part of a larger creative idea. In other words, it’s about campaign themes instead of one-offs. That’s because there are all sorts of wacky posting ideas that can generate engagement, but not real brand affinity. I think this is one of the biggest takeaways of the report, and a place where many marketers can go astray chasing after high engagement numbers. In a plug for my social analytics platform, Zuum has the capability to apply tags to posts for analyzing any grouping of posts. And that’s really the first step — determining what posts go into what campaign. That’s followed by analyzing them as a group, and then comparing metrics. Below you can see a chart showing analysis of four campaigns we analyzed with Zuum. This happens to be a comparison of all video posts from four different campaign themes: Food, art, event and skiing. Four Campaigns Analyzed for Video Viewing Retention Zuum also enables custom selection of any Facebook post metric available. Thus posts can be grouped by tags, then analyzed by any metric you choose. These are the metrics with column headers in yellow, above. Lastly, we can create additional metrics based on those core Facebook metrics. In this case, I’ve divided shares by impressions to know the ratio between those two. I’ve also calculated the % of video views that made it past 10 seconds (which BBDO Comms reported having high correlation with brand affinity). I’ve also created the average views for each video post column, and the views to impressions rate in the last column on the right. We can see the “event” campaign is the clear leader in the metric “% of views reaching 10 seconds or more”. If we take that same “% of views 10 seconds or more” and compare it to other metrics, like the shares-to-impressions ratio, or average reach, there’s a noticeable lack or correlation across the various campaigns. These findings are in support of the About Face report. While views of 10 seconds or more work great for video, we don’t have the same metric for other content types, like photos. However, if we take the principle of a piece of content catching someone’s eye and them wanting more, we can find other ways to measure that for different content formats. With photos, for example, there’s a photo consumption. This is when someone sees a photo in the newsfeed, and then clicks it to get a larger view of it. Pretty similar to someone seeing a few seconds of video and wanting to watch more. Thus by selecting only photos to analyze, and grouping them by campaign, we can determine what percent of photo posts people took the time to expand and view closer. In the “Post Consumptions Analysis” chart below, we’ve done just that with two campaigns consisting only of photo posts, tagged “event” and “skiing”. You can see how the skiing campaign has generated a far greater average reach per post, which is generally a welcome metric. However, if we look at how many people took the critical step to expand the photo for a closer look, which would lead to a photo consumption, we can see that the event campaign outperformed the skiing campaign by 43%. Post Consumptions Analysis I wouldn’t use this approach to compare photos to video views, but for a relative measure between different campaigns, there’s a strong behavioral similarity between someone wanting to see more of a video post, and someone wanting to see more of a photo post. Something caught their eye, and they demonstrated a desire for additional information. If you’ve been around online display advertising much, you may have heard about the report“Natural Born Clickers” Research showing people who click on banners have very little correlation to those making purchases or other conversion actions. Perhaps unsurprisingly, it seems the same thing is now happening on Facebook. One of the more interesting findings in “Face Off” is the idea that clicks are not a good measurement of brand affinity and purchase intent. This is probably counter-intuitive to many digital marketers, as the click is what ultimately leads to an online purchase. But it serves to underscore the importance of selecting the right metric when assessing a post or campaign’s performance. While apparently not all clickers are legitimate prospects, some clickers will move on to the conversion action. So the key is to continue tracking beyond the click and through to the conversion action. Doing so requires site analytics data. As Google Analytics (GA) is used on the majority of websites, I’ll use that as an example. In GA, there are two elements that need to be in place. 1. Goals need to be set up to record conversion actions, and 2. Campaign tracking codes need to be employed on the post links to ensure that GA will recognize all the dark social traffic from those posts as referral visits from the various social media networks. (If you’re not familiar with the concept of dark social, this will explain https://www.techopedia.com/definition/29027/dark-social). Below is an example of the “campaign” tracking parameter showing up in a Campaigns report in GA, under the Campaign column. In this case, I’ve labeled the various marketing efforts with a campaign name. Other parameters which can be tracked are the source, the medium and the content, each of which can also be viewed in this GA view. The report view shows how each of the different campaigns performed across the three funnel stages. Acquisition, Behavior and Conversions. Google Analytics Campaign Report By mapping these numbers back to your content campaign’s posting, impressions and reach data, you can get a clear idea of how your content is performing relative to your goals. This deeper level of prospect authentication should provide a more accurate read on the effectiveness of your content efforts in reaching your goals. The campaign URL tracking codes can be generated with Google’s Campaign URL Builder While research quoted in the About Face report stated sharing having no correlation with conversion actions, that doesn’t necessarily mean sharing has no contribution to brand impact. Especially for new brands looking to increase awareness, shares from loyal customers can introduce new customers to the brand. This behavior happens all day on Facebook. Below is a chart showing the previous two campaigns, “event” and “skiing”, with some additional viral metrics. While the event campaign did well in the Consumptions metric, when it comes to sharing and the resulting viral impressions, you can see the average viral impressions per post favors the skiing campaign. Impact of Sharing Like all data analysis, it really comes back to what your goals are, and which tactics are going to best help you reach them. In a few short years, social media has gone from talking about likes and engagements, to showing a deeper and more nuanced view of how content performs, and what brands can expect in return for their efforts. The key is knowing what you want to get out of social media and the content you post there. Once that’s clarified, then the rest falls into place. Have any preferred methods for evaluating content performance? I’d love to hear about them.Voicebot development insights from the winners of the Actions on Google Developer Challenge
Invocation
User Interface
App directory
Developer Comments
AI: The new creative director or the new creative?
A threat, a tool, or both?
A painfully funny commentary on content marketing
Measuring Brand Impact on Facebook: Choosing the Right Metrics
Social Media’s Engagement Problem
An Alternate Approach to Engagement
Video Viewing Retention Rate
Consumptions Instead of Views
To Click or Not To Click
The Value of a Share
ROI Has Come to Social Media