May 31, 2018

PICA Protocol: A Visualization Prescription for Impactful Data Storytelling – Whiteboard Friday

Posted by Lea-Pica

If you find your presentations are often met with a lukewarm reception, it's a sure sign it's time for you to invest in your data storytelling. By following a few smart rules, a structured approach to data visualization could make all the difference in how stakeholders receive and act upon your insights. In this edition of Whiteboard Friday, we're thrilled to welcome data viz expert Lea Pica to share her strategic methodology for creating highly effective charts.

A Visualization Prescription for Impactful Storytelling

Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Hello, Moz fans. Welcome to another edition of Whiteboard Friday. I'm here to talk to you this week about a very hot topic in the digital marketing space. So my name is Lea Pica, and I am a data storytelling trainer, coach, speaker, blogger, and podcaster at LeaPica.com.

I want to tell you a little story. So as 12 years I spent as a digital analyst and SEM, I used to present insights a lot, but nothing ever happened as a result of it. People fell asleep or never responded. No action was being taken. So I decided to figure out what was happening, and I learned all these great tricks for doing it.

What I learned in my journey is that effective data visualization communicates a story quickly, clearly, accurately, and ethically, and it had really four main goals — to inform decisions, to inspire action, to galvanize people, and most importantly to communicate the value of the work that you do.

Now, there are lots of things you can do, but I was struggling to find one specific process that was going to help me get from what I was trying to communicate to getting people to act on it. So I developed my own methodology. It's called the PICA Protocol, and it's a visualization prescription for impactful data storytelling. What I like about this protocol is that it's practical, approachable. It's not complicated. It's prescriptive, and it's repeatable. I believe it's going to get you where you need to go every time.

So let's say one of your managers, clients, stakeholders is asking you for something like, "What are our most successful keyword groups?" Something delightfully vague like that. Now, before you jump into your data visualization platform and start dropping charts like it's hot, I want you to take a step back and start with the first step in the process, which is P for purpose.

P for Purpose

So I found that every great data visualization started with a very focused question or questions.

  • Why do you exist? Get philosophical with it.
  • What need of my audience are you meeting?
  • What decisions are you going to inform?

These questions help you get really focused about what you're going to present and avoid the sort of needle in a haystack approach to seeing what might stick.

So the answers to these questions are going to help you make an important decision, to choose an appropriate chart type for the message that you're trying to convey. Some of the ways you want to do that — I hear you guys are like into keywords a little bit — you want to listen for the keywords of what people are asking you for. So in this case, we have "most successful." Okay, that indicates a comparison. Different types or campaigns or groups, those are categories. So it sounds like what we're going for is a categorical comparison. There are other kinds of keywords you can look for, like changing over time, how this affects that. Answers or opinions. All of those are going to help you determine your most appropriate visual.

Now, in this case, we have a categorical comparison, so I always go back to basics. It's an oldie but goodie, but we're going to do the tried-and-true bar chart. It's universally understood and doesn't have a learning curve. What I would not recommend are pie charts. No, no, no. Unless you only have two segments in your visual and one is unmistakably larger than the other, pie charts are not your best choice for communicating categorical comparison, composition, or ranking.

I for Insight

So we have our choice. We're now going to move on to the next step in the methodology, which is I for insight. So an insight is something that gives a person a capacity to understand something quickly, accurately, and intuitively. Think of those criteria.

So here, does my display surface the story and answer these questions intuitively? That's our criteria. The components of that are:

  • Layout and orientation. So how is the chart configured? Very often we'll use vertical bar charts for categorical comparison, but that will end up having diagonal labels if they're really long, and unless your audience walks around like this all the time, it's going to be confusing because that would be weird. So you want to make sure it's oriented well.
  • Labeling. In the case of bars, I always prefer to label each bar directly rather than relying on just an axis, because then their eyes aren't jumping from bar to axis to bar to axis and they're paying more attention to you. That's also for line charts. Very often I'll label a line with a maximum, a minimum, and maybe the most important data point.
  • Interpretation of the data and where we're placing it, the location.
    • So our interpretation, is it objective or is it subjective? So subjective words are like better or worse or stupid or awesome. Those are opinions. But objective words are higher, lower, most efficient, least efficient. So you really want your observations to be objective.
    • Have you presented it ethically? Or have you manipulated the view in a way that isn't telling a really ethical picture, like adjusting a bar axis above zero, which is a no-no? But you can do that with a line graph in certain cases. So look for those nuances. You want to basically ask yourself, "Would I be able to uphold this visual in a court of law or sleep at night?"
    • Location of that insight. So very often we'll put our insights, our interpretation down here or in really tiny letters up here. Then up here we'll put big letters saying this is sales, my keyword category. No. What we want to do is we want to put our interpretation up here. This top area is the most important real estate on your visual. That's where their eyes are going to look first. So think of this like a BuzzFeed headline for your visual. What do you want them to take away? You can always put what the chart is here in a little subtitle.
  • Make recommendations. Because that's what a really powerful visual is going to do.
    • I always suggest having two recommendations at least, because this way you're empowering your audience with a choice. This way you can actually be subjective. That is okay in this case, because that's your unique subject matter expertise.
    • Are your recommendations accountable to specific people? Are they feasible?
    • What's the cost of not acting on your recommendations? Put some urgency behind it. So I like to put my recommendations in a little box or callout on the side here so it's really clear after I've presented my facts.

C for Context

The next step in the methodology is C for context. What this is saying is, "Do I have all the data points I need to paint a complete picture, or is there more to this story?" So some additional lenses you might find useful are past period comparison, targets or benchmarks are useful, segmentation, things like geography, mobile device. Or what are the typical questions or arguments that your audience has when you present data? They can be super value contextual points.

In this case, I might decide that while they care about the number of sales, because that's most successful to them, I care about the keywords "conversion rates." So I'm going to add a second bar chart here like this, and I'm going to see there's a different story that's popping out here now.

Now, this is where your data storytelling really comes into play. This particular strategy is called a table lens or a side-by-side bar chart. It's what I recommend if you want to combine two categorical metrics together.

A for Aesthetics

Now, the last step in the methodology is A for aesthetics. Aesthetics are how things look. So it's not about making it look pretty. No, it's asking, "Does my viz comply with brain best practices of how we absorb information?"

1. Decrease visual noise

So the first step in doing that is we want to decrease visual noise, because that creates a lot of tension. So decreasing noise will increase the chance of a happy brain.

Now, I'm a crunchy granola hippie, so I love to detox every day. I've developed a data visualization detox that entails removing things like grid lines, borders, axis lines, line markers, and backgrounds. Get all of that junk out of there, really clean up. You can align everything to the left to make sure that the brain is following things properly down. Don't center everything.

2. Use uniform colors (plus one standout color for emphasis)

Now, you'll notice that most of my bars here have a uniform color — simple black. I like to color everything one color, because then I'll use a separate, standout color, like this blue, to strategically emphasize my key message. You might notice that I did that throughout this step for the words that I want you to pick out. That's why I colored these particular bars, because this feels like the story to me, because that is the storytelling part of this message.

Notice that I also colored the category in my observation to create a connective tissue between these two items. So using color intentionally means things like using green for good and red for bad, not arbitrarily, and then maybe blue for what's important.

3. Source your data

Then finally, you always want to source your data. That increases the trust. So you want to put your platform and your date range. Really simple.

So this is the anatomy of an awesome data viz. I've adapted it from a great book called "Good Charts" by my friend, Scott Berinato. What I have found that by using this protocol, you're going to end up with these wonderful, raving fans who are going to love your work and understand your value. I included a little kitty fan because I can. It's my Whiteboard Friday.

So that is the protocol. I actually have included a free gift for you today. If you click the link at the end of this post, you'll be able to sign up for a Chart Detox Checklist, a full printable PICA Protocol prescription and a Chart Choosing Guide.

Get the PICA Protocol prescription

I would actually love to hear from you. What are the kinds of struggles that you have in presenting your insights to stakeholders, where you just feel like they're not getting the value of what you're doing? I'd love to hear any questions you have about the methodology as well.

So thank you for watching this edition of Whiteboard Friday. I hope you enjoyed it. We'll see you next week, and please remember to viz responsibly, my friends. Namaste.

Video transcription by Speechpad.com


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

May 29, 2018

Getting Real with Retail: An Agency’s Guide to Inspiring In-Store Excellence

Posted by MiriamEllis

A screenshot of a negative 1-star review citing poor customer service

No marketing agency staffer feels good when they see a retail client getting reviews like this on the web.

But we can find out why they’re happening, and if we’re going above-and-beyond in our work, we just might be able to catalyze turning things around if we’re committed to being honest with clients and have an actionable strategy for their in-store improvements.

In this post, I’ll highlight some advice from an internal letter at Tesla that I feel is highly applicable to the retail sector. I’d also like to help your agency combat the retail blues headlining the news these days with big brands downsizing, liquidating and closing up shop — I’m going to share a printable infographic with some statistics with you that are almost guaranteed to generate the client positivity so essential to making real change. And, for some further inspiration, I’d like to offer a couple of anecdotes involving an Igloo cooler, a monk, reindeer moss, and reviews.

The genuine pain of retail gone wrong: The elusive cooler, "Corporate," and the man who could hardly stand

“Hi there,” I greeted the staffer at the customer service counter of the big department store. “Where would I find a small cooler?”

“We don’t have any,” he mumbled.

“You don’t have any coolers? Like, an Igloo cooler to take on a picnic to keep things cold?”

“Maybe over there,” he waved his hand in unconcern.

And I stood there for a minute, expecting him to actually figure this out for me, maybe even guide me to the appropriate aisle, or ask a manager to assist my transaction, if necessary. But in his silence, I walked away.

“Hi there,” I tried with more specificity at the locally owned general store the next day. “Where would I find something like a small Igloo cooler to keep things cold on a picnic?”

“I don’t know,” the staffer replied.

“Oh…” I said, uncomfortably.

“It could be upstairs somewhere,” he hazarded, and left me to quest for the second floor, which appeared to be a possibly-non-code-compliant catch-all attic for random merchandise, where I applied to a second dimly illuminated employee who told me I should probably go downstairs and escalate my question to someone else.

And apparently escalation was necessary, for on the third try, a very tall man was able to lift his gaze to some coolers on a top shelf… within clear view of the checkout counter where the whole thing began.

Why do we all have experiences like this?

“Corporate tells us what to carry” is the almost defensive-sounding refrain I have now received from three employees at two different Whole Foods Markets when asking if they could special order items for me since the Amazon buyout.

Because, you know, before they were Amazon-Whole Foods, staffers would gladly offer to procure anything they didn’t have in stock. Now, if they stop carrying that Scandinavian vitamin D-3 made from the moss eaten by reindeer and I’ve got to have it because I don’t want the kind made by irradiating sheep wool, I’d have to special order an entire case of it to get my hands on a bottle. Because, you know, “Corporate.”

Why does the distance between corporate and customer make me feel like the store I’m standing in, and all of its employees, are powerless? Why am I, the customer, left feeling powerless?

So maybe my search for a cooler, my worries about access to reindeer moss, and the laughable customer service I’ve experienced don’t signal “genuine pain.” But this does:

Screenshot of a one-star review: "The pharmacy shows absolutely no concern for the sick, aged and disabled from what I see and experienced. There's 2 lines for drops and pick up, which is fine, but keep in mind those using the pharmacy are sick either acute or chronic. No one wants to be there. The lines are often long with the phone ringing off the hook, so very understaffed. There are no chairs near the line to sit even if someone is in so much pain they can hardly stand, waiting area not nearby. If you have to drop and pick you have to wait in 2 separate lines. They won't inform the other window even though they are just feet away from each other. I saw one poor man wait 4 people deep, leg bandaged, leaning on a cart to be able to stand, but he was in the wrong line and was told to go to the other line. They could have easily taken the script, asked him to wait in the waiting area, walk the script 5 feet, and call him when it was his turn, but this fella who could barely stand had to wait in another line, 4 people deep. I was in the correct line, pick up. I am a disabled senior with cancer and chronic pain. However, I had a new Rx insurance card, beginning of the year. I was told that to wait in the other line, too! I was in the correct line, but the staff was so poorly trained she couldn't enter a few new numbers. This stuff happens repeatedly there. I've written and called the home office who sound so concerned but nothing changes. I tried to talk to manager, who naturally was "unavailable" but his underling made it clear their process was more important than the customers. All they have to do to fix the problem is provide nearby sitting or ask the customer to wait in the waiting area where there are chairs and take care of the problem behind the counter, but they would rather treat the sick, injured and old like garbage than make a small change that would make a big difference to their customers. Although they are very close I am looking for a pharmacy who actually cares to transfer my scripts, because I feel they are so uncaring and disinterested although it's their job to help the sick."

This is genuine pain. When customer service is failing to the point that badly treated patrons are being further distressed by the sight of fellow shoppers meeting the same fate, the cause is likely built into company structure. And your marketing agency is looking at a bonafide reputation crisis that could presage things like lawsuits, impactful reputation damage, and even closure for your valuable clients.

When you encounter customer service disasters, it begs questions like:

  1. Could no one in my situation access a list of current store inventory, or, barring that, seek out merchandise with me instead of risking the loss of a sale?
  2. Could no one offer to let “corporate” know that I’m dissatisfied with a “customer service policy” that would require me to spend $225 to buy a whole case of vitamins? Why am I being treated like a warehouse instead of a person?
  3. Could no one at the pharmacy see a man with a leg wound about to fall over, grab a folding chair for him, and keep him safe, instead of risking a lawsuit?

I think a “no” answer to all three questions proceeds from definite causes. And I think Tesla CEO, Elon Musk, had such causes in mind when he recently penned a letter to his own employees.

“It must be okay for people to talk directly and just make the right thing happen.”

“Communication should travel via the shortest path necessary to get the job done, not through the 'chain of command.' Any manager who attempts to enforce chain of command communication will soon find themselves working elsewhere.

A major source of issues is poor communication between depts. The way to solve this is allow free flow of information between all levels. If, in order to get something done between depts, an individual contributor has to talk to their manager, who talks to a director, who talks to a VP, who talks to another VP, who talks to a director, who talks to a manager, who talks to someone doing the actual work, then super dumb things will happen. It must be ok for people to talk directly and just make the right thing happen.

In general, always pick common sense as your guide. If following a 'company rule' is obviously ridiculous in a particular situation, such that it would make for a great Dilbert cartoon, then the rule should change.”
- Elon Musk, CEO, Tesla

Let’s parlay this uncommon advice into retail. If it’s everyone’s job to access a free flow of information, use common sense, make the right thing happen, and change rules that don’t make sense, then:

  1. Inventory is known by all store staff, and my cooler can be promptly located by any employee, rather than workers appearing helpless.
  2. Employees have the power to push back and insist that, because customers still expect to be able to special order merchandise, a specific store location will maintain this service rather than disappoint consumers.
  3. Pharmacists can recognize that patrons are often quite ill and can immediately place some chairs near the pharmacy counter, rather than close their eyes to suffering.

“But wait,” retailers may say. “How can I trust that an employee’s idea of ‘common sense’ is reliable?”

Let’s ask a monk for the answer.

“He took the time...”

I recently had the pleasure of listening to a talk given by a monk who was defining what it meant to be a good leader. He hearkened back to his young days, and to the man who was then the leader of his community.

“He was a busy man, but he took the time to get to know each of us one-on-one, and to be sure that we knew him. He set an example for me, and I watched him,” the monk explained.

Most monasteries function within a set of established rules, many of which are centuries old. You can think of these guidelines as a sort of policy. In certain communities, it’s perfectly acceptable that some of the members live apart as hermits most of the year, only breaking their meditative existence by checking in with the larger group on important holidays to share what they’ve been working on solo. In others, every hour has its appointed task, from prayer, to farming, to feeding people, to engaging in social activism.

The point is that everyone within a given community knows the basic guidelines, because at some point, they’ve been well-communicated. Beyond that, it is up to the individual to see whether they can happily live out their personal expression within the policy.

It’s a lot like retail can be, when done right. And it hinges on the question:

“Has culture been well-enough communicated to every employee so that he or she can act like the CEO of the company would in wide variety of circumstances?”

Or to put it another way, would Amazon owner Jeff Bezos be powerless to get me my vitamins?

The most accessible modern benchmark of good customer service — the online review — is what tells the public whether the CEO has “set the example.” Reviews tell whether time has been taken to acquaint every staffer with the business that employs them, preparing them to fit their own personal expression within the company’s vision of serving the public.

An employee who is able to recognize that an injured patron needs a seat while awaiting his prescription should be empowered to act immediately, knowing that the larger company supports treating people well. If poor training, burdensome chains of command, or failure to share brand culture are obstacles to common-sense personal initiative, the problem must be traced back to the CEO and corrected, starting from there.

And, of course, should a random staffer’s personal expression genuinely include an insurmountable disregard for other people, they can always be told it’s time to leave the monastery...

For marketing agencies, opportunity knocks

So your agency is auditing a valuable incoming client, and their negative reviews citing dirty premises, broken fixtures, food poisoning, slowness, rudeness, cluelessness, and lack of apparent concern make you say to yourself,

“Well, I was hoping we could clean up the bad data on the local business listings for this enterprise, but unless they clean up their customer service at 150 of their worst-rated locations, how much ROI are we really going to be able to deliver? What’s going on at these places?”

Let’s make no bones about this: Your honesty at this critical juncture could mean the difference between survival and closure for the brand.

You need to bring it home to the most senior level person you can reach in the organization that no amount of honest marketing can cover up poor customer service in the era of online reviews. If the brand has fallen to the level of the pharmacy I’ve cited, structural change is an absolute necessity. You can ask the tough questions, ask for an explanation of the bad reviews.

“But I’m just a digital marketer,” you may think. “I’m not in charge of whatever happens offline.”

Think again.

Headlines in retail land are horrid right now:

If you were a retail brand C-suite and were swallowing these predictions of doom with your daily breakfast, wouldn’t you be looking for inspiration from anyone with genuine insight? And if a marketing agency should make it their business to confront the truth while also being the bearer of some better news, wouldn’t you be ready to listen?

What is the truth? That poor reviews are symptoms smart doctors can use for diagnosis of structural problems.
What is the better news? The retail scenario is not nearly as dire as it may seem.

Why let hierarchy and traditional roles hold your agency back? Tesla wouldn’t. Why not roll up your sleeves and step into in-store? Organize and then translate the narrative negative reviews are telling about structural problems for the brand which have resulted in dangerously bad customer service. And then, be prepared to counter corporate inertia born of fear with some eye-opening statistics.

Print and share some good retail tidings

Local SEO infographic

Print your own copy of this infographic to share with clients.

At Moz, we’re working with enterprises to get their basic location data into shape so that they are ready to win their share of the predicted $1.4 trillion in mobile-influenced local sales by 2021, and your agency can use these same numbers to combat indecision and apathy for your retail clients. Look at that second statistic again: 90% of purchases are still happening in physical stores. At Moz, we ask our customers if their data is ready for this. Your agency can ask its clients if their reputations are ready for this, if their employees have what they need to earn the brand’s piece of that 90% action. Great online data + great in-store service = table stakes for retail success.

While I won’t play down the unease that major brand retail closures is understandably causing, I hope I’ve given you the tools to fight the “retail disaster” narrative. 85% more mobile users are searching for things like “Where do I buy that reindeer moss vitamin D3?” than they were just 3 years ago. So long as retail staff is ready to deliver, I see no “apocalypse” here.

Investing time

So, your agency has put in the time to identify a reputation problem severe enough that it appears to be founded in structural deficiencies or policies. Perhaps you’ve used some ORM software to do review sentiment analysis to discover which of your client’s locations are hurting worst, or perhaps you’ve done an initial audit manually. You've communicated the bad news to the most senior-level person you can reach at the company, and you've also shared the statistics that make change seem very worthwhile, begging for a new commitment to in-store excellence. What happens next?

While there are going to be nuances specific to every brand, my bet is that the steps will look like this for most businesses:

  1. C-suites need to invest time in creating a policy which a) abundantly communicates company culture, b) expresses trust in employee initiative, and c) dispenses with needless “chain of command” steps, while d) ensuring that every public facing staffer receives full and ongoing training. A recent study says 62% of new retail hires receive less than 10 hours of training. I’d call even these worrisome numbers optimistic. I worked at 5 retail jobs in my early youth. I’d estimate that I received no more than 1 hour of training at any of them.
  2. Because a chain of command can’t realistically be completely dispensed with in a large organization, store managers must then be allowed the time to communicate the culture, encourage employees to use common sense, define what “common sense” does and doesn’t look like to the company, and, finally, offer essential training.
  3. Employees at every level must be given the time to observe how happy or unhappy customers appear to be at their location, and they must be taught that their observations are of inestimable value to the brand. If an employee suggests a solution to a common consumer complaint, this should be recognized and rewarded.
  4. Finally, customers must be given the time to air their grievances at the time of service, in-person, with accessible, responsive staff. The word “corporate” need never come into most of these conversations unless a major claim is involved. Given that it may cost as much as 7x more to replace an unhappy customer than to keep an existing one happy, employees should be empowered to do business graciously and resolve complaints, in most cases, without escalation.

Benjamin Franklin may or may not have said that “time is money.” While the adage rings true in business, reviews have taught me the flip side — that a lack of time equals less money. Every negative review that cites helpless employees and poor service sounds to my marketing ears like a pocketful of silver dollars rolling down a drain.

The monk says good leaders make the time to communicate culture one-on-one.

Tesla says rules should change if they’re ridiculous.

Chairs should be offered to sick people… where common sense is applied.

Reviews can read like this:

Screenshot of a positive 5-star review: "Had personal attention of three Tesla employees at the same time. They let me sit in both the model cars they had for quite time time and let me freely fiddle and figure out all the gizmos of the car. Super friendly and answered all my questions. The sales staff did not pressure me to buy or anything, but only casually mentioned the price and test drive opportunities, which is the perfect touch for a car company like Tesla. "

And digital marketers have never known a time quite like this to have the ear of retail, maybe stepping beyond traditional boundaries into the fray of the real world. Maybe making a fundamental difference.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

May 28, 2018

Tracking Your Link Prospecting Using Lists in Link Explorer

Posted by Dr-Pete

I'm a lazy marketer some days — I'll admit it. I don't do a lot of manual link prospecting, because it's a ton of work, outreach, and follow-up. There are plenty of times, though, where I've got a good piece of content (well, at least I hope it's good) and I want to know if it's getting attention from specific sites, whether they're in the search industry or the broader marketing or PR world. Luckily, we've made that question a lot easier to answer in Link Explorer, so today's post is for all of you curious but occasionally lazy marketers. Hop into the tool if you want to follow along:

Open Link Explorer

(1) Track your content the lazy way

When you first visit Link Explorer, you'll see that it defaults to "root domain":

Some days, you don't want to wade through your entire domain, but just want to target a single piece of content. Just enter or paste that URL, and select "exact page" (once you start typing a full path, we'll even auto-select that option for you):

Now I can see just the link data for that page (note: screenshots have been edited for size):

Good news — my Whiteboard Friday already has a decent link profile. That's already a fair amount to sort through, and as the link profile grows, it's only going to get tougher. So, how can I pinpoint just the sites I'm interested in and track those sites over time?

(2) Make a list of link prospects

This is the one part we can't automate for you. Make a list of prospects in whatever tool you please. Here's an imaginary list I created in Excel:

Obviously, this list is on the short side, but let's say I decide to pull a few of the usual suspects from the search marketing world, plus one from the broader marketing world, and a couple of aspirational sites (I'm probably not going to get that New York Times link, but let's dream big).

(3) Create a tracking list in Link Explorer

Obviously, I could individually search for these domains in my full list of inbound links, but even with six prospects, that's going to take some time. So, let's do this the lazy way. Back in Link Explorer, look at the very bottom of the left-hand navigation and you'll see "Link Targeting Lists":

Keep scrolling — I promise it's down there. Click on it, and you'll see something like this:

On the far-right, under the main header, click on "[+] Create new list." You'll get an overlay with a three-step form like the one below. Just give your list a name, provide a target URL (the page you want to track links to), and copy-and-paste in your list of prospects. Here's an example:

Click "Save," and you should immediately get back some data.

Alas, no link from the New York Times. The blue icons show me that the prospects are currently linking to Moz.com, but not to my target page. The green icon shows me that I've already got a head-start — Search Engine Land is apparently linking to this post (thanks, Barry!).

Click on any arrow in the "Notes" column, and you can add a note to that entry, like so:

Don't forget to hit "Save." Congratulations, you've created your first list! Well, I've created your first list for you. Geez, you really are lazy.

(4) Check in to track your progress

Of course, the real magic is that the list just keeps working for you. At any time, you can return to "Link Tracking Lists" on the Link Explorer menu, and now you'll see a master list of all your lists:

Just click on the list name you're interested in, and you can see your latest-and-greatest data. We can't build the links for you, but we can at least make keeping track of them a lot easier.

Bonus video: Now in electrifying Link-o-Vision!

Ok, it's just a regular video, although it does require electricity. If you're too lazy to read (in which case, let's be honest, you probably didn't get this far), I've put this whole workflow into an enchanting collection of words and sounds for you:

I hope you'll put your newfound powers to good. Let us know how you're using Tracking Lists (or how you plan to use them) in the comments, and where you'd like to see us take them next!


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

May 27, 2018

How Much Data Is Missing from Analytics? And Other Analytics Black Holes

Posted by Tom.Capper

If you’ve ever compared two analytics implementations on the same site, or compared your analytics with what your business is reporting in sales, you’ve probably noticed that things don’t always match up. In this post, I’ll explain why data is missing from your web analytics platforms and how large the impact could be. Some of the issues I cover are actually quite easily addressed, and have a decent impact on traffic — there’s never been an easier way to hit your quarterly targets. ;)

I’m going to focus on GA (Google Analytics), as it's the most commonly used provider, but most on-page analytics platforms have the same issues. Platforms that rely on server logs do avoid some issues but are fairly rare, so I won’t cover them in any depth.

Side note: Our test setup (multiple trackers & customized GA)

On Distilled.net, we have a standard Google Analytics property running from an HTML tag in GTM (Google Tag Manager). In addition, for the last two years, I’ve been running three extra concurrent Google Analytics implementations, designed to measure discrepancies between different configurations.

(If you’re just interested in my findings, you can skip this section, but if you want to hear more about the methodology, continue reading. Similarly, don’t worry if you don’t understand some of the detail here — the results are easier to follow.)

Two of these extra implementations — one in Google Tag Manager and one on page — run locally hosted, renamed copies of the Google Analytics JavaScript file (e.g. www.distilled.net/static/js/au3.js, instead of www.google-analytics.com/analytics.js) to make them harder to spot for ad blockers. I also used renamed JavaScript functions (“tcap” and “Buffoon,” rather than the standard “ga”) and renamed trackers (“FredTheUnblockable” and “AlbertTheImmutable”) to avoid having duplicate trackers (which can often cause issues).

This was originally inspired by 2016-era best practice on how to get your Google Analytics setup past ad blockers. I can’t find the original article now, but you can see a very similar one from 2017 here.

Lastly, we have (“DianaTheIndefatigable”), which just has a renamed tracker, but uses the standard code otherwise and is implemented on-page. This is to complete the set of all combinations of modified and unmodified GTM and on-page trackers.

Two of Distilled’s modified on-page trackers, as seen on https://www.distilled.net/

Overall, this table summarizes our setups:

Tracker

Renamed function?

GTM or on-page?

Locally hosted JavaScript file?

Default

No

GTM HTML tag

No

FredTheUnblockable

Yes - “tcap”

GTM HTML tag

Yes

AlbertTheImmutable

Yes - “buffoon”

On page

Yes

DianaTheIndefatigable

No

On page

No

I tested their functionality in various browser/ad-block environments by watching for the pageviews appearing in browser developer tools:

Reason 1: Ad Blockers

Ad blockers, primarily as browser extensions, have been growing in popularity for some time now. Primarily this has been to do with users looking for better performance and UX on ad-laden sites, but in recent years an increased emphasis on privacy has also crept in, hence the possibility of analytics blocking.

Effect of ad blockers

Some ad blockers block web analytics platforms by default, others can be configured to do so. I tested Distilled’s site with Adblock Plus and uBlock Origin, two of the most popular ad-blocking desktop browser addons, but it’s worth noting that ad blockers are increasingly prevalent on smartphones, too.

Here’s how Distilled’s setups fared:

(All numbers shown are from April 2018)

Setup

Vs. Adblock

Vs. Adblock with “EasyPrivacy” enabled

Vs. uBlock Origin

GTM

Pass

Fail

Fail

On page

Pass

Fail

Fail

GTM + renamed script & function

Pass

Fail

Fail

On page + renamed script & function

Pass

Fail

Fail

Seems like those tweaked setups didn’t do much!

Lost data due to ad blockers: ~10%

Ad blocker usage can be in the 15–25% range depending on region, but many of these installs will be default setups of AdBlock Plus, which as we’ve seen above, does not block tracking. Estimates of AdBlock Plus’s market share among ad blockers vary from 50–70%, with more recent reports tending more towards the former. So, if we assume that at most 50% of installed ad blockers block analytics, that leaves your exposure at around 10%.

Reason 2: Browser “do not track”

This is another privacy motivated feature, this time of browsers themselves. You can enable it in the settings of most current browsers. It’s not compulsory for sites or platforms to obey the “do not track” request, but Firefox offers a stronger feature under the same set of options, which I decided to test as well.

Effect of “do not track”

Most browsers now offer the option to send a “Do not track” message. I tested the latest releases of Firefox & Chrome for Windows 10.

Setup

Chrome “do not track”

Firefox “do not track”

Firefox “tracking protection”

GTM

Pass

Pass

Fail

On page

Pass

Pass

Fail

GTM + renamed script & function

Pass

Pass

Fail

On page + renamed script & function

Pass

Pass

Fail

Again, it doesn’t seem that the tweaked setups are doing much work for us here.

Lost data due to “do not track”: <1%

Only Firefox Quantum’s “Tracking Protection,” introduced in February, had any effect on our trackers. Firefox has a 5% market share, but Tracking Protection is not enabled by default. The launch of this feature had no effect on the trend for Firefox traffic on Distilled.net.

Reason 3: Filters

It’s a bit of an obvious one, but filters you’ve set up in your analytics might intentionally or unintentionally reduce your reported traffic levels.

For example, a filter excluding certain niche screen resolutions that you believe to be mostly bots, or internal traffic, will obviously cause your setup to underreport slightly.

Lost data due to filters: ???

Impact is hard to estimate, as setup will obviously vary on a site-by site-basis. I do recommend having a duplicate, unfiltered “master” view in case you realize too late you’ve lost something you didn’t intend to.

Reason 4: GTM vs. on-page vs. misplaced on-page

Google Tag Manager has become an increasingly popular way of implementing analytics in recent years, due to its increased flexibility and the ease of making changes. However, I’ve long noticed that it can tend to underreport vs. on-page setups.

I was also curious about what would happen if you didn’t follow Google’s guidelines in setting up on-page code.

By combining my numbers with numbers from my colleague Dom Woodman’s site (you’re welcome for the link, Dom), which happens to use a Drupal analytics add-on as well as GTM, I was able to see the difference between Google Tag Manager and misplaced on-page code (right at the bottom of the <body> tag) I then weighted this against my own Google Tag Manager data to get an overall picture of all 5 setups.

Effect of GTM and misplaced on-page code

Traffic as a percentage of baseline (standard Google Tag Manager implementation):


Google Tag Manager

Modified & Google Tag Manager

On-Page Code In <head>

Modified & On-Page Code In <head>

On-Page Code Misplaced In <Body>

Chrome

100.00%

98.75%

100.77%

99.80%

94.75%

Safari

100.00%

99.42%

100.55%

102.08%

82.69%

Firefox

100.00%

99.71%

101.16%

101.45%

90.68%

Internet Explorer

100.00%

80.06%

112.31%

113.37%

77.18%

There are a few main takeaways here:

  • On-page code generally reports more traffic than GTM
  • Modified code is generally within a margin of error, apart from modified GTM code on Internet Explorer (see note below)
  • Misplaced analytics code will cost you up to a third of your traffic vs. properly implemented on-page code, depending on browser (!)
  • The customized setups, which are designed to get more traffic by evading ad blockers, are doing nothing of the sort.

It’s worth noting also that the customized implementations actually got less traffic than the standard ones. For the on-page code, this is within the margin of error, but for Google Tag Manager, there’s another reason — because I used unfiltered profiles for the comparison, there’s a lot of bot spam in the main profile, which primarily masquerades as Internet Explorer. Our main profile is by far the most spammed, and also acting as the baseline here, so the difference between on-page code and Google Tag Manager is probably somewhat larger than what I’m reporting.

I also split the data by mobile, out of curiosity:

Traffic as a percentage of baseline (standard Google Tag Manager implementation):


Google Tag Manager

Modified & Google Tag Manager

On-Page Code In <head>

Modified & On-Page Code In <head>

On-Page Code Misplaced In <Body>

Desktop

100.00%

98.31%

100.97%

100.89%

93.47%

Mobile

100.00%

97.00%

103.78%

100.42%

89.87%

Tablet

100.00%

97.68%

104.20%

102.43%

88.13%

The further takeaway here seems to be that mobile browsers, like Internet Explorer, can struggle with Google Tag Manager.

Lost data due to GTM: 1–5%

Google Tag Manager seems to cost you a varying amount depending on what make-up of browsers and devices use your site. On Distilled.net, the difference is around 1.7%; however, we have an unusually desktop-heavy and tech-savvy audience (not much Internet Explorer!). Depending on vertical, this could easily swell to the 5% range.

Lost data due to misplaced on-page code: ~10%

On Teflsearch.com, the impact of misplaced on-page code was around 7.5%, vs Google Tag Manager. Keeping in mind that Google Tag Manager itself underreports, the total loss could easily be in the 10% range.

Bonus round: Missing data from channels

I’ve focused above on areas where you might be missing data altogether. However, there are also lots of ways in which data can be misrepresented, or detail can be missing. I’ll cover these more briefly, but the main issues are dark traffic and attribution.

Dark traffic

Dark traffic is direct traffic that didn’t really come via direct — which is generally becoming more and more common. Typical causes are:

  • Untagged campaigns in email
  • Untagged campaigns in apps (especially Facebook, Twitter, etc.)
  • Misrepresented organic
  • Data sent from botched tracking implementations (which can also appear as self-referrals)

It’s also worth noting the trend towards genuinely direct traffic that would historically have been organic. For example, due to increasingly sophisticated browser autocompletes, cross-device history, and so on, people end up “typing” a URL that they’d have searched for historically.

Attribution

I’ve written about this in more detail here, but in general, a session in Google Analytics (and any other platform) is a fairly arbitrary construct — you might think it’s obvious how a group of hits should be grouped into one or more sessions, but in fact, the process relies on a number of fairly questionable assumptions. In particular, it’s worth noting that Google Analytics generally attributes direct traffic (including dark traffic) to the previous non-direct source, if one exists.

Discussion

I was quite surprised by some of my own findings when researching this post, but I’m sure I didn’t get everything. Can you think of any other ways in which data can end up missing from analytics?


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!