Michele Kiss's Blog at Web Analytics Demystified - Latest Comments

Re: The curse of bounce rate and ‘easy’ metrics (and why we can do better)

simon — Thu, 30 Apr 2015 05:30:40 -0000

Bounce rate is largely a red herring, especially if used in isolation and will different at different entry points and types of content page. Many people visit throughout a day, often visiting just one page and leaving it open for 30 minutes and the starting a new session. it would appear as 2 bounces in the majority of analytic tools. This can have dramatic distortions on the numbers. Lumps of traffic from specific sources can also have distortions where a poorly defined campaign lands consumers into a deep page and you get the back button issue creating spikes in bounces that overtime drags down the average. I entirely agree that so many sites have "in page" events that are possibly not tracked or links that take users to 3rd party sites so while the user is doing something the metrics do not record this such a video or scroll bar activity. Bounce rate as a word is also "negative" but actually someone coming in to read an article for 20 minutes and leaving is actually positive or scrolling to the bottom of the page is equally positive and they might well return later. Alongside bounce rate analysts should at least be looking at retention metrics to see how many of their visitors are actually returning within 10 days, they should look at "visitor" bounce rates, as opposed to "visit" bounce rates to smooth out the visit issues explained earlier and look at a custom engagement score that is tailored to the type of business. A news site is very different to a retail site and people visiting one page could be doing it for different reasons over a time period where different sites have a different consideration life cycle. Also remember visitors work across devices so you also need to look to cross match devices using matching technology. All stuff we do day to day at Cognesia but always keen to share our thoughts.

Re: Better ways to measure content engagement than time metrics

Yehoshua Coren — Thu, 12 Feb 2015 13:42:35 -0000

Hi Michele,

I really like this post and your suggestions for different metrics that you share at the end.

With new browser technologies, we're now able to get a bit closer to actually use time metrics as an engagement measure. For example, the amount of time that the browser tab was open and in focus. Coupled with the measurement of user interaction that you share so nicely, I think that time metrics can be woven back into our tapestry. I recently wrote about a new way to measure time on page here. http://www.analytics-ninja..... I would love to hear your thoughts.

Best,

Yehoshua

Re: The curse of bounce rate and ‘easy’ metrics (and why we can do better)

cleveyoung — Tue, 13 Jan 2015 18:28:03 -0000

I've seen way too many analysts use them by default without trying to better answer the requester's actual business question. Just because someone asks for a Bounce Rate report doesn't mean that's what they actually want. Most people don't really know what that means, or have enough savvy to use it in context. When an analyst first gets a request for a specific type of report, their first response should be "what is your business question", NOT simply pumping out that report (caveat is if you are sure the requester does know the nuances). If your not sure exactly how to answer that questions fully, that is your first clue that an easy metric like bounce rate is not what you should be providing.

Re: Three Foundational Tips to Successfully Recruit in Analytics

cleveyoung — Tue, 02 Dec 2014 12:12:02 -0000

All three of these points are key, and all too often mishandled. My particular pet peeve is #2; I've read so many job profiles that left me thinking that the person writing the requirements either has no idea what they are really looking for/talking about, or may not have a firm grip on reality. Some are so absurd that the long combination of skills and experience might fit a dozen people in the whole country.

Michele's advice for #1 and #2 are related and invaluable - be honest with both the candidates and yourself about your needs. It's too easy for good workers in this industry to change jobs if they are either misled or miscast into a role they don't feel comfortable with.

Re: The Downfall of Tesco and the Omniscience of Analytics

cleveyoung — Wed, 29 Oct 2014 14:22:57 -0000

Interesting development with Tesco. I worked for a well known and respected grocery chain for many years, and Tesco was often held up as one of the best companies to study in how they did things. My how times change.

You raise a very valid point about not becoming overly dependent upon data - you still need to make decisions, not have the data make them for you. And as much as it pains me to say, you need to take your data analysts insights and recommendations as just one piece of information among many. Ya, my analysis of your email campaign about a great price on milk coincided with a huge lift in milk sales the following week and an even bigger lift in new customers (great success!!), but what I as the analyst was totally unaware of was the Aldi's right down the street had a power outage and was closed for two of those days. No matter how much data we have access to, there are still so many other factors involved that the data cannot be relied upon as the main decision maker. Data is an important and powerful tool, but a tool does not a master craftsman make.

Re: 7 Tips For Delivering Better Analytics Recommendations

K. Weathers — Thu, 18 Sep 2014 21:11:10 -0000

Great tips. Not only are they actionalble, each is a real solution. Thanks!

Re: How to Deliver Better Recommendations: Forecast the Impact!

Parmjeet Kaur — Tue, 15 Jul 2014 03:53:06 -0000

Completely agree with you Michele in making your recommendations impactful by adding numbers or dollar/pound value to impact rather than simply saying "this change will benefit business".

Also the point to note is that the value being suggested is only a recommended benefit value under certain assumptions. If possible, account for seasonality effect as well while calculating the value benefit.

Once the recommendation is acted upon, then it is worth computing the actual benefit and analyze variance in recommended and actual benefit. With this practice, over time your recommendations get closer to actual and your recommendations are more trusted and acted upon.

Re: The Reinvention of Your Analytics Skills!

michelejkiss — Tue, 01 Apr 2014 11:02:04 -0000

100% agree Blair! Great add.

Re: The Reinvention of Your Analytics Skills!

ReevesJB — Mon, 31 Mar 2014 20:14:03 -0000

This is a really awesome list. One thing that has been useful for me is to make sure that I continue to read on a broad range of topics, in addition to the blog/forums/etc that I try (mostly in vain) to keep up with. I'm particularly into non-fiction history, social science or psychology, and I often find that I come across stuff in books I read that strikes me as really relevant to things I'm encountering in my job. It probably more directly affects the perspective I bring to those challenges rather than my hard skills, but both are really important.

Re: Better ways to measure content engagement than time metrics

Anupam Bonanthaya — Tue, 21 Jan 2014 07:52:51 -0000

is there a way to grade content before it goes online ? i mean no google analytics (or similar tracking tools)

Re: Three Things You Need To Move From Reporting To Analysis

michelejkiss — Wed, 25 Sep 2013 13:24:00 -0000

@Jim - Absolutely - great point. Analysis is so fluid, there is rarely just ONE way that you can try to answer a question. Taking two different approaches to answer the same question can be a great way to vet your results - if they're inconsistent you may have real a problem!

Re: Three Things You Need To Move From Reporting To Analysis

Jim — Wed, 25 Sep 2013 11:02:41 -0000

Good article. I would only add that I good analyst also takes the time to fully vet any discovery. It is too often the case that a conclusion is drawn based on a discovery that has not been fully vetted. I often take the time to try to go at the data from a different direction to see if I get the same result. Or I ask a colleague to review my discovery but not the means I took to make the discovery. Then I see if they can query the data and reach the same result.

Re: Better ways to measure content engagement than time metrics

Hadas Sheinfeld — Sun, 22 Sep 2013 02:26:16 -0000

Thanks Michele, I completely agree with your analysis. Moreover, as you mentioned the actual measurement of 'time on site' can be deceiving - the user many not even be in front of their computer!
ClickTale has a unique measurement of 'Engagement Time' - which measures the time visitors were actually active on the page (not only had the browser open).
Activity can be scroll/mouse-move/click, etc. We also add a buffer for 'reading time' - and together we find this metric to be a much more accurate way of measuring engagement.
Obviously this still leaves the question of whether the customer happy/unhappy with the website, which as you described can be specific per website. We recommend to correlate Engagement Time with other engagement metrics (such as various types of conversions) to establish if the goal should be to increase or decrease it.
Once you do that, customers tell us 'Engagement Time' becomes a super-strong metric.

Re: Better ways to measure content engagement than time metrics

michelejkiss — Thu, 19 Sep 2013 20:17:00 -0000

Hi Katy Norris. To do this you ultimately need to send SOME other call, so your analytics tool does not count it as a bounce. You could consider doing that based on scrolling, by setting an event when a user scrolls down. It's not going to tell you 100% that they actually read the content (e.g. I could trigger it by scrolling down to your footer) but it would be better than not knowing anything. You're of course starting to ask a lot of IT, but if it's a business challenge this might be a good solution to explore.

Justin Cutroni has written some cool blog posts about how to do this that might be helpful:

http://cutroni.com/blog/201...
http://cutroni.com/blog/201...

Hope that helps!

Re: Better ways to measure content engagement than time metrics

Katy Norris — Thu, 19 Sep 2013 18:18:02 -0000

Somewhat related, we are looking for a way to get a true bounce rate, by excluding single-beacon visits where the user slowly scrolled through a story, but did not trigger any other event besides that initial page view.

We want to recognize that "They came, they read, they left" isn't a bounce, compared to Avinash's "They came, they puked, they left."

Anyone have any good solutions for that?

Re: Better ways to measure content engagement than time metrics

michelejkiss — Tue, 17 Sep 2013 14:58:48 -0000

@laseaman Definitely agree that in a vacuum that one metric isn't going to tell you "good" or "bad" (the example you give is a great example, and I've done that myself on content sites!) but it's certainly stronger than including navigational pages. Ideally you would be doing multiple or all of the above for a more complete picture, and better yet, supplementing with qualitative measures too.

Re: Better ways to measure content engagement than time metrics

laseaman — Tue, 17 Sep 2013 13:04:56 -0000

Thanks for sharing the tracking "content page views per visit" idea. However, seems that it might still present the same issue: are more content page views per visit a goodness indicator ("yes, I'm finding lots of great content") or a badness indicator ("geez, I can't find the info I'm looking for") Thoughts?

Re: Better ways to measure content engagement than time metrics

Jake Winter — Mon, 16 Sep 2013 10:51:55 -0000

The approach that has worked for our non-monetized, reputation-building content is Paid/Owned/Earned segmentation. This has allowed us to segment what we do to drive traffic (paid/owned) and focus on earned sources and the activities that enable that (sharing, PR, content quality, inbound, etc.). We also track actions individually, but this has helped us share the story that they all feed into the bigger success of building reputation, and that paid/owned visits are not success on their own, but ideally play a support role in enabling the earned ones.

Re: Better ways to measure content engagement than time metrics

michelejkiss — Mon, 16 Sep 2013 10:09:42 -0000

@Sergio Maldonado - My approach is really to look at the business itself before deciding what the right KPIs or primary metrics are. Each business is going to be different so it's about driving what that business cares about. For example, an ad monetised content site is going to have different success measures to a non-monetised site. (Even down to monetising more heavily via video advertising vs. banner advertising!) The above list is a lot of different things you could measure, but deciding which are the most important should really be on a case by case basis, taking in to account the business goals. I don't believe there's one right way that covers all businesses.

Thanks for your comment - see you next week!

Re: Better ways to measure content engagement than time metrics

Sergio Maldonado — Mon, 16 Sep 2013 09:50:01 -0000

Many thanks for a great summary, Michelle. Fully agreed, and it clearly depends a lot on how you monetize your content in this new world of native advertising, inbound marketing and media content hoping to make a living on its own merits (ads, subscription, etc).

So, going for something much more specific, Would you say your primary metrics for this post could be a combination of "number of shares" and "participation" through comments? (only if you can share that, of course)

See you at Accelerate! :)

Re: Data Privacy: It’s not an all or nothing!

michelejkiss — Fri, 06 Sep 2013 14:55:24 -0000

@Gene Jones - You're absolutely right, I don't think companies have done at all well creating "simple yet powerful" privacy controls, and Facebook is a great example of, quite frankly, butchering it.

Thanks for the heads up and info re: 23andMe - DNA is definitely *such* personal data (and I would definitely agree they fall in to the "quantified self" camp.) There are just so many levels of this, because not only is it deciding "data from this service can be shared, data from this service can not be", it's also permissions at the information level, by person. (E.g. in your example - you're happy to share DNA data with your wife but not with insurance.)

Admittedly the tough part is - how do we provide for control at that level, without it being so much that users go, "Ah whatever" and just consent (like they do with every T&Cs they agree to!) It's not an easy problem to solve, but it's definitely a critical one.

Re: Data Privacy: It’s not an all or nothing!

Gene Jones — Fri, 06 Sep 2013 12:37:28 -0000

Michele, thanks for the post.

My biggest takeaway from this post, because it is, I think, the biggest issue, is the construction of "specific controls that address the nuance of privacy permissions [...] in a user-friendly way!"

Thus far, sites which allow for customization haven't done a good job of it. Facebook famously allows for bewilderingly complex controls. Facebook also routinely screws with these options, to the point where the threatened to take action.

If there is to be one rule in consumer directed data privacy, it has to be "listen to the consumer." Facebook fails, consistently, and miserably, at this. They have overwritten explicit user direction on how sharing was to proceed. They have proceeded under opt-out, not opt-in, wherever possible. They, like most companies, have absurdly long terms of conditions.

I think the countervailing argument of a good implementation, is 23andMe, the DNA service. I am a customer, and I think you'd surely agree that they qualify as a self-quantification service. They share many of the problems you mentioned above. I might wish to share my DNA with my relatives (because such sharing is a mutually beneficial action), my spuse (so we can determine what diseases we are carriers for, and the likely impact on any children), but not my health insurance (so they can't deny me based upon pre-existing or likely conditions).

23andMe did a fantastic job. Everything was opt-in. Each opt-in item had a separate page, with simple bullets and highlighted text explaining what I was consenting to. Reassurances were made about legal protection (23andme is federally barred from sharing data with my insurance, for instance).

In other words, 23andme managed my expectations, and thoroughly convinced me they were just as concerned about privacy as I was. Sure, their signup process is much longer (notably, this barrage of consent forms occurs after the purchase funnel, but must be completed prior to using the service. Most people don't forfeit $99 just because the sign-up process takes a few minutes).

Incidentally, the all or nothing approach is one of the biggest problems I have with DNT, although I think it's the industry's fault for not previously adhering to P3P, which did allow for nuanced behavior. Don't mistake that for endorsement of the AdChoices campaign, which is terrible.

Re: Handy Google Analytics Advanced Segments for any website

Chris Bell — Fri, 19 Jul 2013 20:11:05 -0000

Looks like Analytics have just updated their advanced segments; we can finally see smartphones and tablets separately:

http://www.chrisbell.eu/sep...

Re: Self-Quantification: Implications for marketing, personalisation and the privacy debate

Gene Jones — Wed, 17 Jul 2013 13:01:07 -0000

The best example I've seen of informing users of analytics behavior is from Firefox's Crash Reporter:
https://support.mozilla.org...

It clearly explains why they want the data, allows all details to be viewed, and users can control what type of data is sent. Opt-out is also readily possible.

Another great example is 23andme, which shares very personal data: my DNA. They are very careful to explain risks, legal protections, and data throughout the signup process. Medical research is opt-in only. Previous decisions may be reversed at any time. I feel comfortable with 23andme having a million base pairs of my DNA because of their approach. Because of my shared data, I know what genetic diseases I might be at risk for.

I wonder if there is some data too personal to get collected by a central entity. The most interesting aspect of self-quantification to me is the possibility of personal analytics. In personal analytics, data is owned solely by an individual, never given to others, and analytics engines make life improvement recommendations. With health data, communication histories, financial information, and more available as a dataset, I imagine the benefits are endless - as are the abuses.

There should be technological and legal frameworks built around. All to often, we find that the most innocuous of data is actually extremely identifiable. Merely limiting collection isn't enough. We need to automatically detect ethically void queries made by unscrupulous analysts. We need an industry standard of encryption.

Re: The Impact of Self-Quantification: What The Growing Use of Health and Fitness Devices Means for Marketing, Analytics and the Privacy Debate

michelejkiss — Tue, 16 Jul 2013 11:51:52 -0000

ReevesJB - Agree! Honestly the technology is going to accelerate so quickly. I'm really curious to see how it all plays out, what it branches in to, how deep it can go etc.

Simon Arden - Thanks for the comment! The privacy part is definitely the trickier side. It's going to be a real challenge to work through those kinds of uses. For example, is it actually possible to use them to get discounts, but NOT use to discriminate? How do we manage that? Do we get to have our cake and eat it too?

The privacy portion of the post will be up tomorrow so check back! Would love to continue to hear your thoughts and discuss. It's such an interesting topic IMHO and I think there is a lot more to say and discuss.