Featured Stories Issue #48

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

4873

Abstract

f Ads’.</h2>A developer at Jyllands-Posten pointed me to a setting in the performance measuring tool, <a href="http://www.webpagetest.org/">WebPageTest</a>.(WebPageTest is what you use, when you do performance tests. SpeedCurve is actually based on WebPageTest — and the most important things in SpeedCurve are the automated tests and a much better design/UI, at least some of the parts — I’ll get back to that.)What you have to do, before you do a WebPageTest test, is to ask WebPageTest to remove the letters ‘PTST’ from the user agent string (which every browser uses to identify itself):<figure id="f815"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*XitVzBWd_DGzCIjccWOjHw.jpeg"><figcaption></figcaption></figure>(I’ve written <a href="http://ebudvikling.dk/blog/2016/02/15/et-flueben-i-webpagetest-kan-betyde-meget-for-din-performance-maaling/">a blog post in Danish</a> about this nifty little feature.)‘PTST’ is the culprit in all of this. When our ad technology provider AdTech (<a href="http://oneadserver.aol.com/">now a part of AOL</a>) sees a browser with these four magic letters in the user agent, it withholds the ads from rendering. The reason: To avoid wasting ad displays on tests. Which makes sense, when you think about it.Run a test on WebPageTest with this checkbox checked and you get everything. And that’s what we want. I’ve seen tests where the ‘fully loaded’ time (the browser is saying “I’m totally done with loading this site now”) multiplied by 5; that’s a 400% increase! In the same test the total number of requests was multiplied by 3 (200% increase).Oh, and our SpeedIndex value (an expression of how fast the first screen view/viewport loads) increased by 30% in a test.But while WebPageTest can give us the correct data, it can’t automate it. We could do something via <a href="https://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis">the WebPageTest API</a>, but this is something we want to avoid, so as to not have too many products and service to monitor and maintain.We then went back into SpeedCurve, but there was no feature to allow this. But… in the ‘Enterprise’ edition of SpeedCurve you are allowed to use <a href="https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting">the WebPageTest scripting language</a>. One of the things you can do here is set the user agent, which is exactly what we wanted to do.Documents were written, meetings were held, decisions were made. And we (across JP/Politikens Hus, that is Ekstra Bladet, Politiken and Jyllands-Posten) signed up for SpeedCurve Enterprise. O, how we thought we had it made.We now saw SpeedCurve rendering the entire frontpage. Just like we wanted. And we started lacking in the comparisons in SpeedCurve, just as we had expected. Especially compared to the Danish Broadcasting Corporation (which has no ads, since they are funded through Public Service).And the good times kept on coming. SpeedCurve announced that they would now support the same browsers as you can choose between in the developer tools in Google’s Chrome browser. A developer at Politiken tested this and yes, it meant we no longer had to script our user agent. This was a huge plus.Just look at what happened once SpeedCurve updated the browsers and started including ads:<figure id="3451"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*qDl1ZCRvq4Df8XJ_-4E_1w.png"><figcaption></figcaption></figure><figure id="f16c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*S0yQfx9Xvxm3BpTAXBSUTA.png"><figcaption></figcaption></figure><figure id="ca74"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*coOQm61VDAbkCz7rD7TFKA.png"><figcaption></figcaption></figure><figure id="8af5"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*bjoBX9-KatudVZE_IKPv_w.png"><figcaption></figcaption></figure>As you can see, ads have a…certain influence on our front page.These two screenshots from SpeedCurve shows how big a percentage third party stuff (here; ads) take up of the front page:<figure id="db75"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AoHX6PgiVGzvOWyyWKtOPg.png"><figcaption></figcaption></figure><figure id="2d69"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*rAHw9tS-CTQNzbV_B-zfIg.png"><figcaption></figcaption></figure>Notice those percentage numbers…When something takes up almost 80 p

Options

ercent of a websites requests and sites shouldn’t it also receive about 80 percent of the attention?<h2 id="4c1a">PSTS back in, ads back out</h2>Alas, it wasn’t to last. SpeedCurve changed the browsers and reintroduced ‘PTST’ into the user agent string. Therefore; no ads. We noticed this and went back to scripting the user agent. But that didn’t work either. Though it had earlier.I got in touch with the SpeedCurve folks. They told me they had fixed a ‘bug’ and that a test browser should always label itself as such, as Mark from SpeedCurve told me in an email:WPT should always be identifying itself, even if the UA string has been set via scripting.Instead he created <a href="https://github.com/WPO-Foundation/webpagetest/issues/606">an issue</a> with WebPageTest to allow the user to set the user agent (without ‘PTST’) in the scripting language. Nothing has happened since April 25th. Steve Souders (who is the closest you’ll come to a ‘Mr. Performance’) who also works at SpeedCurve has created <a href="https://github.com/SpeedCurve-Metrics/SpeedCurve/issues/62">an issue</a> with SpeedCurve itself to allow us to remove PTST via a checkbox, like in WebPageTest. This issue was created on March 1st.We still had one shot left though: Whitelist a browser with a ‘PTST’ user agent with our ad technology provider to to allow the SpeedCurve test browsers to see the entire page rendered. Unfortunately, this is not possible since it is a “global setting across all client networks”. That means, it would have to be changed across all of the sites that use their technology. According to <a href="http://oneadserver.aol.com/">their own website</a> they have 74 countries with active clients.I then asked if we could allow the browser through if we scripted the user agent to include the word “SpeedCurve”. In effect, their block functionality would allow a browser through if both the words ‘PTST’ and ‘SpeedCurve’ are in the user agent string. But no dice:As long as PTST is in the UA we will block it.<h2 id="c77f">Alternatives?</h2>This is, obviously, a precarious situation for us to be in. We can’t measure the performance of our entire site automatically.The logic step is to look at alternatives. So far I’ve only tried one: <a href="https://calibreapp.com/">Calibre</a> (which was suggested to me by the same colleague who suggested SpeedCurve). I even wrote to the guy behind Calibre up front to be sure that it would include ads. But the same result: A fast, lean website. Which just isn’t the truth ;-)Until SpeedCurve (or WebPageTest) comes up with a change we might look at the initial no-no: Running automated WebPageTest tests through their API. As Jyllands-Posten’s developer suggested, we might be able to get it up and running pretty fast <a href="http://calendar.perfplanet.com/2014/webpagetest-private-instances-in-five-minutes/">using Amazon</a>.So… here we are. Thinking about what to do. Since we can’t automatically measure our entire page render, we can’t really do any performance budgets. We can’t measure any tweaks or changes, either. We could do it via manually tests but that is the last way out.(Also note: Performance budgets are really hard to do, once you’ve got ads in the mix. The load and performance of them vary a lot; week to week, day to day, hour to hour, even banner to banner. Also, the biggest influence on your performance is outside of your control. So ask yourself if a performance budget is the way to go.)If you made it all the way through this article and have either a trick (or a fully fledged automated performance test tool which include ads…) up your sleeve, please leave a comment.Banner ads (and for us; the way they are found, delivered and rendered) are a huge performance culprit but we can’t automate the measurements of that fact. We are stuck with manual tests in WebPageTest — or browser developer tools like those in Google Chrome.(I you found this post by Googling your own frustrations, know this: You are not alone.)<h2 id="4029">Update on June 14th, 2016:</h2>Apparently this isn’t a problem will all ad tech providers: <figure id="e652"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="" allowfullscreen="" frameborder="0" height="undefined" width="undefined"> </div> </div> </figure></iframe></div></div></figure></article></body>

Daily Digest from ILLUMINATION

Featured Stories Issue #48

Selected stories from 9,400 writers supporting 93,500+ readers on ILLUMINATION

Collection of Featured Stories

Easy access to daily selected stories from engaging writers of ILLUMINATION

Index of Published Book Chapters

Easy access to chapters for readers on Medium

Invitation To A Writing Challenge

Share your concerns and aspirations in a story

100 Day Challenge For New Writers

How to grow and become a top writer on Medium in three months

ILLUMINATION

Introducing New Writers

Introducing New Writers

We are onboarding new contributors joining ILLUMINATION, ILLUMINATION-Curated, ILLUMINATION’S MIRROR, ILLUMINATION Book…

Dr Mehmet Yildiz

Introducing an engaging book by prolific author Maria Rattray

In The Name Of My Father

Introducing an engaging book by prolific author Maria Rattray

Thewriteyard

Your Ultimate Guide to No Touch Society

Your Ultimate Guide to No Touch Society

Answers to all those awkward questions

Larry G. Maguire

Machiavelli on How To Play The Power Game

Machiavelli on How To Play The Power Game

In 500 years of human development and technological advancement, has anything really changed in how we do business?

Megan Holstein

How to Look Beautiful Without Wearing Makeup

How to Look Beautiful Without Wearing Makeup

If you put on makeup because you think you look terrible without it, you’re wrong

Elvie

About Me

About Me

Born Filipina and a resident of Singapore. I grew up hopping from one home to the other when my mother died. I was 6…

Fareeha Arshad

How White Are White Lies

How White Are White Lies

One bitter truth behind white lies.

Anna

What I Learned After I Almost Died in Hawaii and How It Changed Me

What I Learned After I Almost Died in Hawaii and How It Changed Me

This scary wake-up call taught me important life lessons.

Veronika Kaufmann

A Girl from Vienna

A Girl from Vienna

an anecdote from my mom’s life who died 10 years ago this month

Michael Hollifield

I Can’t See the Yard for the Weeds

I Can’t See the Yard for the Weeds

When some people can’t see the forest for the trees.

Darryl Brooks

Kids Today Will Never Comprehend the Hardships We Suffered

Kids Today Will Never Comprehend the Hardships We Suffered

It’s a Completely Different World Today

Neha Sandhir S

Inner Light

Feel it radiating from inside out

Abdullah Nadeem

How To Balance Enthusiasm With Discipline

How To Balance Enthusiasm With Discipline

Avoid burning out whilst maintaining high levels of work

Bill Abbate

How Long Will Your Words Last and Will They Matter to Anyone Else?

How Long Will Your Words Last and Will They Matter to Anyone Else?

Why writing with future generations in mind is important

Asmita Karanje

Winners Don’t Quit Is A Big Lie — They Do.

Winners Don’t Quit Is A Big Lie — They Do.

They just know the right time to quit

Ben Ho

Rain to Renew Our Soul

Rain to Renew Our Soul

Morning relaxation thoughts

Anthony Jackson

It Only Takes You 10 Minutes

It Only Takes You 10 Minutes

A poem about time.

Laurie Perez

Alchemy of the Emperor