Featured Stories Issue #53

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

4873

Abstract

f Ads’.</h2>A developer at Jyllands-Posten pointed me to a setting in the performance measuring tool, <a href="http://www.webpagetest.org/">WebPageTest</a>.(WebPageTest is what you use, when you do performance tests. SpeedCurve is actually based on WebPageTest — and the most important things in SpeedCurve are the automated tests and a much better design/UI, at least some of the parts — I’ll get back to that.)What you have to do, before you do a WebPageTest test, is to ask WebPageTest to remove the letters ‘PTST’ from the user agent string (which every browser uses to identify itself):<figure id="f815"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*XitVzBWd_DGzCIjccWOjHw.jpeg"><figcaption></figcaption></figure>(I’ve written <a href="http://ebudvikling.dk/blog/2016/02/15/et-flueben-i-webpagetest-kan-betyde-meget-for-din-performance-maaling/">a blog post in Danish</a> about this nifty little feature.)‘PTST’ is the culprit in all of this. When our ad technology provider AdTech (<a href="http://oneadserver.aol.com/">now a part of AOL</a>) sees a browser with these four magic letters in the user agent, it withholds the ads from rendering. The reason: To avoid wasting ad displays on tests. Which makes sense, when you think about it.Run a test on WebPageTest with this checkbox checked and you get everything. And that’s what we want. I’ve seen tests where the ‘fully loaded’ time (the browser is saying “I’m totally done with loading this site now”) multiplied by 5; that’s a 400% increase! In the same test the total number of requests was multiplied by 3 (200% increase).Oh, and our SpeedIndex value (an expression of how fast the first screen view/viewport loads) increased by 30% in a test.But while WebPageTest can give us the correct data, it can’t automate it. We could do something via <a href="https://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis">the WebPageTest API</a>, but this is something we want to avoid, so as to not have too many products and service to monitor and maintain.We then went back into SpeedCurve, but there was no feature to allow this. But… in the ‘Enterprise’ edition of SpeedCurve you are allowed to use <a href="https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting">the WebPageTest scripting language</a>. One of the things you can do here is set the user agent, which is exactly what we wanted to do.Documents were written, meetings were held, decisions were made. And we (across JP/Politikens Hus, that is Ekstra Bladet, Politiken and Jyllands-Posten) signed up for SpeedCurve Enterprise. O, how we thought we had it made.We now saw SpeedCurve rendering the entire frontpage. Just like we wanted. And we started lacking in the comparisons in SpeedCurve, just as we had expected. Especially compared to the Danish Broadcasting Corporation (which has no ads, since they are funded through Public Service).And the good times kept on coming. SpeedCurve announced that they would now support the same browsers as you can choose between in the developer tools in Google’s Chrome browser. A developer at Politiken tested this and yes, it meant we no longer had to script our user agent. This was a huge plus.Just look at what happened once SpeedCurve updated the browsers and started including ads:<figure id="3451"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*qDl1ZCRvq4Df8XJ_-4E_1w.png"><figcaption></figcaption></figure><figure id="f16c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*S0yQfx9Xvxm3BpTAXBSUTA.png"><figcaption></figcaption></figure><figure id="ca74"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*coOQm61VDAbkCz7rD7TFKA.png"><figcaption></figcaption></figure><figure id="8af5"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*bjoBX9-KatudVZE_IKPv_w.png"><figcaption></figcaption></figure>As you can see, ads have a…certain influence on our front page.These two screenshots from SpeedCurve shows how big a percentage third party stuff (here; ads) take up of the front page:<figure id="db75"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*AoHX6PgiVGzvOWyyWKtOPg.png"><figcaption></figcaption></figure><figure id="2d69"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*rAHw9tS-CTQNzbV_B-zfIg.png"><figcaption></figcaption></figure>Notice those percentage numbers…When something takes up almost 80 p

Options

ercent of a websites requests and sites shouldn’t it also receive about 80 percent of the attention?<h2 id="4c1a">PSTS back in, ads back out</h2>Alas, it wasn’t to last. SpeedCurve changed the browsers and reintroduced ‘PTST’ into the user agent string. Therefore; no ads. We noticed this and went back to scripting the user agent. But that didn’t work either. Though it had earlier.I got in touch with the SpeedCurve folks. They told me they had fixed a ‘bug’ and that a test browser should always label itself as such, as Mark from SpeedCurve told me in an email:WPT should always be identifying itself, even if the UA string has been set via scripting.Instead he created <a href="https://github.com/WPO-Foundation/webpagetest/issues/606">an issue</a> with WebPageTest to allow the user to set the user agent (without ‘PTST’) in the scripting language. Nothing has happened since April 25th. Steve Souders (who is the closest you’ll come to a ‘Mr. Performance’) who also works at SpeedCurve has created <a href="https://github.com/SpeedCurve-Metrics/SpeedCurve/issues/62">an issue</a> with SpeedCurve itself to allow us to remove PTST via a checkbox, like in WebPageTest. This issue was created on March 1st.We still had one shot left though: Whitelist a browser with a ‘PTST’ user agent with our ad technology provider to to allow the SpeedCurve test browsers to see the entire page rendered. Unfortunately, this is not possible since it is a “global setting across all client networks”. That means, it would have to be changed across all of the sites that use their technology. According to <a href="http://oneadserver.aol.com/">their own website</a> they have 74 countries with active clients.I then asked if we could allow the browser through if we scripted the user agent to include the word “SpeedCurve”. In effect, their block functionality would allow a browser through if both the words ‘PTST’ and ‘SpeedCurve’ are in the user agent string. But no dice:As long as PTST is in the UA we will block it.<h2 id="c77f">Alternatives?</h2>This is, obviously, a precarious situation for us to be in. We can’t measure the performance of our entire site automatically.The logic step is to look at alternatives. So far I’ve only tried one: <a href="https://calibreapp.com/">Calibre</a> (which was suggested to me by the same colleague who suggested SpeedCurve). I even wrote to the guy behind Calibre up front to be sure that it would include ads. But the same result: A fast, lean website. Which just isn’t the truth ;-)Until SpeedCurve (or WebPageTest) comes up with a change we might look at the initial no-no: Running automated WebPageTest tests through their API. As Jyllands-Posten’s developer suggested, we might be able to get it up and running pretty fast <a href="http://calendar.perfplanet.com/2014/webpagetest-private-instances-in-five-minutes/">using Amazon</a>.So… here we are. Thinking about what to do. Since we can’t automatically measure our entire page render, we can’t really do any performance budgets. We can’t measure any tweaks or changes, either. We could do it via manually tests but that is the last way out.(Also note: Performance budgets are really hard to do, once you’ve got ads in the mix. The load and performance of them vary a lot; week to week, day to day, hour to hour, even banner to banner. Also, the biggest influence on your performance is outside of your control. So ask yourself if a performance budget is the way to go.)If you made it all the way through this article and have either a trick (or a fully fledged automated performance test tool which include ads…) up your sleeve, please leave a comment.Banner ads (and for us; the way they are found, delivered and rendered) are a huge performance culprit but we can’t automate the measurements of that fact. We are stuck with manual tests in WebPageTest — or browser developer tools like those in Google Chrome.(I you found this post by Googling your own frustrations, know this: You are not alone.)<h2 id="4029">Update on June 14th, 2016:</h2>Apparently this isn’t a problem will all ad tech providers: <figure id="e652"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="" allowfullscreen="" frameborder="0" height="undefined" width="undefined"> </div> </div> </figure></iframe></div></div></figure></article></body>

Daily Digest from ILLUMINATION

Featured Stories Issue #53

Selected stories from 9,400 writers supporting 94,000 readers on ILLUMINATION

Collection of Featured Stories

Easy access to daily selected stories from engaging writers of ILLUMINATION

Index of Published Book Chapters

Easy access to chapters for readers on Medium

Invitation To A Writing Challenge

Share your concerns and aspirations in a story

100 Day Challenge For New Writers

How to grow and become a top writer on Medium in three months

Keri Mangis

Get in the Arena with Us, or Get out of Our Way

Get in the Arena with Us, or Get out of Our Way

Writing in times like these is not about perfection. It’s about bringing our whole humanity to the table.

Karen Madej

ILLUMINATION Book Chapters First Chapter Writing Tips

ILLUMINATION Book Chapters First Chapter Writing Tips

Please help yourself to first attracting readers and then making it easy for them to read the next chapter.

The Maverick Files

To All Illumination Detractors — A Respectful Yet Befitting Response

To All Illumination Detractors — A Respectful Yet Befitting Response

Mud-slinging will most often stain your own clothes too.

Dr Mehmet Yildiz

How To Create A 30,000 Words E-book In A Month

How To Create A 30,000 Words E-book In A Month

Practical steps to produce a manuscript with 1,000-word a day

David Gerken

4 Life Lessons We Can All Learn By Playing Sports

4 Life Lessons We Can All Learn By Playing Sports

And you don’t have to be a good athlete, either.

Alana Rister, Ph.D.

If You Write for Money, You Are A Real Writer

If You Write for Money, You Are A Real Writer

Those who suggest you are not should check their privilege

Alexander Yu

As You Seek, Don’t Forget To See

As You Seek, Don’t Forget To See

How a late basketball great, K-pop star, and kid’s movie offer insights into the meaning of life

Ian Vanderlinden

How to Instantly Improve Your Singing

How to Instantly Improve Your Singing

No, this is not clickbait.

Matthew Prince

Stop the Bullshitting, Too Many Freedom Cages You

Stop the Bullshitting, Too Many Freedom Cages You

Find the balance between freedom and discipline

Marcel Badia

The Necessity of Digital Disconnect to Increase Productivity

The Necessity of Digital Disconnect to Increase Productivity

An increasingly necessary digital detox that has employers on edge

Paola Perez

How To Successfully Stick to a Healthy Diet

How To Successfully Stick to a Healthy Diet

Despite craving unhealthy foods all day long.

Floyd Mori

Japanese Americans Lost Their Homes But Were Not Exactly Homeless

Japanese Americans Lost Their Homes But Were Not Exactly Homeless

It happened during World War II

pst tomowo

What Faith is: Faith is a lifestyle…

What Faith is: Faith is a lifestyle…

Faith is a Lifestyle… a choice lifestyle.. Faith is not a means to an end.

Shawn Ingram

Retest

Retest

A plague doc performs a retest

Jonathan Thomas

I need to make writing my income source

I need to make writing my income source

I’m tired of my wage slavery — AfroPolymath

Priscilla Yeung (Former Midori by the Sea!)

Will Immigrants Go Home First Thing If Something Happens to Their Homeland?

Will Immigrants Go Home First Thing If Something Happens to Their Homeland?

Isolationist’s strange mindset against globalisation

Shawn Ingram

Monolith 2

Monolith 2

FLASH Fiction Challenge 100 — Day 24

Rose Blog