Free AI web copilot to create summaries, insights and extended knowledge, download it at here

Abstract

/a> as the most robust model (<a href="https://arxiv.org/abs/1706.06083">Madry et al.</a>). This fact highlights just how far away we are from robust recognition models – even for simple handwritten digits.In our <a href="https://arxiv.org/abs/1805.09190">recent paper</a>, we introduce a new concept to classify images robustly. The idea is very simple: if an image is classified as a seven, than it should contain roughly two lines – one shorter, one longer – that touch each other at one end. That’s a generative way to think about digits, which is pretty natural for humans and which allows us to easily spot the signal (the lines) even amidst large amounts of noise and perturbations. Having such a model should make it easy to classify the adversarial examples featured above into the correct class. Learning a generative model of digits (say zeros) is pretty straightforward (using a <a href="https://arxiv.org/abs/1606.05908">Variational Autoencoder</a>) and, in a nutshell, works as follows: we start from a latent space of nuisance variables (which might capture things like thickness or tilt of the digit and are learnt from the data) and generate an image using a neural network. We then show examples of handwritten zeros and train the network to produce similar ones. At the end of training, the network has learnt about the natural variations of handwritten zeros:<figure id="9127"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*Y6O2le5_-9PLg_n4iWN_6w.png"><figcaption>A generative model of zeros learns the typical variations of handwritten digits (right side).</figcaption></figure>We learn such a generative model for each digit. Then, when a new input comes along, we check which digit model can best approximate the new input. This procedure is typically called analysis-by-synthesis, because we analyse the content of the image according to the model that can best synthesise it. Standard feedforward networks, on the other hand, have no feedback mechanisms to check whether the input image really resembles the inferred class:<figure id="e38b"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*qfe00YnTC58Up5hOmVuC8g.png"><figcaption>Feedforward networks directly go from image to class and have no way to check that the classification makes sense. Our analysis-by-synthesis model checks what image features are present and classifies according to which class makes most sense.</figcaption></figure>That’s really the key difference: feedforward networks have no way to check their predictions, you have to trust them. Our analysis-by-synthesis model, on the other hand, looks whether certain image features are really present in the input before jumping to a conclusion.We do not need a pe

Options

rfect generative model for this procedure to work. Our model of handwritten digits is certainly not perfect: look at the blurry edges. Nonetheless, our model can classify hand-written digits with high accuracy (99,0%) and its decisions make a lot of sense to humans. For example, the model will always signal low confidence on noise images, because they don’t look like any of the digits it has seen before. The images closest to noise that the analysis-by-synthesis model still classifies as digits with high confidence make a lot of sense to humans:<figure id="5507"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*fjhRFQkEFDMWuwwFv2tEaQ.png"><figcaption>We tried to synthesise unrecognisable images that are still classified as zeros with high confidence by our analysis-by-synthesis model. This is the best we got.</figcaption></figure>In the current state-of-the-art model by Madry et al. we found that minimal perturbations of clean digits are often sufficient to derail the classification of the model. Doing the same for our analysis-by-synthesis model yields strikingly different results:<figure id="f6b0"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*aedBhqczyEb_pd4y9ubzEg.png"><figcaption>Adversarial examples for the analysis-by-synthesis model. Can you guess what the original number was?</figcaption></figure>Note that the perturbations make a lot of sense to humans and it is sometimes difficult to decide into which class the image should be classified. That’s exactly what we expect to happen for a robust classification model.Our model has several other notable features. For example, the decisions of the analysis-by-synthesis model are much easier to interpret as one can directly see which features sway the model towards a particular decision. In addition, we can even derive some lower bounds of its robustness.The analysis-by-synthesis model does not quite match human perception yet and there is still a long way to go (see the full analysis in our <a href="https://arxiv.org/abs/1805.09190">manuscript</a>). Nonetheless, we believe these results are extremely encouraging and we hope that our work will pave the way towards a new class of classification models that are accurate, robust and interpretable. We still have to learn a lot about these new models, least of all how to make inference more efficient and how to scale them to more complex data sets (like CIFAR or ImageNet). We are working hard to answer these questions and are looking forward to sharing more results with you in the future.<h2 id="7aaa">Towards the first adversarially robust neural network model on MNIST</h2>Lukas Schott, Jonas Rauber, Matthias Bethge, Wieland Brendel arXiv:1805.09190</article></body>

Beginner’s Guide to Heavy Metal, Part 6: The Origins of Symphonic Metal

This is especially for readers who love classical music, are metal-curious — but don’t like the screaming!

Anthony Overs has mentioned that symphonic metal can be seen as a “gateway drug” for people who might otherwise be intimidated by metal or its associated stereotypes. Read on — or better yet — play these links to see why he’s right.

Genre definitions can be useful for telling someone roughly what to expect from a new artist or piece of music. We’ve been using genres to organize the world of metal into digestible bites. But genres can also become shackles. Too many musicians and bands find themselves trapped by genre. This is often imposed on them by their fans, some of whom don’t want their favorite artists straying from what hooked them in the first place.

That is unfortunate because the most creative things happen when people stray across preconceived boundaries. This can be true whether they are artists, athletes, or scientists. They just have to have the courage to do it. Some of the most beautiful songs happen where supposedly different genres collide. And then there are situations where genres don’t exist, so the artists must invent them.

If there is a “mainstream” category of metal for North American and British fans, it is the so-called New Wave of British Heavy Metal (UK-based Iron Maiden and Judas Priest being the two biggest and longest lived-they are still touring after more than 50 years); and the West Coast American thrash bands (initially the big four of Metallica, Anthrax, Slayer and Megadeth). Anthony Overs cited most of these bands in Part 1 of this series:

A Beginner’s Guide to Heavy Metal

Part I — origins, classics and a peak down some branches

medium.com

But even as heavy metal was coalescing out of what came before, it began splintering into various styles.

In Part V, Anthony covered the origins of Stoner Rock, one of these splinter factions. Also sometimes known as Desert Rock, it was primarily an American thing.

In continental Europe, metal went off into other directions, one of which was heavily influenced by classical music. As far back as the late 1960s, bands like Cream and Deep Purple were borrowing elements from classical music. Some of the emerging European bands returned to those roots and doubled down on them. This gave us the scene we now refer to as “symphonic metal” or “symphonic rock” (you can consider them pretty much the same thing).

Symphonic rock gets little exposure in the U.S. but is mainstream in Europe, Latin America, and Japan. Deep Purple’s “Concerto for Group and Orchestra” was the first real foray in 1969. They performed it with the London Philharmonic and even got some positive reviews. In the 1970s, people referred to Godspell, Hair, and Jesus Christ Superstar as rock operas, though the music is decidedly pop, not rock. They pale in comparison to modern symphonic rock,

The early symphonic rock bands came out of niche sub-genres like gothic metal, doom pop, shoegaze, and the remnants of synth-based New Wave. What set most of them apart from other rock genres was powerful, often classically-trained, soprano vocals soaring above dark, heavy instrumentation.

Any band that uses symphonic elements (or simulates them using programmed keyboards) and combines them withmore typical metal elements can be considered “symphonic metal.” Some bands do not feature female vocalists, but by far, most of the major bands considered symphonic metal do, in fact, feature front-women.

These front-women are sometimes paired with male co-leads who do “metal” vocals (i.e., growling). The concept has become known as “beauty and the beast.” (Epica are the leading example — see below.)

Overall, symphonic metal is decidedly less dark and morbid than some other types of metal and is sometimes downright inspirational. Growling is not typical of the genre. This is one reason it can be seen as an easy on-ramp for the metal-curious.

The female lead vocalists became the faces and avatars of these bands. This was, in many ways, the first real breakthrough for women in what had otherwise been an extremely macho, sexist corner of the music world. The women were still largely restricted to vocal roles, but that, too, would eventually start to change.

Meanwhile, partly in rebellion against the self-conscious minimalism of grunge and indy-rock, some songwriters began emphasizing classical approaches to composition, with complex arrangements and dense harmonies. One can hear hints, or more, of Bach, Beethoven, Rachmaninoff, and Orff in the work of groups such as Therion and Haggard. Coupling this sort of composition with the “beauty and the beast” concept would give rise to true symphonic metal bands like Nightwish, Within Temptation, and Epica.

The first two clips here are from groups that anticipated symphonic metal. I find the songs gorgeous, but what are they? Gothic? Drone? Shoegaze? Prog rock? They touch all of those styles, but the truth is, we don’t need to label them. They stand on their own.

Theatre of Tragedy (Norway): Final concert (2010; Vocals — Ragnhild “Nell” Sigland)

Theatre of Tragedy were active from 1993 through 2010. Sigland joined in 2004, replacing original vocalist Liv Kristine. The band went through various phases, periodically incorporating death-metal growls that complemented the female vocals and changing the instrumental mix.

In 2010, they announced they were disbanding. They did a farewell tour and were going to memorialize it in a live concert DVD, but their label canceled funding. Their fans crowdfunded it instead and also captured the video footage you will see here.

This is the entire concert. You can click anywhere and find something beautiful happening. Better yet, if you have time, watch the whole thing. If you want to pick and choose, the first comment contains a set list with time markers. My favorites are “And When He Falleth” and “Fade”.

The Gathering (Netherlands): “Travel” (Live, 2014; from their 1998 album “How to Measure a Planet”)

Founded in 1989, The Gathering traverse many genres, variously characterized as gothic, death-metal, doom-pop, progressive rock, and shoegaze. They had several female lead vocalists but hit their peak with Anneke van Giersbergen from 1994 through 2007.

Van Giersbergen is regarded by many as the godmother of symphonic metal. Other top vocalists, including Sharon den Adel, Floor Jansen, Simone Simons, and Asami, cite her as an influence. She left The Gathering to seek more life balance, though she continues to collaborate and make joint appearances with many of those artists.

This is from the band’s 25th anniversary concert, in which their four most important lead vocalists made appearances (including Bart Smits, their original and only male lead singer). Toward the song's end, they gather on stage for some mesmerizing harmonies.

The audio of the concert was released as a live album. In a weird echo of the farewell concert by Theater of Tragedy, the video was captured through an organized collaboration by a dozen or so fans. There is a link to the full concert in the notes to this video.

THERION (Sweden): “Birth of Venus Illegitima” (1998, live, Kraków; Vocals — Martina Astner)

Therion was founded in 1987 as a death-metal band and evolved into something that became known as symphonic metal. Many consider them the first true example of the category.

Like King Crimson, the Pretenders, Tame Impala, and Ayreon, they are essentially the brainchild and ongoing project of one person, in this case Cristofer Johnsson. Almost 100 members have rotated through the band between session and touring musicians. This does not count the full orchestras accompanying them on some of their albums and live concerts. This clip is from their 1998 tour supporting their album “Vovin.”

Epica (Netherlands): “Cry For the Moon” (2003; vocalist — Simone Simons)

Epica are a Dutch band founded in 2002. This piece, which is essentially a hymn to common sense, is heavily influenced by Händel’s Messiah, with a pointedly ironic twist: It is a criticism of religious hypocrisy, specifically of the sexual abuse scandal in the Catholic Church. The Church tried to have this song banned (it didn’t succeed).

The lead vocalist is classically trained soprano Simone Simons. When this was recorded, she was only 18, and her voice was still developing. She doesn’t yet have the operatic throw weight she will have later, but you can hear where things are headed. Several of the backing singers are themselves established metal vocalists, and one of them, Amanda Somerville (the blonde), was also one of Simone’s coaches.

Ignea (Ukraine): “Alga” (2015; vocalist — Helle Bohdanova)

To be taken seriously in American popular music, a song has to be about baring your soul and picking at your emotional scabs. Appreciating art is all about seeing yourself in it, about feeling that the artist somehow “got” you. Really? Since when is narcissism the point of art or music? If that had always been the case, Rembrandt and Beethoven wouldn’t be a thing; Shakespeare would only work if you identify with patricidal pre-medieval despots (in which case you have other issues).

Fortunately, for art’s sake (though maybe not so much for the people living there), there are parts of the world where people don’t have the luxury of being spoiled and self-involved. They have other things to worry about. Here is a band from Ukraine singing about defending their homeland from invaders (guess who). No time for navel-gazing here!

Delain (Netherlands): “Tragedy of the Commons” (2014; vocalists — Charlotte Wessels and Alissa White-Gluz)

Wessels (on our right in the thumbnail) is another Dutch diva who trained in classical and jazz singing before going metal. She doesn’t emphasize the classical “head” voice but uses more of a Broadway musical-style belt.

White-Gluz’ day job is with the Swedish band Arch Enemy, where she specializes in metal vocals (growls and screams). Here, the two of them take turns, alternating the clean soprano and metal vocals with arresting affect. White-Gluz also flexes her incredible range, singing the soprano descant harmonies.

Metal is not typically associated with social protest, but it has its roots in exactly that. This song is an overtly political anthem about humans’ inability to restrain the behavior destroying our planet. I don’t know whether to take it as a warning or a prescient funeral dirge. Either way, I find it stirring.

Nightwish (Finland): “Ghost Love Score” (live, 2013; vocalist — Floor Jansen)

Rock does not lend itself to deep compositions, so only a few rock songs reach the level of true masterpieces. I consider this one of them. Think of a cross between a symphony and a movie score. “Ghost Love Score” has been compared to “Bohemian Rhapsody” and Floor Jansen’s performance to Freddie Mercury’s. Many critics consider this the apotheosis of symphonic rock.

Like Bohemian Rhapsody, it is a true rock opera, but even professional opera singers don’t have the range of vocal qualities called for in “Ghost Love Score”. Freddie was one of a kind, and perhaps he could have pulled this off, but I can’t think of anyone else. Except, of course, for Ms. Jansen.

Side notes: The classical backing track was recorded by the London Philharmonic. The audience shots are eerily similar to the shots from Queen’s epic Live Aid performance at Wembley Stadium.

End Note: