avatarGGWithRabitLIFE

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

7237

Abstract

ing the media world in a fairly unique and trusting way for months now. Most public figures in his position have people advising them on media strategy, what to say, and what not to say — how to frame things so you are not misquoted. Which interviews to take on and which to refuse. Moreover, probably not to allow two solid days behind the scenes to someone who you think may have an agenda.</p><p id="2ef6">As far as I can tell, he doesn’t have this and has embraced and is fully living out his own dictum — “speak the truth and let the pieces fall where they may”.</p><p id="21b3">So far speaking the truth has worked out well for him — he was lucky that Channel 4 News put up the Cathy Newman interview unedited — a decision I’m sure that they quickly regretted. <b>This New York Times piece feels different </b>— that it will solidify impressions of him on either side.</p><p id="6328"><b>“Bad faith changes everything”</b></p><p id="cf22">As Eric Weinstein, Bret’s brother, and another member of the unofficial ‘intellectual dark web’ said — “bad faith changes everything”. It’s possible to have any kind of discussion with people you disagree with so long as they are approaching it in good faith — as soon as they are not, they’re just looking to boost their position, look good in front of others or advance their career within their tribe — as Peterson alleged Cathy Newman was — then true exchange of ideas is impossible.</p><p id="c6f9">I would argue that this journalist is indeed acting in bad faith. Some of the misrepresentations cannot be put down to simple misunderstanding.</p><p id="7c74">The piece of the interview that has been seized upon is this:</p><blockquote id="f129"><p>“Recently, a young man named Alek Minassian drove through Toronto trying to kill people with his van. Ten were killed, and he has been charged with first-degree murder for their deaths, and with attempted murder for 16 people who were injured. Mr. Minassian declared himself to be part of a misogynist group whose members call themselves incels. The term is short for “involuntary celibates,” though the group has evolved into a male supremacist movement made up of people — some celibate, some not — who believe that women should be treated as sexual objects with few rights. Some believe in forced “sexual redistribution,” in which a governing body would intervene in women’s lives to force them into sexual relationships.</p></blockquote><blockquote id="8111"><p>Violent attacks are what happens when men do not have partners, Mr. Peterson says, and society needs to work to make sure those men are married.</p></blockquote><blockquote id="005a"><p>“He was angry at God because women were rejecting him,” Mr. Peterson says of the Toronto killer. “The cure for that is enforced monogamy. That’s actually why monogamy emerges.”</p></blockquote><blockquote id="f00b"><p>Mr. Peterson does not pause when he says this. Enforced monogamy is, to him, simply a rational solution. Otherwise women will all only go for the most high-status men, he explains, and that couldn’t make either gender happy in the end.”</p></blockquote><p id="4fb7">The framing of it makes it sound like he’s advocating for some kind of “government/state enforced” monogamy — which is malicious. He — as anyone who has been listening to his lectures will tell you — is making a more subtle point.</p><p id="924a">That monogamy is an evolved trait to stabilise societies — it’s “enforced” only as a social rule. Many societies in the past have had polygamy or other situations where a small number of men had access to many women (as do many animal societies), and that proved to be unstable and a bad long term solution to social harmony.</p><p id="7529">So in this argument, the reason that monogamy evolves (and is socially enforced) is to avoid the kind of situation where you end up with too many bitter young men wanting to tear things down. Which — whether you agree with his reading or not — seems to be happening.</p><p id="c544">And he also argues — that one of the consequences of the sexual revolution of the 60s is that it has loosened sexual behaviour. When that happens, and there are less social and physiological downsides (pregnancy is not inevitable) — and more sex is being had — the spoils go disproportionately to the most attractive men.</p><p id="7716">He is arguing that there are deeper reasons for the morality that we evolved over our history, and that there are consequences to the kinds of lifestyle experiments that we saw start to accelerate in the 1960s. This is what the essence of Peterson’s message is about.</p><p id="fb0d">You can disagree with Peterson, but many people are recognising that this fits their experience of the world — hence his popularity. That the increasingly boundary-less world we’ve created is not working and we need a reintegration of these traditional values.</p><p id="ad84"><b>Ideological fixation</b></p><p id="89ce">Personally I agree with Peterson that much of our culture and media is in the grip of an ideology that styles itself as open, inclusive and tolerant, but actually has a shadow side of intolerance towards those that don’t share their values.</p> <figure id="3404"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FaMcjxSThD54%3Fstart%3D1511%26feature%3Doembed%26start%3D1511&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DaMcjxSThD54&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FaMcjxSThD54%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" allowfullscreen="" frameborder="0" height="480" width="854"> </div> </div> </figure></iframe></div></div></figure><p id="731c">One of the sacred cows in this is the belief that the only reason that men and women don’t have equal representation in many industries and top jobs is because of discrimination, not differing choices. This was famously the subject of the clash with Cathy Newman.</p><p id="fc16">If these activists (and the liberal left generally) has to accept that there are measurable differences in temperament, values and life choices made by men and women, and that some of them are likely tied to biology and evolutionary history — then the entire edifice of this ‘gender ideology’ movement starts to shake. We would realise that it would be irrational to expect 50/50 representation in many jobs, for example.</p><p id="8022">And realise that this is not just a social movement — this ideological framing — it’s a big industry. There are charities, organisations and groups that are funded on the basis of this that have huge impact on the media conversation, and business practice.</p><p id="e742">So left-wingers who are usually concerned about the impact of money on ideology and political decisions from corporate interests, might wish to look at the fact that these are major financial interests as well. It is not possible for this entire industry to look at the data accurately.</p><p id="b543">I would also frame it in this way that might resonate

Options

with left-wing thinkers. Why are you using capitalist metrics like pay to decide on whether women have achieved equality? Isn’t that just another version of the ‘patriarchy’ controlling your minds? How has “the man” persuaded you that equal pay is the right metric to look for equal respect and value?</p><p id="b245">What makes women actually fulfilled and gives genuine meaning in life? Is it the same as for men? I doubt it, and I personally know many women who made that realisation too late in life to easily have a balanced life that included children.</p><p id="5cc1">But anything that argues against leftist ideology is attacked and smeared. For example the infamous ‘Google memo’ was a case in point, being described as an ‘anti-diversity screed’ throughout the media despite the author, James Damore, specifically making suggestions that would increase the representation of women in tech. This article from the Atlantic — from a writer who doesn’t agree with Damore’s conclusions — gives good context to its misrepresentation: “To me, the Google memo is an outlier — I cannot remember the last time so many outlets and observers mischaracterized so many aspects of a text everyone possessed.”</p><p id="3999">Liberalism as an ideology, with those inside and outside the tribe. Those outside the tribe, like Peterson — deserve scorn, derision, and even misrepresentation.</p><p id="11cc">Another hard to explain mischaracterisation, if the journalist had any familiarity with Peterson’s work is this:</p><blockquote id="33b8"><p>“Mr. Peterson illustrates his arguments with copious references to ancient myths — bringing up stories of witches, biblical allegories and ancient traditions. I ask why these old stories should guide us today.</p></blockquote><blockquote id="f195"><p>“It makes sense that a witch lives in a swamp. Yeah,” he says. “Why?”</p></blockquote><blockquote id="69ec"><p>It’s a hard one.</p></blockquote><blockquote id="0ac8"><p>“Right. That’s right. You don’t know. It’s because those things hang together at a very deep level. Right. Yeah. And it makes sense that an old king lives in a desiccated tower.”</p></blockquote><blockquote id="fc08"><p>But witches don’t exist, and they don’t live in swamps, I say.</p></blockquote><blockquote id="e8f8"><p>“Yeah, they do. They do exist. They just don’t exist the way you think they exist. They certainly exist. You may say well dragons don’t exist. It’s, like, yes they do — the category predator and the category dragon are the same category. It absolutely exists. It’s a superordinate category. It exists absolutely more than anything else. In fact, it really exists. What exists is not obvious. You say, ‘Well, there’s no such thing as witches.’ Yeah, I know what you mean, but that isn’t what you think when you go see a movie about them. You can’t help but fall into these categories. There’s no escape from them.”</p></blockquote><p id="3287">He is deliberately framed here as an old crank, an eccentric who believes in dragons and witches. Yet throughout his lectures he has made clear he is talking psychologically, archetypally and mythologically.</p><p id="67f0">In this view they exist in our mythology in a very real way as representations of psychological realities, for example that the dragon is the mythological representation of the ‘unknown’. Throughout our history if you ventured out into the unknown you could die — but there was no other way to discover new information or new rewards.</p><p id="cc2a">So the dragon is a composite predator of all the animals that used to prey on humans — a cat/snake/bird — and of course in mythology dragons have gold (or virginal women in captivity). The deep psychological story is that by confronting the unknown, you can achieve riches. He’s made that abundantly clear in every lecture.</p><p id="2105">What is becoming ever clearer (and again is something Peterson points out) — the death spiral of the print media is speeding up polarisation — in even the most reputable organisations such as the New York Times start to produce clickbait such as the Jordan Peterson article.</p><p id="4e32">Perhaps the New York Times have decided to take a leaf out of the book of the 4chan culture, and provocateurs like Milo Yiannopolous who made whole careers out of provoking the left into overreaction. In this case — the NYT have placed this article behind a paywall (I’m told) — so to read it, outraged Peterson fans will have to subscribe to the paper.</p><p id="255f">Then to unsubscribe — I know because I just checked — you cannot unsubscribe online, you have to call them. This seems at least unethical.</p><p id="5aaf">The most dangerous part of this whole enterprise is that Peterson has now become pretty much the singular focus of the ramping up of the culture wars — the lightning rod, if you will. Articles like this add hugely to the polarisation he warns about.</p><p id="46ed">On one side you have literally tens of thousands of people (mainly, but not all men) who have had their lives changed, and many claiming actually saved, by listening to Jordan Peterson’s words. On the other side you have a mix of hard core ideological opponents to him, and a vast middle ground who don’t know him well — but are almost certainly thinking that there is no smoke without fire.</p><p id="6c75">He has argued frequently that we are in an increasingly polarised world and that individual actions can have serious consequences, if we don’t act with integrity, or we sacrifice our morality and conscience in any way.</p><p id="c074">This journalist spent two days with Peterson in his house, she seemingly has some familiarity with his work, and yet chooses to characterise him in this way.</p><p id="9c1a">The treatment of Jordan Peterson is speeding up the irrelevance of the mainstream media at an increasing rate. Too many people are aware of his work and who he is and what he believes for the hit pieces to stick.</p><p id="3463">In the language of the internet subculture — the treatment of Jordan Peterson by the mainstream media is showing up their ideology, and Red Pilling an entire generation.</p> <figure id="41f6"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYDxl1stMTCU%3Fstart%3D193%26feature%3Doembed%26start%3D193&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYDxl1stMTCU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYDxl1stMTCU%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" allowfullscreen="" frameborder="0" height="480" width="854"> </div> </div> </figure></iframe></div></div></figure><p id="8453">For more documentaries and interviews, check out the <a href="https://www.youtube.com/channel/UCFQ6Gptuq-sLflbJ4YY3Umw?view_as=subscriber">Rebel Wisdom Youtube channel</a>.</p><p id="0b51">My personal website: <a href="http://davidfuller.tv/">http://davidfuller.tv/</a></p><p id="a4b2">And please consider <a href="https://www.patreon.com/rebelwisdom">supporting us on Patreon</a>.</p></article></body>

[機器學習 ML NOTE] CNN演化史(AlexNet、VGG、Inception、ResNet)+Keras Coding

上一篇我們介紹過CNN的卷積跟池化的運作,也實作過LeNet的模型,接下來我就來簡單介紹CNN的演化史吧!!然後我也用Keras將每個模型的運作給實作出來,概念搭配code會很容易理解模型的想法。

如果對CNN不熟的可以先看這篇~

CNN演化史

我這裡只列出幾個比較重要的突破,也是比較經典的模型。

CNN Classification revolution

為什麼AlexNet會標成紅色呢?因為在2012年這一年中,AlexNet為一個重大的突破,也開始了大CNN時代,CNN開始有了很大的突破,各種CNN的模型也開始出現,也站穩了在影像上面很重要的一席之地,接下來我來簡易介紹各種模型,然後也附上各種Model在分類上的實作(Keras)

LeNet(論文連結)

沒錯,這就是CNN之父,前面也介紹過了卷積跟池化,所以應該可以很容易理解LeNet的運作,就是簡單利用了卷積跟池化,並接上全連結層做分類,這裡我們用Mnist data來實作LeNet training

LeNet Architecture, 1998
Train 5 個epoch就可以達到98.69的精確率
Loss 跟 Accuracy

AlexNet(論文連結)

AlexNet 在2012年的ImageNet LSVRC的比賽上拿了冠軍,而且準確率完勝第二名,那時候造成很大的轟動,也使得CNN開始被重視,變成所謂的CNN大時代,之後的比賽也都是由CNN拿下冠軍,深度學習正式大爆發。

AlexNet structure

AlexNet的特點:

  1. 使用了非線性的激活函數(Activation function):Relu 對於什麼是Activation function,我之後會找時間來介紹,不懂的可以先參考這個網站,在AlexNet以前,大部分的網路都是用tanh當作激活函數
  2. 使用了Data augmentation 和 Dropout 來防止Overfitting
  3. 多個GPU並行運算

之所以為什麼上面架構會分成上下2層去訓練,是因為當初在訓練的時候記憶體不夠所以分成兩塊在GPU上訓練,其實這樣的網路要弄成一層也是可以的,以下這張圖就是一層的網路架構圖,其實是跟LeNet有點類似的,這裡我用開源dataset oxflower17 來實行AlexNet,我也加入了data augumentation,大家可以參考看看。

30個epochs可達到55% 準確值
Loss and accuracy of AlexNet

VGGNet(論文連結)

VGGNet在2014年 ILSVRC 的分類比賽中拿到了第二名(第一名是等等要介紹的InceptionNet),其實VGGNet跟AlexNet之間並沒有什麼太大的差異,只是將網路的深度變深,進而得到更好的結果,我一樣用開源dataset oxflower17 來實行VGG16Net,因為VGG16的深度變深,相對的,VGG的參數就會是AlexNet的好幾倍,訓練時間也會相對的較長。

VGG16的參數量為AlexNet的3倍左右

Inception(GoogLeNet)

GoogLeNet在2014年ILSVRC的分類比賽中拿到了第一名,GoogLeNet做了一個創新,他並不是像VGG或是AlexNet那種加深網路的概念,而是加入了一個叫做Inception的結構來取代了原本單純的卷積層(但他其實也是利用卷積來達成),而他的訓練參數也比AlexNet少上好幾倍,而且準確率相對更好,所以當時才拿下了第一名的寶座,一直到現在,Inception已經衍生到InceptionV4。

GoogLeNet 架構

而GoogLeNet最重要的就是Inception架構:

Inception架構

GoogLeNet將3種convolution跟一個pooling疊在一起,增加了網路的寬度,而且這樣堆疊在一起後,更能擷取輸入圖片的更多的細節訊息跟特徵,那為什麼要加一個1x1的卷積呢?其實他的目前就在於減少圖片的維度跟減少訓練參數,假設我們input的大小為100x100x56,如果我們做一個有125個filter的5x5卷積層之後(stride = 1 , pad = 2),輸出將為100x100x125,那麼這個卷積層的參數就為56*5*5*125 = 175000,如果我們先經過28個filter的1x1的卷積,再經過有125個filter的5x5卷積層之後,輸出一樣為100x100x125,但是參數就減少成56*1*1*28+28*5*5*125=89068,相對少了快一半的參數。

GoogleNet有以下幾種不同的地方:

  1. 將單純的卷積層跟池化層改成Inception架構
  2. 最後分類時使用average pooling來替代了全連結層
  3. 網路加入了2個輔助分類器,為了避免梯度消失的情況

論文連結:

Inception V1, Going Deeper withConvolutions. Inception V2, Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift. Inception V3,Rethinking theInception Architecture for Computer Vision. Inception V4, Inception-v4,Inception-ResNet and the Impact of Residual Connections on Learning.

我們用InceptionV1論文中提到的這個Table來實現GoogLeNet的網路,跟之前一樣,都用開源dataset oxflower17來進行訓練,

Inception的參數為VGG的1/20左右

ResNet殘差網路(論文連結)

我們經過之前VGG的結果,深度學習是不是越就越好呢?如果一直無限延伸的增加層數,是不是結果就會比較好呢?我們來看一下論文裡面的圖,咦!?怎麼56層的網路結果卻比20層的網路差!!!,不是說越深越好嗎?

=======================================

在經過實驗後發現:網路層數的增加可以有效的超加準確率沒錯,但如果到達一定的層數後,訓練的準確率就會開始下降了,這代表說如果網路過深的話,會變得更加難以訓練

=======================================

那為什麼深度越深反而變得更難訓練呢?我們回到神經網路反向傳遞的部份,我們知道要更新參數的方法是利用Loss function對參數的偏”微分”來得出參數調整的梯度,通過不斷的訓練,來找到一個Loss function最小的結果,如果現在是只有1層網路,反向傳遞的梯度也就只需要傳遞1次,相對的,如果現在有100層網路,反向傳遞的梯度就需要傳遞100才能到最前面的參數,所以當網路層數加深時,梯度在傳遞的過程中就會慢慢的消失,這也就是所謂的梯度消失,就會無法對前面網路層的參數進行有效的更新,因此Deep Residual Network就出現了,也就是ResNet。

P.S.想知道關於梯度消失和梯度爆炸的可以先參考這篇文章,寫得很好。

深度殘差網路(Deep Residual Network,簡稱DRN)

重點來了,什麼是DRN呢?這邊就有點小抽象了,可能需要想一下才會理解,我先把殘差網路的架構圖放上來

從上面圖來看,左邊是一般的網路(平原網路),右邊是加入捷徑的殘差網路,我們假設H(x)是我們的期望輸出,原本網路的最後的H(x)就是F(x),而我們可以看出在殘差網路那邊在F(x)加上了原本的輸入x,所以H(x)=F(x)+x,那為什麼要這樣呢?

這裡我們來思考一下,前面我們說到了反向傳遞,是用loss function來決定的,而loss function不外乎就是跟output跟target的值,然而我們說到了梯度消失的問題,假設我們已經學習到比較飽和的準確率的時候,那麼接下來要學習的目標就會變成是”恆等映射”的學習,而什麼是恆等映射呢?其實就是要將輸入 x 近似於輸出 H(x),這樣就可以保持後面的層數中比較不會造成準確率下降。

好吧,或許你看到這裡還是有些不懂,那我們來看一下右邊的殘差網路,殘差網路在輸出的部份加入了一個x,輸出H(x)就會變成H(x)=F(x)+x,那如果我們把F(x)=0時,咦!?就會變成H(x) = x了,這不就是前面所說的恆等映射嗎?所以殘差網路就變成訓練 F(x):H(x)-x ,而目標就是要將F(x)趨近於0,這樣才可以使網路加深,準確率不會下降。

上面一堆文字你是不是快崩潰了,我自己在讀的時候也是快崩潰了XDD,我想了一下,這是不是就跟”以訛傳訛”這句成語是一樣的意思呢?如果傳的人越多,那錯誤率是不是就越高,而殘差網路的捷徑就是直接把話傳給最後一個人來減少錯誤率(如果這想法有錯請幫忙糾正><),大概就是下面這張圖的概念XD

然後論文有說到優化殘差網路比優化原本網路更好,這又是為什麼?我直接來舉個例子會比較好了解: ======================================== 假設輸入x = 5 經過第一層出來後的H(x)為5.1,那麼F(x) = 5.1–5 = 0.1 經過第二層出來後的H(x)為5.2,那麼F(x) = 5.2–5 = 0.2 原本網路的變化率會是(5.2–5.1)/5.1 = 1.96% 那殘差網路的變化率是多少(0.2–0.1)/0.1 = 100% 對的,殘差網路對於調整參數的效果會更好

殘差去掉了相同的主體部份,進而突顯出微小的變化 ========================================

終於把理念講完了(頭痛…),接下來我們來進行實作啦~~~~

我們來看一下論文上面的網路跟卷積核數量,我們會發現一件很奇怪的事,為什麼殘差網路的捷徑有分實線跟虛線的部份,再仔細看一下,虛線的部份的輸入channel跟中間的F(x)channel是不一樣的,所以虛線的部份會再對輸入x進行一次卷積來調整x的維度再進行相加=> H(x)=F(x)+Wx,W就是卷積。我們直接上Code啦(實作ResNet34)!

參數量比Inception多

結論

最後附上全部Model的Github,CNN的演進是很有趣的,有些觀念其實還滿直觀的,只是就是想不出來,這些演進大多都是基於最基本的卷積跟池化演變來的,之後還有一些CNN的變形,Attention,RCNN…等等等的CNN模型用在影像處理上面,不得不說,CNN真的是太有趣了

參考資料

大話CNN經典模型全集(這個是LeNet的,下面有其他模型的連結)

每個CNN模型的論文(上面標題都有連結)

如果有錯還請大家幫忙指正討論

Convolutional Network
Machine Learning
Deep Learning
Recommended from ReadMedium