A very provocative piece. The author uses the word “knowledge” a lot, but not as most of us would understand it; in a way I would consider to be “data” or “output”. When AlphaGo outputs a move, I don’t consider that knowledge, simply the output of a black box machine. When I turn the crank on my meat grinder, what I get is hamburger, not knowledge.
AlphaGo is able to “win” for two reasons: the machine learning algorithm had access to test games with the full range of moves from start to finish, and the rules of the game completely described the Go world — there was no Chaos at work in the rules of the game. If the computer had been fed only data between moves 96 and 121 of all the test games, none before or after, it would not be able to “understand” the game. If the rules of Go allowed for fractional moves, fractional stones to be placed on multiple squares, fractional square on the board, computer learning would have been vastly more complex, perhaps fatally so. AlphaGo was successful only because it had complete access to complete games to learn from. The “Go world” was completely understandable and describable in machine language.
Consider the climate, on the other hand. Any machine learning algorithm will have access to reliably measured data across only one century or two, when a great many forces display very little variance, compared to millenia and eons of climate operation from which the computer will have no data to learn from. Moreover, it will have data for only the 20 or 30 phenomena for which sound measurements have been taken, and for only those few places on the earth where measurements have been thorough. Vast time scales, vast other forces which we can’t measure or which we don’t even know about, vast spaces where no measurements have been taken, the operation of chaos at the smallest and largest scales, render the climate problem almost immune from modelling with any hope of much success.
One last thought. The author considers the model “right” when it produces correct outputs, correct in comparison to the real world. But if we have no clue how the model works, we will be unable to recognize when some condition or group of conditions in the real world have changed in a way that renders the model’s output “wrong”. And then we will rely on that output, unaware that it can’t be correct.