Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Featured Mind boggling machine learning results from AlphaZero

  1. Dec 6, 2017 #1

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    I have always been on the skeptical, but not dismissive, end of judgments about achievemnts and rate of progress in this field. However, the following (please read through carefully) just completely blows my mind:

    https://en.chessbase.com/post/the-future-is-here-alphazero-learns-chess

    A self learning algorithm with no knowledge or base of games, starting only from the rules of chess, within 24 hours is much better than any other existing chess program despite running on hardware 900 times slower!!!!
     
  2. jcsd
  3. Dec 7, 2017 #2
  4. Dec 7, 2017 #3

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    True, but what shocked me was the leap beyond any prior results in machine learning that I’m aware of.
     
  5. Dec 8, 2017 #4
    It would be interesting if some kind of Moore's law could be applied to the evolution of smarter machines, not just that due to faster processing power, but from the use of better algorithms. The people and teams that work on these systems seems to be themselves some kind of brainiacs putting it all together and making it work.
     
  6. Dec 8, 2017 #5

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    It is perhaps worth clarifying that this is an amazing result in machine learning in a closed domain (fixed rules, fixed definition of value). It does not address an open domain, or AI per se, at all.
     
    Last edited: Dec 8, 2017
  7. Dec 8, 2017 #6

    QuantumQuest

    User Avatar
    Gold Member

    I have been playing chess for a long time so I can see the value for us humans regarding the game itself in the long run but clearly it is something beyond our reach to follow.
    From an AI perspective it is very impressive but of more value is (quoting from the article);

    And once again, the basic principle taught in intro courses in CS that advancements in hardware cannot outperform a very efficient algorithm (a fortiori combined with some sort of neural network and its respective processes), is popping up.
     
    Last edited: Dec 8, 2017
  8. Dec 8, 2017 #7
    If you have been following any of the other threads on AI with regards to its impact on society you may have notice a distinct group who believe that AI's impact and in particular rivaling human intelligence is nil even in the next century. In particular our understanding of cognitive processes and the state of the art of computer systems does not support the idea of super general intelligence that might overcome human intelligence.

    The success of machine learning in Go, Dota2(video game) and the new one in Chess which have occurred recently should give those doubter some pause to reconsider their opinions. In particular the rate of increase in the improvement in performance. The article in the OP noted that it would take a decade for AI to compete in Go against humans. It took three years. In another year it defeated the world champion 3-0. Now the AlphaGoZero has beaten that system 100-0 and with 1/12 the processors of the original system.

    Both in Go and in Chess the AlphaGoZero system has "found" new strategies/moves not heretofore identified surprising developers. In Go for example the new strategy was adopted by the world champ and he went on a 22 game winning streak against other players.

    Is there a Moore's law for AI? Of course we will have to come up with a metric to determine it. But there seems to be significant progress in AI in recent years. Note that the AlphaGoZero is a factor of 900 slower than other Chess systems but still outperforms them AlphaGoZero developers seem to keeping in mind the motto for their system "work smarter not harder".
     
  9. Dec 8, 2017 #8

    MathematicalPhysicist

    User Avatar
    Gold Member

  10. Dec 8, 2017 #9

    PeroK

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    I was an average chess player at about 1800-1900 ELO. In terms of the chess playing, computers were better than me almost from the outset. The most remarkable thing to me is the standard to which some humans can play chess. The idea of having any chance against a modern computer is absurd.

    If chess playing or go playing is a measure of intelligence, then computers have been more intelligent than me for decades.

    And, am I really that much less intelligent than Magnus Carlsen? It seems to me that he is more like a machine than a human in terms of his chess playing ability.

    That said, Alpha zeros approach to learning chess is remarkable. What it did to Stockfish in some of those games was beautiful.
     
  11. Dec 8, 2017 #10

    MathematicalPhysicist

    User Avatar
    Gold Member

    Well, I never really enjoyed playing Chess so much, it's just a matter of rote memorizing all the correct combinations.

    Playing football or basketball is a lot more fun, more spontaneous.
    Either way, you need to practice a lot to be a master in something, like the fact that the machines played so many games.
     
  12. Dec 8, 2017 #11

    MathematicalPhysicist

    User Avatar
    Gold Member

    BTW, when I first heard Edward Witten talk, I thought he was a robot.

    He has that voice pattern... :-D

    Reminds me of Blade Runner.
     
  13. Dec 8, 2017 #12
    Certainly fantastic results from the colloborators at DeepMind. The next step is going to be a major hurdle though--we'll have to extend it to model-free control. The Test of Time award winner at NIPS, Ali Rahimi, gave a great talk about the ever more essential need for greater scientific rigor and the development of useful theoretical foundations in Machine Learning such that we might make some insights to approach problems in the field that currently seem almost intractable (i.e. high dimensionality, etc.). For example, one thing he mentioned was how we use batch normalization to accelerate gradient descent, and how this is "explained" by saying batch norm "reduces internal covariate shift." The problem is, we really haven't a clue as to *why* reducing internal covariate shift speeds up gradient descent. Essentially, he wants us to move away from the sort of unguided engineering approach, and move towards a culture of and approach to research more similar to Biology (as opposed to just training our models conventionally and trying to reduce error rate as much as possible, and ultimately just a more empirical 'be creative and see what sticks' kind of approach).

    Interestingly, this has spawned a debate in the community. Prof. Yann LeCun came out and officially disagreed with Rahimi, stating that the state of the art so far has been developed due to the clever engineering of researchers (he took offense to what Rahimi called it: "Alchemy"), and that practical technology almost always precedes the theory developed to fully explain the technology (i.e. Watt's steam engine came before Carnot Cycle--you can't criticize Watt for not being Carnot, basically). I think the essence of the debate is whether the current culture of research will be beneficial or detrimental for the field going forward--Rahimi seems to think it is becoming detrimental, since it's still a challenge to pedagogically deliver ML to students without any anchoring theoretical foundations (you actually have to "get good" at machine learning by developing a lot of experience and intuition while training models, without being able to anchor back to any real first principles), and the need to have proper explanations for the workings of our ML systems in the context of life or death situations involving humans (i.e. autonomous driving, cancer pathology, etc.). Prof. LeCun thinks that this way of doing things is just fine, and that Rahimi shouldn't needlessly criticize the work done so far, instead go and do the theory work, to which Rahimi replied that the talk he gave was basically a public plea for help.

    I see both sides' arguments, and it's been very interesting to follow the discussion so far. Either way, I'm excited to see where we'll be in 5 or 10 years, when we'll have seriously improved and expanded datasets for model training combined with superior hardware like MITs programmable nanophotonic processor or utilizing the phase-change memory work of IBM researchers to allow for massively parallel computing systems, which would be great for ML. Maybe by then we'll have progressed a good bit on developing theoretical foundations in ML.
     
    Last edited: Dec 8, 2017
  14. Dec 8, 2017 #13
     
  15. Dec 8, 2017 #14

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    To contrast with this video, Magnus is a big fan of Monty Python, an interest shared with Viswanathan Anand, and there are videos of them doing skits together. He is also a fan of Donald Duck - this is apparently a Norway thing - Donald Duck is quite popular there.
     
    Last edited: Dec 9, 2017
  16. Dec 8, 2017 #15

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    As for blindfold play, this is a specialty with the current record being 48 simultaneous blindfold games:

    https://www.chess.com/news/view/timur-gareyev-plays-blindfold-on-48-boards-5729

    (In the above video, Magnus plays 10 simultaneous blindfold games, the most he has ever done. The record that stood for decades was by Miguel Najdorf during WW II of 45 simultaneous blindfold games. Of interest about this is that one player in Timur’s exhibition had also played in Najdorf’s. Even more remarkable is that Najdorf’s exhibition was motivated in part to try to let his family in Nazi Germany know he was ok and alive. He had fled to Argentina and there was no normal communication method. He figured correctly that his feat would be covered even in Germany and his family would see it. Postwar, it was verified that his idea had worked.)
     
    Last edited: Dec 8, 2017
  17. Dec 8, 2017 #16

    Delta²

    User Avatar
    Homework Helper
    Gold Member

    Well to be honest I have my doubts, though it seems astonishing to beat the top engines that are based solely on search and evaluation algorithms and rely on NPS computational power (Nodes(positions) per Second processed(searched and evaluated)),

    still reading the article it says that the engine as white prefers to play the English opening (1.c4 ...) or the Queen's Gambit openings (1.d4 d5 2.c4 ...).

    According to Bobby Fischer, one of the top players in all the history of chess, best first move for white is 1.e4 .
     
  18. Dec 8, 2017 #17

    mfb

    User Avatar
    2017 Award

    Staff: Mentor

    For some reason I trust the preference of an algorithm that beats all algorithms that consistently beat all humans more than the preference of a human. e4 might be a good move against humans, but apparently not against much stronger opponents.
     
  19. Dec 8, 2017 #18

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    Fischer said that, but in his critical match against Spassky in 1972, he played English and QGD frequently as white. In fact his first win as white in this match was an English that transposed to a QGD.
     
  20. Dec 8, 2017 #19

    Delta²

    User Avatar
    Homework Helper
    Gold Member

    I don't think its good to trust a computer program more than a human Top GM(GrandMaster).

    The problem with humans against engines is that humans (even strong GMs) lack behind in terms of tactical processing of a given position in board. However humans are better in positional processing of the position.

    Since time is a factor (both players start with a limited time) and humans spent a lot of time calculating the tactical complexities of a position (a human even a GM, might spend 5min for something that an engine can process in 0.005min, like for example to see an elegant queen sacrifice that leads in a forced mate in 5 moves) that's the main reason humans are getting beaten by engines.

    If we allow for a hybrid of a human GM+a typical engine that analyses positions so that the GM sees the engine analysis for the various possible moves he has in mind, then i believe this hybrid can beat any engine, (stockfish or rybka , or even alphazero or whatever).Or if we allow for very big(slow) time controls, so that the human has a lot of time to think about the tactical complexities of a position, then I believe a human GM has the advantage over any kind of engine.
     
  21. Dec 9, 2017 #20

    PAllen

    User Avatar
    Science Advisor
    Gold Member

    It is true that centaurs (human + computer) beat computer alone for current commercial engines. Human with long time control no longer does. It would certainly be interesting to try too human plus stockfish or Houdini against alphazero. I am not so sure the centaur would win. Some top grandmasters have likened alphazero’s play to perfected Karpov, i.e. immense positional understanding.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Mind boggling machine learning results from AlphaZero
Loading...