AI Summer
AI made a lot of progress and entire Silicon Valley is catching up to it. Almost more than 40% of SP500 companies earning calls mentioned AI.
Now AI models can solve most complex problems. As of 2023:
- LLM is able to write MBA exams and get top grade
- LLM can solve international olympiad math and perform on par with gold medalist
- AlphaGo was able to play Go and beat the top player
- OpenAI model was able to win DOTA game
- Meta’s model was able to play diplomacy games (war stragey which involves planning, negotiation, etc)
AI Winter
There are lot of optimism about LLM lately. In 1970s, There were similar optimisim shown for AI, the world had spent a lot of investment and hope on AI, however it didn’t end well. The researches and investors failed to grasp the complexities and overestimated the capabilities of the modeling.
Some notable highlights:
- Machine Translation was considered important by CIA and USA during cold war era. However, the translation tool that failed badly. The famous example is traslation of
the spirit is willing but the flesh is weakis tranlated tothe vodka is good but the meat is rotten(in Russian)
- They concluded that MT is expensive and slower than human translation.
- Single layer neural network (shallow network) have failed to deliever the results.
- They didn’t know how to train multi-layer perceptrons.
- No back-propagation was invented yet.
- There was debate on the topic “The general purpose robot is a mirage” in UK. After that, many AI research facilities were dismantled.
- DARPA decided to spent money on mission oriented research. AI researches were considered to produce unlikely anything practical.
- Speech recongnization system developed in CMU would detect only if the words are spoken in specific order. DARPA felt duped and cancelled the contract.
- Generally scientists and engineers avoided using the term AI for the fear of being viewed as impractial dreamers.