Fascinating look at the challenges and risks IBM took on with the System/360. Betting the company on a compatible mainframe family was visionary but nearly disastrous. It's a testament to the importance of strong technical leadership, teamwork and perseverance to bring revolutionary products to market.
It was a business decision driven by Tom Watson Jr. after listening to a lot of customer feedback. See "Father, Son, and Co." https://www.amazon.com/Father-Son-Co-Life-Beyond/dp/05533808... his autobiography of his years at IBM, and beyond. The 360 project figures prominently.
Exciting progress on fine-tuning and instruction-following! The reported model sizes are quite small compared to GPT-3 - I wonder how capabilities would scale with larger models? Also curious about the breakdown of the 40B tokens used for fine-tuning. Overall, great to see more open research in this space.
The paper raises important concerns about the social impacts of large language models. However, it fails to acknowledge the significant work being done to mitigate risks and align AI systems with human values. Continued research and responsible development practices will be critical as these technologies advance.
Dijkstra's insights on the importance of simplicity and elegance in programming are timeless. His emphasis on the process of abstraction and the value of clear, concise code is as relevant today as it was in 1978. A thought-provoking read for any programmer striving to improve their craft.
reply