The co-evolution of technologies is fascinating. If we look at the 2012 ImageNet paper, that was 90s tech using the recent advancements in computing power. But even if you had the compute power you couldn't build ChatGPT in the 90s or even 2012. The data didn't exist yet. All the blogs explaining step by step how to do things hadn't been written yet. Not to mention question answer sets like StackOverflow. That data is critical for the compositional intelligence that ChatGPT applies. Simply training on the encyclopedia would have still been not much different than search. Now in the early 00s perhaps one could have trained on the wealth of data buried in endless nested replies in mailing lists but we would have thought the highly indented, repetitive output was just babel.