It seems like we have artificial general knowledge, but not artificial general understanding; language models have very good breadth but they fail to make links between things. They spout all the right words but lack actual understanding.

Interactive tutoring sessions, first between people and then between models that have been trained on such sessions could bring them up to speed, and pass general wisdom from larger models to smaller ones. Allowing us to spend weights on things that matter and offload the rest to knowledge databases.