The problem is SEO.

You can’t just use keywords anymore because every site out there is actively exploiting that system. Being the search engine everyone uses inherently means everyone will try to cheat to be overrepresented. It’s an adversarial system now, which inherently forces an arms race between the search engine and web sites, meaning a search engine has to constantly evolve to stay ahead.

They’ve failed to do that and I definitely don’t like the end product now, but there’s nothing they can do to be what they were 10 years ago either. The web has changed.