Had a shower thought: Is all this evangelization of AI among knowledge workers just training the LLMs to eventually push all of us out of our own jobs?

Think about it:

• Increasingly we’re being pushed (and pushing each other) to embrace AI and embed it into our workflows. It’s being positioned as a 10-100x booster
• Every prompt, every reply helps the AI tool understand not just what we ask, but how we respond to it - helping it optimize its response over time
• Eventually it gets so good at the use cases for which we’re “collaborating” with it now – strategy sparring, initiative planning, prototyping, product engineering, content creation, you name it – that at some point you’re no longer needed as a decider. Your boss can just use Claude or ChatGPT or whatever to get the outputs that you used to provide
• Then your boss’s boss can do the same with your boss’ output, and so on
• Unless the one(s) in charge consciously decide that employing people is of value to the company, there’s nothing to really stop this over a long time-scale

Is this a legitimate train of thought? Am I justified in thinking that eventually all jobs in software can become obsolete over the long haul, or is this overkill? Chat, what do we think of this?