This is the best-informed comment so far about what generative AI works, what it's good for, and what it isn't.
I'm a software engineer with several decades of experience, and after extensive experimentation with ChatGPT and Bard (Google's effort) I have come to the following conclusions:
- They are hopeless at mathematical / algebraic manipulation. They will drop terms or fail to perform symmetric operations on boths sides of equations or inequalities. They have no "understanding" of mathematical rules.
- If you present a problem and the AI can identify a corresponding design pattern, the AI can generally successfully apply the design pattern to provide a solution.
- If the AI can't identify an applicable design pattern, it may attempt to apply an incorrect design pattern and provide an invalid solution.
- If you point out that the provided solution is incorrect the AI will either assert that it is correct, or will provide an alternative invalid solution.
- While the AI can provide reasonable first-cut models (such as data tables or object classes) if you attempt to build this out into a real-world solution the AI will start to forget its earlier design decisions and start re-engineering previously accepted entities. If you ask it to revert to the accepted versions it is just as likely to forget everything you have previously established and produce something completely generic and unacceptable.
- In essence, if the generative AI has not seen the solution in its training data it is not capable of "developing" the solution.
You don't have to be a programmer to see that generative AI is applying standard patterns to its responses. If you ask it to write a short story about "Goldilocks sneaks into the bears' house while they are out. When the bears return they find Goldilocks inside." You will get a response with this pattern:
- Repetition / expansion of your scenario.
- The bears return and express their dissatisfaction to Goldilocks.
- Goldilocks realises she behaved badly and determines to do better.
You will not get:
- Any insight into why Goldilocks invaded the bears' home.
- Any twists or evidence of creativity. The bears will never suggest to Goldilocks how she could monetise home invasion in return for a cut of the proceeds, for instance.
So any job that involves any of these things is likely to disappear relatively quickly:
- Repetitive tasks with little variation and no need for innovation: manufacturing, food preparation and service, retail services.
- Digestion of large amounts of information and extracting just the relevant material: legal, general medical, accounting, investment advice.
- Application of established patterns to well-understood problems: architecture, engineering, building, scheduling, project management, software development, commercial art and literature.
Jobs that require creativity, innovation and novel thinking are likely to continue for longer:
- Deal with unpredictability requiring frequent re-prioritisation and innovative responses: executive assistants, emergency workers, police, military
- Cutting edge research.
- Developing novel solutions: brilliant architects, brilliant engineers, ace software developers, untethered artists, specialist medical workers
- Resolving wicked problems: senior management, justice, government
(20min delay)