Here is the onboarding scenario I'm envisioning:
The AI is put on a laptop/desktop/workstation that it has root permissions on
It's given access to a shared work chat (Slack, Discord, whatever) where it can ask questions/request additional permissions/etc
It needs to setup a workstation that can pull, build, and edit at least one of the org's primary codebases.
It needs to make one accepted pull request (or equivalent concept).
The workstation can be the local machine or a cloud instance
Time limit of five business days.
In short: an AI that can go through the onboarding process for a software engineer.
Caveats:
Resolves NO if no one tries to do this (which seems reasonably likely)
The pull request does not need to be interesting or important. A "hello world" PR is fine.
YES does not require that an AI be as good at this as a human engineer, but it can't be significantly specialized for one environment. It needs to work with at least three different environments (e.g. setup Python locally, setup typescript on AWS, and C++ on Azure).