The security problem with AI-generated workflows is not that they fail. It is that they work and nobody can explain how.
“Teams are dealing with a truly dangerous problem, automation that works, but that no one understands,” said Yelena Mujibur Sheikh, a cybersecurity engineer at BNSF Railway, writing for Dark Reading. The observation is blunter than most vendor threat reports permit themselves to be and it is more useful for it.
The pattern Sheikh describes is consistent across organizations that have moved quickly on AI tooling, a developer or analyst uses an AI assistant to generate a workflow, the workflow solves an immediate problem, it gets deployed, and it quietly accumulates access permissions it was never deliberately granted. Nobody reviews the underlying logic. Nobody maps what data it touches. It runs.
The Permissions Problem Is Structural, Not Accidental
AI-generated code tends to request broad permissions because that is the path of least resistance when generating functional output. The model is optimising for code that runs, not code that is least-privileged. The result is automation that routinely holds more access than any human reviewer would consciously approve.
Veracode’s research on AI-generated code security found that insecure patterns and vulnerable dependencies appear with regularity in code produced by large language models, not because the models are malicious but because they are trained on codebases that contained those patterns. The output reflects the input. If the training data included poorly scoped service accounts and hardcoded credentials, the generated code will reproduce them.
The second structural issue is what Sheikh and others call shadow automation, AI-generated workflows deployed outside formal IT governance, often by business teams who have direct access to low-code or no-code platforms. These workflows do not appear in asset inventories. They are not covered by access reviews. Security teams cannot monitor what they cannot see.
Poisoned Models and Cascading Outputs
The more serious risk is upstream. When an AI model that feeds into an automated decision workflow is compromised or manipulated, the consequences do not stay local. A poisoned model can silently approve fraudulent transactions, generate false business intelligence or misroute data in ways that look like normal operation until the damage is already done.
Dark Reading reported a related attack class in May 2025, fake bug reports submitted to AI coding agents that cause the agent to introduce malicious code into a repository at scale. The agent does not know the bug report is fabricated. It processes the input, generates a fix and commits it. No human reviews the change before it reaches production in many CI/CD pipelines configured to trust agent output.
That is the cascade problem in concrete form. One manipulated input, processed by a trusted automated system, propagates through every downstream dependency. The speed that makes AI-assisted development attractive is the same property that makes it dangerous when the input cannot be trusted.
The structural problem Sheikh describes is real and well-evidenced. The inflation of that risk into headline-ready statistics is where commercial interest tends to distort the picture.
Three Controls Worth Implementing Before the Next Deployment
None of the mitigations here are novel. They are the same controls that apply to any automated system, applied consistently to a category of automation that organizations have been treating as exempt.
Review AI-generated code before deployment, not after. This means treating AI-assisted output the same as any external dependency, static analysis, dependency scanning, and a human reviewer who understands what the code does and what it can access. Accepting agent output directly into production pipelines without this step is not a speed optimization. It is an unreviewed change control.
Scope permissions at the point of creation, not after the fact. AI-generated workflows that request broad access should be challenged at review, not trimmed once a security audit catches the exposure. Least-privilege defaults need to be enforced at the platform level, not left to the developer to apply manually.
Map your shadow automation now. Any team using no-code or low-code AI platforms to build workflows that connect to production data or systems should be required to register those workflows with IT. The inventory does not need to be complex. It needs to exist. You cannot monitor drift in workflows you did not know were running.
References
- AI-Generated Workflows Are a Silent Security Disaster
- Fake Bug Report Hijacks AI Coding Agents at Scale
- AI-Generated Code Security Risks
This post is also available in:
Svenska
