Back to Blogs
Industry Insights

AI Coding Assistants: Faster Developers or Faster Bugs? What Studies Found

Google and Microsoft report growing AI-written code, but independent research raises harder questions about developer productivity, code security, and the risks of vibe coding.

S

Simranjit Kaur

AI Trainer

Oct 22, 2026
12 min read
AI Coding Assistants Boost or Bugs

AI coding assistants promise a tempting shortcut: describe what you want, accept the generated code, and ship faster. Tools that autocomplete functions, explain unfamiliar code, write tests, and scaffold applications have become part of everyday software development. The rise of AI coding assistants has also helped popularize vibe coding building software by describing an idea in natural language and relying heavily on AI to implement it.

But there is a difference between generating code, delivering software faster, and delivering software that is reliable and secure. A model can produce code that looks convincing, passes a quick demonstration, and still contain broken assumptions, hidden defects, or exploitable vulnerabilities.

This article compares public claims from Google and Microsoft leaders with two important 2025 research findings: METR’s controlled study of experienced developers and Veracode’s GenAI Code Security Report. The goal is not to declare AI coding either a miracle or a failure. It is to understand where it helps, where it adds risk, and how teams can review AI-generated code responsibly.

What Are AI Coding Assistants?

AI coding assistants use large language models and related systems to help developers create, explain, refactor, debug, and test software. Depending on the tool, they may provide inline autocomplete, chat-based code generation, repository-aware suggestions, or agentic workflows that edit files and run commands.

Common uses include generating boilerplate, drafting unit tests, translating code between languages, explaining legacy functions, creating prototypes, and suggesting fixes for errors. These capabilities can reduce repetitive work, but their output depends on the quality of the prompt, available project context, tool permissions, model limitations, and the developer’s ability to verify the result.

The Big Claim: AI Is Already Writing a Significant Share of Code

What Sundar Pichai Said About Google

In April 2025, Alphabet and Google CEO Sundar Pichai said that more than 30% of new code at Google was being generated by AI, up from a reported 25% in October 2024. In a separate discussion, he described Google’s estimate of an approximately 10% increase in engineering velocity across the company attributable to AI.

These are meaningful adoption and productivity signals, but they measure different things. The share of code generated with AI is not the same as the share of engineering work completed faster. Nor does either figure, by itself, tell us how much review, debugging, testing, or rework the generated code requires.

Source: Moneycontrol’s report on Pichai’s April 2025 remarks and the interview transcript discussing engineering velocity.

What Satya Nadella Said About Microsoft

At Meta’s LlamaCon event in April 2025, Microsoft CEO Satya Nadella said that perhaps 20–30% of code in Microsoft’s repositories and some projects was being written by software—that is, AI. He also noted that results differed across programming languages, with stronger progress in some areas than others.

Again, the statement indicates substantial use of AI-assisted development. It does not establish that every generated line is accepted unchanged, that all teams see the same benefit, or that code quality and security improve at the same rate as code generation.

Source: TechCrunch’s report on Nadella’s remarks.

Why AI-Written Code Percentages Do Not Prove Developers Are Faster

A code-generation percentage answers a narrow question: how much code is attributed to AI under a company’s definition? Productivity is broader. It includes understanding requirements, navigating a codebase, reviewing suggestions, testing edge cases, fixing regressions, handling security, and maintaining the software months later.

To evaluate productivity, teams should look beyond lines of code or accepted suggestions. Useful measures include:

  • Cycle time: how long it takes a change to move from a clear task to a reviewed, deployable result.

  • Review and rework time: how much effort is spent correcting AI output.

  • Defect escape rate: how many defects reach later testing or production.

  • Security findings: whether generated changes introduce vulnerabilities or unsafe dependencies.

  • Maintainability: whether the next developer can understand and safely modify the code.

A team that generates twice as much code but spends the extra time debugging it may not have improved productivity. More output is not automatically more value.

What METR’s 2025 Trial Found About Developer Speed

In a randomized controlled trial published in 2025, Model Evaluation and Threat Research (METR) studied 16 experienced open-source developers working on 246 tasks in mature repositories they already knew well. The developers used early-2025 AI tools, primarily Cursor Pro with Claude models, on tasks randomly assigned to allow or disallow AI use.

Before the tasks, participants expected AI to reduce completion time by about 24%. Afterward, they estimated that AI had made them roughly 20% faster. Yet the measured result went the other way: with AI tools available, developers took 19% longer to complete the tasks in this experiment.

This result challenges the assumption that AI assistance always speeds up experienced developers. In a complex, familiar codebase, the time needed to inspect suggestions, provide context, correct mistakes, and ensure that changes fit the project can outweigh the time saved generating code.

Important limitation: this was a small study involving a particular group of developers, repositories, tasks, and tools available in early 2025. It should not be generalized to every developer, beginner, task, or newer AI system. METR later reported that its subsequent experiment faced selection and measurement problems, making its newer estimates weak evidence for the size of any productivity effect.

Sources: METR’s 2025 study paper and METR’s February 2026 update on its experiment design.

What Veracode’s 2025 Report Found About AI Code Security

Speed is only one part of the story. Veracode’s 2025 GenAI Code Security Report assessed code generated by more than 100 large language models across Java, JavaScript, Python, and C#. In the report’s test tasks, 45% of generated code samples introduced a detectable OWASP Top 10 security vulnerability.

The report also found that models often produced syntactically correct code without reliably choosing a secure implementation. For example, a prompt may ask for a database query but fail to specify parameterized queries. A model may satisfy the functional request while choosing a pattern that exposes the application to SQL injection. Similar risks can arise with cross-site scripting, unsafe data handling, weak access controls, and other security-sensitive operations.

That 45% figure needs careful interpretation. It comes from Veracode’s designed evaluation tasks and security tests; it is not a measurement of the percentage of all AI-generated code in real production systems that is vulnerable. The finding does, however, reinforce a practical rule: functional correctness and security are separate properties, and both need to be tested.

Source: Veracode’s 2025 GenAI Code Security Report summary and the full report.

Vibe Coding Security: Where the Risk Enters

Vibe coding can be useful for prototypes, learning exercises, internal tools, and quickly exploring an idea. Risk rises when developers accept generated changes without understanding them, especially when the application handles user data, payments, credentials, private files, or business-critical workflows.

Common Risks in AI-Generated Code

  • Insecure defaults: code may omit authentication, authorization, input validation, rate limiting, or safe error handling.

  • Injection vulnerabilities: SQL, shell, or other commands may be constructed unsafely from user-controlled input.

  • Secrets exposure: API keys, tokens, credentials, or private configuration may be placed in source code or logs.

  • Dependency risk: suggestions may use outdated, unnecessary, misspelled, or untrusted packages.

  • Broken access control: a feature may work in a demo but allow users to access another user’s records or restricted actions.

  • Over-permissioned agents: coding agents may be able to read files, run commands, or modify a repository beyond what a task requires.

  • False confidence: polished explanations and passing happy-path tests can conceal missing edge cases or unsafe assumptions.

These problems are not exclusive to AI. Human-written software can contain the same defects. AI changes the workflow by making it easier to produce large amounts of code quickly, including code that the person accepting it may not fully understand.

A Practical Checklist for Reviewing AI-Generated Code

Use this checklist before merging or deploying code produced by an AI coding assistant. The higher the impact of the application, the more rigorous the review should be.

1. Understand Every Meaningful Change

  • Can the developer explain what the code does and why the approach is appropriate?

  • Check the complete diff rather than reviewing only the snippet shown by the assistant.

  • Remove unrelated changes, unnecessary complexity, and code that cannot be justified.

2. Validate Inputs and Enforce Access Control

  • Validate type, length, format, and allowed values at trust boundaries.

  • Enforce authorization on the server for every sensitive operation; do not rely only on hidden UI controls.

  • Use parameterized database queries and safe APIs instead of string-built commands.

3. Protect Secrets and Sensitive Data

  • Never hard-code passwords, private keys, or production tokens.

  • Use approved secret management and least-privilege credentials.

  • Check logs, error messages, prompts, and test fixtures for exposed personal or confidential data.

4. Test Behaviour, Not Just Syntax

  • Run unit, integration, and regression tests relevant to the change.

  • Test invalid input, boundary values, unauthorized access, failures, and unexpected states.

  • Verify that tests actually assert the intended security property instead of merely repeating the implementation’s assumptions.

5. Run Security and Dependency Checks

  • Use approved static application security testing (SAST), software composition analysis (SCA), and secret scanning.

  • Check dependency names, versions, provenance, known vulnerabilities, and license requirements.

  • For web applications, test common OWASP risks, including injection and broken access control.

6. Constrain Coding Agents

  • Give agents only the file, repository, network, and command permissions they need.

  • Review shell commands and destructive operations before execution.

  • Keep sensitive production credentials out of the development environment and require human approval for high-impact actions.

7. Require Accountable Human Approval

  • Assign a named reviewer who understands the changed component and its risk.

  • Document important assumptions and any accepted limitations.

  • Do not waive review because the code came from a popular model or passed a demo.

How Developers and CEOs Should Measure AI Coding Success

For engineering leaders, the right question is not simply, “How much code did AI write?” It is, “Did AI help us deliver a better result at a lower total cost and acceptable risk?” A practical evaluation should compare similar tasks with and without AI, track time spent on review and rework, and measure defects, security findings, maintainability, and developer experience.

Teams should segment results by task type and developer experience. AI may be especially useful for boilerplate, documentation, test scaffolding, and explaining unfamiliar APIs, while complex architectural changes or security-sensitive code may require more careful supervision. Results should be measured in the team’s own environment rather than assumed from vendor demonstrations or company-wide adoption figures.

What This Means for Students and Aspiring Developers

AI coding assistants are useful tools to learn, but relying on them without understanding fundamentals creates a fragile skill set. Developers still need programming logic, data structures, debugging, testing, databases, networking basics, secure coding, and the ability to read unfamiliar code.

A strong learning approach is to attempt a problem independently, use AI to compare approaches or explain errors, and then verify the solution through tests and documentation. Ask the assistant to explain trade-offs, generate edge cases, and identify possible security weaknesses but treat those answers as hypotheses to validate, not proof.

For learners exploring AI courses in Jalandhar, at Thread Security Education prioritizing practical training that combines AI-assisted development with Python or another programming foundation, Git, testing, application security, code review, and real projects. A useful course should teach you how to evaluate generated output, not simply how to prompt a tool to produce it.

How Thread Security Approaches Responsible AI Skills

At Thread Security, the important conversation is not whether developers should use AI coding assistants. It is whether they can use them with the technical judgement required to build dependable software. AI, software development, and cybersecurity increasingly overlap, so developers benefit from learning both how to build features and how to test the assumptions behind them.

When comparing AI training or technical programmes, look for hands-on projects, debugging practice, secure coding exercises, testing workflows, and review of real failure cases. These are the skills that help turn AI-generated output into software that can be maintained and trusted.

Conclusion: Faster Code Is Not Always Faster Software

Google’s and Microsoft’s statements show that AI-generated code is already a significant part of software development at major technology companies. METR’s 2025 trial shows that AI assistance can also add time in some real-world tasks, while Veracode’s report highlights security weaknesses in generated samples. These findings are not contradictory: adoption, productivity, and security are different measures.

The sensible approach is neither to reject AI coding assistants nor to trust their output blindly. Use them where they help, measure the full development cycle, review the diff, test edge cases, run security checks, constrain agent permissions, and keep humans accountable for the code that ships. The goal is not to generate the most code. It is to deliver software that works, remains maintainable, and protects its users.

Call to Action

In this week run the 10 point checklist on one of your own sets of data and make notes on what is missing. Looking to have your data pipelines and AI systems reviewed? Check out threadsecurity.org. To enhance the skills of ML security in Jalandhar, visit Thread Security Education.

ARTICLE TOPICS
#Thread Security
#Generative AI
#AI Coding Assistants
#Vibe Coding Security
#AI-Generated Code
#Software Development
#AI Code Security
#Secure Coding
#Developer Productivity
CUTTING-EDGE AI TRACK
100% Practical Defense Labs

AI Security & LLM Red Teaming Masterclass

Learn to identify and mitigate prompt injections, jailbreaks, data poisoning, and model theft in production AI systems with sandboxed exercises.

S

Simranjit Kaur

VERIFIED

AI Trainer · Thread Security

Academic Verified · E-E-A-T

Faculty mentor & research guide at Thread Security Education.

Specializations: AI Trainer
CONNECT:LinkedIn

Related Research & Articles

View All
Business Email Compromise Invoice Fraud
Industry Insights

Business Email Compromise: How Invoice Fraud Works and How Finance Teams Can Stop It

Business Email Compromise can turn routine invoices into costly fraud. Learn how payment redirection works and how finance teams can stop it.

By Akshay Kumar

Ransomware Payment  Risk vs Recovery
Industry Insights

Should Companies Pay Ransomware Demands? What Law Enforcement, Insurers and Data Say

Ransomware payment decisions are complex. Explore what law enforcement, cyber insurers and recovery evidence say before a company pays.

By Akshay Kumar

AI Banking Fraud Detection
Industry Insights

AI Fraud in Banking: How Banks Flag Suspicious Transactions

AI fraud detection in banking helps flag suspicious transactions, but false positives and human review remain important.

By Simranjit Kaur