AI-Only Code Reviews and SOC 2 Change Management: Experiences and Compliance Insights Requested | Vanta Community

Vanta Community Icon

AI-Only Code Reviews and SOC 2 Change Management: Experiences and Compliance Insights Requested | Vanta Community

Vanta Community
APP
·
·
This post is now synced! Check it out here:
AI-Only Code Reviews and SOC 2 Change Management: Experiences and Compliance Insights Requested
⤵️ Any future comments will be automatically posted to the thread
Rudy M.
·
·
Thanks for surfacing this topic. It's on my radar to work out what to do about soon. We've not actively moved yet, but we do plan to get there soon ... as soon as I'm done with my SOC2 Audit that is about to kickoff. So I didn't want to add any change this close to Audit. I'm also keen on a coffee chat on the topic in general, as there's many unknowns to think about, but is a topic I need to work out how best to unlock for my org. There's some tooling I need to get in place (on my side) before we get there though, to catch any significant drift that might be caused by agentic code review principals. Super keen to hear how others are approach the topic too.
👍1
1
Jason L.
·
·
RK K., the key phrase in your setup is "certain tiers of changes". This is how you are going to be able to sell it to an auditor and customers (assuming they ask). Formally identifying the tiers and capturing the process you use for determining the "risk" associated with those tiers, you simply add a few edits to your change management standard that says "X risk level or higher changes require formal change approval. Changes below that level of risk are eligible for automatic integration". Explain to the auditor (and document somewhere) that the compensating controls for the lower risk changes include the ability to roll it back, staged rollout (canary testing), or whatever else you choose. For the integration, this might be difficult. The effectiveness of scoping will likely be mixed. If it doesn't work, you will need to disable Vanta's testing and rely on alternative evidence collection.
💡2
RK K.
·
·
Great advice, thanks Jason L.! Is this something your team is doing today?
Rob G.
·
·
Really great questions and topic RK! We still have a HITL for all reviews, but I've put in a feature request to Vanta to be able to pull in claude code reviews, since it is a part of our process and evidence I show our auditors. Also curious what soc 2 auditors would think of this and if that'd suffice for the review without the human. My gut would tell me no, but I can see the case otherwise
💡1
1
Eric S.
·
·
We are exploring starting with a simple control that states something like the following: The company has configured an agentic AI process to automatically review all code merges for security and quality issues prior to production release. When issues are discovered, the agent does not approve the merge. The second part of that sentence could be further adjusted to specify ...when cybersecurity specific issues are discovered...
💡1
Rudy M.
·
·
some useful reading on the topic -> ideas.fin.ai/p/ai-is-approving-our-pull-requests RK K. - curious to hear if you progressed any further in your discovery?
👀1
RK K.
·
·
Hi Rudy M., no update since last. Thank you for sharing the article, this is helpful
Rudy M.
·
·
Thanks. Would love to hear where you land, once you get further please. It's a topic I presume most of us will need to address in some form over the next few weeks/months. Thanks
👍1
💯1
Rudy M.
·
·
more reading context -> funnel.io/devblog/the-integration-of-ai-code-reviews-at-funnel
Rob G.
·
·
I just straight up asked our auditors (A-LIGN - SOC 2, ISO 27001, ISO 42001, FedRAMP). Will let you all know what they say
2
Rudy M.
·
·
That would be super helpful, thanks. I plan to ask my auditors too, once I'm through our current SOC2 audit round.
👍🏼1
Rob G.
·
·
A-LIGN's take on AI code reviews (AI-only vs. human-in-the-loop): Bottom line: They're not seeing fully AI-only code review (no human) accepted as standard. Most clients keep a human at the approval/push-to-prod step, especially for higher-risk changes. Auditors are tool-agnostic — what matters is a controlled, accountable, risk-based process. Core evidence stays the same; AI review just adds AI-governance evidence on top. ISO 27001: Not prescriptive about the tool or the technology. Needs an effective, risk-based change process with clear accountability and oversight. ISO 42001: Human-in-the-loop is tiered by risk — high: human approves before the action; medium: human monitors and intervenes as needed; low: minimal/periodic human involvement, and could make a case no humans are needed for low-risk changes as long as automated process is in line with organizational change management process and controls. AI governance must be integrated into the broader GRC environment. SOC 2: AI in review is accepted, provided control activities exist around the AI tool/agent. Note: AICPA is updating SOC 2 for emerging tech — draft updates expected Q4, clearer requirements by year-end. FedRAMP 20x: Likely fine. CM KSIs already expect automation (persistent, automated testing/validation of changes). AI review with proper guardrails fits 20x's direction. Evidence: Existing artifacts unchanged (PR record, review comments, approval, CI results, branch protection, audit log) — only reviewer identity changes. AI review adds four things: logical access (who can access/modify agents, in the UAR), drift/hallucination monitoring and response, CI/CD integration config, and CI/CD security thresholds and their update process.