It’s one thing to be embarrassed by the first version of your product capabilities and quite another to ship something for early feedback only to realize you embarrassed your company, your now lost customer, and your own reputation. A Product Engineer is a Software Engineer who also takes on Product Owner/Product Manager responsibilities. Product Engineers are being asked to do more tasks in less time, so turn to using tools like GenAI to vibe code the way to an MVP.
“If you are not embarrassed by the first version of your product, you’ve launched too late.” –Reid Hoffman
LLM coding assistants like Replit, Cursor, and more can be used to crank out code faster than any security expert can keep up. Automated application security scans while developing are the solution to this problem.
Data and Model Poisoning of LLM-Generated Code
Let’s state the problem.
Most software engineers familiar with secure coding are aware of cross-site scripting (XSS) and cross-site request forgery (CSRF) attacks. Even many LLMs will get these right because they are common enough, but large language models (LLMs) may also innocently use incorrectly implemented code when training or more sinisterly, be subject to data poisoning.
Training Data Poisoning refers to intentionally manipulating training data to introduce vulnerabilities and exploitable patterns that LLMs will consume. It's a constantly moving problem, unfortunately, and LLMs will continue to evolve as do the attacks and newly discovered vulnerabilities.
How do we move forward with secure-by-default approaches to code generated by AI?
Semgrep Secure Scanning with Replit
Semgrep is a fast static analysis tool to identify security vulnerabilities in source code before shipping in production. Replit recently introduced a Security Scanner to analyze dependencies and source-code for vulnerabilities as a step before deployment. The findings from Semgrep are used by the Replit Agent to fix the vulnerabilities right away.
- SQL Injection
- Insecure Dependencies
- Cross-site Scripting (XSS)
- Hard-coded Credentials (Secrets)
- and more…
For these reasons, Replit has been referred to as The Safest Place for Vibe Coding. Their approach to beefing up security with App History, Replit Auth, Rollback previews, and of course Semgrep Secure Scans is a sensible approach for rapid development and deployment.
Secure scanning is available on certain plans using your Replit account.
Semgrep Agentic AI Approach with a Cursor MCP Server
Taking a similar approach, Cursor can support a Model Context Protocol (MCP) server that plugs in to the development environment. Then, any generated code can be combined with a Semgrep scan to identify vulnerabilities and then immediately acted upon. In this way, the IDE acts like an agent, running the Semgrep source code scan on its own generated results and then applying fixes.
Source code for the server is available from the Github repo semgrep/mcp after you sign up for a Semgrep account.
Shift-Left for Product Engineers
Product Managers must understand all the business requirements, both functional and non-functional like security. They also must prove their ideas quickly and iteratively, so low-code generative solutions are an effective strategy. It took the industry a long time to recognize and begin shifting security concerns left to put up some guardrails.
LLMs are accelerating development and evolving quickly. Our approach to secure-by-default artificial intelligent agents should be to always scan and then act on the findings. This is the workflow we need to protect us from exploitation and put in place safeguards while still innovating on product.
For more information, check out some of these resources.
From idea to (secure) app: Semgrep + Replit
Security checklist for vibe coding
Safe Vibe Coding on Replit (video demo)
Semgrep maintains the open-source static application security testing (SAST) engine and continuously improves upon our professional security rules to keep pace with the latest security concerns with Artificial Intelligence.