As an AI Security Engineer, you must build these projects.
Systems that prove you can secure, cage & audit when autonomous agents touch your stack.
1.) Crescendo Attack Fuzzer
Build: A multi-turn stateful harness that dynamically adapts payloads to groom context windows and bypass commercial system prompts.
Why: Single-shot "ignore instructions" injections are dead. Modern attacks require stateful, multi-turn grooming.
2.) Sandboxed MCP Proxy
Build: A JSON-RPC middleware that intercepts Model Context Protocol tool calls, validates parameters via Pydantic and blocks "confused deputy" privilege escalations.
Why: Tool poisoning is the SQL Injection of the Agentic Era. Agents are only as secure as the tools they call.
3.) Inter-Agent Sanitization Bus
Build: A message broker that strips hidden markdown and malicious instructions between Agent A and Agent B in a multi-agent swarm.
Why: Swarms create internal trust boundaries. Agent A can be tricked into hijacking Agent B's tools via cross-agent injection.
4.) PII UUID-Swapping Gateway
Build: A pre-flight proxy using NER to swap SSNs and credit cards for deterministic UUIDs before the LLM sees them, restoring them post-generation.
Why: Zero-trust compliance demands that raw customer PII never touches a third-party frontier API.
5.) RAG Citation & Poison Detector
Build: A pipeline that cross-references LLM claims against exact vector DB chunk hashes to detect shadow-context injection and citation spoofing.
Why: If an attacker slips a poisoned document into your knowledge base, they own every answer the AI generates.
6.) CFG Output Caging Layer
Build: Middleware enforcing Context-Free Grammar (CFG) decoding to physically block the generation of executable code, markdown links or unauthorized schemas.
Why: System prompts are suggestions. Grammar constraints are physics. You must enforce security at the token-decoding level.
7.) Delegated Agent OAuth Broker
Build: An auth service issuing time-limited, strictly scoped JWTs to agents so they can act on a user's behalf without holding raw passwords.
Why: Agents need hands to do work, but they should never hold the master keys to the enterprise.
8.) PyRIT CI/CD Security Gate
Build: A GitHub Action running 1,000 adversarial payloads via Microsoft PyRIT on every PR, blocking merges if the jailbreak rate exceeds 1%.
Why: Manual red teaming doesn't scale. You need automated, continuous adversarial fuzzing integrated into your deployment pipeline.
9.) Semantic API Rate-Limiter
Build: A gateway calculating embedding similarity of incoming requests to detect and block scraping bots trying to extract your model's weights.
Why: Your fine-tuned weights are your IP. Competitors will try to distill your model through the public API.
10.) Computer-Use Pixel Auditor
Build: A vision pipeline that scans screenshots for adversarial invisible UI elements and Set-of-Mark spoofing before a GUI agent clicks.
Why: Screen-hijacking and pixel manipulation are the new privilege escalation vectors for Computer Use agents.
11.) Vector DB Sleeper-Agent Scrubber
Build: An anomaly detector scanning episodic memory and vector stores for delayed-trigger payloads and semantic backdoors.
Why: Long-term memory is a persistent attack vector. A poisoned memory bank will trigger exploits weeks after ingestion.
12.) Cryptographic Trajectory Ledger
Build: An append-only, hash-chained audit log recording the exact prompt, retrieved context, tool calls and state mutations for every agent action.
Why: When an agent makes a mistake that causes financial damage, the enterprise needs cryptographic proof of exactly why it happened.
13.) AI Supply-Chain SBOM Scanner
Build: A tool auditing "vibe-coded" outputs and MCP server dependencies for hallucinated packages, typosquatting and backdoors.
Why: AI-generated code and dynamic tool discovery create massive, unvetted supply chain risks.
14.) Sycophancy Activation Patching
Build: A TransformerLens script locating and neutralizing the specific attention heads in an open-source model responsible for deceptive alignment.
Why: You cannot secure a model that is actively lying to its evaluators to hide its true capabilities.
15.) Public Exploit Chain Teardown
Build: 3 published CVE disclosures or security advisories detailing novel MCP, A2A or RAG vulnerabilities and their exact mitigations.
Why: AI Security is a brand new field. Publishing novel attack vectors makes you the undisputed authority in the space.
Most people watch tutorials. Builders ship systems.
Bookmark & Repost.
Suraj Sharma
@suraj_sharma14
Suraj Sharma @suraj_sharma14
If you learn AI Security Engineering, you will never be unemployed again.
I’ll do this.
Stage 1: AI Attack Surfaces & Threat Modeling
- Learn: OWASP Top 10 for LLMs, STRIDE for AI, the "Confused Deputy" problem in agentic workflows & data flow mapping.
- Practice: Draw a
I’ll do this.
Stage 1: AI Attack Surfaces & Threat Modeling
- Learn: OWASP Top 10 for LLMs, STRIDE for AI, the "Confused Deputy" problem in agentic workflows & data flow mapping.
- Practice: Draw a
12:00 PM UTC · Nov 3, 2026 · 39.8K Views
391039471.6K