Ai Red Teaming Expert

Build and scale the "Safety & Trust" engine for our agentic AI ecosystem. You will be the technical lead responsible for ensuring our AWS Bedrock-based agents meet the highest standards of FCA Operational Resilience , DORA , and EU AI Act (Art. 15) compliance.

Core Responsibilities

- Automated Red Teaming: Implement adversarial testing (Garak, Pyrit, AgentDojo) directly into CI/CD pipelines with automated release gating.

- Centralised Eval Platform: Operate a firm-wide service to measure success rates, uncertainty, hallucination, and bias across all non-deterministic systems.

- Secure Architecture: Map OWASP LLM Top 10 and agentic threats to technical controls; manage AWS Bedrock Guardrails and Knowledge Bases.

- AI Supply Chain: Own the AI-BOM , ensuring supply chain integrity, signed artifacts, and drift monitoring.

- Regulatory Evidence: Produce the technical documentation and robust testing evidence required for EU AI Act Article 15.

Technical Requirements

- AWS Bedrock Expert: Hands-on experience with Bedrock Agents, Knowledge Bases, and model lifecycle management.

- AI/ML Depth: Strong grasp of FMs, RAG, tool-use, and the failure modes of agentic workflows.

- Security & Compliance: Deep knowledge of NIST AI RMF, OWASP LLM Top 10, and UK/EU financial regulations (FCA/DORA).

- Testing Automation: Proven ability to build measurement frameworks for drift, memorization, and adversarial robustness.

Qualifications

- Significant experience in UK Financial Services .

- Expertise in automated adversarial testing and evaluation at scale.

- Ability to bridge the gap between complex AI engineering and rigid regulatory requirements.

Randstad Technologies is acting as an Employment Business in relation to this vacancy.