AI Assurance & Governance
About the Role
About the company:
devx labs is an AI consulting firm that ships code. We help enterprises in India, Southeast Asia and the US design, build and run AI systems that move a business metric. Our clients work in retail, telecom, BFSI, the public sector, manufacturing, media and hospitality.
We organise our capabilities into five practices, each owning one layer of a client's AI stack: Platform Engineering, Data & Decision Intelligence, Agents & Enterprise Autonomy, AI-Native Experiences, and AI Assurance & Governance.
We partner with OEMs and AI labs to bring the most complex problems in enterprise AI to devx.
About the role:
AI Assurance & Governance owns whether AI can be trusted. We prove that AI is correct, safe and compliant before it ships, and keep it that way as models, data and rules change.
The practice covers:
- Evals and benchmarking: golden sets, model judges calibrated against people, and release gates
- Red teaming and AI security, including prompt injection and agent tool security
- Agent identity, least-privilege access and audit-grade observability
- Model migration playbooks for every model swap
- AI management systems such as ISO/IEC 42001, and the mapping of regulation to controls and evidence
The cloud security baseline belongs to Platform Engineering, and building the agents to Agents & Enterprise Autonomy. Keeping assurance separate means no team grades its own work.
Responsibilities
Shape and win the work
- Run discovery with client CISOs, CTOs, and risk and compliance teams, and take stock of every AI system and agent in use.
- Assess AI risk: threat models, impact assessments and the regulations that apply in each market.
- Design the control approach for each deal, covering evals, guardrails, identity and release gates, and write the assurance sections of proposals and SOWs.
- Shape red-team exercises and eval pilots that show clients where they stand.
Hand over and advise
- Hand the control design to the delivery team with clear eval targets, release gates and a decision log.
- Advise the delivery team on evals, red teaming and agent security when they need it.
- Review model migrations and releases against the agreed gates.
Drive go-to-market with partners
- Co-sell with partner teams at OEMs and AI labs, build joint sales plays, and help clients
- qualify for partner funding.
Build the practice
- Turn what each deal teaches into reusable assets: eval suites by industry, threat-model templates, regulatory control maps and ISO/IEC 42001 readiness kits.
- Write points of view, speak at events, and help others in the practice grow.
Scope scales with seniority, from solutioning parts of a deal to owning client assurance programmes, partner relationships and the practice roadmap.
Requirements
Evals and testing
- Designing eval suites from real tasks: golden sets, trajectory checks for agents, and pass rates across repeated runs
- Calibrating model judges against human labels, and knowing where they are biased
- Eval tools such as Google's agent evaluation, Inspect, promptfoo, DeepEval, Ragas and Langfuse
AI security and red teaming
- OWASP Top 10 for LLM Applications and for Agentic Applications, and MITRE ATLAS
- Direct and indirect prompt injection, and defensive designs such as CaMeL and plan- then-execute
- Red-team tools such as PyRIT, garak and promptfoo, and measuring attack success rate
- Guardrails such as Model Armor, NeMo Guardrails and Llama Guard
- MCP security: tool poisoning, OAuth 2.1 authorisation and server approval
Identity, observability and change
- Agent identity and least privilege with Google Agent Identity or Microsoft Entra Agent ID
- Audit-grade tracing with OpenTelemetry, and groundedness checks
- Model migration through side-by-side regression and canary releases
Governance and regulation
- ISO/IEC 42001 and 42005, the NIST AI RMF and the CSA AI Controls Matrix
- Regulation across our markets: India's DPDP Act and RBI guidance, Singapore's agentic AI framework and MAS guidelines, Vietnam's AI law, the EU AI Act and US state laws
Good to have: ISO/IEC 42001 lead implementer or lead auditor training; securitycertifications such as CISSP or CCSP; a Google Cloud security certification.
Benefits & Perks
- AI tool credits and freedom to experiment with modern approaches.
- Performance-based appraisal at 6 months + annual appraisal cycle.
- Competitive pay with a structured CTC breakup.
- Work with clients across industries and domains.
- Hybrid model with work-from-home support and flexible working hours.
- No-cap leave policy — take time off responsibly, no fixed limits.
- Health insurance with spouse and kids coverage included.
- Yearly outings, and an open environment for innovation.
Reimagining customer experience, from first principles.
devx labs is an AI-native consulting firm that reimagines business operations around AI capabilities to deliver customer experience transformation for forward-thinking brands. We combine the unbiased perspective of young, AI-native talent with the seasoned judgment of experienced practitioners — solving for outcomes, not shipping deliverables for their own sake.
Reimagining every customer touchpoint across the entire journey.
Optimizing downstream operations to enhance customer experience delivery.
The foundation that makes AI-native experiences possible.