Senior AI Governance & QA Auditor

As an AI Governance and QA Auditor at UPS, your role will involve evaluating, auditing, and ensuring the responsible, ethical, and compliant use of Artificial Intelligence and Machine Learning systems. You will play a crucial part in maintaining trust, fairness, transparency, and regulatory alignment across AI/ML initiatives while driving continuous improvement in model development, deployment, and monitoring practices. Collaboration with data scientists, engineers, risk/compliance teams, and business stakeholders to establish and enforce AI governance frameworks will be a key aspect of your responsibilities, leveraging advanced tools like IBM Watsonx Governance. **Key Responsibilities:** - **AI Assurance & Quality Auditing** - Design framework for technical auditing of AI/ML systems that avoid costly mistakes. - Conduct independent audits of various AI/ML systems covering areas like Gen AI, Deep Learning, Machine Learning, Natural Language Processing, and Computer Vision. - Verify compliance with benchmark standards, external regulations, and ethical AI principles. - Assess data quality, feature engineering practices, and model development pipelines for reproducibility, fairness, and bias mitigation. - Evaluate ML model performance, robustness, explainability, and alignment with intended business outcomes. - Review deployment processes, model versioning, and monitoring practices for traceability and accountability, utilizing Watsonx Governance and other enterprise AI governance tools. - **AI Governance & Risk Management** - Define, implement, and maintain AI governance frameworks, policies, and best practices. - Identify AI risks (bias, drift, adversarial attacks, misuse) and recommend mitigation strategies. - Ensure compliance with AI regulations and standards such as EU AI Act, NIST AI RMF, ISO 42001, GDPR, and industry-specific frameworks. - Establish governance checkpoints across the AI lifecycle and leverage Watsonx Governance for automated policy enforcement, lineage tracking, and audit readiness. - **Compliance & Ethical AI** - Audit AI/ML systems for fairness, inclusivity, accountability, and transparency. - Maintain model documentation, datasheets, and audit trails consistently. - Provide assurance reports for regulators, clients, and internal executives. - Stay updated on emerging AI regulations and adjust governance policies accordingly. - **Collaboration & Advisory** - Partner with Data Science, MLOps, and Product teams to embed assurance-by-design practices. - Educate stakeholders on AI risks, governance policies, and responsible AI principles. - Act as an internal advisor for audits, compliance reviews, and external certifications. **Qualifications & Skills:** - **Education** - Bachelors degree in Computer Science, Data Science, Statistics, Engineering, or a related quantitative field. - Masters/PhD or certification in AI Governance, Risk & Compliance, or Data Ethics is preferred. - **Required Experience** - 6+ years of overall professional experience, with at least 3 years in AI/ML model development, assurance, risk management, or audit functions. - Hands-on expertise with Data Science workflows, ML model evaluation, and MLOps pipelines. - Previous experience in regulatory compliance, IT audit, or QA within financial services, healthcare, or regulated industries is a plus. - **Technical Skills** - Strong understanding of machine learning, deep learning, and generative AI. - Familiarity with AI auditing tools like Fairlearn, Aequitas, SHAP, LIME, Explainable AI frameworks. - Proficiency in Python, SQL, and model evaluation libraries. - Knowledge of cloud platforms (AWS, Azure, GCP) and enterprise governance solutions such as IBM Watsonx Governance, Vertex AI Model Monitoring, and Watson OpenScale. - **Governance & Regulatory Skills** - Knowledge of AI risk management frameworks (NIST AI RMF, ISO 42001, OECD AI Principles). - Understanding of data protection and privacy laws (GDPR, HIPAA, CCPA). - Ability to translate technical findings into executive-level risk and compliance reports. - **Soft Skills** - Strong analytical and critical thinking skills with attention to detail. - Excellent communication and stakeholder management abilities. - Capacity to act as an independent, objective voice in AI governance matters. As an AI Governance and QA Auditor at UPS, your role will involve evaluating, auditing, and ensuring the responsible, ethical, and compliant use of Artificial Intelligence and Machine Learning systems. You will play a crucial part in maintaining trust, fairness, transparency, and regulatory alignment across AI/ML initiatives while driving continuous improvement in model development, deployment, and monitoring practices. Collaboration with data scientists, engineers, risk/compliance teams, and business stakeholders to establish and enforce AI governance frameworks will be a key aspect of your responsibilitie