Senior Architect, Quality Engineering

As an experienced QA AI Architect, your role will involve designing and leading the architecture of AI-driven Quality Engineering systems for next-generation AI-powered products. You will be responsible for building scalable frameworks for testing and validating AI systems, focusing on Large Language Models (LLMs), embeddings, vector databases, and modern AI pipelines. Your work will include collaborating with product, engineering, and AI teams to design intelligent QA solutions. Your passion for redefining quality engineering for AI products will play a key role in this position. **Responsibilities:** - **AI Architecture & LLM Systems:** - Architect and build AI-powered quality engineering platforms using LLMs and SLMs - Evaluate and integrate open-source models such as Llama and Mistral - Design engineering workflows and Retrieval-Augmented Generation (RAG) pipelines - Architect systems for LLM evaluation, guardrails, and validation frameworks - Optimize architectures for token efficiency, cost, and performance - **Embeddings & Vector Search:** - Design and implement embedding pipelines for semantic retrieval - Implement vectorization strategies for documents, test artifacts, and product knowledge bases - Architect vector storage using technologies such as: - Pinecone - Weaviate - Milvus - FAISS - Optimize chunking and indexing strategies for large knowledge repositories - **Model Optimization & Fine-Tuning:** - Implement model fine-tuning pipelines using LoRA / PEFT techniques - Evaluate model performance, reliability, and hallucination mitigation - Manage training datasets and model evaluation metrics - **Backend & System Architecture:** - Design microservice-based AI system architectures - Build scalable backend services and APIs for model inference and evaluation - Develop data pipelines supporting AI workflows and large-scale inference systems - **MLOps & AI Lifecycle:** - Build MLOps pipelines for model deployment, monitoring, and retraining - Implement CI/CD pipelines for AI systems - Enable model observability, performance monitoring, and version control - **OCR & Document Intelligence:** - Design systems for OCR-based document processing - Integrate OCR pipelines with embedding and AI workflows - Build solutions for document understanding and QA automation **Required Skills:** - **AI & LLM Systems:** - Strong understanding of LLM and SLM architectures - Experience working with open-source LLMs - Prompt engineering and RAG architectures - Model evaluation frameworks and hallucination control - Token optimization strategies - **Embeddings & Vector Databases:** - Experience working with embedding models and semantic search - Hands-on experience with vector databases such as: - FAISS - Pinecone - Weaviate - Milvus - Infuse AI with Automated Test Failure analysis - Crawl Product/Test code to figure out whether issue is probable Product/Test/Env issue - **Programming & AI Frameworks:** - Strong Python programming skills - Experience with AI frameworks such as: - LangChain - Llama - Index - Backend development using FastAPI / Flask / Node.js - **Backend & Cloud Architecture:** - Microservices architecture and REST APIs - Containerization using Docker - Experience working on AWS / Azure / GCP - **MLOps:** - Model deployment pipelines - CI/CD for machine learning systems - Monitoring and observability of AI models - **OCR & Document Processing:** - Experience with OCR frameworks such as: - Tesseract OCR - PaddleOCR - **System Foundations:** - Strong understanding of system design, operating systems, networking, and distributed systems **Qualifications:** **Nice to Have:** - Experience building AI agents or autonomous systems - Architect testing frameworks for agentic AI workflows and multi-agent orchestration - Knowledge of LLM evaluation frameworks - Understanding of AI safety and guardrails - Experience architecting AI-powered SaaS products As an experienced QA AI Architect, your role will involve designing and leading the architecture of AI-driven Quality Engineering systems for next-generation AI-powered products. You will be responsible for building scalable frameworks for testing and validating AI systems, focusing on Large Language Models (LLMs), embeddings, vector databases, and modern AI pipelines. Your work will include collaborating with product, engineering, and AI teams to design intelligent QA solutions. Your passion for redefining quality engineering for AI products will play a key role in this position. **Responsibilities:** - **AI Architecture & LLM Systems:** - Architect and build AI-powered quality engineering platforms using LLMs and SLMs - Evaluate and integrate open-source models such as Llama and Mistral - Design engineering workflows and Retrieval-Augmented Generation (RAG) pipelines - Architect systems for LLM evaluation,