AI Quality Analyst (Personalization) - Korean

In this role as an AI Quality Analyst (Personalization) for a French multinational IT services and consulting company, you will be responsible for evaluating a new personalization feature for Gemini. Your main tasks will involve assessing how effectively the AI model utilizes information from past Gemini conversations, Gmail, Google Search, and YouTube activity to enhance responses. This role demands a mix of creativity and analytical skills. You will design prompts based on personal experiences, evaluate the quality of personalized responses, and focus on dimensions like Grounding, Integration, and Helpfulness. Key Responsibilities: - Korean Proficiency: Ability to read and write in Korean proficiently as Korean is the primary language for this project. - Schedule Flexibility: Full-time availability in your local time zone required due to the global, 24-hour operations team. - Exceptional Analytical Thinking: Evaluate nuanced AI responses, specifically focusing on personalization quality. - Creative Prompt Engineering: Design creative multi-turn prompts based on personal context to thoroughly test the model's capabilities. - Strong Evaluation Acumen: Understand personalization concepts to identify incorrect personalization, poor inferences, and forced connections. - Meticulous Attention to Detail: Review responses side-by-side to spot subtle differences in naturalness and overnarrating. - Excellent Written Communication: Write clear, concise rationales for model rankings, referencing specific turn numbers. - Feedback: Provide constructive feedback and detailed annotations. - Communication: Excellent collaboration skills. - Independence: Self-motivated and able to work remotely. - Technical Setup: Desktop/Laptop with a good internet connection. In this role, your responsibilities will include designing and executing conversational prompts, evaluating model responses, analyzing for grounding issues, assessing integration quality, stack-ranking model responses, writing clear rationales, extracting debug info, and maintaining data hygiene by deleting evaluation conversations. In this role as an AI Quality Analyst (Personalization) for a French multinational IT services and consulting company, you will be responsible for evaluating a new personalization feature for Gemini. Your main tasks will involve assessing how effectively the AI model utilizes information from past Gemini conversations, Gmail, Google Search, and YouTube activity to enhance responses. This role demands a mix of creativity and analytical skills. You will design prompts based on personal experiences, evaluate the quality of personalized responses, and focus on dimensions like Grounding, Integration, and Helpfulness. Key Responsibilities: - Korean Proficiency: Ability to read and write in Korean proficiently as Korean is the primary language for this project. - Schedule Flexibility: Full-time availability in your local time zone required due to the global, 24-hour operations team. - Exceptional Analytical Thinking: Evaluate nuanced AI responses, specifically focusing on personalization quality. - Creative Prompt Engineering: Design creative multi-turn prompts based on personal context to thoroughly test the model's capabilities. - Strong Evaluation Acumen: Understand personalization concepts to identify incorrect personalization, poor inferences, and forced connections. - Meticulous Attention to Detail: Review responses side-by-side to spot subtle differences in naturalness and overnarrating. - Excellent Written Communication: Write clear, concise rationales for model rankings, referencing specific turn numbers. - Feedback: Provide constructive feedback and detailed annotations. - Communication: Excellent collaboration skills. - Independence: Self-motivated and able to work remotely. - Technical Setup: Desktop/Laptop with a good internet connection. In this role, your responsibilities will include designing and executing conversational prompts, evaluating model responses, analyzing for grounding issues, assessing integration quality, stack-ranking model responses, writing clear rationales, extracting debug info, and maintaining data hygiene by deleting evaluation conversations.