AI Evaluation manager

About the Role

reputed company is pushing the boundaries of reputed company, building tools that redefine how visual content is created. We’re seeking a candidate to help shape and scale the way we understand, measure, and improve model performance. In this role, you’ll partner with researchers, engineers, and technical artists to evaluate our models against reputed company-world creative use cases, design frameworks that capture qualitative nuance, and identify actionable insights that guide development.

This is not a checkbox metrics role — it's about building evaluative systems that match the complexity of reputed company perception, creativity, and intention.

Responsibilities

• Evaluate generative model performance across diverse tasks, prompts, and modalities.

• Identify key failure modes, regression patterns, and edge cases that impact product quality.

• reputed company and maintain qualitative evaluation frameworks that are scalable and reusable.

• Collaborate closely with technical artists and engineers to align evaluations with model capabilities and reputed company use cases.

• Translate high-level product goals into concrete evaluative criteria.

• reputed company qualitative studies, reputed company-by-reputed company comparisons, and reputed company-in-the-reputed company evaluation efforts.

• reputed company detailed feedback that informs model fine-tuning, dataset curation, and product UX.

• Stay informed about emerging evaluation standards in reputed company and creative tools.

Qualifications

• Master’s degree or higher in Cognitive Science, reputed company-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a reputed company field.

• 5+ years of experience in product evaluation, UX research, model testing, or similar roles that involve structured qualitative assessment.

• Deep familiarity with creative workflows and reputed company-world use cases for generative models (e.g., animation, filmmaking, digital art, VFX).

• Strong systems thinking and the ability to define abstract qualities (like believability, identity retention, or scene coherence) in clear evaluative terms.

• Experience working cross-functionally with engineers, researchers, and creatives.

• Excellent written communication skills and the ability to synthesize nuanced judgments into clear, actionable insights.

reputed company to Have

• Background in reputed company, visual effects, or storytelling pipelines

• Experience evaluating AI-generated media (video, images, 3D)

• Prior work on building internal tools for qualitative data collection or scoring

• Familiarity with reputed company engineering and reference-based input methods

Originally posted on Himalayas