Back
Trust & Safety Evaluator with English from India
Full-time
TELUS Digital is a customer experience and technology company. Note: only candidates currently residing in Finland with a valid work permit will be considered.
About the Role
Trust & Safety Evaluators conduct adversarial testing and safety evaluation of generative AI features. Main tasks are crafting queries, evaluating the safety of generated content and providing critical feedback. This role requires creative thinking about potential misuse, deep cultural and linguistic knowledge, and the ability to identify subtle safety risks.
Key Responsibilities
Write, review and evaluate diverse and challenging queries designed to test the system's limits and expose problematic outputs, targeting specific risk topics.
Design and execute sequences of queries simulating realistic, unfolding conversations.
Craft attack scenarios using techniques like crescendo attacks and context manipulation.
Assign risk ratings to AI-generated content based on safety guidelines.
Detect and articulate potential biases and identify subtle unsafe elements, inconsistencies or problematic implications.
Evaluate cultural and language knowledge across different cultural contexts, maintaining high accuracy under time constraints.
Required Skills
English from India full professional language proficiency and a valid work permit for Finland are mandatory.
Deep cultural awareness of English from India: context, social norms and values, and cultural references.
Query crafting and scenario design, including crescendo attacks and context manipulation.
Risk rating assignment, bias detection in AI outputs, and strong attention to detail while working fast.
Preferred
Background in Creative Writing, Social Media/Community Management, Content Moderation, Digital Marketing, Communications or Journalism.
Prior Trust & Safety experience, familiarity with machine learning concepts, and fluency in additional languages.
Content warning: this role involves exposure to sensitive and potentially disturbing content as part of adversarial testing and safety evaluation.
