Key Responsibilities
- Annotate and evaluate responses from large language models for alignment and reasoning quality
- Develop annotation guidelines for reinforcement learning from human feedback (RLHF)
- Analyze model outputs for coherence, factual accuracy, and ethical considerations
- Collaborate with research teams to improve model performance through feedback loops
- Document annotation processes and maintain quality standards
- Contribute to datasets used for fine-tuning AI models
Requirements
- 3+ years of experience in NLP, AI, or related fields
- Familiarity with LLMs and reinforcement learning concepts
- Strong analytical and critical thinking skills
- Experience with data annotation or evaluation frameworks
- Proficiency in Python and relevant NLP libraries