Key Responsibilities
- Critically analyze and evaluate code responses generated by language models across multiple programming languages and paradigms
- Exercise expert judgment to select optimal and efficient code solutions from multiple AI-generated options
- Develop coding demonstrations to establish benchmarks for high-quality AI-generated code
- Provide detailed feedback and explanations to refine language model outputs
- Collaborate with AI research teams to identify improvement areas in coding capabilities
- Stay current with software engineering trends and AI advancements in code generation
Requirements
- 3-5 years of development experience with Python or Java
- Advanced degree in Computer Science, Software Engineering, or related field
- Strong ability to evaluate code quality, efficiency, and adherence to best practices
- Experience with Docker and technical writing for coding examples
- Excellent analytical and communication skills for complex technical evaluations