Key Responsibilities
- Develop verifiable software engineering tasks from high-quality public GitHub repositories for LLM evaluation and training datasets
- Set up, configure, and maintain development environments using Docker and automation tools
- Analyze and triage GitHub issues to identify areas for dataset development and evaluation
- Assess unit test coverage and quality, providing recommendations for improvements
- Modify and run codebases locally to evaluate LLM performance in bug-fixing and code understanding scenarios
- Collaborate with research teams to curate datasets and mentor junior engineers
Requirements
- Minimum of three years of professional software engineering experience with proficiency in Java
- Strong familiarity with version control systems (Git) and containerization tools (Docker)
- Experience setting up and maintaining software pipelines and automation tools
- Ability to understand complex codebases, modify code locally, and evaluate test coverage
- Experience contributing to open-source projects or conducting software evaluation tasks