Your day at NTT DATA
We are seeking a talented and experienced Prompt Engineer and Gen-AI Architect /Data Scientist to join our team. The ideal candidate will have a strong background in Python, prompt engineering, and experience working with large language models (LLMs) like GPT3.5, GPT4 and open-source models (Mistral, Phi2, Llama). Additionally, the candidate should have a solid understanding of REST APIs, Microsoft Azure services, vector databases, embedding models, and named entity recognition and relation extraction techniques.
What You'll Be Doing
- Develop and optimize prompts for various natural language processing tasks using few-shot, one-shot, and zero-shot learning techniques.
- Collaborate with cross-functional teams to design and implement data-driven solutions.
- Analyze and preprocess large datasets to extract valuable insights.
- Design and develop RESTful APIs using Python and FastAPI.
- Understanding of Microsoft Azure services such as Azure Functions, Azure Storage, Azure Cosmos DB, and Azure Machine Learning.
- Work with vector databases like ChromaDB and Elasticsearch to power Retrieval Augmented Generation (RAG).
- Utilize embedding models to generate meaningful representations of textual data.
- Implement named entity recognition and relation extraction techniques to extract structured information from unstructured text.
- Continuously research and stay up-to-date with the latest advancements in prompt engineering and natural language processing.
Requirements
- Bachelor's or master's degree in computer science, Data Science, or a related field.
- 5+ years of experience in Python programming, with expertise in FastAPI.
- Strong understanding of prompt engineering techniques and experience working with GPT-3.5 and GPT-4.
- Familiarity with REST API design and development.
- Solid foundation in Natural Language Processing (NLP) concepts, techniques, and algorithms.
- Familiarity with machine learning concepts and algorithms, including supervised learning, unsupervised learning, transfer learning, and model evaluation metrics.
- Proficiency in data preprocessing and cleansing techniques using libraries like pandas and NumPy.
- Knowledge of evaluation and analysis techniques for language models, including metrics such as perplexity, BLEU score, and F1 score.
- Experience with prompt optimization techniques such as prompt tuning, prompt decomposition, and iterative refinement.
- Hands-on experience with Microsoft Azure services, including Azure Functions, Azure Storage, Azure Cosmos DB, and Azure Machine Learning.
- Knowledge of vector databases such as Chroma DB and Elasticsearch.
- Experience working with embedding models and understanding their applications.
- Proficiency in named entity recognition and relation extraction techniques.
- Excellent problem-solving skills and ability to work independently as well as collaboratively in a team environment.
- Strong communication skills and ability to present complex ideas to technical and non-technical audiences.