Developers/Programmers (Information & Communication Technology)This job is posted on behalf of Resaro.ai, a startup supported by SGInnovate.BackgroundAs AI adoption increases, the challenge in the next decade is to harness AI safety with the appropriate levels of governance and assurance to build trust in these advanced algorithmic systems. Most enterprises do not have the capability to do this and we are a new AI assurance venture that provides services to validate AI systems for accuracy, robustness, explainability, fairness, privacy and security. We are looking for a senior data scientist with experience in deep learning based language models to work with a category-defining AI assurance venture that will help companies test and audit their AI systems. You will help evaluate and stress-test AI models to make sure they are fit for purpose and safe to be deployed. We value strong technical ability and real world experience and there will be room to solve challenging problems and adopt cutting edge technology into business applications.Job scopeYOU WILLPerform technical AI evaluations, benchmarking and "red-team" tests on large language models, including assessing them for robustness in performance, embedded biases, vulnerability to jailbreaks and prompt injection attacks.Work with the product management team to develop a suite of technical and analytical AI evaluation frameworks and tools that are backed by scientific research and methods. These should assess the robustness, explainability, fairness, privacy, safety and security of AI and machine learning systems, with a strong focus on large language models.YOU ARE ABLE TOThink from first principles and want to tackle the most challenging technical problems from a multi-disciplinary approach e.g., design, engineering and social science.Lead by example regardless of whether you are a manager or individual contributor. You want to work with passionate and talented individuals and people want to work with you.Communicate in an open, frank and respectful manner.Thrive in a fast-paced environment.Navigate uncertainty by being willing to explore, while remaining laser focused on the mission at hand.YOU HAVEExtensive experience as a data scientist training or deploying deep learning based natural language models/large language models in real-world contexts. About 5-6 years of working experience or a relevant postgraduate degree with 1-2 years of working experience.Familiarity with NLP evaluation approaches, preferably with a deep understanding of the failure modes of LLMs including robustness, jailbreaks, embedded biases and possible alignment problems.Ability to be independent productive working with Python-based notebooks and to execute on basic data engineering and visualisation tasks (eg. SQL, using StreamLit to build visualisations).Passion and interest in applied research on the safe and responsible use of AI and with large language models.About usResaro is a leading AI Assurance company, pioneering the field of artificial intelligence safety and reliability. We are a team of AI experts, engineers, and data scientists. Our mission is to ensure that AI systems are developed and deployed in a way that is safe, secure, and aligned with human values. The future of society, business and the world depends on responsible AI adoption.Interested candidates may apply directly on DTC at: https://central.sginnovate.com/hub/marketplace/openings/fceefd4d-787c-4e62-a16a-922dc22ec1d1Your application will include the following questions:Which of the following statements best describes your right to work in Singapore?What's your expected monthly basic salary?Consulting ServicesNot specifiedAt SGInnovate, we're looking for entrepreneurial scientists who have the ambition, passion and expertise to develop Deep Tech solutions to challenging global problems.We build investable companies. We invest in promising Deep Tech startups. We help them knock down barriers — technical, adoption, and market — like no other. We connect them with top talent and bring together great minds across the ecosystem to discuss, share and inspire ideas of innovation.At SGInnovate, we're looking for entrepreneurial scientists who have the ambition, passion and expertise to develop Deep Tech solutions to challenging global problems.We build investable companies. We invest in promising Deep Tech startups. We help them knock down barriers — technical, adoption, and market — like no other. We connect them with top talent and bring together great minds across the ecosystem to discuss, share and inspire ideas of innovation.Don't provide your bank or credit card details when applying for jobs.Researching careers? Find all the information and tips you need on career advice.#J-18808-Ljbffr
