About the Challenge
The innovative use of AI, especially through Large Language Models (LLMs), presents both exciting opportunities and notable challenges. One such challenge is the ability of sophisticated jailbreaking attacks to circumvent existing safety mechanisms, highlighting the need for continued advancements in AI security.
AI Singapore, in collaboration with the CyberSG R&D Programme Office (CRPO), is excited to launch the Global Challenge for Safe and Secure LLMs, an 8-month competition divided into two tracks: Attack and Defense. This challenge aims to advance AI research in developing robust security measures and innovative approaches to mitigate jailbreaking attacks on LLMs. Our goal is to foster safer AI interactions worldwide, particularly in applications with significant societal impact.
Following the successful conclusion of Track 1 of the Challenge, we are thrilled to announce the launch of Track 2, focusing on developing cutting-edge defence mechanisms to fortify LLMs against sophisticated jailbreak attacks.
CHALLENGE TASK
Participants will:
- Develop universal defence methods for three LLMs to enhance their resilience against advanced jailbreak techniques
- Focus on non-tuning-based prompt-level defences that can be applied to any off-the-shelf LLM without modifying its internal parameters
WHO CAN PARTICIPATE
- Researchers and Industry Professionals from around the world
- Individuals interested in AI Security and Safety with experience in Natural Language Processing, Machine Learning and AI Model Robustness
CHALLENGE TIMELINE
• Start of Track 2A: 7 January 2025
• Submission Deadline for Track 2A: 11 February 2025
• Start of Track 2B: 13 February 2025
• Submission Deadline for Track 2B: 25 February 2025
• Notification of Top 5 Teams: By 7 March 2025
The Prize Ceremony is expected to be held between 23 and 28 April 2025
ELIGIBILITY
- Any individual who is at least 18 years of age can form/join a team
- A team can comprise 1 to 6 members
- An individual may not be on more than one team
PRIZES
The top three teams will be declared as the winners of Track 2 and awarded the following cash prizes:
Track 2
- First Place: USD 40,000
- Second Place: USD 20,000
- Third Place: USD 10,000
The top five teams will be provided a stipend of USD 2,000 for one representative per team to attend the prize ceremony.
