Challenges can be seen as a type of game that motivates participants to solve
serious tasks. As a result, competition organizers must develop effective game
rules. However, these rules have multiple objectives beyond making the game
enjoyable for participants. These objectives may include solving real-world
problems, advancing scientific or technical areas, making scientific
discoveries, and educating the public. In many ways, creating a challenge is
similar to launching a product. It requires the same level of excitement and
rigorous testing, and the goal is to attract ”customers” in the form of
participants. The process begins with a solid plan, such as a competition
proposal that will eventually be submitted to an international conference and
subjected to peer review. Although peer review does not guarantee quality, it
does force organizers to consider the impact of their challenge, identify
potential oversights, and generally improve its quality. This chapter provides
guidelines for creating a strong plan for a challenge. The material draws on
the preparation guidelines from organizations such as Kaggle 1 , ChaLearn 2 and
Tailor 3 , as well as the NeurIPS proposal template, which some of the authors
contributed to.
Expert Commentary: The Multi-Disciplinary Nature of Challenge Creation
Creating a challenge is a multi-disciplinary endeavor that requires a diverse set of skills and knowledge. This article highlights the various objectives that competition organizers must consider when developing effective game rules.
One key objective is to solve real-world problems. Challenges often focus on tackling important issues in fields such as healthcare, climate change, or social justice. By framing these problems as games, organizers can motivate participants to come up with innovative solutions. This multi-disciplinary approach encourages collaboration between experts from different domains, such as scientists, engineers, and policymakers, fostering diverse perspectives and ultimately leading to more comprehensive solutions.
Advancing scientific or technical areas is another crucial objective. Challenges can serve as a platform for pushing the boundaries of knowledge and technology. By defining specific tasks or problems to be solved, organizers can drive research and development in niche areas. This interdisciplinary nature encourages participants from various fields, including academia and industry, to collaborate and contribute their expertise.
Additionally, challenges have the potential to make scientific discoveries. By harnessing the collective intelligence and problem-solving capabilities of participants, organizers can uncover new insights and breakthroughs. The competitive nature of challenges facilitates the exploration of uncharted territories and the discovery of unexpected solutions. This dynamic process can lead to significant advancements in science and technology.
Education is another important objective of challenge creation. Challenges provide a learning opportunity for participants, allowing them to acquire new skills and expand their knowledge in specific domains. By participating in challenges, individuals can gain hands-on experience, learn from mentors, and engage in peer-to-peer learning. This multi-disciplinary approach encourages lifelong learning and skill development.
Creating a challenge is like launching a product. Just as a product needs to be exciting and appealing to customers, a challenge must capture the attention and interest of potential participants. Organizers should carefully plan and design their challenge to attract a diverse range of participants from different backgrounds and expertise.
The process of creating a challenge starts with developing a solid plan, similar to a business proposal. This plan outlines the problem to be solved, the objectives of the challenge, the rules and guidelines, and the evaluation criteria. By submitting the proposal to renowned conferences and undergoing peer review, organizers ensure they receive expert feedback and improve the quality of their challenge.
While peer review does not guarantee quality, it is an essential step in critically assessing the impact and feasibility of the challenge. It helps organizers identify potential oversights, refine their objectives, and ensure that the challenge is aligned with the latest research and industry trends. This multi-disciplinary approach ensures that challenges meet high standards of quality and relevance.
In conclusion, challenge creation is a multi-disciplinary endeavor with objectives that go beyond mere entertainment. Challenges have the potential to solve real-world problems, advance scientific and technical areas, make scientific discoveries, and educate the public. By embracing this multi-disciplinary nature, organizers can create impactful challenges that harness the collective intelligence of participants from diverse backgrounds and drive innovation.