Introduction To Meta-Kaggle And Human-AI Teams

Abstract: Meta-Kaggle is a dataset released by Kaggle, an online machine learning competition platform. It includes a range of information on the competitions they have run, including participants, scoreboard results, code written, and more, all compiled into any easy to access SQL database. In this training, participants will be introduced to the dataset, accessing it with SQL commands, and using the dataset to train a model with the Python toolbox SKLearn. I will also describe some research performed as a portion of Project Alloy, in which the dataset was used to predict when code will fail to run.

Project Alloy (funded by DARPA’s Agile Teams program) aimed to develop and implement intelligent machine agents that team with humans (creating hybrid teams) in meaningful and supportive ways. Hosting challenges on a citizen science competition platform provided a unique environment to develop and test hybrid team hypotheses. By dynamically sensing of each team’s state and progress towards a solution, we enabled testing hypothesis formulated about hybrid team management. The combination of team performance monitoring and leaderboard scoring transformed the contest platform into a machine intelligence laboratory. Agents with the ability formulate compelling teams, predict code failures, and provide feedback on model vulnerabilities were created to serve as AI teammates during the competition. By providing machine agents that augment or substitute human roles, we explored a tighter synthesis of human and machine leading to greater resilience and agility under changing project goals and constraints.

Bio: Coming soon!