Shared Task 2017

This year we are running a bigger, better edition of the previous shared task. As before, you will be asked to develop a machine-learned classifier to automatically prioritize posts from an online peer-support forum hosted by ReachOut.com.

ReachOut.com is an online youth mental health service the provides information, tools and support to young people aged 14-25. A key part of this service are the forums, which allow young people to chat anonymously with each other. These forums are carefully moderated by a team of trained professionals and volunteers, who ensure they remain safe, positive and healthy.

This shared task aims to support these moderators by automatically identifying concerning content, so that it can be addressed as quickly as possible. For this task, ReachOut have annotated a corpus of posts with a red/amber/green semaphore that indicates how urgently they require moderator attention. Systems will need to leverage the content of posts, including sentiment, topics, thread context and user history to classify posts as red, amber or green. This year will use a larger, more up-to-date dataset with additional annotated and unannotated posts.

This task motivates research in these specific areas, as well as the broader topic of classifying mental health topics in social media. In addition, ReachOut itself will be able to leverage techniques that they can integrate back into their systems.

If you are interested in participating, please complete the application form here.

This year’s CLPsych Shared Task is again organized by David Milne at the University of Sydney. New this year, we will not require a full system write-up, allowing the task timeline to extend into the summer and task participants to pursue alternative venues for a write-up post-workshop. At the workshop, we plan to hold an open discussion of system trials and successes.

Timeline

18 April – Shared task registration opens; complete application form here.
5 May – Shared task registration closes, training data shared
16 June – Unlabelled testing data shared
23 June – Run submissions due
30 June – Results announced
3 August – CLPsych Workshop

Join the conversation at the CLPsych Google group.