Terms and Condition

By participating in this task you agree to these terms and conditions. If, however, one or more of this conditions is a concern for you, send us an email and we will consider if an exception can be made.

By submitting results to this competition, you consent to the public release of your scores at this website, at the WASSA 2018 website, the Codalab website and in the associated proceedings, at the task organizers’ discretion. Scores may include, but are not limited to, automatic and manual quantitative judgements, qualitative judgements, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers. You further agree that the task organizers are under no obligation to release scores and that scores may be withheld if it is the task organizers’ judgement that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition’s rules. Inclusion of a submission’s scores is not an endorsement of a team or individual’s submission, system, or science. A participant can be involved in exactly one team (no more). If there are reasons why it makes sense for you to be on more than one team, then email us before the evaluation period begins. In special circumstances this may be allowed.

Each team must create and use exactly one CodaLab account.

Team constitution (members of a team) cannot be changed after the evaluation period has begun. No participant can be part of more than one team.

During the evaluation period:

Each team can submit as many as fifty submissions. However, only the final submission will be considered as the official submission to the competition.

You will not be able to see results of your submission on the test set.

You will be able to see any warnings and errors for each of your submission.

Leaderboard is disabled.

Once the competition is over, we will release the gold labels and you will be able to determine results on various system variants you may have developed. We encourage you to report results on all of your systems (or system variants) in the system-description paper. However, we will ask you to clearly indicate the result of your official submission.

We will make the final submissions of the teams public at some point after the evaluation period.

The organizers and their affiliated institutions makes no warranties regarding the datasets provided, including but not limited to being correct or complete. They cannot be held liable for providing access to the datasets or the usage of the datasets.

The dataset should only be used for scientific or research purposes. Any other use is explicitly prohibited.

The datasets must not be redistributed or shared in part or full with any third party. Redirect interested parties to this website.

If you use any of the datasets provided in the shared task, you agree to cite the associated paper. Information will be provided later.