From: Christopher Potts
Date: August 4, 2011 10:58:52 MDT
To: Class list
Subject: Computational Pragmatics: the Slugs-Wugs model and a reminder

Computational Pragmaticists!

Quick reminder: final projects are due on Sunday, August 7. No extensions; I need to file grades on August 8.

Here are the results of the competition to predict Implicit coherence relations in the Penn Discourse Treebank 2:

http://compprag.christopherpotts.net/pdtb-competition.html

Quick summary: the Slugs model did slightly better in terms of overall accuracy, but the Wugs model has a lot to recommend it:

http://compprag.christopherpotts.net/pdtb-competition.html#winner

The fact that there was no objective winner led to a lot of interesting discussion about linguistics and NLP, our goals in building models like this, and different modes of explanation.

There was also some interest in seeing how a joint Slugs-Wugs model would do. I've added those results to the above page:

http://compprag.christopherpotts.net/pdtb-competition.html#compprag

I expected a big performance jump, since the individual models are both effective and implement quite different ideas, but the results were mediocre. Sigh. Do check out this exercise for data that might help with building better models:

http://compprag.christopherpotts.net/pdtb-competition.html#GLM

Many thanks to all of you for your contributions to the course! I thoroughly enjoyed our meetings, and I learned a lot!

---Chris