UKP in Challenges

UKP has participated in a number of research challenges and competitions.  Please follow the links below for detailed descriptions of the tasks and our results.

GermEval-2017 : Shared Task on Aspect-based Sentiment in Social Media Customer Feedback

UKP participated in the GermEval-2017 : Shared Task on Aspect-based Sentiment in Social Media Customer Feedback. We participated in all four subtasks, namely relevance classification, document-level sentiment classification, aspect-category and sentiment detection, and aspect target extraction. The provided data contains customer feedback about Deutsche Bahn AG and was crawled from various web-sources.

We used sentence embeddings and an ensemble of classifiers for two sub-tasks as well as state-of-the-art sequence taggers for two other sub-tasks. First is a modified version of the stacked learner which has shown good performance for the SemEval 2017 Task 10, and second a BiLSTM-CRF using word- and character-level embeddings. Our systems achieved top ranks in two subtasks and placed mid-rank in the other two subtasks.

Further information on the task and the original dataset can be found on the competition website.


You can find the system description here:
undefinedUKP TU-DA at GermEval 2017: Deep Learning for Aspect Based Sentiment Detection

The source code and generated training data can be downloaded from here:

https://github.com/UKPLab/germeval2017-sentiment-detection

Fake News Challenge Stage 1: Exploring how artificial intelligence technologies could be leveraged to combat fake news

A UKP/AIPHES team participated in the Fake News Challenge Stage 1.

The goal of the Fake News Challenge, in general, is to explore, how approaches based on machine learning and natural language processing, can be leveraged to address the fake news problem. Stage 1 of the Fake News Challenge (FNC-1) was focusing on the task of Stance Detection. Stance Detection is the process of estimating the relative perspective (or stance) of a text with respect to a topic, claim or issue. The version of Stance Detection, which was selected for FNC-1, is based on the work of Ferreira and Vlachos.

For FNC-1, estimating the stance of a body text from a news article relative to a headline was chosen as the task. The body text either agrees, disagrees, discusses or is unrelated to the statement represented by the headline. In the competition, the teams were given a labeled data set for system development.
At the end of the competition, an unseen set of test data was provided for system evaluation.
Out of 50 participating teams, UKP/AIPHES came in second with the score of 81.97%.

For more details please refer to the original Fake News Challenge website.

EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION

UKP participated in the SemEval-2017 Task 10: Extracting Keyphrases and Relations from Scientific Publications. In particular, we participated in Task (B): "Classification of identified keyphrases". The goal of this subtask was to classify identified keyphrases from scientific publications into one of the three classes "Material", "Task", and "Process".

Our system is an ensemble of neural techniques: an attention-based Bi-LSTM model, a character-level convolutional neural net and a stacked learner with an MLP meta-classifier. Our system had a micro-F1-score of 0.63 and ranked second out of 5 systems, closely behind the first system with an F1-score of 0.64. Erroneously, we only used about 15% of the available training data. With the full training data our system has an F1-score of 0.69. More information can be found here.

LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test

UKP participated in the Story Cloze Test challenge at the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem 2017). The goal of this competition was to provide a common ground for the evaluation of systems on language understanding. Given four sentences of a story on everyday life events, a system had to identify the correct ending from a set of two predefined ending sentences.

Our system is based on a deep learning architecture combined with a rich set of manually-crafted linguistic features. The system outperformed all known baselines for the task, achieving an accuracy of 71.7 %. The system was placed fourth among 8 systems. More information can be found here.

Event Nugget Detection at TAC 2015

UKP participated in the shared task of Event Nugget Detection at the Text Analysis Conference (TAC 2015). The goal of this shared task was to identify the explicit mentioning of events and to classify the event type. Our system is based on state-of-the-art deep neural networks and uses only minimal preprocessing. The system achieves an F1-measure of 65.31% and is placed first among 14 systems. More information can be found here.

Named Entity Recognition in German at KONVENS 2014

UKP participated in the shared task of GermEval at KONVENS 2014. In this task, systems were expected to automatically annotate nested named entities in German. For the shared task, a new dataset with sentences from German news articles and Wikipedia was created. Our system is based on state-of-the-art deep neural networks combined with specifically designed features. The system achieves an F1-measure of 75.1% and is  placed 2nd among 11 systems. You can read further details on the task and about our system.

Question Answering for University Entrance Exams at CLEF 2014

UKP participated in the university entrance exam task of the CLEF question answering track. In this task, systems were expected to automatically solve open-domain, multiple choice reading comprehension questions about short Englis narrative texts. Our system achieved the second best performance across all competing system in the challenge with a c@1 score of 0.375. More information about our system can be found here.

Sentiment Analysis in Twitter at SemEval 2014

UKP participated in the Sentiment Analysis in Twitter task for SemEval 2014, collocated with COLING conference on 23-24 August, 2014 in Dublin, Ireland. According to the official results,our system (ukp-dipf) achieved the 8th place on tweet level (subtask B) in terms of macro-averaged F-score among 50 systems. On the expression level (subtask A), our system ranked 14 out of 27 systems, based on macro-averaged F-score. For more details please refer to the dedicated page.

Uncovering Plagiarism, Authorship and Social Software Misuse 2013

UKP participated in the first edition of the Author Profiling Task in the PAN Lab at CLEF 2013. The author profiling task aims at revealing certain categorical information about the author, such as his/her age or gender, rather than reveal his/her exact identity. Our system placed 4th among 20 systems in Spanish and 15th among 22 systems in English in terms of accuracy. You can read further details on the task and about our system.

CrossLink2 at NTCIR-10

UKP participated in the  Cross-lingual Link Discovery Task (CrossLink-2) at the  10th NTCIR Workshop (NTCIR-10) held on 18-21 June 2013 at the National Center of Sciences in Tokyo, Japan. CrossLingual Link Discovery (CrossLink) is a task of discovering potential links between cross-lingual documents. At NTCIR-9, the UKP team developed a CrossLink framework consisting of anchor selection, anchor ranking, anchor translation, and target discovery subtasks for English-to-{Chinese, Japanese, Korean} directions. At NTCIR-10, the framework is further extended to work in the reverse direction ({Chinese, Japanese, Korean}-to-English), to find out the properties of the tasks in respect of resources, language pair, language direction as well as link discovery methods and evaluation approach. You can read further details about the task and about our system.

Uncovering Plagiarism, Authorship and Social Software Misuse 2012

In 2012, UKP participated in the Wikipedia Quality Flaw Prediction Task in the PAN Lab at CLEF 2012. According to the official results, our system placed first in terms of precision and second in terms of recall and F1-measure.  You can read further details on the task and about our system.

Helping Our Own 2012

In 2012 UKP participated in the Prepositions and Determiners Task of Helping Our Own, an ongoing shared task in text correction.  According to the official results, our system achieved modest performance in some of the nine tasks, placing as high as 9th among the 14 systems.  You can read further details on the task and our system.

SemEval-2012

UKP participated in the Semantic Textual Similarity Task for SemEval-2012, to be held at *SEM (the First Joint Conference on Lexical and Computational Semantics) on 7–8 June, 2012 in Montreal, Canada.  According to the official results, our system (baer/task6-UKP-run2_plus_postprocessing_smt_twsi) was ranked first in two evaluation measures, and second in the third evaluation measure.

NTCIR-9 (2011)

UKP participated in the Cross-lingual Link Discovery Task (CrossLink) at the 9th NTCIR Workshop (NTCIR-9) held on 6–9 December 2011 at the National Center of Sciences in Tokyo, Japan.  Our system came first in the English-to-Chinese and the English-to-Korean File-to-File with manual assessment and Anchor-to-File with Wikipedia ground truth assessment evaluations using Mean-Average-Precision (MAP) measure.  You can read further details about the task and about our system.

Helping Our Own 2011

In 2011 UKP participated in Helping Our Own, an ongoing shared task in text correction, whose pilot run took place as part of the 2011 Generation Challenges.  Our system achieved the best detection scores for 8 out of 13 error classes (among 6 participating systems). You can read further details about the task and our system.

NTCIR-7 (2008)

UKP participated in the Multilingual Opinion Analysis Task (MOAT) at the 7th NTCIR Workshop, held on 16–19 December 2008 in Tokyo, Japan.  Our system ranked 2nd out of 9 systems in the opinionated sentence classification subtask and 3rd out of 5 systems in the polarity classification subtask.  You can read further details on the competition and our system.

A A A | Drucken Print | Impressum Impressum | Sitemap Sitemap | Suche Search | Kontakt Contact | Webseitenanalyse: Mehr Informationen
zum Seitenanfangzum Seitenanfang