Darmstadt Knowledge Processing Repository

At the UKP Lab, we put a strong focus on developing the software that is the basis for our experiments in a re-usable manner. We call that body of software that we produce the Darmstadt Knowledge Processing Software Repository (DKPro). 

Products

Several products have grown from our DKPro philosophy and have been released under an open source license to the public:

  • CSniper is a search-based annotation tool to help distributed annotation teams finding infrequent linguistic phenomena in large corpora.
  • DKPro Core provides a set of ready to use software components for natural language processing, based on the Apache UIMA framework.
  • DKPro Lab is a lightweight framework for parameter sweeping experiments. It allows you to set up experiments consisting of multiple interdependent tasks in a declarative manner with minimal overhead.
  • DKPro LSR (Lexical Semantic Resources) is a unified API for several lexical-semantic resources.
  • DKPro Similarity is an open source software package for developing text similarity algorithms.
  • DKPro Spelling includes components for real-word spelling error correction and experimental frameworks for mining such errors from the Wikipedia revision history as well as for the "Helping Our Own" shared tasks 2011 and 2012.
  • DKPro Statistics is a collection of open-licensed statistical tools, currently including correlation and inter-rater agreement methods.
  • DKPro TC (Text Classification) is a UIMA-based text classification framework built on top of DKPro Core, DKPro Lab and the Weka Machine Learning Toolkit. It is intended to alleviate supervised machine learning experiments with any kind of textual data.
  • DKPro Uby is a Java framework for creating and accessing sense-linked lexical resources in accordance with the UBY-LMF lexicon model, an instantiation of the ISO standard Lexicon Markup Framework (LMF).
  • DKPro WSD is a modular, extensible Java framework for word sense disambiguation.
  • JOWKL (Java OmegaWiki Library) is an open-source, Java-based application programming interface that allows to access all information contained in OmegaWiki, such as glosses, usage examples, translations and much more.
  • JWKTL (Java Wiktionary Library) is a free, Java-based application programming interface that allows to access the information contained in Wiktionary.
  • JWPL (Java Wikipedia Library) is a free, Java-based application programming interface that allows to access all information contained in Wikipedia.
  • WebAnno is a general purpose web-based annotation tool for a wide range of linguistic annotations.

Team

The principal investigator is Prof. Dr. Iryna Gurevych.

Richard Eckart de Castilho is currently the technical lead.

DKPro is a shared project of all UKP to which all group members contribute.

Teaching

We use DKPro products in our courses:

Awards

The UKP group received two IBM's 2008 Unstructured Information Analytics (UIA) Awards for their DKPro proposals! The award was covered in the 30 June 2008 issue of the Darmstädter Echo.

Publications

Approaches to Automatic Text Structuring

Author Nicolai Erbs
Date September 2015
Kind Phdthesis
AddressDarmstadt
KeyTUD-CS-2015-1228
Research Areas Ubiquitous Knowledge Processing, UKP_s_JWPL, UKP_s_DKPro_Similarity, UKP_s_DKPro_Core, UKP_p_WIKULU, UKP_p_WIWEB, UKP_p_openwindow, UKP_p_DKPro, UKP_a_NLP4Wikis, UKP_a_ENLP
Abstract Structured text helps readers to better understand the content of documents. In classic newspaper texts or books, some structure already exists. In the Web 2.0, the amount of textual data, especially user-generated data, has increased dramatically. As a result, there exists a large amount of textual data which lacks structure, thus making it more difficult to understand. In this thesis, we will explore techniques for automatic text structuring to help readers to fulfill their information needs. Useful techniques for automatic text structuring are keyphrase identification, table-of-contents generation, and link identification. We improve state of the art results for approaches to text structuring on several benchmark datasets. In addition, we present new representative datasets for users’ everyday tasks. We evaluate the quality of text structuring approaches with regard to these scenarios and discover that the quality of approaches highly depends on the dataset on which they are applied. In the first chapter of this thesis, we establish the theoretical foundations regarding text structuring. We describe our findings from a user survey regarding web usage from which we derive three typical scenarios of Internet users. We then proceed to the three main contributions of this thesis. We evaluate approaches to keyphrase identification both by extracting and assigning keyphrases for English and German datasets. We find that unsupervised keyphrase extraction yields stable results, but for datasets with predefined keyphrases, additional filtering of keyphrases and assignment approaches yields even higher results. We present a de- compounding extension, which further improves results for datasets with shorter texts. We construct hierarchical table-of-contents of documents for three English datasets and discover that the results for hierarchy identification are sufficient for an automatic system, but for segment title generation, user interaction based on suggestions is required. We investigate approaches to link identification, including the subtasks of identifying the mention (anchor) of the link and linking the mention to an entity (target). Approaches that make use of the Wikipedia link structure perform best, as long as there is sufficient training data available. For identifying links to sense inventories other than Wikipedia, approaches that do not make use of the link structure outperform the approaches using existing links. We further analyze the effect of senses on computing similarities. In contrast to entity linking, where most entities can be discriminated by their name, we consider cases where multiple entities with the same name exist. We discover that similarity de- pends on the selected sense inventory. To foster future evaluation of natural language processing components for text structuring, we present two prototypes of text structuring systems, which integrate techniques for automatic text structuring in a wiki setting and in an e-learning setting with eBooks.
Website http://tuprints.ulb.tu-darmstadt.de/4959/
Full paper (pdf)
[Export this entry to BibTeX]

Important Copyright Notice:

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
A A A | Drucken Print | Impressum Impressum | Sitemap Sitemap | Suche Search | Kontakt Contact | Webseitenanalyse: Mehr Informationen
zum Seitenanfangzum Seitenanfang