Enriching Europeana Newspapers

The publication of the data will benefit a wide variety of research communities. Ongoing work has already identified interest in the data from the following communities:

• A broad spectrum of historians, or other digital humanities disciplines where topics of interest are included in the newspapers (economic historians, art historians, literary historians, geographers…).
• Linguists, given the linguistic diversity of the source (over 40 languages featured) it will be immensely valuable.
• Computer and information scientists working in the areas of natural language processing, information retrieval, information extraction, text mining and optical character recognition.

Standards for the citation of cultural heritage data in research are much required. Citing such data is not just about providing a stable URL, but also providing the kind of documentation that allows scholars to assess the quality and scope of the data. The latter is especially important when dealing with corpora of mixed-quality data: original data (that may have been produced by different OCR engines and having differing levels of fidelity to the original), manual annotations and automatically created enrichments. Cultural heritage institutions that are making such data available also need such standards. Additionally, they require better metrics and processes to understand the benefits of making such data openly available.

In their current form, the newspapers can be accessed through their metadata fields or by full-text search. It is currently not possible, however, to group newspaper articles based on topics. An enrichment of article-level annotations with named entities and topics will benefit researchers wishing to select a portion of the corpus that is relevant to the topic of their study. In addition, the enrichment will enable a quantitative study of the topics and entities discussed in the news.

We aim to increase awareness among a wide range of scholars of the content of the created dataset as well as methods to access and process it. For that purpose, we will perform a quantitative analysis of a selection of the enriched collection to visualize which were the ‘hot topics’ in the selected period and how this was different across different countries. This will demonstrate to scholars working with the corpus what kind of data and metadata it contains, and how this kind of quantitative analysis can be done, paving the way to more quantitative studies on the newspaper corpus in the future. The aim is to open up discussions about the value of working with big data as well as the problems of dealing with errors in the data (e.g. those caused by OCR or enrichment tools).

In their current form, the newspapers can only be accessed via a web portal, thus computer scientists are unable to obtain the full corpus for machine processing. The availability of the corpus through the EUDAT infrastructure will enable further data and text processing to be applied and derived research datasets to be created. These will further promote the reuse of the corpus by several research communities.