Aim
The project had two main aims:
- Demonstrating that NLP and AI techniques can be used to extract and quantify information related to structural change in documents from an archive of a leading Swedish consultancy firm covering the period 1965-1993.
- Showing that the database contained valuable information.
Database
The database subset used contained approx. 2,300 documents by a single author. Of these, 55% were in Swedish and 35% in English, the rest were in other languages, contained several languages or the language could not be identified. Only the Swedish and English documents were processed.
Methodology
Metadata extraction
The document title and date were extracted for each document. Documents were assigned to clients based on the document title if an organization was mentioned in it.
Language identification
Language identification was performed on the document, page and sentence level. The page-level identification was used as it gave the most consistent results and could account for the fact that some document contain several languages.
Data cleaning
The textual data in the dataset was obtained by converting the scanned archive documents using optical character recognition (OCR). The quality of the results of this process depend strongly on the quality of the original documents.
To improve the quality of the data, we tried to remove some of the noise. This was done primarily by removing pages and sentences with low confidence scores returned by the language identification algorithm. Furthermore, some specific character patterns were also removed (e.g. consecutive punctuation and other non-alphanumeric characters).
Named-entity recognition
Personal names, organization names and place names were extracted using using Bert’s named-entity recognition. This was done separately for the Swedish and English datasets.
The organisation names, personal names and locations extracted from both datasets were combined and an effort was made to canonicalize the different variants (due to different spelling, OCR errors etc.) of the individual named entities to avoid duplication.
The personal name data is not included in the report due to privacy concerns.
POS-tagging and morphological analysis
The dataset were morphologically tagged, i.e. the part-of-speech/word class (noun, verb etc.) of each word was determined along with its grammatical features (number, voice, mood etc.).
We were particularly interested in the verb mood and tense as well as whether a sentence is a question. The analysis was done on the sentence level, e.g., a sentence with a verb with imperative mood was assigned as imperative.
Sentiment analysis
Sentiment analysis attempts to determine the sentiment or attitude of the text, in other words, how positive or negative it is. The result is a scale between -1 (extremely negative) and 1 (extremely positive).
For Swedish, VADER (Valence Aware Dictionary and sEntiment Reasoner), a lexicon and rule-based model was used. For English, the TextBlob library was used.
Topic modelling using Latent Dirichlet Allocation
Topic modelling was done using Latent Dirichlet Allocation (LDA), a generative statistical model that allows sets of observations to be explained by unobserved groups that explain the similarities between some parts of the data.
Keyword and keyphrase extraction and analysis
For the English dataset, LDA was used to find candidate words and phrases that could reveal specific topics in different time periods. These were then manually filtered based on the analysis of the periods in question. In order to discover more contextual information about the selected keywords and keyphrases, co-occurrence matrices were computed for these keywords/keyphrases.
For the Swedish dataset, the keyword selection was done based on the relative word frequency ratios over different periods. The indicated keywords were then filtered and grouped into semantic fields which were further grouped into topics. The scores produced by this method indicate how over- or under-represented a semantic field is in each period.
In order to detect the evolution of semantic meaning surrounding the keywords, word embeddings were used. They represent each word by a multi-dimensional vector, such that the geometry between these vectors captures semantic relations between the corresponding words.
A weighted sum of embeddings of words co-occurring with the keyword in each period was then computed. A cosine similarity matrix to show the contextual semantic meaning variation of a keyword over time was then produced.
Results
The study showed that the archive is a unique database of documents from a very interesting time period of Swedish as well as global industrial history that can provide an valuable reflection of the structural changes that took place in Sweden and globally in the period between 1965 and 1994.
It also demonstrated different methods to retrieve and quantify this valuable information by utilizing various Natural Language Processing methods, Machine Learning algorithms, and AI models.
By extracting time series data focusing on keywords and keyphrases related to structural change, we tried to show how the textual contents of the documents correspond to changes in the environment. This technique could also be used to derive meaningful results related to other topics. Furthermore, we also show how using named-entity recognition to extract organisation names and other names creates possibilities for exploring relations between different individuals and organizations as well as the social networks they create, both on the level of the whole database as well as based around specific individuals or clients.
The morphological and sentiment analysis of the data gave some interesting results, but these proved to be somewhat hard to interpret. However, we believe sentiment analysis in particular can be a valuable research tool used on this and similar datasets.