Testing various classifiers to measure Euroskepticism and Populism in European Union Parliamentary Questions (research on contemporary European policymaking)
(See project description)
Parliamentary questions are one of the key means of oversight in the European Union. This project aims to analyze the ways that this oversight process is used at the European level through structural topic modeling, sentiment analysis, machine learning, natural language processing, and statistical methods, as well as finalize the data structure for a large N, text dataset that will be made publicly available at the conclusion of the research.
- Parsing and reorganizing data to make usable for text analysis, using regex expressions in python. Eventually make dataset publicly available.
- Cleaning, merging and refining several data sources together. Accounting for inconsistent joining keys. Designing and structuring data representation.
- Developing and optimizing a text classification scheme to assess the presence of Euroskepticism in questions by trying different ML classifiers. When I was working on the proect we used tried CNNs, LSTMs. Biggest problem was imbalance in the data, and how to work with it.
- Structural Topic modelling: LDA, DMR
Learned about the way that institutions and oversight work within the complex multilevel governance of the European Union, i.e. the relationships and networks, and strengths and weaknesses, of the relationships between levels of government in Europe.