Machine Learning Helps Us Understand Legislative Texts
Leveraging on legislative text data from the United States, Massimo Morelli, Full Professor at Bocconi Department of Social and Political Sciences finds that the introduction of an independent bureaucracy correlates with more detailed legislation. This finding suggests that lawmakers tend to limit bureaucratic discretion by enacting more detailed legislation as a way to micro-manage implementation. Likewise, he finds that delegation of authority to the State Governor is more likely when the Governor and the two state chambers are controlled by the same party. Studying the same topic with a different methodology, Anthony Bertelli, Full Professor at Bocconi Department of Social and Political Sciences, analyses legislative text data from the European Union and finds that more detailed legislation is usually produced under qualified majority than under unanimity vote.
Political and legislative text can be an incredibly rich source of data capable of illuminating processes such as institutional change, political alliances and legislative strategy. However, processing a large amount of text by hand can be both expensive and very time-consuming. In an effort to automatize this process, Professor Morelli has recently developed, together with Professors Matia Vannoni and Elliot Ash, a new method which can substitute much of the work of the researcher.
In a similar paper, Professor Bertelli uses machine learning to teach to an algorithm the coding technique of the researcher, in order to apply such a technique to larger and more recent data. Together, these two recent publications put Bocconi University at the center of the international debate on the topic.
The method devised by Prof. Morelli relies on a Natural Language Processing (NPL) technique called "Syntactic Dependency Parsing". More precisely, the algorithm takes a look at the raw legislative text and extracts its single grammatical elements, such as the nouns, the verbs and the adjectives together with the syntactic dependency relations between them. Then, by matching the syntactic structure of the sentences in the text with some pre-defined templates, the authors could retrieve the elements they needed for their research application. An example of a relationship between words is the connection between a verb and its direct object, or between an adjective and the noun it refers to. Thus, by telling the algorithm to count the occurrences of some lexical units, the authors could retrieve the elements they needed for their research application.
Prof. Morelli applies this technique to the study of bureaucratic discretion and delegation of authority in the United States. In particular, he finds that legislation containing more provisions is correlated with the introduction of an independent bureaucracy, suggesting that when the bureaucracy becomes more independent, laws and regulations tend to be more precise in order to narrow the liberty of action of bureaucrats. Also, Prof. Morelli finds that under unified government (meaning that the same party holds the State Governor and the two chambers), more provisions delegating power to the Governor are enacted, namely provisions that fit a 'delegation syntactic structure' (e.g. containing a modal "shall" followed by an active verb associated to the subject "Governor"). Hence, in these situations, delegation of authority towards the State Governor is more likely.
Prof. Bertelli's paper also applies his Machine Learning-based technique on authority delegation albeit in the context of the European Union. Together with Jason Anastasopoulos, they replicate and extend a study already conducted by Fabio Franchino finding that delegation and discretion are more likely when the European Union voted by unanimity vis-à-vis qualified majority. The reason behind this finding is that unanimity correlates with more generic and less specific legislation whereas more detailed regulations limit discretion in the implementation. This confirms the original results obtained by Franchino and builds the case for a successful application of this machine learning approach to text analysis to any field where human coding has been profitably employed.
These two approaches to political text analysis are markedly different. On the one hand, the method proposed by Prof. Morelli performs even better than human coding in his application and his results are both promising and encouraging since could possibly expand the domain of analysis beyond what researchers have already coded. However, this method appears to be particularly strong when tackling legislative text but its application to other contexts may still require some work.
On the other hand, Prof. Bertelli's method has the advantage of being easily applicable to a large variety of fields of research. Nonetheless, it relies on the fact that the algorithm has to learn the coding technique employed by a qualified researcher first. Differently, from the "syntactic parsing" method, it cannot substitute human labor entirely nor outperform it at the moment. In spite of this, it has incredible potential in all kinds of social sciences since man-made text coding has been already applied to a large variety of topics and machine learning can possibly be used to expand our knowledge in each of them.
Vannoni, M., Ash, E., and Morelli, M. (2021). "Measuring Discretion and Delegation in Legislative Texts: Methods and Application to US States." Political Analysis, 29(1), 43-57. doi:10.1017/pan.2020.9.
Anastasopoulos, L., and Bertelli, A. (2020). "Understanding Delegation Through Machine Learning: A Method and Application to the European Union." American Political Science Review, 114(1), 291-301. doi:10.1017/S0003055419000522.