A network-based method to harmonize data classifications

Dario Diodato (2018), Papers in Evolutionary Economic Geography #18.43

A frequent problem in research is the harmonization of data to a common classification, whether that is in terms of — to name a few examples — industries, commodities, occupations, or geographical areas. Statistical offices often provide concordance tables, to match data through time or with different classification, but these concordance tables alone are often not sufficient to define a clear methodology on how the matching should be performed. In fact, the concordance tables have, in numerous occasions, a many-to-many mapping of classifications. The issue is exacerbated when two or more concordance tables are concatenated.
In this Jupyter notebook, I discuss a network-based abstraction of this problem and propose, as a general solution, a method that identifies the network components (or the network communities) to make data converge to a new classification. The method simplifies the issue and reduces greatly conversion errors.

CALL FOR PAPERS: Special Issue on Economic Complexity

To download the pdf version, click on icon


Guest editors 
Pierre-Alexandre Balland (Utrecht University & Collective Learning Group, MIT Media Lab), Tom Broekel (Utrecht University), Dario Diodato (CID Harvard), Ricardo Hausmann (CID Harvard), Neave O’Clery (Oxford), and David Rigby (University of California, Los Angeles)

Lead editor 
Elisa Giuliani (University of Pisa)

Economic complexity has emerged as a powerful paradigm to understand key issues in economics, geography, innovation studies, and other social sciences. Owing its popularity, in part, to its cross-disciplinary reach, the concept has shed new light on the variation in standards of living across nations (Hidalgo and Hausmann, 2009), differences in sophistication of technologies (Fleming and Sorenson, 2001), and the heterogeneous distribution of knowledge in space (Balland and Rigby, 2017). This excitement is not limited to academia. A host of policy institutions, ranging from international organizations such as the World Bank, World Economic Forum, European Commission, and OECD to national and local actors, have embedded both the methodology and conceptual framework of complexity into their core toolbox. Hence, as economic complexity moves from the periphery to the core of economic thinking and development policy, this special issue attempts to both reflect on past success and look forward to new research frontiers.
The complexity perspective posits that the knowledge content of a country or a city cannot be found at the intensive margin: knowledge grows not by accumulating more of the same, but by adding new and different elements to existing capabilities. It is this evolutionary, combinatorial process that drives many economic phenomena. While this description of knowledge accumulation is often in direct contrast with leading models of economic growth and development – where technology is typically a homogenous good – the theoretical roots of complexity can be found in both traditional and heterodox economics, from Smith’s division of labor (Hausmann et al, 2011) to information theory (Antonelli, 2011), from Jacob’s externalities (Jacobs, 1969) to urban scaling (Bettencourt et al. 2007, Balland et al., 2018), from agglomeration effects (Glaeser et al. 1992) to network theory (Hidalgo et al., 2007).
Important questions to address include the micro-foundations of economic complexity (how and at what scale is it created? What are its ingredients and where do they reside?), and its relation to traditional concepts such as tacit knowledge, radical innovation, agglomeration, and production networks?

The special issue is organized around four main themes:

  • Micro/theoretical foundations of complexity theory, possibly connecting it to established schools of economic thought or other kinds of literature such as biology or physics
  • New empirical applications of complexity to key issues in economics, geography, and human development
  • Novel approaches to measuring complexity, and studying its evolution over time, organizations and space
  • Implications for policy and firm strategy

Submission process
We welcome full manuscripts of up to 8,000 words maximum (excluding references and appendices). Articles should be submitted online via the Research Policy web-portal. Each paper will be reviewed by two or three referees. We aim to complete the review process with a maximum of two drafts (i.e., a single ‘revise and resubmit’) before a final decision is made — unless special circumstances call for an additional revision round.

March 1, 2019: updated submission deadline for full manuscript
June 1, 2019: decisions and comments sent to authors
October 1, 2019: deadline for final draft
Feb 1, 2020: expected publication

Contact information
For questions regarding the special issue please contact dario_diodato@hks.harvard.edu or oclery@maths.ox.ac.uk

Antonelli, Cristiano, ed. Handbook on the economic complexity of technological change. Edward Elgar Publishing (2011).
Balland, Pierre-Alexandre, and David Rigby. “The geography of complex knowledge.” Economic Geography 93, no. 1 (2017): 1-23.
Balland, P.A., Jara-Figueroa, C., Petralia, S., Steijn, M., Rigby, D., and Hidalgo, C. “Complex Economic Activities Concentrate in Large Cities.” Papers in Evolutionary Economic Geography, no 18 (2018): 1-10.
Bettencourt, Luís MA, José Lobo, Dirk Helbing, Christian Kühnert, and Geoffrey B. West. “Growth, innovation, scaling, and the pace of life in cities.” Proceedings of the national academy of sciences 104, no. 17 (2007): 7301-7306.
Fleming, Lee, and Olav Sorenson. “Technology as a complex adaptive system: evidence from patent data.” Research Policy 30, no. 7 (2001): 1019-1039.
Glaeser, Edward L., Hedi D. Kallal, Jose A. Scheinkman, and Andrei Shleifer. “Growth in cities.” Journal of Political Economy 100, no. 6 (1992): 1126-1152.
Hidalgo, César A., and Ricardo Hausmann. “The building blocks of economic complexity.” Proceedings of the national academy of sciences 106, no. 26 (2009): 10570-10575.
Hidalgo, César A., Bailey Klinger, A-L. Barabási, and Ricardo Hausmann. “The product space conditions the development of nations.” Science 317, no. 5837 (2007): 482-487.
Jacobs, Jane. The economy of cities. Random House (1969).