Page - (000249) - in Knowledge and Networks

245 Overview Our basic measurement question is to assess how common or novel any pairwise combination of prior work is. To determine this, we want to know both the (i) observed frequency of any given pairing of references in the WOS and (ii) the fre- quency of that pairing that would have occurred by chance. Comparing the observed frequency to the frequency expected by chance creates a normalized z-score mea- sure for whether any given pairing appears novel or conventional. To measure the observed frequency of any given pairing in the WOS, we took the following five steps: (1) Took the references listed in a given paper’s bibliography. (2) Considered all pairwise combinations of the papers referenced in the bibliogra- phy of the paper. (3) For each pairwise combination, recorded the two journals that were paired. (4) Repeated steps (1–3) for every paper in the WOS. (5) Counted the aggregate, population-wide frequency of each journal pairing for all referenced pairs from a given publication year. Figure 12.1 presents a stylized example for steps (1–3), showing for a given paper how pairs of references are counted from that paper’s reference list. The algo- rithm repeats this counting process for every article in the WOS and aggregates the counts for each given publication year. Our method counts specific journal pairings, using different journals as a proxy for different areas of knowledge. Journal-level analysis is well positioned to distin- guish domains of knowledge while having precedence in the literature for being relatively transparent, interpretable, and computationally feasible (Bollen et al., 2006; Itzkovitz et al., 2003; Small, 1973; Stringer et al., 2010).1 Having determined the observed frequency of each journal pairing, we consid- ered the frequency distribution for each journal pairing that would have occurred by chance. The null model randomly reassigns the citation links between papers. As further detailed below, the method uses a variation of the Markov Chain Monte Carlo (MCMC) algorithm to randomly switch co-citations between all 17.9 million papers into a synthetic network with 302 million citations (edges), the same number of papers and citations as the observed network. Note that this method preserved the detailed paper-level structure of the global citation network. The number of cita- tions to and from each paper was preserved backward and forward in time. Using this approach, we created 10 synthetic instances of the entire WOS, each with its own set of randomized citation links. For each instance of the WOS, we then repeated steps (1–5) above, calculating the frequency of each co-referenced journal pair. Looking at all 10 randomized cases of the WOS, we generated a distri- 1 Other operationalizations might consider lower resolution pairings using the ISI’s 252 subfield categories, text-based combinations, or conceptualizations for measuring novelty beyond combi- natorial pairs (Rosenkopf & McGrath, 2011). 12 How Atypical Combinations of Scientific Ideas Are Related to Impact:…