Societal interaction of research is not easy to document or measure. It occurs in all kinds of communication channels and is not even limited to the written communi-cation of researchers with external audiences. It also occurs in e.g. teaching, prac-tice training, health care and medication, policy and planning, industrial applica-tions, and technological innovation. In the social sciences and humanities, how-ever, researchers’ written communication for wider audiences is relatively more important in societal interaction.15 This is most prominently the case in legal re-search, where there are even formalized genres and a formal language for written contributions to society that constitute publicly acknowledged sources of law.
These forms of written communication are an important part of the normal so-cietal interaction of legal research and can be defined as “normal everyday inter-actions between organizations that need to create, exchange, and make use of new knowledge to further their goals”.16 Normal societal interaction is mainly an or-ganizational practice and responsibility which is often taken for granted by soci-ety. More attention, not only in the media, but now also in research evaluation sys-tems, is given to extraordinary societal impact which most often occurs at the indi-vidual level and related to extraordinary circumstances. A recent example from Norwegian legal research is the response to a proposed exemption law to control the COVID-19 pandemic in Norway.17 Such cases of individual-level extraordinary impact are all valuable and deserve attention. However, to serve an evaluation to learn from, these cases need to be supplemented with an analysis of normal soci-etal interaction between organizations according to their purposes.
15 Kyvik & Sivertsen. (2013). Økende forskningsformidling. Forskningspolitikk, 36(4), 2013, 16-17.
16 Sivertsen, G., & Meijer, I. (2019). Normal versus extraordinary societal interaction: how to under-stand, evaluate, and improve research activities in their relations to society? Research Evaluation, 29(1), 66–70. doi: 10.1093/reseval/rvz032
17 https://www.jus.uio.no/om/aktuelt/aktuelle-saker/2020/formidlingsprisen-graver.html
4 Societal interaction
Because of limited resources, our study can only contribute in a limited way to the latter ambition by covering three relevant data sources representing publica-tions in genres of societal interaction.
4.2 Delineation, collection, and classification of data
The three selected data sources representing genres of societal interaction are Bokbasen, Norart, and Lovdata. None of these bibliographic data sources are found within the research sector. Norart is a journal indexing service organized by the National Library and funded by the Ministry of Culture. Bokbasen is organized and funded by the major Norwegian book publishers. Lovdata is the major commercial information system serving legal practice in Norway.
Unlike the NSI database that we made use of in chapter 3, none of the three data sources we use here formally interact with and represent the JUREVAL units of evaluation as institutions. Departments or faculties are seldom or never men-tioned in the data. Instead, data can be retrieved using person names (as authors or otherwise mentioned). Our solution is to make these persons represent the in-stitutions that they are affiliated with to measure their organization’s societal in-teraction.
We started by retrieving person names from the NSI database, which also shows their affiliation each year whenever they publish. All researchers affiliated with at least one of the eight units of evaluation and with at least one scholarly publication in legal research (by publication classification) in 2011-2019 were listed as possible identifiers for relevant data. The list contained 522 names in-cluding a few spelling variations in the NSI. All names were used as representa-tions of their unit of evaluation in the search for relevant data. Possible hetero-nyms in each of the three data sources were added to the list, and possible homo-nyms in the retrieved data were treated carefully. The list of names was used to match with author names or mentioned persons in the three data sources.
Two of the databases, Norart and Bokbasen, represent an extension of the pub-lication analysis. We are looking for pubpub-lications beyond the scholarly publica-tions recorded by the NSI, publicapublica-tions that may represent interaction with other audiences in society. Lovdata has all kinds of publications. Most of them are not authored by researchers, but they may refer to publications by researchers. More often, they are official documents from legal processes and decisions in which re-searchers participated. Below is a more detailed description of the three data sources.
4.2.1 Bokbasen
Bokbasen is a database of published books in Norway. It is produced and shared by the Norwegian Publishers Association (NPA). The members represent approx-imately 80 % of the sales from publishers to booksellers in the country. NPA kindly provided us with data for this study. They preselected titles registered as legal lit-erature in Bokbasen and provided bibliographic data for 834 such titles published in 2011-2019. We matched author names with the list of researchers at the units of evaluation, identified overlap with NSI data for scholarly book publishing, and analyzed the representation of books beyond this overlap.
4.2.2 Norart
We followed the same procedure by searching and downloading data from 2011-2019 in Norart, the journal indexing service provided by the National Library of Norway. Here, there was no preselection of legal literature. The names of research-ers in our list could occur as author names or mentioned presearch-ersons in the metadata representing articles in any indexed journal. Some of the indexed journals in Nor-art overlap with the scholarly journals covered by NSI, but items not reported as peer reviewed research articles in NSI, e.g. editorials, discussions, and book re-views, may appear in Norart and not in NSI. Norart also includes several non-aca-demic journals. For our analysis, we extracted (exactly) 1,500 articles in Norart that could be matched to the list of researchers at the units of evaluation. We iden-tified overlaps with NSI data and analyzed the representation of the units of eval-uation in articles and journals beyond this overlap.
4.2.3 Lovdata
Lovdata-Pro is a commercial information source which NIFU subscribed to in 2020 to support this project. It is a service used nationwide in legal practice, and it co-vers all formalized sources of law in Norwegian legal practice within this classifi-cation:
Again, we searched for the names (with different spelling variants) in the list of researchers at the units of evaluation in documents registered by Lovdata in the years 2011-2019. A total of almost 24,000 documents could be matched using per-son names.
Among the nine categories listed above, four are predominant and will be used in our analysis. They cover 97 percent of the matched data:
Nemnder og utvalg mv. (commissions and committees, etc.): 46%
Litteratur (literature): 30%
Rettsavgjørelser (verdicts): 17%
Forarbeider og stortingsdokumenter (parliamentary papers): 5%
In our analysis, we compare the profiles of societal interaction among the units of evaluation and measure how the activity in societal interaction, as represented in Lovdata, compares to the activity in scholarly publishing. We use a simple indica-tor of percentages expressing how the volume of societal interaction compares to the volume of scholarly publishing.