A first snapshot of academics’ media mentions and policy citations in Flanders Belgium

Hans Jonker^* and Florian Vanlee^**

^*Hans.Jonker@vub.be

0000-0002-9923-440X

Centre for R&D Monitoring (ECOOM); Department of Sociology (SOCI), Vrije Universiteit Brussel, Belgium

^** Florian.Vanlee@ugent.be

0000-0001-7384-0201

Department of Communication Sciences, Universiteit Gent, Belgium

Data that show academics’ interactions with media and policy is scarce. Based on open-source database FRIS, we queried the names of all active academics from Dutch-speaking universities in Belgium in BelgaPress and Overton. Explorative analyses reveal distinct sector differences in media mentions and policy citations, with academics from Medical – and Social sciences being the most visible. A small minority of mostly male academics featured very frequently in media and policy, but not in both domains simultaneously. Moreover, academics featured in the media seem are not necessarily quoted more in policy and vice versa.

1. Introduction

Academics are increasingly expected to contribute to the public debate and provide policy advice for public administrations. In the case of the UK, the Research Excellence Framework (REF) not only incentivises academics to produce knowledge for their peers, but requires the documentation of societal impact as well. But since this practice relies on self-reporting, operationalisations of impact are more postulated than demonstrated (Niederkrotenthaler et al., 2011). Hence, current systems encourage universities to invest primarily in stories of heroic scientists – usually male professors – going beyond the confines of academia (Cairney & Oliver, 2020; Dunlop, 2018). In doing so, they forego on a systematic and general analysis of their staff’s performance in media and policy.

We lack data that show individual academics’ media presence and policy uptake on a university scale. While so called visible scientists (aim to) influence public and policy (Goodell, 1977), their effective impact has not yet been demonstrated due to a lack of data (Fahy, 2017). A dataset of all media mentions or all policy citations seems a workable starting point, not to assess impact but rather to provide a first snapshot of heterogeneity in the societal dissemination of academic knowledge. Each datapoint represents a heterogenous coupling (Costas et al., 2021); a documented interaction between science and society.

The case of Flanders is interesting because data on academics are freely available via the FRIS portal (Flanders Research Information Space, 2023). Also, the Belgian database BelgaPress (BelgaPress, 2023) provides detailed data on the visibility of these academics in the media. We chose to focus on written press articles because it allows large amounts of data to be analysed relatively easily. And since 2019, the Overton database offers the possibility to investigate which academics worldwide can be found in policy documents. Our objective in this paper is therefore to explore the characteristics of academics in media and policy documents. The extent to which academics do or do not appear in popular media has not been explicitly examined to date, and moreover, it is far from certain that this form of visibility necessarily translates into policy impact.

2. Methods

We scraped the regional opensource data portal FRIS for all researchers that were active in 2019, current affiliation(s) and total number of scientific publications (Flanders Research Information Space, 2023). Academic staff working for administrative unites were omitted. This produced a database of 31512 individuals. Based on current affiliation, data were enriched with information on sector and discipline, using the Flemish Research Discipline Standard (Vancauwenbergh & Poelmans, 2019). Finally, data were gender-coded with name-based sex data (reduced to female, male, undefined). Gender-neutral names were coded manually.

For data on media mentions, we used the online BelgaPress database (BelgaPress, 2023). Belgian news agency Belga collects all articles that appeared in Belgian newspapers and magazines on a daily basis, and its database allows accurate search functions over a selected time period. As our dataset consisted of Dutch-speaking academics in Belgium, we limited our scope to Dutch-speaking newspapers and magazines. More details and reasons for this approach, as well as partial results of a limited number of social scientists’ media presence, have been reported elsewhere (Jonker et al., 2022). Media appearance data were gathered from 1/01/2019 to 31/12/2019, as 2020 and further years would be deemed atypical due to the covid-19 pandemic. We queried all full names of researchers and noted the amount of written press attention for each individual academic, excluding namesakes. Duplicates from the same outlet were removed. Lastly, we chose to exclude researchers who appeared in the written press in a non-academic capacity, namely those with a political mandate (n = 47), those in the function of a non-academic organization, institution or company (n = 17), and a rest category of journalists, lawyers, athletes, or celebrities (n = 11). Rectors¹ (n = 5) were also not included because of their special mandate.

For data on policy documents, we used the data portal Overton.io (Overton, 2023). In the summer of 2021 we downloaded lists of all affiliated authors and the citations of their work in policy-related documents for each Dutch-speaking university (and university hospital) in Belgium. We then deleted and merged duplicates and removed wrong affiliations and namesakes. The citation delay in policy-related documents in Overton (see: Fang et al., 2020) motivated our choice to compare a total number of policy citations in 2021 to media mentions in 2019, as to leave sufficient time for accumulation of policy document citations to occur.

The result of these queries, was a dataset with following characteristics (see Table 1). Regarding media, 2451 of 31512 (7.8%) of active academics appeared once or more in the written press of 2019. Those academics who did appear in the media in 2019 did so with 5.7 (SD = 16.9) mentions on average. Compared to sector proportions at Flemish universities, academics from Social sciences (21.8% of university vs 31.4% in the media) and Humanities and the arts (10.3% from university personnel vs 14.2% in the media) were overrepresented in the written press of 2019. Academics from Engineering and technology (19.3% vs 9.5%) and Natural sciences (14.7% vs 10.3%) were underrepresented in the media. Male academics (6.7 mentions (SD = 19.8)) appeared on average twice as many times in the in the written press than their female colleagues (3.3 mentions, SD = 5.3).

Regarding policy, 5359 of 31512 (17.0%) academics’ publication(s) got cited at least once in Overton, with 12.9 (SD = 32.4) citations on average. Compared to sector proportions at Flemish universities, academics from Health sciences (26.9% of university vs 40.6% in cited in policy) and Social sciences (21.8% of university vs 27.6% cited in policy) were overrepresented in policy-related documents. Academics from Engineering and technology (19.3% vs 11.8%) and Humanities and the arts (10.3% from university personnel vs 2.2% cited in policy) were underrepresented in policy-related documents. Male academics (15.4 citations (SD = 38.1)) got cited on average almost twice as many times in policy-related documents than their female colleagues (8.7 citations, SD = 18.6).

Table 1. Descriptive characteristics of active academics from FRIS, with sample featured in media (>0) and sample featured in policy (>0).

Note ^a. Natural sciences (NATU), Engineering and technology #ENGI, Medical and health sciences (MEDI), Agricultural and veterinary sciences (AGRI), Social sciences (SOCI), Humanities and the arts (HUMA), based on Vancauwenbergh and Poelmans (2019).

To identify factors determining media visibility and cited research in policy documents, two multiple linear regression models were run, one to predict media mentions (n = 2451), the other to predict policy citations (n = 5373). Both predictions were based on data on research sector, sex (0 = male; 1 = female) and total number of scientific publications, as provided by FRIS (as a proxy for seniority or academic status). These variables were inserted into the models using the Enter method. Given that both media and policy data were characterised by major outliers, the homogeneity assumption was violated. We chose to work with these data anyway given their meaningful significance; incrementally removing the outliers would almost completely erode both variables. Data on seniority (predoc, postdoc, professor, emeritus) was not included in the models, as the variable Publications (FRIS) provided more robust results in terms of explained variance and significance. The variable University was excluded from the models as well due to a problem of multicollinearity.

3. Results

4.1. Multiple regression models

The first multiple regression equation for media (see Table 2) was found to be significant (F(8,2450) = 7.163, p < .001), with R² = .023. Active academics in Flanders predicted media mentions is equal to 3.921 ‑ 3.367 (if female) - .836 (if NATU) - .916 (if ENGI) + 1.604 (if MEDI) - 1.193 (if AGRI) + 4.031 (if SOCI) + 2.958 (if HUMA) + .011 (per scientific publication) media mentions in 2019. Compared with a male academic with 0 publications (reference category), the model predicts significantly less (‑ 3.367; p < .001) media mentions for female academics. Producing one extra publication predicts a slight yet significant effect (.011; p < .001) on appearing once a year in the written press. Academics sectors did not significantly predict media mentions.

Table 2. Multiple regression model for media mentions (dependent), sectors, gender and publications.

Note. * p < .05, ** p < .01, *** p < .001. n_media = 2454. R² = .023. Variable sector not specified not included.

The second multiple regression equation for policy citations (see Table 3) was found to be significant (F(8,5358) = 86,637, p < .001) as well, with R² = .115. Active academics in Flanders predicted policy citations is equal to - 2.123 ‑ 3.174 (if female) + 7.375 (if NATU) ‑.501 (if ENGI) + 7.965 (if MEDI) + 13.548 (if AGRI) + 11.183 (if SOCI) + 1.855 (if HUMA) + .099 (per scientific publication) total policy citations. Compared with a male academic with 0 publications (reference category), the model predicts female academics’ research is significantly less cited (‑ 3.174; p < .001) in policy-related documents. Producing one extra publication predicts a small yet significant effect (.099; p < .001) on a research item being cited in policy. Only academics from AGRI significantly predicted policy citations.

Table 3. Multiple regression model for policy citations (dependent), sectors, gender and publications.

Note. * p < .05, ** p < .01, *** p < .001. n_policy = 5373. R² = .115. Variable sector not specified not included.

Results show that media mentions and policy citations clearly vary by academic sector (see Figure 1). Academics from AGRI showed their visibility clearly in policy papers, but barely attracted media attention. The policy citations model predicts a strong significant effect (13.548; p = .006) for academics active in the relatively small sector (3.9% in Flanders; Table 1). In contrast, HUMA academics were visible mainly in the media, but were scarcely cited by policy documents. Figure 1 also reveals L-shape patterns for academics active in SOCI and MEDI, the most visible sectors in terms of these two variables. The data show that some were more likely to appear in the media, and fewer in policy documents, and vice versa.

Figure 1. Flemish academics’ media mentions and policy citations by academic sector, coloured by gender.

Another striking feature of Figure 1 are the outliers. Both media mentions and policy citations show highly skewed distributions, with the outliers being predominantly male academics in SOCI and MEDI (and HUMA for media). Only a handful of male academics from SOCI and HUMA accomplished both high media visibility and policy impact.

4. Discussion

4.1. Flemish academics in media and policy

Based on FRIS data, this study shows a first snapshot of Flemish academics’ media presence and policy impact. Preliminary results point to two elements for further investigation. Firstly, both proxies of visibility show remarkable outliers. The majority of academics in Flanders did not appear in 2019 media and were not cited in policy documents. But a small minority of mostly male academics frequently featured in the media and were frequently cited in policy. Hence, this study shows that these are different academic profiles. One the one hand there is the well-known figure of the public intellectual or the celebrity scientist, on the other the figure of the much more discrete policy expert. Therefore, contrary to the assumption of Goodell (1977), it seems necessary to break down "visibility" into media attention and policy impact.

Moreover, it is not those who have published the most in the academic system who emerge from this study as "impact heroes" (Cairney & Oliver, 2020; Dunlop, 2018). Publishing more scientific output has a stronger effect on getting more cited in policy documents (.099 citation per scientific publication; Table 3) than getting media attention (.011 mention per scientific publication; Table 2). Both effects however seem rather limited. Nevertheless, the link between publishing more and getting more cited in policy documents is evidently much more direct, while media interactions are likely explained by other factors, such as personal characteristics (Fahy, 2017; Goodell, 1977), time availability (Valinciute, 2020), willingness (Besley et al., 2019), topicality, controversy and successful interactions with journalists in the past (Hubner & Bond, 2022; Jensen et al., 2008).

Secondly, this study highlights visibility’s dependence on academic sectors, showing distinct differences in media attention and policy citations. Results show that the majority of media mentions and policy citations are from MEDI and SOCI sectors, expanding the results of Fang et al. (2020) in the context of policy up-take from Web of Science. This likely signals that there is great variation in policy sensitivity across sectors and institutions (Li et al., 2022), alongside selective journalistic interests (Peters, 2013).

4.2. Limitations

This paper has several limitations. Firstly, the information in FRIS depends on universities supplying their data in current research information systems (CRIS). This means that the list of active researchers in 2019 is likely not exhaustive. In 46 cases, no scientific publications were provided even though this was expected due to the existence of policy citations. These cases were queried in Web of Science to provide the total number of scientific publications. We aim to replace FRIS publications with Web of Science publications in the near future.

Secondly, due to the relative high media concentration in Flanders (Hendrickx & Ranaivoson, 2019), some newspaper articles had the exact same content even though the newspaper was different. Because of the duopoly in the newspaper market in Flanders, content within the same press house is often recycled. Therefore, not every media mention is an equally "substantial" mention; this means that high media profiles are likely to be slightly inflated. Additionally, 2019 was an election year, with increased demand for social science expertise.

Thirdly, Overton (2023) states in their search engine that OpenAlex is missing around 20% of author affiliations, especially for researchers that do not use persistent identifiers. Moreover, research before 2009 becomes harder to identify. We used name-based entity matching to link FRIS academics to authors cited, where small errors cannot be avoided entirely. These factors mean that the proportion of Flemish academics cited in Overton is likely to be slightly higher.

Open science practices

This paper’s data are based on open-source data as provided by several databases. Data on academics in Flanders can be found at the FRIS portal (https://www.researchportal.be/en). Belgian media data can be consulted at BelgaPress (https://www.belga.press/). To investigate policy impact, we queried the Overton database (https://www.overton.io/). We enriched the dataset with gender data. We will make these data available when submitting for publication.

Acknowledgments

The authors are grateful for the support from colleagues Dirk Derom, Kenneth Goossens and Walter Ysebaert from the VUB, and the valuable feedback from Ed Noyons and Rodrigo Costas from CWTS Leiden as well. The authors are grateful for the helpful correspondence with Euan Audie at Overton. We also want to thank Aaron Van Steenlandt (UGent) for his advice with the multiple regression models.

Author contributions

Conceptualization: HJ, FV; Formal Analysis: HJ; Investigation: HJ; Methodology: HJ; Supervision: FV; Visualization: HJ; Writing – original draft: HJ; Writing – review and editing: HJ, FV.

Competing interests

Authors declare no competing interests.

Funding information

This research received no external funding.

References

BelgaPress. (2023). Belga. https://belga.press/

Besley, J. C., O’Hara, K., & Dudo, A. (2019). Strategic science communication as planned behavior: Understanding scientists’ willingness to choose specific tactics. PLOS ONE, 14(10), e0224039. https://doi.org/10.1371/journal.pone.0224039

Cairney, P., & Oliver, K. (2020). How should academics engage in policymaking to achieve impact? Political Studies Review, 18(2), 228-244.

Costas, R., de Rijcke, S., & Marres, N. (2021, 2021/05/01). “Heterogeneous couplings”: Operationalizing network perspectives to study science-society interactions through social media metrics. Journal of the Association for Information Science and Technology, 72(5), 595-610. https://doi.org/https://doi.org/10.1002/asi.24427

Dunlop, C. A. (2018, Sep). The political economy of politics and international studies impact: REF2014 case analysis. British Politics, 13(3), 270-294. https://doi.org/10.1057/s41293-018-0084-x

Fahy, D. (2017, Nov). Historical moments in public understanding of science: 1977, The Visible Scientists identifies a new scientist for the mass media age. Public Understanding of Science, 26(8), 1019-1024. https://journals.sagepub.com/doi/10.1177/0963662517732909?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed

Fang, Z., Dudek, J., Noyons, E., & Costas, R. (2020). Science cited in policy documents: Evidence from the Overton database Altmetrics conference, http://altmetrics.org/wp-content/uploads/2020/11/02_submission_Fang_Dudek_Noyons_Costas-altmetrics20.pdf

Flanders Research Information Space. (2023). Flemish Government: Department of Economy, Science & Innovation. https://www.researchportal.be/en

Goodell, R. (1977). The Visible Scientists. The Sciences, 17(1), 6-9. https://doi.org/10.1002/j.2326-1951.1977.tb01494.x

Hendrickx, J., & Ranaivoson, H. (2019). Why and how higher media concentration equals lower news diversity – The Mediahuis case. Journalism, 22(11), 2800-2815. https://doi.org/10.1177/1464884919894138

Hubner, A. Y., & Bond, R. (2022, 11/21). I Am a Scientist ... Ask Me Anything: explicating the role of past behavioral attitudes on scientists' future public engagement intentions ... Jcom-Journal of Science Communication, 21(7), A02. https://doi.org/Artn A02

10.22323/2.21070202

Jensen, P., Rouquier, J.-B., Kreimer, P., & Croissant, Y. (2008). Scientists who engage with society perform better academically. Science and Public Policy, 35(7), 527-541. https://doi.org/10.3152/030234208x329130

Jonker, H., Vanlee, F., & Ysebaert, W. (2022). Societal impact of university research in the written press: media attention in the context of SIUR and the open science agenda among social scientists in Flanders, Belgium. Scientometrics, 1-18. https://doi.org/10.1007/s11192-022-04374-x

Li, Y., Mao, J., Zhang, L., Wang, D., Shen, S., & Huang, Y. (2022, 2022/09/01). How scientific research incorporates policy: an examination using the case of China’s science and technology evaluation system. Scientometrics, 127(9), 5283-5306. https://doi.org/10.1007/s11192-021-04215-3

Niederkrotenthaler, T., Dorner, T. E., & Maier, M. (2011, 2011/07/25). Development of a practical tool to measure the impact of publications on the society based on focus group discussions with scientists. BMC Public Health, 11(1), 588. https://doi.org/10.1186/1471-2458-11-588

Overton. (2023). Overton.io. https://www.overton.io/

Peters, H. P. (2013, 2013/08/20). Gap between science and media revisited: Scientists as public communicators. Proceedings of the National Academy of Sciences, 110(Supplement 3), 14102-14109. https://doi.org/10.1073/pnas.1212745110

Valinciute, A. (2020, Apr). Lithuanian scientists' behavior and views on science communication. Public Underst Sci, 29(3), 353-362. https://doi.org/10.1177/0963662520907001

Vancauwenbergh, S., & Poelmans, H. (2019, 2019/01/01/). The creation of the Flemish research discipline list, an important step forward in harmonising research information (systems). Procedia Computer Science, 146, 265-278. https://doi.org/10.1016/j.procs.2019.01.075