Public attention to research on Twitter through storytelling: making a narrative out of tweets to a scientific article

Rémi Toupin^*, Florence Millerand^** and Vincent Larivière^***

^*toupin.remi@courrier.uqam.ca

https://orcid.org/0000-0001-6765-0380

Laboratory for communication and the digital (LabCMO), Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Université du Québec à Montréal, C.P. 8888, Succ. Centre-Ville, Montréal, QC, H3C 3P8, Canada

Quantitative Science Studies Lab (QSS Lab), Department of Information Science (DIS), Dalhousie University, Kenneth C. Rowe Management Building, 6100 University Ave, Suite 4010 (4th floor), PO Box 15000, Halifax, NS, B3H 4R2, Canada,

^** millerand.florence@uqam.ca

Département de communication sociale et publique, Laboratory for communication and the digital

(LabCMO), Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Université du

Québec à Montréal, C.P. 8888, Succ. Centre-Ville, Montréal, QC, H3C 3P8, Canada

^*** vincent.lariviere@umontreal.ca

https://orcid.org/0000-0002-2733-0689

École de bibliothéconomie et des sciences de l'information, Université de Montréal C.P. 6128, Succ. Centre-Ville, Montréal, QC, H3C 3J7 Canada and Observatoire des sciences et des technologies, Centre interuniversitaire de recherche sur la science et la technologie (CIRST) Université du Québec à Montréal, C.P. 8888, Succ. Centre-Ville, Montréal, QC, H3C 3P8, Canada

As per the altmetric field, traces left by scholarly documents on social media can be helpful for the evaluation of the societal impact of research. In this study, we explored a storytelling approach based on three methods – network analysis, tweets content analysis and traces interviews – to describe the circulation on Twitter of the paper “Climate change impacts on bumblebees converge across continents” (Kerr et al., 2015). Through this approach, we built a narrative to highlight key events in the dissemination of the Twitter and provide an assessment of its resonance in relation to the communities that engaged with it, for example pollinators conservation organizations or actors of the agricultural sector in Europe. In complementarity with other large-scale methods, the approach developed in our study can provide more contextually based assessments of the public attention to scientific articles, an important endeavour has narrative components become increasingly key to research evaluation.

1. Background

In recent years, technological advancements and sociopolitical developments have contributed to a larger interest for the “social impact” of research. At the crossroads of both dimensions, altmetric research has largely focused on capturing the attention to research based on digital traces left outside of academia, such as social medias or news mention (Holmberg et al., 2019; Robinson-Garcia et al., 2018; Sugimoto et al., 2017). Twitter in particular was the focus of several studies, whether to assess the relations between citations and tweets, study and develop metrics of attention, or examine how research documents circulate on Twitter (Haustein, 2019; Thelwall et al., 2013). As what is measured through indicators of attention to research documents on Twitter remains unclear, there has been a recent shift to move beyond metrics and investigate the contexts of scholarly communication (Araujo, 2020; Arroyo-Machado et al., 2021, 2021; Díaz-Faes et al., 2019). This shift also coincides with a recent uptake of narrative components for the purpose of research evaluation (Bordignon et al., 2023; Leiden Madtrics, 2023).

Meanwhile, environmental issues such as climate change and biodiversity loss garner significant attention in the public sphere as various actors, such as scientists, decision-makers, NGOs, or activists, among others, call for more robust action to prevent dire consequences due to anthropogenic environmental change (Newman, 2017; Ripple et al., 2017, 2022; Walter et al., 2019). In this study, we explore a storytelling approach based on traces left by research documents on Twitter to create narratives that emphasize their resonance. We do so through a case study of the article “Climate change impacts on bumblebees converge across continents” published in the journal Science (Kerr et al., 2015). By relying on a case study, we mobilize a definition of “social impact” at the level of individual articles, thus emphasizing that their resonance is best understood through their specific context of communication (Holmberg et al., 2019).

2. Purpose of the study

In this study, we explore the potential of storytelling to describe how individual scientific articles circulate on Twitter. As such, we contribute to better understand how traces left by scholarly documents in digital environments can be used to provide a contextual and narrative assessment of the social impact of research (Araujo, 2020; Hudson et al., 2022; Weber et al., 2018). This research thus continues to develop a recent trend in altmetrics studies that aim to move beyond metrics to better understand the contexts in which scholarly documents circulate (Arroyo-Machado et al., 2021; Díaz-Faes et al., 2019; Haustein, 2019; Holmberg et al., 2019; Robinson-Garcia et al., 2018).

The approach that we developed to track the attention to singular scientific article builds on the storytelling definition of Dahlstrom (2014) to build narratives that describe the links between various situated events and their impact on specific subjects. Applied to scholarly communication, this approach allows to more finely describe the impact of research objects in the evaluation process, especially as per the current shift to include more qualitative components in research assessment activities (Hudson et al., 2022). In this study, we explore the use of storytelling based on network analysis, tweets content analysis and a trace interview with the main author of the paper “Climate change impacts on bumblebees converge across continents” to highlight how traces left by a research article on Twitter can be helpful to assess its resonance according to the various communities and users that engaged with it. Thus, our research questions are as follows:

RQ1: How can we recreate and describe the circulation of an individual research by using storytelling?

RQ2: How do narratives formulated through this storytelling approach indicative of the public attention to research?

To answer these questions, we focused our analysis on the tweets that included links to the paper “Climate change impacts on bumblebees converge across continents”. As well as generating an important volume of activity on Twitter, this research on the impacts of climate change on bumblebees’ populations in North America and Europe had a significant media potential considering the topics it covered and its publication in Science. By focusing on a singular paper, our study aims to identify the specificities of its communication to provide a targeted and contextual assessment of its attention on Twitter.

3. Data & Methods

Published officially on July 10, 2015, in Science, the article “Climate change impacts on bumblebees converge across continents” shows that the habitable range of bumblebees, important pollinators, declined as a result of climate change (Kerr et al., 2015). The study was led by Jeremy T. Kerr, a professor in macroecology and conservation at the University of Ottawa (UOttawa) and included postdoctoral fellows from UOttawa as well as collaborators from Canada, the US and Europe, mostly as data providers. The project received support from the NSERC (Natural Sciences and Engineering Research Council of Canada) and CANPOLIN. The publication of the paper led to an important press conference by the American Association for the Advancement of Science (AAAS), which resulted in an abundant media coverage in at least 92 news outlets, according to data from Altmetric and Eurêka.cc.

By cross-referencing Clarivate Analytics’ Web of Science (WoS) with Digital Science Altmetric database using the article DOI (Digital Object Identifier), we collected data 408 tweets published by 369 users between July 9, 2015, and June 6, 2017. The Altmetric data was provided as part of a data dump to the Observatoire des sciences et des technologies in October 2018. Collected tweet metadata includes the name and identifier of the user, identifier of the tweet, content, tweet URL as well as time of post. We also collected the Twitter bios of users that either published a tweet or were mentioned. We identified 30 tweets that were not available anymore, either because they were deleted, or their author deactivated their account or were suspended between data collection and analysis. As part of the network analysis, we identified retweets as well as mentions to other users. The resulting network included 406 nodes (users) and 376 edges (retweets or mentions) separated in 66 components and 41 isolated nodes. We also identified 8 larger communities using the Louvain algorithm and produce visuals of the networks in Gephi.

As part of the content analysis of the tweets, we identified the hashtags, the tone and provided a qualitative assessment of the content of the tweet. We also conducted a semi-directed interview using visual support – a network of the circulation of the article on Twitter – with the main author of the paper to elicit its perspective on the publication, communication, and resonance of the paper (Decuypere, 2019; Latzko-Toth et al., 2017; Venturini et al., 2021). When then triangulated all information to recreate the circulation story of the article on Twitter by focusing on key activities, such as tweets that were engaging or attracted more visibility.

4. Results

The majority of the attention toward the article “Climate change impacts on bumblebees converge across continents” on Twitter happened on July 9 and 10, 2015, which coincides with the publication of the article by Science. Most of the attention on Twitter revolved around Science official account (@sciencemagazine), while the other 7 communities detected by using the Louvain algorithm also highlight key aspects of the circulation of the paper on Twitter (Figure 1). One community formed around the main author of the paper (@jetkerr), who was also the most active user with 14 tweets, including 8 retweets and 13 mentions. Other communities in the largest component of the network formed around the account @uottawa and the AAAS communication resources, the co-author of the article @leifr7, as well as the executive director of Buglife @mattashardlow. The three other larger communities developed outside of the largest component, respectively around the accounts @cechr_uod, @b_strawbridge and @agricpolicy.

Figure 1: Circulation network of the research article “Climate change impacts on bumblebees converge across continents” on Twitter based on retweets and mentions.

The first tweet with a link to the paper was published by @leifr7 (Leif Richardson), a co-author on the article and postdoctoral researcher at the University of Vermont at the time of publication. This tweet which emphasizes the publication of the article was retweeted by both the institutional account of the Gund Institute for Environment (@gundinstitute) and its director (@stephen_posner), as well as amateur naturalists like @wurflenii.

The press campaign then picked up as a significant volume of tweets were either retweets of tweets by @sciencemagazine or organized around the public relations services of the AAAS like @scipak or the institutional account of @uottawa. The significance of the press campaign is also highlighted by the main author of the article who argue that it was a key element for its visibility in and out of Twitter.

“[…] Twitter was good to get the message out, but the thing that really catalyzed it all was that the AAAS decided they would do a really organized press event around the paper, and so thew flew here […] and organized a big international press conference on campus”.

“Otherwise, it was going to be another, you know, kind of a meaningful discovery published in a high-profile journal but whose splash was very short. […] Twitter can amplify it in that way, but the nucleus of it was, I think, that Science did the press conference”.

The attention to the article on Twitter significantly gained pace with a tweet from @sciencemagazine highlighting its main conclusion along with the hashtag #free underlining its availability through open access. This particular tweet was retweeted 76 times by scientists (@ameurcherif), professionals outside of academia (@clay_comstock), and users from the general public (stephenwaldron) alike. Work by the AAAS to promote the paper continued for several days, for example through tweets by its news services @eurekalert or a new sharing by @science magazine a week later on July 15.

Meanwhile, two accounts, @alex_smith_ants and @matteashardlow, contributed significantly to the outreach and resonance of the article in their own way. On the one hand, @alex_smith_ants, a professor of molecular ecology at the University of Guelph, published on July 9 a tweet that included a link to a video prepared by York University featuring two co-authors, Sheila Colla and Laurence Packer, while highlighting the role of natural history collections in realizing the study. This tweet also mentioned @jetkerr and @leifr7 and got the attention @ianwmking, a postdoctoral fellow at the University of Guelph prior to becoming science advisor with the Government of Canada. On the other hand, @matteashardlow, executive director of Buglife, takes a more alarmed tone to emphasize that bumblebees are disappearing due to climate change. He then highlights a process developed by is company, the B-Lines, aiming to organize networks of wildflowers to recreate natural habitats for pollinators. This tweet contributed to the creation of a distinct community of entomologists, ecologists and conservation-sensitive users connected to the main component through @matteashardlow (in red, at the bottom of figure 1). Several colleagues and collaborators to the authors also acknowledged their work, such as @bio_diverse, a professor at McGill University, or @jayfitzy, a biologist with the Government of Ontario.

The first Twitter activity of the main author of the paper, @jetkerr, happened in the night of July 9 as a retweet from one of his colleagues at the University of Ottawa, @reeskassen, who acknowledged the importance of the research while emphasizing that “Bumblebees feeling squeezed by climate change. Imp new study by @JeTKerr”. This tweet was also retweeted by the accounts @bio_diverse, @alex_smith_ants and @aaasfellowships, an account representing science policy interns with the AAAS.

In July 2015, a significant part of the tweets came from users based in Europe, such as the professor of evolutionary history at the University of Copenhagen, @pckjaergaard, or the research team on urban pollinators, @readingurbpolls. Significant communities revolved around tweets by @nvonwestenholz, executive director of the Crop Protection Association in Peterborough, UK, and @agricpolicy, chief arable adviser for the National Farmers’ Union.

Tweets started to accumulate at a much slower pace late in July 2015, indicating the critical period of attention to the paper was rather short. Nevertheless, some key events contributed to bursts of attention, such as a tweet by the Centre for Environmental Change and Human Resilience of the University of Dundee (@cechr_uod) on August 1,^, 2015. Outside of the main component, this tweet was retweeted by 29 users, mostly working in the sector of environmental resilience, which resulted in a distinct community of attention. The potential of Twitter to reach new communities was also highlighted by the author.

“Twitter probably served a really important amplification purpose there, and also because of its networking function reached into places where we could just never have thought to go, you know, all over the place.”

Among other usages, the article was shared by users sensitive to environmental issues to attract the attention of policy-makers. For example, on July 10, 2015, @kevinjcoleman called out to @danarohrbacher – republican U.S congressman – by sharing the article in a tweet emphasizing that bumblebees populations are disappearing. In January of 2017, other users called out to the accounts @potus – used by the U.S. president Donald Trump at the time – and @lamarsmithtx21, another republican U.S. congressman, in the same manner.

5. Discussion and conclusion

By triangulating observations from three different set of methods – retweets and mentions network analysis, qualitative tweet content analysis and interviews with visual support –, our study proposes a narrative-based approach to describe the circulation and impact of research articles on Twitter. As social impact evaluation can be best understood and operationalize in the context of individual research article (Holmberg et al., 2019), we explored our approach through a case study of the article “Climate change impacts on bumblebees converge across continents”, an article covering the decline of bumblebees’ populations due to climate change, which attracted a lot of attention on Twitter while benefiting from an important press campaign. Through our case study, we observed the key role played by the AAAS and Science, whether through their press campaign or by actively engaging authors in the communication process, in structuring the circulation of the paper. Our narrative also highlighted how distinct communities could have revolved around specific users, thus fostering a specific resonance for these communities (Arroyo-Machado et al., 2021). For example, the distinct resonance that can be accounted from Twitter users in the agricultural sector or from conservation NGOs adds to qualitative assessment of the article’s social impact.

The press campaign organized by the AAAS in collaboration with the University of Ottawa reaffirms the key role of medias and public relations for the circulation of research in the public sphere. On Twitter, colleagues of the authors, such as @alex_smith_ants and @bio_diverse, mobilized some of the press material to acknowledge the importance of the study and share it to a broader audience. Sharing the article to a broader audience also allowed certain users to mobilize it in Twitter discussions, for example to call out to politicians and decision-makers. Future research could dive deeper into this by investigating the intricate relationship between different media sources for the purpose of knowledge mobilization.

By using a storytelling approach to describe the key events of the circulation of a research article on Twitter, we observed that distinct communities of attention may engage with it. Using Twitter may also serves to amplify the circulation of a research document to new communities or groups that wouldn’t have been made aware otherwise. In our case study, this can be observed through communities that developed around tweets by @agricpolicy or @cechr_uod and which attracted significant attention but are otherwise disconnected from the main component of the circulation network. These communities materialized a Twitter engagement by users based in the UK and working respectively in the crop and agricultural sector as well as in environmental resilience. Other communities formed around the main author (@jetkerr) and one co-author of the article (@leifr7), each gaining mutual attention from several colleagues while also engaging in specific circles.

While narrative approaches are increasingly critical for research evaluation, we recognize the limits to the approach proposed in this study. First, storytelling and case studies such as this one focus on only one article whose resonance is specific and contextualized. While this as the merit of providing a more precise assessment of the attention to individual articles, we recognize that this assessment is valid only in the context of this article. However, the storytelling approach and the triangulation of network analysis, content analysis and interviews can be generalized to other assessment activities and could be helpful for the purpose of narrative-driven research evaluation (Leiden Madtrics, 2023). Readings and interpretations may also differ to some degree between observers, for example by emphasizing certain events over others depending on the motivations behind the assessment or knowledge of the field around the article. Finally, we conducted only one interview for this study. The development of research evaluation activities based on this approach could benefit from multiple interviews, for example with members of the publishing organization that are directly involved in the communication process.

The storytelling approach that we explored in this study builds on previous work in altmetrics and social media scholarly communication to go beyond metrics and provide more contextualized assessment of the traces left by scholarly documents in digital environments (Araujo, 2020; Arroyo-Machado et al., 2021; Díaz-Faes et al., 2019; Haustein, 2019; Holmberg et al., 2019; Robinson-Garcia et al., 2018). Within the limits of the method, this approach brings a complementary view to large-scale studies by mobilizing different tools, like network analysis, content analysis and interviews with visual support, to look at Twitter traces left by a single research article in a social impact assessment perspective. Finally, this approach opens new areas of research, such as the articulation of media sources for social impact assessment purposes or reflections on the nature of attention and impact based on traces left by scholarly documents online.

Open science practices

We firmly believe that open science and open data practices are crucial for effective research assessment activities. While this research is mostly based on proprietary data from Altmetric (Twitter) and the Web of Science, we argue that traces left by scholarly documents in medias and digital environments should be more readily available to researchers or librarians to examine who share relevant research articles and how it is shared. As such, we made our aggregated data from the network analysis and tweet content analysis available on Zenodo (https://zenodo.org/record/7853072) along with the material produced for the purpose of this study. Other relevant material can be found on the Github account of the main author of this paper (https://github.com/toupinr).

Acknowledgments

The authors would like to thank Philippe Mongeon (QSS Lab) as well as Stefanie Haustein, and Juan Pablo Alperin from the ScholCommLab for their help and feedback regarding some aspects of the research. They would also like to thank Jeremy T. Kerr for his contribution to the study.

Author contributions

RT: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Investigation, Methodology, Project administration, Validation, Visualization, Writing – original draft, Writing – review & editing

FM: Conceptualization, Formal Analysis, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing

VL: Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing

Competing interests

The authors have declared that no competing interests exist.

Funding information

This research was funded through a SSHRC Joseph-Armand Bombardier Canada Graduate Scholarship (767-2017-1329), a Coalition Publica Scholarship, as well as CIRST and FSH-UQAM Thesis Writing Scholarships for RT. It also received contributions from the Espace UN (Data Analysis), Observatoire des sciences et des technologies (OST) and Canada Research Chair on the Transformations of Scholarly Communication (Data Collection).

References

Araujo, R. F. (2020). Communities of attention networks: Introducing qualitative and conversational perspectives for altmetrics. Scientometrics, 124(3), 1793–1809. https://doi.org/10.1007/s11192-020-03566-7

Arroyo-Machado, W., Torres-Salinas, D., & Robinson-Garcia, N. (2021). Identifying and characterizing social media communities: A socio-semantic network approach to altmetrics. Scientometrics, 126, 9267-9289. https://doi.org/10.1007/s11192-021-04167-8

Bordignon, F., Chaignon, L., & Egret, D. (2023). Promoting narrative CVs to improve research evaluation? A review of opinion pieces and experiments. Research Evaluation, rvad013. https://doi.org/10.1093/reseval/rvad013

Dahlstrom, M. F. (2014). Using narratives and storytelling to communicate science with nonexpert audiences. Proceedings of the National Academy of Sciences, 111(Supplement_4), 13614–13620. https://doi.org/10.1073/pnas.1320645111

Decuypere, M. (2019). Visual Network Analysis: A qualitative method for researching sociomaterial practice. Qualitative Research, 20(1), 73–90. https://doi.org/10.1177/1468794118816613

Díaz-Faes, A. A., Bowman, T. D., & Costas, R. (2019). Towards a second generation of ‘social media metrics’: Characterizing Twitter communities of attention around science. PLOS ONE, 14(5), e0216408. https://doi.org/10.1371/journal.pone.0216408

Haustein, S. (2019). Scholarly Twitter Metrics. In W. Glänzel, H. F. Moed, U. Schmoch, & M. Thelwall (Eds.), Springer Handbook of Science and Technology Indicators (pp. 729–760). Springer.

Holmberg, K., Bowman, S., Bowman, T., Didegah, F., & Kortelainen, T. (2019). What Is Societal Impact and Where Do Altmetrics Fit into the Equation? Journal of Altmetrics, 2(1), Article 1. https://doi.org/10.29024/joa.21

Hudson, C. G., Knight, E., Close, S. L., Landrum, J. P., Bednarek, A., & Shouse, B. (2022). Telling stories to understand research impact: Narratives from the Lenfest Ocean Program. ICES Journal of Marine Scienc,e 80(2), 394-400. https://doi.org/10.1093/icesjms/fsac169

Kerr, J. T., Pindar, A., Galpern, P., Packer, L., Potts, S. G., Roberts, S. M., Rasmont, P., Schweiger, O., Colla, S. R., Richardson, L. L., Wagner, D. L., Gall, L. F., Sikes, D. S., & Pantoja, A. (2015). Climate change impacts on bumblebees converge across continents. Science, 349(6244), 177–180. https://doi.org/10.1126/science.aaa7031

Latzko-Toth, G., Bonneau, C., & Millette, M. (2017). Small Data, Thick Data: Thickening Strategies for Trace-based Social Media Research. In The Sage Handbook of Social Media Research Methods (1st edition, pp. 199–214). SAGE Inc.

Leiden Madtrics. (2023, March 15). Narrative CVs: A new challenge and research agenda. https://www.leidenmadtrics.nl/articles/narrative-cvs-a-new-challenge-and-research-agenda

Newman, T. P. (2017). Tracking the release of IPCC AR5 on Twitter: Users, comments, and sources following the release of the Working Group I Summary for Policymakers. Public Understanding of Science, 26(7), 815–825. https://doi.org/10.1177/0963662516628477

Ripple, W. J., Wolf, C., Gregg, J. W., Levin, K., Rockström, J., Newsome, T. M., Betts, M. G., Huq, S., Law, B. E., Kemp, L., Kalmus, P., & Lenton, T. M. (2022). World Scientists’ Warning of a Climate Emergency 2022. BioScience, 72(12), 1149–1155. https://doi.org/10.1093/biosci/biac083

Ripple, W. J., Wolf, C., Newsome, T. M., Galetti, M., Alamgir, M., Crist, E., Mahmoud, M. I., Laurance, W. F., & 15,364 scientist signatories from 184 countries. (2017). World Scientists’ Warning to Humanity: A Second Notice. BioScience, 67(12), 1026–1028. https://doi.org/10.1093/biosci/bix125

Robinson-Garcia, N., van Leeuwen, T. N., & Ràfols, I. (2018). Using altmetrics for contextualised mapping of societal impact: From hits to networks. Science and Public Policy, 45(6), 815-826. https://doi.org/10.1093/scipol/scy024

Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. https://doi.org/10.1002/asi.23833

Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do Altmetrics Work? Twitter and Ten Other Social Web Services. PLoS ONE, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841

Venturini, T., Jacomy, M., & Jensen, P. (2021). What do we see when we look at networks: Visual network analysis, relational ambiguity, and force-directed layouts. Big Data & Society, 8(1), 20539517211018490. https://doi.org/10.1177/20539517211018488

Walter, S., Lörcher, I., & Brüggemann, M. (2019). Scientific networks on Twitter: Analyzing scientists’ interactions in the climate change debate. Public Understanding of Science, 28(6), 696–712. https://doi.org/10.1177/0963662519844131

Weber, W., Engebretsen, M., & Kennedy, H. (2018). Data stories: Rethinking journalistic storytelling in the context of data journalism. Studies in Communication Sciences, 2018(1), 191–206. https://doi.org/10.21256/zhaw-4873