BRIDGE Theory: A New Method Combining Quantitative and Qualitative Evaluation

How to build “new standards” after breaking “Siwei” is a hot and difficult issue in the current reform of research evaluation, which urgently needs good theoretical and methodological support. In this regard, this paper puts forward the BRIDGE theory of research evaluation of scientific researchers’ achievements, which is about to integrate the reasonable elements in the quantitative evaluation based on SCI papers into the “new standard” based on peer review, so as to build a bridge between quantitative analysis and qualitative evaluation. BRIDGE theory is an important development of quantitative and qualitative evaluation method, and has a wide application prospect.


How to Understand Breaking the "Siwei"
The "Siwei" evaluation can be referred to as a quantitative evaluation based on SCI papers, while breaking the "Siwei" evaluation mainly refers to breaking the quantitative evaluation based on SCI papers.But how to break it?There are generally two different views at present.The first viewpoint believes that the existing quantitative methods mainly based on SCI papers are fundamentally wrong and have no cure, and should be immediately abolished.The second viewpoint believes that existing quantitative methods played a positive role under historical conditions at that time and were reasonable, and cannot be completely negated.The difference between the above two perspectives mainly lies in how to view the role of quantitative methods such as bibliometric methods in science and technology evaluation, that is, whether quantitative methods such as bibliometric methods have been used incorrectly in science and technology evaluation?Or is it already unusable?The viewpoint of this study is that quantitative methods represented by SCI papers have been misused, and simple quantitative evaluation methods must be broken; However, the quantitative method, especially the literature metrology method, has its own rationality.If used properly, it can still play an important role in scientific and technological evaluation.In fact, various universities, research institutions, and their subordinate research units in China still have a lot of exploration in this area that combines their own characteristics [1,2,3].How to absorb these practical experiences and make good use of these resources is crucial in breaking the "Siwei" and setting new standards.

Concept of BRIDGE Theory
The core ideas of the BRIDGE theory proposed in this study are as follows: (1) A single peer review method is not the most effective method.Especially for China, due to the limited resources of high-level peer experts and excessive human relationships, a single peer review method cannot be a suitable "new standard".Forcible adoption may lead to a return to the problematic peer review path of the 1980s.
(2) The bibliometric methods represented by SCI papers have been misused and must be 'broken'.However, quantitative evaluation based on SCI papers has reasonable elements that can be retained.In scientific and technological evaluation, more than 30 years of practical exploration and application of bibliometric methods can still play an important role.
(3) Establishing a "new standard" is to combine the reasonable elements of existing quantitative methods into the peer review methods that need to be established.Visually speaking, it is to build a "bridge" between quantitative and qualitative evaluation, thus crossing the "gap" between "breaking" and "establishing" in technological evaluation reform, and achieving a balance between the ideal demand and actual supply of peer review.
(4) The BRIDGE theory is essentially a theory about how to conduct qualitative evaluation based on quantitative analysis, and it is the development of a combination of quantitative and qualitative evaluation methods.One end of the BRIDGE theory is linked to quantitative evaluation methods, emphasizing that evaluation should transform collected data, evidence, information, and other materials into comparable and usable data forms according to certain rules, and try to constrain the interference of interest relationships in a single qualitative evaluation, thus reflecting the evidencebased decision-making concept.The other end of the BRIDGE theory is linked to qualitative evaluation methods, emphasizing the quality of research and the achievement of strategic research objectives.On the one hand, it avoids the shortcomings of simple and mechanical quantitative evaluation, and on the other hand, it leverages the reasonable elements of existing quantitative methods.

Six step method of BRIDGE theory
The BRIDGE theory is not only limited to concepts, but can also be applied in practical operations.Its application in practical technology evaluation mainly includes six steps, called the "six step method".The initial letters of the English keywords in these six steps form "BRIDGE".Taking the achievement evaluation in the performance evaluation of scientific researchers in universities and research institutions (commonly referred to as annual assessment) as an example, a brief introduction to the six step method is as follows.
Balance: Maintain the balance of the performance evaluation system.The performance evaluation system for scientific researchers should be in line with the mission and strategy of the unit, and maintain a balance between the evaluation system and its standards between long-term and short-term goals, between the unit's own development and external competition, between the multiple value attributes of scientific research achievements, and the level of scientific research.
Recode: Obtain quantitative analysis results of the results according to the coding system.There are different coding systems for constructing different types of scientific research achievements, with different levels of standards and corresponding values from high to low order.Based on the achievement information filled in by researchers or collected by the unit evaluation working group, each achievement is converted into scores in the corresponding coding system to obtain quantitative analysis results.
Insight: Implement peer expert evaluation of high-level achievements.Obtain a ranking of results from high to low order through the coding system; Submit the highlevel achievements (A-level achievements) in the ranking to experts for evaluation, and select a list of A-+level achievements through expert evaluation.When conducting expert evaluations of high-level achievements, the unit evaluation working group should also try to provide non coding information on the academic impact, economic benefits, social benefits, and other aspects of scientific research achievements, in order to increase the objectivity of expert evaluations.
Dissent: Establish a mechanism for handling appeals and objections.Considering the difficulty in measuring original scientific research results and the uncertainty of scientific research, it is allowed for researchers to appeal to results obtained through quantitative analysis according to the coding system that are not of high order but believe that the quality impact reaches a high order; After passing the preliminary review by the unit evaluation working group, it can be included in the peer expert evaluation along with high-level achievements.
Guide: Use A+'s achievements as a role model.Publicly display achievements rated as A+through various forms such as online promotional reports, organizing academic conference reports, and posting them on the bulletin board; Thus, while reflecting the transparency of evaluation, the role model and guidance role of A+achievements can be played.

Ecology: Form a list of outstanding personnel list by performing respective duties.
Conducting performance evaluation of scientific research personnel requires a good ecosystem of orderly division of labor among stakeholders such as peer experts and unit management experts: the former is responsible for grasping academic quality, while the latter is responsible for grasping management needs.Based on this, the A+results obtained from peer expert evaluations can serve as a necessary condition for evaluation; Based on management needs, determine the number of A+required for evaluation, the weight relationship of indicators behind A+results, and other relevant factors to consider, and ultimately obtain the list of outstanding personnel.

Conclusion
Research evaluation has always been a hot topic and thorny issue in the reform of China's scientific and technological system [4,5].Since 2018, China has introduced a series of reform policies and measures focused on promoting the reform.Although the reform has been widely welcomed, the effect is not obvious.The problem of "what should be built after breaking 'Siwei'" has not been effectively solved.The deep-seated reason is that we do not have a clear understanding of the law of research evaluation and have not yet put forward effective theories to guide reform practice.
In order to solve the dilemma, this paper attempts to find reasonable elements from the methods to be "broken," integrate them into a new standard to be "built," and then put forward the BRIDGE theory and its six steps, which are mainly aimed at evaluating the achievements of scientific researchers.The BRIDGE theory and its six steps can be summarized in the following four points.
(1) The BRIDGE theory holds that there are reasonable elements that can be retained from the quantitative evaluation method based on SCI papers.If properly used, the experience accumulated in the nearly 30-year exploration of research evaluation in bibliometric methods can still play an important role in building "new standards." (2) The core of BRIDGE theory is to effectively integrate the reasonable elements of the existing quantitative evaluation into the "new standard" that is based on peer review, which is essentially the development of an evaluation theory that combines quantitative analysis and qualitative evaluation.
(3) The practical application of BRIDGE theory is expressed through six steps, in which the second step (recode) and the third step (insight) are two key steps in the achievement of hierarchical expert evaluation based on quantitative analysis.
(4) BRIDGE theory is suitable for China's national conditions, as it achieves a balance between the ideal demand and the actual supply of the peer review system.By conducting peer review based on quantitative analysis methods, it limits the over-broad discretion of peer review, avoids the interference of human relationships in peer review to a considerable extent, and preserves high-level peer expert resources.It also allows the role of reasonable elements in the existing quantitative methods to be leveraged.

Discussion
(1) BRIDGE theory is the development of an evaluation theory that combines quantitative analysis and qualitative evaluation.In the practice of research evaluation, there have been some relatively important theories, including theories of evaluation principles, such as the classification evaluation theory and the hierarchical evaluation theory (also known as the Wildebeest Theory)[6], as well as theories with relatively strong operability, such as the evaluation theory that combines quantitative analysis and qualitative evaluation, the representative work system, the international evaluation theory, and the responsible evaluation theory.It is clear that BRIDGE theory is an extension of the evaluation theory that combines quantitative analysis and qualitative evaluation.Although this combined evaluation theory has been widely recognized, how exactly to combine these two methods has not been determined.This is particularly true regarding a general consensus on a quantitative analysis method that supports qualitative evaluation.BRIDGE theory provides a solution by integrating the reasonable elements in the quantitative evaluation method based on SCI papers, that is, by using bibliometrics as a coding system for quantitative analysis.This can be regarded as a contribution to the theory of research evaluation that has explored quantitative methods for more than 30 years in China.
(2) It can be said that BRIDGE theory is directly derived from the practice of research evaluation.Many units engage in the practice of combining quantitative analysis and qualitative evaluation.For example, in the performance evaluation, recruitment review, and professional title promotion review of scientific researchers, quantitative analysis is implemented through the evaluation of the papers (or representative works) provided by the scientific researchers according to the coding system, such as the journal partition table launched by the National Science Library of the Chinese Academy of Sciences, the core journal of Peking University, or the unit's own journal catalog, used by peer experts.This is actually similar to the key second step (recode) and third step (insight) in BRIDGE theory but lacks the six-step specifications.Therefore, it can be said that BRIDGE theory and its six steps are rooted in practice and have wide application prospects.
(3) Although BRIDGE theory is widely applicable, it cannot encompass all cases.For example, the key to utilizing BRIDGE theory is to construct an index coding system that enables the implementation of quantitative analysis, which has varying degrees of difficulty depending on the different types of achievements.Judging from the existing practice, the indexes such as papers, patents, projects, awards, and talents have their own coding systems.However, for some indexes, such as the social benefits of research achievements, it is difficult to find a suitable coding system at this time.Another example is the unpredictability of scientific research.Some research achievements may take years to produce, or their annual work may not be reflected in the coding system.In such cases, the application of BRIDGE theory should be "patched," that is, exceptions similar to those found above should be included in cases where the fourth step (dissent) of BRIDGE theory is applicable.For example, they may be evaluated separately by a peer expert committee.In addition, BRIDGE theory is not applicable to evaluation work that meets the conditions for directly adopting qualitative evaluation methods, such as international evaluation, or that does not have the convenience of quantitative analysis due to special circumstances.