Evaluation of the effectiveness of scientific research. Evaluation of the scientific and scientific-technical effectiveness of R&D Efficiency of R&D

unit of measurement

Parameter values

The scientific effect is characterized by obtaining new scientific knowledge

and reflects the increase in information intended for internal consumption.

The scientific and technical effect characterizes the possibility of using

the results of ongoing research in other R&D and provides

obtaining information necessary for the creation of new technology.

The economic effect is characterized by the saving of living and materialized labor in social production, expressed in terms of cost, obtained by using the results of research.

The social effect is manifested in the improvement of working conditions, etc.

For the final assessment of the results of research, depending on the type of research being performed and the goals set, one of the types of effect is taken as an efficiency criterion, and the rest are used as additional characteristics.

Evaluation of the scientific effectiveness of R&D is carried out using coefficients.

Evaluation of scientific and technical performance is carried out using coefficients.

where K zni is the coefficient of significance of the i-th factor used for

TO dui– coefficient of the achieved level of the i-th factor;

n is the number of scientific performance factors;

m - the number of factors of scientific and technical performance;

When evaluating scientific and scientific - technical performance, various factors are used that affect their quantitative assessment.

For each of the factors, experts establish numerical values significance factor.

The sum of the significance coefficients for all factors can be equal to 1.0.

The coefficient of the achieved level of the factor is established by expert means, and its numerical values are determined taking into account the quality of the factor attribute and its characteristics Cd ≤1.0. The maximum possible value of Knr and Cntr is 1.0.

The closer KNR and KNR to 1.0, the higher the scientific and scientific-technical effectiveness of R&D. Table 9 shows the factors and features that characterize the scientific performance of R&D, and Table 10 shows the factors and features that characterize the scientific and technical performance of R&D. Specific values of these coefficients in relation to the research being carried out can only be established on the basis of experience and knowledge scientists and should be determined by the student with the direct participation of the supervisor of the thesis.

Table 9

Characteristics of the factors and signs of the scientific effectiveness of research.

Scientific performance factor	Kzn	Factor quality	Factor characteristic	Kdu
Novelty of the obtained or expected results	0,5	High Medium Insufficient Trivial	Fundamentally new results, previously unknown to science, have been obtained, a new theory has been created, a new regularity has been discovered. general patterns, methods, methods that allow creating fundamentally new types of technology Positive solution of the tasks set on the basis of simple generalizations, analysis of relationships between factors, extension of known scientific principles to new objects Description of individual elemental facts, transfer and dissemination of previously obtained results, abstract reviews	1,0 0,7 0,3 0,1

End of Table 9

Depth of scientific study	0,35	High Medium Poor	Performing complex theoretical calculations, the results are verified on a large amount of experimental data The complexity of theoretical calculations is low, the results are verified on a limited amount of experimental data Theoretical calculations are simple, experimental verification has not been carried out	1,0 0,6 0,1
Probability of success.	0,15	Major Moderate Minor	Success is very possible, there is a high probability of a positive solution to the tasks set The tasks are theoretically and technically feasible, success is possible Theoretically feasible, the idea is risky, success is doubtful	1,0 0,6 0,3

Table 10

Characteristics of factors and signs of scientific and technical effectiveness of R&D

End of table 10

		Useful	The results will be used in subsequent research and development, in the development of new technical solutions in a specific industry	0,5
Scale of possible implementation	0,3	N/X Sectoral Individual organizations and enterprises	Implementation time, years: up to 3 >5 >10 over 10 Implementation time, years: up to 3 >5 >10 over 10 Implementation time, years: up to 3 >5 >10 over 10	1.0 0,8 0,6 0,4 0,8 0,7 0,5 0,3 0,4 0,3 0,2 0,1
Completeness of the obtained results	0,2	High Average Adequate Insufficient	Methodology, instruction, guidance materials, classifier, standards Terms of reference for applied R&D. Recommendations, detailed analysis, suggestions. Review, information collection.

The expected economic effect of R&D is characterized by the maximum economic effect that can be achieved based on the implementation of the results of this work.

The assessment of the social effect of R&D should be based on a long-term forecast. Some social results achieved on the basis of R&D can be quantified in the appropriate units adopted to measure this result. These results include: work safety, clean air in the room, noise level, illumination, etc. When implementing the results of research aimed at solving social problems, an economic effect is achieved - an increase in labor productivity, the use of labor resources, etc., so it is necessary to determine:

1) coefficient characterizing the performance of work on time

Кс = Тf/Тn, (13)

where Тf is the actual duration of R&D, days;

Tn is the planned duration of R&D, days;

2) labor resource utilization rate

Kt \u003d Trf / Trn, (14)

where Трф is the actual laboriousness of performing R&D, man-days;

Трn is the planned labor intensity of R&D, pers. – day;

3) The coefficient of efficiency of the expenditure of funds.

Kd \u003d Sf / Sn, (15)

where Sf is the actual cost of R&D, rub.

Sn is the planned cost of R&D, rub.

All indicators used to calculate the listed coefficients are closely interconnected.

Reducing the labor intensity of conducting research leads to a reduction in the timing of its implementation. This leads to a decrease in the planned cost value due to a decrease in the basic salary and cost items calculated as a percentage of the basic salary. The presentation includes sheets of a tape or network graph, costing of R&D, coefficients of scientific and scientific and technical performance.

Bibliography

1. Sachko N.S. Organization and operational management of machinery

construction industry: textbook. - 3rd ed. correct - M.:

New Knowledge LLC, 2008.-635s.

2. Organization and planning of engineering production

textbook, ed. Skvortsova. Yu. and Nekrasova L.A. - M.: Higher

school, 2006. -470 p.

3. Feasibility study graduation projects:

ed. Bekleshova V.K. - M.: high school, 1991.-176 p.

Introduction

1. Product Description 2

2. Marketing research 2

3. Determination of the complexity of the implementation of R&D 3

4. Construction of a tape and network schedule for the implementation of research 4

5. Determination of the planned cost of R&D 11

6. Determination of profit and contract price 15

7. Assessment of scientific and technical performance and social

R&D efficiency 15

10. Organization and procedure for the implementation of research

10.3. Methods for assessing the scientific and technical effectiveness of research

The result of research is the achievement of scientific, scientific-technical, economic and social effects. The scientific effect is characterized by the acquisition of new scientific knowledge and reflects the increase in information intended for "intra-scientific" consumption. The scientific and technical effect characterizes the possibility of using the results of ongoing research in other R&D and provides the information necessary to create new products. The economic effect characterizes the commercial effect obtained by using the results of applied R&D. The social effect is manifested in the improvement of working conditions, the improvement of economic characteristics, the development of culture, health care, science, and education. Scientific activity is multifaceted. Its results, as a rule, can be used in many areas of the economy for a long time.

Evaluation of the scientific and scientific and technical effectiveness of R&D is carried out using a system of weighted scoring. For fundamental R&D, only the coefficient of scientific effectiveness is calculated (Table 10.3), and for exploration work, the coefficient of scientific and technical effectiveness is calculated (Table 10.4). Estimates of coefficients can only be established on the basis of the experience and knowledge of scientists who are used as experts. The evaluation of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 10.3

Characteristics of factors and signs of scientific effectiveness of R&D

Factor of scientific effectiveness	Coef. factor significance	Factor quality	Factor characteristic	Coef. reached level
The novelty of the results obtained			Fundamentally new results, new theory, discovery of a new regularity
			Some general patterns, methods, ways to create a fundamentally new product
		Insufficient	Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects
		Trivial	Description of individual factors, dissemination of previous results, abstract reviews
Depth of scientific study			Performing complex theoretical calculations, testing on a large amount of experimental data
			Low complexity of calculations, verification on a small amount of experimental data
		Insufficient	Theoretical calculations are simple, the experiment was not carried out
Degree of probability of success
		Moderate

Table 10.4

Characteristics of factors and signs of scientific and technical effectiveness of R&D

Factor of scientific and technical effectiveness	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. reached level
The prospect of using the results		paramount foam	The results can be applied in many scientific directions
			The results will be used in the development of new technical solutions
		Useful	The results will be used in subsequent research and development
Scale of implementation of results		National economy	Implementation time: over 10 years
			Implementation time: over 10 years
		Individual firms and enterprises	Implementation time: over 10 years
Completeness of results			Terms of Reference for R&D

		Insufficient	Overview, information

In this case, the coefficient of scientific and technical effectiveness is determined by the formula

where k is the number of estimated parameters; - coefficient of influence of the i-th parameter on scientific and technical performance; - coefficient of relative increase of the i-th parameter in comparison with the base value.

For the convenience of performing calculations, the data are summarized in Table. 10.5.

Table 10.5

Assessment of the scientific and technical effectiveness of applied R&D

Efficiency scientific research is established taking into account the various types of manifestation of the beneficial effect that can be obtained when using the results of research. These types of effects can be: a) social effect - is manifested in increasing safety for human life and health, labor safety, in improving working conditions, reducing the likelihood of occupational diseases, increasing environmental safety and in obtaining other socially significant results b) defensive effect - characterizes the significance of the research results for ensuring the country's defense capability. Performance indicators in this case can be: the probability of completing a combat mission, the degree of protection of objects from enemy damage, the level of preservation of state, industrial, military secrets, etc. d.; in) economic effect - characterizes the valuation of the effectiveness of scientific research, manifested in a decrease in the cost of products, works, services, profit growth; G) scientific and technological effect - characterized as the accumulation of new knowledge in the field of properties and phenomena of the material world, manifested in the form of scientific publications, reports, discoveries, defended dissertations, inventions.

Quantification usually lends itself to the economic impact expected from applied research and development. The basis of such an assessment is a comparison of costs (estimated cost of the topic) and results (expected reduction in the cost of production). When calculating other types of effect, the peer review method is often used. For example, when assessing the scientific and technical effect, it may be recommended to use the scientific and technical effect coefficient Ht:

where m is the number of signs of scientific and technical effect;

ri - scoring of the i-th attribute of the scientific and technical effect;

bi is the level of significance of the i-th feature of the scientific and technical effect. Tables 1 and 2 show typical signs of scientific and technical effect and approximate values of ri, bi.

Table 1 - Point values of the signs of the scientific and technical effect

Signs of scientific and technological effect		Point values ri
1. Expected level of novelty of R&D results	a) fundamentally new
	b) relatively new
	c) not new
2. Theoretical level	a) establishment of a new law, theory
	b) deep study of the problem
	c) development of a method, method, program
3. Possibility of practical use of the results	a) within 1 - 2 years
	b) within 3 - 5 years
	c) for 5 years or more
	d) indefinite

Table 2 - Significance of signs of scientific and technical effect

THEORY AND PRACTICE OF MANAGEMENT

Dr. tech. Sciences A. V. Ryzhakova

cand. economy S. V. Manakhov

EVALUATION OF THE EFFICIENCY OF THE RESEARCH

WORKS IN UNIVERSITY1

The article considers the fundamental factors of efficiency (efficiency) of fundamental and applied research work carried out by modern university. A brief description of some methods for evaluating the results of research work is given, a method for calculating the integral indicator that reflects the quality of the results of the scientific activity of the university is proposed. A list of indicators is given, on the basis of which it is possible to evaluate the dynamic rating of the effectiveness of R&D.

Key words and phrases: research work (R&D), fundamental and applied R&D, R&D effectiveness, integral indicator, scientific and technical level indicator, dynamic evaluation of R&D effectiveness.

Scientific work is one of the most important activities of the teaching staff, graduate students and students of any higher educational institution. Holding scientific work at the university provides continuous improvement of the educational process based on fundamental and applied research in the areas of training specialists and implementation in educational activities modern techniques and pedagogical technologies.

The university status of a higher educational institution implies:

Training of highly qualified personnel in a wide range of areas of science, production and public life through postgraduate and doctoral studies;

Participation in the educational process of scientific and pedagogical personnel of the highest qualification;

The presence of generally recognized scientific schools, significant achievements in the field of fundamental and applied research, scientific and methodological innovations;

Raising the scientific qualifications of scientific and pedagogical personnel;

Dissemination of scientific knowledge, cultural and educational activities.

Thus, one of the main tasks of universities is to organize and conduct fundamental and applied scientific research. The results of fundamental scientific research in this case are

1 The article was prepared as part of the research work "Development of a mechanism for project management of the scientific activities of the university" under the state assignment of the Ministry of Education and Science Russian Federation in 2013

theory, method, hypothesis, and applied scientific research - methodology, algorithm, technology, device, installation, device, mechanism, substance, material, product, system (management, regulation, control, design, information), software, database .

The presence of a well-established mechanism for the implementation of regular analysis and evaluation of the results of research work is the most important element in the effective management of the scientific activities of the university. At the same time, if the quantitative assessment of scientific research conducted at the university is easily feasible, then characterizing the quality of research work performed is a difficult task.

The quality and efficiency of research work, the composition of the indicators used and the accuracy of the calculation of the latter depend on the following factors:

The nature of the results obtained;

Liaison with practical bodies;

The scale of R&D implementation;

The duration of the period of research and development and the stage at which the calculation is made (planning, implementation, implementation).

There are features of determining the effectiveness of research work, depending on their type. According to fundamental research, in the course of a comprehensive analysis, the scientific effect is first of all established. If the results of R&D in the future may affect the social side of public life (which is important, for example, for the penitentiary system), then the content of a comprehensive analysis of R&D should include a qualitative assessment of these components of the overall effect.

In addition, the characterization of the effect of fundamental work carried out at the university should include the established impact of these works on the quality of education at the university. The economic effect and economic efficiency of R&D expenditures on fundamental research usually cannot be calculated. In some cases, only a qualitative forecast of the economic rationality of R&D, possible in the future, can be given.

Applied (exploratory) R&D primarily reveals scientific and technical effects. The possibility of analyzing the organizational feasibility of R&D depends on its type. For applied work related to material production, performed on the basis of the use of new phenomena and patterns discovered during the fundamental research, the economic effect of implementing their results and the economic efficiency of R&D costs, as a rule, cannot be calculated. Only the production and other possibilities of using R&D are clarified, and on the basis of a qualitative analysis, a characteristic of their alleged economic significance is given. This determines the range of indicators that change under the influence of the implementation of the results of the work, as well as the possible range of these changes.

Thus, it should be noted that far from every research work, the analysis of rationality can be brought to the calculation of indicators of economic effect.

Currently, a fairly large number of different methods for assessing the effectiveness of research work are known, but there are no unified methods for assessing the results of research work, so the problem of their development is very relevant for modern universities.

The development and application of methods for assessing the economic efficiency of research and development work is one of the most important levers for accelerating scientific and technological progress, concentrating scientific potential in the most important areas of development of the national economy and scattering material, financial and human resources.

The main indicator of the quality of the results of scientific activity of the university, in our opinion, is the indicator of the scientific and technical level (STL) as a generalizing characteristic of the prospects and progressiveness of scientific and technical solutions incorporated in the development, in comparison with existing and previously created ones that belong to one scientific industries. The definition of NTU is made complex method based on an integrated assessment as a function of private assessments (indicators) and weight coefficients characterizing the importance of each indicator.

The result of research is the achievement of a scientific and scientific-technical effect. Their quantitative values are determined using a system of weighted scoring for a group of scientific performance factors. Moreover, these factors differ for fundamental and applied R&D. The most general list of brief description and possible values of the coefficients are presented in Table. 1 and 2.

For fundamental R&D, only the coefficient of scientific effectiveness is calculated (in accordance with the initial data of Table 1), and for applied work, the coefficient of scientific and technical effectiveness is also calculated (in accordance with the initial data of Table 2).

The coefficient of scientific (scientific and technical) effectiveness is determined by the formula

E = £ ErKr,

where k is the number of estimated parameters;

E7 - coefficient of significance of the factor (influence of the 7th parameter on scientific (scientific and technical) performance);

K is the coefficient of the achieved level of the 7th parameter.

In accordance with the proposed methodology, the coefficients are estimated only on the basis of the experience and knowledge of scientists who are used as experts. The assessment of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 1

Characteristics of factors and signs of scientific effectiveness of fundamental R&D

Factor of scientific effectiveness Coefficient of significance of the factor Quality of the factor Characteristics of the factor Coefficient of the achieved level

Novelty of the obtained results 0.5 High Fundamentally new results, new theory, discovery of a new regularity 1.0

Medium Some general patterns, methods, ways to create a fundamentally new product 0.7

Insufficient Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects 0.3

Trivial Description of individual factors, dissemination of previous results, abstract reviews 0.1

Depth of scientific study 0.35 High Performance of complex theoretical calculations, verification on a large amount of experimental data 1.0

Medium Low complexity of calculations, verification on a small amount of experimental data 0.6

Insufficient Theoretical calculations are simple, the experiment was not carried out 0.1

Degree 0.15 Large - 1.0

probability of success Moderate - 0.6

Small - 0.1

Table 2

Characteristics of factors and signs of scientific and technical performance

applied research

Factor Coefficient Quality Characteristic Coefficient

scientific and technical factor factor factor factor

the effectiveness of the significance of the factor of the achieved level

Promising 0.5 Highly Possible Results 1.0

use find application

results in many scientific fields

Important The results will be used in the development of new technical solutions 0.8

Useful Results will be used in subsequent research and development 0.5

Implementation scale 0.3 National Implementation time:

economic results up to 3 years up to 5 years up to 10 years over 10 years 1.0 0.8 0.6 0.4

Industry Implementation period: up to 3 years up to 5 years up to 10 years over 10 years 0.8 0.7 0.5 0.3

Individual firms and Implementation time: up to 3 years 0.4

enterprises up to 5 years up to 10 years over 10 years 0.3 0.2 0.1

Completion 0.2 High Terms of Reference- 1

results on experimental - design work

Insufficient Overview, information 0.4

Evaluation of the effectiveness of research should be carried out at different stages of the implementation of a research project.

The initial evaluation of efficiency (planned efficiency) takes place at the stage of formation of the terms of reference (TOR). It is reflected in the explanatory note and draft TOR, which necessarily describe the planned results and analyze their value in terms of common tasks university.

Evaluation of effectiveness at the time of delivery and defense of work (or stage) is reflected in the minutes of meetings of commissions for the acceptance of research results, in particular, in the form of recommendations for further use with an assessment of expected beneficial effects.

Evaluation of the effectiveness of research after its formal completion is of particular relevance and significance for modern Russian universities. Unfortunately, many R&D ends at the stage of a research report and payment for the work of its participants. The further scientific activity of the department or scientific unit in most universities of the country is not tracked in relation to those R&D that were performed at the department or with the participation of the department or scientific unit. We will call this type of assessment a dynamic R&D rating.

Number of monographs on research topics;

Volume printed sheets in published monographs highlighting the results of a specific R&D;

Number of articles published on research topics;

The number of Ph.D. theses defended using the results of research, indicating the coefficient of use of research results for each dissertation (from 0 to 1);

The number of doctoral dissertations defended using the results of research, indicating the coefficient of use of research results for each dissertation (from 0 to 1);

The volume of work performed on R&D in financial terms (total cost of R&D);

Number of textbooks and teaching aids using the results of research;

Quantity academic disciplines using the results of research;

The number of patents registered based on the results of research;

Number of registered software products created on the basis of R&D results;

Number of new bids submitted based on R&D results;

Number of winning applications created based on results

total amount winning applications created based on the results

Dynamic evaluation of effectiveness may fall if there are no publications, patents, no development, no implementation, the results did not form the basis of dissertations, there are no customers for the results, etc.

Publication indicators;

Indicators of scientific qualification of scientific and pedagogical workers (candidate and doctoral dissertations);

Qualities educational process;

Degrees of commercialization of research results;

The activity of the most qualified part of scientific and pedagogical workers (the desire to enter and further increase the rating of scientific supervisors) in terms of searching for the most promising research topics;

The volume of research carried out, in particular, with the involvement of external sources of financing.

The use of a dynamic R&D rating by higher education institutions will make it possible to more fully ensure continuity between the stages of research work, in particular, between the stage of research development and the stage of commercialization of scientific knowledge.

Bibliography

1. Butenko Ya. A. Implementation of project management in the scientific activity of the university // Bulletin of the Russian University of Economics named after G. V. Plekhanov. - 2013. - No. 9 (63).

2. Manakhov S. V. The development of scientific activity in universities - a new priority of the state scientific and technical policy of Russia // Bulletin of the Plekhanov Russian University of Economics. - 2013. - No. 8 (62).

3. Maslennikov V.V. Organizational Models project management of scientific activities in Russian universities// Bulletin of the Plekhanov Russian University of Economics. - 2013. - No. 9 (63).

4. Saginova O. V., Zavyalova N. B., Shtykhno D. A. Communication of scientific research and educational process in universities // Bulletin of the Plekhanov Russian University of Economics. - 2012. - No. 12 (54).

7.3. Methods for assessing the scientific and technical effectiveness of research

Scientific activity is multifaceted, its results, as a rule, can be used in many areas of the economy for a long time.

Evaluation of the scientific and scientific and technical effectiveness of R&D is carried out using a system of weighted scoring. For fundamental R&D, only the coefficient of scientific effectiveness is calculated (Table 7.3), and for exploration work, the coefficient of scientific and technical effectiveness is also calculated (Table 7.4). Estimates of coefficients can only be established on the basis of the experience and knowledge of scientists who are used as experts. The evaluation of the scientific and technical effectiveness of applied R&D is carried out on the basis of a comparison of the technical parameters achieved as a result of the R&D with the basic ones (which could have been implemented before the R&D was completed).

Table 7.3

Characteristics of factors and signs of scientific effectiveness of R&D

Scientific performance factor	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. achieved level
The novelty of the results obtained			Fundamentally new results, new theory, discovery of a new regularity
			Some general patterns, methods, ways to create a fundamentally new products
		Insufficient	Positive decision based on simple generalizations, analysis of factor relationships, extension of known principles to new objects
		Trivial	Description of individual factors, dissemination of previous results, abstract reviews
Depth of scientific study			Performing complex theoretical calculations, testing on a large amount of experimental data
			Low complexity of calculations, verification on a small amount of experimental data
		Insufficient	Theoretical calculations are simple, the experiment was not carried out
Degree of probability of success
		Moderate

Table 7.4

Characteristics of factors and signs of scientific and technical effectiveness of R&D

Factor of scientific and technical effectiveness	Coef. the significance of the factor	Factor quality	Factor characteristic	Coef. achieved level
Prospects for using the results		paramount	The results can be applied in many scientific areas.
			The results will be used in the development of new technical solutions
		Useful	The results will be used in subsequent research and development
Scale of implementation of results		National economy
			Implementation time: up to 3 years, up to 5 years, up to 10 years, over 10 years
		Individual firms and enterprises	Implementation time: up to 3 years, up to 5 years, up to 10 years, over 10 years
Completeness of results			Terms of Reference for R&D

		Insufficient	Overview, information

In this case, the coefficient of scientific and technical effectiveness is determined by the formula

where k is the number of estimated parameters;

coefficient of influence of the i-th parameter on scientific and technical performance;

– coefficient of relative increase of the i-th parameter in comparison with the base value.

For the convenience of performing calculations, the data are summarized in Table. 7.5.

Table 7.5

Assessment of the scientific and technical effectiveness of applied R&D

Parameter	Unit of measurement	Coef. influence	Parameter values
			achieved