Paired comparison in simple examples. Ranging. Paired comparison method. Scoring method. Group assessments. Ranking of alternatives

Ranking as an evaluative research method is a procedure as a result of which the analyst, based on his knowledge and experience, places the objects under study in order of preference. He selects the best object, superior to all others in some attribute (set of attributes) and assigns to it an indicator equal to 1, characterizing the ordinal place (rank) of the evaluated object among other objects. From the remaining options, the best one is again selected and rank 2 is obtained, etc. The ordinal scale based on the ranking results must satisfy the condition that the number of ranks is equal to the number of ranking objects.

The ranking results in a sequence of preferences:

A > B > D > D > B - preferences;

Accordingly: 1, 2, 3, 4, 5 - rank.

where > is a preference sign;

A, B, C, D, D - objects of analysis.

The ranks of a number do not make it possible to draw a conclusion about how much or how many times an object is preferable in comparison with another.

The advantage of the method is its simplicity. The disadvantage is the practical impossibility of organizing a large number of objects.

Paired comparison is a method that allows you to establish preferences for objects of analysis when comparing all their possible pairs. A pairwise comparison of objects may assume that one of the objects is more preferable than the other.

Pairwise comparisons can be represented as a matrix. As a result, the options are ranked by the sum of preferences.

There are variants of multiple comparisons, which differ in that not pairs of objects are compared sequentially, but their triplets, quadruples, etc.

The scoring method is a procedure for assigning numerical values to objects of analysis on a given scale. In this case, continuous and discrete scales can be used. In the first case, the estimates belong to any point of some limited numerical interval, in the second, the estimates correspond to integers. The scales are characterized by minimal and maximum number points. The upper and lower limits of the scale can have both positive and negative values. The best object is considered to be with maximum value assessments.

The method of expert (group) assessment of phenomena is most often used in the practice of assessing complex systems. As a rule, this method is used to obtain quantitative values when it is impossible to calculate them.

When implementing this method, the following sequence of actions is assumed:

- selection of experts;
- determination of the point scale;
- assessment by experts for all compared objects of analysis;
- assessment of the consistency of expert opinions;
- calculation of group assessment for each object;
- use of the obtained estimates for practical purposes (for example, compiling rankings of objects of analysis).

It should be noted that expert assessments contain both narrowly subjective features inherent in each expert, and collectively subjective ones inherent in the panel of experts. The former are eliminated during the processing of individual expert assessments, the latter do not disappear, no matter what processing methods are used.

To quantify the degree of agreement between experts' opinions, the coefficient of concordance (coherence) W is used, which allows one to evaluate how consistent the opinions of the participants in the examination are. Its value is within 0< W < 1, где W=0 означает полную противоположность, a W=1 полное совпадение оценок. Практически достоверность считается хорошей, если W более 0,7.

A low value of the concordance coefficient indicates a weak agreement of expert opinions.

There is a method of an expert commission - a variation of the group examination method. The expert commission method is based on identifying a single collective opinion by specially selected experts when discussing the problem posed and alternatives for solving it as a result of certain compromises.

Laboratory work No. 1

Paired and sequential comparison methods

Goal of the work.

Master the methods: paired comparisons, sequential comparisons.

Theoretical part.

Methodology for solving unstructured problems. Classification and general characteristics expert assessment methods

It is advisable to divide all methods of expert assessments into 2 classes:

1. Methods for forming individual expert assessments, and an individual expert can be used: to obtain information such as an interview; free conversation, conversation based on the question-answer principle; cross-examination, etc. To collect initial data in the method of paired comparisons and others. For consultations with decision makers and system analysts.

2. Methods for forming collective expert assessments, and a group of experts can be used:

· for teamwork round table(commission method - a meeting to resolve a certain issue; brainstorming method; court method, etc.);

· to collect initial data using the Delfi method, etc.;

· for business games;

· to develop a script;

· to build a tree of goals

Among the promising methods of expert assessments is the Delfi method. It is based on a carefully developed procedure of sequential individual interviews with experts using questionnaires. Surveys are accompanied by constant informing of experts about the results of processing previously received responses. The examination is carried out in several rounds until an acceptable convergence in the experts’ judgment is obtained. The median of the experts' final answers is accepted as a collective expert assessment.

The Delfi method is continuously improved through the use of computers and its use in combination with other methods. New modifications of the method provide increased versatility, speed and accuracy of obtaining collective expert assessments (Delfi method - conference, etc.).

Principles of formalization of heuristic information

The heuristic information received from experts must be presented in a qualitative form that is convenient for processing and analysis. In this case, the following types of scales are used to formalize heuristic information:

1. a classification scale that allows you to study the objects under study using certain numbers;

2. an order scale, which allows you to order the objects under study according to any criterion;

3. interval scale, which allows you to assign relative values to the objects under study numeric values;

4. a ratio scale that allows you to assign absolute numerical values to the objects under study.

Here is an example of scales for formalizing heuristic information:

The Harrington scale has an analytical description in the form of a utility function:

y = exp[-exp(-x)], y∈,

where x is the value under study in the range [-6;6]

Using the Harrington scale, vector estimates with different dimensions can be reduced to dimensionless form.

Paired comparison method

The method involves the use of an expert who evaluates the goals. Z 1, Z 2, ..., Zn.

According to the method, pairwise comparisons of targets are carried out in all possible combinations. In each pair, the most preferred goal is highlighted. And this preference is expressed using a rating on some scale. Processing the evaluation matrix allows you to find the weights of the goals that characterize their relative importance. One possible modification of the method is as follows:

1. a matrix of binary preferences is compiled, in which the preference of goals is expressed using Boolean variables;

2. The price of each goal is determined by summing the Boolean variables along the corresponding row of the matrix.

Sequential comparison method

One possible modification of the method is as follows:

1. All goals are arranged in the form of an array in descending order of their importance and preliminary assessments of the goals are assigned. In this case, the first goal of the array receives a score of 100, and the remaining goals are assigned scores that reflect their importance.

2. The first goal of the array is compared with all possible combinations of lower goals by 2. If necessary, the assessment of the first goal is adjusted. The second goal of the array is compared with all possible combinations of lower goals in 2. If necessary, the assessment of the 2nd goal is adjusted, etc.

3. The adjusted estimates are recorded and the target weights are calculated based on them.

Example of work done

Example 1. Paired comparison method:

The expert assesses 4 goals that are related to solving the transport problem.

Z 1 - build a subway

Z 2 - purchase a 2-decker bus

Z 3 - expand the transport network

Z 4 - introduce light rail

1. Let’s create a matrix of binary preferences:

Z i / Z j	Z 1	Z 2	Z 3	Z 4
Z 1
Z 2
Z 3
Z 4

The table is constructed based on the results of reasoning performed as follows:

1.1. We are considering line No. 1 – corresponding to the construction of the metro. We begin to compare the construction of a subway and the option of purchasing a 2-decker bus, we intuitively conclude that the construction of a subway will solve the problem much better than purchasing a 2-decker bus.

1.2. Therefore, in the table opposite Z 2 we put one. Those. if one is written in a row (for us this is option Z 1), then this solution option is better than the solution corresponding to the column (for the case considered, it is better than Z 2).

2. Determine the price of each goal (add it in rows)

C 1 =3; C 2 =0; C 3 =2; C 4 =1

These numbers already characterize the importance of objects. We normalize, because These numbers are not convenient to use.

3. Claim weights of targets.

V 1 =3/6=0.5; V 2 =0; V 3 =0.33; V 4 =0.17

Examination:

We therefore obtain the order of preference for goals:

Z 1, Z 3, Z 4, Z 2

Formulation of the problem from example 1:

Example 2. Sequential comparison method:

The expert evaluates 4 goals that are related to solving the transport problem (tasks from example 1).

1. Arrange the goals in the form of an array and assign preliminary estimates Z 1 , Z 3 , Z 4 , Z 2 .

We give points:

1.1. We choose the best one and give it 100 points, let the construction of the subway be the best solution to the problem →p 1 =100;

1.2. We assume that buying a 2-decker bus will solve the problem by only 10%→ p 2 =10;

1.3. We assume that expanding the transport network will solve the problem by 60%→ p 3 =60;

1.4. And the introduction of a high-speed tram will solve the problem by 40%→ p 4 =40.

2. Write down the resulting values in descending order:

p 1 =100, p 3 =60, p 4 =40, p 2 =10

3. Let's compare the goals and adjust their estimates. An adjustment must now be made to the resulting distribution.

We compare the most profitable option with the sum of any two.

3.1. Compare the construction of the metro and the sum of points Z 3 ÇZ 4

Z 1 ⇔ (Z 3 ÇZ 4)

We get 100 ⇔ sum 60+40 i.e. 100 ⇔ 100. We are thinking: is it effective to replace the construction of the metro by expanding the transport network and introducing a high-speed tram? Now, according to the assigned points 100>100, such a replacement is equally effective. What we don't agree with. Therefore, we are adjusting the estimates by adding another 25 points to the subway construction option, because we believe that this option is 25 times better than the expansion of the transport network and the introduction of high-speed trams combined→ p 1 =125.

3.2. We repeat our reasoning for two more cases:

Z 1 ⇔ (Z 3 ÇZ 2)

Z 1 ⇔ (Z 4 ÇZ 2)

3.3. It remains to compare the last option: the second place winner (Z 3) with the sum of points of those who took third (Z 4) and fourth place (Z 2).

Z 3 ⇔ (Z 4 ÇZ 2)

We get 60 ⇔ the sum is 40+10 i.e. 60 ⇔ 50. We are thinking: is it effective to replace the expansion of the transport network with the introduction of high-speed trams and the purchase of double-decker buses? Now, based on the assigned points 60>50, such a replacement is not effective. We agree with this

4. Write down the adjusted estimates and calculate the weights of the goals:

p 1 =125; p 3 =60; p 4 =40; p 2 =10

V i =125/ (sum of all estimates) = 0.53; V 3 =0.25; V 4 =0.17; V 2 =0.04

the sum of all V i must be equal to 1.

We therefore obtain the order of preference for goals: Z 1,Z 3,Z 4,Z 2

Formulation of the problem from example 2:

Self-administered task

1. Solve the problem using the method of sequential solutions:

To promote goods and services on the market, the holding needs to carry out additional advertising events. An expert from the sales department analyzes four options for resolving this issue:

1. Creation of an online store;

2. Introduction of round-the-clock operation, increase in personnel;

3. Opening of another branch;

2. Solve the problem using the paired comparison method:

As a result of the effective use of foreign investment and competent policies, the enterprise received significant profits. The manager and investors approved an expert to solve the problem of choosing an object to which funds will be allocated for development. The following goals are proposed to the expert:

1. Construction of a fitness center for employees on the territory of the enterprise;

2. Ordering a corporate website project;

3. Investing in a large construction project;

3. Solve the problem using the method of sequential solutions:

As a result of the successful activities of the bank and the demand for its services, management faces the problem of organizing further uninterrupted provision of services to the population, expansion, and attracting new clients. To do this, the expert is tasked with determining the most successful solution to the issue:

1. Opening of an additional branch in the city;

2. Acquisition of a building of the required size for the relocation of the bank and its expansion;

3. Introduction of round-the-clock operation, increase in personnel;

4. Solve the problem using the method of sequential solutions:

Funds have been allocated from the republican and local budgets to the healthcare sector; the expert assesses the most needy and important areas of medicine to receive subsidies.

1. Replacement of all equipment that has served its standard period with new ones;

2. Installation of expensive modern equipment in specialized centers and dispensaries;

3. Opening of clinics in densely populated neighborhoods;

4. Construction of a drug treatment center;

5. Solve the problem using the paired comparison method:

The Prospect company wants to make maximum profit. To do this, management invited 3 experts to select the best alternative from those proposed:

1. opening your own production;

3. expansion of the sales market;

4. reducing prices in order to increase turnover.

6. Solve the problem using the paired comparison method:

The company management wants to reward employees for exceeding the plan. To do this, the expert is tasked with determining the most successful solution to the issue:

1. issue a one-time profit;

2. arrange a corporate party;

3. give paid leave;

4. increase your salary.

7. Solve the problem using the method of sequential solutions:

Parents decided to reward their children for excellent studies. To do this, they invited 4 experts to choose the best option:

1. increase in pocket money;

2. a trip to the sanatorium;

3. allowed to walk until 23:00

8. Solve the problem using the paired comparison method:

The university management decided to contribute to the cultural enrichment of students. To do this, management invited 3 experts to select the best alternative from those proposed:

1. free theater tickets;

2. free tickets to the exhibition;

3. free movie tickets;

Related information.

It has not changed since 2009 and is morally outdated. This year we decided to update it: simplify the structure, rewrite the texts, and most importantly, fundamentally change the design.

There is always a certain amount of randomness in the choice of design. Based on the needs of users and analysis of their behavior, it is possible to develop requirements for the content of pages and the structure of the site, but why are certain solutions used in the design of pages? best case scenario explained by the brand book.

We had to choose a design that would appeal to users and justify it to management. If the design caused a sharp rejection among users, all the work on updating the site would be in vain. If management had not approved the design, we would not have been able to deliver the project. Therefore, we first had to collect design requirements from management, and then check how potential visitors to the new site reacted to it. We used the method of paired comparisons for this, slightly adapting it to our tasks.

What is the “paired comparison method”?

The paired comparison method allows you to select one option from several when the selection criterion is directly or indirectly related to emotions, so it is difficult to justify. If you need to decide which tastes better (apples, pears, oranges or peaches), which company is better to work for (Google, Apple, Amazon or Facebook), or what to wear for a walk today (jacket, sweater or sweatshirt), the paired comparison method will help do it.

The author of the method is Louis Leon Thurstone, one of the founders of psychometry, the science of measurement. mental processes. Some sources claim that Thurstone developed this method to create a crime severity scale.

To conduct the study, pairs are made from all the options so as to go through all possible combinations. Respondents are asked to choose from each pair the option that seems more suitable to them according to the criterion specified by the researcher (for example, “which version of the main page is more attractive”). The option that was chosen more often than others wins.

Research Tools

On the Internet you can find services that allow you to use the paired comparison method for decision making (for example, 1000minds or TransparentChoice), or even mobile applications. However, they are not suitable for our purposes because they do not allow you to upload images and involve respondents in the assessment. Some survey services (for example, Oprosso) allow you to present pairs of images to respondents, but do not allow you to limit the time of presentation, present images at the scale the researcher needs, and do not take into account a dozen other subtleties that are important for graphic design research.

MyDecision - Smart Comparisons - an application for choosing from several alternatives using paired comparisons. You can use it to decide which bar to go to or which smartphone to choose

Therefore, we asked our developers to make us our own service for paired comparisons. It allows you to display images through a browser, conduct small surveys, for example, to find out the socio-demographic data of respondents and get feedback from them about the reasons for their choice, and most importantly, it is free for us.

This is what pairs of images look like when presented through USABILITYLAB’s own service

Application method 1: choosing the best option

Selecting the best option from several is a classic way of applying the method. To conduct such a study, several simple requirements must be met.

You can only compare objects of the same type. When choosing clothes for a walk, you can compare a jacket with a sweatshirt, but you cannot compare a jacket with jeans; When choosing a design for a website, you can compare different design options for the main pages, but you cannot compare the main pages with the internal ones. Otherwise the results will be meaningless.
Presentation time should be 3-5 seconds. If respondents have more time, they begin to look at the images, read the texts and evaluate the options not according to the criterion that you asked them (for example, “attractiveness”), but according to their own, unknown to you (for example, “informativeness of texts”).
For the same reason, it is better to try to make all options as similar as possible in terms of those parameters that should not affect the decision. If you are comparing designs, try to make all the text and photos on the pages the same, otherwise respondents will start comparing them.
Each pair must appear at least twice. The second time, the options should be swapped: the one that was on the left should be placed on the right. This is necessary in order to exclude the influence on the results of those respondents who, due to inertia, the desire to quickly complete the study or other reasons unknown to us, more often choose the option located on the same side of the screen (for example, they more often choose any options located on the left).

We conducted a similar study for Russian Standard Bank. We needed to evaluate design options for the main page and two internal pages mobile application jar. The old design was compared with three new options.

Design options for the main page of the Russian Standard Bank mobile application. 0 - old design, 1, 2, 3 - new options

From all the options, we made pairs: the main screen with the main screen, internal screens with internal ones. All pairs uploaded to their web application. Each pair was presented twice, so respondents saw each option 12 times: 6 on the left and 6 on the right. In total, respondents rated 72 pairs. As a result, option 1 won: it was chosen in 69% of cases. Old version interface received the worst rating: it was chosen only in 35% of cases.

Application Method 2: Gathering Design Requirements

If you show a person a page of a website and ask: “Do you like it?”, then, most likely, you will not hear anything specific in response. They will answer you that yes, on the whole, they like it, or no, on the whole, somehow not very much. If you present pages in pairs and ask: “Which one is better?”, material for discussion immediately appears: this page is better because, for example, it uses large font, white background, and a kitten is drawn in the lower left corner.

This is how we collected requirements for the site from our management. We found several Russian and foreign corporate websites whose design could become a source of inspiration for our own. We then printed out the home pages of these sites and took them to our CEO. We showed the printouts in pairs and asked which option was better and why. So we found out what elements she pays attention to, what she likes, and what, on the contrary, seems repulsive.

Based on these requirements, we compiled a technical specification for the designer. The designer prepared six versions of the main page for us, and one of them, having agreed with the management, we immediately put into work.

Six options for the home page of the new USABILITYLAB website

It remains to check what reaction these options will cause among potential site visitors, especially the one that we ourselves have chosen for ourselves. First of all, we were worried about the manifestations of negative emotions: if “our” option would cause sharp rejection, the design would have to be redone. If negative emotions were caused by other options, this would serve as additional confirmation of the correctness of our choice.

Application method 3: collecting feedback from the target audience

To gather feedback from our target audience, we decided to show them all six design options under the guise of a paired comparison study. We again used our online service.

We recruited respondents for the study through Facebook. To do this, Dmitry Silaev created a post with an invitation to the study, where he spoke about its essence and promised a reward to the participants.

Post by Dmitry Silaev with an invitation to research

We weren't really interested in which option would win; Much more important to us were the respondents’ comments received via Facebook or through the feedback form at the end of the study. Therefore, we made some methodological errors: we increased the presentation time from 5 to 120 seconds and showed each pair only once, otherwise the study would have taken too much time.

We managed to collect about 300 questionnaires, of which 140 were completely filled out. Most of the comments we received were related to the content of the photographs used in the mock-ups. For example, one respondent wrote that “the photo of a child in one of the designs is so cute that I kept wanting to choose that design.” Although “our” version did not win, it took an honorable third place, and we managed to get feedback from “fans”, which means that all our goals were achieved.

Conclusion

The method of paired comparisons helps to choose the best option from several, but its application does not end there. Using it, we were able to collect requirements for the design of our new website, and then submit this design to management. Work on the site is almost finished: the texts are now being polished and the site is being checked for accessibility for the visually impaired. So very soon you will be able to evaluate it in person.

After we spent webinar, in which they talked about the method, representatives of Oprosso contacted us. We sent them a list of recommendations on what features should be added to their service so that it could be used to conduct full-fledged paired comparison studies. Now we will monitor updates to this service, and if our recommendations are implemented, we will use it on some of our projects.

We also plan to develop our paired comparison service. For example, we set our developers the task of making the service interface adaptive so that respondents could complete the study from different devices. We are also going to improve the work with research results: add the ability to upload results at any time and filter by time and completeness of filling out questionnaires.

If you have tasks similar to those we described in this article, write to us at. We will provide you with access to our service and help you plan your research.

Faculty of Economics and Management

Department mathematical methods and models in economics

Coursework on the topic: “Decision-making for choosing the optimal range of goods for

Coursework assignment

in the discipline “Methods of making management decisions” at

topic “Decision-making for choosing the optimal range of goods for

based on the method of paired comparisons"

1.Consider theoretical aspects making management decisions based on the method of paired comparisons.

2. Solve the problem of choosing the optimal product range based on the method of paired comparisons.

Introduction........................................................ ...........................................3

1 General information about the method of paired comparisons...................................................6

1.1 Paired comparison matrices.................................................................. ............6

1.2 Determining the degree of significance of criteria and assessing consistency

pairwise comparison matrices................................................................. ..................8

1.3 Advantages and disadvantages of the paired comparison method....................................23

2 Solving the problem of choosing the optimal product range based on

method of paired comparisons......................................................... .................26

Conclusion................................................. .................................thirty

List of sources used......................................................... ....32

Introduction

The paired comparison method is extremely powerful, general, and can in principle be used for any set of objects or stimuli that can be compared with each other in some way. psychologically in a real way. For example, in a simple psychophysical task such as loudness perception, a series of tones of varying intensity are presented in pairs and the subject is asked to decide which of the pair is louder. In more complex domains, such as emotions or aesthetics, one can ask subjects to compare pictures based on beauty, smells based on pleasantness, faces based on similarity, and so on.

The power of this procedure is determined by its application to the data multivariate methods and factor analysis techniques to discover the underlying dimensions along which judgments are made.

The paired comparison method is a very general procedure for measuring (scaling) objects or stimuli and estimating the dimensions that define them. In the standard, complete paired comparison method, each item in a set is presented for evaluation in pairs with every other item in that set [ 1 ].

Its author is American psychologist Louis Leon Thurslone. He developed the method in the first half of the twentieth century. Since then it has been widely used in precision natural sciences, such as mathematics and physics, and in the humanities, for example, psychology, sociology. The method also takes into account the laws of logic.

It is designed to study preferences, in which the respondent must, from all possible paired combinations of objects offered to him, choose the most preferable one in accordance with a given criterion. The result is a pairwise comparison matrix in which the sum of the row elements gives an idea of the respondent's ranking of all objects.

IN in this case aij will be equal to 1.

If the respondent preferred the i-th object to the j-th object, and aij = 0 in all other cases, i,j = 1,n

The degree of preference by the respondent for object i is determined as the sum of units in the corresponding row of the matrix. Paired (or pairwise) comparisons when not large number objects - the most accurate and reliable method for identifying preferences. It is usually used to identify experts’ preferences “in their pure form.” It is believed that it is much easier to do qualitative comparison two objects rather than expressing your preferences on a point or ranking scale. This assessment method is considered non-reactive; it does not impose a priori conditions on respondents. Considering some preference random variable reflecting the true ratio of the characteristics of the compared objects, one can set the task of determining the probability of the true ratio of the compared objects (models of Bradley, Terry, Lewis, etc.) [3].

Paired comparison analysis is a very good tool for weighing relatively important but different choices. This applies in cases where priorities are not entirely clear or, conversely, it is clear to the manager that “everything is important.”

This tool helps you identify your most important selection priorities, present information clearly, and make the best choice.

It is used when making decisions in the absence of accurate and objective data. And also, when comparing completely different factors. For example, you need to decide where to invest: in the market for office supplies or legal services. Such comparisons are much more complex than comparing several types of paper with each other.

Tukey's method of paired comparisons is one of the methods of analysis of variance, designed for pairwise comparison of the average values of the dependent variable in individual groups in a factorial experiment. The F-test, which allows us to reject the null hypothesis of no differences between group means, does not answer the question of which groups have different mean values. The easiest way to find out is to compare the mean values of the trait in all groups in pairs (if there are k groups in the experiment, k(k - 1)/2 comparisons are needed). The results of comparisons are presented in the form of a table, which indicates which significant statistical differences were found between the average values of which groups. To test the null hypothesis that the mean values of a characteristic are equal in groups with numbers i and j against the alternative hypothesis, which is that the mean values of these groups are different, the following criterion is used:

where n is the number of objects in each group;

yi and yj are the average values of the characteristic in groups with numbers i and j; MSSvngr - intragroup mean square.

This criterion has a distribution of studentized range with numbers of degrees of freedom:

, (3)

where n is the number of objects in each group; k is the number of groups in the experiment.

The null hypothesis is rejected if the calculated value is greater than ql-a . M.P.S. Tukey is used only if the size of all groups in the experiment is the same. In other cases, and when more complex comparisons are required, multiple comparison methods are used.

The method of paired comparisons allows you to determine the significance of differences in the position of certain objects in the hierarchy and solve other similar problems.

The purpose of the work is to study the possibility of using the method of paired comparisons as a tool for making management decisions.

The object of study is the problem of choosing the optimal nomenclature

The subject of study is the method of paired comparisons.

To achieve this goal, it is necessary to solve the following

1) Consider the theoretical aspects of making management decisions based on the method of paired comparisons;

2) Solve the problem of choosing the optimal product range based on the method of paired comparisons.

1 General information about the paired comparison method

1.1 Paired comparison matrices

A ratio scale is used to establish the relative importance of elements and construct a pairwise comparison matrix. This scale allows the decision maker to assign certain numbers to the degrees of preference of one compared object over another.

Table 1-Relationship scale (degree of significance of actions) [ 5 ].

Significance degree	Definition	Explanation

	Equal importance	Two actions contribute equally to the goal
	Some predominance of the significance of one action over another (weak significance)	There are reasons for preferring one of the actions, but these reasons are not convincing enough
	Significant or strong significance	There is reliable evidence or logical judgment to show that one of the actions is preferable
	Obvious or very strong significance	Compelling evidence favoring one action over another
	Absolute significance	Evidence for preference for one action over another in highest degree convincing
	Intermediate values between two adjacent judgments	A situation where a compromise solution is necessary
Reciprocals of the above non-zero quantities	If action i, when compared with action j, is assigned one of	If consistency was postulated upon obtaining N numeric

The validity of this scale has been proven theoretically by comparison with many other scales. When using the specified scale, the decision maker, comparing two objects in the sense of achieving a goal located at a higher level of the hierarchy, must match this comparison with a number in the range from 1 to 9 or the inverse value of the numbers. In cases where it is difficult to distinguish so many intermediate gradations from absolute to weak preference or this is not required in specific task, a scale with fewer gradations can be used. In the limit, the scale has two ratings: 1 - objects are equivalent; 2- preference for one object over another.

Filling out square matrices of paired comparisons is carried out according to the following rule. If element Ei dominates element E2, then the matrix cell corresponding to row E1 and column E2 is filled with an integer, and the cell corresponding to row E2 and column E1 is filled with its inverse. If element E2 dominates Ei, then the integer is placed in the cell corresponding to row E2 and column E1, and the fraction is placed in the cell corresponding to row E1 and column E2. If elements E1 and E2 are equally preferred, then ones are placed in both positions of the matrix.

To obtain each matrix, the expert or decision maker makes n(n - 1)/2 judgments (here n is the order of the paired comparison matrix).

Let us consider in general an example of forming a matrix of paired comparisons.

Let E1, E2, ..., En be a set of n elements (alternatives) and V1, V2, ..., vn be their weights or intensities, respectively. Let us compare in pairs the weight, or intensity, of each element with the weight, or intensity, of any other element of the set in relation to their common property or goal (in relation to the “parent” element) [4].

The matrix of paired comparisons is presented in Table 2:

The matrix of paired comparisons has the property of inverse symmetry, i.e.

where aij=vi / vj.

When making pairwise comparisons, you should answer next questions: Which of the two elements being compared is more important or has a greater impact, which is more likely and which is preferred. When comparing criteria, it is common to ask which criterion is more important; when comparing alternatives in relation to a criterion, which alternative is more preferable or more likely.

1.2 Determining the degree of significance of criteria and assessing the consistency of paired comparison matrices

Currently, the problem of increasing the efficiency of innovation project management is relevant. Since the financing of innovative projects in most cases is carried out by various investment companies, each of which has its own management features, assigned tasks and history of existence, to reduce the risks associated with investing in innovative projects, their own specific means and management tools are invented. Successful work on the analysis of innovative projects is based on the use of numerous methods used both in the construction general model working with projects and at individual stages of the project process within the company.

One of the most important stages of working with projects in a company is the examination of these projects. During the examination process, an application that will either be rejected or become a funded project is subjected to diverse studies in which experts from various fields participate.

Expertise, as a rule, is a process in which a group of highly qualified and highly specialized experts participates, the result of which is a set of expert opinions or one consolidated opinion.

However, to obtain information from experts that reliably reflects the prospects and shortcomings of the project being analyzed, it is not enough to simply find good experts. To obtain correct and indicative expert opinions, it is necessary to determine the criteria by which experts should analyze the application.

Differences in criteria among different companies can be explained by both different financial situations and different priorities and goals. For this reason, each organization must independently form its own list of project evaluation criteria that are important to it.

However, once this list is compiled, all companies are faced with the task of determining the relative importance and significance of the criteria. To solve this problem, various methods can be used [2].

The most common method is scoring, in which each criterion is given a specific score and the relative importance of the criteria can be assessed by comparing the scores assigned to them. Today, the method of forming criteria weights has become quite widespread. main idea This method consists of pairwise comparison of criteria. All criteria intended for project analysis are assessed by constructing a matrix of paired comparisons. A pairwise comparison matrix is a matrix in which a criterion located in a row is compared with all criteria listed in the columns of the matrix (Table 1). For example, if criterion No. 1 is more important than criterion No. 2 by a12 times, then element (1, 2) of the matrix is equal to a12. Based on this, the main diagonal of the matrix is always filled with ones

Table 1 - Paired comparison matrix

	Criterion 1	Criterion 2	Criterion 3

Criterion 1
Criterion 2
Criterion 3

It is logical to assume that if criterion No. 1 is more important than criterion No. 2 by a12 times, and criterion No. 2 is more important than criterion No. 3 by a2 times, then criterion No. 1 should be more important than criterion No. 3 by exactly a12-a23 times. However, for matrices filled out by real people, this is not always the case. This is due to the fact that the matrix of judgments is filled out by an expert, who may make an error in determining the relative importance of the criteria according to psychological reasons. One of the objectives of the method is the desire to reduce the influence of the human factor on the final semantic result. To determine the degree of data correctness in a completed matrix, the concept of a matrix consistency measure is introduced. To clarify the definition of a fully consistent matrix, its general form is given (Table 2).

Table 2 - General view of the agreed matrix.

	Criterion 1	Criterion 2	Criterion 3

Criterion 1
Criterion 2
Criterion 3

To process the values of the resulting comparison matrix, a consistency index is introduced, which shows the presence of a logical connection between the assessed indicators. To find the consistency index of a positive inversely symmetric matrix (the pairwise comparison matrix has these properties), it is necessary to find the maximum eigenvalue of the matrix and its dimension.

The consistency index is calculated using formula (1):

where L max is the maximum eigenvalue; n is the dimension of the matrix.

If the matrix is consistent, then the assumption that if criterion No. 1 is more important than criterion No. 2 by a12 times, and criterion No. 2 is more important than criterion No. 3 by a23 times, then criterion No. 1 should be more important than criterion No. 3 exactly a12 a23 times, is always true . For such a matrix, IS equals zero. However, as a rule, when analyzing data obtained by experts, the matrix is not completely consistent.

In the developed method, it is proposed to use a rating scale for pairwise comparison of criteria, which contains numerical indicators from 1 to 9 and their reciprocals. The values of the 1:9 scale reflect nine degrees of superiority of one criterion over another, with five values being basic (1,3,5,7,9) and four intermediate values (2,4,6,8). If the criterion under consideration is not more, but less important than the one with which it is compared, this relationship is also described by nine degrees of comparison, but represented by the reciprocal values: 1, 1/2, 1/3, ... , 1/9 [ 1 0 ].

When carrying out the procedure for comparing criteria, experts fill out the corresponding matrices. Each expert is required to fill out only the upper part of the matrix (above the main diagonal), since when using this technique it is assumed that if criterion i, when compared with criterion j, is assigned one of the numbers in the range , then criterion j, when compared with criterion i, is assigned the opposite value.

After the expert fills out the matrix of paired comparisons, it is necessary to check the consistency index of the matrix. For this

Using formula (1), the IS of the matrix is calculated and compared with the average consistency index of random matrices of the same order. The relationship between these indices is called the consistency ratio (CR).

Currently, for the scale, scientists have calculated random consistency indices (SI) for inversely symmetric matrices with dimensions from 1 to 15 (Table 3), taken as a basis when analyzing the resulting matrices for consistency. In this method, an OS value less than or equal to 0.10 is considered acceptable.

Table 3 - Average random consistency indices for matrices of different orders

Of course, using a scale of 1 to 9 to analyze the importance of criteria has its advantages. However, in a number of cases, especially if it concerns such a complex aspect as the analysis of innovative projects, this scale is not only redundant in nature, but can also cause additional error in the process of an expert giving an appropriate assessment when paired comparisons of various criteria.

Based on an analysis of the opinions of practitioners working in the field of innovation, who often have to deal with various types of comparisons, it was revealed that it is advisable to use a more categorical scale of 1:5 (Table 4). This is due to the specifics of the area for which the hierarchy analysis method is being adapted. This article is about comparing innovative projects, while the method of paired comparison of objects is used to determine the weights of the criteria by which projects will subsequently be compared. Since the resulting criteria weights can significantly influence the decision made on a project, it is necessary that the scale by which the resulting weights are determined be unambiguous and specific.

Thus, we can conclude that using a 1:5 scale is more convenient in practice, since each numerical value has a clearly defined semantic interpretation. Besides,

comparison of criteria using such a scale will be characterized by a greater degree of expert confidence. This is an important fact, since in this case we are talking about working with innovations, which means the situation is complicated by various risks. When choosing any methods for organizing work with innovative projects that have a priori a large degree of uncertainty, it is necessary, in order to avoid the accumulation of total errors, to choose methods that themselves have the lowest possible degree of uncertainty.

In addition, a clearer definition of signs, which is provided by the 1:5 scale, allows us to specify the situation without a significant loss of accuracy, on the one hand, and with a significant increase in the comfort of using this scale, on the other hand. Due to the identified need, the scale from 1 to 9 was replaced by a scale from 1 to 5. For this scale, the values of each of the points assigned were described (Table 4)

Table 4 - Modified relative importance scale

Intensity relative importance	Definition	Explanation

	Incomparable	The expert finds it difficult to compare
	Equal importance	Equal importance of criteria i and j
	Not significant degree of importance	Criterion i is not significantly more important than criterion j
	Significant degree of importance	Criterion i is significantly more important than criterion j
	Intermediate values between two adjacent scale values	A situation when a compromise solution is necessary, 2 - criterion i has a slight advantage over criterion j, 4 - criterion i has a noticeable advantage over criterion j
Reciprocals of the above numbers	If criterion i, when compared with criterion j, is assigned one of	Justified assumption

To provide more comfortable comparison conditions for experts, an additional scale item was introduced - null value. The expert has the opportunity to put 0 when comparing two criteria if he believes that the criteria are incomparable or the comparison is extremely difficult for him personally.

Greater comfort when using the 1:5 scale is explained by the ease in differentiating criteria ratings (Figure 1). Three scale values: 1,

3 and 5 act as the main ones when assessing relative importance, and 2 and 4

They are compromise, intermediate choices.

Figure 1 - Key divisions of the 1:5 scale

Since the scale has been changed, to correctly check the consistency of the corresponding matrices, it is necessary to calculate the consistency indices of random matrices of this type. To solve this problem, 100 random matrices of order 3, 4 and 5 were generated for the selected scale. The SI obtained as a result of the calculations are indicated in Table 5.

Table 5 - SI on a rating scale from 1 to 5.

As an example, to calculate the consistency ratio for matrices of order 3, formed on a scale of 1:5, an analysis of the acceptable level of matrix consistency was carried out. The threshold for the 1:9 scale is 10%. To establish a threshold value when using a 1:5 scale, a simulation was carried out, which consisted of analyzing the values of the OS matrix for various deviations of expert assessments from assessments corresponding to a fully consistent matrix.

As part of the simulation, matrices with the following deviations were analyzed: an increase in one value by 1 step; -decrease one value by 1 step; -increase two values by 1 step; -decrease two values by 1 step; -increase three values by 1 step; -decrease three values by 1 step; -increase one value and decrease another value by 1 step; -increase two values and decrease one value by 1 step; -increase one and decreasing two values by 1 step; - increasing 1 value by 2 steps; - decreasing 1 value by 2 steps.

For each matrix dimension, a similar analysis was performed for the five different original matched matrices. As a result of the simulation, a value of 12.7% was obtained, which corresponds to the maximum consistency ratio when the expert’s opinion deviates by one step from the value of a fully consistent matrix. A value of 12.7% was chosen as the threshold for acceptable consistency of the matrix, compiled on a 1:5 scale.

The threshold value of 12.7% is reasonable for a matrix of dimension 3. For matrices of other dimensions, the threshold value of the OS should be calculated not only taking into account the analysis of deviations of the matrix from a completely consistent one, but also taking into account the fact that when comparing a larger number of criteria, the expert’s error may increase.

Thus, taking into account the features inherent in everyday practical activities when assessing innovative projects, a modification of the method was carried out. The main goal of the modification is to increase the effectiveness of the method when used by highly specialized specialists to assess the prospects and technical feasibility of innovative projects. The use of this modification of the method for determining the relative importance of criteria when evaluating a project can increase the efficiency and reliability of such a stage of work with innovative projects as preparation for the examination.

This point is extremely important, since it is according to the criteria established at this stage that the project is further assessed, and the correct correlation of the criteria among themselves in terms of importance allows us to draw a correct conclusion on the project.

The consistency index is a quantitative assessment of the inconsistency of comparison results (for the system as a whole, for nodes of the same cluster, or for clusters that have a common vertex) [8]. It should be borne in mind that there is no obvious connection between the reliability and consistency of comparisons. Contradictions in comparisons arise due to subjective errors of experts. The consistency index is independent of comparison scales, but depends on the number of pairwise comparisons. The consistency index is a positive number. The fewer contradictions in comparisons, the less value consistency index. When using the method of comparisons with the standard, the value of the consistency index is zero.

Relative consistency of the comparison matrix is the ratio of the consistency index to the average statistical value of the consistency index with a random selection of comparison matrix coefficients. Relative consistency for the system as a whole is characterized by a weighted average of the relative consistency across all comparison matrices.

Data can be considered practically consistent (sufficiently consistent) if the relative consistency value is less than 0.1. This conclusion is true for both cluster data and system-wide data.

Index of consistency between the positions of the two groups (consistency index)

Consider with three parties A, B and C and the following distribution of votes: A - 50 seats, B - 49 seats and C - 1 seat. Let's assume that for some reason parties A and B do not join the coalition. Let us calculate the Banzhaf index for this case:

Now the value of the Banzhaf index for batch B turned out to be zero, and for batch C it increased almost 2 times. Naturally, the question arises of how to assess the possibility of parties joining a coalition. For this purpose, the work uses the index of consistency between the positions of the two groups, which is described below.

The relationship between the two groups of parliamentarians is naturally reflected in the voting results. Groups holding similar political positions, having common interests and, accordingly, being in “good” relations will initiate agreed upon decisions and will support them when voting. On the contrary, if parliamentarians are in “bad” relations, then on most issues on which there are different points of view, they will vote differently, opposing their decision to the position of the “opponent” 1.

The consistency index was built on the basis of the conformity index proposed in the paper to determine how much the split in a given faction on a certain vote differs from the split in the entire legislature on the same vote. The concept of conformity does not have any evaluative meaning, and the index of conformity itself characterizes the degree of “similarity in split” between the position of a group of deputies and the position of the entire legislative body. The index was determined using the following formula:

The index value varies in the range from 0 to 1, it takes into account both the difference between p and q, and the “level of support for the issue” p. For the same value |p - q| a lower index value will be achieved with values

p, close to 4.

An index of the consistency of two groups of legislators in a single vote can be constructed using two different approaches leading to a common result. Further, in both cases, q ± and q 2 means the share of those who voted “for” in the 1st and 2nd groups

In the first approach, the index is first calculated as an index c* of conformity for one of the groups and the “common position” of the group:

The index value is 1 if the positions of the groups are the same (q1=q2), and equal to 0 if the positions are “opposite” (for example, q1=0 and q2=1).

However, using the value (q1+q2)/2 as a “general position” leads to a shift (increase) of the threshold value for. The index value in a "threshold situation" when the position of one group

is equal to q t =1, and the other q 2 = 1/2 is equal to c* Q, 1) = 2 / 3 [ 1 7].

To bring the index threshold to 1/2, the following transformation can be applied using the inverse function of function (q,1):

In the second approach, the position of one group is declared “general”, having a more specific position (for which the modulus of the difference (distance) is greater than 1/2). Accordingly, the formula for calculating the index is:

A simple check shows that formulas (3) and (4) lead to the same index value.

The consistency index constructed on the basis of two approaches has the following properties:

1) (value varies between 0 and 1) (5)

2) c (q!, q 2) = c (q 2, q t) (commutativity, “equality” of groups) (6)

3) c(1,1/2)=1/2 (threshold value, exceeding which means a change

bad attitude to good) (7)

4) c((symmetry with respect to the threshold

meanings, “equality” of positions “For” and “Against”).

As an estimate of the consistency index on an “average” per month, this work used the average value of the index over a series of m specially selected votes.

The selection of votes to assess the consistency index “on average” per month was carried out according to several criteria, reflecting different aspects of the information content of the vote for political demarcation between factions, parliamentary groups and individual deputies. The practice of monitoring actual voting in the State Duma, in conditions where non-participation in voting to a greater extent means the deputy’s disagreement with the issue than absence or a neutral position, makes criteria based on the share of “For” votes in the general list of deputies or in factions preferable.

In general, the procedure for selecting votes is carried out in two stages. At the beginning, votes are highlighted in which, even with a small number of votes cast “Against”, there is a significant (in terms of the share of those voting “For”) divergence of positions for at least two factions. For each vote, the difference between the maximum and minimum share of votes “For” by faction is calculated, then votes are selected for which this characteristic is not less than a given level (for the first Duma - not less than 0.5, for the second - not less than 0.6 and for the third - not less 0.7).

Next, “insignificant” votes on obviously passable and obviously unpassable “private” issues are excluded from the resulting list (in such votes, usually the number of “For” votes is at least 300-320 or no more than 30). Finally, voting is excluded from the list in which the discrepancy is due to “technical” reasons, which subsequently lead to a re-vote, or the passivity of one of the factions when voting on a clearly passable issue, etc.

Determination of homogeneity of goods and services [1-8].

“Homogeneous” - belonging to the same genus, category, identical.

In accordance with paragraph 6 of Article 1483 of the Civil Code Russian Federation(hereinafter referred to as the Code) designations that are identical or confusingly similar to the trademarks of other persons cannot be registered as trademarks in relation to homogeneous goods.

According to paragraph 3 of Article 1484 of the Code, no one has the right to use, without the permission of the trademark owner, designations similar to his mark in relation to goods for the individualization of which the trademark is registered, or for homogeneous goods, if as a result of use there is a likelihood of confusion. Thus, the law does not allow the possibility of individualizing homogeneous goods from different manufacturers with the same (identical) or confusingly similar trademarks.

To establish the homogeneity of goods, the following characteristics are taken into account: - type (type) of goods; - consumer properties of goods;

Functional purpose of goods (scope and purpose of use); - type of material from which the goods are made; - interchangeability and complementarity of goods; - conditions for the sale of goods (general place of sale, implementation of sales routes - mainly through retail or wholesale); - circle of consumers of goods; - predominant or traditional way of using goods; - duration/short-term use of goods; - cost of goods (expensive or not expensive);

Other signs.

Based on the results of the analysis of the listed characteristics, the examination may come to a conclusion about the homogeneity or heterogeneity of the goods. If the applied for designation is identical or confusingly similar to another trademark(s) in relation to goods that the examination recognizes as homogeneous, then your trademark registration will be denied.

In relation to consumer goods, to establish the homogeneity of goods, a more strict approach is applied than in relation to goods for industrial purposes.

When purchasing consumer goods, inexpensive goods, the attention of consumers is often reduced, therefore, the likelihood of confusing the trademarks that manufacturers use to mark the product is quite high. When purchasing expensive goods or complex equipment, buyers, as a rule, are extremely attentive and the likelihood of confusion is much lower than in cases of purchasing everyday goods.

There is a fairly widespread opinion among persons who are not specialists in the field of trademark registration that all goods grouped in one class of the ICGS are always homogeneous, while goods located in different classes of the ICGS are always not homogeneous. This is far from true. Not only goods located in different classes ICGS, even goods and services can be considered homogeneous. And within the same class there may be heterogeneous goods.

Let's consider examples of establishing the homogeneity of some goods: for example: - goods of class 05 of the ICGS “mineral waters for medical purposes” can be recognized as homogeneous to goods of class 32 of the ICGS - “mineral waters (drinks)” of class 32 of the ICGS; - “baby food products” belonging to class 05 can be recognized as similar to goods of class 29 of the ICGS - “dairy products” and goods of class 30 - “milk porridge”.

As examples when goods and services can be recognized as homogeneous, the following can be cited: - the product “computer programs”, included in class 09 of the ICGS, can be recognized as homogeneous to the service contained in the list of services of class 42 of the ICGS - “composing programs for computers” ; - “clothing”, a product classified in class 25 of the ICGS is homogeneous with services of class 40 - “tailoring”; - goods and services associated with these goods are often considered homogeneous, for example: goods of class 12 “cars” and services of class 37 – “car repair and maintenance”.

The stronger the similarity of the designations under consideration, the higher the likelihood that the consumer may confuse them; accordingly, the larger and wider the range of goods that the examination will regard as homogeneous.

To search for identical and similar trademarks and establish the homogeneity of goods Guidelines Rospatent provides an approximate list of corresponding classes.

Table 6 - List of corresponding classes.

As an example illustrating the approach to assessing the homogeneity of goods that, at first glance, it would seem, are not homogeneous, one can cite the Resolution of the Supreme Arbitration Court of the Russian Federation on the trademark “AMRO NEVSKOE”. This judicial act finally resolved the dispute between JSC "Vena", which is the owner of the rights to the trademark "NEVSKOE" under certificate N 189158 with a priority date of 04/07/1998 for goods of classes 21, 32 (including beer), 33 (alcoholic beverages) , 42 (providing food and beverages) ICTU and Black Jack-1 LLC, which is the owner of a combined trademark with the verbal element “AMRO NEVSKOE” under certificate N 241119 with a later priority date (02/05/2001) for a number of goods of class 29 (processed peanuts, shrimp, processed almonds, processed nuts, chips, fish, salted fish, dried fish, processed squid, dried squid) and class 30 ICGS.

The Vienna Society filed an objection to the registration of the AMRO NEVSKOE trademark with the Patent Chamber, indicating: the contested designation is confusingly similar to the applicant’s trademark and misleads the consumer regarding the manufacturer of the goods or the person ensuring their appearance on the market, and, therefore, the registration of such a trademark contradicts the requirements of paragraph 3

Article 6 and paragraph 1 of Article 7 of the Trademark Law (at the date of consideration of the dispute, the Trademark Law was in force).

The Presidium of the Supreme Arbitration Court of the Russian Federation put an end to this dispute, noting that:

“Since in this case there is a collision of two not identical, but similar trademarks registered in relation to also not identical goods and services, in order to protect the first trademark, the presence of similarity with it of a later mark and the threat of its confusion with this mark must be established by the court .

It should be recognized that a risk of confusion exists if one trademark is perceived to be another or if the consumer understands that they are not talking about the same trademark, but believes that both trademarks belong to the same enterprise. Such a threat depends on several circumstances: firstly, on the distinctiveness of the sign with earlier priority; secondly, from the similarity of the opposing signs; thirdly, from assessing the homogeneity of the goods and services indicated by the mark.

Comparing the designations “NEVSKOE” and “AMRO NEVSKOE”, the court of first instance rightfully assumed that the first trademark in relation to beer has significant distinctiveness. This is one of the most popular beer brands in Russia with a fairly large market share and recognition among consumers. Therefore, the fact that the trademark “NEVSKOE”, derived from the word “Neva”, is derivative and not original is not decisive. As a result of its use on the market for several years (since 1998), this trademark has acquired sufficient distinctiveness in relation to beer. The strengthening of distinctiveness is influenced by the presence of a series of trademarks with the specified verbal element at the Vienna Company. The designation “NEVSKOE” is fully included in the company’s trademark and occupies a dominant position in it. As the court of first instance correctly noted, the recognition of the named element as dominant is due to the fact that consumers do not associate the other element “AMRO” with a word that has any semantic meaning.

In addition, according to Articles 3 and 4 of the Trademark Law, the right to a trademark is limited to the goods and services specified in the certificate, but its protection extends not only to those objects that it designates, but also to similar ones not mentioned in the document of protection. Homogeneity is recognized in fact if goods, due to their nature or purpose, can be attributed by consumers to the same source of origin.

When establishing the homogeneity of goods, the following circumstances must be taken into account: the type (type) of goods, their consumer properties and functional purpose (scope and purpose of use), the type of material from which they are made, the complementarity or interchangeability of goods, the conditions for their sale (including general place of sale, sale through a retail or wholesale network), circle of consumers, traditional or preferential way of using goods.

When examining the homogeneity of the product - beer (class 32), services for providing food and beverages (class 42) and the previously listed food products included in class 29 of the ICGS, the court of first instance came to the correct conclusion that the traditional use of these beer snacks food products, the conditions of their sales (joint sale of beer and beer snacks), and the common circle of consumers for them indicate the homogeneity of the compared goods and services.

In addition, the prevailing idea in society about the complementarity of such goods as beer and beer snacks when consumed indicates their general (holistic) perception by a significant circle of consumers.” The pairwise comparison matrices are presented in Table 7.

Table 7 - Paired comparison matrix

The pairwise comparison matrix is inversely symmetric. The diagonal consists of units. The elements of the matrix a t y are determined on a scale of relative importance.

If the element A t exceeds the element Ау, then an integer is entered in row i, column j, and the inverse number is entered in row j, column i.

The main problem of filling out the matrix of paired comparisons is ensuring the transitivity of judgments.

If E ± > E 2, and E 2 > E 3, but E< Е 3 то суждения не транзитивны, матрица не согласована [ 2 0 ].

Calculation of element importance coefficients using a matrix of paired comparisons:

Consistency Index:

where is the maximum eigenvalue of the matrix;

n - matrix size (number of elements being compared).

Consistency ratio: OS=IS/K

Average consistency values of random matrices:

	Matrix size

Consistency criterion for the matrix of paired comparisons: O C<0,1 -02 Если ОС>0.1 - the matrix is not consistent [2 1].

1.3 Advantages and disadvantages of the paired comparison method

The main advantages of the method are as follows

It is possible to measure the unevenly changing importance of indicators, which is so necessary for solving most practical economic problems;

During the analysis process, the expert does not focus on all indicators at once, but only on two, compared at each this moment, which makes work easier and, therefore, helps improve its quality; - you can get a large number of comparisons of each indicator with others, which increases the accuracy of the assessment and opens up the opportunity to study quality more aspects of the research object than when using other methods;

It is possible to obtain not only the average assessment of the indicator given by each expert, but also the dispersion of this assessment, which makes it possible to further conduct a more in-depth economic and mathematical analysis

The paired comparison method is the simplest of the existing classification tests, since it involves comparing only two samples of a product

The advantage of the paired comparison method over ranking is that it is easier to make a judgment since the supervisor only has to compare two people at a time. Second advantage

The fact is that it provides the opportunity to put people with the same abilities on the same level.

The method of paired comparisons makes it possible to conduct a rigorous, statistically based analysis of the consistency of expert opinions and to identify whether the estimates obtained are random or not. Undoubtedly, the procedure of the paired comparison method is more complicated than the simple ranking method, but simpler than the sequential comparison method.

This method is very simple and it allows you to study a larger number of objects (compared, for example, with the rank method) and with greater accuracy.

The disadvantage lies in the need to perform a huge number of pairwise comparisons if you have to evaluate large groups. A supervisor with 60 employees will have to perform 1,770 comparisons! If the comparison is carried out according to five separate parameters, then the number of pairwise comparisons will increase fivefold. Typically, the paired comparison method is used in two cases: either when assessing small groups, or when assessing only one parameter - the overall efficiency of production activities

One disadvantage of pairwise comparisons is how experimental method is that comparing n stimuli requires making (n - 1) x (n / 2) judgments. For example, for 10 stimuli we need to obtain 45 judgments, and if we wanted to scale a set of 50 stimuli, we would need 1225 judgments.

The main disadvantage of the paired comparison method is that there are no ready-made software developments.

To use this method, you will have to instruct competent company employees to develop a special program or order it from a third-party developer. Additionally, there are no proven business valuation solutions based solely on pairwise comparisons, but rather a mathematical half-baked product that serves to create scaling tools. In addition, the test procedure itself looks rather monotonous, which affects the objectivity of research conclusions. And the quality of the results directly depends on the number of assessments and indicators assessed, as well as the correct selection of pairs and their unambiguous interpretation.

The paired comparison method is generally considered to be superior to direct ranking. This point of view is not entirely correct, and here's why.

The choice of method is always determined by the research situation and the objectives of the study. Naturally, if the basis for ranking can be unambiguously formulated, then the method of paired comparisons gives an excellent result and this method should be chosen. But there are situations when a clearly understood ranking basis is impossible and not really necessary.

The disadvantage of the method is the increase in the labor intensity of the procedure as the number of objects increases: already with 12-15 objects the procedure becomes labor intensive. In addition, different pairs of objects are sometimes compared by respondents according to different criteria, which leads to intransitivity of preferences. The method is widely used in expert assessments.

Testing is a common way to test knowledge. This method is valuable because it can be carried out remotely (which is convenient when there is a high territorial distribution of personnel), and also because it does not take much time and is quite simple to administer. The main limitation of testing is low validity when assessing many important competencies. It is almost impossible to assess competencies that are not directly related to knowledge (persuasive communication, team management, etc.) using testing. In addition, the development of tests “tailored” to the specifics of the Bank’s activities is a difficult and expensive process. Another disadvantage of testing is the high risk of rejection of the assessment results by both those being assessed and management.

2 Solving the problem of choosing the optimal product range based on the method of paired comparisons

Let's consider the process of purchasing the following washing powders:

Let us describe one of the ways to practically give quantitative content to the comparison of objects, actions or circumstances and build a corresponding table of comparisons.

In order to choose the highest quality and inexpensive washing powder, we will create a matrix of paired comparisons. For comparison, we need a scale of relative importance.

Table 1 - Relative importance scale.

Intensity relative importance	Definition	Explanation

	Incomparable	The expert finds it difficult to compare
	Equal importance	Equal contribution of two activities to the goal
	Moderate superiority of one over the other	Experience and judgment give slight superiority to one activity over another.
	Significant or strong superiority	Experience and judgment give strong superiority to one activity over another.
	Significant superiority	One of the activities is given such a strong predominance that it becomes practically significant

The matrix of paired comparisons is constructed according to the following rules:

If Myth and Tide are equally important, we enter the number 1 in position (Myth, Tide) of the comparison table,

If Myth is slightly more important than Tide - number 3,

If Myth is significantly more important than Tide - number 5,

If the Myth is clearly more important than the Tide - the number 7,

If the Myth is absolutely superior in importance to the Tide - the number is 9.

The numbers 2, 4, 6 and 8 are used to facilitate trade-offs between scores that differ slightly from the base numbers.

Rational fractions are used when it is desirable to increase the consistency of the entire matrix with a small number of judgments.

Suppose that by comparing washing powders 1, 2, 3 and 4, we obtain a comparison table that leads to an inversely symmetric matrix.

Matrix A is called inversely symmetric if for any i and k the relation holds:

aki = 1 / aik (1)

From this, in particular, it follows that aii = 1.

Matrix A is said to be consistent if for any i, k and l the equality holds:

aik* aki = ail (2)

Thus, the ideal comparison matrix is inversely symmetric and consistent.

The following statement is true.

THEOREM. A positive inversely symmetric matrix is consistent if and only if the order of the matrix and its largest eigenvalue are the same [2, 9].

Table 2 - Paired comparison matrix

Let's describe several ways to approximate the calculation of an eigencolumn

1st method:

1) sum up the elements of each row and write the results in a column,

2) add up all the elements of the found column,

3) divide each of the elements of this column by the resulting amount.

2nd method:

1) sum up the elements of each column and write the results in the column,

2) replace each element of the constructed column with its inverse,

3) add up the column elements from the reciprocals,

3rd method:

1) sum up the elements of each column,

2) divide the elements of each column by their sum,

3) add up the elements of each row of the resulting matrix,

4) write the results in a column,

5) divide each of the elements of the last column by the order of the original matrix n.

4th method:

1) multiply the elements of each row and write the results in a column,

2) extract nth root degrees from each element of the found column,

3) add up the elements of this column,

4) divide each of these elements by the resulting amount.

Each of these four methods, when applied to an ideal matrix, leads to the same exact result. Using one of the methods for approximate calculation of the eigenelements of this matrix (to be specific, the second), we found the eigencolumn, eigenvalue, and IS:

The sum of all elements of the resulting own column (it is called the priority column) is equal to 1. It allows us to summarize the analysis of the comparison table: among the compared elements 1,

2, 3 and 4 the highest priority is Myth (68%), followed by Tide (16%),

Dosya (9%) and Ariel (6%) respectively.

Conclusion

The dynamism and novelty of modern national economic problems, the possibility of the emergence of various factors influencing the effectiveness of decisions, require that these decisions be made quickly and at the same time be well justified. Experience, intuition, a sense of perspective, combined with information, help specialists more accurately select the most important goals and directions of development, find the best options for solving complex scientific, technical and socio-economic problems in conditions where there is no information about solving similar problems in the past.

Using the method of paired comparisons helps to formalize the procedures for collecting, summarizing and analyzing the opinions of experts in order to transform them into a form that is most convenient for making an informed decision.

But it should be noted that the method of paired comparisons cannot replace either administrative or planning decisions; it only allows one to replenish the information necessary for preparing and making such decisions.

The widespread use of paired comparisons is justified only where more accurate methods cannot be used to analyze the future.

Methods for paired comparisons are constantly being developed and improved. The main directions of this development are determined by a number of factors, including the desire to expand the scope of applications, increase the degree of use of mathematical methods and electronic computer technology, and also find ways to eliminate emerging shortcomings.

Despite the successes achieved in last years in development and practical use method of paired comparisons, there are a number of problems and tasks that require further methodological research and practical testing.

It is necessary to improve the system for selecting experts, increasing the reliability of group opinion characteristics, developing methods for checking the validity of comparisons, and studying hidden reasons that reduce the reliability of paired comparisons.

However, even today paired comparisons in combination with other mathematical and statistical methods are an important tool for improving management at all levels.

When solving the problem of choosing the optimal product range, we can draw the following conclusion: when purchasing washing powders, first we compared them, then, based on the scale of relative importance, we chose the best one, which turned out to be Myth powder. This type of powder is of the highest quality, relatively inexpensive in price, and is also in demand among buyers on the market.

Method of pairwise comparisons Pairwise comparison of those assessed among themselves on certain qualities and subsequent mathematical ranking in descending order

To assess the significance of consumer properties and functions, the method of pairwise comparison of properties and the method of prioritization are usually used.

Example. The method of ranking objects using the pairwise comparison method can be considered using a simple example. In table Figure 3.2 shows an example of an expert ranking six assessment objects using the pairwise comparison method. When performing an assessment, the expert compares pairs of objects as follows. He denotes the preference of one object over another by 1, otherwise he simply puts 0.

Ranking of six objects using pairwise comparison method

Example. In table Table 1.1 shows the ranking data of six Q objects by an expert using the pairwise comparison method. When performing an assessment, the expert compares pairs of objects. He denotes the preference of one object over another as 1, otherwise he denotes the situation as 0. In particular, the expert, as can be seen from the first row of the table. 1.1, preferred the first object to the second and considered that the first object was inferior to the third. In addition, the expert preferred the first object to the fourth, fifth and sixth. Therefore, in the end he received the sum of the ranks of the first object equal to four. The sum of the scores of each object compared to every other object is given -. naya in the last column of the table. 1.1, and is the result of the measurement on the order scale. The ranked series looks like Q4

At the next stage, using the pairwise comparison method, experts assess the significance of situations. The pairwise comparison technique consists of simultaneously presenting two situations to an expert, who must choose the situation that is most significant in terms of its impact on the final results of the activity. All situations of each block are compared in pairs with each other. Based on the comparison results, significance coefficients for individual situations are calculated (by all experts).

When developing standards for the labor intensity of functions performed by management personnel of commercial and sales divisions of organizations, factors influencing the labor intensity of functions were identified. Their list is ranked by the method of pairwise comparisons (see Table 2.13).

Taking into account the results of the method of pairwise comparisons of factors and the above provisions, the following main factors have been identified that influence the number of management personnel in managing the commercial and sales activities of the organization

Drawing up the initial version of the attachment. When solving a transport problem in a network form using a computer, the initial transportation scheme is chosen arbitrarily. When drawing up a transportation scheme manually, the initial option can be obtained for the entire polygon or for its individual parts using the simplest methods of attaching the method of pairwise comparison of options, the method of differences, the method of circular dependence, but with the calculation of avoiding clearly oncoming and unnecessarily long-distance transportation, etc.

The method of pairwise comparison of options is used when there are only two points of consumption and two points of production.

It is proposed to select the OPF on the basis of expert methods, in particular the well-known method of pairwise comparison, by conducting a collective examination by group members (Tables 5.7-5.9).

Pairwise comparison scaling is useful when the number of brands is limited because it requires direct comparison and obvious choice. However, with a large number of brands, pairwise comparisons become very cumbersome. Among other disadvantages is the possibility of violating the transitivity assumption, which will lead to bias in the results if the order of presentation is changed. Pairwise comparisons have little to do with the market within each choose from a variety of options. It is also possible that respondents prefer one object to others, but they do not like it at all. Box 8.2 shows some new aspects of using scales.

Paired comparison method

There are two complementary methods of ranking by weighting coefficients and a pairwise comparison method.

The classification of methods is shown in Fig. 6.15. In many ways, it resembles the classification of methods for analyzing problems with an infinite number of feasible solutions, however, it also has its own characteristics associated with the finiteness of the number of solutions. Thus, after constructing the solution matrix (3.5), finding effective points is carried out by a simple, specially organized search of all solution options and their pairwise comparison. This procedure remains effective with a sufficiently large number of options and criteria, so the issue of identifying an effective set does not cause any difficulties and will not be considered further. Let us only emphasize that we can talk about an effective set only when the direction of improvement is given

Pairwise comparison. When forming expert assessments, an order scale is often used. The question of comparison is decided according to the principle of better or worse, more or less. This is largely due to the peculiarities of the psychology of a person who compares objects in pairs. Therefore, when constructing an order scale and a ranked series, experts should offer a pairwise comparison method.

Method of complete pairwise matching. To avoid the possible error of favoring some property /th over property / not because it is more important, but because it was accidentally placed first in the pair when compared using the second pairwise matching method, comparison is made not only in the order property / - property /, but also in reverse order property / - property / .

Pairwise comparison. When using the expert method, an order scale is often used - assessment based on the principle of better or worse, more or less. This is due to the peculiarities of human psychology, which usually compares objects in pairs. Therefore, to obtain a ranked series of evaluated objects, it is preferable for experts to propose a pairwise comparison method. When performing an assessment, an expert, in the simplest case, compares pairs of objects in the following way: the preference of one object over another is denoted by 1, otherwise he denotes the situation as 0. The sum of all assessments for one object gives its overall comparative assessment. Let's give a simple example of evaluation.

The assessment results obtained by the method of paired comparisons have the least difficulties and the greatest thoroughness. According to this method, the expert does not examine all objects at once, but in pairs. The expert’s task is significantly simplified and comes down not to assigning ranks, but to comparing each pair of workers and choosing three alternatives: better, worse, and the same. The method of paired comparisons is a type of expert assessment method, the use of which will be discussed later.

Because of this, the result of pairwise comparison most accurately reflects subjective preference, because the least restrictions are imposed on the choice here and the method does not impose a priori conditions on the expert.

Assessing the significance of consumer properties can be done using pairwise comparison and prioritization methods. The value of Q can also be calculated

What value orientations will a young specialist prefer when choosing his future job? Rank them using the method of paired comparisons (Table 5.3).

We have developed a methodology for determining the optimal number of management personnel for organizations in the electrical industry. The latter are characterized by small-scale and multi-product production, big amount consumers of their products, which were decisive circumstances when choosing a set of factors influencing the number of management personnel. When developing standards for the labor intensity of functions performed by management personnel of commercial and sales divisions of organizations, factors influencing the labor intensity of functions were identified. In order to identify the most important factors, their general list was ranked using the method of pairwise comparisons.

The next set of issues that need to be resolved within the framework of the ORGPRO business game is the choice of the organizational and legal form of the designed enterprise. Modeling of the choice of OPF in ORGPRO is carried out on the basis of expert methods, in particular the well-known method of pairwise comparison, through a collective examination by the team participating in the game. The criteria for choosing OPF in the model are considered

The most convenient form of implementing this principle is the situation when the whole is taken as a unit, and the component factors (parts) are expressed in fractions of a unit. The weight of each factor in fractions of one in sociological and mathematical models is determined, as a rule, by experts using social method pairwise comparison, covered in sufficient detail in the literature.

Blind testing of soft drinks, while consumer decisions are heavily influenced by factors such as self-perception and branding, may be a poor indicator of potential market success. The launch of New oke is an example of this situation. In blind paired comparison testing, New oke had a clear advantage, but the launch of the new brand was less successful, mainly because image plays a big role in the purchase of soft drinks.

A number of stochastic methods for solving the stated optimization problem of parallelization of calculations have been developed. In the first method - the stochastic method of pairwise optimization of subgraphs - the search for an optimal solution is carried out through mutual (stochastic) transfer of vertices between different pairs of subgraphs of the algorithm graph. The second method - the Monte Carlo method of random walk of the vertices of the algorithm graph through subgraphs - is based on identifying the vertices of the algorithm graph with some particles performing random walks across subgraph areas in a potential force field, the role of the potential of which is played by the minimized functional. The most probable state of such a system of particles corresponds to the minimum potential --and, therefore, is the desired solution. The search for such a state is carried out by the Monte Carlo method using a special simulated annealing procedure. The third method, the stochastic steepest descent method, is based on the use of a discrete analogue of the gradient of the functional being minimized. All developed methods are implemented in software and are part of the PARALLAX program system. The created programs were tested and their work was compared using simple examples.

American certification practice excludes typification of procedures for this process and is focused on individual assessments. Frequency varies at different enterprises (on average - once a year). However, during the reorganization of the Chrysler company, its director, Lee Iacocca, conducted certification on a quarterly basis. The expert appraisers are usually the manager-manager, i.e. the boss of the person being certified, the expert council (committee of supervisors), colleagues and subordinates of the person being certified, third-party specialists, the person being certified (self-assessment method). Combinations of these groups of evaluators are possible. To establish an assessment, various methods are used: questionnaires with closed or semi-open questions, graphic employee rating scales, questionnaires various types, methods of monitoring workers (especially in critical situations), methods of classification, pairwise comparison of qualities, management by objectives45. The latter is associated with setting specific measurable and developmental goals for the employee, which is done in cooperation between the boss and the employee, with subsequent assessment of the degree of achievement of the goals.