Businesspeople love to measure things. We measure sales and profits, which is easy enough, but then we also want to measure everything else. This is called the ‘administrative bias’ of the business world: using measurements as a guide to decision-making, and ignoring anything that cannot be measured.
There are specialists who believe that everything can be measured, from customer satisfaction and organizational climate, to effective communication and anything else that might concern you. When measurements are used, they argue, we are more objective and can better establish clear goals and objectives. No one can argue with that noble aim, but when measurements are used as universally valid tools, we are, more often than not, likely to head the wrong way.
Let’s use as an example the way ‘satisfaction’ is often measured. You are likely to find questions that ask respondents to give an answer between ‘1’ and ‘5,’ where ‘1’ means ‘very dissatisfied’ and ‘5’ means ‘very satisfied.’ Can you use these scales and get reliable answers across cultures and nations? In my opinion, the answer is an emphatic ‘no!’
An experience a colleague of mine recently had illustrates this point. He is a consultant for a very large and well-known firm, and the project he was involved in dealt with a merger. The client was interested in knowing how well the members of the smaller company involved in the merger were adapting. His data consisted of a large survey (translated from English) that measured in different ways how well they had adjusted to the merger.
The answers to most of the questions were above ‘4,’ on a scale of ‘1’ to ‘5,’ where the higher numbers indicated a more positive attitude toward the company and the merger. Thus, he could argue that in general terms the adaptation to the new organization had been successful. To complement these objective results, he decided to do qualitative interviews with various groups of people. In these qualitative interviews, he discovered that most people were very unhappy about the merger, and that most people did not feel that they belonged to the new organization. In other words, he obtained contradictory results! He decided to pay attention to the qualitative data, and threw away the ‘objective’ results as irrelevant information.
Indeed, in my experience, Mexicans very often give a higher rating to the company they work for when objective scales are used, as compared to the results you might encounter when you do these kinds of studies in the United States. Stated another way, a result of ‘3’ does not mean the same in the United States as it does in Mexico. In Mexico, extreme negatives are avoided, so when you get a ‘3’ it may be the same as a ‘1’ in other cultures. A theater director in Mexico tells me that when he asks his friends how they liked his latest show, he knows ‘good’ means ‘so-so’ and ‘so-so’ means ‘terrible’.
Researchers have known about these cultural variables for a long time. U.S. psychologists Gerardo Marín and Barbara VanOss Marín have explained at length the various problems associated with using instruments across cultures, and in particular with conducting research among U.S. Hispanics. Studies have shown, for example, that this group tends to give more extreme answers to questions where a scale is used, and less likely to give the ‘moderate’ answers.
According to the authors, this preference is based on a cultural style that sees extreme answers as more sincere. Other studies show that certain cultures have a greater tendency to give ‘yea-saying answers.’ Therefore, when the same instrument is used to compare people from different cultures, you are likely to get very different numbers, and will probably reach false conclusions.
I am not arguing that we should avoid measuring because of these problems. But, unless an instrument has been adopted culturally and has been tested for validity, measuring is likely to produce a very poor reflection of reality.
