Psychometrics in the Construction Industry

Psychometrics in the Construction Industry Abstract This dissertation is about evaluating the use of psychometrics in the construction industry. The document will also present the main aims and objectives of the study. A comprehensive literature study has also been carried out with the objective of gaining an in-depth understanding of the topic of psychometrics. The literature review chapter will identify a selection of the testing methods that are available on the market, test design, usage within industry and the associated costs with such tests. A methodology and potential constraints have also been identified, and finally a timetable has been proposed. The survey indicated that companies are beginning to realise the benefits of psychometric testing, 1.1 Introduction This dissertation proposal has been compiled as part of the BSc (Honours) in construction management. It is necessary for the individual to produce a research project on the subject of his/her own choice that is construction related. The topic chosen for the purposes this dissertation is that of psychometric science. 1.2 Nature of the Problem Few people in the construction industry would disagree that effective team building is essential to success; however bringing together a bunch of individuals does not automatically produce a team (Franks 2002). Teams have to be assembled by taking into account many factors, including individual skill and particularly the personal characteristics of the individual member, psychometric testing is one way of ensuring the correct team members are recruited, thus maximizing the capabilities of the team. Psychometrics covers a range of testing methods from IQ and aptitude tests to personality questionnaires and numbercy indicators. Many organisations now use this form of testing routinely in their recruitment processes especially at senior level, according to Corcoran (2005) it forces an examination and sound understanding of the competencies inherent in the role in question and links the candidate’s competencies with those to the role. Hampton (2002) suggests that these tests are not just limited to recruitment; they can also be used in career development to raise awareness of the individual’s own strengths and weaknesses. The need for this research stems from the fact that compared to other industries where the use of psychometrics is widespread; construction firms seem to be blind to its benefits. As an industry that relies heavily on good teamwork, construction could gain from a wider implementation of psychometrics in both recruitment and team development. Also if survey statistics are to be believed, up to 56% of candidates exaggerate their skills/experience on their CV. Making a recruitment decision purely on the content of a CV and interview alone, could prove to be unreliable. Whilst a CV certainly provides an overview of a candidate’s ability, it doesn’t necessarily illustrate their suitability for the role. Likewise, an applicant may handle an interview extremely well but not have the capability to deliver the role itself. Psychometric testing therefore attempts to add a third dimension to the team building/recruitment process. The need for this research is therefore obvious and will be designed to test the following hypothesis: â€Å"Psychometric testing is critical to successful team building in the Irish construction industry.† 1.3 Rationale As an industry that relies heavily on good teamwork, it is essential that that the right people are selected for the right jobs. This rationale for this dissertation is to establish whether or not construction could benefit from the wider implementation of psychometric testing, and also to evaluate the current usage of psychometric testing in the industry. Establishing the usage of psychometric testing is a critical part of this research, it establishes the current trends in the construction industry towards recruitment and team development and once this is established it may help organisations see the way forward. To achieve this, a literature review, a pilot study and postal questionnaires shall be conducted. Initial research has indicated that the construction industry is slow to realise the benefits of testing in both recruitment and team development. Employers need to be made aware of both the advantages and disadvantages of psychometrics which enable them to make an informed decision as to whether or not psychometrics is appropriate to their organisation. 1.4 Aims The general aim of this research is to investigate if psychometric testing is either a beneficial or a counterproductive exercise to a construction organisation in the selection of team members. To establish the extent of use of psychometrics within the Irish Construction Industry. To gain a comprehensive understanding of the topic of psychometrics. 1.5 Objectives of the Research To provide a general overview of the topic of psychometric science. To review the range of testing methods available. To establish the apparent advantages and disadvantages to a construction organisation in adopting psychometric testing. To assess the current usage of psychometrics within the Irish construction industry. 1.6 Hypothesis â€Å"The benefits of Psychometric Testing are not fully appreciated in the Irish Construction Industry† 1.7 Outline Methodology The methodology section for this dissertation will provide a rationale for the choice and use of particular strategies and tools for gathering and analyzing the data. A quantitative research strategy was used as part of the methodology, which helped in the execution of both the literature review and the questionnaire. The literature review was one the first stages of the methodology; this was used to highlight the importance of using psychometric testing in both recruitment and team building. The methodology also involved the production of structured questionnaire; these were compiled to assess the current usage of psychometric testing among organisations within the Irish Construction Industry. A pilot study of the questionnaire was first conducted to provide a trial run for the questionnaire, which involves testing the wording of the questions and identifying any ambiguous questions. A detailed analysis of the results was then produced; it will consist of questions, answers, tables, figures and charts to represent the data. Finally an interview was conducted with a member of an organisation utilising psychometrics as part of their recruitment process. Methodology 2.1 Introduction This chapter will set out to outline the methods of data collection that were adopted for the purposes of the dissertation. The methodology chapter for this dissertation of 4 main parts: A general definition and review of the approach to the research. A justification for the selection of the hypothesis, aims and objectives. A description of the data collection and analytical procedures. A review of the methodological complexities that were encountered, including the limitations and constraints on the research. 2.2 Research Strategy A primarily quantitative research strategy was adopted for the purposes of this dissertation. The rationale for the selection of this method is that quantitative research is used when the individual needs to identify facts about a concept, question or an attribute. This ties in closely with the objectives and aims of the report. Naoum (1998) describes quantitative research as â€Å"an inquiry into a social or human problem, this closely relates to the initial aims objectives of the research† 2.3 Theoretical Framework This dissertation will be approached using a both sociological and psychological framework. The former of which is concerned with cultural roles, norms and values and the structuring and functioning of society. Also explored will be the psychological aspects such as personality traits and relationships between individual team members. 2.4 Historical Framework One of the first attempts to scientifically measure the differences between individuals was made by Sir Francis Galton in the 19th Century when he tried to show how the human mind could be systematically mapped into different dimensions, identifying how individuals differed in terms of their ability. The psychological approach and the first published test can be credited to Sir Alfred Binet. 2.5 Psychological Framework Psychometric testing is fundamentally a form of psychological assessment, Postal Questionnaires Postal questionnaires along with a cover letter will be distributed to a number of Irish construction firms to gather information on the current usage and the popularity of psychometrics within the Irish construction industry. Before these questionnaires are distributed, it will be necessary to complete a pilot study before the final data is collected. The pilot study will provide a trial run for the questionnaire, and its main aim is to test the wording of the questions, identify any ambiguous questions and test the techniques being used Once the pilot study is complete, the main questionnaire will be distributed. To establish an accurate sample of the industry, it will be necessary to get a minimum number of responses to the main questionnaire. This number will be determined by the constraints of time imposed on the research, however it is expected that this will be in the region of 10-15 responses. This study will be primarily aimed at the larger construction firms in the industry. A random selection procedure will be adopted, to ensure that a representative sample of the firms is taken. Where possible, questionnaires will be delivered and collected to ensure a high response rate. Otherwise when replies are not received, the enquiries will be followed up by subsequent emails and phone calls.Tables, graphs, bar charts and histograms have been identified as a means of presenting the data from the questionnaires. The Research Sample Due to the relatively high costs involved in the implementation of psychometric instruments, the smaller organisations in the industry tend to avoid using this method of recruitment and team development. To establish an accurate representation of the industry, the research sample was confined to those in the Top 100 Irish construction companies (The Times, 2007). This sample was further refined down 35 using a random selection procedure. The types of organisation were sampled from a number of different disciplines within the construction industry, including: 1. Civil Engineering Contractor’s 2. Building Contractor’s 3. Building Services Contractor’s 4.Project Management organisations 5. Engineering Consultants 6. Multi-disciplinary organisations Literature Review This paper reviews the literature on psychometric testing by employers, and considers whether or not psychometric testing is a critical element of the recruitment process. It will seek to outline the fundamental benefits associated with the concept of psychometric testing and determine current trends regarding the usage of these tests in the Irish Construction Industry. According to Reber (1995) psychometric testing has been defined as â€Å"Pertaining to mental testing in any of its facets, including assessment of personality, evaluation of intelligence, determining aptitudes.† Rust (2004) classifies these tests into two broad categories: those that assess ability and those that assess personality. The way that organizations operate has changed dramatically in the last 30 years, there are now fewer levels of management than there were, and management styles tend to be less autocratic. In addition, there appears to be a move towards more knowledge based and customer focused jobs, which means that individuals have more autonomy even in the lower levels within organizations (Edenborough 2004). The effects of these changes mean that a person’s personality is seen as more important now than it was in the past. The structure of the modern workforce within the Irish Construction Industry is also rapidly changing and teamwork is becoming increasingly important, here a psychometric test can prove an invaluable tool, not only does it make sure that each team has the required skills but it also ensures that the people who make up team can work well together or that they can â€Å"gel effectively†. An article in China Staff magazine (2003) points out that this doesn’t mean that everybody in the team should be the same, rather than that you end up with a combination of personalities who can work well together. Each type of test may not suit every organisation, the design of a testing procedure must be based on what the user is wishing to achieve from the results. Burke (1995) suggests that the various types of â€Å"tests are designed for a purpose and the use of a particular test will vary according to the objectives of the assessment. In order to ascertain which tests you should use, it is necessary to gain an in-depth understanding of the characteristics necessary to perform well at the job. Recruitment and Selection Many organisations now use psychometric testing routinely in their recruitment process especially at a senior level. It forces an examination and sound understanding of the competencies inherent in the role in question and links the candidate’s competencies with those of the role. It makes for sound selection decisions and identifies development areas for the successful candidate (Corcoran 2005). Ideally, if psychometrics is used at the initial selection stage, employers are immediately aware of any areas ability or personality where an employee might not be an exact match for the job, and can make informed decisions as to whether the employee should be selected and trained, or rejected. Type of Testing Available Personality Measurement Tests These tests are used to assess how a person is likely to react to situations with people, the types of relationship they prefer to establish with others, and the motivational factors that influence a person’s actions. These are often known as tests of disposition. These tests do not generally contain questions to which there is only one correct answer. The answers given to these questions tend to reflect a tendency to interpret situations or respond to other people in particular ways. Burke (1995) outlines the typical qualities assessed by disposition tests; these include anxiety, sociability, perseverance, dominance, fear of failure and resistance to stress. A common approach to in psychometrics is to see personality as a combination of traits. A trait can be any characteristic way of behaving, thinking, feeling or operating. The measure of these ‘traits’ has always been a matter of interest for many years, it has been recognized that personality is of great impo rtance in peoples success at work, no less than the approach they take in other aspects of their life (Edenbourgh 2005). Psychologists often refer to these as the ‘big five’ personality traits. Openness appreciation for art, emotion, adventure, unusual ideas, imagination, curiosity, and variety of experience. Conscientiousness a tendency to show self-discipline, act dutifully, and aim for achievement; planned rather than spontaneous behavior. Extraversion energy, positive emotions, urgency, and the tendency to seek stimulation and the company of others. Agreeableness a tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others. Neuroticism a tendency to experience unpleasant emotions easily, such as anger, anxiety, depression, or vulnerability; sometimes called emotional instability. The argument runs that personality questionnaires do not test behavior directly but rather how the respondent chooses to describe his or her own behavior. It also argued by Cronbach (1966) that such questionnaires or self-report inventories, as they are sometimes known, indicate typical behavior. One corollary to this view by Reber(1995) is that such behavior is not very amenable to change, these type of tests have no right or wrong answers as such, however there is certainly the right and wrong personality mixes for certain jobs. Ability and Aptitude Tests ‘Aptitude’ and ‘Ability’ tests are not always precisely separated; many of the tests in these categories are used to look at behavior, often of an intellectual or cognitive nature. These are usually tests of general intelligence, which can be regarded as mental horsepower or as measure of a person’s ability to process information. Tests of ability generally assess the broader areas of what a person can do. While scores on such tests are influenced by education and training, they are not designed to assess specific areas of knowledge or skill. Examples of ability tests are: measures of verbal reasoning, numerical reasoning and mechanical reasoning. China Staff magazine (2003) defines ability testing â€Å"as a measure of a persons potential to learn the new skills needed for a new job or to cope with the demands of a training course.† Tests of aptitude on the other hand are often used to assess how well an individual is likely to perform in a training programme or in a job Edenbourgh (2005) attempts to distinguish between the two: The term ‘aptitude’ is usually reserved for those tests directed at predicting whether skill in a particular area can be acquired. ‘Ability tests’ on the other hand tend to be reserved for measures of less job-specific though often still job-related intellectual tests. Attainment Tests These are often referred to as achievement tests; these are more commonly used in relation to educational assessment rather than in occupational settings. They represent standard ways of assessing the amount of skill currently reached or attained by an individual in a particular area. Tests of attainment are often linked with ability and can be quite similar; the difference is that they test specifically what people have learnt e.g. mathematical ability or typing skills. Of course what people have learned does depend on their ability in that domain in the first place, so the scores of the two types of test are linked. Another differentiation between the two is that ability tests are retrospective in nature; they focus on what has been learnt and on what a person knows and can do now. Ability tests on the other hand are prospective: they focus on what candidates are capable of learning and achieving in the future or on their potential to learn. (China Staff Magazine 2003) Assessment Centres Psychometric testing often takes place within the context of an assessment centre. Edenbourgh (2004) points outs that organisations use a range of selection methods, including interviews, group exercises, role-playing, in-basket exercises as well as psychometric testing in order to select from a group of job applicants. Candidates are likely to encounter tests that will simulate situations that are likely to be encountered in a particular role. Assessment centre selection methods can take one or two days to complete, and because of the cost and time taken these are generally only associated with larger organisations. Assessment centers are usually used after the initial stages of the selection process and usually follow the initial job interview. They are commonly held either on employers’ premises or in a hotel and are considered by many organizations to be the fairest and most accurate method of selecting staff. This according to Psychometric Success is because â€Å"a number of different selectors will get to see the individual over a longer period of time and have the chance to see what he/she do, rather than what the individual says they can do.† Tests that are carried out within these assessment centres are often conducted by HR personnel however it is often the case that external consultants will be brought in to administer the tests. Ability to distinguish between candidates Any selection tool should be able to distinguish between candidates in two ways: 1) The tool should be accurate and precise. Any differences in results between candidates should be real and not simply random or due to errors or inaccuracies. This is known as reliability, and psychometric tools have consistently been shown to be more reliable than other common selection methods. 2) The differences in results between candidates should reflect differences in their ability to perform in the job. This is known as criterion-related validity. As shown in Diagram 3, psychometric tools do predict job performance well, provided that the right tool has been chosen, for example, via a careful analysis of the attributes required for the job. Differences in results do therefore relate to differences in ability to perform well in the job. The importance of getting this right is one reason why most instruments are only sold to those who have been trained in their use. Reliability One of the main problems associated with psychometric testing is the issue of test reliability. For instance, if there is high competition for a job  then there is strong incentive for candidates to present themselves favorably.   Similarly candidates can be tempted to respond to personality questionnaires in a way they assume the employer wants.   If we consider a construction related job, how many candidates will disagree with the statement I enjoy working in team environment? Edenbourgh (2005) identifies one way of increasing the reliability in personality questionnaires, this is to use forced-choice formats whereby candidates are presented with a combination of statements and must indicate a preference between them, this method of combining items is known as ‘ipsative’ scaling. This can be contrasted with the ‘normative scaling’, in which the candidate is instructed to choose one statement from a selection of 2 or more. For example in an ipsative test, a question might ask a candidate to choose whether they are more hard working or creative.   In this case the candidate cannot make themselves look good on both of the statements, it therefore can be said that this method can increase the reliability of certain tests and give the assessor a greater profile of the candidate’s personality. Disadvantages to Psychometric Testing In many cases, psychometric tests and questionnaires have been put together by people with little background in psychometrics and they may have very little actual utility and value for the purposes for which they are marketed. Although British Psychological Society (BPS) has a standard qualification system, it is not unusual for tests to be used by people who are not adequately trained to score them. Also just because a person attends a recognized training course it cannot be guaranteed that they will use tests and questionnaires correctly since some instruments, particularly personality questionnaires, require considerable experience and the possibility of poor interpretation of results is ever-present. Another disadvantage of psychometric testing is the use of personality questionnaires to assess an individual’s ability or skill in a particular area. For example, if a person scores highly on a test called Leadership, this does not mean that he or she will actually possess a high level of leadership skill, instead it means that the individual has the basic personality characteristics that are commonly found amongst effective leaders and, with sufficient experience and given the development of certain necessary skills, has the potential to become an effective leader. (Team Focus 2004) Cost The costs associated with these tests are often substantial. This implies that employers are unlikely to be using them merely in order to follow a management trend, but because they firmly believe that the tests are useful in recruiting job applicants with the right skills and attributes. According to a recent survey by Incomes Data Services (IDS 2000), the sums involved in testing can be quite substantial. Assuming that employers prefer to administer and interpret the tests themselves, rather than employing external consultants, then the costs will include initial training in test use, since it is necessary to be qualified in order to use tests. Further costs will include start-up kits such as manuals or computer software, and consumables such as question-and-answer booklets for test candidates. (Jenkins 2001) In order to become a trained assessor, a person needs to obtain certification from the British Psychological society (BPA), and requires the completion of two courses, Level A and Level B. The BPS Level A covers the use and interpretation of ability tests. The BPS Level B covers the use and interpretation of personality tests. In general the test companies will often provide courses leading to these qualifications. These courses usually take about five days for each of Level A and Level B. The average cost per trainee is about â‚ ¬1,900 for Level A and about â‚ ¬2,200 for Level B. It is possible to train some employees to be test administrators, rather than test users, although at least one person in the organisation must be a qualified test user if the company is to be permitted to buy tests. Test administrators can brief candidates prior to testing, and hand out and collect test papers, but cannot score tests, except under supervision, nor interpret the results. Source: IDS survey of British companies Training to become a test administrator is often cheaper than becoming a test user and a fee of perhaps â‚ ¬1000 would be charged for a test administrator course. However, it is likely that larger companies would want to have several people qualified to test user standard, as well as more employees qualified to administer the tests. According to Edenborough (2004) the major component of cost is the expense of training company staff to be able to obtain and utilize the tests properly, precise costs of psychometric tests will vary but say; in a recruitment context it is unlikely that the expenses are to be more than a couple of percent of a salary. The benefits concerned are also likely to dwarf such figure. Estimating the cost-benefit of psychometric instruments Companies can examine the effect of a recruitment process on a company’s performance by analyzing how well a candidate performs in the role and therefore how much they add to productivity, and balancing this against the investment in their recruitment. Research across a range of organisations suggests that the amount an employer can save, per employee recruited, per year, is: (Validity of the test) multiplied by (Caliber of those selected) multiplied by (Standard deviation of job performance) Minus (Cost of the selection) divided by (Proportion of applicants selected) This formula, known as the utility equation, may look complex, but it has successfully been used by HR professionals in order to justify the use of psychometric instruments and other objective assessment methods. (McHenry 1990) Calculating the cost benefits of psychometric instruments The utility equation can be used to calculate the financial benefit of using psychometric testing as part of an organisations selection process. An example of this would be a company recruiting a person for a position with a â‚ ¬60,000 salary, is possible to compare the financial benefit achieved by using a recruitment process of unstructured interviews that have a validity of 0.18, compared to using personality questionnaires and ability tests that have a combined validity of 0.60. (See table in Appendix 1) Using the utility equation and a conservative estimate of the extent to which this position will impact on company productivity, it can be predicted that using psychometric tools would increase the organization’s productivity by â‚ ¬8,520 compared to using unstructured interviews. This calculation assumes that the candidate remains in the job for one year; if they remain for three years the productivity increase can be as much as â‚ ¬25,160. Source: McHenry (1990) Current Usage within Industries According to Psychometric Success testing is now used by over 80% of the Fortune 500 companies in the USA and by over 75% of the Times Top 100 companies in the UK. Information technology companies, financial institutions, management consultancies, local authorities, the civil service, police forces, fire services and the armed forces all make extensive use of use psychometric testing.   In 2006 a survey identified that well over 2,500 personality questionnaires are available on the market, this number is increasing each year as dozens of new companies appear with their own ‘new’ products. The reasons for test use suggest that the perceived objectivity of tests, their predictive abilities, as well as their ability to filter out unsuitable candidates were important reasons for test use in both the public and private sectors, however according Jenkins (2001) equal opportunities legislation may have also encouraged employers to use tests as part of a drive to fairer selection. There are also a number of studies linking increases in test use to the spread of greater professionalism in the human resource management function, and to multi-national companies imposing standard selection procedures throughout their businesses. It has been estimated that it can cost organisations the equivalent of the whole of the first years costs (salary and any other benefits) to put right a wrong recruitment decision. Research reported in a number of test manuals shows that using good assessments professionally as part of the recruitment process dramatically increases the number of right dec isions you make. In order to establish reasons for organisations use of psychometric testing, it was necessary to look at statistics carried on British organisations, due to the limited availability of published research in Ireland. Surveys carried out by Bevan and Fryatt (1988) and by Psychometrics in the Construction Industry Psychometrics in the Construction Industry Abstract This dissertation is about evaluating the use of psychometrics in the construction industry. The document will also present the main aims and objectives of the study. A comprehensive literature study has also been carried out with the objective of gaining an in-depth understanding of the topic of psychometrics. The literature review chapter will identify a selection of the testing methods that are available on the market, test design, usage within industry and the associated costs with such tests. A methodology and potential constraints have also been identified, and finally a timetable has been proposed. The survey indicated that companies are beginning to realise the benefits of psychometric testing, 1.1 Introduction This dissertation proposal has been compiled as part of the BSc (Honours) in construction management. It is necessary for the individual to produce a research project on the subject of his/her own choice that is construction related. The topic chosen for the purposes this dissertation is that of psychometric science. 1.2 Nature of the Problem Few people in the construction industry would disagree that effective team building is essential to success; however bringing together a bunch of individuals does not automatically produce a team (Franks 2002). Teams have to be assembled by taking into account many factors, including individual skill and particularly the personal characteristics of the individual member, psychometric testing is one way of ensuring the correct team members are recruited, thus maximizing the capabilities of the team. Psychometrics covers a range of testing methods from IQ and aptitude tests to personality questionnaires and numbercy indicators. Many organisations now use this form of testing routinely in their recruitment processes especially at senior level, according to Corcoran (2005) it forces an examination and sound understanding of the competencies inherent in the role in question and links the candidate’s competencies with those to the role. Hampton (2002) suggests that these tests are not just limited to recruitment; they can also be used in career development to raise awareness of the individual’s own strengths and weaknesses. The need for this research stems from the fact that compared to other industries where the use of psychometrics is widespread; construction firms seem to be blind to its benefits. As an industry that relies heavily on good teamwork, construction could gain from a wider implementation of psychometrics in both recruitment and team development. Also if survey statistics are to be believed, up to 56% of candidates exaggerate their skills/experience on their CV. Making a recruitment decision purely on the content of a CV and interview alone, could prove to be unreliable. Whilst a CV certainly provides an overview of a candidate’s ability, it doesn’t necessarily illustrate their suitability for the role. Likewise, an applicant may handle an interview extremely well but not have the capability to deliver the role itself. Psychometric testing therefore attempts to add a third dimension to the team building/recruitment process. The need for this research is therefore obvious and will be designed to test the following hypothesis: â€Å"Psychometric testing is critical to successful team building in the Irish construction industry.† 1.3 Rationale As an industry that relies heavily on good teamwork, it is essential that that the right people are selected for the right jobs. This rationale for this dissertation is to establish whether or not construction could benefit from the wider implementation of psychometric testing, and also to evaluate the current usage of psychometric testing in the industry. Establishing the usage of psychometric testing is a critical part of this research, it establishes the current trends in the construction industry towards recruitment and team development and once this is established it may help organisations see the way forward. To achieve this, a literature review, a pilot study and postal questionnaires shall be conducted. Initial research has indicated that the construction industry is slow to realise the benefits of testing in both recruitment and team development. Employers need to be made aware of both the advantages and disadvantages of psychometrics which enable them to make an informed decision as to whether or not psychometrics is appropriate to their organisation. 1.4 Aims The general aim of this research is to investigate if psychometric testing is either a beneficial or a counterproductive exercise to a construction organisation in the selection of team members. To establish the extent of use of psychometrics within the Irish Construction Industry. To gain a comprehensive understanding of the topic of psychometrics. 1.5 Objectives of the Research To provide a general overview of the topic of psychometric science. To review the range of testing methods available. To establish the apparent advantages and disadvantages to a construction organisation in adopting psychometric testing. To assess the current usage of psychometrics within the Irish construction industry. 1.6 Hypothesis â€Å"The benefits of Psychometric Testing are not fully appreciated in the Irish Construction Industry† 1.7 Outline Methodology The methodology section for this dissertation will provide a rationale for the choice and use of particular strategies and tools for gathering and analyzing the data. A quantitative research strategy was used as part of the methodology, which helped in the execution of both the literature review and the questionnaire. The literature review was one the first stages of the methodology; this was used to highlight the importance of using psychometric testing in both recruitment and team building. The methodology also involved the production of structured questionnaire; these were compiled to assess the current usage of psychometric testing among organisations within the Irish Construction Industry. A pilot study of the questionnaire was first conducted to provide a trial run for the questionnaire, which involves testing the wording of the questions and identifying any ambiguous questions. A detailed analysis of the results was then produced; it will consist of questions, answers, tables, figures and charts to represent the data. Finally an interview was conducted with a member of an organisation utilising psychometrics as part of their recruitment process. Methodology 2.1 Introduction This chapter will set out to outline the methods of data collection that were adopted for the purposes of the dissertation. The methodology chapter for this dissertation of 4 main parts: A general definition and review of the approach to the research. A justification for the selection of the hypothesis, aims and objectives. A description of the data collection and analytical procedures. A review of the methodological complexities that were encountered, including the limitations and constraints on the research. 2.2 Research Strategy A primarily quantitative research strategy was adopted for the purposes of this dissertation. The rationale for the selection of this method is that quantitative research is used when the individual needs to identify facts about a concept, question or an attribute. This ties in closely with the objectives and aims of the report. Naoum (1998) describes quantitative research as â€Å"an inquiry into a social or human problem, this closely relates to the initial aims objectives of the research† 2.3 Theoretical Framework This dissertation will be approached using a both sociological and psychological framework. The former of which is concerned with cultural roles, norms and values and the structuring and functioning of society. Also explored will be the psychological aspects such as personality traits and relationships between individual team members. 2.4 Historical Framework One of the first attempts to scientifically measure the differences between individuals was made by Sir Francis Galton in the 19th Century when he tried to show how the human mind could be systematically mapped into different dimensions, identifying how individuals differed in terms of their ability. The psychological approach and the first published test can be credited to Sir Alfred Binet. 2.5 Psychological Framework Psychometric testing is fundamentally a form of psychological assessment, Postal Questionnaires Postal questionnaires along with a cover letter will be distributed to a number of Irish construction firms to gather information on the current usage and the popularity of psychometrics within the Irish construction industry. Before these questionnaires are distributed, it will be necessary to complete a pilot study before the final data is collected. The pilot study will provide a trial run for the questionnaire, and its main aim is to test the wording of the questions, identify any ambiguous questions and test the techniques being used Once the pilot study is complete, the main questionnaire will be distributed. To establish an accurate sample of the industry, it will be necessary to get a minimum number of responses to the main questionnaire. This number will be determined by the constraints of time imposed on the research, however it is expected that this will be in the region of 10-15 responses. This study will be primarily aimed at the larger construction firms in the industry. A random selection procedure will be adopted, to ensure that a representative sample of the firms is taken. Where possible, questionnaires will be delivered and collected to ensure a high response rate. Otherwise when replies are not received, the enquiries will be followed up by subsequent emails and phone calls.Tables, graphs, bar charts and histograms have been identified as a means of presenting the data from the questionnaires. The Research Sample Due to the relatively high costs involved in the implementation of psychometric instruments, the smaller organisations in the industry tend to avoid using this method of recruitment and team development. To establish an accurate representation of the industry, the research sample was confined to those in the Top 100 Irish construction companies (The Times, 2007). This sample was further refined down 35 using a random selection procedure. The types of organisation were sampled from a number of different disciplines within the construction industry, including: 1. Civil Engineering Contractor’s 2. Building Contractor’s 3. Building Services Contractor’s 4.Project Management organisations 5. Engineering Consultants 6. Multi-disciplinary organisations Literature Review This paper reviews the literature on psychometric testing by employers, and considers whether or not psychometric testing is a critical element of the recruitment process. It will seek to outline the fundamental benefits associated with the concept of psychometric testing and determine current trends regarding the usage of these tests in the Irish Construction Industry. According to Reber (1995) psychometric testing has been defined as â€Å"Pertaining to mental testing in any of its facets, including assessment of personality, evaluation of intelligence, determining aptitudes.† Rust (2004) classifies these tests into two broad categories: those that assess ability and those that assess personality. The way that organizations operate has changed dramatically in the last 30 years, there are now fewer levels of management than there were, and management styles tend to be less autocratic. In addition, there appears to be a move towards more knowledge based and customer focused jobs, which means that individuals have more autonomy even in the lower levels within organizations (Edenborough 2004). The effects of these changes mean that a person’s personality is seen as more important now than it was in the past. The structure of the modern workforce within the Irish Construction Industry is also rapidly changing and teamwork is becoming increasingly important, here a psychometric test can prove an invaluable tool, not only does it make sure that each team has the required skills but it also ensures that the people who make up team can work well together or that they can â€Å"gel effectively†. An article in China Staff magazine (2003) points out that this doesn’t mean that everybody in the team should be the same, rather than that you end up with a combination of personalities who can work well together. Each type of test may not suit every organisation, the design of a testing procedure must be based on what the user is wishing to achieve from the results. Burke (1995) suggests that the various types of â€Å"tests are designed for a purpose and the use of a particular test will vary according to the objectives of the assessment. In order to ascertain which tests you should use, it is necessary to gain an in-depth understanding of the characteristics necessary to perform well at the job. Recruitment and Selection Many organisations now use psychometric testing routinely in their recruitment process especially at a senior level. It forces an examination and sound understanding of the competencies inherent in the role in question and links the candidate’s competencies with those of the role. It makes for sound selection decisions and identifies development areas for the successful candidate (Corcoran 2005). Ideally, if psychometrics is used at the initial selection stage, employers are immediately aware of any areas ability or personality where an employee might not be an exact match for the job, and can make informed decisions as to whether the employee should be selected and trained, or rejected. Type of Testing Available Personality Measurement Tests These tests are used to assess how a person is likely to react to situations with people, the types of relationship they prefer to establish with others, and the motivational factors that influence a person’s actions. These are often known as tests of disposition. These tests do not generally contain questions to which there is only one correct answer. The answers given to these questions tend to reflect a tendency to interpret situations or respond to other people in particular ways. Burke (1995) outlines the typical qualities assessed by disposition tests; these include anxiety, sociability, perseverance, dominance, fear of failure and resistance to stress. A common approach to in psychometrics is to see personality as a combination of traits. A trait can be any characteristic way of behaving, thinking, feeling or operating. The measure of these ‘traits’ has always been a matter of interest for many years, it has been recognized that personality is of great impo rtance in peoples success at work, no less than the approach they take in other aspects of their life (Edenbourgh 2005). Psychologists often refer to these as the ‘big five’ personality traits. Openness appreciation for art, emotion, adventure, unusual ideas, imagination, curiosity, and variety of experience. Conscientiousness a tendency to show self-discipline, act dutifully, and aim for achievement; planned rather than spontaneous behavior. Extraversion energy, positive emotions, urgency, and the tendency to seek stimulation and the company of others. Agreeableness a tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others. Neuroticism a tendency to experience unpleasant emotions easily, such as anger, anxiety, depression, or vulnerability; sometimes called emotional instability. The argument runs that personality questionnaires do not test behavior directly but rather how the respondent chooses to describe his or her own behavior. It also argued by Cronbach (1966) that such questionnaires or self-report inventories, as they are sometimes known, indicate typical behavior. One corollary to this view by Reber(1995) is that such behavior is not very amenable to change, these type of tests have no right or wrong answers as such, however there is certainly the right and wrong personality mixes for certain jobs. Ability and Aptitude Tests ‘Aptitude’ and ‘Ability’ tests are not always precisely separated; many of the tests in these categories are used to look at behavior, often of an intellectual or cognitive nature. These are usually tests of general intelligence, which can be regarded as mental horsepower or as measure of a person’s ability to process information. Tests of ability generally assess the broader areas of what a person can do. While scores on such tests are influenced by education and training, they are not designed to assess specific areas of knowledge or skill. Examples of ability tests are: measures of verbal reasoning, numerical reasoning and mechanical reasoning. China Staff magazine (2003) defines ability testing â€Å"as a measure of a persons potential to learn the new skills needed for a new job or to cope with the demands of a training course.† Tests of aptitude on the other hand are often used to assess how well an individual is likely to perform in a training programme or in a job Edenbourgh (2005) attempts to distinguish between the two: The term ‘aptitude’ is usually reserved for those tests directed at predicting whether skill in a particular area can be acquired. ‘Ability tests’ on the other hand tend to be reserved for measures of less job-specific though often still job-related intellectual tests. Attainment Tests These are often referred to as achievement tests; these are more commonly used in relation to educational assessment rather than in occupational settings. They represent standard ways of assessing the amount of skill currently reached or attained by an individual in a particular area. Tests of attainment are often linked with ability and can be quite similar; the difference is that they test specifically what people have learnt e.g. mathematical ability or typing skills. Of course what people have learned does depend on their ability in that domain in the first place, so the scores of the two types of test are linked. Another differentiation between the two is that ability tests are retrospective in nature; they focus on what has been learnt and on what a person knows and can do now. Ability tests on the other hand are prospective: they focus on what candidates are capable of learning and achieving in the future or on their potential to learn. (China Staff Magazine 2003) Assessment Centres Psychometric testing often takes place within the context of an assessment centre. Edenbourgh (2004) points outs that organisations use a range of selection methods, including interviews, group exercises, role-playing, in-basket exercises as well as psychometric testing in order to select from a group of job applicants. Candidates are likely to encounter tests that will simulate situations that are likely to be encountered in a particular role. Assessment centre selection methods can take one or two days to complete, and because of the cost and time taken these are generally only associated with larger organisations. Assessment centers are usually used after the initial stages of the selection process and usually follow the initial job interview. They are commonly held either on employers’ premises or in a hotel and are considered by many organizations to be the fairest and most accurate method of selecting staff. This according to Psychometric Success is because â€Å"a number of different selectors will get to see the individual over a longer period of time and have the chance to see what he/she do, rather than what the individual says they can do.† Tests that are carried out within these assessment centres are often conducted by HR personnel however it is often the case that external consultants will be brought in to administer the tests. Ability to distinguish between candidates Any selection tool should be able to distinguish between candidates in two ways: 1) The tool should be accurate and precise. Any differences in results between candidates should be real and not simply random or due to errors or inaccuracies. This is known as reliability, and psychometric tools have consistently been shown to be more reliable than other common selection methods. 2) The differences in results between candidates should reflect differences in their ability to perform in the job. This is known as criterion-related validity. As shown in Diagram 3, psychometric tools do predict job performance well, provided that the right tool has been chosen, for example, via a careful analysis of the attributes required for the job. Differences in results do therefore relate to differences in ability to perform well in the job. The importance of getting this right is one reason why most instruments are only sold to those who have been trained in their use. Reliability One of the main problems associated with psychometric testing is the issue of test reliability. For instance, if there is high competition for a job  then there is strong incentive for candidates to present themselves favorably.   Similarly candidates can be tempted to respond to personality questionnaires in a way they assume the employer wants.   If we consider a construction related job, how many candidates will disagree with the statement I enjoy working in team environment? Edenbourgh (2005) identifies one way of increasing the reliability in personality questionnaires, this is to use forced-choice formats whereby candidates are presented with a combination of statements and must indicate a preference between them, this method of combining items is known as ‘ipsative’ scaling. This can be contrasted with the ‘normative scaling’, in which the candidate is instructed to choose one statement from a selection of 2 or more. For example in an ipsative test, a question might ask a candidate to choose whether they are more hard working or creative.   In this case the candidate cannot make themselves look good on both of the statements, it therefore can be said that this method can increase the reliability of certain tests and give the assessor a greater profile of the candidate’s personality. Disadvantages to Psychometric Testing In many cases, psychometric tests and questionnaires have been put together by people with little background in psychometrics and they may have very little actual utility and value for the purposes for which they are marketed. Although British Psychological Society (BPS) has a standard qualification system, it is not unusual for tests to be used by people who are not adequately trained to score them. Also just because a person attends a recognized training course it cannot be guaranteed that they will use tests and questionnaires correctly since some instruments, particularly personality questionnaires, require considerable experience and the possibility of poor interpretation of results is ever-present. Another disadvantage of psychometric testing is the use of personality questionnaires to assess an individual’s ability or skill in a particular area. For example, if a person scores highly on a test called Leadership, this does not mean that he or she will actually possess a high level of leadership skill, instead it means that the individual has the basic personality characteristics that are commonly found amongst effective leaders and, with sufficient experience and given the development of certain necessary skills, has the potential to become an effective leader. (Team Focus 2004) Cost The costs associated with these tests are often substantial. This implies that employers are unlikely to be using them merely in order to follow a management trend, but because they firmly believe that the tests are useful in recruiting job applicants with the right skills and attributes. According to a recent survey by Incomes Data Services (IDS 2000), the sums involved in testing can be quite substantial. Assuming that employers prefer to administer and interpret the tests themselves, rather than employing external consultants, then the costs will include initial training in test use, since it is necessary to be qualified in order to use tests. Further costs will include start-up kits such as manuals or computer software, and consumables such as question-and-answer booklets for test candidates. (Jenkins 2001) In order to become a trained assessor, a person needs to obtain certification from the British Psychological society (BPA), and requires the completion of two courses, Level A and Level B. The BPS Level A covers the use and interpretation of ability tests. The BPS Level B covers the use and interpretation of personality tests. In general the test companies will often provide courses leading to these qualifications. These courses usually take about five days for each of Level A and Level B. The average cost per trainee is about â‚ ¬1,900 for Level A and about â‚ ¬2,200 for Level B. It is possible to train some employees to be test administrators, rather than test users, although at least one person in the organisation must be a qualified test user if the company is to be permitted to buy tests. Test administrators can brief candidates prior to testing, and hand out and collect test papers, but cannot score tests, except under supervision, nor interpret the results. Source: IDS survey of British companies Training to become a test administrator is often cheaper than becoming a test user and a fee of perhaps â‚ ¬1000 would be charged for a test administrator course. However, it is likely that larger companies would want to have several people qualified to test user standard, as well as more employees qualified to administer the tests. According to Edenborough (2004) the major component of cost is the expense of training company staff to be able to obtain and utilize the tests properly, precise costs of psychometric tests will vary but say; in a recruitment context it is unlikely that the expenses are to be more than a couple of percent of a salary. The benefits concerned are also likely to dwarf such figure. Estimating the cost-benefit of psychometric instruments Companies can examine the effect of a recruitment process on a company’s performance by analyzing how well a candidate performs in the role and therefore how much they add to productivity, and balancing this against the investment in their recruitment. Research across a range of organisations suggests that the amount an employer can save, per employee recruited, per year, is: (Validity of the test) multiplied by (Caliber of those selected) multiplied by (Standard deviation of job performance) Minus (Cost of the selection) divided by (Proportion of applicants selected) This formula, known as the utility equation, may look complex, but it has successfully been used by HR professionals in order to justify the use of psychometric instruments and other objective assessment methods. (McHenry 1990) Calculating the cost benefits of psychometric instruments The utility equation can be used to calculate the financial benefit of using psychometric testing as part of an organisations selection process. An example of this would be a company recruiting a person for a position with a â‚ ¬60,000 salary, is possible to compare the financial benefit achieved by using a recruitment process of unstructured interviews that have a validity of 0.18, compared to using personality questionnaires and ability tests that have a combined validity of 0.60. (See table in Appendix 1) Using the utility equation and a conservative estimate of the extent to which this position will impact on company productivity, it can be predicted that using psychometric tools would increase the organization’s productivity by â‚ ¬8,520 compared to using unstructured interviews. This calculation assumes that the candidate remains in the job for one year; if they remain for three years the productivity increase can be as much as â‚ ¬25,160. Source: McHenry (1990) Current Usage within Industries According to Psychometric Success testing is now used by over 80% of the Fortune 500 companies in the USA and by over 75% of the Times Top 100 companies in the UK. Information technology companies, financial institutions, management consultancies, local authorities, the civil service, police forces, fire services and the armed forces all make extensive use of use psychometric testing.   In 2006 a survey identified that well over 2,500 personality questionnaires are available on the market, this number is increasing each year as dozens of new companies appear with their own ‘new’ products. The reasons for test use suggest that the perceived objectivity of tests, their predictive abilities, as well as their ability to filter out unsuitable candidates were important reasons for test use in both the public and private sectors, however according Jenkins (2001) equal opportunities legislation may have also encouraged employers to use tests as part of a drive to fairer selection. There are also a number of studies linking increases in test use to the spread of greater professionalism in the human resource management function, and to multi-national companies imposing standard selection procedures throughout their businesses. It has been estimated that it can cost organisations the equivalent of the whole of the first years costs (salary and any other benefits) to put right a wrong recruitment decision. Research reported in a number of test manuals shows that using good assessments professionally as part of the recruitment process dramatically increases the number of right dec isions you make. In order to establish reasons for organisations use of psychometric testing, it was necessary to look at statistics carried on British organisations, due to the limited availability of published research in Ireland. Surveys carried out by Bevan and Fryatt (1988) and by