Chapter 3

Methodology–Delphi Study

Introduction

The focus of this study was to obtain information on leading virtual teams. Leadership may also be thought of as a management or coordination function. The Delphi study or technique was used to determine if there are emerging patterns or consensus on leadership practices and information technologies used in leading virtual teams. A case study (described in Chapter 4) was used to obtain more detailed information regarding one virtual team.

Introduction to the Delphi technique

The Delphi technique was developed by the Rand Corporation in the 1950’s. It is a method for the "systematic solicitation and collation of judgments on a particular topic through a set of carefully designed sequential questionnaires interspersed with summarized information and feedback of opinions derived from earlier responses" (Delbecq, Van de Ven, & Gustafson, 1975, p. 10). This technique does not require that participants be collocated or meet face-to-face, thereby making it useful to conduct surveys with qualified people over a wide geographic area.

The Delphi technique can be used in planning situations to achieve a number of objectives.

  1. To determine or develop a range of possible program alternatives.
  2. To explore or expose underlying assumptions or information leading to different judgments.
  3. To seek out information which may generate a consensus on the part of the respondent group.
  4. To correlate informed judgments on a topic spanning a wide range of disciplines.
  5. To educate the respondent group as to the diverse and interrelated aspects of the topic (Delbecq et al., 1975, p. 11).

Delbecq, et al. (1975) described the Delphi technique by the following process:

  1. Develop a set of questionnaires. Typically three or four questionnaires mailed to the respondents are used to obtain the required data.
  2. Identify, select, and contact the respondents. Frequently a nominating process is used whereby key individuals may nominate someone with the expertise to participate in the study.
  3. Select a sample size. Thirty is frequently used as an upper bound due to limited numbers of new ideas, three or four people is probably too few, and usually between ten and twenty people is reasonable.
  4. Develop the first questionnaire, mail it to the respondents, and obtain the responses. For example, the questionnaire could contain ten to fifteen open-ended questions.
  5. Analyze the data. Responses to questions could be grouped or categorized by frequency or other criteria.
  6. Develop the second questionnaire, mail it to the respondents, and obtain the responses. This questionnaire is developed from the first questionnaire responses. It might request the respondents to review responses that have been categorized and rate them according to a scale.
  7. Analyze the data. This analysis should tally the votes for each of the responses; determine various statistics such as mean, mode, standard deviation, and so on for each response; and finally, summarize additional responses for the next round.
  8. Develop the third questionnaire, mail it to the respondents, and obtain the responses. Provide the respondents with the outcome of the second round and ask them if they would like to reconsider their own second round responses.
  9. Analyze the data. Determine if there is a consensus.
  10. Complete the final report (pp. 87-107).

Application of the Delphi Technique

The purpose of the Delphi technique in this study was the third item mentioned above, namely, to seek out information which may generate a consensus on the part of the respondent group regarding leadership practices and information technologies used in virtual teams (Delbecq et al., 1975).

A three phase process was used–that is, three rounds of questionnaires were used. Respondents or participants were identified by a nominating process as having some expertise in virtual teams (Delbecq et al., 1975). These participants were then contacted by electronic mail to determine if they would be interested in participating.

The first round of the Delphi method asked the participants to respond to thirteen specific questions on virtual teams. The second round used questions developed from responses to the first questionnaire. The participants were asked to rate each statement on a 1 to 5 scale (strongly disagree to strongly agree) and to optionally comment on each statement. The third round used the same statements as the second round and asked the participants if they would like to modify their answers based on the responses of the other participants (Delbecq et al., 1975).

Selection of Participants

Participants in this study were selected by nominations of people (or self-nominations) with some expertise on virtual teams. Nominators were identified from organizations such as AT&T, the Boeing Company, World Future Society, and Universities. Each nominator was asked to provide a list of names of people with expertise in virtual teams who may be willing to be participants. Identified participants were contacted to see if they would be willing to participate. Once a target list of thirteen names was obtained, round 1 began. The number of participants in a Delphi study varies. As Delbecq et al. (1975) indicated, "the size of the respondent panel is variable. With a homogenous group of people, ten to fifteen participants might be enough" (p. 89).

Data Collection Process

Once the participants were identified, three rounds or phases were used to collect data in response to a series of questions.

Delbecq, et al. (1975) wrote,

Delphi is essentially a series of questionnaires. The first questionnaire asks individual to respond to a broad question. (Delphi questions might focus upon problems, objectives, solutions, or forecasts.) Each subsequent questionnaire is built upon responses to the preceding questionnaire. The process stops when consensus has been approached among participants (p. 83).

This dissertation stopped at three questionnaires or rounds which seems fairly typical of many studies. Consensus or trends towards consensus were documented at the conclusion of round 3.

Round 1. Questions used in this round were as follows. The overall question for this Delphi Process is: How do you lead virtual teams–teams of people you don’t see on a day-to-day basis or may have never met–in terms of leadership practices and information technologies?

The following three research questions relate to the overall question, but are more specific and are the focus of the study.

This last question includes leadership practices such as bringing the team together for face-to-face interaction if necessary, communications, coaching, individual recognition, and team celebrations.

Examples of virtual teams include teams of people who work at home or telecommute and a project composed of team members who live in different cities.

Examples of leadership, management, or coordination practices and behaviors include planning, guiding, encouraging, coaching, mentoring, and rewarding team members; setting examples; and communications.

The following thirteen questions were used for the first questionnaire. This researcher was looking for data and examples as responses to these questions, not opinions. This was stated at the beginning of all data collection processes.

  1. What was the mission or purpose of the team and how was this mission or purpose created or shaped?
  2. What has happened so far on the project? When did it start? What phase are your in? What’s next for the project?
  3. What is the structure or organization of the team?
  4. What was your role on this team (team leader, committee chair, team member)?
  5. Who did you primarily interact with (all team members, project leaders, etc.) and how? Did you have any previous interaction with these people before this project? How?
  6. Where were the team members physically located? Why were they not all collocated?
  7. How were the team members selected and why?
  8. What information and communication technologies (telephone, e-mail, conferencing, project management software, etc.) did the team use?
  9. Can you provide examples of when team members met face-to-face? Did sub-teams meet more frequently? Was face-to-face contact important? Why?
  10. Can you provide examples of where individual or team accomplishments were celebrated? If not, would there have been any if the team had been collocated?
  11. Can you provide any examples of team members being challenged, encouraged, rewarded, participating in key decisions, coached, or listened to? How was their morale?
  12. Have you learned anything interesting about being on a virtual team? What would you have done differently if the team had been collocated?
  13. What were some of the major problems or challenges attributable to working in a virtual environment (e.g., trust, collaboration, leadership, communicating, decision-making, team or individual focus)?

Round 2. Responses to the first round were categorized or grouped by frequency or similarity of response in order to reduce the number to a manageable level but yet keeping the essential meaning of the responses. Participants were asked to rate the categorized responses from round 1 on a scale of 1 to 5, with 1 being "strongly disagree" and 5 being "strongly agree."

Round 3. The purpose of this round was to determine if there was a consensus. Responses from the round 2 were analyzed by determining the mode for each response. Participants were asked to review their response and the modal response, respond again using the same rating scale, and add any comments regarding the responses.

Final Analysis of Data. Responses from round 3 were then analyzed to determine if there was a consensus by looking at the responses and their modes.

Survey Instruments

The survey instrument containing the questions listed above is also provided in the Appendix and was the questionnaire in round 1 of the Delphi technique. This survey instrument was developed by the researcher in discussions with committee members and other researchers. Questionnaires for Delphi rounds 2 and 3 were developed from the responses to Delphi round 1.

Validity. These instruments (questionnaires 1, 2, and 3) were reviewed by other researchers to establish the face validity (do the questionnaire items appear to measure what the instrument purports to measure?). The purpose of a review is to improve the questions, format, and the scales (Creswell, 1994).

Reliability. In addition to validity, these instruments were tested for reliability, that is, limitations in replicating the study. Creswell (1994) wrote, "statements about the researcher’s positions–the central assumptions, the selection of informants, the biases and values of the researcher–enhance the study’s chances of being replicated in another setting" (p. 159). These assumptions, selection, process, and so forth are documented. Other researchers were asked to comment on this documentation and modified the instruments and supporting documentation where it was necessary.

Confidentiality

Responses to the Delphi questionnaires were treated with complete confidentiality.