Qualitative data collection

Chapter 6: Qualitative data collection

Book Name: Research Methods in Applied Linguistics

Writer: Professor Zoltan Dornyei

Professor: Dr.Zoghi, M     

Summarized :Saeed Mojarradi Ph.D. Candidate

 Monday, November 19, 2018

Quantitative inquiry can be divided into two distinct phases:

1-data collection

2-and data analysis

They typically follow each other in a linear manner. In accordance with the flexible and emergent nature of the qualitative research process, qualitative data collection and analysis are often circular and frequently overlap.

Furthermore, it is sometimes problematic to decide whether a certain qualitative method refers primarily to data collection, data analysis or a unique, combined design—grounded theory and the case study are good examples of thig ambiguity.

It is fair to say that research methodology texts in general have been inconsistent in how they have broken down the broad umbrella term ‘qualitative research’ into specific data collection and data analytical methods, techniques, and designs.

 6.1 Qualitative data

Qualitative data can come from many sources, it is usually transformed into textual form resulting in hundreds (if not thousands) of pages of transcripts and field.

I believe it is important to highlight two main characteristics of a typical qualitative dataset: its tendency to become increasingly long and its rather unfocused and heterogeneous nature.

To put it broadly, qualitative data tends to be bulky and messy.

In qualitative research there are no explicit restrictions on what can be considered ‘data’ and as Richards (zoos) concludes, the researcher in a qualitative project often starts out by treating everything around a topic as potential data.

Qualitative data expands quickly, and novice researchers often find that the real challenge is not to generate enough data but rather to generate useful data. In fact, a serious problem in qualitative research can be too much data, which is augmented by the fact that ‘qualitative data are messy records’ (p. 34), usually consisting of a mixture of field notes, transcripts of various recordings as well as documents of a diverse nature and length. This does not mean that qualitative data cannot produce valuable results, but it does mean that processing such sizeable and heterogeneous datasets can involve a lot of work.

Accordingly, the design of most qualitative data collection is fluid and open-ended, and researchers are not required to plan all elements of the project at the outset. While this is in line with the discovery-oriented character of qualitative inquiry, Richards (Zoos) also stresses that possibly the most common cause of problems in qualitative data collection is the lack of any plans for data reduction. As she argues, ‘Novice researchers can very easily frame the scale of a project in terms of how widely they might need to spread their net, rather than how wide, realistically, the spread can be’ (p. 2o).

6.2 Sampling in qualitative research

S straightforward: we need a sizeable sample to be able to iron out idiosyncratic individual differences. Qualitative research, on the other hand, focuses on describing, understanding, and clarifying a human experience and therefore qualitative studies are directed at describing the aspects that make up an idiosyncratic experience rather than determining the most likely, or mean experience, within a group (Polkinghorne zoos).

Accordingly, at least in theory, qualitative inquiry is not concerned with how representative the respondent sample is or how the experience is distributed in the population. Instead, the main goal of sampling is to find individuals who can provide rich and varied insights into the phenomenon under investigation so as to maximize what we can learn. This goal is best achieved by means of some sort of ‘purposeful’ or ‘purposive’ sampling.

Purposive and theoretical sampling

Because there is always a limit to how many respondents we can contact or how many sites we can visit, we have to make some principled decisions on how to select our respondents, at which point to add additional participants to the sample and when to stop gathering more data. Even when our sample consists of one case only, we need to select the aspects of that case that we will focus on. (See ‘within-case sampling’ in Section 6.2.4.)

Therefore, a qualitative study must have a sampling plan describing the sampling parameters (participants, settings, events, processes), and this plan should line up with the purposes of the study. Punch (zoos) suggests that if it is not clear to us which cases, aspects, or incidents to study, it is usually worth devoting more work to developing the initial research questions.

6.2.2 Iteration, saturation, and sample size

Researchers are in agreement that the participant selection process should remain open in a qualitative study as long as possible so that after initial accounts are gathered and analyzed, additional participants can be added who can fill gaps in the initial description or can expand or even challenge it. This cyclical process of moving back and forth between data collection and analysis is often referred to as ‘iteration’. Although iteration is a key process in qualitative sampling, it cannot go on forever.

 

6.2.3 Specific sampling strategies

Purposive sampling can follow a number of different strategies depending on the research topic and setting. When designing the sampling plan, we also need to take into account feasibility issues (in terms of time, money, respondent availability) and— what is often ignored—saturation considerations. The more cohesive/homogeneous the sample, the faster the saturation, but at the same time, the narrower the scope of the project. The following list of the most common qualitative sampling strategies offers a sense of how wide we need to throw the net to be able to reach saturation in different research areas.

Relatively quick saturation can be achieved by the following three inter-related sampling strategies because they all aim at selecting participants who are similar in some respect:

• Homogeneous sampling the researcher selects participants from a particular subgroup who share some important experience relevant to our study (for example, they have participated in a study-abroad programme). In this way, this strategy allows us to conduct an in-depth analysis to identify common patterns in a group with similar characteristics.
• Typical sampling

The researcher selects participants whose experience is typical with regard to the research focus (for example, they all study a foreign language as a school subject at an intermediate level with moder-ate success). This strategy assumes that we have a profile of the targeted attributes possessed by an ‘average’ learner. Although we cannot generalize from the results because we cannot claim that everybody will have the same experience, we can list the typical or normal features of the experience,

• Criterion sampling

The researcher selects participants who meet some specific predetermined criteria (for example, company executives who failed an important language exam). We can also gain valuable insight into an issue if, rather than selecting typical participants, we intentionally look at the whole range of possible responses, including very special cases

• Maximum variation sampling

The researcher selects cases with markedly different forms of experience (for example, 1…2. learners from all develop-mental levels). This process will allow us to explore the variation within the respondents and it will also underscore any commonalities that we find: if a pattern holds across the sampled diversity, we can assume that it is reasonably stable.

• Extreme or deviant case sampling

Following the same logic as maximum variation sampling the researcher selects the most extreme cases (for example. the most motivated and demotivated learners). On the one hand, this allows us to find the limits of the experience; on the other hand, if even such cases share common elements, they are likely to be real core components of the experience.

• Critical case sampling

The researcher deliberately targets cases which offer a dramatic or full representation of the phenomenon, either by their intensity or by their uniqueness (for example, in a language attrition study examining people who have completely forgotten an 1.i they used to speak). Their case may be taken as the most salient or comprehensive manifestation of the phenomenon under scrutiny; in such situations researchers are not only interested in what they find but also in what they do not, because something that does not occur in such salient cases is unlikely to happen elsewhere.

As it often happens, the most practical and feasible (and of course most common) sampling strategies are the least sound from a theoretical point of view. Three such ‘less principled’ strategies are particularly well known:

• Snowball or chain sampling

The starting point of this strategy is a principled list of key respondents, who are then asked to recruit further participants who are similar to them in some respect central to the investigation. This chain reaction can reach far, which is ideal in situations where the experience in question is rare.

• Opportunistic sampling

This is an unplanned and potentially haphazard procedure in the sense that it is followed on the spur of the moment: while working in the field, the researcher sometimes comes across respondents who are ‘too good to miss’ and a decision to include them is made on the spot. The problem is that they are not always exactly what is needed, yet their selection is very much in line with the emergent nature of the qualitative inquiry.

• Convenience sampling

This is the least desirable but the most common sampling strategy, at least at the postgraduate research level. It is not purposive but largely practical: the researcher uses those who are available. Of course, in an ideal world nobody would employ a convenience sample, but research (and particularly postgraduate research) all too often happens in less-than-ideal circumstances, under considerable time or financial constraints. One redeeming feature of this sampling strategy is that it usually results in willing participants, which is a prerequisite to having a rich dataset. On the other hand, saturation may not happen at all. Thus, this strategy may save time, money, and effort, but at the expense of credibility (Miles and Huberman 1994).

6.2.4 Within-case sampling

Although qualitative researchers are becoming aware of the importance of principled, purposive sampling, the process is usually applied only to select-ing respondents, that is, to ‘between-case sampling’. In some qualitative methods (for example, ethnography or case study) we need to extend our sampling plan to also include ‘within-case sampling’ (i.e. selecting data from the potentially available data pool concerning a participant), because we have to make regular decisions about when and how to collect data from a particular respondent, what aspects of the case to direct our attention to and which activities, locations, or events to focus on. In these decisions we need to be just as purposive with our choices as with the selection of informants.

6.3 Ethnography

A good starting point in our exploration of the specific qualitative methods of discovery is ethnography, because this approach embodies in many ways the essence of the qualitative inquiry. In fact, ‘ethnography’ has frequently been used as a synonym for ‘qualitative research’ and Hammersley and Atkinson (1995: 1), for example, started their influential monograph on ethnography by stating, For the purposes of this book we shall interpret the term “ethnography” in a liberal way, not worrying much about what does and does not count as examples of it’.

The main goal of most ethnographic research is to provide a ‘thick description’ of the target culture, that is, a narrative that describes richly and in great detail the daily life of the community as well as the cultural meanings and beliefs the participants attach to their activities, events, and behaviors.

6.3.1 Main features of an ethnographic study

According to Harklau (2005), a hallmark of classic ethnographic research is that it involves firsthand ‘participant observation’ in a natural setting, and most studies that frame themselves as ‘ethnographic’ include some degree of this method. However, we saw that ethnographers also utilize several other data collection techniques and, in fact, Harklau points out that multiple data sources are usually considered desirable. So, if the method of data collection is not a key determinant of the ethnographic approach, what are the features that define it? The following three points are emphasized most frequently in the literature; they will need little elaboration here because they have been described earlier when discussing the qualitative approach in general. (See Section 2.3.2.)

• Focusing on participant meaning

In ethnographic studies the participants’ subjective interpretation of their own behaviors and customs is seen as crucial to understanding the specific culture. Therefore, a central aspect of ethnography is to find ways of looking at events through the eyes of an insider.

• Prolonged engagement in the natural setting

The subtleties of the participants’ meaning (and most often, multiple meanings) cannot be uncovered unless the researcher immerses him/herself in the culture and spends an extended period living there, observing the participants and collecting data. Therefore, a minimum stay of 6–I z months is usually recommended to achieve the necessary prolonged engagement.

• Emergent nature

Because the ethnographer is entering a new culture, the exact focus of the research will evolve contextually and ’emerge’ in situ only after some fieldwork has been done.

6.3.2 Main phases of an ethnographic study

An ethnographic study involves a complex process of ‘getting in’ and ‘coming out’ which can be described as a sequence of four relatively distinct phases (Morse and Richards 2002; see also Richards 2003):

• The first phase involves entering what is for the researcher a strange environment. The ethnographer needs to negotiate the entry with the gate-keepers and then find a way, reason, or adequate role for fitting in.
• The second phase is easier in many ways than the first because by now the ice has been broken and the researcher has become familiar with the participants and the routines in the setting. Nonparticipant observation is now in full swing and the critical task of finding key informants begins along with conducting initial interviews with them. At this stage the ethnographer also needs to start analyzing the preliminary data to develop some initial ideas and concepts.
• The third phase is the most productive research phase. Acculturation has been completed and the researcher has been accepted and now feels ‘at home’ in the setting. This allows him/her to employ a variety of techniques to collect increasingly focused data, which is used to verify initial hypotheses and ideas and to develop broader theoretical concepts. Data analysis at this stage is characterized by ‘progressive focusing’, involving sifting, sorting, and reviewing the data (Cohen et al. z000).
• The final phase is the necessary withdrawal. This may be an emotionally taxing closure phase evoking a feeling of loss, and the ethnographer needs to make sure that he/she disengages from the field in a way that brings as little disruption to the group or situation as possible. The focus is on data analysis and additional data is collected only to fill gaps, to resolve ambiguities, and to validate previous findings.

6.3.3 Strengths and weaknesses of ethnographic research

The ethnographic approach is particularly useful for exploring uncharted territories and understanding social processes from the participants’ perspective. It is an excellent way of ‘crossing cultures’ and gaining insight into the life of organizations, institutions and communities.

In short, ethnography is ideal for generating initial hypotheses about something totally unknown. Hornberger (1994: 688) also highlights the capacity of ethnography to take a holistic view and focus on the whole picture that ‘leaves nothing unaccounted for and that reveals the interrelatedness of all the component parts’. As she goes on to argue,

The value here is that the approach allows, indeed, it is the very essence of the approach to ensure, comparison and contrast between what people say and what people do in a given context and across contexts in order to arrive at a fuller representation of what is going on. It is not enough for ethnographers to ask teachers about their communicative approach to ESL teaching; they must also observe it in action.

6.4 Interviews

The second qualitative method of inquiry to be described in this chapter is conducting interviews. Interviewing is a frequent part of the social life surrounding most of us: we can hear interviews on the radio, watch people being interviewed on television, and we ourselves often participate in interviews of various types either as interviewers or interviewees.

As Miller and Crabtree (1999) point out, the interview genre with its turn-taking conventions and expectations for participant roles, etiquettes, and even linguistic phrases is usually shared cultural knowledge. It is exactly because interviewing is a known communication routine that the method works so well as a versatile research instrument—in fact, although there is a range of qualitative research techniques available for researchers, the interview is the most often used method in qualitative inquiries.

6.4.1 Main types of interviews

One-to-one interviews can be divided into different types according to the degree of structure in the process and whether there are single or multiple interview sessions. Let us start with the latter, less often mentioned aspect. Single or multiple sessions

The typical qualitative interview is a one-off event lasting about 3o-6o minutes. However, as Polkinghorne (zoos) argues, one-shot interviews are rarely able to produce the full and rich descriptions necessary for worth-while findings. Drawing on Seidman’s work he recommends that researchers administer a sequence of three interviews with the same participant to obtain sufficient depth and breadth.

The first interview breaks the ice and develops rapport, while also providing a quick sweep of the areas to be investigated later. The interval between the first and the second interviews allows the inter-viewer to prepare a more made-to-measure interview guide for the second session and it also offers the interviewee the chance to think more deeply about the experience.

As a result, the second interview is more focused than the first. Finally, having analyzed the transcripts of the first two sessions, in the third interview-the researcher can ask any ‘mop-up’ or follow-up questions to fill in and to clarify the account. It must be noted that the multiple session format outlined here is not the same as a longitudinal interview study (see Section 4.4) because the purpose of the three sessions is not to document temporal changes but to arrive at a full account. In a longitudinal interview study the multiple sessions would need to be organized differently, with the first one or two interviews creating the baseline knowledge and the subsequent, regularly occurring interviews focusing on how and why the particular phenomenon under study changes.

Structured interviews

The second main categorizing principle of interviews is the degree of structure in them, with one extreme being the ‘structured interview’. In this format, the researcher follows a pre-prepared, elaborate ‘interview schedule/guide’, which contains a list of questions to be covered closely with every interviewee, and the elicited information shares many of the advantages (for example, comparability across participants) and disadvantages (for example, limited richness) of questionnaire data. Such tightly controlled interviews ensure that the interviewee focuses on the target topic area and that the interview covers a well-defined domain, which makes the answers comparable across different respondents. The other side of the coin is, however, that in a structured interview there is generally little room for variation or spontaneity in the responses because the interviewer is to record the responses according to a coding scheme.

Unstructured interviews

The other extreme, the ‘unstructured interview’ (sometimes also referred to as the ‘ethnographic interview’), allows maximum flexibility to follow the interviewee in unpredictable directions, with only minimal interference from the research agenda. The intention is to create a relaxed atmosphere in which the respondent may reveal more than he/she would in formal contexts, with the interviewer assuming a listening role.

No detailed interview guide is prepared in advance, although the researcher usually thinks of a few (i-6) opening questions (sometimes called ‘grand tour’ questions) to elicit the interviewee’s story. During the interview, the researcher may ask an occasional question for clarification and may give reinforcement feedback as any good communication partner would to keep the interview moving, but interruptions are kept to a minimum. It is easy to see that for an unstructured interview to be successful it is indispensable that the interviewer establishes very good rapport with the interviewee. This kind of interview is most appropriate when a study focuses on the deep meaning of particular phenomena or when some personal historical account of how a particular phenomenon has developed is required. In-depth interviews can also be used where exploratory work is required before a more focused (for example, quantitative) study can be carried out.

Semi-structured interviews

In applied linguistic research most interviews conducted belong to the `semi-structured interview’ type, which offers a compromise between the two extremes: Although there is a set of pre-prepared guiding questions and prompts, the format is open-ended and the interviewee is encouraged to elaborate on the issues raised in an exploratory manner. In other words, the interviewer provides guidance and direction (hence the `-structured’ part in the name), but is also keen to follow up interesting developments and to let the interviewee elaborate on certain issues (hence the ‘semi-‘ part).

Because of the great popularity of this interview format, most of the following recommendations on question wording and interview conduct will be geared at semi-structured interviews in particular.

The semi-structured interview is suitable for cases when the researcher has a good enough overview of the phenomenon or domain in question and is able to develop broad questions about the topic in advance but does not want to use ready-made response categories that would limit the depth and breadth of the respondent’s story. This format therefore needs an ‘interview guide’ which has to be made and piloted in advance. Usually, the interviewer will ask the same questions of all of the participants, although not necessarily in the same order or wording, and would supplement the main questions with various probes—see below.

Preparing for the interview and designing the interview guide

The complete interview process involves a series of carefully designed steps, with the preparation starting well before the first interview session. After the initial sampling plan has been finalized and ethical issues such as informed

A few trial runs can ensure that the questions elicit sufficiently rich data and do not dominate the flow of the conversation.

The main function of the interview guide (or ‘interview schedule/protocol’) is to help the interviewer in a number of areas:

(a) by ensuring that the domain is properly covered and nothing important is left out by accident; (b) by suggesting appropriate question wordings; (c) by offering a list of useful probe questions to be used if needed; (d) by offering a template for the opening statement; and (e) by listing some comments to bear in mind.

It might be advisable to combine this guide with an ‘interview log’ and thus leave space in it for recording the details of the interview (for example, participant, setting, length) as well as for the interviewer’s comments and notes.

Question types and wording issues

There are a variety of questions we can include in the interview guide, but we need to bear in mind that these only provide the framework and the real meaning is so often uncovered through exploratory and unstructured responses that deviate from the interview schedule.

• The first few questions

These are particularly important in an interview, not so much from the content point of view but rather because they set the tone and create initial rapport. If the interviewees feel that they can do themselves justice when answering these initial questions, this will make them feel competent, help them to relax and consequently encourage them to open up. This is why researchers often start with easy personal or factual questions (for example, about the respondent’s family or job). The quality of the subsequent responses will to a large extent depend on the climate of trust that we create in this initial ice-breaking period.

• Content questions

Patton (2002) points out that on any given topic, it is possible to ask any of six main types of question focusing on: (a) experiences and behaviors, (b) opinions and values, (c) feelings, (d) knowledge, (e) sensory information (i.e. what someone has seen, heard, tasted, smelled, etc., and even what the interviewer would have seen or heard if he/she had been in a particular place), and (f) background or demographic information.

These six categories concern different aspects of the participant’s overall view/experience of the phenomenon and therefore we can get a rounded picture by including in our interview guide questions that tap into each dimension.

• Probes

The emergent nature of qualitative interview data can be enhanced by applying various probes that use what the interviewee has said as a starting point to go further and to increase the richness and depth of the responses. Probes may include detail-oriented and clarification questions but a technique often used in person-centered psychotherapy is to simply take a salient content word used by the respondent and ask to elaborate (`You have used the word “freedom” mice—what exactly does it mean to you/do you mean by that …?’). Patton (low.) also mentions an interesting probe, the ‘contrast probe’ which asks about how a particular experience/ feeling/action/term compares to some other, similar concept.

• The final closing question

This permits the interviewee to have the final say. Several scholars have noted in the literature the richness of the data that simple closing questions such as the following ones can yield: ‘Is there anything else you would like to add?’ or ‘What should I have asked you that I didn’t think to ask?’

With regard to the wording of the questions asked in a qualitative interview, the literature contains a great deal of advice. Some of this advice is self-evident (for example, ‘don’t use words that the interviewee does not understand’), but some other suggestions are truly practical and useful—Patton (20oz), for example, offers particularly detailed guidelines. Similar to the wording of written questionnaire items (see Section 5.2.3), there are some rules of thumb about how to formulate our questions.

Two particularly important rules are that we should avoid (a) leading questions (`It was frustrating, wasn’t it …?’) and (b) loaded or ambiguous words and jargon. In general, short and relatively simple questions that contain only one idea work best. Using words that make sense to the interviewee and reflect his/her worldview help to connect to the respondent and improve the quality of the interview data.

6.4.3 Conducting the interview

While it is true that practice usually makes us more relaxed and polished interviewers, I have found that simply being familiar with certain key issues regarding interview conduct is already helpful and there are several techniques that we can implement right from the start with good effect. So let us examine key components of the interviewing process.

Recording the interview

There is general agreement in the literature that if we want to use the content of a semi-structured or unstructured interview as research data, we need to record it—taking notes is simply not enough as we are unlikely to be able to catch all the details of the nuances of personal meaning; furthermore, note-taking also disrupts the interviewing process. However, we must be aware of the fact that many people do not like to be recorded, and therefore we must discuss this aspect with the interviewee in advance. Recording has a technical and a theoretical, aspect.

Conducting the interview

A good qualitative interview has two key features: (a) it flows naturally, with the various parts connecting seamlessly. We must remember that we are there primarily to listen (and not speak!) — This is, in fact, the first of Robson’s (2oo2) general recommendations on interviewing (presented in Table 6.0. We must let the interviewee dictate the pace without being rushed or interrupted. Even if there is a silence, we need to be patient and resist stepping in too quickly with a new question; and (b) it is rich in detail.

Once we have successfully established the flow of the interview, we need to sustain it in an unobtrusive way. There are a number of useful techniques to keep the interview on track:

• Carry-on feedback Backchannel ling signals (for example, nods, `uh-huh’ noises, one-word utterances like ‘yeah’) are offered by any sympathetic listener as part of normal communication and the interviewer must appear to be a particularly sympathetic listener. Miller and Crabtree (1999) also highlight the important role of small gestures such as the ‘attentive lean’ and the ‘eyebrow flash’, and we can add here the ‘sympathetic smile’.
• Reinforcement feedback just as in a normal conversation, we need to provide from time to time reinforcement, indicating that we are pleased with the way the interview is going and confirming that the interviewee’s efforts are worthwhile. Everybody likes to be praised!
• Negative reinforcement

What shall we do when the interview is not going as well as we think it could? After all, people often like to talk and go off at a tangent in a way which is not ‘spontaneously emerging personal meaning’ but merely waffling. If we give regular short reinforcement feedback (for example, nodding, uh-huhs) then simply withholding these and replacing them with an interjected question or a shift in our position might be enough of a nudge to get back on track. Alternatively, as Patton (2002) points out, some clever phrases might be used for inoffensive interruption and refocusing (for example, ‘Let me stop you here for a moment and go back to what you said earlier to make sure that I understood you well’).

• Encouraging elaboration In cases when the interviewee is not very forth-coming about a certain topic, we can use various probes (discussed in Section 6.4.2), including ‘silent probes’ that involve remaining quiet to indicate that we are waiting for more. More direct calls for elaboration include the ‘echo prompt’ (repeating the last word spoken by the respondent), low-inference paraphrasing or reflective summary and clarification questions.

Ending the interview

We can signal that the interview is nearing the end by using pre-closing moves such as summarizing or recapping the main points of the discussion. These also have a significance content-wise because they allow the interviewee to correct anything that we may have misunderstood and to make any additional points. In fact, as discussed in connection to the interview guide (Section 6.4.1), we may wish to give the respondent an explicit opportunity to make any comments that might be relevant/important but which have not been covered in the rest of the interview CI have no further questions.

Do you have anything more you want to bring up, or ask about, before we finish the interview?’ Kvale 1996: 128). Whether we do this or not, we need to be careful not to end le interview on a difficult topic—just like in oral language exams, we need to include a final winding down phase by steering the interview towards positive experiences.

Strengths and weaknesses of interviews

The interview is a natural and socially acceptable way of collecting information that most people feel comfortable with and which can be used in a variety of situations and focusing on diverse topics to yield in-depth data.

The inter-viewer’s presence allows for flexible approaches, probing into any emerging new issue, while the interview guide helps to maintain a systematic coverage of the domain. Because of the high social profile of interviewing most of us will have several good interviewer models in our minds and therefore even beginning researchers are likely to be able to obtain rich data in their very first interviews.

The main weakness of the interview is that it is time-consuming to set up and conduct, and that it requires good communication skills on the part of the interviewer, which not all of us have naturally.

 

6.5 Focus group interviews

Focus group interviews are sometimes treated as a subtype of interviewing but because both the format and the interviewer’s role are considerably different in the interviewing process, I find it more appropriate to keep the two methods separate. Focus group interviews— as the name suggests—involve a group format whereby an interviewer records the responses of a small group (usually 6-12. members).

This is obviously an economical way to gather a relatively large amount of qualitative data and therefore focus groups are used for a variety of purposes in many different fields. The name originally comes from market research but now the terms ‘focus group interview’ and `group interview’ are used interchangeably. The method has become familiar to the larger public when political parties have started to use it to gauge voter reaction to some planned policies and group interviews also feature frequently on television.

The focus group format is based on the collective experience of group brain-storming, that is, participants thinking together, inspiring and challenging each other, and reacting to the emerging issues and points.

6.5.1 Characteristics of the focus groups

The size of a focus group ranges between 6–io (sometimes i 2.) people. Fewer than six people would limit the potential of the ‘collective wisdom’ whereas too large a size makes it difficult for everyone to participate. When designing a focus group study, the two key technical questions to decide are (a) whether to have homogeneous or heterogeneous people in a group; and (b) how many groups to have.

• Composition

Although, in line with the principles of maximum variation sampling (see Section 6.2.3), heterogeneous samples consisting of dissimilar people could in theory be useful in providing varied and rich data that covers all angles, it has been found that the dynamics of the focus group works better with homogeneous samples.

6.5.2 Conducting the focus group interview

In focus group interviews the interviewer is usually referred to as the ‘moderator’, and this special name reflects the fact that the researcher’s role differs from that in one-to-one interviews. Although they still need to ask questions, during the session they need to function more as facilitators of the discussion than as interviewers in the traditional sense. Because the dynamic of the focus group is one of the unique features of this method, the researcher’s role inevitably involves some group leadership functions, including making sure that nobody dominates the floor and that even the shyer participants have a chance to express their views.

 

 

6.5.3 Strengths and weaknesses of focus group interviewing

Focus groups are highly versatile: as mentioned earlier, they can be used in a wide range of areas, from market research to political opinion probing, and in educational contexts they are also often used for programme evaluation to assess the effectiveness of a particular course to understand what was or was not working and why. People do not usually mind participating in focus groups—in fact, they tend to find the sessions enjoyable and stimulating—and the interviews typically yield rich data. These merits, coupled with the quick turnaround, make the method popular.

Because of the flexible and information-rich nature of the method, focus groups are-often used in mixed methods research. Although they can be used as a stand-alone method of inquiry, this is not that common except for some areas where focus groups are well established (most notably market research) because researchers often feel that they have more control over the content and can elicit deeper and more ‘unedited’ personal meaning in one-to-one interviews. In applied linguistic research they have been widely used for generating ideas to inform the development of questionnaires and subsequent deep interviews.

The downside of focus group interviews is that they need quite a bit of preparation to set up, and to do them well the moderator needs to be able to carry out several functions simultaneously. There is also more need to improvise because the number of questions to be asked following the interview guide is relatively low and a lot of the content emerges from well-managed group discussion facilitated by probe questions. With regard to the content of the elicited material, Smithson (z000) mentions two possible limitations: the tendency for certain types of socially acceptable opinion to emerge and for certain types of participant to dominate the research process. Finally, an important technical point is that transcribing such an interview can be rather difficult because of the number of people (i.e. voices) involved. For this reason, researchers sometimes also prepare a video recording of the interviews —ancillary to audiotaping — so that they can identify who is speaking at any one time.

6.6 Introspective methods

Ever since the beginnings of psychological research at the end of the nineteenth century, psychologists have been trying to find ways of obtaining information about unobservable mental processes such as thoughts, feelings, and motives. One obvious source of information about these processes is the individual him/herself, and the various ways of eliciting self-reflections from respondents is usually referred to under the umbrella term ‘introspective method’. It subsumes several different approaches that all aim at helping the respondents to vocalize what is/was going through their minds when making a judgement, solving a problem or performing a task.

The data generated by this methodology is called a ‘verbal report’ or ‘verbal protocol’ and therefore introspective methods are also referred to as ‘verbal reporting’ or ‘protocol analysis’. We must note here that there is some inconsistency in the terminology in the area, because sometimes ‘introspection’ is used as a higher-order term that includes any form of self-report such as diaries, interviews, and even surveys, and when the term is applied so broadly, verbal reporting is seen only as a subset.

6.6.1 Think-aloud technique

One of the main proponents of introspective methods in psychology, Ericsson (2002), explains that the closest connection between thinking processes and verbal reports are found when participants are instructed to verbalize their ongoing thoughts while focusing on a task. This technique has come to be known as ‘think-aloud’ and it involves the concurrent vocalization of one’s `inner speech’ without offering any analysis or explanation. Thus, respondents are asked to verbalize only the thoughts that enter their attention while still in the respondent’s short-term memory. In this way, the procedure does not alter the sequence of thoughts mediating the completion of a task and, according to Ericsson, can therefore be accepted as valid data on thinking. The resulting verbal protocol is recorded and then analyzed.

6.6.2 Retrospective interview or stimulated recall

In ‘retrospection’, the respondents verbalize their thoughts after they have performed a task or mental operation. In such cases, the relevant information needs to be retrieved from long-term memory and thus the validity of retrospective protocols depends on the time interval between the occurrence of a thought and its verbal report. Ericsson (zoos.) concludes that for tasks with relatively short response latencies (less than 5-10 seconds), subjects are able to produce accurate recollections, but for cognitive processes of longer duration, the difficulty of accurate recall of prior thoughts increases.

How can we improve the quality of the retrospective data? Based on Gass and Mackey (z000), Mackey and Gass (zoos), the studies in Fxrch and Kasper (1987) as well as my own experience, the following recommendations can be made:

• We need to try and keep the interval between the task and the retrospective interview as short as possible. Often (for example, in strategy research) the researcher first needs to transcribe and analyses the respondent’s speech to make the retrospective session really meaningful because only such a thorough engagement with the text can reveal certain subtle issues that require clarification; however, even in such cases the time lapse should not exceed two days (and should preferably be less than 24 hours).
• The richer the contextual information and stimulus about the target event, the better. Therefore, listening to a recording is better than merely looking at the transcript, and watching a video recording is superior to listening to an audio recording. • We should only encourage the recall of directly retrievable information (for example, ‘what were you thinking of?’) rather than explanations or interpretations.
• If possible, the respondents should not be informed of the subsequent retrospective interview (or the exact details of what it will involve) before the completion of the task so that the foreknowledge does not affect their performance.
• Gass and Mackey (z000) point out that stimulated recall does not require extensive participant training because simple instructions and a direct model are often enough. We need to be careful not to cue the respondents into any aspects of the study that are unnecessary and might affect their responses.
• During the retrospective interview we should try and involve the respond-ents as much as possible in volunteering data (for example, ask them to stop the cassette recorder when they remember something that might be useful) and even when we highlight parts to comment on (for example, by stopping the playing of the tape at critical points), we should avoid leading questions or any other researcher interference.
• If possible, the retrospective interview should be in the respondent’s Li (or in the language the respondent chooses).
• The whole retrospective procedure needs to be piloted. Gass and Mackey (woo) offer complete sample protocols for stimulated recall procedures in oral interaction settings.

6.6.3 Analyzing introspective data

Just because introspective reports come ‘straight from the horse’s mouth’, they cannot be taken as the ultimate revelations about thought processes. Rather, as Kasper (1998) emphasizes, they represent information about what is going through someone’s mind when addressing a task or issue and the underlying cognitive processes need to be inferred from these verbal reports just as with other types of data. In other words, verbal reports must be seen only as one valuable source of data that needs to be subjected to qualitative data analysis similar to all the other qualitative data types.

6.6.4 Strengths and weaknesses of introspective methods

The major advantage of the use of introspective methods is obvious: by applying them we can potentially gain access to mental processes that are central, for example, to language processing and production but which are inacces-Qualitative data collection.

As Ericsson concludes, the theoretical and methodological questions raised about verbal reports have never cast doubt on people’s ability to recall part of their thought sequences and all major theoretical frameworks concerned with thinking have advocated the use of verbally reported sequences of thoughts.

6.7 Case studies

The ‘case study’ — as the term suggests—is the study of the ‘particularity and complexity of a single case’ (Stake 1995: xi). What is a ‘case’? Cases are primarily people, but researchers can also explore in depth a programme, an institution, an organization, or a community.

In fact, almost anything, can serve as a case as long as it constitutes a single entity with clearly defined boundaries. Research studies sometimes describe a series of ‘multiple cases’• this is fine as long as each individual case is considered the target of a separate study. For example, Duff (in press) describes her ethnographic case study of bilingual education in Hungary as having at least three levels: the first level was one case, the country itself; the second level involved three cases, the three schools studied, and the third level included as many as eight cases, the teacher participants.

How are the selected cases studied? Case study researchers usually combine a variety of data collection methods such as interviews, observation and document archives. Although case studies are typically discussed under the label of qualitative research (because a single case cannot be representative of a population), actual case studies often include quantitative data collection instruments as well such as questionnaires.

Thus, the case study is not a specific technique but rather a method of collecting and organizing data so as to maximize our understanding of the unitary character of the social being or object studied. In many ways, it is the ultimate qualitative method focusing on the ‘Particular One’.

6.7.1 Main features of a case study Stake distinguishes three types of case study:

 

(a) The ‘intrinsic case study’ is undertaken to understand the intriguing nature of a particular case. That is, the case is of interest not because it illustrates something or represents other cases but because of its own value or specialty;

(b) The `instrumental case study’ is intended to provide insight into a wider issue while the actual case is of secondary interest; it facilitates our understanding of something else; and

(c) The ‘multiple or collective case study’ where there is even less interest in one particular case, and a number of cases are studied jointly in order to investigate a phenomenon or general condition. Thus, a multiple case study is, in effect, an instrumental case study extended to several cases. As one of the leading case study researchers in applied linguistics, Duff (z0°6), describes, most of her students conduct case studies with 4-6 focal participants in one or more sites, and this multiple case study format can be seen as fairly typical. Duff explains that choosing six cases initially means that even if there is attrition among the participants (and there usually is), there will likely be 3-4 cases remaining.

6.7 The generalizability issue

The main reservation about case studies, particularly if they concern individuals rather than social objects, is their generalizability: how can knowledge coming from one inevitably idiosyncratic source be of wider relevance?

As a result, this issue has been the subject of heated discussions in the literature and warrants further attention here too. As we saw above, in intrinsic case studies the case represents some unique or not-yet-understood feature which might explain not only to the researcher but also to the research audience the relevance of the case to our general understanding of a wider domain.

For example, few people would question that a case study of a ‘language savant’ (an individual with less-than-average mental skills but with very special language abilities) or a uniquely gifted language learner would be a worthwhile and a widely sought-after investigation in applied linguistics.

The real issue, then, is what Stake (1995, 1005) has termed the instrumental case study, where we examine a case to gain insights into a more general matter. There are two points to consider here:

• Analytic generalization

Duff (in press) points out that although the concept of ‘generalization’ is usually understood as generalizing to populations, it can also refer to the generalization to theoretical models, resulting in `analytic generalization’. The early case studies in applied linguistics (see Section 6.7.3 below), many of which fall under the instrumental category, represent this approach well because they led to the formulation of several theoretical principles and models that are still considered relevant today.

• Purposive sampling

In instrumental case studies a key issue underlying the broader relevance of the investigation is the sampling strategy that results in the selection of the particular case. Indeed, Duff (in press) confirms that case selection and sampling are ‘among the most crucial considerations in case study research’. Section 6.2.3 presents various qualitative sampling strategies that fall under the broader category of ‘purposive sampling’; I would argue that if conducted well, several of these strategies—such as typical, criterion, extreme/deviant, and critical case sampling—will lead to cases whose study can have a lot to offer to the wider research community, particularly in terms of new conceptualizations and novel propositions.

6.7.3 Case studies in applied linguistic research

Two recent reviews of case studies, by Duff (in press) and van Lier (2005), provide convincing evidence that the case study approach has been productive and highly influential in applied linguistics. In fact, as both scholars point out, the foundations of the whole discipline were based on the results of the first wave of case studies of language learners in the 1970s and 198os (by researchers such as Hatch, Schmidt, Schumann, and Wong-Fillmore).

These studies are still widely cited in the literature because they helped to shape our collective thinking in substantial ways. Case studies did not lose their significance after the foundations had been laid and applied linguistic research entered into a new phase of fine-tuning the broad observations made by the first generation scholars.

6.7.4 Strengths and weaknesses of case studies

The case study is an excellent method for obtaining a thick description of a complex social issue embedded within a cultural context. It offers rich and in-depth insights that no other method can yield, allowing researchers to examine how an intricate set of circumstances come together and interact in shaping the social world around us. When done well, as Duff (in press) concludes, case studies display a high degree of completeness, depth of analysis and readability, and they are effective in generating new hypotheses, models, and understandings about the target phenomena. Thus, the method is highly recommended for exploring uncharted territories or making sense of a particularly problematic research area, and it can provide an unparalleled understanding of longitudinal processes.

My personal feeling is that because of the heightened vulnerability of this method in terms of idiosyncratic unpredictability and audience criticality, in most cases it may be worth using (a) multiple-case designs or (b) case studies in some combination with other methods.

6.8 Diary studies

Diary-writing is a pervasive narrative form. (McDonough and McDonough 1997: I2I) Diaries have been used for hundreds of years to record the events of people’s everyday lives, but the diary study as a data collection method has been employed by social researchers only since the 1970s. Diary methods were initiated in the field of psychology to study emotions and moods across situations in daily experience and they have also been used in family psychology to obtain data about intimate aspects of the life of couples (Laurenceau and Bolger zoos).

In general, diaries offer the opportunity to investigate social, psychological, and physiological processes within everyday situations and, as Bolger et al. (2003) summarize, asking research participants to keep regular records of certain aspects of their daily lives allows the researcher to capture the particulars of experience in a way that is not possible using other methods.

6.8.1 Recording diary entries

Diary studies have often been classified into three categories depending on when the diary entries are to be made: ‘interval-% ‘signal-% and ‘event-contingent’ designs (Bolger et al. 2003). The interval-contingent design requires participants to report on their experiences at regular, predetermined intervals (for example, every afternoon at 4 pm). Signal-contingent designs’ rely on some signaling device such as a pager, a programmed wristwatch or a phone call to prompt participants to provide diary reports. This design is often used when studying momentary experiences of people such as psychological states (for example, happiness or stress).

6.8:2 Strengths of diary studies

Diaries offer a range of special features that are difficult or impossible to replicate using other data collection methods The most important ones are as follows: • Diaries allow the researcher an unobtrusive way of tapping into areas of people’s lives that may otherwise be inaccessible (Gibson 1995)

• We have seen in Chapter 2 that qualitative research involves the study of people in their natural contexts, with as little obtrusion as possible—diary studies can satisfy this requirement to a large extent.
• A further element of the qualitative inquiry is to try and elicit the participants’ own descriptions and interpretations of events and behaviors—in diary studies the participants inevitably become co-researchers as they keep records of their own feelings, thoughts, or activities. Diary data is by definition an insider account.
• Diary methods enable the researcher to study time-related evolution or fluctuation within individuals by collecting data on many occasions from the same individuals. Thus, diary studies are appropriate for looking at temporal variation in dynamic processes, investigating for example how people change or respond to certain stimuli. Diary studies are more sensitive to such questions than many other longitudinal designs (see Section 4.3) because they typically involve more frequent data recording that can capture changes with increased fidelity.
• Diary studies can provide ongoing background information that can help resolve ambiguity regarding causal direction between variables. In applied linguistics, Schmidt and Frota (1986) reported, for example, that diary entries helped them to confirm the existence of certain acquisitional orders.
• Diary studies offer a self-report format that reduces inaccuracies stemming from not remembering something correctly, because when writing their entries participants recall recent rather than distant events (van Eerde et al. 7.005).

6.8.3 Weaknesses of diary studies

If diary studies have as much potential as described in the previous section, why don’t we use them more? There are at least two main reasons for the dearth of diary studies in applied linguistics. One is simply that this method is rather novel and therefore it is not yet covered in standard methodology courses and texts. Thus, many scholars are simply not familiar with it sufficiently to consider it a realistic option when preparing a research design. The other reason is that the method has some serious weaknesses which make researchers think twice before attempting to use it. Let us have a look at these:

• The first and most obvious limitation of diary studies is that the inform-ant needs to be not only literate but actually comfortable at writing diary entries. As Gibson (1995) points out, one way of overcoming the problem of illiteracy would be to allow the informant to keep an audio or video diary.
• The second source of weakness is that a diary study is very demanding on the part of the informant. First of all, it requires a detailed training session to ensure that participants fully understand the protocol. Second, to produce regular, high quality data, diary studies must achieve a level of participant commitment and dedication rarely required in other types of research (Bolger et al. 2003).
• A further, seemingly trivial but in actual practice rather serious shortcoming of diary studies is that they are vulnerable to honest forgetfulness, where participants fail to remember the scheduled response times or fail to have the diaries at hand (Bolger et al. 1003); alternatively, they may be too tired or simply not in the mood to prepare their scheduled entry. This, of course, can defeat the main benefit of diaries, their ability to obtain accurate, real-time information.
• It has also been found that the length and depth of diary entries show considerable variation. .

6.8.4 Practical suggestions to facilitate diary studies

Despite all the problems they might be associated with, diary studies could be used to good effect more often in applied linguistic research. Therefore, let me make some practical points to facilitate this practice:

• The multiple benefits of diary studies would warrant in many cases at least an attempt to implement a diary study. I have seen encouraging examples of successful diary studies in educational contexts where the researchers were in regular touch with the informants (for example, they or their research assistants/associates were teaching the respondents).
• Researchers have tried different ways of increasing the informants’ motivation to persist with the study and make high quality entries at the scheduled time. (See Bolger et al. zoos; Gibson 1995.) First, the procedures can be made as convenient and user-friendly as possible, with making portable, pocket-size diaries and preprint the dates and times of expected responses onto the diary sheets in order to keep participants on track. In certain studies the diary can be further structured by adding, for example, key questions—when? Where? What? Who? — on the diary pages. Second, a regular gentle check-up and, if needed, ‘nudge’ by a research assistant with whom the informant has developed good rapport has been found to be beneficial. Third, in some instances researchers have used rewards as an incentive to boost compliance—this can either be money or some gift token.
• Finally, even if there are gaps in the diary entries, these can be filled in by follow-up interviews. In this case the diary entries have a similar function to the written or auditory recall prompts used in stimulated recall method-ology. (See Section 6.6.2.)

6.9 Research journals

One of the best ways to begin to define what interests you and your thoughts about what you already know is with a research journal. (Hatch and Lazaraton 1991: io) Research journals are diaries kept by the researchers themselves during the course of a research project rather than by the participants of a study concerning the topic of the investigation (which was discussed in the previous section); to make this distinction clear, I will refer to these researcher notes as ‘journals’ even though in the literature the terms ‘diary’ and ‘journal’ are often used synonymously.

6.9.1 How to keep research records

Scientific data needs to meet certain requirements of reliability and validity, and therefore if we decide to use our research journal as a data source, we need to pay attention to some technical details. If we keep a pencil and paper journal, the minimal requirement is to have a properly pre-bound folder with each page numbered and each entry dated, but some researchers recommend a more formalized approach.

Silverman (zoos), for example, outlines an organization framework consisting of four categories:

(I) observation notes about experiences,

(2) Methodological notes about how and what kind of data were collected,

(3) Theoretical notes describing hunches, hypotheses and ideas, and (4) personal notes containing feeling statements (concerning for example, satisfaction, surprise, shock, etc.) and other subjective comments.

Cryer (woo: 99) recommends that research students should record in their logbooks and journals the following:

• What you do, and where, how, when, and why you do it.
• What you read.
• What data you collect and how you process it.
• Any outcomes of the data analysis.
• Particular achievements, dead ends, and surprises.
• What you think or feel about what is happening, and any ideas that may be relevant for your research.
• Anything else that is influencing you. With the increasing availability and use of personal, and especially portable, computers more and more scholars transfer some of the traditional paper and pencil records to electronic files.