Chapter 6: Qualitative data collection
Book Name: Research Methods in Applied Linguistics
Writer: Professor Zoltan Dornyei
Professor: Dr.Zoghi, M
Summarized :Saeed Mojarradi Ph.D. Candidate
Monday, November 19, 2018
Quantitative inquiry can be divided into two distinct phases:
2-and data analysis
They typically follow each other in a linear manner. In accordance with the flexible and emergent nature of the qualitative research process, qualitative data collection and analysis are often circular and frequently overlap.
Furthermore, it is sometimes problematic to decide whether a certain qualitative method refers primarily to data collection, data analysis or a unique, combined design—grounded theory and the case study are good examples of thig ambiguity.
It is fair to say that research methodology texts in general have been inconsistent in how they have broken down the broad umbrella term ‘qualitative research’ into specific data collection and data analytical methods, techniques, and designs.
6.1 Qualitative data
Qualitative data can come from many sources, it is usually transformed into textual form resulting in hundreds (if not thousands) of pages of transcripts and field.
I believe it is important to highlight two main characteristics of a typical qualitative dataset: its tendency to become increasingly long and its rather unfocused and heterogeneous nature.
To put it broadly, qualitative data tends to be bulky and messy.
In qualitative research there are no explicit restrictions on what can be considered ‘data’ and as Richards (zoos) concludes, the researcher in a qualitative project often starts out by treating everything around a topic as potential data.
Qualitative data expands quickly, and novice researchers often find that the real challenge is not to generate enough data but rather to generate useful data. In fact, a serious problem in qualitative research can be too much data, which is augmented by the fact that ‘qualitative data are messy records’ (p. 34), usually consisting of a mixture of field notes, transcripts of various recordings as well as documents of a diverse nature and length. This does not mean that qualitative data cannot produce valuable results, but it does mean that processing such sizeable and heterogeneous datasets can involve a lot of work.
Accordingly, the design of most qualitative data collection is fluid and open-ended, and researchers are not required to plan all elements of the project at the outset. While this is in line with the discovery-oriented character of qualitative inquiry, Richards (Zoos) also stresses that possibly the most common cause of problems in qualitative data collection is the lack of any plans for data reduction. As she argues, ‘Novice researchers can very easily frame the scale of a project in terms of how widely they might need to spread their net, rather than how wide, realistically, the spread can be’ (p. 2o).
6.2 Sampling in qualitative research
S straightforward: we need a sizeable sample to be able to iron out idiosyncratic individual differences. Qualitative research, on the other hand, focuses on describing, understanding, and clarifying a human experience and therefore qualitative studies are directed at describing the aspects that make up an idiosyncratic experience rather than determining the most likely, or mean experience, within a group (Polkinghorne zoos).
Accordingly, at least in theory, qualitative inquiry is not concerned with how representative the respondent sample is or how the experience is distributed in the population. Instead, the main goal of sampling is to find individuals who can provide rich and varied insights into the phenomenon under investigation so as to maximize what we can learn. This goal is best achieved by means of some sort of ‘purposeful’ or ‘purposive’ sampling.
Purposive and theoretical sampling
Because there is always a limit to how many respondents we can contact or how many sites we can visit, we have to make some principled decisions on how to select our respondents, at which point to add additional participants to the sample and when to stop gathering more data. Even when our sample consists of one case only, we need to select the aspects of that case that we will focus on. (See ‘within-case sampling’ in Section 6.2.4.)
Therefore, a qualitative study must have a sampling plan describing the sampling parameters (participants, settings, events, processes), and this plan should line up with the purposes of the study. Punch (zoos) suggests that if it is not clear to us which cases, aspects, or incidents to study, it is usually worth devoting more work to developing the initial research questions.
6.2.2 Iteration, saturation, and sample size
Researchers are in agreement that the participant selection process should remain open in a qualitative study as long as possible so that after initial accounts are gathered and analyzed, additional participants can be added who can fill gaps in the initial description or can expand or even challenge it. This cyclical process of moving back and forth between data collection and analysis is often referred to as ‘iteration’. Although iteration is a key process in qualitative sampling, it cannot go on forever.
6.2.3 Specific sampling strategies
Purposive sampling can follow a number of different strategies depending on the research topic and setting. When designing the sampling plan, we also need to take into account feasibility issues (in terms of time, money, respondent availability) and— what is often ignored—saturation considerations. The more cohesive/homogeneous the sample, the faster the saturation, but at the same time, the narrower the scope of the project. The following list of the most common qualitative sampling strategies offers a sense of how wide we need to throw the net to be able to reach saturation in different research areas.
Relatively quick saturation can be achieved by the following three inter-related sampling strategies because they all aim at selecting participants who are similar in some respect:
The researcher selects participants whose experience is typical with regard to the research focus (for example, they all study a foreign language as a school subject at an intermediate level with moder-ate success). This strategy assumes that we have a profile of the targeted attributes possessed by an ‘average’ learner. Although we cannot generalize from the results because we cannot claim that everybody will have the same experience, we can list the typical or normal features of the experience,
The researcher selects participants who meet some specific predetermined criteria (for example, company executives who failed an important language exam). We can also gain valuable insight into an issue if, rather than selecting typical participants, we intentionally look at the whole range of possible responses, including very special cases
The researcher selects cases with markedly different forms of experience (for example, 1…2. learners from all develop-mental levels). This process will allow us to explore the variation within the respondents and it will also underscore any commonalities that we find: if a pattern holds across the sampled diversity, we can assume that it is reasonably stable.
Following the same logic as maximum variation sampling the researcher selects the most extreme cases (for example. the most motivated and demotivated learners). On the one hand, this allows us to find the limits of the experience; on the other hand, if even such cases share common elements, they are likely to be real core components of the experience.
The researcher deliberately targets cases which offer a dramatic or full representation of the phenomenon, either by their intensity or by their uniqueness (for example, in a language attrition study examining people who have completely forgotten an 1.i they used to speak). Their case may be taken as the most salient or comprehensive manifestation of the phenomenon under scrutiny; in such situations researchers are not only interested in what they find but also in what they do not, because something that does not occur in such salient cases is unlikely to happen elsewhere.
As it often happens, the most practical and feasible (and of course most common) sampling strategies are the least sound from a theoretical point of view. Three such ‘less principled’ strategies are particularly well known:
The starting point of this strategy is a principled list of key respondents, who are then asked to recruit further participants who are similar to them in some respect central to the investigation. This chain reaction can reach far, which is ideal in situations where the experience in question is rare.
This is an unplanned and potentially haphazard procedure in the sense that it is followed on the spur of the moment: while working in the field, the researcher sometimes comes across respondents who are ‘too good to miss’ and a decision to include them is made on the spot. The problem is that they are not always exactly what is needed, yet their selection is very much in line with the emergent nature of the qualitative inquiry.
This is the least desirable but the most common sampling strategy, at least at the postgraduate research level. It is not purposive but largely practical: the researcher uses those who are available. Of course, in an ideal world nobody would employ a convenience sample, but research (and particularly postgraduate research) all too often happens in less-than-ideal circumstances, under considerable time or financial constraints. One redeeming feature of this sampling strategy is that it usually results in willing participants, which is a prerequisite to having a rich dataset. On the other hand, saturation may not happen at all. Thus, this strategy may save time, money, and effort, but at the expense of credibility (Miles and Huberman 1994).
6.2.4 Within-case sampling
Although qualitative researchers are becoming aware of the importance of principled, purposive sampling, the process is usually applied only to select-ing respondents, that is, to ‘between-case sampling’. In some qualitative methods (for example, ethnography or case study) we need to extend our sampling plan to also include ‘within-case sampling’ (i.e. selecting data from the potentially available data pool concerning a participant), because we have to make regular decisions about when and how to collect data from a particular respondent, what aspects of the case to direct our attention to and which activities, locations, or events to focus on. In these decisions we need to be just as purposive with our choices as with the selection of informants.
A good starting point in our exploration of the specific qualitative methods of discovery is ethnography, because this approach embodies in many ways the essence of the qualitative inquiry. In fact, ‘ethnography’ has frequently been used as a synonym for ‘qualitative research’ and Hammersley and Atkinson (1995: 1), for example, started their influential monograph on ethnography by stating, For the purposes of this book we shall interpret the term “ethnography” in a liberal way, not worrying much about what does and does not count as examples of it’.
The main goal of most ethnographic research is to provide a ‘thick description’ of the target culture, that is, a narrative that describes richly and in great detail the daily life of the community as well as the cultural meanings and beliefs the participants attach to their activities, events, and behaviors.
6.3.1 Main features of an ethnographic study
According to Harklau (2005), a hallmark of classic ethnographic research is that it involves firsthand ‘participant observation’ in a natural setting, and most studies that frame themselves as ‘ethnographic’ include some degree of this method. However, we saw that ethnographers also utilize several other data collection techniques and, in fact, Harklau points out that multiple data sources are usually considered desirable. So, if the method of data collection is not a key determinant of the ethnographic approach, what are the features that define it? The following three points are emphasized most frequently in the literature; they will need little elaboration here because they have been described earlier when discussing the qualitative approach in general. (See Section 2.3.2.)
In ethnographic studies the participants’ subjective interpretation of their own behaviors and customs is seen as crucial to understanding the specific culture. Therefore, a central aspect of ethnography is to find ways of looking at events through the eyes of an insider.
The subtleties of the participants’ meaning (and most often, multiple meanings) cannot be uncovered unless the researcher immerses him/herself in the culture and spends an extended period living there, observing the participants and collecting data. Therefore, a minimum stay of 6–I z months is usually recommended to achieve the necessary prolonged engagement.
Because the ethnographer is entering a new culture, the exact focus of the research will evolve contextually and ’emerge’ in situ only after some fieldwork has been done.
6.3.2 Main phases of an ethnographic study
An ethnographic study involves a complex process of ‘getting in’ and ‘coming out’ which can be described as a sequence of four relatively distinct phases (Morse and Richards 2002; see also Richards 2003):
6.3.3 Strengths and weaknesses of ethnographic research
The ethnographic approach is particularly useful for exploring uncharted territories and understanding social processes from the participants’ perspective. It is an excellent way of ‘crossing cultures’ and gaining insight into the life of organizations, institutions and communities.
In short, ethnography is ideal for generating initial hypotheses about something totally unknown. Hornberger (1994: 688) also highlights the capacity of ethnography to take a holistic view and focus on the whole picture that ‘leaves nothing unaccounted for and that reveals the interrelatedness of all the component parts’. As she goes on to argue,
The value here is that the approach allows, indeed, it is the very essence of the approach to ensure, comparison and contrast between what people say and what people do in a given context and across contexts in order to arrive at a fuller representation of what is going on. It is not enough for ethnographers to ask teachers about their communicative approach to ESL teaching; they must also observe it in action.
The second qualitative method of inquiry to be described in this chapter is conducting interviews. Interviewing is a frequent part of the social life surrounding most of us: we can hear interviews on the radio, watch people being interviewed on television, and we ourselves often participate in interviews of various types either as interviewers or interviewees.
As Miller and Crabtree (1999) point out, the interview genre with its turn-taking conventions and expectations for participant roles, etiquettes, and even linguistic phrases is usually shared cultural knowledge. It is exactly because interviewing is a known communication routine that the method works so well as a versatile research instrument—in fact, although there is a range of qualitative research techniques available for researchers, the interview is the most often used method in qualitative inquiries.
6.4.1 Main types of interviews
One-to-one interviews can be divided into different types according to the degree of structure in the process and whether there are single or multiple interview sessions. Let us start with the latter, less often mentioned aspect. Single or multiple sessions
The typical qualitative interview is a one-off event lasting about 3o-6o minutes. However, as Polkinghorne (zoos) argues, one-shot interviews are rarely able to produce the full and rich descriptions necessary for worth-while findings. Drawing on Seidman’s work he recommends that researchers administer a sequence of three interviews with the same participant to obtain sufficient depth and breadth.
The first interview breaks the ice and develops rapport, while also providing a quick sweep of the areas to be investigated later. The interval between the first and the second interviews allows the inter-viewer to prepare a more made-to-measure interview guide for the second session and it also offers the interviewee the chance to think more deeply about the experience.
As a result, the second interview is more focused than the first. Finally, having analyzed the transcripts of the first two sessions, in the third interview-the researcher can ask any ‘mop-up’ or follow-up questions to fill in and to clarify the account. It must be noted that the multiple session format outlined here is not the same as a longitudinal interview study (see Section 4.4) because the purpose of the three sessions is not to document temporal changes but to arrive at a full account. In a longitudinal interview study the multiple sessions would need to be organized differently, with the first one or two interviews creating the baseline knowledge and the subsequent, regularly occurring interviews focusing on how and why the particular phenomenon under study changes.
The second main categorizing principle of interviews is the degree of structure in them, with one extreme being the ‘structured interview’. In this format, the researcher follows a pre-prepared, elaborate ‘interview schedule/guide’, which contains a list of questions to be covered closely with every interviewee, and the elicited information shares many of the advantages (for example, comparability across participants) and disadvantages (for example, limited richness) of questionnaire data. Such tightly controlled interviews ensure that the interviewee focuses on the target topic area and that the interview covers a well-defined domain, which makes the answers comparable across different respondents. The other side of the coin is, however, that in a structured interview there is generally little room for variation or spontaneity in the responses because the interviewer is to record the responses according to a coding scheme.
The other extreme, the ‘unstructured interview’ (sometimes also referred to as the ‘ethnographic interview’), allows maximum flexibility to follow the interviewee in unpredictable directions, with only minimal interference from the research agenda. The intention is to create a relaxed atmosphere in which the respondent may reveal more than he/she would in formal contexts, with the interviewer assuming a listening role.
No detailed interview guide is prepared in advance, although the researcher usually thinks of a few (i-6) opening questions (sometimes called ‘grand tour’ questions) to elicit the interviewee’s story. During the interview, the researcher may ask an occasional question for clarification and may give reinforcement feedback as any good communication partner would to keep the interview moving, but interruptions are kept to a minimum. It is easy to see that for an unstructured interview to be successful it is indispensable that the interviewer establishes very good rapport with the interviewee. This kind of interview is most appropriate when a study focuses on the deep meaning of particular phenomena or when some personal historical account of how a particular phenomenon has developed is required. In-depth interviews can also be used where exploratory work is required before a more focused (for example, quantitative) study can be carried out.
In applied linguistic research most interviews conducted belong to the `semi-structured interview’ type, which offers a compromise between the two extremes: Although there is a set of pre-prepared guiding questions and prompts, the format is open-ended and the interviewee is encouraged to elaborate on the issues raised in an exploratory manner. In other words, the interviewer provides guidance and direction (hence the `-structured’ part in the name), but is also keen to follow up interesting developments and to let the interviewee elaborate on certain issues (hence the ‘semi-‘ part).
Because of the great popularity of this interview format, most of the following recommendations on question wording and interview conduct will be geared at semi-structured interviews in particular.
The semi-structured interview is suitable for cases when the researcher has a good enough overview of the phenomenon or domain in question and is able to develop broad questions about the topic in advance but does not want to use ready-made response categories that would limit the depth and breadth of the respondent’s story. This format therefore needs an ‘interview guide’ which has to be made and piloted in advance. Usually, the interviewer will ask the same questions of all of the participants, although not necessarily in the same order or wording, and would supplement the main questions with various probes—see below.
Preparing for the interview and designing the interview guide
The complete interview process involves a series of carefully designed steps, with the preparation starting well before the first interview session. After the initial sampling plan has been finalized and ethical issues such as informed
A few trial runs can ensure that the questions elicit sufficiently rich data and do not dominate the flow of the conversation.
The main function of the interview guide (or ‘interview schedule/protocol’) is to help the interviewer in a number of areas:
(a) by ensuring that the domain is properly covered and nothing important is left out by accident; (b) by suggesting appropriate question wordings; (c) by offering a list of useful probe questions to be used if needed; (d) by offering a template for the opening statement; and (e) by listing some comments to bear in mind.
It might be advisable to combine this guide with an ‘interview log’ and thus leave space in it for recording the details of the interview (for example, participant, setting, length) as well as for the interviewer’s comments and notes.
Question types and wording issues
There are a variety of questions we can include in the interview guide, but we need to bear in mind that these only provide the framework and the real meaning is so often uncovered through exploratory and unstructured responses that deviate from the interview schedule.
These are particularly important in an interview, not so much from the content point of view but rather because they set the tone and create initial rapport. If the interviewees feel that they can do themselves justice when answering these initial questions, this will make them feel competent, help them to relax and consequently encourage them to open up. This is why researchers often start with easy personal or factual questions (for example, about the respondent’s family or job). The quality of the subsequent responses will to a large extent depend on the climate of trust that we create in this initial ice-breaking period.
Patton (2002) points out that on any given topic, it is possible to ask any of six main types of question focusing on: (a) experiences and behaviors, (b) opinions and values, (c) feelings, (d) knowledge, (e) sensory information (i.e. what someone has seen, heard, tasted, smelled, etc., and even what the interviewer would have seen or heard if he/she had been in a particular place), and (f) background or demographic information.
These six categories concern different aspects of the participant’s overall view/experience of the phenomenon and therefore we can get a rounded picture by including in our interview guide questions that tap into each dimension.
The emergent nature of qualitative interview data can be enhanced by applying various probes that use what the interviewee has said as a starting point to go further and to increase the richness and depth of the responses. Probes may include detail-oriented and clarification questions but a technique often used in person-centered psychotherapy is to simply take a salient content word used by the respondent and ask to elaborate (`You have used the word “freedom” mice—what exactly does it mean to you/do you mean by that …?’). Patton (low.) also mentions an interesting probe, the ‘contrast probe’ which asks about how a particular experience/ feeling/action/term compares to some other, similar concept.
This permits the interviewee to have the final say. Several scholars have noted in the literature the richness of the data that simple closing questions such as the following ones can yield: ‘Is there anything else you would like to add?’ or ‘What should I have asked you that I didn’t think to ask?’
With regard to the wording of the questions asked in a qualitative interview, the literature contains a great deal of advice. Some of this advice is self-evident (for example, ‘don’t use words that the interviewee does not understand’), but some other suggestions are truly practical and useful—Patton (20oz), for example, offers particularly detailed guidelines. Similar to the wording of written questionnaire items (see Section 5.2.3), there are some rules of thumb about how to formulate our questions.
Two particularly important rules are that we should avoid (a) leading questions (`It was frustrating, wasn’t it …?’) and (b) loaded or ambiguous words and jargon. In general, short and relatively simple questions that contain only one idea work best. Using words that make sense to the interviewee and reflect his/her worldview help to connect to the respondent and improve the quality of the interview data.
6.4.3 Conducting the interview
While it is true that practice usually makes us more relaxed and polished interviewers, I have found that simply being familiar with certain key issues regarding interview conduct is already helpful and there are several techniques that we can implement right from the start with good effect. So let us examine key components of the interviewing process.
Recording the interview
There is general agreement in the literature that if we want to use the content of a semi-structured or unstructured interview as research data, we need to record it—taking notes is simply not enough as we are unlikely to be able to catch all the details of the nuances of personal meaning; furthermore, note-taking also disrupts the interviewing process. However, we must be aware of the fact that many people do not like to be recorded, and therefore we must discuss this aspect with the interviewee in advance. Recording has a technical and a theoretical, aspect.
Conducting the interview
A good qualitative interview has two key features: (a) it flows naturally, with the various parts connecting seamlessly. We must remember that we are there primarily to listen (and not speak!) — This is, in fact, the first of Robson’s (2oo2) general recommendations on interviewing (presented in Table 6.0. We must let the interviewee dictate the pace without being rushed or interrupted. Even if there is a silence, we need to be patient and resist stepping in too quickly with a new question; and (b) it is rich in detail.
Once we have successfully established the flow of the interview, we need to sustain it in an unobtrusive way. There are a number of useful techniques to keep the interview on track:
What shall we do when the interview is not going as well as we think it could? After all, people often like to talk and go off at a tangent in a way which is not ‘spontaneously emerging personal meaning’ but merely waffling. If we give regular short reinforcement feedback (for example, nodding, uh-huhs) then simply withholding these and replacing them with an interjected question or a shift in our position might be enough of a nudge to get back on track. Alternatively, as Patton (2002) points out, some clever phrases might be used for inoffensive interruption and refocusing (for example, ‘Let me stop you here for a moment and go back to what you said earlier to make sure that I understood you well’).
Ending the interview
We can signal that the interview is nearing the end by using pre-closing moves such as summarizing or recapping the main points of the discussion. These also have a significance content-wise because they allow the interviewee to correct anything that we may have misunderstood and to make any additional points. In fact, as discussed in connection to the interview guide (Section 6.4.1), we may wish to give the respondent an explicit opportunity to make any comments that might be relevant/important but which have not been covered in the rest of the interview CI have no further questions.
Do you have anything more you want to bring up, or ask about, before we finish the interview?’ Kvale 1996: 128). Whether we do this or not, we need to be careful not to end le interview on a difficult topic—just like in oral language exams, we need to include a final winding down phase by steering the interview towards positive experiences.
Strengths and weaknesses of interviews
The interview is a natural and socially acceptable way of collecting information that most people feel comfortable with and which can be used in a variety of situations and focusing on diverse topics to yield in-depth data.
The inter-viewer’s presence allows for flexible approaches, probing into any emerging new issue, while the interview guide helps to maintain a systematic coverage of the domain. Because of the high social profile of interviewing most of us will have several good interviewer models in our minds and therefore even beginning researchers are likely to be able to obtain rich data in their very first interviews.
The main weakness of the interview is that it is time-consuming to set up and conduct, and that it requires good communication skills on the part of the interviewer, which not all of us have naturally.
6.5 Focus group interviews
Focus group interviews are sometimes treated as a subtype of interviewing but because both the format and the interviewer’s role are considerably different in the interviewing process, I find it more appropriate to keep the two methods separate. Focus group interviews— as the name suggests—involve a group format whereby an interviewer records the responses of a small group (usually 6-12. members).
This is obviously an economical way to gather a relatively large amount of qualitative data and therefore focus groups are used for a variety of purposes in many different fields. The name originally comes from market research but now the terms ‘focus group interview’ and `group interview’ are used interchangeably. The method has become familiar to the larger public when political parties have started to use it to gauge voter reaction to some planned policies and group interviews also feature frequently on television.
The focus group format is based on the collective experience of group brain-storming, that is, participants thinking together, inspiring and challenging each other, and reacting to the emerging issues and points.
6.5.1 Characteristics of the focus groups
The size of a focus group ranges between 6–io (sometimes i 2.) people. Fewer than six people would limit the potential of the ‘collective wisdom’ whereas too large a size makes it difficult for everyone to participate. When designing a focus group study, the two key technical questions to decide are (a) whether to have homogeneous or heterogeneous people in a group; and (b) how many groups to have.
Although, in line with the principles of maximum variation sampling (see Section 6.2.3), heterogeneous samples consisting of dissimilar people could in theory be useful in providing varied and rich data that covers all angles, it has been found that the dynamics of the focus group works better with homogeneous samples.
6.5.2 Conducting the focus group interview
In focus group interviews the interviewer is usually referred to as the ‘moderator’, and this special name reflects the fact that the researcher’s role differs from that in one-to-one interviews. Although they still need to ask questions, during the session they need to function more as facilitators of the discussion than as interviewers in the traditional sense. Because the dynamic of the focus group is one of the unique features of this method, the researcher’s role inevitably involves some group leadership functions, including making sure that nobody dominates the floor and that even the shyer participants have a chance to express their views.
6.5.3 Strengths and weaknesses of focus group interviewing
Focus groups are highly versatile: as mentioned earlier, they can be used in a wide range of areas, from market research to political opinion probing, and in educational contexts they are also often used for programme evaluation to assess the effectiveness of a particular course to understand what was or was not working and why. People do not usually mind participating in focus groups—in fact, they tend to find the sessions enjoyable and stimulating—and the interviews typically yield rich data. These merits, coupled with the quick turnaround, make the method popular.
Because of the flexible and information-rich nature of the method, focus groups are-often used in mixed methods research. Although they can be used as a stand-alone method of inquiry, this is not that common except for some areas where focus groups are well established (most notably market research) because researchers often feel that they have more control over the content and can elicit deeper and more ‘unedited’ personal meaning in one-to-one interviews. In applied linguistic research they have been widely used for generating ideas to inform the development of questionnaires and subsequent deep interviews.
The downside of focus group interviews is that they need quite a bit of preparation to set up, and to do them well the moderator needs to be able to carry out several functions simultaneously. There is also more need to improvise because the number of questions to be asked following the interview guide is relatively low and a lot of the content emerges from well-managed group discussion facilitated by probe questions. With regard to the content of the elicited material, Smithson (z000) mentions two possible limitations: the tendency for certain types of socially acceptable opinion to emerge and for certain types of participant to dominate the research process. Finally, an important technical point is that transcribing such an interview can be rather difficult because of the number of people (i.e. voices) involved. For this reason, researchers sometimes also prepare a video recording of the interviews —ancillary to audiotaping — so that they can identify who is speaking at any one time.
6.6 Introspective methods
Ever since the beginnings of psychological research at the end of the nineteenth century, psychologists have been trying to find ways of obtaining information about unobservable mental processes such as thoughts, feelings, and motives. One obvious source of information about these processes is the individual him/herself, and the various ways of eliciting self-reflections from respondents is usually referred to under the umbrella term ‘introspective method’. It subsumes several different approaches that all aim at helping the respondents to vocalize what is/was going through their minds when making a judgement, solving a problem or performing a task.
The data generated by this methodology is called a ‘verbal report’ or ‘verbal protocol’ and therefore introspective methods are also referred to as ‘verbal reporting’ or ‘protocol analysis’. We must note here that there is some inconsistency in the terminology in the area, because sometimes ‘introspection’ is used as a higher-order term that includes any form of self-report such as diaries, interviews, and even surveys, and when the term is applied so broadly, verbal reporting is seen only as a subset.
6.6.1 Think-aloud technique
One of the main proponents of introspective methods in psychology, Ericsson (2002), explains that the closest connection between thinking processes and verbal reports are found when participants are instructed to verbalize their ongoing thoughts while focusing on a task. This technique has come to be known as ‘think-aloud’ and it involves the concurrent vocalization of one’s `inner speech’ without offering any analysis or explanation. Thus, respondents are asked to verbalize only the thoughts that enter their attention while still in the respondent’s short-term memory. In this way, the procedure does not alter the sequence of thoughts mediating the completion of a task and, according to Ericsson, can therefore be accepted as valid data on thinking. The resulting verbal protocol is recorded and then analyzed.
6.6.2 Retrospective interview or stimulated recall
In ‘retrospection’, the respondents verbalize their thoughts after they have performed a task or mental operation. In such cases, the relevant information needs to be retrieved from long-term memory and thus the validity of retrospective protocols depends on the time interval between the occurrence of a thought and its verbal report. Ericsson (zoos.) concludes that for tasks with relatively short response latencies (less than 5-10 seconds), subjects are able to produce accurate recollections, but for cognitive processes of longer duration, the difficulty of accurate recall of prior thoughts increases.
How can we improve the quality of the retrospective data? Based on Gass and Mackey (z000), Mackey and Gass (zoos), the studies in Fxrch and Kasper (1987) as well as my own experience, the following recommendations can be made:
6.6.3 Analyzing introspective data
Just because introspective reports come ‘straight from the horse’s mouth’, they cannot be taken as the ultimate revelations about thought processes. Rather, as Kasper (1998) emphasizes, they represent information about what is going through someone’s mind when addressing a task or issue and the underlying cognitive processes need to be inferred from these verbal reports just as with other types of data. In other words, verbal reports must be seen only as one valuable source of data that needs to be subjected to qualitative data analysis similar to all the other qualitative data types.
6.6.4 Strengths and weaknesses of introspective methods
The major advantage of the use of introspective methods is obvious: by applying them we can potentially gain access to mental processes that are central, for example, to language processing and production but which are inacces-Qualitative data collection.
As Ericsson concludes, the theoretical and methodological questions raised about verbal reports have never cast doubt on people’s ability to recall part of their thought sequences and all major theoretical frameworks concerned with thinking have advocated the use of verbally reported sequences of thoughts.
6.7 Case studies
The ‘case study’ — as the term suggests—is the study of the ‘particularity and complexity of a single case’ (Stake 1995: xi). What is a ‘case’? Cases are primarily people, but researchers can also explore in depth a programme, an institution, an organization, or a community.
In fact, almost anything, can serve as a case as long as it constitutes a single entity with clearly defined boundaries. Research studies sometimes describe a series of ‘multiple cases’• this is fine as long as each individual case is considered the target of a separate study. For example, Duff (in press) describes her ethnographic case study of bilingual education in Hungary as having at least three levels: the first level was one case, the country itself; the second level involved three cases, the three schools studied, and the third level included as many as eight cases, the teacher participants.
How are the selected cases studied? Case study researchers usually combine a variety of data collection methods such as interviews, observation and document archives. Although case studies are typically discussed under the label of qualitative research (because a single case cannot be representative of a population), actual case studies often include quantitative data collection instruments as well such as questionnaires.
Thus, the case study is not a specific technique but rather a method of collecting and organizing data so as to maximize our understanding of the unitary character of the social being or object studied. In many ways, it is the ultimate qualitative method focusing on the ‘Particular One’.
6.7.1 Main features of a case study Stake distinguishes three types of case study:
(a) The ‘intrinsic case study’ is undertaken to understand the intriguing nature of a particular case. That is, the case is of interest not because it illustrates something or represents other cases but because of its own value or specialty;
(b) The `instrumental case study’ is intended to provide insight into a wider issue while the actual case is of secondary interest; it facilitates our understanding of something else; and
(c) The ‘multiple or collective case study’ where there is even less interest in one particular case, and a number of cases are studied jointly in order to investigate a phenomenon or general condition. Thus, a multiple case study is, in effect, an instrumental case study extended to several cases. As one of the leading case study researchers in applied linguistics, Duff (z0°6), describes, most of her students conduct case studies with 4-6 focal participants in one or more sites, and this multiple case study format can be seen as fairly typical. Duff explains that choosing six cases initially means that even if there is attrition among the participants (and there usually is), there will likely be 3-4 cases remaining.
6.7 The generalizability issue
The main reservation about case studies, particularly if they concern individuals rather than social objects, is their generalizability: how can knowledge coming from one inevitably idiosyncratic source be of wider relevance?
As a result, this issue has been the subject of heated discussions in the literature and warrants further attention here too. As we saw above, in intrinsic case studies the case represents some unique or not-yet-understood feature which might explain not only to the researcher but also to the research audience the relevance of the case to our general understanding of a wider domain.
For example, few people would question that a case study of a ‘language savant’ (an individual with less-than-average mental skills but with very special language abilities) or a uniquely gifted language learner would be a worthwhile and a widely sought-after investigation in applied linguistics.
The real issue, then, is what Stake (1995, 1005) has termed the instrumental case study, where we examine a case to gain insights into a more general matter. There are two points to consider here:
Duff (in press) points out that although the concept of ‘generalization’ is usually understood as generalizing to populations, it can also refer to the generalization to theoretical models, resulting in `analytic generalization’. The early case studies in applied linguistics (see Section 6.7.3 below), many of which fall under the instrumental category, represent this approach well because they led to the formulation of several theoretical principles and models that are still considered relevant today.
In instrumental case studies a key issue underlying the broader relevance of the investigation is the sampling strategy that results in the selection of the particular case. Indeed, Duff (in press) confirms that case selection and sampling are ‘among the most crucial considerations in case study research’. Section 6.2.3 presents various qualitative sampling strategies that fall under the broader category of ‘purposive sampling’; I would argue that if conducted well, several of these strategies—such as typical, criterion, extreme/deviant, and critical case sampling—will lead to cases whose study can have a lot to offer to the wider research community, particularly in terms of new conceptualizations and novel propositions.
6.7.3 Case studies in applied linguistic research
Two recent reviews of case studies, by Duff (in press) and van Lier (2005), provide convincing evidence that the case study approach has been productive and highly influential in applied linguistics. In fact, as both scholars point out, the foundations of the whole discipline were based on the results of the first wave of case studies of language learners in the 1970s and 198os (by researchers such as Hatch, Schmidt, Schumann, and Wong-Fillmore).
These studies are still widely cited in the literature because they helped to shape our collective thinking in substantial ways. Case studies did not lose their significance after the foundations had been laid and applied linguistic research entered into a new phase of fine-tuning the broad observations made by the first generation scholars.
6.7.4 Strengths and weaknesses of case studies
The case study is an excellent method for obtaining a thick description of a complex social issue embedded within a cultural context. It offers rich and in-depth insights that no other method can yield, allowing researchers to examine how an intricate set of circumstances come together and interact in shaping the social world around us. When done well, as Duff (in press) concludes, case studies display a high degree of completeness, depth of analysis and readability, and they are effective in generating new hypotheses, models, and understandings about the target phenomena. Thus, the method is highly recommended for exploring uncharted territories or making sense of a particularly problematic research area, and it can provide an unparalleled understanding of longitudinal processes.
My personal feeling is that because of the heightened vulnerability of this method in terms of idiosyncratic unpredictability and audience criticality, in most cases it may be worth using (a) multiple-case designs or (b) case studies in some combination with other methods.
6.8 Diary studies
Diary-writing is a pervasive narrative form. (McDonough and McDonough 1997: I2I) Diaries have been used for hundreds of years to record the events of people’s everyday lives, but the diary study as a data collection method has been employed by social researchers only since the 1970s. Diary methods were initiated in the field of psychology to study emotions and moods across situations in daily experience and they have also been used in family psychology to obtain data about intimate aspects of the life of couples (Laurenceau and Bolger zoos).
In general, diaries offer the opportunity to investigate social, psychological, and physiological processes within everyday situations and, as Bolger et al. (2003) summarize, asking research participants to keep regular records of certain aspects of their daily lives allows the researcher to capture the particulars of experience in a way that is not possible using other methods.
6.8.1 Recording diary entries
Diary studies have often been classified into three categories depending on when the diary entries are to be made: ‘interval-% ‘signal-% and ‘event-contingent’ designs (Bolger et al. 2003). The interval-contingent design requires participants to report on their experiences at regular, predetermined intervals (for example, every afternoon at 4 pm). Signal-contingent designs’ rely on some signaling device such as a pager, a programmed wristwatch or a phone call to prompt participants to provide diary reports. This design is often used when studying momentary experiences of people such as psychological states (for example, happiness or stress).
6.8:2 Strengths of diary studies
Diaries offer a range of special features that are difficult or impossible to replicate using other data collection methods The most important ones are as follows: • Diaries allow the researcher an unobtrusive way of tapping into areas of people’s lives that may otherwise be inaccessible (Gibson 1995)
6.8.3 Weaknesses of diary studies
If diary studies have as much potential as described in the previous section, why don’t we use them more? There are at least two main reasons for the dearth of diary studies in applied linguistics. One is simply that this method is rather novel and therefore it is not yet covered in standard methodology courses and texts. Thus, many scholars are simply not familiar with it sufficiently to consider it a realistic option when preparing a research design. The other reason is that the method has some serious weaknesses which make researchers think twice before attempting to use it. Let us have a look at these:
6.8.4 Practical suggestions to facilitate diary studies
Despite all the problems they might be associated with, diary studies could be used to good effect more often in applied linguistic research. Therefore, let me make some practical points to facilitate this practice:
6.9 Research journals
One of the best ways to begin to define what interests you and your thoughts about what you already know is with a research journal. (Hatch and Lazaraton 1991: io) Research journals are diaries kept by the researchers themselves during the course of a research project rather than by the participants of a study concerning the topic of the investigation (which was discussed in the previous section); to make this distinction clear, I will refer to these researcher notes as ‘journals’ even though in the literature the terms ‘diary’ and ‘journal’ are often used synonymously.
6.9.1 How to keep research records
Scientific data needs to meet certain requirements of reliability and validity, and therefore if we decide to use our research journal as a data source, we need to pay attention to some technical details. If we keep a pencil and paper journal, the minimal requirement is to have a properly pre-bound folder with each page numbered and each entry dated, but some researchers recommend a more formalized approach.
Silverman (zoos), for example, outlines an organization framework consisting of four categories:
(I) observation notes about experiences,
(2) Methodological notes about how and what kind of data were collected,
(3) Theoretical notes describing hunches, hypotheses and ideas, and (4) personal notes containing feeling statements (concerning for example, satisfaction, surprise, shock, etc.) and other subjective comments.
Cryer (woo: 99) recommends that research students should record in their logbooks and journals the following: