Com 473--Baldwin—Communication—Illinois State University

Grounded Theory Evaluation Exercise

Updated 9/12/08

Objectives: Through this exercise students should be able to:

• Demonstrate a solid understanding of the principles of grounded theory, as put forth by Strauss & Corbin (1998)
• Develop and evidence skills of thoughtful, respectful, and balanced evaluation of research
• Illustrate an understanding of some of the complex issues involved in making research “solid,” “well-done,” or “worthwhile” (i.e., of various ways to increase the “worth,” “validity,” or “value” of qualitative research.
• Articulate an understanding of standard criteria for evaluating qualitative research, including but not limited to terms and strategies put forward by Lincoln & Guba (1985).

Details: To accomplish this exercise, students should:

• You will evaluate a research article provided to you by the instructor that uses grounded theory in developing its conclusions (or locate your own).
• You may work individually or in pairs on this assignement.
• You should provide a solid evaluation of the article primarily in terms of criteria for judging qualitative research in general or grounded theory specifically. Some thoughts:
• A good evaluation considers both strengths and limitations
• You should consider several of each (strengths and limitations) with some detail and clear examples of the stance you take. Don’t just point something out—make sure it is clear if it is a strength or a limitation of the article (or both at the same time in different ways).
• Make sure you let the reader know where you are getting your criteria from.
• You might use some of the notions from Lincoln & Guba Ch. 11, or some standards from other sources. You can “mix & match” as long as you use some generally accepted criteria in addition to your own solid reasoning and tell the reader where the information is from. A good evaluation synthesizes the various tools available for a “thoughtful” rather than a “lockstep” evaluation.
• You may want to provide a brief paragraph summary of the article in the beginning of the paper (general direction, purpose, method of the study), but beyond that, we need only details pertinent to your actual evaluation. This paper is an evaluation, not an abstract.
• Your final paper will likely be about 3-4 pp. long, DS, standard font, 1 inch margins. If you cite sources, be sure to provide refs at the end, but they can be at bottom of last page instead of on a separate page.
• Feel free to discuss the article with Baldwin and/or with one another before you write!

Observation Evaluation Page

Grounded Theory Evaluation ISU# Last 5: _________ Score __ /50

	Lo				Hi
Overall writing, structure o Balance of depth and breadth of analysis (10 pts) o Organization/cohesion of paper as a whole (10 pts)	4	8	12	16	20
	Your Points: xx/ 20 Comments: xyz
Analysis (evaluation of the article) o Application of a diversity of criteria (10 pts) o Appropriate use of criteria (Are criteria used, interpreted, applied to aspects of article correctly?) (10 pts) o Appropriate consideration of article (Is article covered sufficiently? Are aspects relevant to GT assignment covered?) (10 pts)	6	12	18	24	30
	Your Points: xx/30 Comments: xyz
WRITING: There is not a specific grade for writing quality; excellent quality might curve a marginal grade upward. Excessive errors, especially in things such as spelling, grammar, punctuation (the most obvious rules) can lead to a grade deduction
TOTAL (out of 50): A balance between objective components and my subjective impression of the essay as a whole

Example:

Fulano Fernández

Megano Menéndez

Grounded Theory Evaluation

Browning, L.D. (1978). A grounded organizational communication theory
derived from qualitative data. Communication Monographs, 45,93-109.

Process Criteria: How is research accomplished?

Looking at this article on the whole, it is obvious that Browning is attempting to
follow the tenets of grounded theory in his research. This portion of the paper will focus
on how well Browning adhered to the process criteria for grounded theory in this research project. First, Browning states "rather than reviewing and interpreting the literature in organizational communication ... the project emphasized the search for and the discovery of grounded organizational communication variables by using data collection methods (in-depth interviews and non-participant observation)" (p 93). We believe that this statement indicates a dedication to using grounded theory. We will establish two sections of process criteria and use these to critique Browning's use of grounded theory. Following this, we will address product and practical criteria. The assumption that qualitative data is more useful than quantitative data will be examined in the first section of process criteria and then we will examine Browning's use of comparative analysis to generate conceptual categories from evidence in the second section dealing with process criteria.

Browning conducted interviews and observations in this experiment in a qualitative fashion continually over a relatively prolonged period of three months. These multiple methods were used in this study to triangulate the results. The researcher detailed the organization chosen and its favorable characteristics of being very intensely communicative and having three levels of employees located in the same building. However, no explanation was given as to which subjects within this hierarchy were selected for interviews and observations or why. A more detailed sampling description would be helpful for the reader of this study.

We feel that one of the strengths of this article was the use of categorization, from
development to revision to the creation of hypotheses. In the beginning of the study, interviews and observations were conducted simultaneously. After each day of this first phase, the interviewer, Hawes, and the observer, Browning, met to discuss their field notes of each instance. While this process was detailed in the article, specific references to the researcher's thoughts and feelings were omitted. Emergent design was used during this phase in the form of modifying questions and selecting important events to observe. This change in sampling came from the daily peer debriefing sessions in this early stage rather than from categories that emerged from the interviews and observations after the completion of those tasks. One criticism of this sampling change was that the modifications were not specified in much the same way as the original sampling procedures were not detailed. This form of emergent design is a variation from the grounded theory practice of developing categories throughout the study as no categories were established until after data collection.

Following the conclusion of observations and interviews, the data was converted into
manuscript form which then was broken down further into 233 interview incidents and 193 observation incidents. This breakdown of these mutually exclusive units was based on completeness of thoughts or acts performed by individuals. Endings to these units were signaled by a change in subject. Descriptive category labels were developed from these incidents.

Descriptions of settings, placement of people, key phrases, non-verbal behavior,
and outcomes were interpreted as contextual clues in the development of categories. Using
these, the evolution from descriptive to construct categories using open categorization was detailed. This was performed initially by the researcher who then had two judges review, suggest changes and eliminations, and sharpen and improve the accuracy of the category. This process served as a form of peer debriefing. Examples of these new categories which resulted from open coding include power (with high and low dimensions), requests for information (ranging from simple requests to demands to requests for approval, etc.), and pressure (from absence of reinforcement, having to oppose a powerful person, complying with another's wishes, etc.). From here, the categories were subordinated into core categories such as the power-advancement cluster (encompassing categories like power, personal advancement strategies, expression of distrust, etc.). Some of these categories were found to be present in multiple clusters thereby reflecting relationships between clusters and allowing the suggestion of hypotheses. One example in the power-advancement cluster is that as interpersonal power becomes more salient, personal advancement strategies will become more numerous. The formulation of these propositions as testable is apparent but the sheer number of them (36) makes explict testing and replication recommendations difficult. In the conclusion of the article, the researchers suggest that future research should address these propositions. Since these hypotheses were not tested after they were formed in this study, no contradictions or exceptions emerging through negative case analysis or any other form of reflection were noted. The hypotheses were not surveyed through peer debriefing or member checks according to what was presented in the article. In the conclusion, it was suggested that some of these hypotheses were consistent with prior research; however, specific hypotheses or the previous studies they reinforced were not named explicitly.

Product Criteria: What does the finished product look like?

In our eyes, Browning was successful in following many of the criteria of grounded theory in his effort to generate credible research and findings. From this criteria we used to evaluate the processes of this study, there were, however, some areas that Browning lacked in. He did not explicitly state the sampling techniques used, develop categories as the data collection was ongoing, give the reader any indication of his thoughts or feelings during the process, suggest testing methods for his hypotheses, or perform negative case analysis or member checks. We believe that, despite these shortcomings, Browning’s work has credibility for the number of grounded theory criteria he did meet.

The transferability of this study is somewhat questionable. Description of the organization is limited to who the subjects are, what they do, where the communication takes place, and why the organization is an attractive research setting. However, details of the observations and interviews, such as settings, placement of people, key phrases, non-verbal behavior, and outcomes are not present in the article. To transfer this study, one would have to contact the researcher to find out the details of these topics because they are not present in the article. Therefore, we feel that there is not adequate description given in this area. The other parts of the article, especially the areas of categories and clustering, have a sufficient amount of data and exemplars.

Browning finds a great number of hypotheses that need testing. As we mentioned above, the researcher derived 36 propositions based on the data analyzed which would suggest that future studies should examine these propositions and attempt replication. According to Browning, hypotheses found in the study apply to past research as well as creating new research opportunities.

The completeness of this article can be called into question. As mentioned previously, more details about the setting are necessary as are variations in theoretical explanation. Browning sets forth categories based on the observations without discussing any problematic instances or possible criticisms or limitations of his work. The process was recounted in sufficient details but the broader conditions of the experiment need more attention.

The categories derived and the way the hypotheses are developed seem reasonable to the reader. The categories are explained in detail both by use of definitions and exemplars and have explanations of all dimensions found. The relationships expected by the hypotheses are also explained as is the reasoning behind the propositions. The results of the study seem plausible to us.

Throughout this article, Browning remains out of arm’s reach by giving little detail on his role in the experiment. He tells us nothing of his biases, none of his thoughts or hunches, or of any self-criticisms or limitations he finds in his work. This makes Browning seem to favor the objectivist side of the continuum by distancing himself from the research; however, this is problematic in qualitative research aimed at developing grounded theory because this notation of self-reflection is important for other researchers who are attempting to interpret or replicate the findings of this study. It is obvious by looking through his description of categories that Browning can pay attention to details and can be thorough, but he is not when it comes to finding limitations through external audits or describing personal feelings about the study.

Practical Criteria: What is done after theory is completed?

The question that this third portion of the abstract will briefly address is what is the relationship of Browning’s theory to truth and reality. Before answering this question, two general assumptions about grounded theory must be outlined. The first assumption is that grounded theories are never established forever. The way in which the theories are developed allows for endless elaboration and partial negation. Second, grounded theories are limited by time. That is, theory conditions are established and valid primarily for the context (the course of a given research project) that they were developed within. Any changes to these conditions at any level of the conditional matrix will affect the validity of the theories. Consequently, grounded theories can become outdated overtime. With regards to Browning’s research, it appears that the developed theory fits these criteria well. The categories and testable propositions were clearly developed from the research project data and not from previous research. In his article, Browning explains that the categories and hypotheses are specifically geared for the current research project and that subsequent testing must be done in order to determine if they are situation specific or can operate consistently in organizations with similar characteristics. Additionally, he does make several recommendations regarding how the categories can be operationalized and the hypotheses can be tested, which are quite useful for future research on this topic.