Structure and Format Guidelines

Michael Bieber
This set of guidelines was originally designed for a course, so it includes grading guidelines, which most readers can ignore. Notes: 1. Follow the guidelines here on structuring and formatting your review. 2. Include each category in a separate section or subsection. Ensure that you discuss each category! If a category does not apply! then actually write in that sub"section that the category does not apply." #e will be loo$ing for each category when grading. %. &eference other articles you've read! if appropriate. (ust as in the articles you've read! include a reference mar$er where you refer to each in your review! and provide full bibliographic references in a bibliography section at the end of your report. ). *our final report should be ) full pages or more if measured in +imes ,ew &oman 1-pt single spaced"! which is e.uivalent to /01- pages in most #eb browsers. +his does not include your bibliography! citation analysis! diagrams or large gaps between parts of your report. +here is no ma1imum length. 2. 3o not copy any part of the article into your review. If you want to use more than %0) of the author's words! then use .uotation mar$s! and add a page number from the article if available" when you cite this .uotation. 4. 5roof0read your review for grammar and clarity as well as spelling. 6pell chec$ers do not fi1 grammar. *ou should be sure that you have caught all mista$es and written clearly. Most word processors have a grammar chec$er 0 use it! /. For the printed copy of your review! turn in 7ust a plain stapled blac$ 8 white copy of your review. 3o not put your review in a folder. 9void fancy covers! colors or fonts. #e want to 7udge your content! not visual presentation. :. 5lease reread the note on plagiarism posted in the syllabus and on the course #eb site.

Additional References to Outside Materials

In sections 6-14, utilize at least 3 citations to articles outside those on the class reading list. se these articles to !ac" u# your analysis, to descri!e or address issues, im#lications or recommendations for further research or de$elo#ment.

Grading Notes
%e ha$e included the grading criteria in these guidelines. nless otherwise noted, the categories are: great; fully discussed! e1plained well and well07ustified

o$; partial discussion< not e1plained well or well07ustified poor; barely discussed =ero; not discussed

Length Grading
%e ta"e & #oints off for each full #age missing 'i.e., less than 4 full #ages( and 3 #oints off for e$ery half #age missing. Note that large ga#s !etween sections will not count towards the full #age length.

Editing Grading
)as the re$iew !een #rooofread for !oth s#elling and grammar* If not u# to 3 #oints off for #oor grammar and u# to 3 #oints off for #oor s#elling.

Preparing for Your Revie

The #ur#ose of research articles is to #resent new or refine conce#tual ideas, or to #resent new e$idence for conce#tual ideas. + research article re$iew results from critically e,amining a research article. -ou will ha$e to read your article several times to understand it fully enough to re$iew #ro#erly. .ften, com#aring your article to others will hel# you determine its /uality. +lso, thin" a!out the article and its research ideas in terms of each of the different wee"0s conce#ts and framewor"s we study in class. 1o the ideas in the article fit all the categories of a##ro#riate class conce#ts, models and framewor"s* This often will hel# you see things the authors missed, thin" a!out things the authors write in a new light, or see that the authors indeed co$ered a to#ic thoroughly.

Outline of the Article Revie

2lease include the following categories in your article re$iew. 1. Full Bibliographic &eference (-3 if missing) 2. Introduction; >b7ectives! 9rticle 3omain! 9udience! (ournal and ?onceptual@Emprical ?lassification (8) %. Aery Brief 6ummary (4) ). &esults (8) 2. ?lass &eadings (4) 4. ?ontributions (8) /. Foundation (4) :. 6ynthesis with ?lass Materials (12 +8 extra credit) B. 9nalysis 8 9dditional 9nalysis (4 +8 extra credit) 1-. Ceneral ?riti.ue (10 + 6 extra credit)

11. Further ?riti.ue of a ?onceptual 9rticle 0or0 (12) Further ?riti.ue of an Empirical 9rticle (12+2 extra credit) 12. Issues listed by the author" (6+3 extra credit) 1%. Issues in your opinion" (6+6 extra credit) 1). Impact (9) 12. Duestions (6) 14. 9nnotated Bibliography (-5 if missing) 1/. ?itation 9nalysis 9ppendi1 (6) In addition, you can lose #oints if your re$iew is too short or #oorly edited. 3ee the 4rading Notes section a!o$e.

!" Full #i$liographic Reference

3tate the full !i!liogra#hic reference for the article you are re$iewing 'authors, title, 5ournal name, $olume, issue, year, #age num!ers, etc.( Im#ortant: this is not the !i!liogra#hy listed at the end of the article, rather the citation of the article itself6 Grading: -3 if missing

%" &ntroduction' O$(ectives) Article *omain) Audience) +ournal and ,onceptual-Emprical ,lassification
Note: For the on-line reviews done in some class sections, this category may be broken up into several separate subcategories. For the written review, please discuss all of these subcategories together as follows. 2aragra#h 1: 3tate the o!5ecti$es 'goals or #ur#ose( of the article. %hat is the article0s domain 'to#ic area(* 2aragra#h 7: 9udience; 6tate the article's intended audience. 9t what level is it written! and what general bac$ground should the reader have< what general bac$ground materials should the reader be familiar with to understand the articleE 9ppropriate (ournalE; #hy is the 7ournal appropriate or inappropriate" for this articleE ?hec$ the mission statement or purpose of the 7ournal itself from its cover or its #eb site." 2aragra#h 3: 3tate whether the article is 8conce#tual8 or 8em#irical8, and why you !elie$e it is conce#tual or em#irical. 9m#irical articles and conce#tual articles ha$e a similar o!5ecti$e: to su!stantiate an argument #ro#osed !y the author. %hile a conce#tual article su##orts such an argument !ased on logical and #ersuasi$e reasoning, an em#irical article offers em#irical e$idence to su##ort the argument. 9m#irical articles offer su!stantial, detailed e$idence which the authors analyze using statistical methods. 9m#irical articles must include

hy#otheses 'or #ro#ositions(, detailed research results, and 'statistical( analyses of this em#irical e$idence. 9m#irical research includes e,#eriments, sur$eys, /uestionnaires, field studies, etc, and to limited degree, case studies. :once#tual articles may refer to such em#irical e$idence, !ut do not #ro$ide the detailed analysis of that e$idence. .f course, !oth ty#es of articles can use real life e,am#les to !ac" u# their #oints. ;ust !ecause an article #ro$ides e,am#les, does not necessarily mean that it is em#irical. 'The lesson to ta"e home is not to consider a conce#tual article to !e an em#irical one 5ust !ecause it #ro$ides some summarized or some unanalyzed data.( Grading: b!ectives: great - 3" ok - #" poor - $ Grading: %udience&'ournal %ppropriateness: great - 3" ok - #" poor - $ Grading: (onceptual vs. empirical: great - #" ok&poor - $

." /er0 #rief Summar0

<or our article re$iews, we do not want you to s#end much s#ace summarizing the article. Instead we are more interested in your analysis of the article. Thus, in this section, summarize the article only $ery !riefly '7-3 #aragra#hs(. If #ossi!le, use the I3 research #aradigm as the format of your summary, !ut remaining $ery !rief: 5aragraph 1; what is the problem or opportunity being addressed 5aragraph 2; which solution is proposed the solution could be a new model or a theory that e1plains the problem" 5aragraph %; what evidence is put forth that this solution is appropriate If this is an empirical article! be sure to briefly describe what $ind of empirical study was done as part of the evidence" Grading: great - )" ok - #" poor - $

1" Results
=ery !riefly summarize the im#ortant #oints 'o!ser$ations, conclusions, findings( and 8ta"e home messages8 in the article. 2lease do not re#eat lists of items in the articles - 5ust summarize the essence of these if you feel they are necessary to include. Grading: great - *" ok - +" poor - #

2" ,lass Readings

1. 1oes this article directly cite any of the class readings, i.e., does any class reading a##ear e,#licitly in its !i!liogra#hy or reference section* If not, state this e,#licitly. If so, clearly descri!e how the authors use the cited article. )ow does the article you are re$iewing relate to and>or !uild u#on the class article it cites* If this article does not cite any class readings then 5ust state this. ,-f you do not state this e.plicitly, you will not receive credit for this section./ 1o not discuss any

other readings, such as other readings on the same to#ic or !y the same author. 3a$e any discussions of similar articles for your synthesis section !elow. 7. 1o any of the class readings cite your article '!esides the te,t!oo"(* If so, clearly descri!e how. If no class readings cite your article, then write in your re$iew 8No class readings cite this article.8 ,-f you do not state this e.plicitly, you will not receive credit for this section./ ?e sure to add all references you cite to the !i!liogra#hy. Grading: great - )" ok - #" poor - $ 0-f none, then score ) by default if this has been stated e.plicitly.1

3" ,ontri$utions
+n article ma"es a 8contri!ution8 !y adding to the "nowledge of researchers in a research field. +n article can ma"e a contri!ution to the research field in many ways. 1oes it #ro$ide a new way to loo" at a #ro!lem* 1oes it !ring together or 8synthesize8 se$eral conce#ts 'or framewor"s, models, etc.( together in an insightful way that has not !een done !efore* 1oes it #ro$ide new solutions* 1oes it #ro$ide new results* 1oes it identify new issues* 1oes it #ro$ide a com#rehensi$e sur$ey or re$iew of a domain* 1oes it #ro$ide new insights* +lso, is it salient 'rele$ant and current( to a #articular scientific issue or managerial #ro!lem* +re the issues addressed introduced in a way that their rele$ance to #ractice is e$ident* %ould answers to the /uestions raised in the article li"ely to !e useful to researchers and managers* Note: 1o not discuss the contri!utions of the technologies the article descri!es, !ut rather the contri!utions of the article itself6 The article0s contri!utions should !e original. To the !est of your "nowledge, are they* +re the article0s ta"e-home messages new* 1escri!e each contri!ution clearly in a se#arate #aragra#h or !ullet #oint. 1iscuss why the contri!ution is im#ortant. +lternati$ely, if you !elie$e the article ma"es no contri!utions, e,#lain why clearly. Grading: great - *" ok - +" poor - #

4" Foundation
4ood research often is !uilt u#on theories and framewor"s that other researchers ha$e de$elo#ed. 3ometimes articles will !e su!stantially !ased u#on this #rior wor", and refer !ac" to it in some detail. 'Not all research articles will do this.( %hich theoretical foundations does this article and research !uild on, if any* In what ways* Include references>citations of the foundation wor". '-ou can determine this in #art from the wor"s the article cites.(

Note, howe$er, that most wor"s cited are not core foundational wor", !ut rather 5ust su##ort certain as#ects of the article. 3imilarly, do not confuse a general discussion of related to#ics as foundational wor". If the article does not !uild u#on "ey #ieces of #rior research, then write in your re$iew 8This article does not !uild u#on any foundation research.8 ,-f you do not state this e.plicitly, you will not receive credit for this section./ Grading: great - )" ok - 3" poor -$ 0-f none, then score ) by default if this has been stated e.plicitly1

5" S0nthesis

ith ,lass Materials

The synthesis section should !e at least one full #age. 3ynthesis means analyzing a #articular to#ic !y com#aring and contrasting it with, and thin"ing a!out it from the $iew#oint of, the class materials from across the semester. These materials include the articles, models, framewor"s, guidelines and other conce#ts we0$e co$ered. '.f course, only certain materials will !e rele$ant for any gi$en article.( Note: -ou ha$e to do this synthesis6 -ou need to relate this article to other things we ha$e studied, so !y definition you will not find this analysis in the article itself6 1iscuss the article0s research ideas and results in terms of any rele$ant materials co$ered in class or which you ha$e found in the readings. -ou can also chec" the conce#ts in the 8to "now8 lin" on the 8/uic" lin"s8 #ortion of the course %e! site. :ite these readings e,#licitly, including their source in the !i!liogra#hy and a !i!liogra#hic mar"er in the te,t 'e.g., @Turoff et al., 1AAAB(. -ou also could analyze the a##roach the author too" to the article0s analysis and discussion. 1iscuss the article0s a##roach and results in terms of one or more of the framewor"s, etc., from the te,t or readings, or any you find elsewhere. <or e,am#le, if the authors discuss any ty#e of information system, you could use +lter0s %:+ analysis to e,amine how they a##roached that information system. Try to do this for all the models and framewor"s, etc., which a##ly to your article. +s #art of this analysis, reference other articles you0$e read, when a##ro#riate. :om#are the a##roach, results and contri!ution with all articles a!out similar to#ics or with a similar a##roach. <or e,am#le, if your article de$elo#s a new framewor", com#are it with ?andyo#adyhah0s framewor" criteria 'and $ice $ersa - whoe$er does ?andyo#adyhah0s article could test his criteria on framewor"s from the other readings(. Include any articles you cite in the !i!liogra#hy and use !i!liogra#hic mar"ers in the te,t. <or all of these, do your synthesis com#arison in as much de#th as you can6 Grading: four items up to #2 points total ,$# points plus * points e.tra credit/ - for each item: great - + ok - #" poor - $ Great: discussed deeply and relating the article in detail with the synthesi3ed models and frameworks. 4: the synthesi3ed information is only discussed in general

6" Anal0sis

Note: 5any people assume this category is the same as 6General (riti7ue6. -t is not. General (riti7ue is a different category from this, and follows below. %hat has changed since the article was written* )ow do it0s lessons, ideas and theories still a##ly* To what e,tent has its issues !een resol$ed* Grading: great - )" ok - #" poor - $

Additional Anal0sis
.#tionally, try a##lying the article0s models, framewor"s and guidelines, etc. yourself. 1o you find them useful* In addition, you may o#tionally add your own additional analysis in a se#arate su!section. '1o not re#eat the author0s analysis in the #a#er - you could summarize this as #art of the results section.( Grading: this section is e.tra credit only: great - *" ok - +" poor - #

!7" General ,riti8ue

In this section you should state your o#inions of how well 'or #oorly( the authors did their research and #resented the research results in the article. -our criti/ue can contain !oth #ositi$e and negati$e comments. ;ustify and e,#lain in detail each of your criti/ue #oints in a se#arate #aragra#h of at least 4-& sentences. The following are suggestions only: 3oes it build upon the appropriate foundation i.e.! upon appropriate prior research"E 3id the authors choose the correct approach! and then e1ecute it properlyE Fow confident are you in the article's results! and whyE 9re its ideas really new! or do the authors simply repac$age old ideas and perhaps give them a new nameE 3o the authors discuss everything they promise in the article's introduction and outlineE #hat are the article's shortcomings faults" and limitations boundaries"E 3id it discuss all of the important aspects and issues in its domain topic area"E In what way should the article have made a contribution! but then did notE 3o the authors ma$e appropriate comparisons to similar events! cases or occurrencesE Fow complete and thorough a 7ob did the authors doE 3o the authors include an ade.uate discussion! analysis and conclusionsE 3id they 7ustify everything ade.uatelyE 3id they provide enough bac$ground information for the intended audience to understand itE For you to understand itE #ere there ade.uate and appropriate e1amples and illustrationsE <or full credit, as" yourself these /uestions when 5ustifying your criti/ue #oints: why@why notE howE

what distinguishes the differences@different approaches! and in what waysE

Grading: four items up to $8 points total ,$2 points plus 8 points e.tra credit/ - for each item: great - )" ok - #" poor - $

!!c" Further ,riti8ue of a ,onceptual Article

CCC only for conce#tual articles

0adapted from guidelines from 9r. 9an :obey, Georgia ;tate <niversity1 + criti/ue of a conce#tual article e,amines the logic of the arguments made !y the authors. ?oth strengths and wea"nesses should !e identified in a criti/ue. 9,#lain and 5ustify each of your criti/ue #oints in at least 3-4 sentences. 4i$e e,am#les whene$er #ossi!le. To the !est of your a!ilities, discuss each of the following categories in a se#arate #aragra#h: 1. D.4I:+D :.N3I3T9N:-: 1o any #arts of the article or research contradict or in$alidate other #arts* If so, ha$e the authors ac"nowledged and e,#lained this ade/uately* 7. :.)9E9N:9: 1oes the article ma"e sense* 1id the authors a##roach this article 'and this research( sensi!ly* 1oes the article de$elo# an argument that follows a coherent line of reasoning* +re the !oundaries of the argument reasona!ly well defined* 1oes the argument antici#ate most, if not all, ri$al arguments* 1oes the article flow in a logical se/uence* 1o later #arts !uild logically u#on earlier #arts* 3. 3 ?3T+N:9: 1oes the article #ro$ide an argument or a line of reasoning that offers insight into im#ortant issues, or does it merely summarize #re$ious studies in a shallow way that does not reflect de#th of analysis* 1oes the article #ro$ide ways 'a model, framewor", guidelines, etc.( to guide future thin"ing a!out the issue's( the author is addressing* 4. <.: 3: Is there a clear audience that the authors address* %as the article written at the a##ro#riate le$el for this audience* Grading: for each: great - 3 ok - #" poor - $

!!e" Further ,riti8ue of an Empirical Article

CCC only for em#irical articles

0adapted from guidelines from 9r. 9an :obey, Georgia ;tate <niversity1 + criti/ue of an em#irical article e,amines the strength of the em#irical e$idence su##orting the author0s argument. ?oth strengths and wea"nesses should !e identified in a criti/ue. 9,#lain and 5ustify each of your criti/ue #oints in at least 34 sentences. To the !est of your a!ilities, discuss each of the following categories in a se#arate #aragra#h: 1. :D+EIT-: Is the article0s #ur#ose and argument clear* 1o the researchers clearly de$elo# a ma5or research /uestion, #ro#osition, or hy#othesis that is to !e

e$aluated in the em#irical study and discussed in this article* If the study is e,#loratory '#reliminary(, is sufficient 5ustification for an e,#loratory strategy gi$en* 7. T)9.E9TI:+D 4E. N1IN4: Is the researcher0s argument grounded in more !asic theory* Is it clear whether the structure of the em#irical study 'i.e., what they do( was deri$ed from theory, or 5ust made u#* In theory-!uilding articles, is the need for new theory ade/uately esta!lished* 3. 193I4N .< E939+E:) IN=93TI4+TI.N: Is it clear e,actly how the em#irical study was carried out* Is the design of the research a##roach 'field study, e,#eriments, /uestionnaires, etc. - !oth contents and how they will !e used( ade/uate to address the common threats to internal and e,ternal $alidity* )a$e a##ro#riate controls !een esta!lished, and is the selection of research sites 5ustified* +re the hy#otheses and e,#eriments, etc., significant* 4. F9+3 E9F9NT: 9m#irical studies can ha$e /uantitati$e measurements 'i.e., numeric results( and /ualitati$e or su!5ecti$e measurements. +re the measures used ade/uately descri!ed 'i.e., what is measured in the study and how(* +re data on the relia!ility and $alidity of these measures re#orted* 1oes the article feel anecdotal or solidly su##orted with e$idence* <or e,am#le, in case or field studies, are the results well documented* Is it clear who the su!5ects were, and with whom inter$iews were carried out* %ere im#ortant results cross-chec"ed, i.e., determined across a range of su!5ects or 5ust gotten from one or two su!5ects* &. +N+D-3I3: Is the analysis of em#irical data conducted #ro#erly* 1o the data conform to the re/uirements of any statistical tests used* +re /ualitati$e data ade/uately descri!ed and #resented* 6. 1I3: 33I.N +N1 :.N:D 3I.N3: In discussing the results of the em#irical study, do the authors remain true to the actual findings of the study* +re the claims made in the conclusion of the article actually su##orted !y the em#irical data* If the study is e,#loratory, do the authors offer research /uestions or hy#otheses for future research* G. ?I+393: 1o the !iases of the authors affect the design of the research or the inter#retation of the results* +re the authors aware of #otential !iases and the affect on the study* Grading: for each: great - # ok&poor - $

!%" &ssues 9listed $0 the author:

%hat o#en /uestions or issues has the author stated remain unresol$ed* 1iscuss each in a se#arate #aragra#h of &-1H sentences. 9ach issue0s #aragra#h should ta"e the following format: what is the issueE why do you believe this is an important issueE in what way is it unresolved suggestions for resolving it 0 if you give your own suggestions instead of or in addition to the authors'! then precede each with GI would propose ...G If it has been resolved since the article was written! then state how it was resolved.

Note: If you ha$e any criti/ues in this section, they most li"ely !elong in the 4eneral :riti/ue section instead. Grading: 3 items up to = points total ,8 points plus 3 points e.tra credit/ - for each item: great - 3" ok - #" poor - $

!." &ssues 9in 0our opinion:

Dist se$eral o#en /uestions or issues which remain unresol$ed in your o#inion* <or e,am#le, what #ossi!le future research /uestions could arise from this article* 1iscuss each in a se#arate #aragra#h of &-1H sentences. 9ach issue0s #aragra#h should ta"e the following format: what is the issueE why do you believe this is an important issueE in what way is it unresolved suggestions for resolving it Note: If you ha$e any criti/ues in this section, they most li"ely !elong in the 4eneral :riti/ue section instead. Grading: ) items up to $# points total ,8 points plus 8 points e.tra credit/ - for each item: great - 3" ok - #" poor - $

!1" &mpact
To determine how much im#act this article has had, do a citation analysis. 1iscuss what this citation analysis shows, and whyI don0t 5ust list the citations6 '3ee the :itation +nalysis 4uidelines '.doc( and )andout '.#df( #osted on the course %e! site.( If the article has no citations, then write in your re$iew 8I found no citations in the 3cience :itation Inde,, the 3ocial 3ciences :itation Inde, or on the Internet.8 Then clearly e,#lain why you !elie$e there were no citations at all. If you found citations in some inde,es or on the Internet !ut not the others, then e,#lain this as well. Include your citation lists in an a##endi, to your re$iew 'see !elow for details(. Grading - impact discussion: great - 3" ok - #" poor - $

!2" ;uestions
Dist three insightful /uestions of your own, arising from this article. 1o not as" definitions, !ut rather /uestions that really ma"e one thin". Grading: 3 7uestions, up to 8 points total - for each 7uestion: great&ok - #" poor $

!3" Annotated #i$liograph0

<or e$ery item you ha$e cited in your re#ort, you need a full reference and an annotation e,#laining it. This includes references to any class materials, as well as the three additional citations utilized in sections 6-14. 1. Hist the full bibliographic references authors! title! 7ournal name! volume! issue! year! page numbers! etc." for anything you have cited in your review. IM5>&+9,+; +his is NOT the bibliography listed at the end of the article. It is the bibliographic references for any readings you yourself referred to inside your review. 2. #rite 20) sentences describing the article. %. #rite 20% sentences describing why you cited it. +lso, !e sure that you ha$e included a !i!liogra#hic mar"er to each 'such as @?ie!er J 3mith, 7HH1B( in the te,t of your re$iew. Grading: -+ if missing references" -3 if you mention the authors e.plicitly in your te.t and put the references in this bibliography section, but forget to e.plicitly place citation markers in your te.t.

!4" ,itation Anal0sis Appendi<

There is a se#arate #age on the course %e! site descri!ing citation analysis. This a##endi, will ha$e three sections: the citations you found in the 6cience ?itation Inde1 the citations you found in the 6ocial 6ciences ?itation Inde1 the citations you found through a thorough #eb search on the Internet If the article has no citations for any of these three, then write in that section 8I found no citations in the @3cience :itation Inde, or the 3ocial 3ciences :itation Inde, or on the InternetB.8 Note, if your article has more than 7H citations, you only need to include a selection of them: 6tate how many citations each inde1 has and the #eb search found Hist 102 citations for each year in which the article has been cited. +ry to include citations from several different 7ournals spread over your selection E Include a citation analysis to see who has cited it and how. Grading - citation analysis: # points for each of the regular inde.es, and # points for the -nternet search" if the article is from $==* or later then: $ point for each of the regular inde.es, ) points for the -nternet search.