305-0007-SYW

Q.1.

“Does Monitoring deter future cheating? The case of external examiners in Italian Schools”

In this paper the authors propose to study the effect of monitoring school exams on cheating. We will
start by analyzing the research approach and methodology of the authors, followed by the data, and later
results.

Methodology:

The paper proposes to study the effect monitoring has on cheating propension by analyzing the effect of
said monitoring on test exam results, standard deviation, and a specific cheating index: INVALSI. The
dependent variable is indexed by school and region. To assess whether monitoring has a lagged effect,
the regression also includes the regressor of interest (monitoring) lagged for two previous periods. The
authors then add to the regression certain controls for school size and region, as well as a vector of
characteristics X(irt). The authors note that the error term is clustered by school.

It should be noted that the vector X, which controls for characteristics (and it is lagged), is very broadly
and dubiously defined. If it is related to a whole school, for which we do not have information about
(average size, number of classes and students), it might be incorrect to use, given that only 1 to 2 classes,
per school, are externally monitored, we could be forcing school characteristics on a single class.
Furthermore, by allowing the characteristics to be lagged we might be mudding inference: “Past”
characteristics of students/school should not be affecting the current students results, other than trough
current characteristics (which are probably “sticky”/correlated with past ones).

Regarding other fixed effects present: the region by year dummies and its interaction terms are discussed
very briefly, with little information or reasoning presented. Such a topic should have been expanded on,
particularly knowing that Italian regions differ greatly, as the authors state, in terms of human capital
(north vs south). A more in-depth analysis of within region variation would have been interesting.

The authors also do not discuss the occurrence of a string of external monitoring, in which monitoring
could happen in t, t-1 and t-2, and what would be the impacts when this effects overlap.

Data:

The authors present the following information: they have 22,984 observations, out of 6,790 institutes. So
the authors have a sample of 3,3 classes per institute. This seems a relative low number, given that 1 to 2
classes per institute are externally monitored, that would leave 1,3 to 2,3 classes as the “control” group
for comparison. So, even though the number of groups is large, the elements per group is extremely low.
The authors will cluster results per institute, as well as the error term. Given that there is such a small
sample to compare, questions are raised about the validity of this method.

It is not mentioned what type of monitoring other classes receive, and argued why such a monitoring is
substantially different.

It is referred that the probability of a school being selected to be monitored is based on its number of
students. As such we can be analyzing schools and results with different characteristics, if we assume
schools of different sizes have different student populations (per example rural schools are normally
smaller than urban schools and can present different characteristics in teaching/students/amenities).

1
305-0007-SYW

Results:

As mentioned before the small number of elements per cluster has the possibility to impact results, given
that error term and standard deviations are calculated at cluster (school) level.

The distinction between student pressure and cheating is never made or addressed. The authors do not
mention how being externally examined might negatively impact student performance. This is partially
addressed by utilizing the Cheating index (although such is never explained).

We can see that the results in t-1 are less significant than in t, and assuming the error clustering to not be
entirely correct, we could have insignificant results. If such would happen, the argument for the effects
on t being owed simply to student pressure would gain significance.

Q.2.

“information presentation and firm response: Evidence from fertility clinics”

In this paper the author proposes to study how information disclosure about a service can change and
impact firm behavior. We will start by analyzing the research approach and methodology of the authors,
followed by the data, and later results.

Methodology:

In this case information regarding birth clinics, in 2003, started being published differently, with more
relevance being given to a certain index. The author suspects the change in information would lead clinics
to behave in a way which would increase their perceived quality in regard to this index. The author is
particularly interest in the difference responses of firms which face competition or are monopolies. As
such the regressor of interest in Competition, measure as a dummy, with a value of 1 if the firm faces
competition.

The author then uses a Dif-in-Dif method, with competition as the main regressor, and After2003 as the
treatment.

The key assumption in a DiD is the parallel trends assumption, that both the control and treatment group
have the same trend, pre-treatment, and would have the same post-treatment, was it not for treatment.
So, we would need a population which we can subdivide in treatment and control, on the sole fact of
being treated or not. The author, however, never proves such. Since the information disclosed is
applicable to every clinic, there is no control group to speak off after 2003, since all clinics are in treatment.
The author argues that the control and treatment group are defined by being a monopoly or being a
competitive firm, and that they have parallel trends. Given that being a monopoly or a competitive firm
is a clinic characteristic, rather than a specific state dictated by treatment, we would reject the author
claim. Furthermore, both monopolies and non-monopolies were subject to treatment (information
disclosure), so no control group remains after 2003.

The author never discusses the error term, which in this case can be of concern, given that some
correlation between clinic location and dependable variable is expected (state trends and serial

2
305-0007-SYW

correlation). Since we have such wide range of clinic location, clustering by state or region could have
increase inference. Also, no controls mentioned for fertility and or birth rate.

Data:

A valid critique to data would be the exclusion of Metropolitan Areas, which is not fully argued in the
paper. By removing Large cities and urban areas we are sub-sampling data, creating a sample bias, in this
case our analysis will be based in clinics on rural or small urban communities, which characteristics can
differ significantly to the overall population, from socio-economic factors to race, etc… So, by excluding
Metropolitan areas a section of population is being ignored. This event is even more pernicious when
accounting that developed countries have a high percentage of population living in Metropolitan and
urban areas (normally around 50-70%). So by excluding said areas the author is sampling a specific
population with specific characteristics which do not represent the whole population of interest.

External validity suffers from this fact, and it is dubious on whether we can claim it as a RCT.

Results:

As seen previously the key assumption of the model do not hold, and standard errors should be taken
with a grain of salt, as such, results are unreliable and not explanatory. They do present a difference in
attitudes between monopolies and competitive firms, but the model is wrongly built, and inference
dubious, based on that.