The Cashless Debit Card Evaluation: does it really prove success?

THE CASHLESS DEBIT CARD E V A L U AT I O N : D O E S I T R E A L LY PROVE SUCCESS? J HUNT Centre for Aboriginal Economic Policy Research ANU College of Arts & Social Sciences CAEPR TOPICAL ISSUE NO. 2/2017 Series Note The Centre for Aboriginal Economic Policy Research (CAEPR) undertakes high-quality, independent research to further the social and economic development and empowerment of Indigenous people throughout Australia. For more than 25 years, CAEPR has aimed to combine academic and teaching excellence on Indigenous economic and social development and public policy with realism, objectivity and relevance. CAEPR maintains a substantial publications program, including Research Monographs, Discussion Papers, Working Papers and Topical Issues. All CAEPR Working Papers are refereed and are produced for rapid distribution to enable widespread discussion and comment. All CAEPR publications are available in electronic format for free download from CAEPR’s website: caepr.anu.edu.au CAEPR is located within the Research School of Social Sciences in the College of Arts & Social Sciences at the Australian National University (ANU). The Centre is funded from a range of sources, including ANU, the Australian Research Council, industry and philanthropic partners, and Australian state and territory governments. As with all CAEPR publications, the views expressed in this Topical Issue Paper are those of the author(s) and do not reflect any official CAEPR position. Dr RG (Jerry) Schwab Director, CAEPR Research School of Social Sciences College of Arts & Social Sciences The Australian National University September 2017 caepr.anu.edu.au The Cashless Debit Card evaluation: does it really prove success? J Hunt Janet Hunt is Deputy Director and Senior Fellow at the Centre for Aboriginal Economic Policy Research, Research School of Social Sciences, College of Arts & Social Sciences, Australian National University. Topical Issue No. 2/2017 An electronic publication downloaded from <caepr.anu.edu.au>. For a complete list of CAEPR Topical Issues, see <caepr.anu.edu.au/publications/ topical.php>. Centre for Aboriginal Economic Policy Research Research School of Social Sciences College of Arts & Social Sciences The Australian National University Front cover image: Terry Ngamandarra Wilson, Gulach (detail), painting on bark, private collection © Terry Ngamandarra, licensed by Viscopy, 2016 Topical Issue No. 2/2017 iii Centre for Aboriginal Economic Policy Research Acknowledgments I am grateful to Rob Bray and Francis Markham for assistance in preparing this analysis of the Orima Wave 2 report. However, all responsibility for the accuracy of information in it and for the analysis remains mine. I would also like to thank an anonymous reviewer for valuable comments on an earlier draft. Acronyms iv Hunt ANU The Australian National University CAEPR Centre for Aboriginal Economic Policy Research CDCT Cashless Debit Card trial caepr.anu.edu.au Contents Series Note ii Acknowledgments iv Acronyms iv Introduction 1 Assessing the report’s findings 1 Interpretation of the findings 2 Notes 3 References 3 Figures Fig. 1. Reported assaults in Kununurra and Wyndham, Western Australia, 2010–17 2 Topical Issue No. 2/2017 v Centre for Aboriginal Economic Policy Research vi Hunt caepr.anu.edu.au Introduction T he evaluation report on the Cashless Debit Card trial (CDCT) in Ceduna and the East Kimberley (Orima Research 2017) was recently released with much fanfare. The Minister for Human Services, Alan Tudge, claimed the trial a huge success, and the Prime Minister was in Western Australia on 3 September, saying with great conviction: It’s seen a massive reduction in alcohol abuse, in drug abuse, in domestic violence, in violence generally; a really huge improvement in the quality of life, not just for the families who are using the Cashless Welfare Card, but for the whole community. But above all, above all it’s an investment in the future 1 of the children. Someone needs to tell them that the report does not say that. Indeed, the authors qualify a number of their apparently positive findings with various caveats, but, at the same time, the evaluation itself has serious flaws, so even these findings are contestable. Despite this, the trials are continuing, and new rollouts of the Cashless Debit Card are proposed elsewhere. Social policy analyst Eva Cox has highlighted many of the problems with the survey design, the way interviews were conducted and the ethics of the process (Cox 2017), all of which suggest that the results presented should be treated with great scepticism. Her criticisms of the evaluation process are valid, but what, if anything, can be drawn from the data that are presented, flawed as they are? And what about what isn’t presented? Is there any evidence that this trial is achieving its stated objectives? Assessing the report’s findings People interviewed for the evaluation reported that they drank less than before the trial began. However, such recall over a year is not likely to be very reliable. And, because respondents had to give their ID to the interviewer, they may have said exactly what they thought the interviewer wanted to hear, and certainly would not have incriminated themselves about any behaviours. Participant reports of change in the community may be more accurate than their reports of change in their personal alcohol use, but, in this case, the results are very mixed. For example, according to the report, in the East Kimberley, 20% of respondents said there has been more drinking and 18% said there has been less. In Ceduna, 14% said more, 23% said less, but 25% couldn’t say. The largest proportion in each site said that the level of drinking was the same. It is very unclear why there is such enormous variation in these views, and this is not investigated further – all of which means it is hard to draw conclusions. And no data on alcohol sales in the trial sites are provided to supplement participant and other reports. Overall, the data on alcohol use raise many more questions than they answer. The report also suggests that there is now reduced gambling in the trial sites; however, a number of qualifications to this in the report were completely ignored by the Minister and the Prime Minister. These include that a reduction in gambling was not the case in the East Kimberley, where both participants and nonparticipants2 in the trial were more likely to say that they thought gambling had increased. For Ceduna, a 12% reduction in gambling revenue over a year in a much broader region (Ceduna itself hosts only 40 of 143 poker machines covered by these data) could be due to the CDCT, but equally could be due to other factors across the region. The evaluation report does not investigate this further, so one cannot draw the conclusion that the CDCT has led to reductions in gambling in either location. The data about illegal drug use are likely to be the least reliable of all. Importantly, the results may be considerably affected by the publicity about drug testing of welfare recipients, particularly just before the Ceduna interviews in May 2017. So, the report’s claim that people reported a drop in illegal drug use must be treated with great scepticism; in addition, the small number of respondents on this issue means that the data reliability is low. The theory behind the trial was that, if drinking, drugs and gambling decreased, violence would decrease and people would feel safer. The report acknowledges that that there was ‘no statistically significant change’ in people’s feelings of safety, and concerns for safety at night remained, particularly in the East Kimberley. Finally, and perhaps most concerning of all, are the data that the report ignored. Assault incidence reports from the Western Australia Police rose sharply around the time the CDCT began in the East Kimberley in mid-2016 (Fig. 1).3 This is consistent with reports by East Kimberley CDCT participants of an increase in violence since the trial began, and certainly should have been investigated by evaluators. Was there something specific happening in Kununurra that led to this increase? Did the CDCT have anything to do with it or not? Although in Wyndham the increase in assaults since late 2015 and early 2016 is far less, the trend is still slightly up. The Kununurra data illustrate that the CDCT does not seem to have solved the violence problem there, whatever its cause. Topical Issue No. 2/2017 1 Centre for Aboriginal Economic Policy Research Interpretation of the findings So how might we interpret these findings? First, perhaps, despite all the flaws in the evaluation, there has actually been positive change on the ground in relation to the three behaviours targeted (alcohol consumption, gambling and illegal drug use). If that is the case, these changes do not appear to have affected the key harms that the program was supposed to address, namely safety and violence. The other possibility is that the program is not reducing the alcohol, drug and gambling behaviours it was meant to target. This could be because people are finding ways around the constraints of the card, or because the problems require far more than a card to solve. In either case, the program is not working, and the theory of change needs revisiting. The complex and interrelated problems of drug and alcohol abuse, poverty, unemployment, poor or overcrowded housing, and violence do need solutions that will improve the overall wellbeing of adults and children. These solutions are likely to be multifaceted and undertaken with strong engagement of the people whose lives they are meant to improve, but should not be imposed in a punitive way. Senator Patrick Dodson has called the trial ‘a public whip’ (Wahlquist 2017), and one of the trial’s early influential Kimberley advocates is now saying it is not working (Davey 2017). This is a costly program for taxpayers to support (costing up to $18.9 million, excluding GST, according to Conifer [2017]) if we cannot be confident that it is making a significant contribution to improved outcomes. There is also an opportunity cost for Indigenous communities because this funding may achieve better outcomes if spent in other ways. F I G . 1. Reported assaults in Kununurra and Wyndham, Western Australia, 2010–17 80 70 Assaults per month 60 50 40 30 20 10 0 2010 2011 Kununurra 2012 2013 Wyndham 2014 2015 12-month rolling average Note: The vertical black line indicates 1 June, when the rollout of the CDCT in the East Kimberley was almost complete. Source: https://www.police.wa.gov.au/Crime/Crime-Statistics-Portal 2 Hunt 2016 2017 caepr.anu.edu.au Notes References 1. https://www.malcolmturnbull.com.au/media/address-tothe-wa-liberal-party-state-conference-3-september-2017 Conifer D (2017). Centrelink cashless welfare card trial costing taxpayers $10,000 per participant. ABC News, 2 May, www.abc.net.au/news/2017-05-02/ cashless-welfare-trial-costing-taxpayers-$10kper-participant/8488268. 2. The nonparticipant result was not statistically significant, however. 3. These data need to be treated with caution because there may have been a major change in policing behaviour in Kununurra that contributed to the sharp rise in assault incidence reports. Cox E (2017). Much of the data used to justify the welfare card is flawed. The Guardian, 7 September, https://www.theguardian.com/ commentisfree/2017/sep/07/much-of-thedata-used-to-justify-the-welfare-card-isflawed?CMP=share_btn_link. Davey M (2017). Aboriginal leader withdraws support for cashless welfare card and says he feels used. The Guardian, 23 August, https://www. theguardian.com/australia-news/2017/aug/23/ aboriginal-leader-withdraws-support-forcashless-welfare-card-and-says-he-feels-used. Orima Research (2017). Evaluation report on the Cashless Debit Card trial evaluation: final evaluation report, report for the Australian Government Department of Human Services, Canberra. Wahlquist C (2017). Pat Dodson says cashless welfare card a ‘public whip’ to control Indigenous people. The Guardian, 22 August, https://www. theguardian.com/australia-news/2017/aug/22/ pat-dodson-says-cashless-welfare-card-apublic-whip-to-control-indigenous-people. Topical Issue No. 2/2017 3