The recent headlines comparing HDI values for Papua New Guinea’s Western Province to those of Zimbabwe and the DRC have certainly caught people’s attention. The comparison has been made by Professor Mark McGillivray and is based on research that he presented at a recent conference at Deakin University. Unfortunately, the research is based on data that is mostly more than 10 years old so it cannot be expected to show us today’s realities.
Professor McGillivray has estimated values for a Human Development Index (HDI) at the Provincial and District level in PNG. This approach is well established and combines various measures of income, health and education into a single numerical index which can then be used for comparison. A composite measure like the HDI can only ever be as good as the data that is used to create it. In this case the data has been taken from the ‘Provincial Profiles’ [pdf] which was published by the PNG National Research Institute in 2010 and collates information at the Provincial and District level from a number of Government sources. Within this report the data on life expectancy used by Professor McGillivray dates back to the 2000 Census. The measure of education is a hybrid of the adult literacy rate measured in the 2000 Census and the School enrolment rate measured by the Department of Education in 2007.
The third component of the HDI is a measure of income. Provincial GDP data is not readily available for PNG so Professor McGillivray used national GDP measure and data from 2002 on the proportion of the population in each Province living within 5Km of a National road. These two measures were then combined to form a Provincial GDP measure. The rationale for this approach is that proximity to a national road is correlated with market access and opportunities for income generation. However, this ignores other modes of transport and other factors known to affect incomes in PNG such as household and environmental characteristics.
The district rankings produced by Professor McGillivray’s analysis differ significantly from the ranking produced in a similar analysis [pdf] carried out by the PNG National Economic and Fiscal Commission (NEFC) in 2004. The NEFC study used the same data from the 2000 census to measure health and education outcomes but used the incidence of poverty estimated in a World Bank poverty mapping study [pdf] for the income component of the index. The outcome was a ‘District Development Index’ that was explicitly based on the HDI methodology.
The scatter chart below shows the correlation between the rankings of each district in the two studies with lower rankings (bigger numbers) indicating lower levels of development:
Looking at the scatter plot we can see that the methodology used by Professor McGillivray produces a ranking that is substantially different to the ranking generated by the NEFC. This is confirmed by Spearman’s rank correlation coefficient which, at 0.53, is lower than one would expect from two indices that purport to measure the same thing using very similar data.
It is also relatively easy to see that, compared to the ranking produced by NEFC, Professor McGillivray’s methodology heavily penalises districts in Milne Bay and Western Provinces. The reason for this is almost certainly that these two Provinces have significantly lower rates of access to National Roads than other Provinces. For example, in 2002, the proportion of people in Western and Milne Bay living within 5km of a National Road was 17.2% and 21.1% respectively. In contrast the average rate for the remaining 17 provinces is almost 50%.
The weak correlation between the two indices and the dramatic shift in the rankings for districts in Western Province and Milne bay suggest that careful attention should be given to estimation of the income component of the HDI. On face value, the approach taken by NEFC is preferable since it is available at a much higher level of geographical disaggregation and takes account of a wider range of relevant factors. It also avoids findings of extremely low development in districts such as North Fly and Alotau which have significant economic activity but are within Provinces that do not have an extensive network of National roads.
Notwithstanding questions about the best methodology, the real issue ought to be the availability of good quality recent data. The PNG National Statistical Office (NSO) is currently working to process and analyse data from the 2010 Household Income and Expenditure Survey and the 2011 Census. This data will permit a wealth of analysis and will enable a clear assessment of PNG’s human development achievements over the past decade. Considering just how important this is, and how useful more frequent data collection would be, it’s surprising that the NSO is not given more support from the PNG Government or foreign donors. In a week of news stories based on old data, this story has gone untold.
David Freedman is an ODI Fellow and Economist at PNG Sustainable Development Program.
The research should not have been publicised in the public domain if it is preliminary. In any case current is more valid and should be published in public domain for the information of the public especially development agencies.
The danger here is that such publicity give wrong informaiton to people who don’t know the realities of Western Province.
Lydia
I was initially moved to comment on this research because of media coverage that lacked the many caveats and qualifications provided by Mark in his response. I’d like to thank Mark for acknowledging the preliminary nature of his research and carefully elucidating the range of issues that will need to be addressed to arrive at a better HDI measure.
It is certainly unfortunate that findings that “will undoubtedly be revised heavily before being used for further analysis” have received so much media attention. In addition to the coverage in the Australian media, the press release that was issued by Deakin University has been reprinted in both of the national newspapers in Papua New Guinea and has aroused significant comment in the press and online forums.
The reporting has focused on the apparent low levels of development in Western Province. As such I think it’s entirely appropriate to discuss other studies that have used similar data and have found far higher levels of development in Western Province. Notwithstanding the technical differences between the indices that were correctly identified by Mark, I maintain that it is of interest that the low level of access to National Roads in Western Province appears to be one of the main factors causing its low HDI score.
In due course new data and improved methodologies should help to give a far clearer picture of the progress in human development across PNG. In the meantime, I would politely suggest that Mark McGillivray can’t have his cake and eat it. If, as he says, “it is premature to compare the HDI scores presented at the conference with the work of the NEFC in 2004 or for that matter any other indicators of development levels in PNG”, then surely it is also premature to compare the preliminary HDI scores he computed with the values published for other countries by the UNDP?
I would like to end my comment by echoing the thanks from Matt. This is an important topic and the debate and public interest in measuring development that Mark has stimulated is very timely.
Thank Mark,
You have stimulated an interesting debate and obviously there are methodological issues. But isn’t the key issue that your conclusions drawn on Western Province’s level of development are misleading because they are based on old data? I think what David is trying to show using more recent data, albeit not with directly comparable methodologies, is that the situation in Western Province may not be as bad as the headlines from your report. The solution is to a) note that the headlines on Western Provinces relative level of development are based on old and perhaps weak data, b) look at more recent data to see if HDI is likely to have got better in the last decade, and c) to get NSO (and their donors) to publish better data to enable more rigorous analysis.
Matt
The issue of human development achievements in PNG and the disparities they reveal is an issue of fundamental importance. To this extent it is pleasing that David Freedman picks up on this in his comment, which is based on a powerpoint file used during a 15 minute presentation at the Deakin conference. Yet the comment reflects some key misunderstandings of the research in question, and misrepresents its purpose. I will quickly outline the grounds on which I make this statement.
As was made clear at the Deakin conference the research is a first attempt to derive internationally comparable sub-national HDI scores for PNG provinces and districts. These scores will be the subject of significant scrutiny and will undoubtedly be revised heavily before being used for further analysis. The estimates of income per capita, which are currently extremely crude, will be the subject of significant further work. It is obvious that there are determinants of income per capita in addition to roads, and that for some parts of PNG roads might not be especially important in this regard. The main point I make here is that given the very preliminary nature of the research in question, it is premature to compare the HDI scores presented at the conference with the work of the NEFC in 2004 or for that matter any other indicators of development levels in PNG.
David Freedman notes that the research is based on data that are mostly more than 10 years old, labeling this as unfortunate. The HDI scores presented at the conference will, after the further scrutiny mentioned above, serve as baseline for monitoring. They are in no way intended to show today’s realities, to the extent that quantitative measures can do this. (This point was made to David prior to the submission of his comment, but he has ignored it). The intent is to update these scores using those calculated using more recent information (including that mentioned by David in his comment), and then compare these newer scores with the baseline information.
David Freedman’s comparison of the HDI scores presented at the Deakin conference (putting aside that this is a misuse of these scores) with the NEFC’s District Development Index is reflects two key errors or misunderstandings.
First, the HDI scores are intended to be internationally comparable, so that achievements in human development in PNG can be compared with those in other developing countries. What this in effect involves is comparing the PNG sub-national HDI scores with those published in the UNDP Human Development Report. The latter use estimates of income per capita, not poverty incidence, which is a very different measure. In this context, the question of whether it is better to use the poverty incidence or an estimate of income per capita simply does not arise, and to ask it is quite misleading.
Second, the NEFC’s District Development Index and the HDI index presented at the Deakin conference use different methodologies. The former is formed by taking the arithmetic mean of achievements in health, education and poverty, the latter by taking the geometric mean of achievements in health, education and income. In other words, the former is calculated by adding together these achievements, and the latter by multiplying them by each other. Each index also uses different ways of normalizing these achievements. The HDI index presented at the Deakin conference is consistent with current UNDP HDI methodology, while the NEFC’s index follows an older methodology. This means that even if I used exactly the same achievements as the NEFC to form the index presented at the Deakin conference, the resulting indices would yield different rankings. David’s comparison of these rankings totally misses this point, and that his reasoning for the difference in rankings is flawed. Of course the difference will be due to one index using poverty incidence and the other using an estimate of per capita income, but it is also due to their different statistical properties.
I will end one a more positive note. David Freedman is totally correct in pointing to the need for good quality data. The NSO clearly requires more support to produce such data.