Modelling or muddling? Economic analysis of the PNG LNG Project

Remnants of the former Panguna mine buildings, PNG (madlemurs/Flickr/CC BY-NC-ND 2.0)
Remnants of the former Panguna mine buildings, PNG (madlemurs/Flickr/CC BY-NC-ND 2.0)
Written by Mark McGillivray

Double or nothing: the broken economic promises of the PNG LNG Project’ is an April 2018 report by the Australia Jubilee Research Centre. It was authored by Paul Flanagan and Luke Fletcher.

It critiques the 2008 ACIL-Tasman macroeconomic modelling of the Exxon-led PNG LNG project. As the report notes, this project since 2014 has shipped approximately 7.9 million tonnes of natural gas per year from the gas fields of the PNG Hela region, with this gas being liquefied at a plant close to Port Moresby and shipped to buyers in Asia.

A key prediction of the ACIL-Tasman modelling was that the LNG project would double PNG’s GDP. Other predictions regarding the impacts of the project in PNG included an 84 percent increase in household incomes, 42 percent increase in employment and an 85 percent increase in government expenditure.

‘Double or Nothing’ states that ‘the proponents [of the project] positioned it as a major transformational project for the PNG economy, based around the central claim of a doubling of GDP’ and that ‘caveats around assumptions [of the modelling] were lost’ (p. 5). It is true that some proponents did this, and that the claim influenced the approval of the project in 2009 and found its way into the 2011/12 PNG election campaign. But not all proponents did. Many were skeptical of the ACIL-Tasman modelling, although believing that with appropriate institutional and policy settings the project had the potential to transform the PNG economy and contribute to many positive economic and social benefits.

The report argues that these and other forecasts of beneficial impacts were wrong, spectacularly so in many cases. Some of its findings, alongside the predictions of the ACIL-Tasman modelling, are set out in the following table.

ACIL-Tasman model prediction Double or nothing finding
A doubling of (100 percent % in) GDP An increase of 10%.
An 84% increase in household incomes A fall of 6%;
A 42% increase in employment A fall of 27%;
An 85% increase in government expenditure A fall of 32%
A 58% increase in imports A fall of 73%.

The ‘Double or Nothing’ findings are based on its own economic modelling. This modelling compares actual economic outcomes to those that, according to the report, would likely have been observed in the absence of the LNG project. They are not comparisons of ACIL-Tasman modelled predictions against actual outcomes, but those predicted by the ‘Double or Nothing’ modelling against these outcomes. The comparisons in the above table are of one set of predictions against another.

How valid or empirically robust are the ‘Double or Nothing’ predictions? The predictions are based on an assumption, which is that in the absence of the LNG project, the PNG economy would have continued to grow at the rate it did between about 2005 and 2009. Would the economy have grown at this rate in the absence of the project?

If the answer to this question is ‘no’, then the ‘Double or Nothing’ conclusions are wrong. It is remarkably silent on the importance of this key assumption to its findings, with no real attempt to credibly justify or evaluate it.

Without such a justification, ‘Double or Nothing’ unfortunately adds little to understanding of the impacts of the PNG LNG project, and its findings must be taken with a pinch of salt.

Here is to hoping that, one day, there will be empirically robust and credible modelling of the impacts of the PNG LNG project so as to truly inform policy choices that can benefit the people of PNG.

image_pdfDownload pdf

Mark McGillivray

Mark McGillivray is a Research Professor of international development at Deakin University's Alfred Deakin Institute. He works on aid effectiveness and allocation and measures of achieved well-being.

9 Comments

  • In response to Vailala, thank you so much for this additional information. I was not aware of these formal requirements, so I appreciate the information. Two broad comments. First, you mention the earlier Gas to Australia analysis down by ACIL-Tasman (which was done in 2002) as well as the 2008 ACIL-Tasman analysis on the PNG LNG project. The differences in these forms of analysis are remarkable. The first analysis did not include the complex modelling projections of the 2008 analysis – it focused on direct benefit flows as well as expected flow-over effects. There was much less spruiking of unrealistic macro-economic claims we found in the second report.
    This is in fact an important reason why examining the claims of the 2008 ACIL-Tasman report are so important. PNG is entering into important contract negotiations of some major potential future projects. The key lesson is do not trust reports from commercial project proponents. Prepare your own independent analysis. Bring in outside experts. PNG should not once again be mis-led again by such unrealistic claims.

    • Thank you Paul F for your comments. I must apologise for being rather too brief in my comment and I hope these additional comments prove helpful.

      The first Acil report (Gas to Australia) faced quite simple modelling tasks for a dry (specification) gas supply to a Townsville baseload and a potential expansion into a then immature Australian hubbing system. The second Acil report (LNG Project) faced much more complicated issues and found its answers from somewhere beyond the planet Mars. In mitigation it must be said that this second report was prepared at a time when ‘excessive optimism’ seemed normal, i.e. just before the GFC. As a report commissioned to meet a PNG legislative requirement it was, of course, a public document.

      It should also be noted that ExxonMobil have since the 1990s emphatically insisted that the returns from a gas project will be small, very small.

      As far as PNG is concerned the die was cast more than 20 years ago following the realization that PNG had little oil but abundant supplies of stranded gas. The ‘going for gas’ scenario was progressively developed over several decades as a scheme for the rational development of a PNG petroleum industry. The PNG government expended quite large sums of money in developing this scenario involving geotechnical surveys, legislative development and institutional strengthening and training schemes. Insofar as GoPNG and developer fiscal arrangements are concerned about 97% of the content of these terms can be found in the Oil and Gas Act and a number of other Acts, including, of course, the arrangements for taxation. This means that most of the negotiations of relevance for future resource development contracts arise in the context of the bankability studies. Apart from the staging of equity calls and GoPNG assurances that these will be met the most important item is likely to be the fiscal stability contract.

      For more insight into the LNG Project you need to fully comprehend the core substance of the unincorporated joint venture agreement. Under this scheme GoPNG as licence equity holder and co-venturer participates pro rata in the debt, risk, costs and income streams of the LNG Project as do other concession holder co-venturers and the operator ExxonMobil. As operator ExxonMobil must justify all operational costs to the co-venturers and, of course, acquit all sales income to the respective licence shares. It should also be emphasised that PNG has now accumulated many years of experience as a joint venture participant.

      Detailed scrutiny is made of all aspects of the LNG Project at the time of bankability studies. Independent studies are commissioned for this purpose. Lender party scrutiny is intense so as to eliminate ‘optimism bias’. Country risk analysis is also added at this point. Risk is assessed as a number, 0-6. Nigeria and Yemen rate 6, PNG 5, Indonesia 4, Quatar 3. This number plays into the financial calculations and any lender counter-party lay-offs.

      The ‘jewel in the crown’ of the LNG Project is that it will pay a cash benefit to many thousands of entitled landowner beneficiaries over many years. If these payments are made down to household level we can confidently expect an improvement in the welfare of women and children. Instead these very deserving people face the barriers erected by anthropological consultants in the form of ‘zone ILGs’, ‘umbrella sharing groups’, metaphors of Huli land tenure as a hotel and the benefit sharing process as the ‘cutting of the pig’. Fantasies all guaranteed to produce prebendarism.

      Unanticipated by the Acil report are the post 2011 development of the US shale gas play, the impact of this on Henry Hub prices and the rise of POP contracts, etc. These developments have in turn impacted on Asian gas prices and given rise to developments such as Asian price hubbing and a marked fall in LNG prices since 2014. There is a likelihood that there will be a long term trend of further downward pressure on LNG prices.

      Did the Acil report cause PNG’s current budget crisis? Or is it just another budget episode?

      Vailala

      For more information on how the PNG LNG Project sits within the global context see –

      Sophia Ruester “Financing LNG Projects and the Role of Long-Term Sales-and-Purchase Agreements”, Deutsches Institut für Wirtschaftsforschung, Berlin 2015.

    • I should have added the following lines to the end of the comment: ‘Doing so also can create unrealistic expectations of local benefits that can also have destabilising implications. Going well beyond models and their limitations, the Jubilee Australia analysis also indicates the crucial importance of policy responses to any major project. PNG should learn the lessons that poor policy responses induced by major resource projects can undermine promised benefits. More broadly, the report raises important questions about PNG’s development path and its experiences with a repeat of the resource curse. Pretty valuable lessons from that “dead sheep”.’

  • In response to Mark, the “Double or Nothing” report did a sensitivity analysis along the lines he suggests. It is summarised on page 17 of the report. The results of two other growth trend paths were considered (in addition to the 4.8% growth path assumed for GDP). The first was a 3% growth trend path (which should be the minimum basis for a country with a 3% population growth rate ie no growth per capita). This indicated there was still a 71 percentage point overestimate of GDP growth in the ACIL-Tasman modelling. This supported the robustness of the report’s conclusions of a massive over-estimation of the benefits from the project. The second trend path examined was what would have been required to generate the extraordinary ACIL-Tasman prediction of a 97% benefit gain relative to 2016 actual GDP. The answer was around minus 3% for every year – so the ACIL-Tasman model would have needed to assume PNG would otherwise have been in a very serious recession every year for over 8 years. This is clearly absurd. So the report did contain the key results of the sensitivity which was done on the underlying trends. Actually, the sensitivity analysis modeled -3%, 0%, 3%, 4%, 5%,6%,7%, 8% and 9% underlying growth assumptions. During the peer review process in early April (such reports go through an extensive peer review process prior to publication, including with PNG contacts and other economists), the view was formed that such analysis was too detailed both for reasons of length and excessive technical detail. Figures above 4.8% simply made the ACIL-Tasman report even more unrealistic and did not need to be included to support the report’s numerical conclusions. The 3% and negative 3% were considered the most relevant points for a policy discussion. So the sensitivity analysis was summarised and still included on page 17 of the report. This still provided enough information to confirm the robustness of the report’s conclusions – that within any reasonable assumptions of an underlying growth path, the ACIL-Tasman report was hopelessly optimistic and inaccurate. The original detailed analysis will be placed on the pngeconomics.org website shortly. With the key results of a sensitivity analysis included, I’m not sure what issues Mark has with the report. Any more detailed modelling of potential project impacts inevitably relies on assumptions – the underlying hard data for PNG has not been updated for decades. Predictions are about assumptions – and one needs to be transparent about these, provide a basis for these (a credible trend analysis providing official sources and time periods for this and reasons why the trend analysis could not cover a longer period reliably), and include a sensitivity analysis. The Jubilee Australia report did this while the ACIL-Tasman report did not. So the column on the left is from a black box with all the numbers assuming a negative growth path of on-going recessions over 8 or 9 years to be accurate. The column on the right comes from a transparent report including a summarised sensitivity analysis and based on reasonable projections for most developing economies of a sustainable growth path somewhere between 3 and 7%. The figures on the right are unquestionably much more credible than those on the left. I’m surprised that Mark does not acknowledge this. The issue is not about whether they are perfectly accurate – economics is not that exact a science. However, surely the Jubilee Australia figures are much, much more credible? The “Double or Nothing”report conclusions are robust, certainly much more so that the ACIL-Tasman figures.

  • Thanks to Vailala for an incisive and useful set of comments.

    Vailala makes a very good point, that to now attack the ACIL-Tasman report on the basis of its shortcomings seems to be like “savaging a long dead sheep”.

    I fully agree, and alluded to this in my original post, when I referred to many being skeptical of the ACIL-Tasman modelling when it was originally released.

    To conclude some 12 years later that its projections were wrong doesn’t add much, I respectfully suggest.

    What we should be focussing on is how to better model the PNG economy and the impacts of projects and policy stance within it, the LNG project in particular. This was the most fundamental point of my post, and let me again state that here is to hoping that this happen.

    Mark

  • It may be worthwhile to add a little background to the discussion of the ‘Double or Nothing’ report.

    The 2006/7 Acil report is a revised version of a report prepared for the Gas to Australia project. This earlier report and the report under discussion were prepared to meet legislative requirements for reports of this character.

    The Organic Law on Provincial Governments and Local-level Governments has relevant provisions for a ‘cost and benefit analysis’.

    S 115 deals with the control of natural resources and S 116 Specifies the Resource Development Process and the responsibility of the National Economic and Fiscal Commission –

    S 116 (2) The National Economic and Fiscal Commission shall carry out a cost and benefit analysis for and in relation to the development of natural resources.

    The Oil and Gas Act S 174. refers to this Organic Law provision –

    (1) The total benefits granted in accordance with this Act to project area landowners and
    affected Local-level Governments and affected Provincial Governments and any other persons or
    organizations shall not, when added to other costs incurred by the State in the course of the
    development or operation of a petroleum project, exceed 20% of the total net benefit to the State from that petroleum project as determined in a cost-benefit analysis under Section 116 of the Organic Law on Provincial Governments and Local-level Governments.
    (2) No commitment shall be made by the State pursuant to Section 171 or 173 unless the
    Minister is satisfied that, following the provision of any benefits or grants which might be agreed under those sections, Subsection (1) is complied with.

    The PNG National Economic and Fiscal Commission has a general power to make use of documents prepared by other persons.

    In common with many others I attached little significance to the Acil report for a number of reasons. These included the estimation of oil price futures over an unsustainably long time, the incorporation of the 3 price case scenarios within the broader economic analysis, the very ‘rubbery’ modelling of the PNG economy and a susceptibility to the common CBA problem of ‘optimism bias’.

    It may well be that the main purpose for having the Acil study done was to put a tick into the box on the ‘to do list’. To expect that the Acil report could have aptly modelled the changes in the LNG markets for wet gas between 2006 and 2018 is, I think, unrealistic. To now attack the report on the basis of its shortcomings seems to me to be like savaging a long dead sheep.

    Vailala

  • This is a quick response to the comments of Paul and Terrence.

    I have read the entire report, in particular page 17, footnote 22 and the appendices.

    My fundamental point remains.

    It needs to be made very clear to the reader, especially those without modelling expertise, that if the report’s assumed growth path is wrong then the results it reports will be wrong. This includes the results in Table 1.

    Related to this, it also needs to be made clearer that the baseline referred to in the third column of Table 1 is not an actual baseline, but the assumed growth path. Many readers would not be aware of this, I would suggest.

    On the growth path itself, what ideally needs to be done is to credibly model the growth path that would prevailed in the absence of the LNG project, and not rely on an assumed path.

    But I acknowledge that this is no easy task, and data availability might rule it out.

    In these situations, what is often done (and needs to be done), is to provide a number of growth paths, such as high, low and expected on the basis of available evidence, and report results that correspond to each.

    This is not, as Terrence might suggest, a case of needing more data. It is simply a matter of spending more time with existing data, and being very clear about the caveats that apply to results that are reported.

    Such an approach is consistent with the report’s call (on page 40), for a code of conduct for modellers, which I fully support.

  • Lending a vote of support to Paul here. He did a valuable job, in a context that is hardly data rich. As he says, there’s also the appendices if you want more details.

    It would be great if there was more detail again still, but at some point that task has to fall to governments, who have the information and resources.

    Thanks Paul

  • A quick initial response as I hope that I’ll be allowed to give a more detailed response in a future blog. First, we are in furious agreement that there is need for better modelling of the PNG economy – it was in fact a recommendation of the Jubilee Australia report. Second, it is not clear that Mark has read the full report. The issue of underlying growth trends was recognised as the most challenging issue in the report – see lead question in Appendix 2. The report did include key outcomes from a sensitivity analysis of alternative trend assumptions (page 17) indicating alternative realistic figures of underlying growth would not change the report’s conclusions. A blog will go into more details why the conclusions of the JA report should not be treated with a “pinch of salt” – there are fundamental issues about PNG’s future path of development. Cheers. Paul

Leave a Comment