Version classiqueVersion mobile

Infrastructure Investment in Indonesia

 | 
Sally Wilson
, 
Colin Duffield
, 
Kevin Kin Peng Hui

6. Comparative Efficiency. Analysis of Australian and Indonesian Ports

Felix Kin Peng Hui, C.F. Duffield, Andrew Chin et Hanlong Huang

Texte intégral

6.0 Introduction

1Logistics is a critical element of a country’s trading ability and is central to the economic growth of the country, since it enables effective connection of trade through both domestic and international logistics networks. Due to the close geographic proximity of Australia and Indonesia, trade plays an important role in each country’s economy (DFAT 2019). In 2016, Australia came in 8th and 11th place as the principal import source and export destination for Indonesia respectively. Given the pivotal role of trading between the two countries, it is important to establish and analyse the efficiency of the major ports of the two countries. For this project, the Port of Melbourne was chosen as the focus for analysis as it is the largest container port in Australia. The Port of Surabaya was chosen as the comparison study port in Indonesia. The major ports in Jakarta (Port of Tanjung Priok), Sydney (Botany Bay) and Perth were also used in the comparative analysis.

2Port performance is extremely important for supporting the economy of the hinterland (Hung, Lu and Wang 2010; Lam and Yap 2016), which plays an important role in the logistics supply chain. The efficiency of the ports needs to be analysed and studied in order to improve the competitiveness of the port and terminal within the country and the region.

3The Logistics Performance Index (LPI) captures the assessment of the entire logistic performance through a series of inputs and outcomes of the logistics supply chain and environment (Ojala and Celebi 2015). These indicators help regulators define the areas for policy regulations as well as inputs for operational assessment, such as customs, infrastructure, services quality and other service delivery performance that deals with cost, time and reliability outcomes.

4Ports are an essential part of the logistic supply chain. In a similar manner, port efficiency can be measured through inputs and outputs, generally with a concept similar to the logistics performance index. Given the multiplicity of ports and types of cargoes handled, the choice of indicators for analyzing inputs and outputs, as well as their units of measurement, need to be carefully considered. The primary measures of the operational performance of ports are ship turnaround time; and crane handling rate in the port (Chung 1993). These measures are dependent on the port’s infrastructure, available resources, types of cargoes (bulk, container TEU) and logistical interfaces. Asset and financial performance are important inputs and outputs to measure the port efficiency as they reflect the berth throughput, berth utilisation rate, and rate of return and turnover. Hence these variables were used in the efficiency analysis in this research. Due to the different ownership status of different ports and terminals, many past studies such as Chen, Pateman and Sakalayen (2017), Tongzon and Heng (2005) and Yuen, Zhang and Cheung (2013) have provided different views on how ownership structure can influence their efficiency and competitiveness. Therefore, part of this study also discusses how ownership structure might affect the efficiency of Australian and Indonesian container terminals.

5The rest of this chapter provides a comparative analysis of the efficiency of Australian and Indonesian ports and terminals. The study also includes comparison with the Port of Shanghai, which is the largest container port in the world.

6.1 Literature Review

6.1.1 Logistics and Port Efficiency

6Based on a World Bank (2016) report on port efficiency, Germany is ranked first on overall efficiency in the world and Singapore is ranked first in the Asian region, in terms of overall efficiency (LPI indicator) (see Fig. 6.1).

Image 100000000000029E000001067D58D1BA75BE5127.jpg

Fig. 6.1 Overall efficiency of ports in Asian region (incl. Australia) on country basis

(Figure by the authors based on World Bank 2016 data)

7In the same report, the port infrastructure index showed that Australia and Indonesia are rated at 86 % and 61 % respectively, significantly behind Singapore the top ranked in the Asia region (see Fig. 6.2).

Image 100000000000029B000000D49EB902750C4304A7.jpg

Fig. 6.2 Relationship between infrastructure and overall efficiency on country basis

(Figure by the authors based on World Bank 2016 data)

8It is interesting to note that Data envelopment analysis (DEA) was used to compare the overall port efficiencies over the provision of infrastructure. There appears to be a linear correlation between the provision of infrastructure and the port efficiency index (see Fig. 6.2). Cullinane et al. (2006) also observed that infrastructure investments and provisions have an influence in port operational efficiencies. It is clear that both Australia and Indonesia have room for improvement in relation to worldwide best practices.

6.1.2 Indonesia

9The major international ports in Indonesia are located at Tanjung Priok, Jakarta and Tanjung Perak, Surabaya. These ports are close in distance to Australia and are central to Indonesia’s logistics system, providing a strategic gateway for trade to the hinterland in Indonesia. They also provide gateways for domestic trade connections to neighbouring islands and provinces. This is an important role considering that these are critical infrastructure for a country with a large population of 255.5 million (DFAT 2014). Inter-island shipping alone accounts for more than 60 % of the nation’s sea cargo activities (World Bank Group 2013).

6.1.2.1 Port of Surabaya

10The Port of Tanjung Perak is a major transportation hub in East Java and it serves as a gateway to the collection and distribution of goods around the country. Tanjung Perak contributes significantly to the economic development in the whole of Eastern Indonesia (Logistics Capacity Assessment 2017). It currently has an annual container throughput of 3.1 million TEU. Terminal Petikemas Surabaya (TPS) and Terminal Teluk Lamong (TTL) are two of the main terminals that handle containers and bulk in the port. TPS is 51 % owned by the Government State Owned Enterprise Pelindo III and 49 % by DP World, a large port operator based in the Middle-East.

11Recent research by Seo et al. (2012), using DEA analysis, indicated that Surabaya has a relatively low efficiency rating when compared with other ASEAN ports. However, other studies that compared terminal level efficiency, found TPS to be a relatively efficient terminal in Indonesia, in terms of container throughput and utilisation (Andenoworih 2010; Syafaaruddin 2015). TPS is well equipped with modern facilities as well as being well-connected to industrial parks by rail and roads. Syafaaruddin (2015) used DEA analysis to show that TPS has a high ratio of capacity utilisation based on technical inputs.

12Terminal Teluk Lamong (TTL) opened in 2015. It is owned by Pelindo III, a State Owned Enterprise and the first green port in Indonesia (Terminal Teluk Lamong 2015). It has world-leading infrastructure facilities and is also the first in Indonesia to implement “semi-automated equipment in yard services, automation gate-system and online transaction” (Terminal Teluk Lamong 2015). Recent studies by Rahmanto (2016) found that the Teluk Lamong Terminal is still a congested terminal with low port capacity issues. In recent years it has been considered to be one of the low performing ports of Surabaya due to its poor road infrastructure around the terminal. The principal difference between TPS and TTL is that TPS has private sector involvement (49 % owned by DP World). It would be interesting to see whether private sector involvement plays a role in port efficiency (between TPS and TTL), and whether high tech, advanced, green automotive infrastructures can improve port efficiencies.

6.1.2.2 Port of Jakarta

13The Port of Tanjung Priok (PTP) is the busiest port in Indonesia and is managed by PT Pelabuhan Indonesia Pelindo II, one of the four state-owned corporations that manage ports in Indonesia (IPC 2017). PTP is also home to the Jakarta International Container Terminal (JICT), a container terminal that is majority owned by Pelindo II under government control (Koperasi Pegawai Maritim) at 51 %, with the remaining 49 % under Hutchison Port Holdings.

14Andenoworih (2010) and Syafaaruddin (2015) both used DEA analysis and reported consistent results showing that the Port of Tanjung Priok and JICT have relatively good efficiency scores, ranking highly against other smaller ports. It was noted that that Port of Tanjung Priok and JICT are situated within the high regional economic activity areas of Indonesia. However, both studies noted that there are still bottleneck issues with congestion and low port capacity issues. On the other hand, Afriansyah et al. (2017) explained that the Port of Tanjung Priok has both current internal and external issues associated with the port, one being that operational efficiency of the port is low and the other being that the bad integration of the information system causes long dwelling time. Tanjung Priok has a dwelling time of 4.58 days, which was attributed to the less than ideal information management system. Hill (2014) found that Tanjung Priok is the only port in ASEAN having not provided importers with a priority lane. Overall operational efficiency is low, with slow customs handling causing congestion issues, consistent with the findings of Afriansyah et al. (2017). This shows dwell time is an important input that has been used in past analysis and is a useful parameter in our study.

6.1.3 Australia

6.1.3.1 Port of Melbourne

15The Port of Melbourne (POM) is the largest and busiest container and multi cargo port in Australia, with an annual container throughput of 2.64 million TEU (PR Newswire 2017). According to the Department of Foreign Affairs and Trade (2017), in the year 2014, Australia’s top exports are iron ore, coal, gas, wheat (bulk), while top imports are crude petroleum, motor vehicles (roll on-roll off).

16DP World and Patrick Terminals serve as stevedores for West and East Swanson Dock respectively. In 2016, it was announced that the Lonsdale Consortium had acquired the right to operate the Port of Melbourne for the next fifty years. The government believed that by doing this, they could simultaneously allow private sector involvement in the port and gain access to funds for the government budget, in the process receiving AUD 9.7 billion from leasing the commercial and management rights of the port.

17Ghadehi, Cahoon and Nguyen (2016) highlighted that the Port of Melbourne lacks an intermodal rail that may have allowed loading and unloading to occur outside the dock. This view is shared by Lubulwa, Malarz and Wang (2011), who reported that container haulage mode from terminals are 95 % trucks and 5 % rail, well below the 30 % target for rail in statistics obtained from 2010. The poor development of terminal rail infrastructure was reported to have caused inefficiency and congestion problems within the Port of Melbourne.

6.1.3.2 Port of Botany, Sydney

18The Port of Botany in Sydney is managed by the New South Wales Ports Consortium. It holds a ninety-nine-year lease to the state-owned assets of the port. It is the second major container port in Australia, ranking after Melbourne, with an annual container throughput of 2.28 million TEU and 4.7 million tonnes of bulk handled at the port. The Botany Port has three private stevedores that co-manage container terminal berths: Patricks, DP World and Hutchinsons, Port of Botany, Sydney, was reported to be and deemed efficient using the variable return DEA model, but inefficient under the constant return to scale assumption (according to analysis from Tongzon (2001)). Similar to Fremantle Port (see below), it was suggested that the port undertake structural and technical reform to raise the efficiency level. To further add to the scope of the project, we included Port Botany and its container terminals for a comparative analysis against Indonesian ports using DEA techniques.

6.1.3.3 Port of Fremantle

19Fremantle Port (Harbour) is the largest general cargo port of Western Australia and fourth largest container port of the country (Fremantle Ports 2017). It is strategically managed by Fremantle Ports, a Western Australia Government trading enterprise. Port of Fremantle is an important gateway from the western part of the country to the world, with annual container TEU’s of 0.72 million handled (Maritime Report 2016). Compared to the other international ports in this efficiency analysis, it is slightly smaller in scale in terms of land area as well as annual container throughput handled.

20Cheon, Dowall and Song (2010) highlight that Port of Fremantle is relatively inefficient when compared to other larger scale international ports. It was found that despite government restructuring of port ownership, efficiency was still not improved, given that ports of this size and scale should really focus on large scale port technical improvement rather than terminals structure that can improve their short-term scale efficiency. Another study by Tongzon (2001) stated that the port has a major slack in terminal area usage and labour input. The two studies strengthened the case that Fremantle Port may need strong and effective government reform to improve the technical areas of the port in order to achieve higher levels of trade volumes and efficiency.

6.1.4 China

2.1.4.1 Port of Shanghai

21The Port of Shanghai is a major international port in China and has the highest container throughput in the world, with 36.5 million TEU annually (World Shipping Council 2017). Shanghai International Port (Group) Company Limited (SIPG) is the sole operator of the public terminals of the port. Wu and Goh (2010) and Yuen, Zhang and Cheung (2013) both stated that it is a relatively efficient port in terms of ownership structure, hinterland size and container terminal efficiency performance. Their studies used DEA (both CCR and BCC models) to reduce any scale differences in the efficiencies, and the results obtained were consistent. Wu and Goh (2010) used indicators of supply chain factors as inputs to the DEA analysis. These include: customs clearance, review procedures, and import and export lead time. Privatisation of the port was also found to be beneficial and had some positive influences on port efficiency (Yuen, Zhang and Cheung 2013). In this project, Shanghai port is used as a benchmark to provide an additional reference point to the comparisons of Indonesian and Australian ports.

6.1.5 Data Envelopment Analysis (DEA)

22Data envelopment analysis (DEA) is a common tool used to measure efficiency based on the inputs and outputs variable of processes in port operations. DEA has recently been applied by several researchers to investigate the efficiency and productivity of port logistics operations. However, most of the inputs and outputs from past studies are directly related to the physical infrastructure of the port, such as cranes, number of berths, berth length, quay length, yard area etc. and with output of container throughput (TEU) (Almawsheki and Shah 2014). Kevin et al. (2004) analysed the application of DEA to container port production efficiency. Ada and Lee (2007), So et al. (2007), Salem et al. (2008) and Van Dyck (2015) have all conducted DEA efficiency analysis on Malaysia, Northeast Asia, Middle Eastern and West African seaports respectively. Since then, a significant number of port efficiency analyses have been completed using DEA. It is an appropriate tool for investigating the relative efficiency of selected ports in Australia and Indonesia. In this way, their efficiency as major trading partners can be compared and recommendations for improvement provided.

6.1.6 Private Sector Involvement

23It is shown that port ownership structures have an influence on efficiency, and privatisation may not necessarily be beneficial to the port efficiency (Cullinane, Ji and Wang 2005; Chen, Pateman and Sakalayen 2017). Joint venture arrangements of port organisations and public/private partnership have been critiqued over the years. Tongzon and Heng (2005), Yuen, Zhang and Cheung (2013), Panayides et al. (2015) and Wanke and Barros (2015) did not fully reject the view that privatisation has no relationship to port efficiency. They concluded that it may bring some benefits to the management and operational activities of the ports. However, the government at the same time needs public participation in their reformed policies in order to fully maximise the potential of privatisation of ports.

6.1.7 Current Knowledge Gap

24From the above literature review, it is apparent that previous studies focused largely on port level efficiency as a whole. Furthermore, most of this research was conducted more than five years ago. There is thus a gap in the understanding of efficiency at port terminal level. This study offers more insights for owners, port operators, stakeholders and future researchers on port operations and improvement opportunities. This study aims to do this by measuring and comparing efficiencies at port and terminal levels for the Ports of Melbourne, Fremantle, Botany, Surabaya, Jakarta, Shanghai and their container terminals. This chapter further investigates the effects of privatisation on efficiency of these ports and terminals. Due to the close proximity of Indonesia and Australia and the important trade relationship, this study provides valuable insights into performance of the major ports in Indonesia and Australia.

6.2 Methodology

6.2.1 Data Envelopment Analysis (DEA)

25The Data Envelopment Analysis model was used to quantify and measure the efficiency of ports, focusing on port and container cargoes. DEA models allow for multiple inputs and multiple outputs without strong a-priori assumptions regarding production technology or error structure. There are two basic DEA models generally used in the applications. The first assumes constant returns to scale (CRS) and is named the DEA-CCR model after its authors Charnes, Cooper and Rhodes (1978). The second assumes variable returns to scale (VRS) and is called the DEA-BCC model, named after its authors Banker, Charnes and Cooper (1984). The efficacy of a Decision Making Unit (DMU) can be measure by weighted input variables.

6.2.2 Input and Output Variables

26As outlined implicitly by Charnes et al. (1978), DEA models assume that factor inputs and factor outputs are discretionary. They are controllable and can be set up by the decision-maker. Based on earlier research, the input and output variables used in the port efficiency analysis are summarised in Table 6.1. The analysis presented in this chapter uses the latest available data sets from ports from 2015 annual reports and official government data.

6.2.2.1 Crane Rate

27Crane rate is computed as the total number of containers handled divided by the total elapsed crane time. It is interpreted as a proxy measure for the productivity of capital at a container terminal.

6.2.2.2 Ship Rate

28Ship rate is the average number of containers moved on or off a ship in an hour.

Table 6.1 Input and output variables used in the port DEA analysis.

Image 100000000000029E0000018D2480633EF715C558.jpg

(Table by authors: based on data sourced from: Kevin et al. (2004), Ada and Lee, So et al. (2007), Salem et al. (2008), Cullinane and Wang (2010), Van Dyck (2015))

6.2.3 Mathematical Formulation of DEA

29Let yk = {y1k,y2k,…, yS k} and xk = {x1k,x2k,…, xM k} be the vectors of outputs and inputs for DMU k (k = 1, 2,.., n), where s and m are the number of outputs and inputs respectively. Outputs and inputs are converted into weighted virtual entities by the values of the production factors (ur and vi).

30For DMUk, the virtual output is calculated as in equation (1) and the virtual input is calculated as in equation (2). The efficiency is calculated as in equation (3).

Image 100000000000020A000000981F2529121BE39B5E.jpg

31Subject to

Image 100000000000020E0000009B90AE90F990F3ECED.jpg

32Where, j is the number of DMU being evaluated in DEA. k is a generic DMU and θk its efficiency. Solving this fractional problem for each DMU, the efficiency scores 0 < θk<1, (k = 1,2,…, n). The DMUs with θk= 1 are considered as efficient, and the ones with θk < 1 are inefficient.

33The efficiency score θk, obtained from the CCR model, represents the overall efficiency of DMU k. The most efficient selected ports can be set to the maximum efficiency DMU (θk = 1)

Image 1000000000000218000000C48D9E74B8458DBCA0.jpg

6.2.4 Returns to Scale Structure

34A DEA model can be either a Constant Returns to Scale structure (CRS) also known as DEA-CCR model, or a Variable Returns to Scale (VRS) known as DEA-BCC model, depicted in Fig. 6.3. In the case of a CRS, it is assumed that an increase in the inputs consumed would lead to a proportional increase in the outputs produced. In the VRS model, the outputs produced do not vary proportionately with the increase in inputs. They may increase; remain constant; or decrease with an increase in the inputs. The CRS version is more restrictive than the VRS, and usually yields a fewer number of efficient units. This also results in lower efficiency scores among all DMUs. The CRS is considered a special case of the VRS model.

Image 10000000000002560000014668081ECB76FB5A85.jpg

Fig. 6.3 Computing efficiency frontier in VRS and CRS model

(Figure by the authors)

6.2.5 Scale Efficiency

35The scale efficiency of each DMU has been estimated using the efficiency scores obtained under the CCR and BCC models. In fact, the efficiency observed under the CRS model is the overall measure of technical and scale efficiency whilst the one deriving from the VRS model is pure technical efficiency. The scale efficiency can be used to indicate the efficiency of the DMU.

Image 100000000000020100000031B031E6060AFAF885.jpg

6.3 Results and Findings

6.3.1 Data Analysis

6.3.1.1 Port

36To streamline the DEA analysis, the input data is divided into two groups: functional and operational. The functional inputs consist of land size, length of berths, number of berths and number of cranes — effectively describing the existing facilities and infrastructure of the port. These parameters indicate the physical hardware of the port and the ability of the port to handle the throughput objectively. In contrast, the operational inputs consist of operating expense, net assets and number of employees — in other words, the financial assets and human labour have been used as inputs in the ports. These parameters indicate the software of the ports and how much recourse has been used to operate the ports. The outputs used in DEA are the same for both functional and operational inputs: container throughput and bulk throughput.

37The sample ports for DEA are shown in Table 6.2 while the characteristics of the variables used to estimate the relative efficiency of the sample ports are presented in Table 6.3. As shown in Table 6.3, the standard deviations of the port data variables are significantly large. This is due to the size and scale difference in the sample ports, especially for the Port of Shanghai, which is one of the biggest ports in the Asia Pacific area and consists of terminals and large throughput volume. Hence, this further strengthens our initial idea of analysing container terminals of the selected ports, where datasets gathered are more complete and more precisely reflect the operational side of the terminals.

Table 6.2 Sample ports for efficiency comparison (Table by the authors)

Country

Sample Ports

Australian

Port of Melbourne

Australian

Port of Fremantle, Fremantle

Australian

Port Botany, Sydney

Indonesia

Tanjung Priok Port, Jakarta

Indonesia

Tanjung Perak Port, Surabaya

China

Port of Shanghai

Table 6.3 Descriptive port statistics for input and output variables for DEA

Image 100000000000029A000000B8EB3FEF0E5BD47BB7.jpg

Image 10000000000002940000011E15C72E690BFC3468.jpg

*Std. Dev = Standard deviation

(Table by the authors based on data gathered from various publically available data sources related to ports listed in Table 6.2)

6.3.1.2 Container Terminal

38To conduct the efficiency analysis at the terminal level, a typical container terminal from each of the sampled ports was selected. The sample of container terminals is shown in Table 6.4 and the characteristics of the variables used to estimate the relative efficiency of the sample container terminals are presented in Table 6.5.

Table 6.4 Sample of container terminals for efficiency comparison (Table by the authors)

Country

Sample Port

Sample Container Terminal

Australia

Port of Melbourne, Melbourne

Swanson Dock

Australia

Port of Fremantle, (WA)

North quay terminal (Fremantle)

Australia

Port Botany (Sydney)

DP World Container Terminal

Indonesia

Tanjung Priok Port (Jakarta)

Jakarta International Container
Terminal

Indonesia

Tanjung Perak Port (Surabaya)

TTL — Terminal Teluk Lamong

Indonesia

Tanjung Perak Port (Surabaya)

TPS — Terminal Petikemas

China

Port of Shanghai (Shanghai)

Pudong International Container
Terminal

Table 6.5 Descriptive container terminal statistics for inputs and outputs variables for DEA

Image 100000000000029E000001544700D4B2F61FC5F7.jpg

(Table by the authors, based on data gathered from various publically available data sources related to ports listed in Table 6.4)

6.3.2 Efficiency Comparison Based on DEA Result

6.3.2.1 Port

39The inputs and outputs shown in Table 6.5 were used for the DEA analysis to determine efficiency, while Table 6.6 shows the efficiency computed, based on these variables. This demonstrates that the Australian ports are more efficient than the Indonesian ports when comparing the functional input: land size, length of berths, number of berths and number of cranes. Port of Melbourne, Fremantle Port, Port Botany (Sydney) and Port of Shanghai are relatively efficient since the scale efficiency equals to 1 while Tanjung Perak Port (Surabaya) and Tanjung Priok Port (Jakarta) are relativity inefficient since their scale efficiencies are less than 1. The Surabaya port is less efficient than Jakarta port with scale efficiencies of 0.861 and 0.910 respectively (Table 6.6). Both these ports scored low efficiencies in the CCR and BCC models.

40The Australian ports are relatively efficient when comparing the operational input: operating expense, net assets and number of employees (Table 6.7). On the other hand, Tanjung Perak Port (Surabaya) is relatively inefficient as scale efficiency is less than 1 (0.863), but Tanjung Priok Port (Jakarta) is relativity efficient as scale efficiency equals 1 when comparing operational inputs.

Table 6.6 Port Efficiency score for functional inputs based on DEA models (Table by authors)

Country

Port

CCR
Efficiency

BCC
Efficiency

Scale
Efficiency

Australia

Port of Melbourne

1.000

1.000

1.000

Australia

Fremantle Port

1.000

1.000

1.000

Australia

Port Botany (Sydney)

1.000

1.000

1.000

Indonesia

Tanjung Perak Port (Surabaya)

0.488

0.567

0.861

Indonesia

Tanjung Priok Port (Jakarta)

0.598

0.657

0.910

China

Port of Shanghai

1.000

1.000

1.000

41Generally, the Australian ports and the Chinese ports are relatively efficient compared to the Indonesian ports. Comparing Indonesian ports, Tanjung Priok Port (Jakarta) is more efficient than Tanjung Perak Port (Surabaya).

Table 6.7 Port Efficiency score for operational inputs based on DEA models (Table by authors)

Country

Port

CCR
Efficiency

BCC
Efficiency

Scale
Efficiency

Australia

Port of Melbourne

1.000

1.000

1.000

Australia

Fremantle Port (Perth)

1.000

1.000

1.000

Australia

Botany Port (Sydney)

1.000

1.000

1.000

Indonesia

Tanjung Perak Port (Surabaya)

0.807

0.935

0.863

Indonesia

Tanjung Priok Port (Jakarta)

1.000

1.000

1.000

China

Port of Shanghai

1.000

1.000

1.000

6.3.2.2 Comparison of Container Terminals

42The major container terminals in each port were analysed for detailed study. The container terminal efficiency was calculated with the following inputs: terminal area, length of berths, number of berths and number of cranes with regard to different outputs (crane rate, ship rate and container throughput). As shown in Table 6.8, in terms of crane rate, Terminal Teluk Lamong (TTL), North Quay Terminal (Fremantle), DP World Container Terminal (Sydney) and Pudong International Container Terminal (Shanghai) are relativity efficient since the scale efficiency equals to 1 while Jakarta International Container Terminal (JICT), Terminal Petikemas (Surabaya) (TPS) and Swanson Dock (Melbourne) are relativity inefficient due to scale efficiency less than 1. Swanson Dock in Port of Melbourne is inefficient in the CCR model but efficient in the BCC model.

Table 6.8 Container terminal Efficiency in terms of Crane Rate (Table by authors)

Container Terminal

CCR
Efficiency

BCC
Efficiency

Scale
Efficiency

Jakarta International Container
Terminal (Jakarta)

0.624

0.751

0.830

TPS — Terminal Petikemas (Surabaya)

0.971

0.999

0.973

TTL — Terminal Teluk Lamong (Surabaya)

1.000

1.000

1.000

Swanson Dock (Melbourne)

0.845

1.000

0.845

North Quay Terminal (Fremantle)

1.000

1.000

1.000

DP World Container Terminal (Sydney)

1.000

1.000

1.000

Pudong International Container
Terminal (Shanghai)

1.000

1.000

1.000

43In terms of ship rate, JICT, TPS, TTL and Swanson Dock (Melbourne) are relatively inefficient as scale efficiency is less than 1 (Table 6.9).

44In terms of container throughput (Table 6.10), Swanson Dock (Melbourne), DP World Container Terminal (Sydney) and Pudong International Container Terminal (Shanghai) are relatively efficient as scale efficiency equals to 1. North Quay Terminal is inefficient in the CCR model but efficient in the BCC model.

Table 6.9 Container terminal Efficiency in terms of Ship Rate (Table by authors)

Container Terminal

CCR
Efficiency

BCC
Efficiency

Scale
Efficiency

Jakarta International Container
Terminal (Jakarta)

0.767

0.794

0.966

TPS — Terminal Petikemas (Surabaya)

0.817

0.903

0.905

TTL — Terminal Teluk Lamong (Surabaya)

0.869

0.950

0.915

Swanson Dock (Melbourne)

0.858

1.000

0.858

North Quay Terminal (Fremantle)

1.000

1.000

1.000

DP World Container Terminal (Sydney)

0.988

1.000

0.988

Pudong International Container Terminal (Shanghai)

1.000

1.000

1.000

Table 6.10 Container terminal Efficiency in terms of Container Throughput (Table by authors)

Container Terminal

CCR
Efficiency

BCC
Efficiency

Scale
Efficiency

Jakarta International Container
Terminal (Jakarta)

0.744

0.818

0.910

TPS — Terminal Petikemas (Surabaya)

0.661

0.960

0.689

TTL — Terminal Teluk Lamong (Surabaya)

0.348

0.950

0.367

Swanson Dock (Melbourne)

1.000

1.000

1.000

North Quay Terminal (Fremantle)

0.725

1.000

0.725

DP World Container Terminal (Sydney)

1.000

1.000

1.000

Pudong International Container Terminal (Shanghai)

1.000

1.000

1.000

45From the above Tables 6.6 to 6.10, Australian container terminals are generally more efficient than Indonesian container terminals. Where Indonesian container terminals are concerned, JICT shows, on average, around 80 %–90 % efficency in each aspect. TPS and TTL have high efficiency scores in terms of crane rate and ship rate but relativity low scores in terms of container throughput. TTL in particular had a significantly lower efficiency score in terms of container throughput ability 36.7 %, while the other container terminal TPS scored 68.9 % efficiency (Table 6.10). For the sampled Australian container terminals, Swanson Dock in the Port of Melbourne is less efficient in terms of crane rate and ship rate with around 85 % but very efficient in terms of container throughput. In contrast, North Quay Terminal in the Port of Fremantle is efficient in terms of crane rate and ship rate but inefficient in container throughput. As one of the top container terminals in China, Pudong International Container Terminal in the Port of Shanghai is relatively efficient and scored 100 % effiency in all aspects, under both the CCR and BBC model.

46Different port and terminal efficiency factors are discussed in the section below, where the framework presented by Cheon, Dowall and Song (2010) is used as a guideline. This report affirms their framework, using it to explain the areas of improvement required by ports, as explained in the section above with CCR model efficiency, BCC pure technical efficiency and scale efficiency.

Table 6.11 Sources of efficiency gains (after Cheon, Dowall and Song (2010))

Category (DEA models)

Areas of improvement

Technical efficiency

Improve utilization and optimisation of terminals
Crane and facilities improvement
Labour reforms

Overall progress efficiency

Container trade volume

Scale efficiency

Governing and managing structure reform
Better decision making and investment climate

6.4 Discussion

47The improvement percentage of inefficient units based on constant crane rate, ship rate and throughput are shown in Fig. 6.4.

Image 10000000000002950000012EC00E739919CDC22C.jpg

Fig. 6.4 Improvement percentage* of inefficient units based on constant crane rate, ship rate and throughput (Figure by authors)
*vertical axis % improvement in efficiency required. Also note that Shanghai is not depicted as Shanghai is relative efficient at 100 %

48From Fig. 6.4, JICT has the highest improvement rate required to match the optimal required efficiency score and improve its terminal operations. This figure is based on the percentage improvement of each inefficient unit of terminals on crane rate, ship rate and throughput efficiency. Sydney and Shanghai both performed relatively well with improvements of 4 % and 0 % overall. Fremantle and Melbourne have a slack improvement of around 16 % and 10 % respectively. The Indonesian terminals have the highest room for efficiency improvements with Jakarta (JICT) 50 %, TPS 35 % and TTL 30 % overall. A summary of the container terminal efficiency scores is shown in Table 6.12.

6.4.1 Indonesia

49Overall, from the port efficiency results shown in Tables 6.6 and 6.7, both of the Indonesian ports examined are ranked less efficient relative to the Australian and Chinese ports. The Port of Tanjung Perak in Surabaya, on the whole, had a lower efficiency score compared to the Port of Tanjung Priok in Jakarta. Andenoworih (2010) and Syafaaruddin (2015) both confirm that Jakarta is efficient overall in Indonesia due to its advantageous location within a high regional economic activity area. Based on the results, the major determinants of port efficiency are the following outputs: annual container throughput, and ship rate. This is where Tanjung Priok outperforms Surabaya with higher throughputs in both categories (see Fig. 6.12)

Table 6.12 Summary of Container Terminal Efficiency scores (Table by authors)

Image 100000000000029C000001B4C1006A469276E4AB.jpg

50From the DEA results, it can be seen that JICT also has a better efficiency score using container throughput as the output. However, it has a lower crane rate within the terminal. This is supported by earlier research by Afriansyah et al. (2017) and Wiradanti et al. (2016), where slow landside customs handling as well as the poorly integrated information system of the port caused major congestion issues in the terminal. The DEA results shown in Tables 6.7 to 6.10 highlight that crane rate and the efficiency score at JICT is the lowest among terminals, suggesting a poor handling rate of containers when loading/unloading, which increases ship dwell time. The ship turnaround time at the terminal alone is ten hours more than the closest rival TPS and TTL. Low infrastructure expenditure and low quality of workers with high capital input are common issues in Indonesia (Firdausy 2005). The different models of DEA, BCC, CCR and Scale model gave similar outputs, lending support to the finding from the literature review that Jakarta is inefficient in seaside and landside operations. The Port of Tanjung Priok in Jakarta has three times more cranes and employees than the Port of Melbourne, yet still ranks lower in efficiency. This is supported by the study from Wiradanti et al. (2016), which reported that the Indonesian government has approved funding for further expansion and upgrade of the port to improve the sea/land operations and ease congestion.

51Both terminals in Surabaya have high efficiency in terms of sea-side crane operations. Because annual throughput data was not available for TTL, an estimation is used for the container TEU throughput at 40 % capacity of its full capacity of 1.6 million annual TEU (Seatrade Maritime 2017). However, the DEA analysis showed TTL is inefficient as an overall terminal, since it had low container throughput. It also has a high net asset value, and this does not equate to the profits and expected throughput, due to low usage. Rahmanto (2016) confirms this, describing TTL as a low usage terminal with “state-of-art” facilities, but with poor road/rail infrastructure outside the terminal. The terminal needs to improve its container volume to achieve the desired efficiency score and improve its terminal usage capacity. Having its own gas-fired power plant in the terminal (Terminal Teluk Lamong 2015) did not help improve the issue in any way or help increase the business volume to the port corporation. Firdausy (2005) strongly suggests implementation of institutional reform to solve the poor governance and investment climate that could change the sustainability of the terminal business. It will also lead to better decision making in investments that the terminal desperately needs in terms of multimodal transport from the terminal to the outside world (Rahmanto 2016).

52To validate our findings of inefficiency at TTL, a sensitivity analysis was done by introducing a new estimate for the annual throughput. It is expected that annual container throughput will increase by 0.4 million each year (Seatrade Maritime 2017), hence 60 % capacity was used as the estimation (0.96 million). The sensitivity analysis results did not generate a huge difference to the earlier assumption with a new efficiency score of 0.522. This means that, with its expensive new equipment and facilities, the trade volume needs to increase for the port to reach its potential optimal efficiency. This in turn would increase the utilisation rate of the port capacity.

53The findings from the TPS are consistent with the research findings of Groenveld and Wanders (2009). Table 6.12 shows that slight improvements are needed for crane rate, ship rate and landside operations in order to boost efficiency. Compared with TTL and Swanson Dock in Melbourne, TPS is doing quite well in terms of seaside operational function, where the crane rate in DEA Table 6.12 is higher than that of Melbourne’s Swanson Terminal. However, the container throughput efficiency score is lower than that of Jakarta International Terminal, and the Australian and Chinese terminals. From the CCR model, it is ranked second, behind JICT, and in the BCC model it is ranked first, ahead of JICT. Research conducted by Syafaaruddin (2015) supports this finding that, overall, TPS has achieved near-full capacity of the terminal and has generated the second highest container volume behind JICT. It is placed behind Jakarta in the scale efficiency score, which is an aggregate of the pure technical efficiency and general efficiency. This indicates that TPS either has to improve the quality of its workers to improve crane rates, or decrease the trade volumes to match the ideal optimal efficiency level.

54Surabaya’s Tanjung Perak port as a whole did not achieve a good efficiency score, ranking last in the port ranking. Seo et al. (2012) used a similar DEA CCR model in their analysis with a larger sample size. This supports our DEA analysis for Tanjung Perak, where both the output and input-orientated model showed the lowest efficiency scores. It is suggested that port managers can mitigate this by improving port operations and management. An analysis of the relative efficiencies of the two major terminals in Tanjung Perak (TPS and TTL) indicates that both terminals contribute to the overall port performance with their relatively low crane rate and container output DEA efficiency scores.

55Looking at ownership structures, JICT and TPS both have private sector involvement while TTL is wholly owned by a State Owned Enterprise. Our analysis shows that JICT and TPS are performing better in overall efficiency compared to TTL. Although TTL is better equipped with modern facilities, it is still outperformed by the two Indonesian terminal counterparts in overall performance. As stated by Tongzon and Heng (2005), government ports may gain benefits in allowing private sector participation, which may introduce better decision making in structural management and assist the port to become a more efficient performer and be more profitable.

56Figures 6.5 and 6.6 show areas requiring improvements for the terminals. Port of Tanjung Priok in Jakarta requires moderate (yellow) improvement in all areas: seaside technical operations as well as landside, especially customs clearance (red). Tanjung Perak port needs a high (red) level of improvement in landside infrastructure development for optimal efficiency, predominantly caused by the inefficient performance of Teluk Lamong.

Image 100000000000029B00000078A2B3E5066FA23713.jpg

Fig. 6.5 Port of Tanjung Priok (Jakarta) logistics flow chart

(Figure by authors)

Image 100000000000029B0000008617D010F5B7414DC3.jpg

Fig. 6.6 Port of Surabaya logistics flow chart

(Figure by authors)

6.4.2 Australia

57Due to inaccessibility of data, landside analysis of Melbourne’s Swanson Dock could not be quantified. However, a study from Infrastructure Victoria (2017) shows that the landside multimodal infrastructure of Swanson Dock to outside destinations is efficient, and can sufficiently accommodate future population and demand growth in the long term. It has been suggested that rail transport of containers should be developed to share the logistics load, increasing from the current 10 % rail usage (Infrastructure Victoria 2017). The focus for the terminal is still to upgrade the seaside technical facilities in order to improve crane rates. Efficiency scores of 0.84 crane rate and 0.86 ship rate (Table 6.12) are not sufficient for a port that handles 2.64 million TEU’s annually. This finding confirms the conclusion of Lubulwa, Malarz and Wang (2011) and Ghaedhi, Cahoon and Nguyen (2016): that on-dock rail infrastructure and crane facilities are behind other advanced terminals in the world, especially TPS and TTL. The lack of connectivity between terminals and docks adds time and cost to the freight system, as trucks are needed to service the gap. Infrastructure Victoria (2017) also strengthens this view by stating the need to upgrade berth capacity and yard with on-dock rail to improve the crane rate. By increasing “rail’s mode share of container haulage”, it will have a positive improvement on the operational and management practice of the terminal.

58In terms of operational efficiency, North Quay container terminal in Fremantle did not have an issue at terminal level analysis. It did however receive a score of 0.725 in overall container throughput efficiency (Table 6.12), lower than JICT, and other Australian and Chinese Terminals. Looking at the container throughput, North Quay has the lowest annual throughput. Although it is efficient in handling containers, the throughput result suggests that the size and the scale of the terminal may be insufficient when compared with larger terminals, in terms of the container throughput volume at terminal level. Cheon, Dowall and Song (2010) stressed that scale and size of ports may play a crucial role in determining the efficiency as the volume of trade does not meet up with other larger volume terminals. Volume throughput needs to increase by 15–20 % to meet optimal efficiency.

59Port Botany in Sydney is efficient in the DEA analysis, supporting the findings of Tongzon (2001). Slight improvement can be made in the handling of crane and ship rates, where they are just below the score of 1. DP World container terminal also achieved scores of 1 for crane rate, ship rate and overall throughput efficiency rate output. Zahran et al. (2015) confirms this finding that Sydney has a relatively good efficiency rating when they performed DEA against revenue and throughput in their study. The Port of Melbourne should upgrade and focus on investing in new cranes and on dock facilities (yellow), in order to improve seaside operations, as shown in Fig. 6.7.

Image 100000000000028B0000007A5C358B212744A593.jpg

Fig. 6.7 Port of Melbourne logistics flow chart

(Figure by authors)

6.4.3 Opportunities for Future Research

60Throughout the stages of acquiring data and performing DEA analysis, it was recognised that there were limitations in the completeness of the data collected on which to base the analysis. This data was mostly obtained from public sources. Future research into Australian and Indonesian port efficiency would benefit from sourcing data directly from the ports. The current DEA approach used in this study did not consider the temporal scale efficiency. It would be beneficial in future research to include datasets of various time periods to investigate temporal changes which can further strengthen the DEA results. This project mostly emphasised the seaside crane handling structure and overall container throughput rate. It is highly recommended that for a complete port operational review study, landside data from ports and terminals should be included in the analysis. Such considerations would improve the methodology and the expanded dataset would provide further insights into other issues, such as connectivity to the other terminals and inland ports.

6.5 Conclusion

61Indonesian ports require more efficiency improvement, and this can be realised from improving the sea-side technical operations and quality of workers. Labour reform may be required in Jakarta’s Port of Tanjung Priok, while Surabaya’s Tanjung Perak Port may require better institutional and policy reform to improve the investment climate and raise throughput volume to achieve optimal efficiency. Indonesian ports and terminals, which are mostly State Owned Enterprises, may benefit more from private sector involvement. Here, a more transparent management structure and system could help improve overall port performance. In general, the efficiency of Indonesian port terminals is lower than that seen in Australian port terminals in all aspects — seaside and landside — although there is healthy competition within the terminal level.

Bibliographie

References

Almawsheki, ES, Shah, MZ and Al-Iraqi, AS 2014. Efficiency analysis of container ports: A review, paper presented at the conference proceedings of 8th SEATUC Symposium, 4–5 March, Johor Bahru, Malaysia.

Andenoworih, TH 2010. Measuring relative container terminal efficiency in Indonesia, PhD Thesis, Erasmus University, Rotterdam, The Netherlands, thesis.eur.nl/pub/33240

Afriansyah, AR, Febriani, E, Fahmi, QA and Krismawanti, DD, n.d. Online integration system as a solution of reducing dwelling time using management informations system approach.

Chen, PSL, Pateman, H and Sakalayen, Q 2017. ‘The latest trend in Australian port privatisation: Drivers, processes and impacts’, Research in Transportation Business and Management, 22, pp. 201–13.

Cheon, S, Dowall, DE and Song, DW 2010. ‘Evaluating impacts of institutional reforms on port efficiency changes: Ownership, corporate structure, and total factor productivity changes of world container ports’, Transportation Research Part E: Logistics and Transportation Review, 46: 4, pp. 546–61, https://doi.org/10.1016/j.tre.2009.04.001

Cullinane, K, Ji, P and Wang, TF 2005. ‘The relationship between privatization and DEA estimates of efficiency in the container port industry’, Journal of economics and business, 57: 5, pp. 433–62, https://doi.org/10.1016/j. jeconbus.2005.02.007

Cullinane, K, Wang, TF, Song, DW and Ji, P 2006. ‘The technical efficiency of container ports: comparing data envelopment analysis and stochastic frontier analysis’, Transportation research part A: Policy and practice, 40: 4, pp. 354–74, https://doi.org/10.1057/mel.2011.15

Chung, KC 1993. Port Performance Indicators, Transportation, Water and Urban Development Department, The World Bank, www.siteresources.worldbank.org/INTTRANSPORT/Resources/336291-1119275973157/td-ps6.pdf

Department of Foreign Affairs and Trade (DFAT) 2016. Indonesia, Australian Government, Australia, www.dfat.gov.au/trade/resources/documents/indo.pdf

Department of Foreign Affairs and Trade (DFAT) 2017. Australia’s trade at a glance, Australian Government, Australia, www.dfat.gov.au/trade/resources/trade-at-a-glance/pages/asean.aspx

Department of Foreign Affairs and Trade (DFAT) 2019. Indonesia-Australia comprehensive economic partnership agreement, Australian Government, Australia, www.dfat.gov.au/trade/agreements/not-yet-in-force/iacepa/Pages/indonesia-australia-comprehensive-economic-partnership-agreement.aspx

Firdausy, CM 2005. Productivity performance in developing countries, country case studies: Indonesia, report for the United Nations Industrial Development Organisation (UNIDO), www.unido. org/file-storage/download

Fremantle Ports 2017. Fremantle ports Western Australia, www.fremantleports.com.au/About/Pages/default.aspx

Ghadehi, H., Cahoon, S and Nguyen, H 2016. ‘The role of rail in the Australian port-based container market: challenges and opportunities’, Australian Journals of Maritime and Ocean Affairs, 8–1, pp. 52–72, https://doi.org/10.108 0/18366503.2016.1173632

Groenveld, R and Wanders, S 2009. ‘Computer simulation model International Container Terminal Tanjung Perak, Surabaya, Indonesia’, in Proceedings of the International Workshop Harbour, Maritime and Industrial Logistics, Modelling and Simulation, Genoa, Italy: SCS European Publishing House.

Hill, H (Ed.) 2014. Regional dynamics in a decentralized Indonesia, Singapore: Institute of Southeast Asian Studies.

Hung, SW, Lu, WM and Wang, TP 2010. ‘Benchmarking the operating efficiency of Asia container ports’, European Journal of Operational Research, 203: 3, pp. 706–13, https://doi.org/10.1007/s12544-015-0187-z

Infrastructure Victoria 2017. Second container port advice — Evidence base discussion paper, Victoria, Australia: Infrastructure Victoria.

IPC 2017. KOJA Container Terminal, www.indonesiaport.co.id/menu/koja-ct.html

JICT 2013. JICT shareholder at glance, www.jict.co.id/?x0=andx1=68andx2=article

Lam, JS and Yap, WY 2006. ‘A measurement and comparison of cost competitiveness of container ports in Southeast Asia’, Transportation, 33: 6, pp. 641–54, https://doi.org/10.1007/s11116-006-7474-4

Lin, JY 2011. ‘China and the global economy’, China Economic Journal, 4: 1, pp. 1–14, https://doi.org/10.1080/17538963.2011.609612

Logistics Capacity Assessment 2017. Indonesian Surabaya Port of Tanjung Perak.

Lubulwa, G, Malarz, A and Wang, SP 2011. An investigation of best practice landside efficiency at Australian container ports, proceedings of the Australasian Transport Research Forum, 29–30 September, Adelaide, Australia.

Ojala, L and Çelebi, D 2015. The World Bank’s Logistics Performance Index (LPI) and drivers of logistics performance, prepared for the roundatable ‘Logistics development strategies and their performance measurements’ at the International Transport Forum, 9–10 March, Queretaro, Mexico, www.itfoecd.org/sites/default/files/docs/ojala.pdf

Panayides, PM, Parola, F and Lam, JSL 2015. ‘The effect of institutional factors on public–private partnership success in ports’, Transportation Research Part A: Policy and Practice, 71: C, pp. 110–27, https://doi.org/10.1016/j.tra.2014.11.006

PR Newswire 2017. Lonsdale Consortium comprises Future Fund, QIC, GIP and OMERSacquire 50-year lease of Port of Melbourne, www.prnewswire.com/news-releases/lonsdale-consortium-comprising-future-fund-qic-gip-andomers-acquire-50-year-lease-of-port-of-melbourne-300330084.html

Rahmanto, WP 2016. Kandangan dry port project: an option of solution for congestion: case of Lamong Bay Terminal, Surabaya, Indonesia: World Maritime University Dissertations, 528, pdfs.semanticscholar.org/2ad6/2cb9138dee9bd957b0e5e9 eb83d840067837.pdf

Wee, V 2015. Indonesia awards concessions for Teluk Lamong, Makassar ports, Seatrade Maritime News, www.seatrade-maritime.com/news/asia/indonesiaawards-concessions-for-teluk-lamong-makassar-ports.html

Seo, YJ, Ryoo, DK and Aye, MN 2012. ‘An analysis of container port efficiency in ASEAN’, Journal of Navigation and Port Research, 36: 7, pp. 535–44.

Syafaaruddin, DS 2015. Evaluation of container terminal efficiency performance in Indonesia: Future investment, Rotterdam, The Netherlands: Erasmus University.

Terminal Teluk Lamong 2015. Terminal overview.

Wanke, PF and Barros, CP 2015. ‘Public-private partnerships and scale efficiency in Brazilian ports: Evidence from two-stage DEA analysis’, Socio-Economic Planning Sciences, 51, pp. 13–22, https://doi.org/10.1016/j.seps.2015.06.002

Wiradanti, B, Pettit, S, Abouarghoub, W and Beresford, AKC 2016. Trends in trade and port development of rising economies: Mexico and Indonesia, paper presented at the Logistics Research Network Conference, The Chartered Institute of Logistics and Transport (CILT), Hull University, United Kingdom:

World Shipping Council 2017. Top 50 world container ports, www.worldshipping.org/about-the-industry/global-trade/top-50-world-container-ports

World Bank Group 2013, Transport in Indonesia.

World Bank 2016. Connecting to Compete 2016 Trade Logistics in the Global Economy. The Logistics Performance Index and Its Indicators, www.openknowledge.worldbank.org/bitstream/handle/10986/24598/Connecting0to00n0the0global0economy. pdf?sequence=1andisAllowed=y

Wu, YCJ and Goh, M 2010. ‘Container port efficiency in emerging and more advanced markets’, Transportation Research Part E: Logistics and Transportation Review, 46: 6, pp. 1030–42, https://doi.org/10.1016/j.tre.2010.01.002

Tongzon, J 2001. ‘Efficiency measurement of selected Australian and other international ports using data envelopment analysis’, Transportation Research Part A: Policy and Practice, 35: 2, pp. 107–22, https://doi.org/10.1016/S09658564 (99) 00049-X

Tongzon, J and Heng, W 2005. ‘Port privatization, efficiency and competitiveness: Some empirical evidence from container ports (terminals)’, Transportation Research Part A: Policy and Practice, 39: 5, pp. 405–24, https://doi.org/10.1016/j. tra.2005.02.001

Yuen, ACL, Zhang, A and Cheung, W 2013. ‘Foreign participation and competition: A way to improve the container port efficiency in China?’, Transportation Research Part A: Policy and Practice, 49: C, pp. 220–31, https://doi.org/10.1016/j.tra.2013.01.026

Zahran, SZ, Alam, JB, Al-Zahrani, AH, Smirlis, Y, Papadimitriou, S and Tsioumas, V 2015. ‘Analysis of port authority efficiency using data envelopment analysis’, Maritime Economics and Logistics, 19: 3, pp. 518–37, https://doi.org/10.1057/mel.2015.33

Auteurs

Professor of Engineering Management, Deputy Head of Department (Academic), Dept. of Infrastructure Engineering, The University of Melbourne.

Master of Engineering graduate, Dept. of Infrastructure Engineering, The University of Melbourne.

Master of Engineering graduate, Dept. of Infrastructure Engineering, The University of Melbourne.

CC-BY-4.0

Le texte seul est utilisable sous licence CC BY 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Acheter

Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search