Reviews and Responses for Methodology to Quantify and Attribute ATM Inefficiencies and Their Impact on Fuel Consumption

Gabriele Sirtori; Laurent Joly; Melissa I. Hofmann;
This web version is automatically generated from the LaTeX source and may not include all elements. For complete details, please refer to the PDF version.

Original paper

The DOI for the original paper is https://doi.org/10.59490/joas.2026.8469

Review - round 1

Reviewer 1

In this paper, the authors present an analysis of the performance of several flights over Europe. In my opinion, for such an analysis, which is not remarkably new, the methodology used is quite inaccurate. I recommend the authors to address the following comments so that the paper can be ready for publication:

1) It is not clear to me which is the optimal trajectory the authors are comparing each trajectory to. As I understand from Section 3.1., it seems that the authors are using one "universal" optimal flight profile, corresponding to that of an A320. This is highly inaccurate, as each aircraft has its own performance characteristics, leading to different optimal altitudes, speeds, etc. Each aircraft model has a different optimal Top of Descent, Top of Climb, optimal cruise altitude, etc. Additionally, the effects of weather should be considered as well to obtain the optimal profile. Could the authors ellaborate on that so that it is clearer to me how are they comparing the actual and optimal profiles?

2) Linked to my previous comment, in Figure 2 the authors present a set of values for several aircraft models. They are not actually "performance indicators"; they are just performance values or characteristics for several aircraft models, so the authors should rename this figure and related explanations. In addition, this is just a small set of the performance characteristics for several aircraft models. And, on top of that, these values correspond to actual flown values but not optimal ones, right? Therefore, bigger differences would be observed among different aircraft models if focusing on the optimal values (actually, there are already big differences in the flown values as well). In my opinion, it is very hard to justify the fact that the authors are using as an optimal profile that of an A320, specially when conducting an efficiency study of this kind.

3) The authors present in Table 1 a set of selected city pairs but, in the end, only results for one city pair are presented. Therefore, the information presented in this Table does not seem very relevant to this publication, as none of the other city pairs are analyzed or even mentioned throughout the paper.

4) Typos: - Section 2.2.: "has been selected, as shown in ??" –> I guess it’s Table 1? - Section 3.1.: "most optimal" –> optimal Recommendation: Revisions Required

Reviewer 2

This paper tackles an important and timely problem: the quantification and attribution of ATM inefficiencies using real flight data and filed flight plans. The overall objective of linking operational inefficiencies to fuel burn and climate impact is well motivated, and the use of largely open-source trajectory data is a significant strength of the work. However, after reviewing, I felt that the specific research question or set of tightly defined research objectives remains unclear throughout much of the manuscript. As a result, it is difficult to assess exactly what new scientific insight is being delivered beyond the construction of a preliminary methodological pipeline. Clarifying the central analytical focus, strengthening figure interpretation, and more carefully addressing the reproducibility implications of the proprietary data would significantly improve the paper. As a minor point, the authors should also carefully check the manuscript for compilation and reference formatting issues; for example, there is a broken reference placeholder in Section 2.2 that should be fixed. (1.) While the background and motivation up through the end of the introduction are generally well written, I was expecting a clearly articulated statement of the central research question or set of research objectives at the close of the section. By the time the paper transitions into Section 2, it remains somewhat unclear what precise scientific / research question the authors are attempting to answer. The paper lists several factors that contribute to flight operations inefficiency and ATMrelated inefficiencies, but these are not explicitly tied to a focused investigative goal. Clarifying what specific hypotheses are being tested, or what precise attribution or quantification problem is being addressed, would substantially strengthen the framing of the work. (2.) Given the paper’s emphasis on open data and reproducibility, it is somewhat unclear how accessible the Thales-provided datasets truly are to the broader research community. While the authors note that these proprietary datasets enrich the analysis, the extent to which independent researchers could realistically recreate the full workflow remains uncertain. It would be valuable for the authors to discuss whether a faithful proxy for the Thales dataset could be constructed using only open sources, and if so, what limitations such a proxy would impose on the results. A more explicit discussion of this reproducibility gap would improve transparency. (3.) Some of the figures and associated results would benefit from clearer explanation and physical interpretation. For example, in Figure 4, the temporal offset between the reference trajectory and the actual flight profile is somewhat confusing. It is not clear whether this shift is attributable to specific air traffic flow management initiatives, strategic delays, or other operational constraints. As presented, the figure shows an offset but does not clearly explain its operational origin or analytical significance. Additional explanation in the text would help the reader properly interpret what is being demonstrated. (4.) The authors may wish to more explicitly engage with / look into recent related work by Marek Travník and R. John Hansman on flight efficiency, congestion, and trajectory-based performance analysis. Although the work from Travnik and Hansman relies more heavily on satellite-based aircraft trajectory data, there may be important conceptual and methodological commonalities that could be leveraged here. Recommendation: Revisions Required

Response - round 1

Response to reviewer 1

In this paper, the authors present an analysis of the performance of several flights over Europe. In my opinion, for such an analysis, which is not remarkably new, the methodology used is quite inaccurate. I recommend the authors to address the following comments so that the paper can be ready for publication:

1) It is not clear to me which is the optimal trajectory the authors are comparing each trajectory to. As I understand from Section 3.1., it seems that the authors are using one "universal" optimal flight profile, corresponding to that of an A320. This is highly inaccurate, as each aircraft has its own performance characteristics, leading to different optimal altitudes, speeds, etc. Each aircraft model has a different optimal Top of Descent, Top of Climb, optimal cruise altitude, etc. Additionally, the effects of weather should be considered as well to obtain the optimal profile. Could the authors ellaborate on that so that it is clearer to me how are they comparing the actual and optimal profiles?

Response

Section 3.1 has been clarified to account for your comments. In particular, at the time of the initial submission which wrongly carries the traces of that perspective, we were getting prepared to use a reference trajectory which, in contrast to previous efficiency studies (e.g. referring to orthodromic routes) would be compatible with ATM constraints. This is not of interest to this publication, as all metrics are specific to a particular flight phase and have their own specific reference, as is clarified now and detailed in each metric definition paragraph. The choice you are disagreeing with, of standardizing all operations to the A320 was described as a option for the next phase of the research and is not effective in this paper. Please find the updated paragraph below and in the reviewed paper. "At this stage, however, a reference optimal trajectory from the origin airport to the destination airport is not required as the proposed primary or phase-specific metrics are referred to a particular phase or segment of the flight and each segment-specific reference is either based on a particular feature of the trajectory in this flight segment (i.e the presence of level flight after the TOD - Top of Descent) or on the flight plan, which is considered to be the proxy of the best trajectory accounting for aircraft type, weather and airspace conditions at the time of flight. As opposed to the use of theoretical optimal (such as the great-circle path), the intent is to use segment-specific references achievable in the ATM real-world constraints and then to aggregate the consequent primary metrics into a compound one. Among the primary metrics, some are based on the estimation of the fuel consumption on the segment, whose accuracy varies across the spectrum of various aircraft types and depending on the chosen performance model. To eliminate aircraft-related uncertainties on fuel consumption and move towards ATM-specific efficiency metrics, we propose to consider all flights to be operated by one reference aircraft. For short-haul ones, considered in the present results, we resort to the Airbus A320 (and to its openAP performance model), which is the most common on short-haul flights. As each aircraft has indeed its own set of optimal climb, cruise and descent performance, the consequences on the output of efficiency evaluation due to the proposed deletion of the aircraft variable needs to be assessed. Figure 2 (in the paper) shows key performance indicators for all the aircraft types operating on the target route, LEPA-EDDL, as a post-process of real flight data on the selected city pair. More specifically, the four boxplots show, for each aircraft type, the observed climb range, cruise altitude, cruise Mach number, and descent range. The median values in red, interquartile ranges within the blue or orange boxes, and outliers (dots) illustrate the variability of operational practices between aircraft types as well as within the same type. The biggest dispersion concerns the BCS3 cruise performance, showing a lower altitude and speed, which is coherent with the fact that it is a regional aircraft. Similarly, a lot of dispersion is seen concerning the descent range, which includes all the distance flown after the TOD, including the eventual holding. It is important to note that these indicators correspond to actual operational conditions on the selected city pair, with all aircraft types subject to the same constraints, and therefore do not represent optimal aircraft performance values. As such, the observed dispersion cannot be attributed solely to intrinsic aircraft performance, whose optimal values are very hard to find

as they are proprietary to the OEMs. Enforcing fuel burn calculations based on a single reference aircraft (e.g., A320) thus appears to introduce homogenization in an attempt to adopt an aircraft-neutral approach that makes sure that uncertainties in aircraft performance models do not interfere with ATM efficiency assessment solely based on flight plans and trajectories. Further analyses and validation of such approach will follow, before a definitive implementation in the methodology..

2) Linked to my previous comment, in Figure 2 the authors present a set of values for several aircraft models. They are not actually "performance indicators"; they are just performance values or characteristics for several aircraft models, so the authors should rename this figure and related explanations. In addition, this is just a small set of the performance characteristics for several aircraft models. And, on top of that, these values correspond to actual flown values but not optimal ones, right? Therefore, bigger differences would be observed among different aircraft models if focusing on the optimal values (actually, there are already big differences in the flown values as well). In my opinion, it is very hard to justify the fact that the authors are using as an optimal profile that of an A320, specially when conducting an efficiency study of this kind."

Response

The wording "performance indicators" is not an existing specific jargon precisely liked to something else, so it is a correct phrasing to represent a set of performance values. The definition of reference/optimal profile and the fact that the plot shows operational (from flight data) and not optimal values (from aircraft data sheets, for example) have been clarified in the reply to the previous comment.

3) The authors present in Table 1 a set of selected city pairs but, in the end, only results for one city pair are presented. Therefore, the information presented in this Table does not seem very relevant to this publication, as none of the other city pairs are analyzed or even mentioned throughout the paper.

Response

The objective of presenting the entire list of selected city pairs was to offer an insight into the upcoming project development, but since the table is not relevant for the work shown in the paper detailing the work at this point, only a general description of the selected city pairs is left in the paper to clarify the project objectives and context of development. (at least one end in Europe, short, medium and long haul routes)

4) Typos: - Section 2.2.: "has been selected, as shown in ??" –> I guess it’s Table 1? - Section 3.1.: "most optimal" –> optimal Recommendation: Revisions Required

Response

Thanks for pointing out the typos; they have been fixed.

Response to reviewer 2

This paper tackles an important and timely problem: the quantification and attribution of ATM inefficiencies using real flight data and filed flight plans. The overall objective of linking operational inefficiencies to fuel burn and climate impact is well motivated, and the use of largely open-source trajectory data is a significant strength of the work. However, after reviewing, I felt that the specific

research question or set of tightly defined research objectives remains unclear throughout much of the manuscript. As a result, it is difficult to assess exactly what new scientific insight is being delivered beyond the construction of a preliminary methodological pipeline. Clarifying the central analytical focus, strengthening figure interpretation, and more carefully addressing the reproducibility implications of the proprietary data would significantly improve the paper. As a minor point, the authors should also carefully check the manuscript for compilation and reference formatting issues; for example, there is a broken reference placeholder in Section 2.2 that should be fixed. (1.) While the background and motivation up through the end of the introduction are generally well written, I was expecting a clearly articulated statement of the central research question or set of research objectives at the close of the section. By the time the paper transitions into Section 2, it remains somewhat unclear what precise scientific / research question the authors are attempting to answer. The paper lists several factors that contribute to flight operations inefficiency and ATMrelated inefficiencies, but these are not explicitly tied to a focused investigative goal. Clarifying what specific hypotheses are being tested, or what precise attribution or quantification problem is being addressed, would substantially strengthen the framing of the work.

Response

A summary of the goal of this paper and of the general project has been added to the end of chapter 1: The long-term objective of this work is to obtain a unique efficiency indicator, based on the aggregation of several local indicators referred to to various phases of flight and to both vertical and horizontal efficiency. This indicator can be declined to a single flight, a specific airport pair or network. The preliminary work shown here details the data manipulation and some of the selected relevant metrics that will be aggregated coherently later in the project. You are thus right in pointing towards something that looks as a "preliminary methodological pipeline" but we think new outcomes in this early phase where already valuable for the ATM efficiency scientific community. We have tried to make that clear in the new version of the introduction.

(2.) Given the paper’s emphasis on open data and reproducibility, it is somewhat unclear how accessible the Thales-provided datasets truly are to the broader research community. While the authors note that these proprietary datasets enrich the analysis, the extent to which independent researchers could realistically recreate the full workflow remains uncertain. It would be valuable for the authors to discuss whether a faithful proxy for the Thales dataset could be constructed using only open sources, and if so, what limitations such a proxy would impose on the results. A more explicit discussion of this reproducibility gap would improve transparency.

Response

This dataset has been made available specifically in the context of this project; nonetheless, paragraph 2.1 details how far the proposed analysis can go, referring only to open-access data (its usage might potentially be very computationally costly, for example, to obtain the weather conditions along the flight path). The remaining access limitations to the different versions of the flight plans and the associated metrics, if using only open-access data, has been made clearer.

(3.) Some of the figures and associated results would benefit from clearer explanation and physical interpretation. For example, in Figure 4, the temporal offset between the reference trajectory and the actual flight profile is somewhat confusing. It is not clear whether this shift is attributable to specific air traffic flow management initiatives, strategic delays, or other operational constraints. As presented, the figure shows an offset but does not clearly explain its operational origin or analytical significance. Additional explanation in the text would help the reader properly interpret what is being demonstrated.

Response

The purpose of Figure 4 is to detail the vertical flight efficiency parameters: this particular flight is very representative of the low VFE (limited adherence to the planned flight level) with respect to both the first and last flight plans. The flight also showed a brief interruption of the descent slightly below 20,000 ft, detailing a less than unitary CDO. There is effectively a shift of approximately 40 minutes between the initial and the last, so-called preactive, Flight Plan, the former also matching the actual departure time. This delay has no influence on the considered metrics, which are position-based and not timebased, meaning that the planned altitude is compared to the actual flown altitude at the same position. The precise cause of the delay has not been investigated, as we are interested on in-flight inefficiencies i.e. a flight taking longer than planned, rather than looking at sources of delay occurring on the ground (minor maintenance, airport congestion and slots delaying the departure or other). This particular flight has matched its planned duration well. These comments have been added to the text.

(4.) The authors may wish to more explicitly engage with / look into recent related work by Marek Travník and R. John Hansman on flight efficiency, congestion, and trajectory-based performance analysis. Although the work from Travnik and Hansman relies more heavily on satellite-based aircraft trajectory data, there may be important conceptual and methodological commonalities that could be leveraged here. Recommendation: Revisions Required

Response

The paper "Preliminary Risk Analysis and Fuel Benefits of High-Altitude Free Fight Regions" by Mina Cezairli, ..., Marek Travnik et al. has been added as a justification to the importance of the en-route vertical flight efficiency. Thanks for the hint, other publications by this author will be assessed.

Review - round 2

Reviewer 1

1. Please update the manuscript’s reproducibility statement section with a DOI or URL of the repo. This is mandatory for JOAS papers.

2. The final section of the paper, "Outlook" could be updated and turned into "Conclusion". Currently, the paper reads as if it is not complete. I understand that the research is still ongoing. However, it might be better for readers to have a clear overall understanding of what has been archived at this stage.

3. Figure 3 is a bit hard to read. I suggest using one subplot for each aircraft type

4. Figure 6 should use three distinct colors. Currently, the two blue colors look very similar

5. small correction: openAP -> OpenAP

6. In the Author contributions section, use the authors’ names (instead of "first author", etc)

Reviewer 2

Reviewer 3

Response - round 2

Response to reviewer 1

1. Please update the manuscript’s reproducibility statement section with a DOI or URL of the repo. This is mandatory for JOAS papers.

Response

A link to the git repo has been added.

2. The final section of the paper, "Outlook" could be updated and turned into "Conclusion". Currently, the paper reads as if it is not complete. I understand that the research is still ongoing. However, it might be better for readers to have a clear overall understanding of what has been archived at this stage.

Response

The outlook section has been renamed to conclusion and the level of achievements at the time of the paper has been clarified.

3. Figure 3 is a bit hard to read. I suggest using one subplot for each aircraft type

Response

The figure has been updated with a better scale that enables better readability of the behaviour of the different aircraft types, despite maintaining a single figure.

4. Figure 6 should use three distinct colors. Currently, the two blue colors look very similar

Response

The colors have been updated for increased readability.

5. small correction: openAP -> OpenAP

Response

Fixed, thanks.

6. In the Author contributions section, use the authors’ names (instead of "first author", etc)

Response

Fixed, thanks.

Response to reviewer 2

Response to reviewer 3