OF TEST PILOTS AND THEIR LITTLE UNDERSTOOD WAYS

TEST PILOT

Ten thousand grand for a dozen g's And no one can take a thirteen of these And all I do for this princely dough

Is climb as high as the ship will go, And dive till the needle hits the pin

In the gadget that shows how fast I've been.

Then throw her in reverse control To see if I and the ship stay whole.

If the wings stay on and the fittings tight, And the elevators function right,

And the ailerons don't flutter away,

Or pressures buckle where pressures may, I climb again with a like intent

To prove that it wasn't an accident.

Pick up g's with a spinning fall To see if she's weak that way at all. I loop for balance and stall for glide

And whipstall hard with flaps out wide. I spin wheels up and spin wheels out And spiral and slip and slam about

In landings rough as the tank can stand To prove each step the designer planned.

Then I taxi into an open ditch And don't get paid for the sonofabitch.

- Anon¹

This article has been a long time in the making. Realization dawned early, and then repeatedly over the course of 26 years, since acquiring an obscure ‘ETP’ (Experimental Test Pilot) suffix in the Navy List, that the ‘System’ never really understood much of what test pilots did or ought to do for their board and keep. To paraphrase a pithy life lesson handed down by one hoary old Chief Test Pilot at the Indian Air Force’s ASTE (Aircraft & Systems Testing Establishment), “You will rarely be understood in the wider Service. Most of those you interact with will dislike you for your persistent questioning of everything. Yet, beware those who venerate you, accelerating you towards mistakes born of hubris. Those guys will kill you!”

The Test Pilot (TP) cadre is a rarely seen and little understood element buried within the folds of the Fleet Air Arm, whose overall cadre management itself is an ever-evolving conundrum. Therefore, on the cusp of an impending resurgence in carrier aviation, giving voice to the “Test Pilot” cadre is clearly in the best interests of the Navy. Having personally enjoyed the hedonistic privilege of twenty additional years in the cockpit, over and beyond the frontline aviator’s typical flying career and given free run of three major aircraft programmes,² some debts are owed. Passing a few insights down the chain of my successors while spreading the word to the uninitiated, might yet earn some redemption.

We have always aspired to be a “Builder’s Navy”, a concept which has got a second wind in recent years, at least with regard to surface ships. However, there has never been a corresponding impulse within Naval Aviation, largely due to the absence of any serious indigenous contenders hitherto. Unfortunately, just as viable indigenous options have begun to emerge, a fair degree of disdain for such programmes has crept in, fuelling a bizarre race to the bottom. While the Navy justifies its circumspection on the low prioritisation for naval programmes by indigenous aircraft design agencies, they in turn point to the Navy’s apparent soft corner for imported aeroplanes/rotorcraft.³ Surprising indeed, considering that the stellar availability of indigenously built aircraft⁴ during the post-Pokhran sanctions ought to have convinced even the most die-hard of the utility of indigenous capability, even if it was of the build to print variety.

Notwithstanding this tussle, geopolitical turmoil and the obvious drawbacks of dependency on imports to this degree, could—and should—deepen the commitment to indigenous aircraft for the Navy. Whole-hearted backing for the Twin Engine Deck Based Fighter (TEDBF) and Naval Multi-Role Helicopter (NMRH) programmes could thus crystallise into reality faster than the weaknesses plaguing indigenous programmes could be made good. Criticism of indigenous efforts thus threatens to turn into a self-fulfilling prophecy wherein mistrust of indigenous aircraft development actively prevents investment in vital long-gestation ingredients, particularly those reliant on high-value human skill sets, thereby contributing to their subsequent failure. With an already simmering debate about the future relevance of aircraft carriers, this muddle can only get worse for the Navy. What could be worse than a hard-fought recapitalisation of embarked aviation being authorised, only to discover that the tools necessary to implement it have been allowed to wither?

This article represents lessons learnt through the lived experience of a TP with an extended career in the Indian Navy. While it sacrifices some of the studied formality usually associated with this publication, it is hopefully not at the cost of being dismissed as frivolous. It also eschews the “more” correct terminology of ‘Experimental Test Pilot’ or ‘Flight Test Crew’ in favour of a simpler “Test Pilot.” To the Flight Test Engineers (FTE) out there, especially those comrades in arms who kept me safe through an extended testing career, please do forgive the kneading of FTEs into the folds of this generic term.

The Nature of the Beast

In trying to understand test pilots, one can do no better than to inspect this closely guarded guild’s eligibility criteria. Rather than attendance of any qualifying course, the Society of Experimental Test Pilots recognizes “quantum of practice of the flight-testing profession”⁵ as the sole criteria for eligibility. Actual participation in test programmes, irrespective of prior qualification, allows unfettered access to full membership and beyond. Conversely, mere attendance of a formal flight test course only entitles one to a time-limited associate membership. The entry barrier is further raised, based on the type of flight testing practiced, with only “Experimental”⁶ or “Developmental”⁷ flight testing considered qualifying for primary membership. All other forms, like ‘Production Testing’ of new build follow-on airframes or testing of minor ‘Engineering Changes’ are limited to associate membership. Contrary to widespread belief, the standard in-service “Air Test” following major maintenance is not considered true flight testing.⁸

Acceptance of this understanding, though contrary to what is commonly believed, clearly places test pilots at the heart of the aerospace vehicle design industry and any form of pathbreaking aerospace research. That Orville & Wilbur Wright were the first to succeed at heavier than air flight, despite many who tried before them, is surely not unrelated to their fastidiously detailed and methodical test programmes.⁹ From the world’s first structured usage of a wind tunnel to their methodical ‘build a little, test a little’ flight development programme, any modern TP would instantly recognise the kernel of his trade.

Unlike these totally integrated origins, the modern aerospace industry is clearly delineated into “creators” and “users.” Despite being deeply embedded within the ‘creator’ side, modern test pilots originate inside the ‘user’ community of each specific branch of aerospace endeavour (transport/combat/surveillance etc.). Straddling the engineering world of ‘build’ and the rough and tumble of ‘operate’, test pilots inevitably adopt the mantle of translator/interpreter for all conversations between these communities.

The Atmanirbhar Dilemma¹⁰

After years of enjoying the dubious distinction of being amongst the top arms importers,¹¹ the ruinous economics and strategic risks of importing weapons and restrictions on key technologies have finally made India serious about indigenous aerospace creation.¹² Very tellingly, this drive is not aimed at acquiring some unique warfighting capability currently unavailable on the global market.¹³ Introspecting on why we still covet imported weapons inexorably leads to the “shortfall in indigenous capabilities” trope, while the pushback alleges that similar shortcomings are tolerated in imported systems. Clearly, our deep-rooted longings originate elsewhere.

The allure of any widely admired product often lies in the delicate balance of engineering compromises made by its creators. However, these are hard to see for anyone unfamiliar with the world of practical engineering. Investigating this train of thought would most likely identify a widespread lack of appropriate “Systems Engineering” and “Design” attitudes in our indigenous consciousness. This reacts particularly negatively with our other unfortunate cultural predisposition which is to silo any activity as per legacy job descriptions. Inevitably, this leads to institutional neglect of multi-disciplinary skill sets and activities. Words like ‘certification’ and ‘integration’ take on negative connotations and the requirements definition process is reduced to a collation of single line specifications independently inserted by each stakeholder.¹⁴ Naturally, this reflects as intolerance of compromise on our indigenous systems whereas, for imported systems, we seem blissfully unaware of the wise compromises made on what we consider great weapons.¹⁵ It also makes it particularly difficult to implement disruptive ideas as these most often exploit cross-discipline application of extant technologies.

One particularly alarming characteristic, peculiar to the Indian MoD customer is that it has become increasingly acceptable to aver “our job is to defend the nation, not build weapons” and the capstone “we want robust products not research projects.”¹⁶ While that may be understandable in the throes of a combat operation, it is quite unacceptable if we truly believe that military hardware needs to be built rather than bought. This, in any case, clearly runs counter to global best practices which underpin their admittedly superb products.¹⁷

Even the most cursory dip into Systems Engineering lexicon will point to the ‘V’ diagram depicted. And onwards to the inescapable conclusion that, for good design of major systems, nearly half the ‘V’ process needs to be conducted by the customer himself and not the designer. In fact, the first three steps depicted have spawned a whole new discipline of ‘Requirements Engineering’ which only an enlightened and fully engaged customer can execute. This is the stage where test pilots are perhaps most critical to the process as is argued further in the article.

Most will recognise ‘Verification’ as requiring structured testing and precise measurements cross-referenced against the customer’s requirements. Repetitive invocation of ‘Verification’ along the right half of the ‘V’ clearly makes the case for subject matter experts in the field of testing, ergo TPs. The Indian MoD customer is quite inclined to view intermediate ‘verification’ testing as only of interest to the boffins, satisfying himself with sporadic reviews of the results by inhouse ‘translators.’¹⁸ But these too need to be TPs, even if handicapped by their own lack of deeper participation in the programme.

When early negative results inevitably spark a war of words between system developer and customer, the vocabulary liberally employed by both sides default to the words of the translator/interpreters/prognosticators! With weighty decisions riding on their words, their credentials with both sides of the spectrum need to be impeccable. This, in essence is the origin story of the modern Test Pilot!²²

So, Then What?

With this background one could easily mistake the role of the TP to be narrowly confined solely to testing. Not so evident is the role of the TP as a developer in his own right. Particularly true for the Indian context, he is normally the only person in the design room to have experienced the scenarios and flight conditions being discussed in addition to being the only one with holistic practical experience of the operational roles being designed for. It is only to be expected that many design ideas have, and will continue to, originate from test pilots.²³

The benefits of full-cycle deep involvement will become easier to understand when we graduate from being satisfied with token indigenous content, towards truly leveraging mastery over technology for combat success. In times of crisis, advanced nations regularly explore radical technical options at breakneck speed for operational answers. Obviously, this requires creation, testing and fielding of a novel capability at breakneck speed e.g. the ‘Super STOL’ modification of a C-130 Hercules to enable landing and take-off from a football field in Tehran.²⁴ Such opportunities only arise if one has created deep scholar-developer-operator bench strength. The daily drip feed of videos from the Ukraine conflict only serves to validate this, albeit at comparatively elementary levels of system complexity.

Looking towards an increasingly unmanned/autonomous future, it is easy to see the continued or even increased relevance of the ‘Test Pilot’ when his role is framed in such systems engineering vocabulary. Even for short term outright purchases, most of the steps outlined are equally relevant even for outright purchase of an aircraft. As the saying goes, ‘even to buy an aircraft correctly requires one to know how to build one!’

The role of the Test Pilot as a developer, designer and systems engineer in indigenous designing should not be viewed as a digression. Rather, it is the rampant ignorance of the key principles of Systems Engineering and its embedded offspring, Requirements Engineering, that explains most of our indigenous development problems. Visions of a vibrant ‘Atmanirbhar’ India are unrealistic without these vital ingredients.

To Make a Test Team

“Concevoir une machine volante n'est rien; Fabriquer est peu; L'essayer est tout” ²⁵

Ferdinand Ferber

French aviation pioneer (1862-1909)

The curriculum of the modern Test Pilot is unlike any other academic course particularly in the military system of education. For starters, selection for the course is preceded by a comprehensive theoretical, practical and oral exam to identify pilots with the right aptitude and approach to flying. Apart from teaching operational pilots an engineering vocabulary, the formal course content is largely based around set-piece flight test techniques that have developed over time.

Continuous and comprehensive evaluations are conducted during which the real learning occurs: the art of figuring out how to acquire knowledge rather than accumulation of a specified set of facts. No inference or conclusion is allowed to go unchallenged or accepted as axiomatic. More than the absolute accuracy of parameters measured, the logical train of inferences to identify operational implications and resulting recommendations for achieving of a designated role is emphasised. None of this is possible without the paperwork to reduce all such thoughts to a highly stylised train of relentless logic.

Having endured such a curriculum bordering on epistemology, the graduate is expected to be supremely confident of being able to get you the answer to any question, if not already possessing it. Tribal knowledge avers that the most dangerous test pilots are those who already know the answers to everything. The true hallmark is one of relentless scepticism and pride in learning something new every day. Relentless questioning of all shibboleths inevitably leads to awareness of more exceptions to any simplistic rule than adherents. Not for nothing are test pilots cautioned against appearing for ‘gen-tests’ designed for lesser mortals when one knows a little too much to accept simple True/False dichotomies. Then there is the joke about them always beginning any answer with “depends….”. When one takes an already insufferable fighter pilot²⁶ and turns him into a test pilot, he acquires the smug confidence to state that if he can’t understand what you are saying, then it’s your inability to explain, not his lack of comprehension!

But as mentioned earlier in the context of SETP membership, the qualifying course is hardly considered open license to access the full range of industry flight test work, just as an undergraduate course in medicine or surgery is unlikely to be considered qualifying for handling the most delicate medical cases. Only with years of experience of multiple programmes for various operational roles and of differing complexity, can one claim the right to exercise influence over a clean-sheet design or weigh-in on critical decisions about the future of any air vehicle.

It is in this regard that the Indian flight-testing milieu deviates significantly from global practice. Despite the ASTE flight test course being indistinguishable in content and quality from other schools worldwide, paucity of major programmes denies most Indian test pilots the opportunity to practically acquire the necessary credentials.²⁷ The bulk either quickly lapse back into the wider service to preserve their career prospects or at best take on administrative oversight roles for acquisition/modification programmes. In this author’s experience as Chief Test Pilot of the National Flight Test Centre at Bangalore for over a decade, re-training of already ‘qualified’ test pilots to ready them for entry level testing tasks on the LCA programme took approximately a year in good times and at least twelve months in other cases! Only then were they equipped to offer definitive opinions in an unsupervised environment.

What Do They Do?

Aerospace programmes tend to be expensive enterprises with an ever-present risk of hull loss. During initial development, the risks are further accentuated by an overall lack of validated data. With modern complex systems, the inter-relationships between various system functions can have many unintended consequences yielding fault mechanisms that are very hard to anticipate.²⁸ Similarly, the deterministic nature of older systems where one switch operation only led to a specific action, has long since been replaced with context variant responses due to complex algorithms. As AI driven systems proliferate, this would be further replaced with probabilistically varying responses requiring supervision for proximity of risk and understanding / endorsement of the complex strategy chosen by the AI. Testing such complex systems usually falls into one of three bins:

• Testing that desired functions occur as they were designed to,²⁹ as and when commanded. Techniques to accomplish this form the bulk of traditional test pilot course syllabus. Surprisingly, these take up only a small fraction of a Test Pilot’s attention on the job. The bulk of test artefacts in this category tend to be of aeromechanical nature or simple systems controls sans complex software.

• Demonstration of the desired performances/capabilities specified for the programme, typically related to kinematic behaviour in all six degrees of freedom. This usually follows a slow build-up of risk, commencing from an extremely safe spot in the flight envelope and proceeding incrementally towards the maxima/minima sought. Mandatory margins above the defined safe limits also often need investigation by test pilots. The techniques to achieve this are relatively easy to teach but, lacking the pucker factor, incredibly hard to learn in a training course, since one already knows the limits of the aircraft. Hence the skill set for this category is only learnt on the job over several programmes. Safe progress is assured only if the incrementally experienced behaviour matches that which is expected. To develop the necessary anticipation of what is appropriate, a deep understanding of the design is essential. Achieving this level of empathy is directly impinged upon by a surprisingly common misconception that Test Pilots are needed only in time for the vehicle to take flight and hence their presence can be confined to the flight phases.³⁰ Nothing could be further from the truth. While there is a heavy “flight envelope” bias, implicit in this observation is that most modern platforms use complex software controls to achieve most aeromechanical envelopes. Hence testing in this category often requires investigation of complex control systems rather than purely classical aeromechanics/dynamics.

• Testing that ‘undesired’ behaviours do not occur when they are not tolerable.³¹ This is far easier said than done for complex systems, as the task is akin to proving that the haystack does not contain a needle! “You can only see what you understand” is my personal flight test twist on the old Judaic saying “You only see what is in you”!³² The level of understanding needed to truly achieve this category of testing borders on empathy and resonance with the design concept in its entirety. The deep chemistry needed is achievable only if the tester has participated in the design process from genesis and lived through its various iterations including understanding of the stubs remaining from abortive efforts. The value of such deep involvement is clearly demonstrated by the accident-free history of the LCA programme as compared to every other aircraft development programme in India’s history!³³ Aircraft under test in this category tend to be preponderantly software-heavy systems with great emphasis on multi-system interaction and complex adaptation of operating logic to circumstances. With AI pushing its way in, it is self-evident that testing of this category is about to explode in its complexity!

Even so, modern systems tend to offer huge amounts of internal data during testing which feed into computationally powered design and data analysis tools. Therefore, it is tempting to construe that the role of human observation has significantly reduced or been eliminated in modern flight test. In fact, today, design teams are snowed under mountains of data. While it is often trivial to extract and process these for the predicted results, it is the search for anomalies which continues to pose problems. These cannot be predicted (otherwise they wouldn’t be anomalous to begin with!) and hence need to be tracked backwards by correlation with particular event sequences or by association with other coincident phenomena.

Since the Test Pilot occupies the seat closest to any flight event, he is typically in a good position to contribute richly to post-facto data analyses. One must therefore expect the highest standards of fidelity and credibility from this tribe when it comes to their observations as no one else can fill this role. Obviously, the rapidity with which events occur in real-time prevent deep on-the-fly opinion formulation. Instead, test pilots are expected to be exceptionally observant for all occurrences and comment insightfully on these observations, as they occur. With reams of data available offline, it is these observations or even simple annotations of specific time instances that are of irreplaceable value during subsequent data analysis. Often the anomaly is not measured directly by any single parameter and requires significant processing of non-obvious parameter combinations to tease out the truth. Accordingly, good test pilots are expected to be extremely loquacious in the cockpit and develop a rich repertoire of adjectives to describe different types of vibrations, thumps and burbles!

With modern test programmes employing real-time analysis techniques, a lot of discussion ensues in-flight between test points, with the TP having to take the final call with regard to continuation/adjustment of the subsequent test plan. Contrary to the Hollywood stereotype, a test pilot’s finest moments usually take the form of a prescient, but anti-climactic mission abort, whose real import becomes apparent only when data are analysed. Post-facto fear in the data analysis room is a novel experience reserved for the privileged few.³⁴ It is this singular role, the responsibility to look ahead, that forms the basis of the modern Test Pilot’s enduring mystique. For, besides serving as a true measure of his skill, it also serves as the fount of the Test Pilot’s veto power in a programme.³⁵

The test process executed thus far can obviously be expected to throw up problems that need resolution, a task possible only if the underlying cause is correctly identified. That the symptom is not the problem, or in formal Bayesian speak, “Correlation does not imply Causation”³⁶ forms the essence of most robust conversations with the design teams and programme management. The ragged edges of the Test Pilot’s academic qualifications often begin to show in such discussions with the Boffins. Hence, rather than duelling with formulae, the wise TP is best advised to take on the role of ensuring that all the facts add up to a plausible hypothesis. It is here that the Test Pilot’s second trump card comes to the fore – his cross-functional knowledge and thought processes, since virtually all the others in the room would be single domain experts who often err by conjuring up unique and increasingly exotic explanations for each symptom individually. Even in the presence of the other cross-functional participants viz. Integration and Certification teams, he is advantaged in possessing a hands-on feel for the problem.

Once the synoptic data has been analysed, the Test Pilot is expected to take the lead in prognostication of the implications of these results with respect to achievement of goals (look ahead at ‘validation’). This is vital for understanding by the programme management as they are technically/operationally challenged by their familiarity and proximity to the technology. Finally, when it comes to recommending solutions to problems, the Test Pilot is expected, like any other member of the design team, to contribute towards ideation of feasible options. Here again, the Test Pilot is expected to offer unique perspectives due to his cross-functional mindset, allowing solutions to be implemented in domains different from the problem while also setting the limits on acceptability of prospective solutions or limitations being proposed to the customer. This remains a high-value uniquely non-transferrable task. It is often in these situations that he realises how alone he is – when the designer proposes a sub-optimal solution, and the customer is inclined to accept it due to schedule and /or cost constraints despite his protestations!

In short, the Test Pilot is expected to be impeccable in the fidelity of his observations; have really good instincts for prognosticating implications; bring a holistic approach to analysing underlying causes; and, when discussing solutions, try and keep everyone honest while adding his own original ideas to the fray! It should be no surprise if this reads like the idealised description of a Systems Engineer, because that is what they ought to be, in spirit if not in letter.

The Test Pilot’s responsibilities do not end here as there are many aspects of the aero-space development and procurement processes that benefit significantly from his unique perspectives. Typically, when major decisions are to be made about the future of a programme, it is only the TPs who have actually experienced the developmental article first hand. The bulk of awareness for the rest is via discussions during reviews which naturally gloss over positive achievements and dwell deeply only on the problem areas. The oversight chain, often and repeatedly, needs to be persuaded to see the big picture by the only person in the room who has experienced the positives. Naturally, effectiveness in this role hinges upon the individual TP’s credibility and reputation!

Extrapolating this responsibility, it is natural that the best salesperson for an air vehicle is its Test Pilot. Apart from credibility based on ‘been there – done that,’ he is capable of practically demonstrating the capabilities he claims for the product. The ultimate distillation of this persona is the display/demonstration pilot where, unlike the gloom and doom atmosphere of a programme review, he has a clean slate and your rapt attention to show what he wants to – the strengths of the aircraft! It is thus indeed a travesty that we have allowed the LCA Tejas to be displayed to prospective customers by the operator (IAF) rather than by its developer.³⁷

Square Pegs…… The Test Pilot Personality

Unlike all other military specialisations, the Test Pilot is uniquely exhorted to achieve individual excellence. As opposed to Chetwodian allegiance³⁸ to some ‘other’. One can therefore surely anticipate eccentric mutations of the strong personality traits of the gene pool from which they are drawn.³⁹ Yet, much of this deviance is to be expected, given the unavoidably high-risk nature of the job. Unlike most other dangerous professions, it is the individual’s knowledge and actions that determine the actual level of risk rather than a probabilistic roll of the dice. Danger takes on a much more individualised form that can be combated only by a lonelier type of courage. Knowledge and its companion, imagination, need to be leveraged for mere survival more than success. Forced obeisance to the gods of physics and algorithms is thus ensured by the core job description of proving/disproving whether something actually works and then backing it up with data, rather than just be taken at one’s word. Given the type of education he has acquired, the Test Pilot is denied the immunity card of “how can you expect me to have known that?” It’s, therefore, not without reason that two-thirds of the SETP’s code of professional ethics⁴⁰ relates to Integrity and Accountability.

Worship of any lesser understood god is normally feasible only by withdrawal into a monastic sect to pursue one’s own higher truth. Unfortunately, the Test Pilot is positioned precisely at the interaction boundary of multiple interest groups, Tech /Engg /Ops /Finance /Foreign Military Industrial Complex etc., each powered by humungous financial muscle and the inertia of legacy processes. As an axiom, when the going gets tough, the Test Pilot spends more time in the board room than in the cockpit. Hence any thoughts of a cloistered existence are rudely denied. Each of these interest groups marches to its own piper and so their contrast with the core Test Pilot ethos permeates every interaction.

One key distinction that colours several interactions is that the TP usually knows more about the overall subject than any of these interest groups individually.⁴¹ This is usually based on its novelty and the Test Pilot being forced to the lead in understanding it by virtue of being the one who must actually operate it. Even when dealing with relatively proven technologies, he is still usually more cross-functionally aware, and even when dealing with other Test Pilots outside the programme, much more deeply immersed in the specific process. Hence an inverse knowledge gradient in every interaction is inevitable, running generally counter to the hierarchical grain. The contrast is particularly acute with operational military command chains which are otherwise based on knowledge advantage as one moves up the hierarchy. Constant friction is therefore guaranteed,⁴² with even the stylised language thrust upon by the job itself becoming a source of conflict!⁴³ Unless continuously lubricated by a widespread understanding of the Test Pilot personality and his role in the larger scheme of things, the friction takes its toll. Either the round hole gets swaged into acceptance of the square peg, or the square hole forges splines onto the round peg. Virtually all accidents during development can be squarely blamed on malignant imbalance due to one side or the other overplaying its archetype. Heresy is usually tolerated only in extraordinary times e.g. the frenzy of combat. But the Test Pilots operating stage is nearly always a peacetime setting, well back from the frontline, where such energy is usually deemed ‘inappropriate.’ Even so, the messianic zeal powering these heretic Test Pilots needs to be tolerated, solely because it draws its energy from the potentially huge opportunity cost in lives on the frontline. Success in operations is directly at stake if one fails to properly execute a test programme or push for necessary changes. Unfortunately, those losses are all in the future tense and it is very easy for a test pilot to be typecast as a prophet of doom. My usual Freudian malapropism is to introduce myself to new audiences as a “professional cynic by training” instead of “sceptic”!

And yet TPs are equally often the only ray of sunshine in the room, being inherently equipped to see an issue in its entirety and offer the most balanced views. It certainly helps being the only ones to have sampled the ‘good’ when embroiled in discussions usually dwelling exclusively on the ‘bad.’

Why Does the Indian Navy Need Them?

A good way to round out this ode to my calling, would be to answer that question. Circa 2010, I was tasked to comment on a proposal floated to form a Naval Flight Test Squadron. Even at that time, the quantum of flight test and aircraft development related work looming ahead was growing ominously. For the first time major aircraft upgrades had been attempted by the IN and two such programmes had been successfully executed.⁴⁴

On the rotary wing side, the benefit of captive Ship-Helicopter Operating Limits (SHOL) clearance ability was beginning to be appreciated.

Hence, while the ‘capability’ requirement was clearly becoming understood, the growing ‘capacity’ crunch was sought to be resolved by a prescient proposal to formalise a flight test organisation within the Navy. Since then, the Naval Flight Test Squadron has come to fruition and a comprehensive Navy Order has been promulgated to streamline organisational issues and processes. Given that it has been functional for approximately seven years since its commission in 2017, it would be fitting to stir some introspection on how we have progressed with a suggested list of questions. Given this author’s distance from the Navy’s currently prevalent situation, it would be incorrect to construe these as leading questions. Rather they should be viewed for what they are: - a dispassionate attempt to lay down a “mid-term review” with professional integrity:

• Have we been able to adequately influence/manage the massive influx of new ship-classes all of which would have needed the benefit of good design for adequate SHOLs? Meanwhile, have we also been able to deal with the recurring SHOLs task for the older classes?

• Have we been able to handle the induction of several new helicopter launched weapons programmes with the necessary level of test pilot input?

• Is our bench strength correctly poised to guide/steer/manage the emerging opportunity to design a good ship-borne multi-role helicopter, specifically with regard to implementing lessons learnt from the ALH programme?

• Has our flight-testing capability acquired the strength to support the Navy’s multi-carrier programme ambitions? This would potentially have to simultaneously handle 65000-ton leviathans populated with an indigenous TEDBF and armed with a wide array of new-build indigenous weapons. We currently have a once in a century opportunity to develop a highly optimised ‘aircraft carrier weapon system’ with simultaneous coordinated development of all three major constituents viz. ship-aircraft-weapons. Of all the ingredients needed to realise these dreams, there is only one that no other agency besides the Navy can provide: TPs. For everything else, enhanced financial outlay can solve most problems. Hence this question is particularly critical to fulfilment of our ambitions.

• Have we been able to solve the core dilemma affecting the test pilot training pipeline viz. the double-bind of achieving the mandated qualifying flying experience within the age/seniority bracket appropriate to a viable Naval career? Hopefully the temptation to dilute these criteria has been resisted. While that may give the appearance of success in the short term, it ignores the rest of the follow-on testing experience bucket that will still need to be filled to make a competent test pilot. One can potentially buy the TP course but not the rest of the ingredients. The true reckoning of any such compromises will come when any such test pilot weighs in on some weighty decision e.g. recommends cutting up the deck of a carrier to modify the lift shape so as to avoid some performance constraint on the aircraft. At that stage, the shortfalls of a lightweight resume’ would permit even the most sagacious advice to be roughly dismissed as coming from an insufficiently experienced Test Pilot. While these may be construed as unduly rough words, they are meant to point out the pitfalls of any attempt at affirmative action and also perhaps to ease the way for any specific individuals who would face these challenges. In any case such a sidestep, tempting as it seems, doesn’t offer any pathway to the minimum sustainable structure which necessarily requires test pilot oversight of other test pilots’ work. A good example exists in the form of the IAF which has ensured sufficient strength to create qualified oversight gradients, besides possessing all the basic ingredients to sustain a viable flight-testing pipeline, including a school and a dedicated testing organisation.

• Has the option of borrowing test pilots temporarily from the IAF been adequately debated and laid to rest as unworkable? If not, besides amounting to a negation of all that has been laid out before, has the IAF committed to the additional burden despite their own extensive needs (LCA Mk 2, AMCA etc)?

• Has the IN been able to create spare test pilot capacity to explore the remaining scope of flight-testing roles e.g. evaluations prior to purchase or acquisition and accident investigations?⁴⁵

Hopefully this article will trigger a comprehensive review of the existing situation, to be followed by that most worthy of Test Pilot activities, continuous iteration for improvement. Only then would growth become sustainable/cumulative and Navy flight test become a learning organisation. While wishing the Navy success in creation of the pillars to support its vast ‘Atmanirbhar’ ambitions, these thoughts hopefully should at least provide an insight into the workings of a Test Pilot’s mind. In many ways, a worldview based on the premise that the centre of the universe lies right between the Test Pilot’s eyes.

Endnotes

1. Exists as a scrap from some magazine many years ago. Remains anon despite strenuous efforts to trace its origins

2. Namely, “LUSH” - Limited Upgrade of the Sea Harrier (which was anything but limited!), the IAF’s LCATejas and the Light Combat Aircraft (Navy).

3. The case of the Advanced Light Helicopter is particularly instructive in this regard. Very public disparagement by either side, as exemplified by the following series of articles in the Livefist blog, only served to distract from the inescapable work necessary to make such programmes a success. Yashodan Marathe, “The Truth Hurts, Says Indian Navy’s 1st Dhruv Flight Commander,” Livefistdefence.com, 10 June 2020, https://www.livefistdefence.com/the-truth-hurts-says-indian-navys-1st-dhruv-flight-commander/; Anil Bhambani, “Navy-Dhruv Spat: Let’s Stop Fighting, HAL Test Pilot Says,” Livefistdefence.com, 11 June 2020, https://www.livefistdefence.com/navy-dhruv-spat-lets-stop-fighting-hal-test-pilot-says [Author’s Note: This has disturbing parallels in the LCA(Navy) saga wherein, every Navy Day CNS interview unfailingly and very publicly criticised what should always have been a technology development programme, all the while sparing the MiG-29K boondoggle any similar treatment]; Abhijit Singh, “Why the Indian Navy Is Unhappy With Its Carrier-Based Light Combat Aircraft Project”, The Diplomat, 03 Dec 2016, https://thediplomat.com/2016/12/has-the-indian-navy-canceled-its-carrier-based-light-combat-aircraft-project/; Comptroller and Auditor General of India, Report of the CAG for the Year Ended March 2015: Union Government (Defence Services) Navy and Coast Guard – Report No. 17 of 2016, 54-62, https://cag.gov.in/webroot/uploads/download_audit_report/2016/Union_Defence_Services_Na vy_and_Coast_Guard_Report_No_17_of_2016.pdf .

4. Dornier-228, Chetak and Kiran, as compared to the heavily sanctions afflicted frontline of Sea Harriers and Seakings.

5. The Society of Experimental Test Pilots, Standard Operating Procedures 4-4: Instruction to Membership Applicants, 16 February 1972, https://www.setp.org/images/Initial_and_Upgrade_ to_Member_Instructions.pdf.

6. Experimental flight test requires that the aerospace vehicle itself is the article under test, either due to the novelty of its overall design, underlying engineering concept or the conditions in which it is to be flown. The basic ability of the machine to fly should be in question for it to qualify as “experimental”. This typically covers 1st flights, envelope expansion, initial performance or flying qualities testing of new or significantly modified vehicles.

7. Developmental flight test requires that some major system of the aircraft be the ‘article under test’, not the aircraft itself. This typically covers structural changes, software changes, weapons testing etc. or systems tests where the system is under development prior to fielding.

8. The new Navy Order on Flight Testing (NO 13/2023), does adopt a much broader umbrella in defining forms of flight testing. Yet, buried in the details are mandatory qualification criteria for specific types of tests which effectively narrow the definition to attendance of a formal flight test course. Bewilderingly, some vital “on ground” activities, which ought to be the exclusive preserve of only highly qualified experimental TPs, are left open to PTPs. The new Navy Order (NO) definitely deserves a whole workshop format discussion, drawing in all forms of experience to make it more robust, implementable and in tune with global industry practices.

9. National Aeronautics and Space Administration, NASA publication EG-2003-02-007-GRC: Learning to Fly – The Wright Brothers’ Adventure, https://www.nasa.gov/wp-content/uploads/2009/07/58225main_Wright.Brothers_508.pdf.

10. “Self-reliant”. Refers to a Govt of India initiative to achieve self-reliance in industrial capacity and technology

11. Pieter D. Wezeman et al., “SIPRI Fact Sheet: Trends In International Arms Transfers, 2023,” Stockholm International Peace Research Institute, March 2024, https://www.sipri.org/sites/ default/files/2024-03/fs_2403_at_2023.pdf.

12. Ministry of Defence, “Defence Production and Export Promotion Policy-2020”, Press Information Bureau, 03 August 2020, https://pib.gov.in/PressReleasePage.aspx?PRID=1643194

13. Ministry of Defence, “Defence Production and Export Promotion Policy-2020.”

14. The settings/reference material provided for the perennial Exercise “Jahaz Banao” at the Defence Services Staff College clearly demonstrate how insidious the problem is.

15. The glaring lack of wing fold on a Rafale-M offers a classic example system of systems engineering driven solutions, with ‘compromise’ on the aircraft to avoid manoeuvre performance penalties while shifting the alleviation onto another more tolerant system viz. lifts dimensions on the aircraft carrier PA Charles de Gaulle.

16. While few would officially state this so explicitly, it is true that every invocation of “atmanirbharta” is also reflexively caveated with assertions that undermine the core commitment to a Builder’s Navy viz. that one cannot/should not ‘buy’ any critical capability. Be it statements in Parliament or the IN Strategy document, every official invocation to military preparedness unfailingly keeps the door to “buying” weapons open thereby implying that one can indeed “buy” worthwhile military capability. It is also widely believed that acquisitions process tactical considerations shape QRs as much as the capabilities sought. RK Bhonsle, “Indigenous Weapons Programmes Need Armed Forces Support”, United Services Institute, 29 August 2017, https://www.usiofindia.org/strategic-perspective/indigenous-weapons-programmes-need-armed-forces-support.html; Indian Navy, Naval Strategic Publication 1.2: Ensuring Secure Seas: Indian Maritime Security Strategy, NSP 1.2, 2015, 131-132, https://www.ssri-j.com/MediaReport/Document/IndianMaritimeSecurityStrategyDocument25Jan16.pdf [Author’s Note: NSP 1.2 sees fit to repeat the caveat on consecutive pages when dealing with indigenisation/self-reliance/self-sufficiency]; Dhruva Jaishankar, “The Indigenisation of India’s Defence Industry,” Brookings Impact Series 082019 - 01 , August 2019 , 13 , https://www.brookings.edu/wp-content/uploads/2019/08/The-Indigenisation-of-India-Defence-Industy-without-cutmar-for-web.pdf.

17. Edward Luttwak and Eitan Shamir, The Art of Military Innovation- Lessons from the IDF. Essentially the theme of the whole book but especially Chapter 6.

18. In days gone by, customers had the luxury of repeating all the key verification tests with their own crew as encapsulated in Mil-D-8708C, “General Specification for Demonstration of Aircraft….”. This has become simply unaffordable / unviable for modern programmes. Instead, Integrated Project Teams have been the norm for over twenty years.

19. Shawn Wasserman, “Model-Based System Engineering – Beyond Spreadsheets,” engineering.com, 23 March 2014, https://www.engineering.com/story/model-based-system-engineering---beyond-spreadsheets.

20. “Air Force Operational Test and Evaluation Center,” Air Force, August 2021, https://www.af.mil/About-Us/Fact-Sheets/Display/Article/104538/air-force-operational-test-and-evaluation-center/

21. This author was very surprised to hear a 30-year design veteran of a major aircraft program confess that usage of iteration as a tool for success, including questioning the requirements and assumptions, was a completely novel idea!

22. Sadly, not taught explicitly as the reason for their existence, even to Test Pilots!

23. Opinion is often divided on this aspect even amongst TPs with some insisting that they must remain aloof of the design itself in order to retain objectivity as evaluators. But this denies engineering teams their earliest and precious insights into a design problem. Its value has been repeatedly proved historically by the number of early designers who were TPs themselves (the Wrights, Geoffrey DeHavilland, Willi Messerschmitt….).

24. Functioning of the US Joint Test Directorate during Operation Credible Sport, a hostage rescue concept attempted in June 1980, immediately following the failure of Op Eagle Claw, the US attempt to rescue its diplomats from Iranian militias. Dario Leone, “Operation Credible Sport” Y M C - 1 3 0 H g o i n g t o m u s e u m ” , t h e a v i a t i o n g e e k c l u b , 1 1 J u l y 2 0 1 8 , https://theaviationgeekclub.com/operation-credible-sport-ymc-130h-going-to-museum/#google_vignette.

25. Translation: ""To design a flying machine is nothing; To build one is something; To experiment is everything" in Les progrès de l'aviation depuis 1891 par le vol plané, Revue d'Artillerie, mars 1904. In his memoir, The Story of the Winged-S, Igor Sikorsky renders the quote as "To invent a flying machine is nothing; to build it is little; to make it fly is everything". Sourced from https://en.wikipedia.org/wiki/Ferdinand_Ferber#cite_note-13. The origins of this quote proved particularly hard to track down as it is routinely ascribed, mistakenly of course, to the German aviation pioneer Otto Lilienthal: https://www.aphorismen.de/zitat/102083.

26. The bulk of military test pilots are drawn from the fighter stream as it offers greatest flexibility in subsequent employment on test programmes and wider acceptability of uncomfortable pronouncements! With the bulk of all test pilots worldwide being ex-military, this broad-brush assumption is reasonably accurate.

27. With the latter-day democratization of flight test courses via emergence of civilian test pilot schools, one now hears horror stories of fresh graduates launched headlong into major developmental programmes overseas, often as head of flight testing [ Gökhan Virlan| h t t p s : / / w w w. l i n k e d i n . c o m / i n / g ö k h a n - v i r l a n - 3 3 6 0 2 0 8 5 / ] ; A r i f A t e s | https://www.linkedin.com/in/arif-ates-0943a3143/! That is the precise threat hanging over our Navy’s ambitions (TEDBF and NMRH). But that should be acceptable only in emerging aerospace industries, not one with aspirations like India’s.

28. Nancy G Leveson and John P Thomas, System Theoretic Process Analysis Handbook (2018), 4. https://psas.scripts.mit.edu/home/get_file.php?name=STPA_handbook.pdf

29. Leveson and Thomas, System Theoretic Process Analysis Handbook, 36.

30. Not as far-fetched as it sounds. Some outrageous suggestions for Test Pilot availability, bordering on “daily wagers”, had to be fought off on the LCA (Navy) programme!

31. Nancy Leveson, ibid

32. From Sefer Ba'al Shem Tov al HaTorah [https://judaism.stackexchange.com/questions/29326/source-for-the-baal-shem-tovs-idea-of-seeing-ones-own-flaw-in-others].

33. Wholly ascribable to the full-time and long-term test crew involvement mandated for the LCA programme by the responsible agency (NFTC, Bangalore) as compared to much higher turnover, ad-hoc/shallow engagements employed for other programmes. Unfortunately, each of these other programmes has suffered a hull loss during development e.g. the multiple system characteristics and crew awareness related factors leading up to the crash of NAL’s prototype Saras VT-XRM. “Final Investigation Report On Accident To National Aerospace Laboratories, Bangalore Saras Pt2 Aircraft Vt–Xrm At Seshagirihalli Near Bidadi (Karnataka) On 6th March 2009,” is very illustrative (specifically Para 3.1 “Findings”) [Source: Aviation Safety Network, https://reports.aviation-safety.net/2009/20090306-0_SARA_VT-XRM.pdf].

34. “It is only those among His servants who have knowledge, that fear Allah”, Surah Fatir (35:28 Quran)

35. Expected to be confined to elements with safety implications. But then most Test Pilots are no shrinking violets with strong opinions on most aspects of the programme!

36. ‘The Book of Why’, Judea Pearl. The slogan actually sums up virtually the whole book as well as the author (Judea Pearl)’s life work

37. Some may claim that this has the 2nd order effect of emphasising domestic customer endorsement of the product. However, the orientation of an operator does not permit the same level of creativity, flexibility and commitment needed to stand out in a crowded field that includes the biggest players of the global MIC, besides making it very difficult to access the latest of continuous enhancements that an OEM can deploy. In fact, the exact reverse argument could be made - that the product is no longer evolving!

38. “The safety, honour and welfare of your country come first, always…… men you command…. etc.”

39. “Personality Characteristics of Fighter Pilots and Ground Personnel,” Journal of Military Psychology, vol. 30 no. 1 (2018), 70-78, https://www.tandfonline.com/doi/full/ 10.1080/08995605.2017.1420977 . This draws further from the already pre-selected pool of pilots in general: https://ntrs.nasa.gov/api/citations/ 20000105204/downloads/20000105204.pdf .

40. “Code of Professional Ethics, The Society of Experimental Test Pilots, 21 February 2020, https://www.setp.org/about-setp/cope/

41. There is a time and a place for humility, but this is not one of them.

42. The Quick and the Dead – The perils of post-war Test Flying, memoirs of Gloster Company Test Pilot Bill Waterton

43. TPs are expected to lay out recommendations using ‘must’ for essential, ‘should’ for highly desirable and ‘could’ for desirable ones. The obvious lack of deference to authority in ‘must’ often upsets military hierarchies. Or the other trigger word “trivial”- a beautifully precise way of saying ‘mathematically simple’ rather than ‘worthless’.

44. LUSH (Sea Harrier) and the IL-38SD Sea Dragon upgrade.

45. Not touched upon at any length in this paper but an equally vital form of payback to the Navy.