As part of the JALLC’s Programme of Work for 2019, the JALLC Advisory and Training Team conducted a five-day Lessons Learned Management Course (LLMC) from 21 to 25 October at the NATO School Oberammergau, Germany. This was the first time that such a course had been trialled.
The LLMC was attended by 22 students from NATO Command Structure HQs, CoEs and Allied Nations who already had an advanced level of Lessons Learned (LL) expertise. The aim was to prepare students for their duties in the planning, administration, and execution of LL Staff Officer functions and services within NATO.
The course covered NATO LL reference documents and concepts, the NATO LL Portal, how to undertake LL Training for Points of Contact and Key Leaders, and LL documentation for Exercises and Operations.
The LLMC provided an opportunity for experienced LL personnel to enhance and strengthen their knowledge of NATO's LL Capability as a learning organization, by providing a closer look at the LL process. The students also analysed the reasons why LL Capabilities within organizations might not be fully efficient or effective.
Day 3 of NLLC 19 began with two presentations on Lessons Learned (LL) from our Partner nations.
Ms Jane Chirwa, the African Union Commission knowledge management officer, spoke about LL from African Standby Forces and Peace and Security Operations.
She said that in 2016 the AU Peace and Security Council noticed that there was very little documentation on the challenges and successes of operations, particularly the long-running AU Mission in Somalia.
A LL Process, drawing from the experience of other international organizations including NATO, was therefore set up, with guiding principles of early stakeholder involvement, internal (member state) drivers, non-attribution of issues raised and safe spaces for sharing, sharing of information and cyclical feedback loops.
This LL Process has been a success and, for the future, the aim will be to enhance it by continuing to learn from NATO’s approach to LL, as well as learning from internal experiences, and benefiting from LL training.
Colonel (ret.) Yuri Pashchuk, of the National Army Academy, Ukraine, spoke about the approach to improving the Ukrainian Army LL Capability.
The current system had been inherited from the Soviet Union in 1991, with improvements being made as a result of learning from operations in Afghanistan. The system has since changed to meet the needs of modern warfare and the current global security threat environment.
The challenges that the Ukraine faced are similar to those faced by all organizations in developing a LL Capability as have been presented at this event.
The Ukraine’s priority was now to embrace interoperability going forward and a roadmap to get there was produced in December 2018.
The focus of the day then moved to New Technologies.
Mr Stefan Olaru, research analyst at the JALLC, introduced the Panel, reminding the audience of the state of play after the New Technologies Event 2018. He provided a brief overview of progress since then, which was followed by a set of related updates.
The first of these updates was from Commander Dietmar Teufel, HQ SACT Innovation Branch Senior Analyst, who spoke about innovation and exploiting disruptive technologies.
In this context, he highlighted the ACT innovation hub, a network of scientists and academics which was launched in 2012, which can be asked find innovative solutions to complex problems, amongst others through issuing innovation challenges to experts inside and outside of NATO; and ACT’s innovation branch, created in May 2019, which is tasked with matching solutions to ACT’s problems and pain points.
He then gave a brief overview of the Emerging and Disruptive Technologies (EDT) Roadmap. The aim of the EDT Roadmap is to exploit technologies to inform policy changes, inform defence planning, feed future capabilities, and ensure interoperability by design. EDT Roadmap activities are prioritized through five main Lines of Effort.
He noted that SACT’s intention is clear – we should not just talk about New Technologies but should deliver through them.
There were then three short updates from the JALLC’s own research analysts.
Referring to the EDT Roadmap, Mr Stefan Olaru stepped onto the stage again to speak about the LL Enabling Line of Effort of the EDT Roadmap, providing an overview of its status.
He noted that SACT’s call for a new LL toolset in early 2018 had been well accepted and, in response, the JALLC organized the New Technologies Event 2018, inviting representatives from NATO and the Nations, industry and academia, to look at how existing New Technology could potentially improve the NATO LL Capability, and in particular the NATO Lessons Learned Portal.
The next step for the JALLC was to conduct analysis on all the data gathered at that Event which resulted in the identification of a number of challenges and corresponding technology that had the potential to address those challenges from the review of some 112 technological applications and products on the market.
Finally, the JALLC has since sponsored NCIA to collect data from Exercise TRJE 18– a total of 3.9 terabytes – and is using that dataset to investigate the potential of Big Data analysis tools.
Mr Filipe Vieira then spoke about the exploration of LL data using data science tools.
The aim was to consider how to capture and analyse the largely untapped dataset of LL from operations, missions and exercises, and from data already uploaded to the NATO LL Portal including from existing Lessons, documents, and reports.
If machine learning could analyze large amounts of data for keywords and trends, analysis could be done much more efficiently and at the pace needed.
The aim was for the technology to help analysts to analyse, not to have spend time extracting the nuggets of information.
A particular challenge would be to ensure that whatever was developed was replicable for other datasets.
Finally, Ms Mihaela Racovita spoke about the JALLC’s project on NATO Exercise Big Data exploration.
Big Datasets, characterized by volume, velocity, and variety of the data, may contain embedded Observations, Best Practices, and Lessons Identified that are never reported, as well as contextual information about Lessons, and clues about the needs of Lessons Learned users.
The Alliance needs to learn how to access that information as quickly as possible and navigate the challenges in doing so.
In the context of this particular project, the JALLC was considering:
how we can use Exercise data to increase the quality of NATO LL Portal products;
how we can help people access the right LL information at the right time;
whether we can extract tacit knowledge; and
whether we can construct predictive LL models.
Expectations need to be managed; Big Data was not a universal cure.
We will need to learn from experimentation and from our failures as well as our successes.
Colonel Paul Malessa, HQ SACT CAPDEV, then presented the new governance model for NATO’s common-funded capability delivery.
After the Cold War, the urgency in delivering new capabilities reduced and the timescales for capability development have relaxed.
With the current speed of technological development, there is a renewed sense of urgency to develop the capabilities the Alliance will need to meet new and challenging threats at the speed of relevance.
A new governance model has therefore been instituted, which has to be followed from beginning to end to be effective. The value added is a reduction in timescales, clearer responsibilities, greater transparency and flexibility, and full coverage of DOTMLPFI.
There was then a set of presentations providing a range of external, Non-NATO perspectives on New Technologies.
Mr Miguel Pinto Luz, the Deputy Mayor of Cascais, the Municipality of the NLLC 2019 Venue, gave a local perspective on how Cascais has been a Smart City for 650 years.
The city has evolved from being technology driven to being technology enabled, led by the city council. The next step will be for citizens and companies to use the technology to co-create the city with the council.
The key is data capture, coordination, and management, which will free up elected politicians to focus on being strategists, visionaries, and social builders.
Mr Al Musgrove of US J7 then provided a national perspective from the United States, speaking about the US Joint LL Information System.
He described its good points, and of course its challenges, noting that a new system is being developed and is expected to be ready in two years.
The aim is for the new system to be interoperable internally with other US systems and externally with NATO.
Mr Charles Macmillan, of the European Commission Joint Research Centre, gave the International Organization perspective, speaking about the Europe Media Monitor.
He explained how in 2002 the European Commission had replaced its “cut and paste” news monitoring system for its spokespeople with a tool that scoured 11,000 relevant news sites and 3,000 articles each day.
The aim is to provide information in usable format – with keyword searching, alerts on clustering and trends, and the top 10 stories by each of the 70 languages.
In the afternoon, there was the opportunity for the audience to present questions to the panel and listen in on the discussion as well as react via the Crowdicity app, the live interactive tool accompanying the NLLC 19.
The Panel discussed how the rapid deployment of New Technologies would impact on their work, the importance of incremental and modular development of systems, how the procurement systems helped and hindered, how to stimulate innovation through technology, as well as the extent to which technology could replace human effort.
The final event of the New Technologies Panel was to invite a representative sample of the NLLC audience on stage to present and explain their needs from a new LL tool.
The aim was to prompt a flurry pf ideas to be captured by the Crowdicty App to help inform the conference outcomes and to help the JALLC better understand what the NATO Lessons Learned Community was looking for in any new Lessons Learned Tool to be developed in the (near) future.
The ideas being submitted and related comments were on screen in real time for the audience to see, and the results were immediately displayed in a Word Cloud diagram (see picture).
Brigadier General Bogdan Cernat then closed the conference with some final thoughts, noting that the event had been both productive and informative. He noted that that the three days of the NATO Lessons Learned Conference had made it clear to him that the NATO Lessons Learned Community needs to better recognize where it needs to change and to actively seek the pain points to find where that change was necessary. He also noted that change inevitably means a risk of failure, but that we cannot be afraid to fail, because to fail is to learn.
There will be much more from the conference in the magazine-style summary that will be published soon and made available on the JALLC’s website at www.jallc.nato.int and on the NATO Lessons Learned Portal.
In the meantime, you can find an impression of the final day of NLLC 19 in the Gallery.
Day 2 of NLLC 19 focused on providing SACT and SACEUR with recommendations on how to make the NATO Lessons Learned (LL) system more relevant and effective in improving NATO’s warfare capability.
It began with Rear Admiral James Kirk, Deputy Commander and Chief of Staff of the Joint Warfare Centre (JWC), speaking about his experience of transforming LL from Exercises into doctrine.
He explained how NATO Exercises are becoming increasingly complex, with more stakeholders and participants, and diverse training audiences.
Their value is not just in testing HQ processes, but also in exposing the different levels of command to each other’s ways of planning.
Exercises are testbeds to put new organizations and doctrines under scrutiny. They are where we can muster manpower and materiel and try them out.
It was important to test in challenging circumstances and to understand that what hurts us makes us stronger. We adapt at the point of pain.
Ms Louise Hoehl, NATO Communication and Information Agency (NCIA), then spoke about the Agency’s experiences of doing LLs at Exercise Trident Juncture 2018 (TRJE 18).
NCIA supports over 20 Exercises each year, but TRJE 18 was on a different scale with much greater complexity. Senior management at NCIA was therefore keen to use TRJE 18 as an opportunity to capture its own lessons and to make its own improvements.
NCIA experienced some of the challenges all NATO organizations face when doing LL, such as how to capture observations, turn them into Lessons Identified and then actually learn those Lessons.
Air Commodore Philip Lester, UK Development Concept and Doctrine Centre, spoke about LL in the Space and Cyber domains.
He said that we need to get better at remembering the Lessons that had already been learned, before considering new Lessons.
It was important that LL and doctrine should provide the baseline of understanding from which leadership can make their decisions.
Resources are not used effectively if leadership assumes too quickly that a situation is totally new. It is important to get the jumping off point right.
It is a challenge for both NATO and the Nations to integrate the new domains of Space and Cyber into the existing operational domain structure of air, land, and sea. What we need for the future is an integrated five-domain concept – not three domains plus two.
Learning Lessons at the level of an individual Nation is difficult. It is much more so in an Alliance of 29. He reminded the audience of the indirect benefits from the NATO LL Process from, for example, liaison with colleagues: the journey is important as well as the destination.
In the fourth of this set of presentations, Mr Cornelious (Ham) Doraton, US Army NATO Interoperability/Standardization Representative, spoke about standardization and interoperability LL.
He reminded participants that the military is small in comparison with society as a whole, but its role is very important.
Standardization is vital both in NATO and in everyday life. Standards are the glue that holds NATO together.
He asked the audience why interoperability is important to NATO, the answer being because the whole is greater than the sum of the parts.
He described how he has worked with the JALLC over the years to embed interoperability standardization through the Capable Logistician Exercises.
Participants then had the opportunity to discuss the issues raised on the Crowdicity App with the Panel of speakers.
During the Panel discussion, there was a call for greater synergy of the JALLC, the JWC, and the Joint Forces Training Centre (JFTC), also known as JJJs, both in general, and specifically to improve the NATO LL Process.
In that connection, there was a need to engage more with the Centres of Excellence and specific Communities of Interest as the Subject Matter Experts.
In the second set of presentations, Professor Heidi Hardt of the University of California, Irvine, spoke about overcoming barriers to learning within the Alliance and how international organizations develop institutional memory.
Her research from 2015 with 120 NATO elites had indicated that there could be inadvertent barriers to LL in NATO.
She noted that her research suggests a preference for informal rather than formal methods for capturing and sharing Lessons organizationally, such as memos or Food for Thought papers.
Professor Hardt also raised the point that NATO’s military rotation and civilian personnel contracting policy can make it difficult to retain institutional memory.
Dr. Henrik Heidenkamp, ACO SHAPE Strategic Management Planning, presented on the links between LL and Strategic Management.
The subject was timely, given that Allied Command Operations (ACO) had recently revised its Strategic Management directive, including adding references to the NATO LL capability, and was about to publish its new Strategic Management Plan.
The aim of ACO’s Strategic Management was to enhance the efficient and effective development of ACO. It can do so more effectively, validly and reliably if it is based on LL.
A Lessons-based execution of the ACO’s Strategic Management System can also improve the justification for ACO’s resource requirements.
Dr. Heidenkamp said that it was important to align and integrate processes so that the Strategic Management community knew about the LL community and vice versa.
As the conference had already heard from various speakers, there are cultural, organizational, and educational challenges for NATO which affect Strategic Management and the linkage with LL.
Dr. Heidenkamp said that his point is not that we cannot make progress. We have to accept, however, that these challenges are inherent to the organization and are not going to change in the short term.
We should define success against the organization as it is, rather than the organization as we think it is or would like it to be.
Finally, Dr Tom Dyson of Royal Holloway University of London spoke about LL best practice from the perspective of the individual Nation and a bottom-up approach.
Dr Dyson described the British Army’s experience of a high-level review team to look at observations in the first instance, with remedial action then overseen by a cross-functional team.
Dr Dyson described factors that can make a difference to organizational culture and can incentivize people to behave in certain ways.
Again, there was the chance for discussion of all the ideas raised by the Panel, with interesting conversation about intelligence sharing, appropriate tools for analysis, the role of artificial intelligence and machine learning, how to present LL information in a way that would find traction with NATO leadership, and how to provide safe spaces for honest reflection.
Today was the first day of the NATO Lessons Learned Conference 2019 (NLLC 19), hosted by the JALLC.
Participants from NATO, the Nations and academia came together to share, discuss, learn, and challenge.
This year, the theme of the NATO Lessons Learned Conference is improving NATO’s Lessons Learned capability to enhance NATO’s warfighting capability.
Supreme Allied Commander Transformation General André Lanata welcomed participants to the conference by video.
He said that their mission for the conference was an important and timely one. The character of warfare evolves, and NATO needed to Observe, Orientate, Decide and Act faster in the new operating environments.
The NATO Lessons Learned (LL) Process has to ensure that we learn faster and better from our experiences. He encouraged us to think added value, and bring our ideas to shape the NATO LL Process.
The JALLC’s Commander Brigadier General Bogdan Cernat welcomed participants to the highly innovative NOVA School of Business and Economics, in Lisbon, and to what would be the Smartest NATO Lessons Learned Conference in terms of technology so far.
He said that this event combined the annual NLLC with the follow-on to the very successful NATO New Technology Event 2018, to maximize synergies and opportunities.
There are two key elements in the drive for more effective Warfare Development. The first is Lessons Learned, to ensure that we learn faster and better from our experiences.
The second is New Technology, to ensure that we understand how technology is changing the very nature of future military conflict and that we can harness it for the benefit of the Alliance.
Brigadier General Ilmars Lejins, Assistant Chief of Staff, Joint Force Development, opened the day’s presentations, speaking about the status of implementing the NATO Lessons Learned Capability and the way ahead.
He discussed the gaps in governance, the JALLC, the NATO LL Portal (NLLP) and NATO’s LL culture, and how the new NATO-wide Lessons Learned Plan for 2020 to 2025 would add value and help to fill those gaps.
Colonel Manuel Santos, SHAPE J5, said that the warfighting capability that NATO needed was constantly evolving.
The past few years had seen an emphasis on readiness, multinationality, and responsiveness, all of which increased the need for interoperability.
The aim of having highly capable, deployable and trained forces required a culture of interoperability by design, and not as an afterthought.
Before lunch, there was a demonstration by Crowdicity of its collaboration and ideas-generating platform, and everyone was encouraged to use it to log ideas, questions and comments and respond to the daily NLLC 19 challenge.
After lunch, Lieutenant Colonel Fabrizio Ottaviani, JALLC, spoke about the status of the NATO LL Capability, from the JALLC perspective.
He highlighted certain aspects of the implementation of the NATO LL Capability that seemed to require attention.
difficulties in filling LL posts and keeping post-holders focused on LL;
hierarchical and functional issues;
the use of generic rather than HQ-specific documentation;
effective responsibility for recommendations sent to HQs; and
the LL mindset and culture.
The root cause seems to be the current prioritization given to LL in NATO.
LTC Ottaviani noted that action at any level would be likely to make a difference, and participants were asked to think about what they could do to address potential challenges.
Mr David Noon, JALLC, spoke next about the insights JALLC had obtained from supporting the LL process at Exercise TRIDENT JUNCTURE 2018 (TRJE 18).
From its analysis, the JALLC had identified differences between the execution of the NATO LL Process during routine HQ activity, and the execution during a major exercise, such as TRJE 18
These differences were likely to be the result of:
the increased volume of lessons identified during an Exercise, and the speed at which they entered the system;
the complexity of the situations described in the data, which went beyond the expertise of any specific HQ;
the raw state of the data; and
keeping the relevant people engaged in the NATO LL Working Groups and analysis after the end of the Exercise, once they had returned to regular duty.
Although many issues are raised during an Exercise, these do not necessarily find their way into the NLLP, and can potentially create a real-world risk.
As a result of the analysis of the NATO LL Process during TRJE 18, it is becoming clear that the NATO LL Process and the reporting process for Exercises are not meshing well. For example, the First Impressions Report for an Exercise has to be produced within 15 days, whereas defining a Lesson Identified takes considerably longer under the NATO LL Process.
Lieutenant Colonel Rigo Genz, JALLC, then presented the recent developments in training for the NATO LL Capability.
LL Training was added in early 2018 as a fourth pillar to the NATO LL Capability – in addition to LL Structure, Process, and Tools.
As well as the online two-hour course for general awareness, the JALLC Advisory and Training Team (JATT), and the five-day LL Staff Officer Course for Subject Matter Experts, a new NATO LL Management Course will be offered for the first time the week after NLLC 19 in Oberammergau, Germany.
This course will provide the opportunity for hands-on experience and a deeper dive into the subject.
The final presentation of the afternoon was provided by Commander Pedro Viegas, JALLC, on the subject of NLLP developments.
He described the development of the NLLP, partly the result of increased data migration from the NATO Command Structure.
Commander Viegas highlighted that:
The JALLC has implemented over 180 improvements to the way that the NLLP worked.
In the short term, the JALLC is working on the NLLP Improvement Plan 2019.
In the medium term, there will be further NLLP improvement, implementation of the new NATO LL Plan 2020-25, and the integration of New Technologies.
It is expected that the NATO Communication and Information Agency will take over some of the technical functions of the NLLP from 2020.
In the longer term, the focus will be on a new LL Toolset with plans to discontinue the present NLLP from 2025.
There was then the opportunity for participants to discuss three topics emerging from the Crowdicity App discussion, with participants rising to Brigadier General Bogdan’s challenge to come up with their big ideas, bright ideas, and best ideas.
On 03 October 2019, Vice Admiral Alfonso Fernández de Córdoba, Deputy Chief of Staff to Allied Command Transformation’s (ACT) Joint Force Development Directorate, visited the JALLC in order to improve awareness and understanding of the JALLC's role within NATO, the NATO Lessons Learned Capability and the JALLC's contribution to the Alliance’s objectives.
The visit started with an Office Call with Brigadier General Bogdan Cernat, the JALLC’s Commander, followed by a tailored briefing during which Vice Admiral Fernández de Córdoba was able to discuss the NATO Lessons Learned Processes and the JALLC’s activities and products.
Vice Admiral Fernández de Córdoba departed with words of appreciation for the opportunity to gain a better understanding of the JALLC. He also added that in this unique period of change for ACT and NATO, he was looking forward to working with the JALLC over the next few years.
The JALLC is always ready to open its doors to visitors who want to find out more about what the organization does and how Lessons Learned contribute to the transformation of the Alliance. Over the past couple of weeks, the JALLC has received visits from three distinguished groups: the Directorate of Research, Doctrine, Organization and Materiel (DIDOM) from Spain, 50 participants on the Portuguese Air Force Senior Enlisted Course and their supervisors, and 32 participants on the Portuguese Junior Diplomatic Service Officers Course and their professors.
The JALLC has a basic tour and visit programme for its visitors which is tailored to meet each visiting group’s specific interests. Our visitors might like to know more about how the NATO Lessons Learned Capability works, or perhaps how Joint Analysis contributes to the NATO Lessons Learned Process, or even how the JALLC’s Advisory and Training Teams reach out to the Allies and partners across the globe. Each visit is unique and is accompanied by a team of JALLC staff assigned especially for the purpose because of their expertise or background to ensure our visitors have the most informative and enjoyable visit possible.
Visitors are invited to sign the book of honour and pictures are taken to commemorate the visit.
These recent three visits were no different and the JALLC’s Command Group and staff had the pleasure of meeting and exchanging ideas and views with the visitors in presentations, discussion sessions, and office calls.
This year’s conference will return to a more traditional conference format with a fascinating variety of speakers from NATO, Nations, Partners and Academia. You will have the chance to share insights into the latest LL developments in NATO, engage with leadership perspectives on LL, learn from studies into what works and what doesn’t work and how to incorporate best practices into your everyday work, discover new tools to support your work, and be part of the requirements definition for NATO's new LL toolset
Among our distinguished speakers this year, we are honoured to welcome General Ilmars Lejins, NATO HQ SACT Assistant Chief of Staff Joint Forces Development, Rear Admiral James A. Kirk, Chief of Staff NATO Joint Warfare Centre and published authors Prof. Heidi Hard and Dr Tom Dyson.
We will also use a new high tech approach to audience participation which will enable your full active participation throughout. We look forward you joining us and the rest of the NATO LL Community for a riveting three days at our modern beachfront conference venue.