Day 3 of NLLC 19 began with two presentations on Lessons Learned (LL) from our Partner nations.

Ms Jane Chirwa, the African Union Commission knowledge management officer, spoke about LL from African Standby Forces and Peace and Security Operations.

She said that in 2016 the AU Peace and Security Council noticed that there was very little documentation on the challenges and successes of operations, particularly the long-running AU Mission in Somalia.

LL Process, drawing from the experience of other international organizations including NATO, was therefore set up, with guiding principles of early stakeholder involvement, internal (member state) drivers, non-attribution of issues raised and safe spaces for sharing, sharing of information and cyclical feedback loops.

This LL Process has been a success and, for the future, the aim will be to enhance it by continuing to learn from NATO’s approach to LL, as well as learning from internal experiences, and benefiting from LL training.

Colonel (ret.) Yuri Pashchuk, of the National Army Academy, Ukraine, spoke about the approach to improving the Ukrainian Army LL Capability.

The current system had been inherited from the Soviet Union in 1991, with improvements being made as a result of learning from operations in Afghanistan. The system has since changed to meet the needs of modern warfare and the current global security threat environment.

The challenges that the Ukraine faced are similar to those faced by all organizations in developing a LL Capability as have been presented at this event.

The Ukraine’s priority was now to embrace interoperability going forward and a roadmap to get there was produced in December 2018.

The focus of the day then moved to New Technologies.

Mr Stefan Olaru, research analyst at the JALLC, introduced the Panel, reminding the audience of the state of play after the New Technologies Event 2018. He provided a brief overview of progress since then, which was followed by a set of related updates.

The first of these updates was from Commander Dietmar TeufelHQ SACT Innovation Branch Senior Analyst, who spoke about innovation and exploiting disruptive technologies.

In this context, he highlighted the ACT innovation hub, a network of scientists and academics which was launched in 2012, which can be asked find innovative solutions to complex problems, amongst others through issuing innovation challenges to experts inside and outside of NATO; and ACT’s innovation branch, created in May 2019, which is tasked with matching solutions to ACT’s problems and pain points.

He then gave a brief overview of the Emerging and Disruptive Technologies (EDT) Roadmap. The aim of the EDT Roadmap is to exploit technologies to inform policy changes, inform defence planning, feed future capabilities, and ensure interoperability by design. EDT Roadmap activities are prioritized through five main Lines of Effort.

He noted that SACT’s intention is clear – we should not just talk about New Technologies but should deliver through them.

There were then three short updates from the JALLC’s own research analysts.

Referring to the EDT Roadmap, Mr Stefan Olaru stepped onto the stage again to speak about the LL Enabling Line of Effort of the EDT Roadmap, providing an overview of its status.

He noted that SACT’s call for a new LL toolset in early 2018 had been well accepted and, in response, the JALLC organized the New Technologies Event 2018, inviting representatives from NATO and the Nations, industry and academia, to look at how existing New Technology could potentially improve the NATO LL Capability, and in particular the NATO Lessons Learned Portal.

The next step for the JALLC was to conduct analysis on all the data gathered at that Event which resulted in the identification of a number of challenges and corresponding technology that had the potential to address those challenges from the review of some 112 technological applications and products on the market.

Finally, the JALLC has since sponsored NCIA to collect data from Exercise TRJE 18– a total of 3.9 terabytes – and is using that dataset to investigate the potential of Big Data analysis tools.

Mr Filipe Vieira then spoke about the exploration of LL data using data science tools.

The aim was to consider how to capture and analyse the largely untapped dataset of LL from operations, missions and exercises, and from data already uploaded to the NATO LL Portal including from existing Lessons, documents, and reports.

If machine learning could analyze large amounts of data for keywords and trends, analysis could be done much more efficiently and at the pace needed.

The aim was for the technology to help analysts to analyse, not to have spend time extracting the nuggets of information.

A particular challenge would be to ensure that whatever was developed was replicable for other datasets.

Finally, Ms Mihaela Racovita spoke about the JALLC’s project on NATO Exercise Big Data exploration.

Big Datasets, characterized by volume, velocity, and variety of the data, may contain embedded Observations, Best Practices, and Lessons Identified that are never reported, as well as contextual information about Lessons, and clues about the needs of Lessons Learned users.

The Alliance needs to learn how to access that information as quickly as possible and navigate the challenges in doing so.

In the context of this particular project, the JALLC was considering:

  • how we can use Exercise data to increase the quality of NATO LL Portal products;
  • how we can help people access the right LL information at the right time;
  • whether we can extract tacit knowledge; and
  • whether we can construct predictive LL models.

Expectations need to be managed; Big Data was not a universal cure.

We will need to learn from experimentation and from our failures as well as our successes.

Colonel Paul MalessaHQ SACT CAPDEV, then presented the new governance model for NATO’s common-funded capability delivery.

After the Cold War, the urgency in delivering new capabilities reduced and the timescales for capability development have relaxed.

With the current speed of technological development, there is a renewed sense of urgency to develop the capabilities the Alliance will need to meet new and challenging threats at the speed of relevance.

A new governance model has therefore been instituted, which has to be followed from beginning to end to be effective. The value added is a reduction in timescales, clearer responsibilities, greater transparency and flexibility, and full coverage of DOTMLPFI.

There was then a set of presentations providing a range of external, Non-NATO perspectives on New Technologies.

Mr Miguel Pinto Luz, the Deputy Mayor of Cascais, the Municipality of the NLLC 2019 Venue, gave a local perspective on how Cascais has been a Smart City for 650 years.

The city has evolved from being technology driven to being technology enabled, led by the city council. The next step will be for citizens and companies to use the technology to co-create the city with the council.

The key is data capture, coordination, and management, which will free up elected politicians to focus on being strategists, visionaries, and social builders.

Mr Al Musgrove of US J7 then provided a national perspective from the United States, speaking about the US Joint LL Information System.

He described its good points, and of course its challenges, noting that a new system is being developed and is expected to be ready in two years.

The aim is for the new system to be interoperable internally with other US systems and externally with NATO.

Mr Charles Macmillan, of the European Commission Joint Research Centre, gave the International Organization perspective, speaking about the Europe Media Monitor.

He explained how in 2002 the European Commission had replaced its “cut and paste” news monitoring system for its spokespeople with a tool that scoured 11,000 relevant news sites and 3,000 articles each day.

The aim is to provide information in usable format – with keyword searching, alerts on clustering and trends, and the top 10 stories by each of the 70 languages.

In the afternoon, there was the opportunity for the audience to present questions to the panel and listen in on the discussion as well as react via the Crowdicity app, the live interactive tool accompanying the NLLC 19.

The Panel discussed how the rapid deployment of New Technologies would impact on their work, the importance of incremental and modular development of systems, how the procurement systems helped and hindered, how to stimulate innovation through technology, as well as the extent to which technology could replace human effort.

The final event of the New Technologies Panel was to invite a representative sample of the NLLC audience on stage to present and explain their needs from a new LL tool.

The aim was to prompt a flurry pf ideas to be captured by the Crowdicty App to help inform the conference outcomes and to help the JALLC better understand what the NATO Lessons Learned Community was looking for in any new Lessons Learned Tool to be developed in the (near) future.

The ideas being submitted and related comments were on screen in real time for the audience to see, and the results were immediately displayed in a Word Cloud diagram (see picture).

Brigadier General Bogdan Cernat then closed the conference with some final thoughts, noting that the event had been both productive and informative. He noted that that the three days of the NATO Lessons Learned Conference had made it clear to him that the NATO Lessons Learned Community needs to better recognize where it needs to change and to actively seek the pain points to find where that change was necessary. He also noted that change inevitably means a risk of failure, but that we cannot be afraid to fail, because to fail is to learn.

There will be much more from the conference in the magazine-style summary that will be published soon and made available on the JALLC’s website at and on the NATO Lessons Learned Portal.

In the meantime, you can find an impression of the final day of NLLC 19 in the Gallery.