The UK Was Warned This Counterterrorism Program Was A Disaster — But Rolled It Out Anyway
Revealed: the inside story of how the British government rolled out a dangerously flawed intelligence-sharing system right as the UK suffered one of its deadliest years from terrorism.
By
by Richard HolmesPosted 1 minute ago
{
"id": 129101895
}
The UK government knew the counterterrorism intelligence-sharing system it had spent years developing was dangerously defective — but rolled it out anyway, causing chaos and confusion among officers scrambling to contain an unprecedented wave of fatal terror attacks, BuzzFeed News and BBC Newsnight can reveal.
The National Common Intelligence Application was built to put all regional information about potential violent extremists into one searchable national database, and improve communication between MI5, Britain’s domestic intelligence agency, and the police. In 2015, authorities heralded its launch as a “landmark event” that would “make a significant contribution” to the fight against terrorism.
But according to hundreds of pages of confidential police emails, intelligence files, and internal police memos reviewed by BuzzFeed News, along with interviews with five sources inside Britain’s counterterrorism police, the database was plagued by crashes and blackouts, and bogged down by mass duplications that made sifting through crucial intelligence almost impossible. Officers warned that the NCIA’s problems were “critical” and that using it risked “intelligence failure.” They repeatedly deemed it “not fit for purpose” — unable to do the job it was built to do — but their concerns were brushed aside. One counterterrorism detective responsible for helping to develop the system, Tony Thorne, said that he tried multiple times to warn higher-ups about its shortcomings. “They ignored me and continued to roll out a broken system,” he said.
During the later stages of the NCIA’s implementation, in 2017, the UK suffered one of its most deadly years from terrorism in recent memory, with five incidents resulting in 40 killed and more than 900 injured. The deadliest was a bombing at the end of an Ariana Grande concert in the Manchester Arena, which killed 22 victims and injured more than 800, the majority of them children. The youngest victim, Saffie-Rose Roussos, was just 8 years old.
Jeff J Mitchell / Getty Images Members of the public look at tributes left in St Ann’s Square for the people who died in the May 2017 terror attack at the Manchester Arena.
{ "id": 129101895 }Before the attack, authorities had briefly considered the bomber, Salman Abedi, a “subject of interest” but had ultimately decided he wasn’t a threat. An ongoing public inquiry has heard that vital information about Abedi slipped through the cracks: MI5 failed to pass key leads to counterterror police in Manchester, and police were in possession of 1,300 text messages between Abedi and a known terrorist but hadn’t linked the texts to their computer profile of him.
“Clunky” and “hit and miss” IT systems hampered both MI5 and the police during that time, according to evidence given at the inquiry. MI5 described the process for its officers in Manchester to send information to the police as “bureaucratic” and “onerous.”
Much about what went wrong — including details as simple as the names of the data systems in question — have remained shrouded in secrecy, with the UK government claiming that further disclosure would harm national security.
But now a BuzzFeed News investigation in collaboration with BBC Newsnight can reveal the NCIA was the primary tool responsible for storing, managing, and sharing intelligence in use in Manchester when the bombing happened — and that officers had been warning as early as 2013 about the very problems identified in the inquiry.
Two officers who operated the NCIA in Manchester around the time of the bombing told BuzzFeed News that they were deeply troubled by its shortcomings. One used the NCIA in the immediate aftermath to help determine whether another attack was imminent — and came to believe the NCIA was a reason why police did not prevent the bombing in the first place.
“A terrorist attack is a perfect storm when a dozen different factors go wrong at the same time,'' the officer said, speaking on condition of anonymity due to fear of being prosecuted for talking about a sensitive national security subject. “But in Manchester, NCIA made up 11 out of 12 factors.”
Echoing problems flagged in documents, five current and former officers from four counterterror units across the UK told BuzzFeed News that the rollout of the NCIA made officers’ jobs more difficult. So much information was either redundant or missing, all five of the sources said, that in response to a potential terrorist threat they were left to make guesses about what information might be on file. Two of the sources said they warned their bosses of critical failures.
In response to detailed questions about this story, a spokesperson for UK Counter Terrorism Policing said in a written statement that the NCIA “is constantly being improved” and the “suggestion that concerns are ignored is simply not true.” “Following the 2017 attacks CT Policing and partners undertook a significant review of UK Counter Terrorism, leading to 104 recommendations for improvement,” the statement said. “The consequent changes have brought about substantial improvements in the way we work, and we have successfully disrupted 32 late-stage terror attacks and saved many lives since 2017. The NCIA has been an essential tool in helping us achieve this.”
Asked how much it cost to build and implement the NCIA, the spokesperson did not respond. At the Manchester inquiry, a high-ranking counterterror official referred to it as a “nine-figure” project.
A spokesperson for the Home Office, the government agency that oversees counterterrorism police, added in a statement that the NCIA had “greatly enhanced” authorities' ability to respond to the Manchester bombing and “manage the huge volumes of intelligence.” The spokesperson added that the public inquiry into the Manchester bombing will decide whether the data systems they used "impacted their ability to prevent the attack."
“It is of the utmost importance that those who were affected by the Manchester Arena attack get the answers they need," the spokesperson said. "The victims, their families, those injured and all those affected by this cowardly attack remain at the forefront of our thoughts.”
The classified documents reviewed by BuzzFeed News, which range from 2013 to 2020, paint a damning picture of the UK government's ability to fight terrorism — and to respond to urgent national security threats with life-or-death stakes. Many of the NCIA’s issues persisted for years after they were flagged — some until at least 2020, more than three years after one of the worst terrorist attacks in the country’s history.
“If we are using NCIA” to deter terrorism, one former counterterror officer told BuzzFeed News, the next big attack is “a matter of time.”
{ "id": 129105302 , "is_fullsize": true }
Alex Williamson for BuzzFeed News
{
"id": 129101895
}
Since around 2003, British counterterror police officers had used a database known as the National Special Branch Intelligence System to share information. The system was secure but sluggish: Each of the UK’s counterterror branches used its own individual database, which led to inconsistencies and incompatibilities from region to region. Full access to records was often slow-going — if an officer from London wanted a report from Manchester, the London officer often had to call Manchester and ask to see it.
The Home Office’s new Apollo project aimed for something more centralized that would make it easier for officers to share information with one another and with MI5. The new system would put “the whole picture of counter-terrorism” at officers’ fingertips, according to an independent report looking at the efficiency and effectiveness of how each police force in England and Wales manages intelligence. Its name, the National Common Intelligence Application, reflected the government’s goal: the entire country’s counterterror force on the same page.
By winter 2013, the police were ready to test out the new system and asked a team of counterterror officers to implement it across Scotland, with a final review once it was complete.
In late November 2013, Police Scotland produced a scathing five-page assessment in which it said the NCIA was “not fit for purpose.”
Police Scotland refused BuzzFeed News’ Freedom of Information request for audits and reports about the system, citing national security, saying its disclosure may highlight “the limitations of police capabilities in this area which may further encourage criminal or terrorist activity by exposing a potential vulnerability.” The release of the document would result in the public being “in fear of more criminal or terrorist activity occurring.”
But BuzzFeed News has examined the still-confidential report. It slams the NCIA for failing to provide accurate results when officers searched for specific subjects. “As intelligence officers there needs to be a confidence that, if certain searches are done, then all of the available results will be returned,” the report states. This shortcoming, the report continues, “would lead to missed intelligence and officers not having the full intelligence picture before them.”
The new system was “slow and cumbersome,” the officers say in the report, and would “struggle” to handle day-to-day counterterrorism work, let alone a major terrorist incident. The report concludes that the problems were “too great to justify the adoption of the NCIA at this time.”
The UK authorities went ahead with the NCIA anyway. On Dec. 2, 2013, a week after the officers filed their cutting review, Scotland became the first region to use the new system, according to the internal documents seen by BuzzFeed News. The other regions were still on the old system, but would switch over in the coming years.
{ "id": 129105171 }
Tori Frenec for BuzzFeed News Tony Thorne
{ "id": 129101895 }Tony Thorne was one of the officers on the Apollo project, advising the team on the task of merging large volumes of data. Thorne, a former counterterrorism officer with the Wales Extremism and Counter Terrorism Unit, said he was shocked by what he saw in Scotland. “We left Scotland with a process that was not complete or adequate in any way,” he said.
The key issues highlighted during the test run reemerged right away, according to emails and internal memos from 2014 and 2015 reviewed by BuzzFeed News.
Officers described a system that was “crashing routinely” and “timing out after 10 minutes,” with glitches so severe that they dramatically increased “the time required to perform a simple task.”
Even basic searches caused trouble. One officer had described how he put in a search term and received a result that was too broad. He started sifting through the documents manually to figure out which ones he actually needed — but as he did so, the system crashed. When he logged back in, he typed in the same search terms and found that “the search result was not the same.”
Officers using the new systems also reported serious difficulties with the very problem the NCIA was aiming to solve: communicating with other forces and agencies. After a suspicious person entered the UK by plane, an officer reported that they received an important intelligence report from officers at the airport in an unreadable format. Another told a member of the Apollo team that the NCIA’s inability to share intelligence with the other regions still using the old system was a critical risk that “may lead to intelligence failure.”
The quality of intelligence that did make it onto the system was often poor. In some cases, the NCIA was deluged with irrelevant information; in others, vital intelligence did not show up on the NCIA at all. One officer complained that the system “auto ingests” documents that had nothing to do with terrorism. “This issue was something that was always talked about,” the officer wrote, “however now we are live there appears to have been nothing more done about it.”
The NCIA was built on the template of an already existing system called the Home Office Large Major Enquiry System (HOLMES), four sources told BuzzFeed News. The problem, one said, is that HOLMES is used to investigate incidents that have already occurred whereas the NCIA is intended to prevent attacks from happening. Another officer told BuzzFeed News that building the NCIA on top of the HOLMES system caused defects that left large amounts of intelligence difficult to find.
Officers echoed these concerns in their emails and official reports. One of the key features borrowed from the HOLMES system was a search tool, much like Google, that was supposed to enable officers to quickly retrieve documents containing a certain word, regardless of where on the record the particular word appeared. If it worked, this would make it far easier to find specific intelligence on potential terrorists from hundreds of thousands of files.
But the search tool wasn’t working. Officers found that if they put in the same search term on multiple occasions, they would often get a different result each time. The search tool also wasn’t able to scan for dates of birth, making it much harder to pinpoint the right document.
This shortcoming dovetailed with another major problem. Early on, it became clear that many duplicated records would make their way onto the NCIA — since it was compiling data from multiple forces that often possessed the same file on a given individual. One internal report seen by BuzzFeed News acknowledges that this would cause a “knock on” effect that hampered analysts. But higher-ups ultimately decided that “no de-duplication would occur” until the entire UK was using the NCIA.
One Manchester-based officer who later started using the NCIA told BuzzFeed News that duplications made finding what you were looking for like “trying to find a needle in a haystack” — such a struggle that “you could miss vital intelligence leads.”
Thorne, the counterterror detective who worked on the NCIA, was growing increasingly concerned. “Unfortunately,” he wrote to colleagues in a February 2014 email, “as we are all fully aware the NCIA has struggled to deliver what was promised and has not been fit for purpose.”
The rollout of the NCIA pressed ahead.
{ "id": 129101895 }In September 2015, the then–senior national coordinator for counterterrorism policing, Helen Ball, wrote a draft foreword to a confidential handbook for officers using the NCIA. In addition to calling the new system a “landmark event,” she promises that “the operational advantages provided by NCIA will deliver levels of interoperability not previously seen.”
The foreword mentions nothing about any difficulty with the new system. But minutes from a meeting involving the police and MI5 in November that year show that substantial issues remained, including some of those first raised by Police Scotland. Documents filed prior to NCIA were still unavailable on the system. The search tool was still causing such severe problems that alternatives were being sought. And officers were still getting an “error message” when they tried to perform simple tasks.
The NCIA was supposed to have been rolled out nationwide by March 2017. But it kept being delayed. One report, announcing a new delay until July 2018, cites "significant resourcing issues." Another, discussing the need to do “data cleansing,” says that the final region, London, would have to wait until at least March 2019.
There was little sign that the NCIA’s core issues were being fixed. As in Scotland, officers in other regions were given the chance to try the new system before the full rollout. One was the North West, which includes Manchester.
A seasoned counterterror officer on Greater Manchester Police’s intelligence unit told BuzzFeed News that he read a review from the North West in early 2014. Echoing the Scottish report, the document deemed the NCIA not fit for purpose, the officer said. As a result, he added, senior officers refused to take the new system on their computers and delayed the rollout in the North West for at least another two years. The region finally began using the NCIA in earnest by March 2017.
{ "id": 129105109 , "is_fullsize": true }
Pa Images / PA Images via Getty Images Helpers attend to people inside the Manchester Arena after the attack.
{ "id": 129101895 }Four months later, on the evening of May 22, 2017, Salman Abedi walked undetected into the lobby of the Manchester Arena and detonated a bomb loaded with nails and screws, killing 22 people and injuring hundreds. Many of the victims were young girls.
Abedi had been known to UK intelligence agencies since 2010. Police had stopped him during a trip to deliver medical supplies and other material to rebels fighting the Muammar al-Qaddafi regime with his father and brother. In 2014 authorities deemed him a “subject of interest” and investigated his links with another extremist. Later that year, police recategorized him as a “closed subject of interest,” meaning he joined a pool of about 20,000 people no longer under active investigation. Yet he continued to communicate with a known terrorist, Abdalraouf Abdallah, who was jailed for helping people travel to Syria to fight for ISIS.
Between 2015 and 2017, authorities continued to receive intelligence about Abedi’s activities. Weeks before the attack, MI5 scheduled a meeting to discuss whether he had reengaged in extremist activity and should be investigated more closely. MI5 later acknowledged, “the plot then moved faster than the process.” Nine days before that meeting was to take place, Abedi carried out his attack.
Evidence given during the ongoing public inquiry into the bombing has revealed that before the attack, MI5 and the police failed to share with each other several key pieces of intelligence relating to Abedi. In 2016, according to a summary of evidence presented at the inquiry, MI5 had “intended” to send counterterror police information that Abedi had traveled between Libya and Istanbul in 2016, but the police had “no record of it being received.” In 2014, the police also failed to share with MI5 the 1,300 messages exchanged between Abedi and a terror suspect, who went on to be convicted, despite secret testimony from MI5 telling the inquiry that “normal practices” would have seen the police share “material of this kind immediately.”
{ "id": 129105686 } Sometimes PowerPoint presentations had to be sent one slide at a time. { "id": 129101895 }Secret testimony from MI5 also revealed how two key pieces of intelligence about Abedi — “the significance of which was not fully appreciated at the time” — were not shared with the police in the months before the attack. An MI5 officer told the inquiry that if such information was found today it would have led to an investigation into Abedi “in conjunction with the police.” However, there was no record of either piece of intelligence being shared with counterterror police.
Officials have also testified that “IT systems” issues made it more difficult for the police and MI5 to communicate during that time.
Sharing intelligence between police and MI5 was a “bureaucratic and process heavy” task during a time when the intelligence agency’s regional team in the North West was “struggling to cope” with an increased workload in the month before the attack, according to a public summary of secret testimony given at the inquiry.
One senior MI5 witness testified that electronic files over a modest size didn’t send properly — sometimes PowerPoint presentations had to be sent one slide at a time. “MI5 officers would ensure that documents did get to the police, but the system was quite involved and it could sometimes be a real effort to push something through,” the summary states. “This was onerous for investigators who were regularly writing and sending messages from MI5 to the police.” A chief inspector responsible for the police’s partnership with MI5 was “concerned about the system of communication at that time,” according to the summary.
The summary did not name the system the officers were referring to — but documents seen by BuzzFeed News show that their criticisms of the technology were strikingly similar to complaints that officers were making about the NCIA more than three years before the Manchester attack.
Asked whether the officers at the inquiry were referring to NCIA, the spokesperson for UK Counter Terrorism Policing did not respond. The inquiry has heard “extensive evidence” from witnesses about “a range of processes and systems, including some IT systems,” the spokesperson said. It is for the chair of the inquiry to “decide whether the systems we used impacted our ability to prevent the attack.”
“The terrible events of the Manchester Arena attack and the suffering of the victims, their families and all those affected reminds us of the critical importance of what we do in UK Counter Terrorism,” the spokesperson added.
The NCIA had only recently been introduced in Manchester at the time of the bombing, and a senior officer testified at the inquiry that communication difficulties between MI5 and counterterror police had been a result of “changes” to their data systems “around this time.”
One former counterterror officer told BuzzFeed News that he had a chance to see the NCIA in action in the days after the attack. Tasked with reviewing intelligence in Manchester to root out any potential further attacks, the officer was distraught by how poorly the NCIA functioned: “It was a big bag of spanners,'' he concluded.
At a moment of tremendous urgency, analysts using the NCIA had to sift through hundreds of files, which included new intelligence coming in from the public, officers, and informants in real time.
But the way in which the system processed data, resulted in around 400 pieces of intelligence about links to Abedi or possible future attacks going unread, the officer told BuzzFeed News. The technology was so “clunky,” he said, that officers didn’t feel confident that they could see everything they needed to. The officer said analysts were still making their way through intelligence up to 12 days after the attack.
{ "id": 129105692 } Inconsistencies in the search results forced officers to make “assumptions,” and left them “frightened of making the wrong decision” when assessing if someone was a risk to national security. { "id": 129101895 }The spokesperson for Counter Terrorism policing said that “in the days and weeks that followed” the attack, the police’s “ability to respond to, and manage, the huge volumes of intelligence we received was enhanced by the use of NCIA.”
The former counterterrorism officer told BuzzFeed News that the experience pushed him over the edge and led him to retire. “I couldn’t see myself having to work with that pile of shit,” he said.
A second officer working in the North West told BuzzFeed News that whoever decided to downgrade Abedi as a threat couldn’t have been confident in their decision if they made it using the NCIA. Inconsistencies in the search results forced officers to make “assumptions,” and left them “frightened of making the wrong decision” when assessing if someone was a risk to national security, the officer said.
Tony Thorne had watched news coverage of the Manchester bombing at home that night and asked himself: “Did the system I built have something to do with this?”
Thorne retired from the force in 2018. But he said the NCIA continued to gnaw at him. He said he felt so responsible for its failings that in October 2020 he wrote a letter to Prime Minister Boris Johnson — copying in Home Secretary Priti Patel, the heads of London and Manchester police forces, and the head of MI5 — outlining what he felt went wrong.
{ "id": 129105206 }
Tori Frenec for BuzzFeed News Tony Thorne
{ "id": 129101895 }In his letter, Thorne also outlines his own personal health struggles, the “incompetence and abuse” he experienced during his medical retirement, and his belief that he is owed compensation for how his injuries were handled. He also details the grievances he had with his local police force in Wales and the misconduct he believed he had witnessed there. But he devotes much of the letter to detailing his criticisms of the NCIA. The system, he writes, “contributed [to] if not [was] the full blown fault why the below attacks occurred.”
He then lists the attacks that might have gone differently — not just the Manchester Arena bombing but the four others that occurred that year:
In March 2017, an Islamic extremist drove a car across Westminster Bridge, killing four pedestrians on the sidewalk and injuring dozens more before crashing his car into railings outside the Houses of Parliament and fatally stabbing an on-duty police officer. That June, three terrorists wearing fake explosive vests drove a van into pedestrians on London Bridge before going on a stabbing spree. Eight people were killed. Later that month, a man was killed and many more were injured after a right-wing extremist drove a van into Muslim worshippers gathered near a North London Islamic center. Then in September, an explosive device was detonated on a London Underground train after it arrived at Parsons Green station during rush hour, injuring 23 people.
By the time these attacks took place, documents and interviews show, the NCIA was supposed to be up and running in the majority of forces nationwide. But delays meant that London’s Metropolitan Police Service remained stuck on the old system — and one of the most urgent problems identified by officers in their early reviews was that regions not yet on the NCIA had severe trouble communicating with those regions that were using it. One serving officer at the Metropolitan police said these communication issues were still ongoing in 2017.
The Police Integrity Unit at the Home Office replied to Thorne’s letter in January 2021, saying they had forwarded his concerns “to the Counter-Terrorism Headquarters.”
BuzzFeed News and BBC Newsnight asked the government whether it had responded to the concerns raised by Thorne and other officers. The spokesperson for Counter Terrorism Policing did not answer, saying only that the NCIA is “constantly being improved” and “any suggestion that concerns are ignored is simply not true.”
But an internal Home Office memo seen by BuzzFeed News shows that as of August 2020, authorities were still scrambling to fix at least one of the NCIA’s major flaws. The memo reports that about 1.7 million of its records — 20% of what was on the system at the time — were duplicates.
The memo proposes outsourcing a “deduplication solution,” which was likely to take several months to complete. “Due to this slow process, the benefits of this exercise will be slow to materialise,” the memo says, “yet it’s vital they are delivered” in order to “reduce the risk of intelligence failure.”
A serving officer at the Metropolitan police said in an interview in July last year that the NCIA’s search results were still inconsistent. His bosses had told him to run each search a minimum of four times over a 24-hour period to ensure nothing is missed. But the officer said this was “not a viable option” when working on imminent threats or actual attacks.
He felt that this left him with two options: Either go back into the old, pre-NCIA system to find what he’s looking for or work “blind.” ●
Introducing Jobbguru: Your Gateway to Career Success
The ultimate job platform is designed to connect job seekers with their dream career opportunities. Whether you're a recent graduate, a seasoned professional, or someone seeking a career change, Jobbguru provides you with the tools and resources to navigate the job market with ease.
Take the next step in your career with Jobbguru:
Don't let the perfect job opportunity pass you by. Join Jobbguru today and unlock a world of career possibilities. Start your journey towards professional success and discover your dream job with Jobbguru.
Originally posted on: https://www.buzzfeednews.com/article/richholmes/uk-manchester-bombing-counterterrorism-failures