European Knowledge Hub and Policy Testbed for Critical Infrastructure Protection (EU-CIP) aims to consolidate information regarding critical infrastructure protection/critical infrastructure resilience (CIP/CIR) research to generate high quality, reliable and timely evidence to enhance the EU’s capability to benefit from research projects’ outcomes, technologies and policies. EU-CIP data collection and desk research methodology establishes an integral component for continuous desk research and data collection from various relevant sources.

Developed Automated Methodology

The EU-CIP project developed an automated data collection and desk research methodology shown in Fig. 1. This methodology is a continuous method of collating and synthesizing existing secondary data using both qualitative and quantitative research methods from various relevant sources, an initial process for desk research.

Fig 1. EU-CIP Automated Data Collection and Desk Research Methodology: Contributors: ENG, NRS, Innov-acts, EU-VRi

The methodology consists of the following six stages:

1 - Identify research topic: This stage identifies what information is to be collected. these topics are identified:

• Policies and standards relevant to CIP/CIR
• CIP/CIR technologies (including cutting edge tech)
• technology analysis (risk assessment, threat detection etc.)
• CIP/CIR research outcomes
• Research and innovation projects
• Sectorial white papers (covering the CER sectors)
• Local legislations report
• Capability needs and gaps in CIP/CIR
• State-of-the-art technologies techniques, methods and tools
• Technological, industrial, legal and ethical issues,
• CIP/CIR future trends
• Lessons learnt and best practices
• Market analysis on main CIP/CIR-related markets

2- Collect raw data: A variety of information collection modalities are considered such as desk research, interviews with sector specific consortium stakeholders, questionnaires on specific topic, workshops, and further secondary research based on proper curation of the collected data. The information to be collected is mainly focused on (i) security research activities and outcomes for CIP/CIR infrastructures, (ii) security solutions and technologies for CIP/CIR infrastructures with emphasis on cutting edge technologies that advance the state of the art and fill-in existing gaps in CIP/CIR infrastructures, (iii) security and infrastructure resilience polices, and (iv) related standards in collaboration with standards development organizations.

3 - Analyse collected data: This stage leverages methodologies and best practices for analysis of collected data such as the adaptation of the COIN (Context, Observation, Impact, and Next steps) methodology. This can be used to extract data-driven insights and composite policy indicators for policy makers in different areas, and tools for continuous desk research for conducting systematic desk research about CIP/CIR technologies, research initiatives, standards, and policies. For this purpose, different data sources can be utilized including research projects and their outcomes, standardization bodies, CIP/CIR research literature, technology reviews and research reports, and market analysis reports

4 - Curate collected data: This stage is responsible for proper curation of collected data from CIP/CIR by creating, organizing and maintaining data to support decision-making, academic needs, and scientific research. The main objective of this stage is to make data findable, accessible, traceable, and classifiable.

5 - Extract relevant information and insights: This stage facilitates the extraction of the collected data from CIP/CIR to give meaningful foresight and insights by applying different techniques such as statistics, data visualization, and software tools to identify patterns, trends, correlations and legal and ethical issues. For CIP/CIR, it is necessary to extract insights about gaps in current knowledge and solutions, technologies that can address these gaps, research outcomes and practices that can be employed.

6 - Present relevant information and insights: This stage presents the extracted information and insights from the collected data in a concise, neat and clean way. Although different formats such as a dashboard, a report, or a live demo can be selected for this purpose, data visualization seems to be a powerful tool. This practice can be useful to identify relevant patterns and key results for comparing different critical scenarios and their applicability for CIP/CIR sectors.

Applying the Methodology for Data Collection and Analysis

In this study, we extended the previous survey and applied developed methodology for data collection and analysis, however data curation stage was not completely used. Applying the developed methodology, this study identified two research topics: (i) capability needs and gaps in CIP/CIR, and (ii) technology analysis (risk assessment, threat detection, etc.), a questionnaire data collection modality has been applied, and excel has been used for analysis, extraction and presentation. Using both qualitative and quantitative research methods the following stages of this methodology were used: a) collect raw data based on questionnaires on specific topic, b) analyze collected data, c) extract relevant information and insights, and d) present relevant information and insights. We designed the questionnaire of the survey through a rigorous review of different articles and discussions with the experts from EU-CIP consortium and the European Cluster for Securing Critical Infrastructures (ECSCI) cluster.

We have a total of 30 responses from 25 organizations where both industry (56.7%), and academia (43.3%) actively participated. We identified which capabilities needs (top three from predefined options) are more important for CIP/CIR based on the respondents’ involvement as shown in Fig. 2. It can be observed from Fig. 2 that the specific capability needs with the respective percentile information are as follows: enhanced adaptability (33.3%), reduced response times (26.7%), increased transparency (10%), improved detection capabilities (56.7%), improved risk and impact assessment capabilities (36.7%), better integration of security tools (36.7%), solutions addressing cascading effects (13.3%), transformation of proactive and adaptive protection tools and methods (13.3%), better exploitation of information (10%), risk prediction and anticipation (33.3%), and training, reskilling and upskilling (43.3%). It can be analyzed that improved detection capabilities, training, reskilling and upskilling are the most selected capability needs from both industry and academia.  

Fig.2 Responses on capability needs in the CIP/CIR domain

As a next step we identified what the capability gaps (top three based on pre-defined options) are most important for CIP/CIR based on the respondents’ involvement as shown in Fig. 3. It can be observed from Fig. 3 that the specific capability gaps with their respective percentile information are as follows: poor automation (10%), lack of proper control of interconnectedness (13.3%), poor alignment of resilience indicators (13.3%), lack of agreed standards-based stress-testing procedures (13.3%), problems with the classification of IoT devices (10%), scalability in the Mitigation of Distributed Denial of Service (DDoS) attacks (10%), development and deployment of AI-based systems (40%), lack of holistic security management systems (53.3%), gaps in emergency management processes (40%), inability to cope with dynamically evolving threats (56.7%), and poor awareness about modern CIP/CIR challenges (40%). It can be observed that inability to cope with dynamically evolving threats and lack of holistic security management systems are the topmost capability gaps.

Fig.3 Responses on capability gaps in the CIP/CIR domain

From previous surveys, the following mostly voted state-of-the-art technologies and tools for CIP/CIR are as follows:

• Cyber-Physical Threat Intelligence
• Security Risk Assessment
• Impact assessment Tools
• Digital Twins
• Anomaly Detection
• Cybersecurity Tools
• Modelling of cascading effects and interdependencies between CIs

In view of that, experts were also inquired what the key challenges or obstacles are associated with the adoption and implementation of state-of-the-art technologies in the CIP/CIR domain. The top identified key challenges were lack of regulatory and policy standards, lack of quality data, high resources consumption for adoption of advanced technologies, lack of adequate training for operators, complexity for integration with legacy systems, and enlargement of the attack surface, among others. Also, it was identified that prevention (50%) as a major disaster phase, in terms of applying technology and tool development for CIP/CIR. The other important stages are preparedness (32.1%), response (25%), recovery (35.7%), and others (3.6%).

Fig.4 Responses on disaster phases in the CIP/CIR domain

In this blog, we have compiled and analyzed some of the important data of the questionaries. There are still other significant data for how and what information will be fed to the EU-CIP FAIR Observatory based on the inputs from some of the partners of EU-CIP. Additionally, there is also a possibility that the collected data be relevant to other EU-CIP activities.

Outlook and Future Directions

Based on the development of the methodology and the exercise of its application the developed methodology establishes a data collection, information monitoring and analysis methodology for continually collecting, analyzing, curating, extracting and presenting information and insights about CIP/CIR infrastructures. The EU-CIP project can conduct a continuous desk research on CIP/CIR research outcomes, R&I projects, technologies, standards, and policies to be up to date with related developments, and to have a continuous stream of information for further analysis and curation. As it has been suggested the Innovation Watch tool (developed in H2020 ENCIRCLE project) can be integrated to automate the process of periodically searching, aggregating and consolidating the web-based data ensuring prompt up-to-date information in the observatory. The EU-CIP project will use this methodology to generate value-added CIP/CIR reporting on aggregated and consolidated view of (i) the capability needs and gaps, (ii) the state-of-the-art  technologies, techniques, methods and tools that can contribute to fill the identified capability gaps, and (iii) outcomes (including on technological, industrial, legal and ethical issues), future trends, lessons learnt and best practices derived from past and current security research projects. Several supporting tools such Templates, Survey tool, Innovation Watch tool, Slido to be used during workshop (https://www.slido.com/, etc. need to be evaluated and integrated. One of the future directions can also be to integrate AI and feedback loop with the developed methodology to automate the process of periodically searching, aggregating and consolidating the web-based data.

By Habtamu Abie and Sandeep Pirbhulal    

Norsk Regnesentral (NRS)