The emergence and rise of Generative Artificial Intelligence (AI) have marked a significant milestone in the field of AI, which transforms the way humans and enterprises interact with technology. In just few months, this new AI technology is revolutionizing numerous industries. As a prominent example, among the various applications of Generative AI, ChatGPT has gained considerable attention and experienced very high adoption rates. ChatGPT, powered by advanced language models like OpenAI's GPT-4, can understand and generate human-like text, making it an invaluable tool for a wide range of purposes such as customer support, content creation, and virtual assistants. The ease of integration and adaptability of ChatGPT across different platforms and industries have contributed to its widespread adoption. As another example, there are numerous GenAI tools (e.g., GitHub Copilot) that can analyse and produce software code, which boosts the productivity of programmers and software analysts.

As Generative AI (GenAI) evolves, the capabilities and potential applications of tools like ChatGPT will expand to almost every industry and application domain. Cybersecurity seems to be no exception to this rule. GenAI is currently transforming the cybersecurity landscape by presenting both significant opportunities and challenges. Specifically, modern GenAI tools empower organizations to develop adaptive defence strategies and anticipate potential threats through machine learning and deep learning techniques. However, the same technology can also be exploited by malicious actors to create sophisticated cyber-attacks. This is the reason why security professionals must stay updated on the latest GenAI advancements. Modern security officers must prepare to harness GenAI to strengthen security measures and safeguard valuable digital assets. At the same time, they must be ready to confront hackers and adversaries using GenAI technologies to compromise key digital assets.   

Generative AI as a Powerful Cybersecurity Tool

GenAI can nowadays serve as powerful tool that can help cybersecurity teams to greatly improve the automation, productivity, and intelligence of their security operations. Some of the most prominent ways GenAI improves existing cybersecurity tools include:

• GenAI-Powered Intrusion Detection: One of the main characteristics of Generative AI tools is their ability to understand and generate complex data patterns. Such capabilities can play a crucial role in enhancing cybersecurity measures. For instance, one of the most prominent cybersecurity applications of GenAI is the development of advanced intrusion detection systems. By training AI systems on vast amounts of network traffic data, GenAI tools can learn to identify patterns and anomalies that may signify an ongoing cyber-attack. This enables organizations to detect and respond to threats more effectively and in real-time.
• Sophisticated Simulation Capabilities: GenAI can be used to create realistic and diverse simulations of cyber-attack scenarios. The latter simulations can allow cybersecurity professionals to test and improve their defence strategies. GenAI accelerates the generation of diverse, yet relevant attack scenarios. At the same time, it boosts the creation and simulation of complex attack strategies beyond conventional rule-based simulators. In particular, GenAI tools have the remarkable capability to model complex data patterns and generate realistic simulations, especially when trained on a diverse range of attack scenarios and techniques. Based on such training future GenAI tools will be able to generate new and previously unseen cyber threats that mimic the tactics and behaviours of real-world adversaries.
• Improved Preparedness and Training: The development of sophisticated simulations of cyber-attacks based on GenAI tools allows cybersecurity professionals to assess the resilience of their defence systems and identify potential weaknesses in their infrastructure. Additionally, these simulated attacks can be used for training purposes, helping security teams to develop their skills in detecting, analysing, and mitigating cyber threats. Overall, by exposing organizations to a wide variety of simulated attack scenarios, GenAI enables them to be better prepared for the ever-evolving landscape of cyber threats.
• Proactive Risk Identification and Secure Software Development: Apart from bolstering defence mechanisms, GenAI can also be employed to proactively identify and remediate security vulnerabilities. In this direction, the code analysis capabilities of GenAI can be leveraged. Specifically, by analysing large datasets of software code, GenAI can learn to recognize patterns associated with security flaws and suggest appropriate fixes. This can accelerate the process of vulnerability detection and patching, while at the same time helping organizations to develop more secure software from the outset.
• GenAI Powered Sandboxes: GenAI simulations can be also combined with sandbox environments for testing and experimentation. Sandboxes are testbed-like environments that are designed to support security testing, analysis of malicious software elements, as well as training of security professionals in realistic, nearly operational conditions. The integration of GenAI generated cyber threats into sandbox environments enables organizations to safely experiment with various defence strategies and countermeasures within controlled and safe environments that do not jeopardise the operation of their actual systems and data. This blending of sandboxes and GenAI tools also provides excellent opportunities for the continuous refinement of security measures. In this direction, the outcomes of the experiments are fed into GenAI tools to enable the generation of even more sophisticated, realistic, and challenging attack scenarios.
• Password Generation: GenAI tools can be also used to generate strong and unique passwords, making it more difficult for attackers to gain unauthorized access to sensitive systems and data.

The above-listed capabilities are already integrated in cyber-security tools and techniques. For instance, few weeks ago, Google unleased a Large Language Model (LLM) (SEC PaLM), that is trained on security use cases, while providing advanced cybersecurity analysis capabilities.

Generative AI as Enabler of Sophisticated Malicious Cyber-Attacks

Unfortunately, GenAI is not always a friend of cybersecurity officers. There are many cases where it can be also used by malicious parties to launch sophisticated attacks against organizations. Here are some of the ways adversarial parties are using GenAI:

• More convincing Phishing Attacks: GenAI tools can be used to create of highly convincing phishing emails. This is because they leverage LLMs that can be trained on very large datasets of legitimate emails. Attackers can therefore generate context-aware and personalized emails that are more likely to deceive recipients into clicking on malicious links or divulging sensitive information. In the next few years, we expect GenAI tools to lead to an increased success rate of phishing campaigns thanks to the great level of customization and precision of the text that they generate.
• Deepfakes Generation and Misinformation Spreading: Deepfakes are fabricated images, videos, or audio recordings that manipulate the appearance or voice of a person, making it seem as if they said or did something they never actually did. Many attackers use already deepfakes to create convincing fake identities, impersonate high-profile individuals, or spread misinformation. As a result, deepfakes and misinformation can enable a host of malicious activities, including social engineering attacks, blackmailing, or even discrediting individuals and organizations. Unfortunately, as GenAI advances in a rapid pace, the authenticity and quality of deepfakes will only improve, which will make it very challenging to distinguish between real and fake content. Moreover, the unprecedented penetration of GenAI tools will lower the time and effort needed for adversaries to create deepfakes and misinformation at scale. This will undoubtedly present new challenges for cybersecurity professionals in the ongoing battle against cyber threats.

Overall, GenAI represents a great opportunity for improving the cyber-resilience of modern organizations, while at the same time creating a host of new challenges. In this context, cyber-security professionals have no other option than learning how GenAI tools operate, including their inner-working and capabilities. Moreover, industrial organizations like critical infrastructure providers must invest in the training of security officers in GenAI technologies and tools. In parallel, it is very important for policy makers to develop a proper regulatory framework that can foster the benefits, while minimizing the risks of GenAI. The AI Act is certainly a first step in this direction. Nevertheless, as GenAI evolves in an unprecedented pace, additional regulatory initiatives, including security-focused initiatives must be undertaken.

The EU-CIP Coordination and Support Action (CSA) is closely monitoring GenAI developments in the cybersecurity areas, including emerging capabilities, technologies, and tools. As part of the project’s Knowledge Hub (to be launched later in 2023) we plan to provide access to resources (e.g., training resources, whitepapers, reports with foresight on new technologies) to help security professionals and critical infrastructure operators improve their GenAI knowledge and skills.

INNOV, June 2023