There has been a lot of talk about Artificial Intelligence (AI) in recent years. It is certainly a polarizing subject. While it raises hopes about the future of technology and what humanity is capable of, it also raises questions around human control and technological determination. There are those who worry that Artificial Intelligence is going to ‘take people’s jobs’, or even take over the world, and that the world will end up like a dystopian ‘Terminator’ style film.
Despite all its controversy, AI and related tools work to enhance human life and wellbeing. They also have a long way to go before they could become threatening. However, advancements to AI technology are rapidly speeding up. For example, the changes between the first iteration of Open AI’s Chat GPT and GPT 4 are monumental.
When ChatGPT launched in 2018, it offered basic conversational capabilities. GPT-2 introduced improvements in text generation and understanding. GPT-3 dramatically scaled up the model’s size, enhancing its ability to understand context and perform tasks without explicit training. GPT-3.5 refined these capabilities further, leading to today’s GPT-4, which offers superior understanding, creativity, and the ability to interpret and generate various data types beyond text, such as images and graphs.
The same goes for cybersecurity and the use of AI in operational and defensive environments.
What is Generative AI, and an LLM?
The release of ChatGPT raised public awareness of generative AI and its capabilities exponentially. It has also led to a fast rise of other new AI releases, including Stability AI (Stable Diffusion), Google (Bard) and Microsoft (Bing Chat and Copilot).
All these generative AI tools are based on Large Language Models (LLMs). Algorithms that apply deep learning against large data sets to process and predict new content.
For example, GPT-3.5 is the model set on which the free/public version of ChatGPT is based. GPT-4 improves on GPT-3.5 and is the model set used for paid users of ChatGPT and by Bing Chat / MS Copilot services.
Different generative AI tools are designed to focus on producing different types of content, including but not limited to:
- ChatGPT produces text-based content (this can now be extended using plug-ins for paid users).
- MidJourney produces images from text prompts.
- ElevenLabs focuses on text to speech and voice cloning.
- ai can analyze a video, change what that person is saying and even how their mouth moves, into any language, seamlessly.
- Intercom/Liveperson etc. focus on customer chatbot functions on websites.
An LLM is a very large data set that can be used to create data points based on text input, and use all of the information at its disposal to calculate the best answer. But this means that LLMs are only as good as their data sets. All AI tools are highly dependent on their training. The process starts by asking a question, it provides an answer, then confirms if the answer was acceptable and correct. This positive reinforcement ensures similar worded questions have the same outputs. If data is not clean, the answers will not come back correct. More questions will then need to be asked to correct the output. Then the LLM has new data and learns from it.
Errors frequently occur. It is all based on mathematical calculations, therefore is limited to using what is written no matter the sentiment, emotion or social input (like sarcasm). It is all taken very literally. And the only way it can lean and reduce errors is to tell it what a right answer looks like. Without this, the outputs could end up being completely false, as they were when Google had to take their AI engine, Gemini AI, offline when it was generating images that were radically inaccurate and not even close to being reliable.
The effectiveness and safety of LLMs, particularly in critical fields like cybersecurity, depend heavily on the integrity and quality of their training data. Persistent attempts by malicious actors to introduce false information pose significant challenges, potentially compromising the model’s outputs and, by extension, the security postures of those relying on these tools for information and guidance.
This underscores the importance of continuous monitoring, updating and curating sources of used in training LLMs, as well as developing robust mechanisms to detect and mitigate the influence of incorrect information.
Operational Applications:
Operational applications of AI are diverse, and are already significantly impacting various domains.
AI chatbots have become prevalent in customer services as an example, providing efficient and scalable solutions, with chatbots such as Drift and LivePerson becoming increasingly commonplace. This is to speed up the qualification in or out for customers having their questions answered without human interaction. Saving money for suppliers operationally and better standardizing customer engagement responses.
In security, AI is being integrated into SOAR (Security Orchestration, Automation, and Response) products for straightforward tasks like modifying firewall rules or managing IP addresses, enhancing response capabilities.
Meanwhile, process management benefits from AI’s ability to convert complex human-driven processes into automated, systematic functions, or in other words, turning qualitative inputs into quantitative data.
AI also plays a significant role in creating marketing materials such as images and videos, as well as in analyzing large data sets, whether numerical or not.
Essentially AI is invaluable for standardizing responses in tasks with repetitive functions that encounter minor variations. With a sufficiently large data set, nearly any task can be automated, showcasing its transformative potential across industries.
Offensive Applications:
If you consider how you can essentially apply any outcome to an LLM depending on the data set, there is an abundance of breached data and open sourced information online. Previous breaches from hashes, to source code and customer information, to armies of hackers that all share information. This means there is a good chance that AI will be used to change small things in previously used breaches to bypass security, based on signature detection, for example.
The most prominent ways in which threat actors are currently using generative AI tools include:
- Prompt Engineering jailbreaks – These are prompts like the “Do Anything Now” prompt which is intended to bypass the core behavioral prompts applied by the owner of the generative AI tools (example – the limitations placed by OpenAI on ChatGPT).
- Malware coding – In the same manner which software engineers are using generative AI tools to speed up their coding tasks, malware writers (particularly script kiddies) are using these tools to accelerate their capabilities.
- More effective Phishing / Social Engineering campaigns – Generative AI tools are being used to write more convincing phishing emails free of typical grammatical errors.
- Privacy concerns – Different tools may have different levels of privacy and incorrect usage may result in data leakage. An example here is that the public / free version of ChatGPT uses data input from prompts for further training of the model, whereas the paid version provides an option to exclude your data from being used for training. There was a specific case making headlines where Samsung banned use of tools like ChatGPT after discovering that staff uploaded sensitive code to the platform.
- Shadow AI usage – Similar to the principle of shadow IT, where multiple business units or individuals within an organization were purchasing and utilizing cloud services without the knowledge of the IT team, there is a concern that users may behave similarly with the use of AI.
Defensive Applications
When you address enforceable cyber security using AI and LLMs, you must be clear about why you are bringing in AI in the first place. Is it alert fatigue? Is it slow SIEM data sets based on SQL that take hours to compute an output? Is it a complicated process that if left to humans, has a chance of going wrong (JML is a good example)? Whatever it is, be tight on defining its function.
It is all still based on data and fine tuning. Whatever the function is that you define for defensive capabilities in security, it will make rules and decisions based on the input of that data telemetry.
For example – multiple vendors are using AI and LLMs to make faster decisions based on trends for product development, feature enrichment and utilization, for example. Benefit? Faster decisions based on decent, clean data. No subjectivity. It is evidence-based.
- Dynamic watchlists for activity and devices
This leverages AI to monitor and evaluate activities and devices in real-time, identifying potential threats based on historical data and evolving patterns. It dynamically adjusts watchlists, ensuring the system focuses on the most relevant threats at any given time. This adaptability reduces the chances of overlooking emerging threats and decreases false positives, making security monitoring more efficient.
- Custom rule functionality based on data
AI and LLMs enable sophisticated, data-driven rules for cybersecurity systems to be created. These rules can adapt to changing threat landscapes by learning from new data, allowing for a more responsive and proactive security posture. And, of course, custom rules crafted from deep insights into an organization’s specific data environment help in pinpointing unusual or malicious activity with greater accuracy.
- Minor variations in signature-based detection
Traditional signature-based detection methods can be rigid, failing to catch malware that has been slightly modified. AI enhances these methods by identifying minor variations in known malware signatures, so offers the ability to catch modified threats that would otherwise slip through. This application of AI makes signature-based detection more robust and less susceptible to evasion techniques.
- Tailor made threat intelligence (external threat inputs + customer telemetry data = company specific threat intelligence)
By combining external threat intelligence with an organization’s specific telemetry data, AI can develop customized threat intelligence, applicable to the unique security concerns of the company. This tailor-made approach ensures defensive strategies are highly relevant and effective, focusing on protecting against the most pertinent threats.
- Big Data Analytics
Given that the vast amount of data generated by modern enterprises can be overwhelming for traditional security tools, AI excels in analyzing big data, extracting meaningful patterns and identifying anomalies that might indicate a security threat. This enables organizations to leverage their data more effectively in defending against attacks.
- Deep behavioral analytics
Deep behavioral analytics go beyond simple pattern recognition, analyzing the nuanced behaviors of users and systems to identify malicious activity that might not trigger traditional detection mechanisms. By understanding the baseline of normal behavior, AI-driven systems can detect deviations that suggest a compromise or attack, often before the damage is done.
Conclusion
Generative AI and its applications are still in their infancy. We are barely scratching the surface of the possibilities. The strides made between GPT-1 and GPT-4 are huge, and it is exciting for the future.
How we govern these AI tools is going to be paramount for both countries and organizations.
But until they can discern emotional responses, the threat is purely mathematical. AI generated threats are growing and they will not stop. Just make sure if you bring AI in, it is for the right reasons – and is locked down as tight as it can be.
