Maher maman. My name is Maher, and I am a lead security researcher in the Global Research Analysis Team at Kaspersky. My team and I, we work on cyber threat analysis, advanced persistent threat analysis, and the like. We cover, obviously, the Global Threat Landscape, but we are focused on different regions as well. A few housekeeping notes before we start. This session, obviously, is recorded. We will have short Q&A session or section at the end of a webinar. So feel free to use the Q&A section if you have anything you'd like to ask. All right, so without further ado, let's get going. So for the past few years, we've heard a lot that AI and machine learning are the silver bullets, right, for cybersecurity. Sorry, Mahir. Yep. I'm just wanting to tell you that we can't see your slides. Oh, you can't? No. I still, okay. Now I know what's happening. Now you should see it. Okay. Yes, that's perfect. Yeah, it was still post. All right. So one second and we're there. Okay. So yeah, I was, as I was saying for the past few years, you know, we've heard a lot that AI and machine learning are the silver bullets for cybersecurity and they will solve all the things to us and that we can use this technology to automate, you know, our way to safety and it will be the go-to tool for anything we have in the technology or in IT in general. And to be fair, the AI and ML did have or made the significant impact to our lives. But what you're seeing here on screen isn't a simulation. It's a glimpse really of the reality behind the headlines here. It's the constant change and the constant cyber attacks and the change in cyber attacks and the evolving threat landscape, which occasionally, and it's becoming even trendy and increasingly being sophisticated on the different levels. AI and ML did excel at known threats and they still do excel at known threats, even with the scale of detecting threats. But at the moment, an attacker deviates from the expected or from the usual shape of the pattern of the attack. The moment they introduce new techniques or new novelties, that's when the algorithms stumble. And this is where automation have issues. So that's when the carefully crafted models break down as well. And in those critical moments, when the automated defenses fail, it's not more AI we need. It's really us as human beings. So today in this session, we're going to explore those failure points, understand why AI and ML isn't perfect. And discuss why investing in skilled human analysts is more critical now than ever before. We'll look into some real world examples where human intuition and expertise were the only things really that standing between us and an organization and a catastrophic breach. Because it's important to understand the different things and the limitations and the limitations and the limitations of the technology by itself. So, I always like to start by establishing some context, some background so we can all be on the same page here. Now, what is AI? Can be, is it, obviously, as a cyber security industry, there's a lot of terminologies. And there's a lot of buzzwords, of course, but it's like different, each technology has different meanings depending on the context, of course. Now, I'm not sure if you know it, but there are three main types and categories of AI or artificial intelligence. And these three categories. And these three categories, whenever I ask, like, you know, in 2022, when ChatGPT started kicking off, and really, it made a lot of waves, even to date, and then we've seen a lot of expansions in different types of LLMs and chatbots. But I started by asking, like, do you know where ChatGPT falls under? Like, is it considered a narrow AI? Is it considered a narrow AI? Like, strong AI? Or super intelligent AI? And I do get mixed responses. But most of the responses will go into, like, a general AI. It's a strong AI because it's smart, it's giving a lot of things, and so forth. And then when I show this snippet here, that really deep dive into the different types of AIs and the different sections and the different categories and subcategories, then we'll see that the chatbot, or the LLM, or what is called the large language model, which we'll explain later on, but ChatGPT in this case, is really a part of a system. subcategory here, that is called generative AI models, or generative models, which are a category inside a category inside a category, and it's part of the weak AI. Now, we'll explain later on why this is weak, and what is the real power behind it, and how it works, and the risks. But it's important to know that this is what we are seeing now in chatbots. It's really nothing major, or groundbreaking, or groundbreaking thing that can change a lot of things. And it's still not considered general AI, or strong AI, that it can think on its own, it needs data, it needs a lot of things. Now, of course, many companies out there have, or did a lot of work on the narrow AI categories. General AI is what everyone else is also aiming for. Some specialized professors in this field say there are, we may not see general AI even before 100 years. Everyone is working toward that, of course, but there are technical or hardware limitations. There are different limitations. There are different limitations out there. But again, most of what you see nowadays is under narrow AI. And general AI is what everyone is seeking for. Now, the purpose, of course, is to automate, but we want to smartly automate things. And we will go into the different things that we want to automate here. But basically, the purpose of any artificial intelligence is to, to act fast on different things. And there are three learning modes, or maybe two learning modes, but there is a learning mode that you need to teach the artificial intelligence. So we need to teach the artificial intelligence to do what we want it to do. And it goes through a certain process, of course. And that data is not necessarily available all the time. Sometimes available, on the cloud, on the cloud, on the cloud, sometimes available in special storage, and so forth. And since we have different components here, mainly the data is on the cloud, with such thing like if you think about ChatGPT, the data is on the cloud, that you're submitting the data to the cloud, risks remain there as any cloud-based solution out there. Like think about webmail, then Gmail or Hotmail that you use, it's similar kind of risks. Now, of course, AI and the chatbots do introduce additional risks out there, which we'll talk about later on. So let's get into the four main terminologies or definitions that can establish, again, our understanding of this. And we'll start with the artificial intelligence or AI. And really, the artificial intelligence is the broad term. It's like when we say malware, it's a general term that contains a keylogger, ransomware, and so forth. And here, it's the same. Artificial intelligence is the broad term that has narrow AI, strong AI, superintelligent AI, and the like. And the point here is to create some kind of a machine or software. It's a software at the end of the day that relies on a hardware. And that software will, we want it to mimic or to behave like a human thinking to a certain extent. Meaning, imagine you are entering your office and you are having the guard, the security guard at the gate and validating each person. Now, each person has their own ID and maybe their fingerprint. And the security guard is validating those things. So what we want in this case is, of course, we want the software or some kind of artificial intelligence to validate the employees while they are entering the company in a way that we don't need a human standing there anymore. And therefore, we want to create something that has some kind of intelligence that we can teach it who is considered a valid employee and who is not. So we need to give it certain criteria that the artificial intelligence software or system can rely on to acknowledge is this an employee that can enter the facility or it's not and it should be rejected. And you can see that with the biometrics, for example. You can see that with the machines that can allow a person inside the access cards, the biometrics with the fingerprints and so forth. So that's in general what is artificial intelligence. But what is machine learning then? Machine learning, as we saw in the beginning, it is a subcategory of artificial intelligence. It is a subcategory of artificial intelligence. And machine learning is basically machine to machine kind of, so it's a software at the end of the day that can learn by itself from the data that we provided. So in this case, for example, we were speaking of the employees, we will tell it that an employee should have a first name, last name, and the employment ID, the employment ID, for example, and joining date. And we give it all these information to learn from. Now once the machine, the software itself learns from this data, it will then take this data and go in and ask the employee, whoever is there for their details. Now, if they supply the right details, they will be allowed in. Otherwise, they will be blocked. So in this case, the machine learning is replacing the human behavior in a computer system that can take action based on the validation process. Now, of course, the machine learning extends way more than this. And we can, if we provided the right data with the right models, which we'll talk about in a bit, it can also help us identify patterns. Patterns in the sense, like we see this employee and this employee are entering the facility in the morning, leaving in the evening, but these employees are not following the same process. Maybe they are entering and they are getting out late. So it sees a pattern and then it can show that this is anomalous behavior and this is legitimate behavior. So this is, again, again, again, again, again, again, again, again, what is machine learning. So again, it's a subtype of the broad term artificial intelligence. But then it comes with the LLM and all these buzzwords nowadays since ChatGPT in 2022 and many others nowadays. And LLM is again, and LLM is again on the chatbot sometimes, or GeneTurtive AI and other times depending on the context. But basically it's another subset or another type of artificial intelligence that can ingest a large amount of data and it can build an estimation on what is the next likely word that comes in the sentence. So, for example, if I have this text that you are seeing on screen now, if I type large in the chatbot, the chatbot itself will go back and it will look into the database that it has, the model that it has, and will look at it and say, okay, the user typed large. What is the most likely word that comes after large based on the data set? And it will give you a random thing because it doesn't have context. But when you start typing large language, it will probably bring model as a suggestion after large language. And that means based on the database of the LLM, it has calculated that model comes usually with high likelihood after language, which is after large. So, there you go that with the LLM, it doesn't have context. It just ingests a large amount of data. So, just chatbot itself just ingested a large amount of data, build a data set or a model that says the next likely keyword in the sentence is this with this likelihood or this estimate with this percentage. Now, if you think about it, this brings a lot of interesting capabilities or abilities. So, just a lot of interesting capabilities. And in this case, we can say, create to me an essay. So, it will go in and it will, it will, it will, chat GPT will go into its database and will say, okay, usually with essays, these keywords are likely there. So, it will assemble something based on the high likelihood of, or the statistical high likelihood in the database. So, this has also some risks, which we'll discuss later on. But at a higher level, large language models are simply a statistical analysis of the keywords in the text. When you ingest a lot of data, you will have a higher number of statistics inside the database, which will bring you the text that you generate. So, what is a rack? So, what is a rack then? Not sure if you heard it, but this is a common word as well in, currently nowadays in artificial intelligence and machine learning, which is retrieval augmented generation. In LLMs, when you create this database, it goes by file. So, the database is a file, which is also called the model. Now, the model that contains all the statistics is based on the ingested data that is fixed in time. So, therefore, when you go to, if you remember when you went to the first time to chat GPT, it will tell you that I have data up to date, up to this specific date. It was probably 2021, December 2021 or something. And that's because chat GPT, as the machine itself or the software itself, ingested data up to December 2021. Any data beyond December 2021 is not there in chat GPT, at least back then, the initial setup. And that brings a challenge, really, because the model itself, the artificial intelligence itself doesn't have up-to-date data. Even if it gets worse, even with private data, even with private data, because it doesn't have your private data if you want to use it, statistics, specific documents, financial trends, and so forth. So, the community created something else that's called retrieval augmented generation. And in this case, you will still be used. So, this is not an artificial intelligence or machine learning on its own, but it's a methodology. technology. And this is a technology used in contact or in connection with the LLM. So, in this case, we'll still use an LLM model. But the RAG in this case will allow the LLM to connect to an external holder that contains the data, the private data that we own. And you can do this on chat GPT. You can do this offline, even if you are kid, if you care about privacy and so forth. So in this case, I will still use ChatGPT, but I have something called knowledge. That knowledge or the context is simply a folder that you connect to ChatGPT. You will put your own files inside that folder and then you will be able to query ChatGPT or attach ChatGPT with the data that you want. So with those in mind, that context in mind, is AI really new? And of course, if we go back in time, we see the initial adoption of the technology at large, which is machine learning. It wasn't called AI as far as I remember. But really, the machine learning by itself started by around the year 2000, a little bit of cases or a few cases, maybe before 2000. But it started or the community started, cybersecurity industry started to use it at least in different contexts. Take the early antivirus solutions back then. And they all had this kind of engine that can classify the malware based on signatures, specific signatures. In the sense, even the firewall, the firewall, the old firewalls, they used to have also specific rule set where you can, where they can detect specific patterns in the network and can block it, of course. But since then, things moved forward. And then between 2010 and 2018, the tech really advanced and we started seeing more use of machine learning, but more efficient, more use cases. Even we started seeing it used specifically in SEM solutions, for example. And there we started defining certain rules in the dashboard of the SEM and saying, if you see a user that is not logging in within those timeframes, then flag me an alert or trigger an alert. And many other use cases. But basically, the machine learning will try to connect different things and I will teach it or we will teach it what we should detect. Or what we should flag as malicious or what we should flag as malicious or suspicious, at least for someone to look at. And from there onward, of course, to this year, we started seeing even more capabilities with the AI and ML. And since 2022, at least, we started seeing even a boom in LLM-enabled solutions. Even recently, even recently, even recently, even the MCP servers that allow us even to extend the capabilities of the LLM by itself. We discussed the RAG solution and RAG-enabled solution gives the end user a way to connect personal data or company data to the ChatGPT. So meaning you don't need to update ChatGPT by itself, just connect it to an external database. In this case, again, we are using the MCP solution. In this case, again, we are using the MCP solution. For example, MCP is also another technology that was created recently that can give us direct access to databases. You don't even need to create the folder that contains the data. You just ask the LLM to connect to the database and look for the information that you want. So from there, I mean, there's a lot of solutions that have been created. And we're still seeing this developing. But for the reference, all the machine learning based solutions have been there for quite some time. And they are kind of mature. Now, the LLM based solutions that we are seeing are mainly prototypes in general sense, but there's a lot of work to be done there. So it can be really like a finished product. So with all of this being machine learning based or as we call it now, artificial intelligence under the artificial intelligence of RELA. Back in 1997, IBM created the what they called the supercomputer back then to play chess. And it's played with the world champion back then Kasparov. And nobody called this supercomputer as artificial intelligence. They just called it supercomputer and it played chess. So meaning we still, I think we need to define what is really artificial intelligence. And it's not really new. We've been using this for a long time. And we need to really define what is artificial intelligence. That is the big answer. But again, the machine is machine learning is machine to machine and LLM or genitive AI or the chatbots have given us access as humans giving us as direct access to the data. So it's a human to machine or human to data nowadays. So why do we need this whole thing in the first place? Of course, we don't create solutions by for random random things of course everything has a utility and if we look at the total number of people using the internet it gradually increased now this data maybe can show unique people globally but even i would i would expect it even double if not triple this number just because each person may have nowadays multiple pcs or multiple devices that has internet as well so it depends on how you count the total number of people using the internet but basically this is unique people yet each person has multiple devices connected and this didn't stop here we got even it got even worse to where the number of people using social media so of course with this it gives a lot of exposure and a lot of exposure means a lot of risks a lot of threats a lot of potential for threats and so forth so imagine the number of users using the internet and the technology they will be exposed to different types of threats and viruses and phishing attacks and so forth so if you if you say like we have about 2 billion uh person using uh facebook uh or uh using technology uh you double it for different devices or multiple devices that's 4 billion and at least let's say by by minimum each one has one attack or cyber attack that's that's 4 billion attack uh at least in 2018 at least um so that brings us to the question like how do we analyze all these these kinds of threats we need some kind of these kinds of threats we need some kind of automation in 2005 uh we created something called auto wood becker and auto wood becker is a solution uh that we have built uh internally of course in 2005 which is the core of the uh endpoint protection solutions back then because we needed something to really automatically classify the files but the results we can't use the data and stake now um key log for we provided that this is how a malicious file look like or suspicious file look like and then it will go in and classify the files based on this training model now we'll talk about the the limitation more in the in the next slide but what i wanted to say here is that ai and machine learning has been there for a long time already and with that we have been so far able with the machine learning support not only to classify malware but also to track more than 900 threat groups and campaigns that are happening globally and with the classification and the scale of classifying files and and categorizing files by suspicious or benign and even by the different malware categories we are able to uniquely detect 400 000 unique sample per day and that that's the unique file per day imagine how many millions of files per day we our engines go through and this is to support us as as threat researchers of course because without such solutions we cannot have obviously individuals to look at different files and track different factors we need a solution that can scale detection and can help us in threat hunting and threat analysis so we can really take down malicious stuff on the internet out there and support our partners globally but with that there's always issues and in ml and machine learning there's always issues and limitations as any technology out there but in the terms of limitation you have you have heard for example you have heard for example false positives that the file was detected falsely by the antivirus and and other endpoint protection solutions but also nowadays even with the chatbots even with chat GPT and similar technologies we started seeing also some kind of false positive and that is called hallucination of course hallucination is a term we use with humans that human is hallucinating he doesn't know or she doesn't know what she's saying what she's talking about and LLM or chatbots also do hallucinate why because at the end of the day as we mentioned in the beginning LLMs or chatbots go through an enormous amount of data and will bid statistical analysis on the keywords which means it doesn't have context so even if you ask it a question and the AI doesn't have the back knowledge of the topic you are asking asking for it will still give you something does that something really make sense it doesn't matter to the AI because it's just giving you based on statistical analysis of the keywords it has in the database so therefore we need to understand that there are limitations like we have false positives in the traditional machine learning technologies we also have limitations in LLM based technologies which is nowadays hallucinations So mainly the limitations in AI in ML have well there's a lot of issues and limitations but I think this is a logical separation of these limitations and the first one I think we cannot go without it is dataset. Now I mentioned in the beginning we have different types of teaching for the artificial intelligence or the AI the AI and this teaching we are teaching well there's if you if you want to summarize how do you make an artificial intelligence or machine learning model it's basically few lines of code the critical element or the cornerstone for any machine learning or LLM is the data set itself without a good data set without a relevant data set a cleaned and really standardized data set with that we cannot have really a model we cannot build a model a machine learning or LLM model because the model itself will require certain consistency in the data so meaning if I have a word document that is written in some form or format I need to convert it to the same format as a PDF or as a photo it needs to be in the same format so that the AI or the machine learning algorithm will learn from it in the same way but also the data set have really some importance in the sense that we have we have to have some relevant data in the first place for example if we want to build a data set in the first place we have to build some technologies using LLM or generative AI or machine learning that can forecast financial trends right we need to have the right data for the financial trends we have to have historical data consistent data from different domains and then we prepare the data in a consistent way and then we use it with the machine learning to teach to learn from but also we have seen lots of the data that we have a lot of issues with the data as well that is bias in the data meaning you will only put financial data from a specific country without taking into consideration other external elements so therefore you will have specific outputs or results that are not necessarily realistic because you did not give enough data diverse data that can solve the the bias issue and if we if we give even more data that that becomes something called overfitting and overfitting is basically when you provide a lot of data from the same type the AI will not be able to forecast correctly or will not be able to give you the right output correctly so we have to balance between a little bit of bias little bit of diversity not giving too much from the same type of data so we can get optimum output but on this case as well I participated in an event in the past few months and I was I was hearing the different discussions on the AI and apparently there was a company that that work in the beauty beauty industry and they were building they were talking about building such technology AI based technology that can suggest beauty products or something in the beauty industry for for for their clients and they were teaching the the model or the AI based on data from that specific country so basically the AI will the beauty as defined by that certain country so meaning if you are coming from a different country the AI will not look at you as in the same form that's bias in the data so meaning the standard of beauty in a country might be different to another country the same goes for any type of data so this is what I'm talking about in the bias you have to be diverse as diverse as possible and the data to be consistent so you can get the optimum results but also domain expertise is another issue you cannot have a person who doesn't work in cyber security and tell him like you get this is the data that I want and just go through or teach model for me I need to use it if he doesn't have the domain expertise you'll not be able to understand or interpret the outcome and align and modify and adjust the models or adjust the data even to be consistent to be consistent with the results so in other sense domain expertise is critical as critical as the data set we are providing I think here we can maybe even extend it further and say we may find specific jobs that work with the data with domain expertise so for example we'll see maybe in the future that we some companies are requesting some companies are requesting people with financial experience to work on data set to teach AI I think that that could be plausible in the future but anyway the context is also another limitation AI and machine learning does not have context as I explained at the beginning LLM or chatbots do not have context because simply they are just calculating the next likely word word in the word in the word in the sentence based on statistical analysis now there are models or there are ways that companies are trying to implement so they can minimize the hallucinations and the effect of hallucinations but nevertheless it still does not have context meaning it doesn't know the relation for other events that you are asking maybe there are connections to something else so it doesn't have the context but whenever you start providing additional details it will give you additional details it will give you additional details it will give you the better result and this is where prompt engineering if you heard of this term before prompt engineering is one of the keywords as well out there with associated with LLMs and this is basically how the art of asking the right question to get the right answer from the LLM because I've done several hours of research on prompt engineering itself and if you ask the AI if you ask the AI a specific question or from one angle it gives you one result but you can influence the output if you ask it from a different angle and threat evolution that evolution is I think is one if not the major constant in any AI solution or machine learning solution and the threat evolves it changes form and shape the endpoint protection solutions will detect the signature if it's have high high accuracy that it's malicious but if the threat actor changes the form in a way maybe the ML or the machine learning or the AI will not be able to detect that in the first place and that goes even more when threats or cyber attacks get more sophisticated and complex, it will have even harder time to detect. So we need to keep on teaching as humans, keep on teaching the AI and updating the models with new signatures, with new ways being used by humans. At the end of the day, again, as we mentioned in the beginning, we are creating this technology to help us detect and scale the detection to detect cyber threats. So eventually, maybe until the strong AI or super intelligent AI, if it comes, until then, we will have to keep integrating ourselves with the AI so we can keep updating the models on the new ways. The threats are evolving. So the human element is essential for the machine learning and the artificial intelligence. It's essential to update it. It's essential to maintain it. We cannot live without it. And that's the SOC by itself. The fact that we have SOC level one, level two, level three in security operations center is mainly because of this. Because the SOC level one analyst will get the alerts from the AI or the machine learning. They will filter, validate, and then escalate to level two analysts should they need to. But the fact that we have level one, level two, level three is because we want the humans to validate what the AI or what the ML provided us as suspected alerts. Without it, we cannot operate. And again, we are coming to this after 20 years, almost 20 years of using machine learning. So with the same solutions and the same solutions and the like, we're still getting false positives. We're still getting the need for a human to validate. So at the end of the day, we get this question a lot like, do we, do we, are we really, are we really being replaced or will we get replaced by AI and MLF in the future? The short answer. The short answer is no, at least in the short to midterm. I don't think it will replace anybody. But as we had in the predictions two years ago, I still think that we will see more of the companies, the hiring companies seeking people with AI enabled skills, meaning they, they have to know how to use the artificial intelligence or the LLMs mainly, not artificial intelligence by itself, but the LLMs and the chatbots, they know how to use it. They know how to connect with it and so forth. But basically, at the end of the day, currently, after 20 years, we are still seeing false positives with machine learning based technologies and human oversight is essential in this case, because we need to validate if this is a real threat or not. And it happens a lot. We still see it to date. But even with the LLMs and chat GPT type of solutions, it does hallucinate. And therefore, you will see all kinds of similar companies that says, hey, we are giving you this output, but please validate before you rely on it. Otherwise, you will see a lot of weird things that doesn't relate. You will see in front of you. You will see in front of you a text, of course, a paragraph of some sort, but doesn't necessarily mean or it reflects reality. You have to validate. So humans are really to date are essential. For 20 years, we've been using machine learning and it has evolved really well. We still need a human to validate the false positives on those technologies. So humans are really to think about it. So humans are really to think about it. So humans are really to think about it. So humans have the intelligence to connect with different domains that there is not data at the stage. For example, a human can know what is happening in the external world and then rely on the AI over the world. For example, a human can know what is happening in the external world. So humans are really to think about it. So humans are really to think about it. So humans have more context are able to connect the different context and so forth. But remember as well that humans are really to think about it. But remember as well that humans are the ones updating the technology. The fact that we need humans to update the signatures with new types of form and shapes that gives it by itself that we still need humans to enable the artificial intelligence at least for the mid to short term. Now humans also have more capacity capacity to for problem solving. Of course the AI has fixed type of solving the issues doesn't have extended capabilities. If you notice in chat GPT for example and the like again I'm not picking on chat GPT but it's the most famous LLM out there. It's if you tell it no this is wrong. This is what you provided me is wrong. Then it will apologize and say okay this is wrong. This is the right thing that you requested. This is the right thing that you requested. And then whenever you start exposing that no this is not what I wanted multiple times it will start messing things around. So therefore you will need a person at the end of the day to validate to really think about the output that you were given. So where do we go from here. So where do we go from here. And what did we learn from the past and how we can go about it. So we can really learn from the past solutions that we have created in the past. And I think blending human and machine is the way forward at least for the mid term. Long term if strong AI became reality. I don't super intelligent AI. I think it was only hypothetical. I think it was only hypothetical. We will not reach it. But if strong AI becomes a reality. Then you may find the robots inside your home doing your job. Until then we have to blend in and see and benefit from the technology. But with with our human element involved. And therefore it's important to know how the technology works. Without the buzzwords. Without the buzzwords out there. How the technology works. What are the limitations. How we can benefit from it. How we can use it in the different ways. And the LLM and the chatbots nowadays gave humans the capability to directly connect to the data. To directly connect with the machine. However before in machine learning it was machine to machine. So that's an advantage that we can use. But definitely you cannot and you should not rely on blinding. On the solution. On the solution so far. Many solutions out there are still prototype kind of prototype. And you will need your personal expertise to to be brought in and validate what is the outcome. But again at the end of the day we are here to automate things like I want to scale the detection of files. Are these malicious or benign. Now automation doesn't really. Now automation doesn't really. It's not a silver bullet at the end of the day. It can have some pitfalls. So we need to validate. We need to analyze what is the output and really decipher the outcome. But also at the end of the day if you if you notice like if you ask a person to read a book or an article and summarize it. They will summarize it in their own way. Based on what they think is important to summarize the text. That same thing applies to AI and artificial intelligence. That same thing applies to AI and artificial intelligence. It will go in and summarize the text based on how it thinks it should summarize it. It's not necessarily how it should be or how you how you want it in the first place. So I think it's important to know these these limitations. And again threat actors keep on improving their techniques. AI and ML is models are taught. You teach them from specific type of data. If the new technique is not in the data that you taught the model. The AI will not know that this is a threat because it's learning from the data that you provided. Whether it be a cyber attack. Whether it be a financial fraud. If the AI doesn't know how an attack look like. Or it will not really identify new files that should look suspicious. And I think at the end of the day, whenever we keep learning, we keep evolving, being enabled with the machines and the artificial intelligence and the generative AI, I think we can use it to our advantage. At the same time, we will know the limitations, of course, and then therefore we will not blindly rely on it, which will essentially help us in our daily work, make us more efficient and scale our analysis, our work, make things work even faster and so forth. That's all I have for you. I hope you had a fun time and found the session informative and I look forward to any questions you may have. I think we still have a few more minutes for any questions. Okay, so I see the first question here. What are the key skills human analysts need to develop to remain effective with AI and automation for short? That's an important critical question. I think the domain expertise is what you need to bring in with AI and automation. Because at the end of the day, when you say, when you go in and ask, and this is a common question we got in the beginning, like, or we saw it really trending in the beginning. With the LLM out there, now we'll see cyber threats left and right and anyone from, like, bring in a new kid in the street and then they will create a malware and be a sophisticated threat actor. But that's not the case. Because when you ask the LLM or chat GPT a general question, it will answer you with a general answer. If you ask the LLM, if you ask the LLM, if you ask a specific question, then it will provide you specifics. So you cannot just say, hey, chat GPT, create me a ransomware. It will not create you a ransomware in the first place. Because the ransomware is a software that has a lot of components. So it's a complex software. So but if you are an experienced person in this domain, whether it be offense or defense, in this case, in this case, we are talking about ransomware, but let's say we want to detect, we want to create rules for detecting ransomware, right? So we'll ask the LLM to create for us detection rules. Now, how do you know that this is the right detection rule or not? If you don't have the domain expertise, then you will not be able to validate this. So as a human analyst, really learning the traditional things will help you a lot in analyzing the output of the AI and automation. So without without the right domain expertise, you will not you will fall short. Definitely. Good question. Any more question? Let me see here. Okay, this is done. Yep, if that's all. Thanks again for joining in. Going once, going twice. Thanks all. Have a great day ahead. Bye. Bye.