The hidden risks of shadow AI – National Cyber Security Centre

Our advice & guidance covers a broad range of topicsResources for individuals and organisations in the UK who have experienced an online scam or cyber attack.Find a range of products & services from NCSC and certified 3rd party suppliersWorking with industry, government and academia to support the next generation of researchers, students and cyber security professionalsAll the latest information to help you keep track of what's happeningUnderstanding why staff use unapproved AI tools is key to managing the security challenges they can create.Andriy Onufriyenko via Getty ImagesOver the past few years, the use of artificial intelligence (AI) has grown rapidly in many workplaces with employees increasingly exploring how such tools can be incorporated into their jobs.AI can help people complete tasks more quickly, improve decision-making, save costs and increase productivity.However, organisations’ policies and guidance, which should reflect and manage the risks associated with using these new technologies, have not always developed at the same pace.Rather than preventing them from using AI, this can mean employees turn to using AI tools that have not been approved by their organisation, introducing new cyber security risks that can be hard to identify.Shadow AI describes the use of AI technology which isn’t captured in an organisation’s approved systems and processes. It is a form of shadow IT (or ‘grey IT’).Recent research suggests that using shadow AI is widespread, with one study finding that nearly three-quarters of employees (71%) reported using AI tools that have not been approved by their employer.Where cyber security policies cannot meet business needs, organisations are likely to continue seeing their employees adopt new AI services before they have had time to assess them and provide approved alternatives. This trend is likely to be reinforced as AI capabilities become increasingly affordable and readily available.The use of shadow AI can create risks that organisations may struggle to identify and in turn manage, potentially resulting in breaches and security incidents. For example:Providing shadow AI access to company or customer data likely increases the risk of data breaches, intellectual property loss and failure to meet regulatory requirements.Employees who transfer sensitive or proprietary information to consumer AI services will likely reduce the organisation’s visibility and control over that information. This is because that information may be stored, retained or used to improve the service – outside established security and governance arrangements – unless specific privacy controls are in place.AI agents are complex pieces of software that can have critical security vulnerabilities. If an attacker successfully exploits a vulnerability, they can gain access to the same data, services, and privileges that the agent has legitimate access to.Attackers are highly likely to use agents with looser guardrails to exploit any vulnerabilities or misconfigurations in the wider corporate IT system.The NCSC is not recommending that individuals stop using AI – but when turning to these tools for assistance with a work task, think carefully about which apps and services you are using before you share data.It may feel natural to stick with using the same AI service that you are familiar with from your personal life – but using systems that are not corporately approved can present real problems for your employer.For organisations, the challenge is ensuring that employees have access to AI tools that meet their needs while managing cyber risk appropriately.The use of shadow AI is unlikely to disappear completely. As with shadow IT more broadly, the goal should be to reduce risk rather than assume it can be eliminated. To do this, organisations should:You cannot manage what you do not know. By raising awareness of the risks of shadow AI use within your organisation and understanding the needs of employees, you can help them get the benefits of new technologies while using them securely.The NCSC’s cyber security culture principles.An NCSC blog on thinking carefully before adopting agentic AI.Senior Cloud ResearcherHow to create the right cultural conditions in an organisation that support and encourage people to carry out the desired cyber security behaviours.When it comes to using agentic AI, make sure you can walk before you run.A revolution is underway at the intersection of AI and cyber security.source