- Does this problem really affect my business too?
- What exactly is an AI jailbreak, and how does it work today?
- Why is the chatbot's personality the weak point to protect?
- The Italian context: how exposed are SMEs?
- What concrete scenarios should I expect in my business?
- Are AI vendors' protections enough, or do I need to do more?
- How do you build a concrete four-layer defense?
- What are the most common mistakes that expose SMEs to risk?
- How can I tell if my chatbot is already at risk or under attack?
- So, is it worth investing in chatbot security for my business?
- How we see it at SHM
- Frequently Asked Questions
- Hackers no longer "break" chatbots head-on: they manipulate them through their own identity, tricking them into behaving as if the security rules didn't exist.
- The latest jailbreak techniques bypass security filters in 92-100% of cases, with minimal impact on the model's performance.
- 76% of Italian SMEs have not invested in AI, and 80% of workers use personal, non-company AI tools: a combination that significantly amplifies the risk.
- Four layers of protection — data, monitoring, system prompt, and governance — reduce risk without sacrificing the chatbot's usefulness.
Updated October 13, 2026
From January to April 2026, malware attacks disguised as AI services grew almost fivefold compared with the same period in 2025. If your company uses a chatbot, that chatbot is already a target.
Does this problem really affect my business too?
Yes, and more than you think. SMEs are prime targets precisely because they adopt chatbots without a structured security roadmap.
When people talk about attacks on AI chatbots, many business owners' first reaction is: "That's a big-company problem, not ours." The data says otherwise.
From January to April 2026, Kaspersky's security systems detected over 33,300 malware attacks against SMEs in which the malicious software was disguised as well-known AI services — almost five times more than in the same period of 2025, according to BitMat. The most common lures posed as ChatGPT, Claude, and DeepSeek.
SMEs are prime targets for a simple reason: they have limited security resources but handle valuable data — customer data, contracts, financial information. A poorly configured chatbot becomes the easiest way in.
| Your situation | Risk | why |
|---|---|---|
| Chatbot integrated with a CRM or customer database | High | Can expose third-party data if manipulated |
| Virtual assistant for e-commerce | High | Can authorize unintended actions (discounts, orders) |
| Internal bot for HR or onboarding | Medium-high | Handles sensitive documents and contract data |
| B2B technical support chatbot | Medium | Can reveal architectural details of customer systems |
| Informational chatbot with no data access | Low | Reputational risk, not data exfiltration |
What exactly is an AI jailbreak, and how does it work today?
An AI jailbreak is a technique for bypassing a chatbot's security instructions, tricking it into behaving as if those rules didn't exist. The techniques have evolved a lot since the early years.
In the first wave of commercial chatbots, breaking an AI system was almost trivial. All it took was phrasing a request indirectly or pretending to set up a fictional story. Today, models have received extra layers of protection, and attack techniques have evolved accordingly.
The key concept is the personality exploit. Modern LLMs — Large Language Models, the language engines behind ChatGPT, Claude, and similar tools — are not simple answer engines. They are systems trained to maintain a consistent tone, style, and set of values. That very consistency becomes an attack surface.
A skilled attacker builds conversational scenarios that lead the model to "believe" it is operating in a different context from the real one: by simulating an administrator role, a fictional character, or an authorized test scenario. The analysis published by The Verge documents this evolution in detail.
| Technique | How it works | Estimated effectiveness |
|---|---|---|
| Persona injection | The model is assigned an alternative identity with no restrictions | High |
| Many-shot jailbreaking | Long sequences of examples that gradually shape the model's behavior | High |
| Crescendo attack | Harmful requests are introduced gradually | Medium-high |
| Boundary Point Jailbreaking (2026) | Advanced technique that bypasses security classifiers | 92-100% success |
| Indirect prompt injection | The malicious content comes from external sources (e.g. uploaded documents) | Medium |
The newest technique, Boundary Point Jailbreaking, delivers a security filter evasion rate between 92% and 100%, with a drop in model performance of just 2%, according to Cybersecurity360. In practice: the attacker gets malicious answers while the chatbot still looks perfectly functional.
Why is the chatbot's personality the weak point to protect?
The chatbot doesn't see security instructions as inviolable rules: it treats them as part of the conversational context. Whoever manages to rewrite that context rewrites the rules too.
Every chatbot receives a system prompt: a set of initial instructions that define its behavior. What it can do, what it can't do, how it should respond. These instructions are the main application-level security mechanism.
The problem is structural. The model doesn't clearly distinguish between "rules" and "conversational context." If an attacker builds a convincing enough context, they can implicitly rewrite those rules without touching a single line of code.
The attack surface grows even larger when chatbots are connected to external tools. A model integrated with a customer database or a booking system is not just a source of potentially wrong information: it becomes a vector for data exfiltration or unauthorized actions. That's why, in the world of AI and process automation, chatbot security has become a priority for SMEs too.
Prompt Injection, the injection of malicious instructions into the conversation flow, is the number 1 vulnerability in the OWASP Top 10 for LLM Applications 2025 (OWASP is the international organization that catalogs the most critical software vulnerabilities). According to Vectra AI, it is present in over 73% of production AI deployments analyzed during security audits.
The Italian context: how exposed are SMEs?
Italian SMEs start from a disadvantage: low structured adoption of AI, little training and a widespread Shadow AI phenomenon that amplifies the risks.
In 2025, Italy recorded 116,498 cyberattacks, one every 5 minutes, at a rate 17% higher than the global average, according to Il Giornale delle PMI. The Rapporto Clusit 2026 reports 507 serious incidents in Italy in 2025, 9.6% of the global total.
There's another worrying figure: only 15.7% of Italian SMEs with at least 10 employees use artificial intelligence, compared with 53.1% of large companies (ISTAT 2025 data). Those that do use it often lack adequate security measures.
Shadow AI, the use of personal AI tools not provided by the company, affects 8 in 10 workers, according to the HR Innovation Practice Observatory at Politecnico di Milano (2026 data). This means employees use ChatGPT, Claude or other tools with personal accounts, with no company policies, no logging, no control. Fertile ground for jailbreak and prompt injection.
What concrete scenarios should I expect in my business?
The risk isn't theoretical: it depends on how your chatbot is configured and what data it can access. Here are four realistic scenarios for Italian SMEs.
Personality exploits are not a threat reserved for large corporations. In fact, SMEs are often easier targets because their chatbots are set up quickly, without systematic security testing.
| Company type | Attack scenario | Data at risk | Potential impact |
|---|---|---|---|
| E-commerce | The bot is tricked into confirming unauthorized orders or applying invalid discounts | Payment data, pricing policies | Direct financial loss, reputational damage |
| Professional studio | The bot exposes documents uploaded by customers (contracts, appraisals, financial statements) | Confidential third-party data | GDPR violation, loss of trust |
| B2B manufacturing company | The technical support bot reveals architectural details of customer systems | Technical specifications, configurations | Competitive risk, contractual liability |
| Retail chain with an HR bot | The bot exposes internal policies, contract data or employee information | Employee personal data | GDPR violation, litigation |
As analyses of hacker attacks on SMEs in 2026 also explain, the entry point changes but the goal stays the same: getting access to data that has commercial value or can be used for extortion. The chatbot is simply the new way in.
Are AI vendors' protections enough, or do I need to do more?
Vendor protections are necessary but not sufficient. The problem is partly inherent to the architecture of language models: no vendor can guarantee complete protection.
The main AI model providers, from OpenAI to Anthropic, from Google to Meta, constantly invest in alignment (aligning the model's behavior with desired values) and red teaming (simulating attacks internally to find vulnerabilities before hackers do). Even so, the problem remains open.
The reason is structural: there is still no universal method to cleanly separate security instructions from conversational context. As MIT Technology Review notes, the jailbreak problem is partly inherent to the transformer architecture these models are built on.
Relying solely on vendor protections is therefore not enough. SMEs that run chatbots in production need to add their own layers of security, tailored to their industry and the data the system handles.
How do you build a concrete four-layer defense?
Business chatbot security is built on four distinct levels: data, monitoring, system prompt and governance. Each addresses a specific dimension of risk.
The trade-off between usability and security is real but manageable. An overly restricted chatbot becomes rigid and frustrating. A system that's too flexible is vulnerable. The solution is to design the two goals so they support each other, not to choose between them.
- 1 · IT Manager / VendorLevel 1 — Data perimeterThe chatbot accesses only the data strictly necessary for its function. Principle of least privilege: a customer service bot doesn't need access to financial data. Data segregation reduces the potential damage of an exploit.
- 2 · IT Manager / VendorLevel 2 — Conversation monitoringReal-time logging and analysis of conversations. Anomalous patterns are identified: unusual sequences of questions, attempts to redefine the bot's role, repeated requests on sensitive topics. Anomaly detection tools automate the process.
- 3 · Vendor / Internal teamLevel 3 — System prompt architectureSystem instructions define not only what the bot can do, but also how to recognize and handle manipulation attempts. The system is regularly tested with simulated attack scenarios.
- 4 · Management / Internal teamLevel 4 — Governance and continuous updatesAI security is not a one-off project. It requires periodic reviews, updates to system instructions and team training. Keep an active line of communication with the model vendor for updates on known vulnerabilities.
What are the most common mistakes that expose SMEs to risk?
Most incidents don't come from sophisticated attacks, but from superficial configurations and a lack of internal policies on the use of AI tools.
Shadow AI, the use of personal, non-company AI tools, is closely tied to jailbreak risk. If employees use ChatGPT with personal accounts to process company data, there's no logging, no control, and no way to step in when an incident happens. It's also worth exploring how AI roles in the team shape the governance of the tools you adopt.
| Common mistake | Why it is risky | How to fix it |
|---|---|---|
| Chatbot with access to all company data | One exploit exposes the entire database | Apply the principle of least privilege |
| No conversation logging | Impossible to detect attacks in progress or after an incident | Enable logging and anomaly analysis |
| Generic, untested system prompt | Vulnerable to persona injection and crescendo attacks | Design and test the prompt with attack scenarios |
| No policy on personal AI use | Shadow AI amplifies risk with no control | Define clear policies and train your team |
| Unmonitored model updates | New versions can introduce new vulnerabilities | Keep an open channel with your vendor |
| Chatbot not tested before go-live | Vulnerabilities discovered only after an incident | Internal red teaming before launch |
How can I tell if my chatbot is already at risk or under attack?
There are early warning signs and measurable indicators that help you spot manipulation attempts before they cause real damage.
Phishing and jailbreaks have something in common: they often leave traces in conversations before they do damage. As the analyses of 2026 phishing attacks show, proactive monitoring is the difference between catching an attack in progress and discovering it after the breach.
| Indicator | What it signals | Alert threshold |
|---|---|---|
| Unusual sequences of questions on the same sensitive topic | Crescendo attack attempt | 3+ related questions in one session |
| Requests to redefine the bot's role or identity | Persona injection attempt | Any occurrence |
| Questions about system instructions or configuration | System prompt extraction attempt | Any occurrence |
| Bot responses outside the defined perimeter | Possible exploit in progress | Continuous monitoring |
| Spikes of anomalous sessions at unusual hours | Automated attack | Change of >50% from the average |
So, is it worth investing in chatbot security for my business?
It's worth it if you already have a chatbot in production connected to company data (CRM, customer databases, management systems) and have never run a dedicated security audit. The risk is real, and the cost of an incident far exceeds the cost of a secure setup.
Somewhere in between if you're considering a chatbot but haven't yet defined which data it will access. Before going live, it's worth designing the security architecture together with the system, not adding it afterward.
Not yet if you use a purely informational chatbot with no access to company data and no stored conversations. The risk is low, but keep an eye on how attack techniques evolve and on your internal Shadow AI policies.
How we see it at SHM
For us, artificial intelligence is a tool that should free companies from routine, not create new hidden risks. When we integrate chatbots and virtual assistants for our clients, security isn't a layer added at the end: it's part of the architecture from day one.
We've worked with companies that already had a chatbot in production but had never tested the system prompt against attack scenarios. In one case, a simple audit revealed that the bot could be tricked into disclosing prices reserved for premium customers. It wasn't a sophisticated attack: all it took was phrasing the request indirectly. We redesigned the data perimeter and the system instructions in less than a week.
We believe simplifying digital is the highest form of expertise, and that applies to security too: it's not about building impenetrable fortresses, but systems that work well and are hard to manipulate.
If you're considering adopting or reviewing a business chatbot, our chatbot and virtual assistant development service includes system prompt design and security testing. If you want to understand how to integrate AI in a structured way, the AI training for companies program also covers operational risks. If you'd rather start with the big picture, our strategic AI integration service is the best place to begin.
You can also explore the other articles in the AI and process automation cluster to see how Italian SMEs are tackling these topics in practice.
Frequently Asked Questions
Vendor protections are a necessary starting point, but not a sufficient one. The problem is structural: language models don't clearly separate security instructions from conversational context. The latest jailbreak techniques bypass filters in 92-100% of cases. You need to add your own layers of protection: a data perimeter, conversation monitoring, and a system prompt that is designed and tested.
It depends on the complexity of the deployment and the data the chatbot can access. For a simple chatbot with limited access, an audit and a system prompt review take just a few days. For systems integrated with a CRM or customer databases, the investment is larger but proportionate to the risk: a customer data breach can cost far more in GDPR fines and reputational damage.
It's called Shadow AI: 8 out of 10 workers in Italy use personal, non-company AI tools, according to the Politecnico di Milano. The risk is that company data ends up processed on platforms with no logging, no oversight and no security policies. The solution isn't banning AI, but defining clear policies and providing suitable company tools with the right protections.
Without conversation logging, you can't. The first step is to turn on session monitoring and look for unusual patterns: odd sequences of questions on the same sensitive topic, requests to redefine the bot's role, questions about system instructions. Many attacks leave traces in conversations before they cause real damage.
The GDPR applies to any system that processes personal data, chatbots included. If your chatbot handles customer or employee data, you're responsible for protecting it. A breach caused by a jailbreak is no excuse: you're required to adopt appropriate technical and organizational measures. The European AI Act, which is being phased in, adds further obligations for high-risk AI systems.
If your chatbot has no access to company data and conversations aren't stored, the risk is low. The main issue here is reputational: an attacker could get the bot to respond inappropriately or misleadingly. Still, keep an eye on internal Shadow AI and set clear policies on which AI tools employees can use for work.
AI jailbreaking isn't a technical problem to hand off to your vendor: it's a business risk, and you manage it the same way you manage any other operational risk.
Related articles
Discover more articles exploring similar topics, selected to offer you a more complete and stimulating perspective. Each piece of content is carefully chosen to enrich your experience.
