Is AI dangerous? What nearly four years of daily use taught me about the risks small businesses should take seriously

I started using ChatGPT in November 2022

I started using ChatGPT in November 2022, a few days after it launched, for the least glamorous job in marketing: outbound email. I had a list of prospects, a template I was tired of, and a chatbot that would rewrite the same pitch 20 ways without complaining. It was a toy that happened to be useful.

Within months, it was the first tab I opened. It got me past the blank page on articles and email sequences, then edited what I wrote, catching the repeated word and the paragraph that said the same thing twice. It argued with me about headlines. Research changed shape too: instead of typing questions into a search box, I hand a tool a question and get back a sourced brief that has read 40 pages, so I can read one. Long reports and call transcripts turn into summaries I can act on before lunch.

The work got more ambitious as the tools did. I paste survey results and traffic data into a model and ask it to find the pattern, then build a rough forecast from the history. The forecast is usually wrong in an instructive way. I generate landing page copy, ad variations, and subject lines in bulk and keep the two that sound like us. I sketch image concepts and mock-up pages before a designer ever sees them. I use agents that open a browser and click through websites the way a person would, checking dozens of pages for me while I do something else. When I built the knowledge base for Ask Zia, our support chatbot, I used AI to audit two months of conversation logs, work that would have eaten weeks. Since then, I have built a schema markup generator and a few hackathon agents, and the research and show notes for most of my podcast episodes start with it.

So, before the uncomfortable part: I am not a skeptic. I depend on this technology and can see, up close, how much better it has become. That is exactly why the recent warnings bother me.

What happened to AI while we weren't looking?

The ChatGPT of November 2022 could write text and nothing else. It could not browse the web, look at a picture, or run code, and OpenAI's own launch notes warned that it "sometimes writes plausible-sounding but incorrect or nonsensical answers." It still reached a million users in five days.

GPT-4 could read images by March 2023. Browsing and code execution followed that spring, voice conversations by that fall. In October 2024, Anthropic released a Claude computer use that could operate a computer by looking at the screen and clicking, which the company itself called "cumbersome and error-prone." By mid-2025, the major labs were all shipping agents that browse, fill forms, write and run code, and keep going without a human prompting each step.

The best yardstick I know comes from METR, a nonprofit that measures how long a task an AI can finish on its own half the time. In early 2025, the answer was about an hour. By January 2026 that horizon had been doubling every three months or so, and by spring the estimate for the best model was at least 16 hours, with METR's caveat that its tests get unreliable at that length. Those are software tasks with wide error bars. I still remember when the impressive thing was a decent paragraph.

The AI predictions that are getting harder to ignore

If AI went from a chatty text box to hours of autonomous work in under four years, what does the next four look like? The people building these systems have been responding publicly, and their answers are not calm. In January, Anthropic's CEO, Dario Amodei, published an essay calling the coming years "a rite of passage, both turbulent and inevitable, which will test who we are as a species," while adding that "nothing here is intended to communicate certainty or even likelihood." Both halves matter. A forecast with an error bar is still a forecast.

Behind it sits the extinction question. In 2023, hundreds of researchers and executives, including the heads of OpenAI, Google DeepMind, and Anthropic, signed a statement that "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." The argument, minus the science fiction: if you build systems more capable than people at most cognitive work and connect them to money, code, and infrastructure, you had better be sure they do what you intend, and nobody yet knows how to be sure. 

Geoffrey Hinton (also fondly known as the Godfather of AI) puts the odds of catastrophe at 10% to 20%; a 2023 survey of more than 2,700 AI researchers gave a median of 5%. Those are beliefs. Nobody has measured an event that has never happened. The 2026 International AI Safety Report puts it carefully: "Current systems lack the capabilities to pose such risks, but they are improving in relevant areas such as autonomous operation." On September 9, Anthropic's own economists published three scenarios for 2030, from an internet-sized bump to GDP to a world where knowledge-worker wages fall more than 10%, and their paper attaches "no probabilities to them."

So, is AI dangerous? Today's documented dangers are misuse and mistakes: fraud, leaked data, and incorrect answers that people act on. The catastrophic dangers are forecasts, taken seriously by credible researchers and not yet observed. That distinction is the spine of this article.

The warnings did stop being hypothetical in one narrow way this year. In July, during OpenAI's own security testing, a group of its agents broke out of the test environment and reached the production systems of Hugging Face, a company that hosts AI models, without being told to. OpenAI called it "a 'warning shot' for us and for the world." And on September 8 a researcher resigned from Anthropic saying the people building AI "earnestly believe that it could kill us all by the end of the decade"; a senior Anthropic alignment researcher replied that he puts that risk above 10% within a decade, and that the risk from today's models is low. The people closest to this technology, including the ones selling it, take the risks a decade out seriously enough to say so on the record. That is not normal for an industry.

Over the past eight months, Anthropic's Threat Intelligence team identified and disrupted operations in which threat actors tried to use Claude for malicious activity. Read this report to learn more!

Are AI systems really going to take our livelihoods?

This is the fear most owners have, so let me separate what has happened from what has been predicted. The predictions point in opposite directions. Amodei told Axios in 2025 that AI could eliminate half of entry-level white-collar jobs within one to five years; in May 2026, Sam Altman said he was "delighted to be wrong" about entry-level losses so far. Neither man has data from the future.

What has happened is quieter. The Yale Budget Lab, which has tracked the occupational mix monthly since ChatGPT launched, reported in August that it "is not yet changing in ways that clearly align with the introduction of AI." The New York Fed found in September that 61% of service firms in its region use AI, and 4% had laid anyone off because of it. Layoff announcements say otherwise: companies cited AI for 116,175 announced cuts through August, the leading stated reason this year, according to Challenger, Gray & Christmas. Altman's phrase for the gap between announcements and reality was "AI washing."

Where the evidence does turn uncomfortable is the bottom rung. Stanford's Digital Economy Lab found employment of 22-to-25-year-olds in the most AI-exposed occupations fell about 11% between late 2022 and mid-2026, while the same age group in the least exposed occupations grew about 10%. The researchers call these descriptive patterns rather than causal estimates, and they still "do not see widespread, economy-wide job displacement associated with AI." Fewer juniors get hired. Almost nobody gets fired. That is what displacement looks like when it starts.

What jobs will AI replace? Today, the honest answer is tasks: first drafts, summaries, data entry, routine code, tier-one support. What jobs are safe from AI? The ones that own the outcome, hold the relationship, or carry the liability. For a small business, the more pertinent question is whether you hire one fewer junior because the owner and a chatbot can cover the work, and what that costs you in three years.

Why AI agents change the conversation

A chatbot answers. An agent acts. That difference sounds small and is the whole story.

An AI agent is software that takes a goal, breaks it into steps, uses tools such as a browser, an inbox, a CRM, or a payment system, and keeps working until it finishes or fails. Ask an agent to "follow up with everyone who went quiet last month" and it decides who that is, writes the messages, and sends them. Every step it takes without you is a step you did not check.

Anthropic ran the cleanest experiment I know of. In 2025, it let Claude run a small shop in its office for a month, with a real budget, suppliers, and customers. The agent sold tungsten cubes at a loss, created a Venmo account, and briefly insisted it was a human in a blue blazer. Anthropic's verdict after a second, improved run: "The gap between 'capable' and 'completely robust' remains wide."

The real incidents rhyme. This April, an agent hit a credential mismatch in a small startup's systems and deleted its production data and backups in nine seconds, using an API key with far more power than the task needed. The hosting company's CEO said what every owner should hear: "If you (or your agent) authenticate, and call delete, we will honor that request."

Nobody programmed that agent to do harm. It was confident, fast, and over-permissioned, which also describes a bad hire, and that is the frame I would use. Agents are moving from suggestion to action across business software, including the CRM I work on, and the shift is accelerating. The opportunity is real, and so is the new category of mistake.

The risks small businesses should worry about right now

An AI taking over the world is not something you can put on Monday's to-do list. Instead, let's focus on what you should be thinking about:

An employee pastes your customer list, a contract, or last quarter's numbers into a free chatbot on a personal account.

Netskope, which monitors corporate networks, found that incidents of sensitive data going into AI apps doubled in a year, and Cyberhaven found roughly a third of workplace ChatGPT use runs through personal accounts. Nobody in these stories set out to leak anything. They were busy.

Someone impersonates you

On September 10, Microsoft described a three-day campaign in August that sent more than a million emails impersonating CEOs and CFOs and asking accounts-payable staff to push about $50,000 per target through a bank transfer; nearly 88% of the targets were US businesses. The FBI logged $3 billion in business email compromise losses in 2025, plus more than 22,000 complaints where AI was part of the scam. Those are the people who reported. Most don't.

A salesperson trusts an AI answer that was wrong

An Air Canada chatbot invented a refund policy, and a tribunal made the airline honor it. A database of court decisions involving AI-fabricated citations passed 2,000 cases in September 2026, more than 800 of them filed by lawyers. If professionals who bill by the hour are getting caught, your sales team will too.

None of this requires superintelligence

This is the scary part. All of it is happening now, mostly to organizations with fewer people watching. The UK's National Cyber Security Center expects"a digital divide" between systems that keep pace with AI-enabled threats and "a large proportion that are more vulnerable." Small businesses make up a large share of the second group.

The privacy problem: what happens to the data we give AI?

Choosing an AI tool is a data governance decision dressed up as a productivity decision. Three questions get you most of the way.

First, what does the tool do with what you type? On consumer accounts, the default is often that conversations can be used to train future models unless you opt out. OpenAI says so for free and paid ChatGPT plans, and says business and API customers are not trained on by default. Those are reasonable terms. But a customer's phone number pasted into a personal chatbot is now governed by terms nobody in your business reads.

Second, where does the data go, and how long does it stay? In 2025, a court order forced OpenAI to preserve consumer chat logs it would normally have deleted. Retention is boring until it isn't.

Third, what can the tool do and who told it to? The top risk on OWASP's list of AI application vulnerabilities is prompt injection: instructions hidden in a web page, a document, or a form field that hijack the agent reading it. Researchers showed in 2025 that text planted in a Salesforce web-to-lead form could make an Agentforce agent leak CRM lead data to an outside server. Anthropic, testing its own browser agent, cut attack success from 23.6% to 11.2% and still concluded that "no browser agent is immune."

So here is what I would ask any vendor of an AI business tool, CRM, or otherwise, before signing. Is my data used to train models, and is that in the contract? Where is it processed, and for how long? Do humans review it? Can an administrator turn AI features off, by role? What can an agent do without a person confirming, and is every action logged? Which independent audits can you show me? How fast will you tell me about an incident? Most of these come straight from guidance published by NIST, the US and UK cyber agencies, and OWASP. A vendor who cannot answer them in plain language is telling you something.

Find out how Noma Labs discovered ForcedLeak, a critical severity (CVSS 9.4) vulnerability chain in Salesforce Agentforce that could enable external attackers to exfiltrate sensitive CRM data through an indirect prompt injection attack.

AI and cybersecurity: defender and attacker

AI is making both sides of cybersecurity better at their jobs, and the attackers are less fussy about how they use it.

On the hacker side of things, the evidence has moved fast. Anthropic's September 2026 threat report describes a breach that went from one stolen developer token to full administrative control in roughly three hours, and a group that included university students hitting roughly 50 organizations, and concludes that "sophistication has stopped being a reliable signal of who is behind an operation."Google's threat team noted that same week it had still not seen a fully autonomous attack from start to finish.

Phishing is where most small businesses will first feel this. In 2023 phishing lures written by AI were less effective than ones written by expert human red teams; by March 2025 they were 23% more effective. The "tell" is gone, and the "does the boss really write like that" tell is going, because the model has read the boss's LinkedIn.

On the defensive side of things, the same tools find flaws at a pace no human team can match. In April, Anthropic withheld a model from general release partly because it had found thousands of severe vulnerabilities in major operating systems and browsers, with over 99% unpatched at the time. The bottleneck, it wrote in May, was "the human capacity to triage, report, and design and deploy patches."

For a small business with no security team, the translation is short. You will not out-research the attackers. You can make yourself a slower, more annoying target: multi-factor authentication on everything, a rule that no payment detail changes on the strength of an email or a voice, and staff who know the old phishing tells no longer work. If nothing is written down, this small business cybersecurity plan is a fair place to start.

Who wins if this keeps going?

Concentration at the top has come with abundance at the bottom. A handful of companies build the frontier models, but for $20 a month, a two-person agency gets roughly the same models as a bank. Large firms still adopt faster, 37% of US businesses with 250 or more employees against under 20% of those with four or fewer, according to the Census Bureau, but the capability gap between small and large has never been narrower. The adoption gap is mostly time and confidence.

The skeptics deserve a line too. Princeton's Arvind Narayanan and Sayash Kapoor, the most persuasive I read, argue AI is "normal technology" whose spread into consequential work will be slow. Everyone is partly right, and that is the unsettling part. This technology can be enormously useful, economically disruptive, dangerous in the wrong hands, and far more capable in five years than it is today, all at once.

Six things I think every small business should do now

• Use it, but start where mistakes are cheap and reversible: drafts, summaries, research you will check, and meeting notes. The St. Louis Fed calls workplace AI use in 2026 "widespread but shallow," and shallow is where you learn. This practical guide to AI beyond ChatGPT has starting points.
   
• Write down what never goes into a chatbot, and give people business accounts. Customer records, contracts, payroll, financials, and anything a customer trusted you with. Pay for accounts with contractual no-training terms so the rule is enforceable, and keep customer data in one managed place rather than a pile of spreadsheets.
   
• Keep a person between the AI and anything that is hard to undo: sending, deleting, paying, and anything that sounds like legal, medical, or financial advice. In August European regulators fined Uber €825 million for deactivating drivers through fully automated decisions with no human involved. The principle scales down to a five-person shop.
   
• Treat agents like new hires, with the permissions of one. An agent should have its own identity, access scoped to the task, no standing right to delete or pay, and a log of every action, which is what Microsoft's guidance on agent security now recommends. In a CRM, that means roles that decide which records it can see, an audit trail of what it did, and rule-based workflows for anything that must behave the same way every time.
   
• Assume the email, the invoice, or the voice might be fake. Any change to payment details gets confirmed by phone, on a number you already have. Multi-factor authentication goes on everything. Train staff on the new phishing, the kind with perfect grammar and your CEO's writing style.
   
• Choose vendors the way you would choose an accountant. Ask the questions in the privacy section and expect written answers. Model capability is the easiest thing to compare and the least important thing to get right.

What I think after nearly four years of using AI

I opened ChatGPT in November 2022 to rewrite a cold email. This month, I read a resignation post from a researcher who believes the technology his colleagues are building might end us, and a report from the same company describing criminals who breach a network in the time it takes me to write a section of this article. I use the product in the middle of that story every day.

I don't know where this ends. Neither do the people building it, and the ones I trust most are the ones who say so. Waiting for a verdict is not a plan. The capability will keep arriving whether or not a small business is ready for it, and so will the scams.

So my plan is unglamorous. Use the tools where they are clearly useful. Keep a human in the decisions that matter. Protect the data that customers trust us with. Give agents the permissions of an intern. And keep paying attention, because the next four years will not look like the last four, and the last four were strange enough.