Onyx Digital Intelligence.

AI Companies Build Models. Cybersecurity Companies Defend Systems.


AI companies build models.
Cybersecurity companies defend systems.

Those are different jobs. Treating them as interchangeable is a category error that organisations keep making, usually under pressure to “do something about AI risk” without examining who is actually equipped to do it.

The problem is not the model. It is the system around the model.

Most of the serious security failures involving AI systems do not originate in the core model weights. They originate in how the model is integrated, how it is given tools and privileges, how its outputs are trusted, how its training and fine-tuning data are sourced, how its agents are allowed to act, and how the organisation monitors and contains the resulting behaviour.

Prompt injection, excessive agency, tool over-permissioning, insecure retrieval pipelines, poisoned training or fine-tuning data, model extraction, and supply-chain compromise of the model-serving stack are security problems. They are solved by threat modelling, least privilege, input/output validation, monitoring for anomalous behaviour, secure development practices, and incident response capability.

An AI company can improve the model’s robustness on certain benchmarks. It cannot replace the defensive engineering required to run that model safely inside a real organisation that has real assets and real adversaries.

Three functions that are not interchangeable

The work around advanced AI systems tends to collapse into a single conversation. It should be separated into at least three distinct functions.

Capability engineering is the work of building and improving the model, its training, its infrastructure, and its product features. This is the primary business of AI companies.

Independent evaluation is the work of measuring what the model and its agents can actually do, identifying failure modes, and assessing whether those capabilities cross thresholds that create serious risk. This is the work of organisations such as METR (Model Evaluation & Threat Research). METR develops methods to test autonomous capabilities, investigates real incidents involving AI agents, and publishes findings that are not controlled by the companies whose models are under examination.

Operational defence is the work of securing the deployed environment: managing permissions, detecting attacks, investigating incidents, reducing residual risk, and maintaining the controls that keep a live system from being abused. This is the work of cybersecurity practices.

Two concrete examples illustrate different parts of that operational layer.

Cybersec Clinique, based in South Africa, operates as an evidence-driven cybersecurity practice covering ISO-certified processes, digital forensics, mobile application security, software engineering with a security posture, IT governance, and proactive threat work. Their commercial identity is residual risk and operational defence.

Embroidery, based in Oslo, focuses specifically on the new attack surface created by AI agents. It monitors agent behaviour for misalignment, misuse, and dangerous actions, and alerts security teams when those agents begin to take actions that traditional detection was not built to see. The founders come from cybersecurity backgrounds and are building detection that understands context rather than simply matching patterns.

These organisations sit on different scales and address different slices of the problem, but they share the same structural orientation: they exist to examine and contain risk in live systems rather than to sell model capability.

Incentives and independence

An AI company’s commercial incentive structure is shaped by capability, performance, and product velocity. Security work inside those companies can be genuine and competent. It is still performed inside an organisation whose primary product is the thing being secured. Internal security competence and independent security assurance are different things. Hiring the builder to audit the building remains a weaker arrangement than bringing in parties whose institutional identity is not tied to the product’s success.

Cybersecurity companies are commercial organisations too. They have sales targets, delivery pressures, and reputational concerns. Their label does not automatically make them independent, competent, or objective. The same evidence-based scrutiny applied to AI vendors should be applied to them.

The practical question is therefore not “AI company or cybersecurity company?” It is which combination of capability engineering, independent evaluation, and operational defence is required for the specific risk under consideration, and whether those functions are being performed by parties whose incentives align with residual-risk reduction rather than product narrative.

Accountability remains with the deployer

Even when external specialists are engaged, the organisation that deploys the AI system retains responsibility for its architecture, access controls, data, governance, and operational decisions. Outsourcing security work does not outsource accountability. The deployer decides what privileges the system receives, what data it can reach, and what actions it is allowed to take. Those decisions determine the attack surface more than the model weights do.

The Onyx position

I do not argue that AI companies are incompetent or malicious. I argue that they are solving a different problem. Capability engineering, independent evaluation, and operational defence are distinct disciplines. Organisations that need the second and third should engage parties whose professional identity is built around those functions.

If the question is “How do we make the model more capable?”, that is capability engineering.

If the question is “What can this system actually do, and where do the risks concentrate?”, that is independent evaluation, the category occupied by groups such as METR.

If the question is “How do we stop the deployed system from being used against us, or from failing in ways that create material harm?”, that is operational defence, the category occupied by cybersecurity practices such as Cybersec Clinique and by specialised agent-monitoring approaches such as Embroidery.

Adjacent to secure is not secure.
Capability is not defense.

Sources and scope. This piece is an analytical distinction between professional functions and their incentive structures. METR is cited on the basis of its public mission and published evaluation work as an independent nonprofit focused on measuring autonomous AI capabilities and associated risks. Cybersec Clinique is cited as a concrete example of an evidence-driven cybersecurity practice operating in South Africa, based on their public positioning and service lines. Embroidery is cited as an example of a cybersecurity company focused specifically on monitoring AI agent behaviour for misalignment, misuse and dangerous actions, based on their public positioning. No claim is made that any single organisation is uniquely capable, nor that AI companies employ no competent security engineers, nor that cybersecurity firms are free of commercial distortion. The argument concerns category fit and default organisational reflex. Interpretation of relative risk and hiring priority is mine.

Clayton Bax

Published under ONYX Digital Intelligence
Following the #OnyxAudit methodology.

"Adjacent to true is not true."

Truth has no flag nor favour, only a standard. And it's heavy!

Editors note: I use various AI agents to fetch my research. I check, strip and write the article.
Facts are partly theirs. Interpretation is mine.

#AI #Onyx #South Africa #cybersecurity #defense #hiring #risk