Can we trust an unsecured AI in an operational environment?

12 Jun 2026 By Astrid Cailleux
ML security

A lot of AI systems still look solid… until you take them out of a controlled environment.

In defense, industrial operations, embedded systems, or field deployments, the question is not just about model performance. It’s about behavior under real-world constraints, and possibly when confronted with a malicious user.

AI Is Quietly Moving to the Edge

The shift toward edge and on-device AI is no longer theoretical.

Industry forecasts consistently suggest that over 50% of AI processing will happen at the edge or on-device by 2027 (Gartner and broader industry analyses on edge computing).

The drivers are straightforward:

  • Latency requirements: Often below 50 ms in critical use cases.
  • Uptime targets: Above 99.9% in industrial systems.
  • Connectivity: Unreliable or absent in defense and field environments.
  • Data confidentiality and privacy.

Case 1: Drones and Embedded Vision

A standard pipeline today: Camera → Vision Model → Local Decision-Making.

In practice, this setup is far from trivial once deployed. From research and field observations:

  • Adversarial perturbations can reduce vision model accuracy by up to 80–90% on non-robust architectures.
  • Delays in the range of 100–200 ms can already destabilize real-time control loops (navigation, avoidance systems).
  • Sensor noise or spoofing (GPS, IMU, camera feeds) can lead to persistent misclassification or navigation drift.

Illustration Image

Case 2: Industrial and Critical Infrastructure

In industrial environments (energy, transportation, manufacturing), the weakest link is still data quality.

Studies in IIoT consistently show that up to ~30% of automation incidents are linked to degraded or misinterpreted sensor data. And AI is not improving this rate.

Once you embed an AI layer on top of unstable inputs:

  • Anomalies become harder to interpret.
  • Decision loops become less predictable.

Edge AI may look simpler to deploy, but isn’t necessarily more reliable.

Illustration Image

What Is Actually Being Attacked?

A common misunderstanding is that “AI security” mainly refers to LLM attacks or prompt injection.

In embedded AI, the threat model is broader and more physical.

Key Threat Vectors

  • Sensor spoofing: Manipulating the physical input layer (GPS, LIDAR, camera streams).
  • Model extraction: Reverse-engineering of AI models allowing an attacker to have full access to architecture and weights.
  • Runtime compromise: Tampering with inference pipelines or execution environments on edge devices.
  • Supply chain exposure: Backdoored dependencies, firmware, or hardware components.

Cloud vs. Edge: Fundamentally Different Risk Profiles

Cloud environments offer:

  • Centralized monitoring
  • Fast rollback
  • Continuous patching
  • Controlled infrastructure

Edge and embedded environments do not:

  • Physical access is possible
  • Environments are harder to control
  • Compute and memory are constrained
  • Updates are slow or rare
  • Hardware dependencies are strong

This leads to our next question: “What happens when an attacker can directly access and manipulate the model?”

The Current Blind Spot

A lot of threats are not properly tested yet.

A properly deployed embedded AI system must remain:

  • Stable under adversarial inputs
  • Confidential
  • Protected against tampering

These new properties require dedicated security tooling. As AI moves into defense, mobility, industry, and healthcare, failures can produce real-world consequences.

It’s time to take these challenges seriously.

Sources

Edge AI / Adoption Trends

Industrial IoT / Operational Risk

AI Security / Embedded Systems

Adversarial Machine Learning (Academic References)

Share on: