01 — Start With the Product Problem
Don't Start With the Neural Network
Before selecting:
or:
LLM
define:
What decision must the product make?
Examples:
Is the machine behaving abnormally?
Is a person present?
Which object is visible?
What command was spoken?
Is the sensor signal healthy?
What is likely to happen next?
AI Begins With the Product Decision — Not the Model Architecture.
02 — Do We Even Need AI?
This is a surprisingly important question.
Some problems are better solved using:
control algorithms.
AI Should Be Used When It Improves the Product — Not When It Improves the Marketing Slide.
03 — Classical Algorithm vs Machine Learning
If the relationship can be explicitly described:
a classical algorithm may be simpler.
If the relationship is difficult to describe but can be learned from examples: Machine Learning May Create Value.
04 — Hybrid Architecture
Many of the strongest products combine:
Machine Learning.
Example: Sensor Calibration → DSP Filtering → Feature Extraction → ML Classification
AI Does Not Need to Replace Good Engineering.
05 — Define the AI Output
Possible outputs include:
Classification
Regression
Detection
Segmentation
Anomaly Score
Forecast
Embedding
Natural-Language Output
Model Output Should Map to Product Meaning.
06 — Classification
Question: Which category does this input belong to?
Examples:
Normal / Fault
Object A / B / C
Command 1 / 2 / 3.
07 — Regression
Predict a continuous value.
Examples:
remaining life
quality score
physical parameter estimate.
AI Can Estimate — Not Only Classify.
08 — Object Detection
Identify: What and Where.
Detection Adds Spatial Meaning.
09 — Segmentation
Classify individual pixels or regions.
Useful where precise shape/location matters.
More Detailed Output Usually Costs More Compute.
10 — Anomaly Detection
In industrial systems, a particularly powerful problem is: Does This Look Different From Normal?
This can sometimes be more practical than collecting every possible failure type.
11 — Forecasting
AI may estimate future:
load
trend
failure probability
from historical sequences.
Time-Series AI Adds Time to the Feature Space.
12 — Sensor Fusion AI
Inputs can combine:
Vibration
Current
Temperature
Speed
Several Weak Signals Can Become One Strong Prediction.
13 — Multimodal AI
A product can combine:
Image
Audio
Sensor Data
Text
into a richer model.
Intelligence Can Span Several Sensor Modalities.
14 — Generative AI at the Edge
This is now becoming a real embedded-system category.
On-device models can support:
on sufficiently capable hardware.
MLPerf Mobile v6.0 introduced standardized on-device LLM benchmarks using Llama-family models from 1B through 8B parameters, showing how quickly local GenAI is moving into measurable product engineering.
Edge Generative AI Is Moving From Demo to Benchmarkable System Engineering.
15 — But Not Every Product Needs an LLM
A sensor node detecting bearing failure probably does not need: 8 Billion Parameters.
Use the Smallest Intelligence Architecture That Solves the Product Problem.
This improves:
cost
power
reliability
latency.
16 — AI System Requirements
Before model development, define:
Accuracy
How correct?
Recall
How many important events can be missed?
Precision
How many false alarms are acceptable?
Latency
How fast?
Memory
How much RAM / Flash?
Power
How much energy?
Thermal
How much sustained heat?
Cost
What compute platform can the product afford?
AI Requirements Are Multi-Dimensional.
17 — Accuracy Alone Is Not Enough
Model A: 99.0% accuracy
but: 800 ms latency.
Model B: 98.5%
but: 20 ms latency.
For a real-time product: Model B May Be Better.
18 — False Positive
A false positive says: problem exists, when it does not.
In industrial monitoring this can create: unnecessary service, alarm fatigue.
19 — False Negative
A false negative misses: A Real Event.
Depending on application, its cost may be much higher.
AI Thresholds Should Follow Product Consequence.
20 — Precision / Recall Trade-Off
Moving decision threshold can trade: Sensitivity
against: False Alarm Rate.
The "Best Threshold" Is a Product Decision — Not Only a Data-Science Decision.
21 — Confusion Matrix
A single accuracy percentage hides:
which classes are confused.
Know How the Model Is Wrong.
22 — Class Imbalance
Suppose: 99.9% of machine operation is normal.
A model predicting: normal
all the time can achieve: 99.9% Accuracy
and be completely useless.
Dataset Statistics Can Make Accuracy Misleading.
23 — Dataset Is Product Engineering
A model learns from: Data.
If the data does not represent:
real sensors
real environment
production variation
then the model may learn the wrong problem.
Model Quality Begins With Dataset Quality.
24 — Dataset Definition
Define:
What data?
From which hardware?
Under which operating states?
At what temperature?
From how many units?
Data Source Should Be Traceable.
25 — Train / Validation / Test
Keep independent subsets for: Training, Hyperparameter Development, Final Evaluation.
Don't Measure a Model Using the Same Data It Memorized.
26 — Data Leakage
Leakage can create unrealistically high validation results.
Example: samples from the same physical recording appear in both: training test.
Test Independence Matters.
27 — Device-Level Split
For physical products, splitting by:
Because otherwise the model may learn: characteristics of one sensor unit.
28 — Manufacturing Variation in Dataset
Collect data from:
not only one golden prototype.
Production AI Must Survive Production Hardware Variation.
29 — Environmental Variation
Dataset may need:
Temperature
Lighting
Noise
Humidity
Mounting
Background
variation according to application.
Laboratory Data Is Not Automatically Field Data.
30 — Corner Cases
The rare events often matter most.
Dataset Collection Should Hunt for Difficult Cases — Not Only Common Cases.
31 — Ground Truth
Every training sample needs the correct answer.
That answer is: Ground Truth.
32 — Bad Labels
If 5% of labels are wrong: The Model Is Being Trained to Learn Errors.
33 — Annotation Policy
Different human annotators may disagree.
Define: class boundaries, ambiguity rules.
Ground Truth Needs Its Own Specification.
34 — Annotation Confidence
Some samples are genuinely uncertain.
Uncertainty Should Not Be Hidden as False Certainty.
35 — Sensor Calibration Before AI
Suppose two sensors have different gain/offset.
A model may accidentally learn: Sensor Identity
instead of: Physical Phenomenon.
Calibrate the Measurement Chain Before Asking AI to Learn Physics.
36 — Preprocessing
Model input may require:
normalization
resizing
filtering
feature extraction.
Preprocessing Is Part of the Model.
37 — Training / Deployment Preprocessing Must Match
A common deployment failure:
training uses: RGB normalized 0–1
while device sends: BGR 0–255.
Perfect Model + Wrong Preprocessing = Broken Product.
38 — Preprocessing Version
Preprocessing logic must be versioned with the model.
Model File Alone Does Not Define the AI System.
39 — Feature Engineering
For some sensor problems, engineered features remain highly effective:
RMS
FFT Features
Spectral Peaks
Statistical Features.
Deep Learning Is Not Mandatory for Every Embedded AI Problem.
40 — Raw-Signal Learning
Other systems can learn directly from:
audio.
Remove Manual Features Only When the Data and Compute Budget Support It.
41 — Windowing
Time-series models often operate on windows:
depending on phenomenon.
Window Length Is Latency Architecture.
42 — Overlap
Overlapping windows can increase detection responsiveness but also: compute load.
Sampling Architecture and AI Architecture Interact.
43 — Sample Rate
Higher sample rate captures more information.
But costs:
power.
More Data Is Not Free Intelligence.
44 — Aliasing
AI cannot recover information already destroyed by poor sampling.
Nyquist Still Exists in the AI Era.
45 — Analog Front End Matters
Garbage analog signal → Garbage ADC data → Garbage AI.
A Neural Network Cannot Repair Every Measurement-System Failure.
This directly connects Page 12.
46 — Sensor Noise
Noise may be:
random
correlated
environmental.
Model Training Should Understand Real Sensor Noise.
47 — Data Augmentation
Artificial variations can improve robustness.
Examples:
depending on modality.
Augmentation Should Represent Plausible Product Variation.
48 — Unrealistic Augmentation
If synthetic data creates physically impossible examples: The Model Can Learn Unrealistic Physics.
49 — Synthetic Data
Simulation can help expand difficult datasets.
But: Synthetic → Real Domain Gap
must be understood.
50 — Domain Shift
Model trained on: Factory A
may perform worse in: Factory B.
Real Deployment Distribution Can Change.
51 — Domain Adaptation Direction
Some systems can use adaptation or retraining to handle new environments.
But production governance becomes more complex.
Learning After Deployment Requires Lifecycle Engineering.
52 — Model Architecture Selection
Potential models include:
CNN
RNN / LSTM
TCN
Transformer
Autoencoder
and many others.
Architecture Should Follow Data Structure and Deployment Constraints.
53 — CNN
Convolutional networks remain efficient for many:
problems.
Local Spatial Structure Can Be Exploited Efficiently.
54 — Depthwise Separable Convolution
Architectures such as MobileNet use depthwise-style operations to reduce compute.
Model Architecture Can Be Designed for Edge Efficiency.
55 — Temporal Convolution
Time-series AI can use convolution across time.
Sequence Processing Does Not Always Require Recurrent Networks.
56 — RNN / LSTM
Useful for selected sequential problems.
But recurrent execution can limit: parallelism.
Architecture Affects Accelerator Efficiency.
57 — Transformer
Transformers increasingly appear in:
multimodal systems.
But resource requirements can be larger.
Model Popularity Does Not Override Product Constraints.
58 — Attention
Attention can be powerful but memory traffic may become significant.
Compute Is Not the Only Cost of AI.
59 — KV Cache
Generative transformer inference can require substantial memory for: Key / Value Cache.
At edge scale: Memory Capacity and Bandwidth Can Become the Limiting Resource.
60 — On-Device LLM
For local language models, engineering variables include:
Parameter Count
Quantization
Context Length
KV Cache
Tokens/s
Time to First Token
Memory
Thermal
"Runs an LLM" Is Not a Performance Specification.
61 — Time to First Token
Interactive user experience depends strongly on: Initial Response Latency.
62 — Tokens per Second
Once generation begins:
affects user experience.
63 — Context Length
Larger context can increase:
compute.
Bigger Context Has an Edge Cost.
64 — Model Size
A 1-billion-parameter model at: FP16 requires fundamentally different storage than highly quantized representation.
Numerical Precision Becomes Memory Architecture.
65 — TinyML
At the other extreme: TinyML places AI on highly resource-constrained MCU-class devices.
MLCommons describes TinyML as ultra-low-power inference on systems ranging from small microcontrollers to vector processors and dedicated neural accelerators; MLPerf Tiny v1.4 explicitly measures both performance and optional energy efficiency.
66 — MCU AI
Tiny embedded AI may run: Sensor → Cortex-M → Model → Decision → without Linux or cloud.
Intelligence Can Live Inside a Few Milliwatts.
67 — Always-On AI
Examples include:
wake word
vibration monitoring
occupancy detection.
These workloads often spend most of their time: Waiting.
Therefore idle energy matters enormously.
MLPerf Tiny's streaming wake-word workload explicitly includes the always-listening operating model rather than measuring inference in isolation.
68 — Duty Cycle
An always-on system may: Sense Continuously
but: Run Heavy Inference Occasionally.
Duty-Cycled Intelligence Can Save Energy.
69 — Cascaded AI
Architecture:
Tiny Low-Power Detector → Wake Larger Processor → Run Complex Model
Use Expensive Compute Only When Needed.
70 — Hierarchical Compute
A product might contain:
NPU.
Different Intelligence Workloads Can Belong to Different Compute Domains.
71 — CPU Inference
CPUs provide:
broad operator support.
CPU Is Often the Universal Fallback.
But not always the most efficient engine.
72 — DSP
DSPs can be effective for:
vector workloads.
Traditional Signal Processing and AI Often Share the Same Compute Architecture.
73 — GPU
GPUs provide high parallel throughput for suitable workloads.
Parallel Arithmetic Can Accelerate Neural Networks.
But they may carry:
thermal costs.
74 — NPU
Neural Processing Units are specifically optimized for machine-learning operations.
Dedicated AI Hardware Can Deliver More Inference per Watt.
But only if the model maps well to the accelerator.
75 — TOPS
TOPS usually describes: Trillions of Operations per Second.
But: TOPS Is a Theoretical Compute Metric — Not Product Throughput.
76 — Why TOPS Can Mislead
Two processors with equal: 20 TOPS
can deliver different real performance because of:
operator support
memory bandwidth
compiler
utilization
quantization
model architecture.
Effective Utilization Matters.
77 — Benchmark the Actual Model
Don't ask only: How many TOPS?
Ask: What is the measured latency for our model?
Model-Specific Benchmark > Marketing Number.
78 — End-to-End Latency
Vision AI might be: Exposure → Sensor Readout → ISP → Preprocess → Inference → Postprocess → Application Decision
Inference Time Is Only One Component.
79 — Camera Frame Rate
A model running: 5 ms
does not guarantee: 200 FPS
if camera and pipeline deliver only: 30 FPS.
80 — Pipeline Throughput
System performance is limited by the slowest stage.
Optimize the Pipeline — Not Only the Neural Network.
81 — Zero-Copy
One of the most important Edge AI concepts.
Traditional: Camera → Copy to CPU → Copy to GPU → Copy to NPU
creates:
power.
Avoid Unnecessary Copies.
82 — Zero-Copy AI Pipeline
Potential architecture: Sensor → DMA Buffer → ISP → NPU → Display / Application
with shared buffer infrastructure where platform supports it.
Data Movement Is Part of AI Performance.
83 — LiteRT v2 Direction
Google's current LiteRT v2 CompiledModel architecture explicitly includes:
asynchronous execution.
Edge AI Runtime Architecture Is Becoming Hardware-Aware.
84 — Memory Bandwidth
AI accelerators may perform arithmetic faster than tensors can reach them.
Memory Bandwidth Can Limit AI More Than Compute.
85 — Arithmetic Intensity
Models that reuse data effectively can utilize compute more efficiently.
Model Architecture Interacts With the Memory Hierarchy.
86 — Cache
CPU/GPU cache behavior can strongly affect performance.
Tensor Layout Is a Hardware Performance Variable.
87 — SRAM
On MCU/NPU-class devices: on-chip SRAM can be dramatically faster/lower-energy than external memory.
Keep Working Data Close to Compute.
88 — External DDR
Large models may require DDR.
This introduces:
latency
power
PCB complexity
SI/PI requirements.
AI Model Size Can Become PCB Architecture.
89 — AI Is a Hardware Design Input
If the model requires: 4 GB LPDDR
that requirement affects:
thermal.
AI Requirements Should Exist Before Board Layout Freezes.
90 — Accelerator Interface
NPU may be: inside SoC
or: external accelerator.
External devices introduce:
requirements.
AI Compute Architecture Connects Directly to Pages 09, 21, 22, 23 and 29.
91 — Model Training vs Inference
Training optimizes model weights.
Inference executes the trained model.
Most Embedded AI products perform: Inference on Device.
Training may occur elsewhere.
92 — On-Device Training
Some applications may adapt locally.
But training costs:
much more than ordinary inference.
Local Learning Should Have a Specific Product Reason.
93 — Personalization
A product may adjust selected parameters locally without fully retraining the whole model.
Personalization Can Be Smaller Than Training.
94 — Federated Learning Direction
Distributed devices may contribute to model improvement without centralizing all raw data.
But architecture becomes significantly more complex.
Privacy-Preserving Learning Still Requires System Engineering.
95 — Model Export
Training framework representation must become deployable artifact.
Possible pathways include: PyTorch → ExecuTorch
or: Framework → ONNX → Target Runtime / Compiler
or: TensorFlow → LiteRT
depending on platform.
Training Model and Deployment Model Are Not Automatically the Same Artifact.
96 — ONNX
ONNX provides an open representation for exchanging neural-network graphs across tools/runtimes.
ONNX release management currently lists 1.22.0 as released June 15, 2026, with 1.23 planned for September 1, 2026.
Portable Model Representation Can Reduce Framework Lock-In.
97 — ONNX Does Not Guarantee Deployment
A model converting to ONNX does not guarantee:
target runtime support
accelerator support
equal numerical output.
Model Format Compatibility ≠ Hardware Compatibility.
98 — Operator Support
An accelerator may support:
Then unsupported portions may: Fall Back to CPU.
99 — Graph Partitioning
Runtime may divide model: NPU Segment → CPU Segment → NPU Segment.
Hardware Boundary Can Create Data-Copy Overhead.
100 — One Unsupported Operator Can Matter
A model that is:
can still perform poorly if the remaining 1% creates expensive transfers.
Operator Placement Matters More Than Percentage Supported.
101 — ExecuTorch
ExecuTorch 1.0 supports deployment across CPUs, GPUs, DSPs and NPUs from embedded devices to larger edge platforms. In April 2026 it formally became part of PyTorch Core governance.
Training-to-Edge Toolchains Are Becoming More Integrated.
102 — CMSIS-NN
For Cortex-M products, CMSIS-NN provides optimized neural-network kernels designed to improve performance while minimizing memory footprint.
MCU AI Needs Kernel-Level Optimization.
103 — Runtime Selection
Possible factors:
Model Framework
Target Processor
Accelerator
Operator Support
Memory
Licensing
Lifecycle
Runtime Selection Is Product Architecture.
104 — Vendor AI Compiler
Many NPU vendors provide dedicated compilers.
These transform: Model Graph
into: Hardware-Specific Execution.
105 — Compiler Is Part of AI Performance
Two compiler versions can produce different:
accuracy.
AI Toolchain Is Product Configuration.
106 — Model Compilation
Compiler may:
fuse operators
reorder
quantize
schedule memory.
Deployment Can Transform the Original Graph Significantly.
107 — Operator Fusion
Example:
may execute as one fused kernel.
Less Intermediate Memory Can Improve Performance.
108 — Constant Folding
Static calculations can be performed at compile time.
Don't Spend Device Compute on Values That Never Change.
109 — Graph Optimization
Remove:
redundant conversions.
Model Optimization Is Software Compilation.
110 — Quantization
One of the most important Embedded AI technologies.
Convert numerical representation from: FP32
toward:
where accuracy permits.
Lower Precision Can Reduce Memory and Compute Cost.
111 — INT8
INT8 is extremely important in embedded inference because many accelerators provide strong integer support.
8-Bit Arithmetic Can Make AI Practical on Much Smaller Hardware.
112 — Quantization Is Not Free
Reduced precision can create:
numerical error.
Measure Accuracy After Deployment Quantization.
113 — Post-Training Quantization — PTQ
Train high precision.
Then quantize.
Fast Deployment Path.
But some models lose too much accuracy.
114 — Calibration Dataset
PTQ usually requires representative samples to estimate numerical ranges.
Quantization Calibration Data Should Represent Real Deployment Inputs.
115 — Bad Calibration
If calibration only contains bright daytime images: nighttime activations may clip.
Quantization Quality Depends on Calibration Distribution.
116 — Quantization-Aware Training — QAT
Simulate quantization effects during training.
Teach the Model to Survive the Numerical Format It Will Actually Use.
117 — Per-Tensor vs Per-Channel
Quantization scale can be applied:
globally to tensor
separately to channels
depending on runtime/accelerator support.
Numerical Granularity Can Affect Accuracy and Performance.
118 — Symmetric / Asymmetric Quantization
Different quantization mappings make different trade-offs.
Model Format Must Match Hardware Kernel Expectations.
119 — INT4
Lower-bit quantization is increasingly important for:
memory-constrained inference.
Every Bit Removed Saves Storage — but Increases Numerical Risk.
120 — Mixed Precision
Different layers can use different precisions.
Spend Precision Where the Model Needs It.
121 — FP16
Half precision can reduce:
on platforms with strong hardware support.
122 — BF16
Some accelerators support BF16-style arithmetic.
Numerical Format Is Hardware Ecosystem-Dependent.
123 — Pruning
Remove less-important weights/connections.
Potential benefit: smaller model.
But: Sparse Model Only Runs Faster if the Hardware/Runtime Exploits Sparsity.
124 — Structured Pruning
Remove:
blocks.
Structured Sparsity Is Often Easier for Hardware to Accelerate.
125 — Unstructured Sparsity
Many individual zeros.
This may compress well but not automatically accelerate inference.
Storage Efficiency ≠ Execution Efficiency.
126 — Knowledge Distillation
Train a smaller: Student Model
from a larger: Teacher Model.
Transfer Intelligence Into a Smaller Deployment Footprint.
127 — Architecture Search
Model architectures can be optimized around:
accelerator constraints.
Edge Model Design Can Be Hardware-Aware From the Beginning.
128 — Model Compression Is Multi-Objective
Possible objectives:
while preserving: Accuracy.
Optimization Has No Single Best Model.
129 — Memory Planning
Embedded AI must know:
Weights
Activations
Scratch
Input / Output
Peak RAM Can Be Much Larger Than Model File Size.
130 — Activation Memory
A small model may create huge intermediate feature maps.
Parameter Count Does Not Equal Runtime Memory.
131 — Lifetime Analysis
Intermediate tensors can share memory if their lifetimes do not overlap.
Memory Scheduling Is AI Compiler Engineering.
132 — Arena Allocation
Tiny inference runtimes often pre-plan a shared memory arena.
Deterministic Memory Can Eliminate Runtime Allocation.
133 — Static Allocation
For embedded products: Known Memory Before Runtime
can improve:
reliability.
134 — Heap Avoidance
Repeated dynamic tensor allocation can create:
jitter.
AI Runtime Memory Policy Is Real-Time Policy.
135 — Preprocessing Memory
Camera resize/crop buffers can consume as much memory as the model.
Measure the Whole Pipeline.
136 — Postprocessing
Detection models often require:
NMS or related operations.
The Model Output Still Needs Product Interpretation.
137 — NMS
Non-Maximum Suppression can itself consume meaningful CPU time.
Postprocessing Can Become the Bottleneck After NPU Acceleration.
138 — Accelerator + CPU Balance
If NPU inference takes: 4 ms
but CPU postprocessing takes: 18 ms,
product latency is: Not 4 ms.
139 — Asynchronous Inference
CPU can perform other work while accelerator executes where platform supports it.
Overlap Compute Instead of Serializing Everything.
LiteRT v2 specifically highlights asynchronous execution as part of its newer on-device inference architecture.
140 — Pipeline Parallelism
Example: Frame N → NPU
while: Frame N+1 → ISP
and: Frame N−1 → Application.
Throughput Can Improve Without Reducing Individual Stage Latency.
141 — Batch Size
Cloud AI loves large batching.
Real-time edge systems often use: Batch 1.
Because latency matters.
142 — Throughput vs Latency
Batching can increase throughput but also increase waiting.
Optimize for the Product Metric.
143 — Real-Time Inference
Real-time does not mean: AI is fast.
It means: Inference + Pipeline meets a required deadline.
144 — Worst-Case AI Latency
Average: 12 ms
is not enough if: worst case = 140 ms
and product deadline is: 30 ms.
AI Needs Real-Time Distribution Thinking Too.
145 — RTOS Integration
On MCU products:
AI inference competes with:
sensing.
AI Must Fit Into the Scheduler Budget.
This directly connects Page 27.
146 — Linux Integration
On Linux platforms:
AI competes with:
storage.
AI Performance Must Be Measured Under Real System Load.
147 — CPU Affinity
High-performance systems may allocate workloads among cores.
Scheduling Can Protect AI or Protect Control From AI.
148 — NPU Scheduling
Several models may share one NPU.
Accelerator Is a Shared Resource.
149 — Multi-Model Products
A system may run:
Model Interaction Matters.
150 — Model Priority
A safety/critical anomaly detector may deserve resource priority over:
background analytics.
AI Workloads Need Scheduling Policy.
151 — Thermal Engineering
Accelerators consume power.
Sustained inference can heat:
regulator.
AI Performance Is Thermal Performance.
152 — Thermal Throttling
A product may benchmark: 30 FPS cold,
then fall to:
after 20 minutes.
Benchmark Steady-State Thermal Performance.
153 — Peak vs Sustained AI
Marketing often highlights: Peak.
Products live at: Sustained.
154 — Power per Inference
For battery products: Energy / Inference
may matter more than: Inferences / Second.
MLPerf Tiny explicitly supports energy benchmarking for this reason.
155 — Idle Energy
Always-on AI can spend 99% of time waiting.
Optimize the System Between Inferences Too.
156 — Wake-Up Architecture
Low-power sensor → detects event → wakes AI accelerator.
Intelligence Can Have Power States.
157 — DVFS
Dynamic voltage/frequency scaling may reduce energy when full performance is unnecessary.
AI Performance and Power Can Be State-Dependent.
158 — Battery Impact
Ask: How many inferences per battery charge?
Product Energy Metric Beats Chip Benchmark.
159 — PCB Power Design
NPU/GPU bursts create:
PDN demands.
AI Accelerator Performance Depends on Power Integrity.
160 — AI + PI
Power droop can cause:
unreliable operation.
Model Performance Can Become PDN Performance.
161 — AI + Thermal
Higher accelerator utilization → more heat → lower clock → higher latency.
Electronics Physics Closes the AI Performance Loop.
162 — AI + SI
External DDR or PCIe accelerators require: robust high-speed channels.
Edge AI Can Create High-Speed PCB Requirements.
163 — Hardware / AI Co-Design
Choose:
SoC
RAM
Flash
Sensor
Camera
Power
based partly on the intended model.
Don't Finish the Board and Then Ask Where the AI Fits.
164 — Sensor Resolution
A 12 MP camera creates far more data than a 1 MP sensor.
More Pixels Create Compute and Memory Cost.
165 — More Resolution ≠ More Accuracy
The target object may not require full sensor resolution.
Optimize Information — Not Pixel Count.
166 — Region of Interest
Crop/select important regions.
Process the Data That Matters.
167 — Frame Skipping
Not every frame needs full inference.
Temporal Redundancy Can Save Compute.
168 — Adaptive Inference
Normal state: low-rate AI.
Suspicious state: high-rate AI.
Intelligence Workload Can Follow Product Risk.
169 — Early Exit Direction
Some model architectures can stop inference early when confidence is sufficient.
Easy Cases Do Not Always Need Full Compute.
170 — Cascade Models
Small model: screening.
Large model: confirmation.
Spend Compute According to Uncertainty.
171 — Confidence
Model probability is not automatically: Real-World Confidence.
Calibration matters.
172 — Probability Calibration
A model saying: 90% should ideally have interpretable reliability if the application uses probability quantitatively.
Confidence Needs Validation.
173 — Decision Threshold
Output: 0.62
may be:
depending on threshold.
Threshold Is Part of Product Firmware.
174 — Threshold Versioning
Changing threshold can alter field behavior without changing model weights.
AI Configuration Is Product Configuration.
175 — Uncertainty
Some inputs are outside training distribution.
A strong system can sometimes recognize: I Don't Know.
176 — Out-of-Distribution — OOD
Model trained on: motors
may see: hammer noise.
High Neural Confidence Does Not Guarantee Familiar Input.
177 — Reject Option
Products can define: uncertain → request another measurement / use fallback.
Not Every Input Needs a Forced AI Answer.
178 — Fallback Algorithm
If AI fails or confidence is too low: What Happens?
Possible architecture: AI → if uncertain: Classical Logic / Safe Product State.
AI Should Fit Inside the Overall Reliability Architecture.
179 — Explainability
Some applications need insight into: Why the model produced this result.
The required depth depends on product/market.
180 — Feature Attribution Direction
Engineering tools may inspect which:
signal ranges
image regions
features
influence predictions.
Explainability Can Help Debug Models — Not Only Explain Them to Users.
181 — AI Failure Analysis
When prediction is wrong:
don't only ask: Why did the network fail?
Ask:
Sensor?
Label?
Preprocessing?
Quantization?
Runtime?
Threshold?
Environment?
AI Failure Is End-to-End Failure Analysis.
182 — Golden Model vs Deployed Model
Maintain: Reference High-Precision Model and Embedded Deployment Model.
Compare outputs.
Deployment Optimization Should Be Measured Against a Reference.
183 — Numerical Correlation
For selected samples: Training Framework vs. ONNX vs. Target Runtime vs. NPU
Find Where Outputs Begin to Diverge.
184 — Layer-by-Layer Comparison
If the final output differs: compare intermediate tensors where tooling permits.
Debug Numerical Drift Like a Signal Chain.
185 — Quantization Regression
Track accuracy after every:
compiler
runtime
quantization
change.
Performance Optimization Must Not Silently Destroy Accuracy.
186 — Dataset Regression
Keep a fixed validation suite.
Every Model Release Should Be Compared Against the Same Evidence.
187 — Hardware-in-the-Loop AI Testing
Run actual:
with controlled stimuli.
Test Intelligence on the Physical Product.
188 — Recorded Data Replay
Replay known sensor data through the embedded pipeline.
Repeatable Input Enables Repeatable AI Validation.
189 — Live Hardware Test
Recorded data cannot expose:
sensor timing
optics
noise
synchronization.
Real Sensors Still Matter.
190 — Camera Calibration
Vision AI performance depends on:
sensor.
Optical Manufacturing Can Become AI Accuracy.
191 — Lens Variation
Different focus or optical alignment between units can shift model accuracy.
Vision AI Is Mechatronic Manufacturing Too.
192 — Microphone Variation
Acoustic AI depends on:
microphone sensitivity
enclosure
port geometry.
Audio Model Performance Can Be Mechanical Design Performance.
193 — Vibration Sensor Mounting
A vibration model can change dramatically if sensor mounting changes.
Mechanical Transfer Function Becomes Dataset Distribution.
194 — Manufacturing Calibration
Production may calibrate:
sensor offset
camera
microphone
accelerometer
before AI operation.
Calibrated Hardware Creates More Stable AI Inputs.
195 — End-of-Line AI Test
Manufacturing test can verify:
AI Can Be a Production-Test Requirement.
196 — Reference Stimulus
Apply known:
input and verify expected response range.
AI Product Manufacturing Needs Functional Intelligence Testing.
197 — Not Every Unit Needs Full Dataset Testing
Production testing should verify: Manufacturing Integrity.
Full model validation belongs primarily to engineering qualification.
Design Validation and Production Test Serve Different Purposes.
198 — Model Version
The finished product includes: Firmware Version
Model Version.
The Model Is Product Software.
199 — Model Hash
A cryptographic hash can identify exact model artifact.
"Model v3" May Not Be Enough for Traceability.
200 — Hardware / Model Compatibility
Model may depend on:
NPU generation.
Correct Model Must Reach Correct Hardware.
201 — Model OTA
Connected products may update models separately from full firmware.
AI Lifecycle Can Move Faster Than Firmware Lifecycle.
202 — Signed Models
If model behavior affects product operation: Model Authenticity Matters.
Do not accept arbitrary model files.
203 — Model Rollback
A new model can reduce field performance even if it technically runs.
AI Update Needs Known-Good Recovery.
204 — Canary Model Deployment
Deploy to:
first.
Monitor:
resource use.
Fleet AI Should Be Rolled Out Gradually.
205 — Shadow Evaluation
A new model can sometimes run in parallel without controlling the product.
Compare: Current Model Decision vs. New Model Decision.
Evaluate Before Granting Control.
206 — Model Drift
Over time, the real-world data distribution can change.
Examples:
machine wear
new environment
new user behavior.
The World Changes Even If the Model Does Not.
207 — Data Drift
Input distribution moves.
Detect the Change Before Assuming Model Quality Is Constant.
208 — Concept Drift
Relationship between input and correct output itself changes.
Old Labels May No Longer Describe New Reality.
209 — Accuracy Without Labels
Field systems often do not have immediate ground truth.
Therefore monitoring may use:
confidence
distribution
alarm rates
as proxy indicators.
Field AI Monitoring Is Hard Because Truth Arrives Slowly.
210 — Feedback Loop
When reliable labels eventually become available: Field Error → Dataset → Retraining → Validation → New Model.
AI Products Need a Learning Lifecycle.
211 — Dataset Versioning
Dataset used for model:
should be identifiable.
Reproduce the Training Evidence.
212 — Training Code Version
Model artifact depends on:
Framework.
The Model Is the Output of a Build Process.
213 — Model Reproducibility
A professional release should trace: Dataset → Training Configuration → Model → Conversion → Quantization → Compiler → Deployment Artifact.
AI Needs Configuration Management Like Firmware.
214 — MLOps Meets Embedded Engineering
Cloud MLOps often ends at: Model Artifact.
Embedded AI continues: Runtime → Hardware → Power → Thermal → Physical Sensor → Manufacturing.
Edge MLOps Must Include the Device.
215 — AI SBOM Direction
In addition to ordinary software dependencies, future lifecycle management increasingly needs visibility into:
libraries.
AI Supply Chain Is Part of Software Supply Chain.
216 — Security of AI Models
Threats can involve:
unauthorized replacement
model theft
malicious input
depending on product.
AI Adds New Assets and Attack Surfaces.
217 — Model Confidentiality
Proprietary models may represent valuable IP.
Deployment Architecture Should Consider Whether Model Extraction Matters.
218 — Adversarial Inputs
Machine-learning systems can sometimes respond unpredictably to carefully constructed inputs.
AI Security Is Not Solved by TLS.
The required defense depends on application/threat model.
219 — Sensor Spoofing
A physical attacker may manipulate:
sensor input.
AI Security Begins at the Sensor.
220 — AI + Cybersecurity
The device still needs:
Secure Boot
Authenticated Update
Access Control
Model Integrity.
Intelligent Product Must First Be a Secure Product.
221 — Privacy
On-device inference can reduce the need to transmit raw:
sensor data.
MLCommons and PyTorch both highlight local processing as one of Edge AI's privacy/responsiveness advantages.
Sometimes the Best Cloud Privacy Architecture Is Not Sending the Raw Data to the Cloud.
222 — Local Feature Extraction
Device can upload: Result
instead of:
Example:
bearing_health = 0.83
instead of uploading continuous vibration.
223 — But Preserve Debuggability
If only prediction is stored: How do engineers investigate false predictions?
Selected event capture may be needed according to privacy and storage requirements.
224 — Edge / Cloud Partition
Ask:
What Must Be Local?
latency
privacy
offline control.
What Belongs in Cloud?
fleet analytics
model training
long-term trend.
Intelligence Can Be Distributed.
225 — Cloud-Assisted AI
Small local model can decide: uncertain.
Then send selected data to larger cloud model.
Escalate Difficult Cases.
226 — Local-First AI
Architecture: Local AI handles normal operation.
Cloud extends capability.
The Product Should Not Stop Being Intelligent When Wi-Fi Disappears.
227 — Generative AI Hybrid
Edge model can handle: local private interactions.
Cloud model handles: larger reasoning, when connectivity/permission allows.
Model Placement Can Be Dynamic.
228 — Edge AI Gateway
An industrial gateway can aggregate several sensors and run AI centrally at the edge.
Not Every Leaf Sensor Needs Its Own NPU.
229 — Multi-Camera Edge AI
One edge computer may serve several cameras.
Now resource planning includes:
memory bandwidth
decoder/ISP
NPU scheduling.
AI Architecture Becomes Shared Infrastructure.
230 — Deterministic Resource Limits
At scale, one stream should not consume all accelerator resources.
AI Needs Resource Governance.
231 — AI Runtime Failure
What happens if:
NPU driver crashes
model cannot load
memory allocation fails?
AI Failure Needs Product Recovery.
232 — Fallback Compute
Potentially: NPU unavailable → CPU fallback
if latency still acceptable.
Redundancy Can Exist Across Compute Engines.
233 — But Fallback Must Be Qualified
CPU fallback may use:
miss deadline.
"It Runs" Does Not Mean "It Is an Acceptable Fallback."
234 — Watchdog
An AI task can hang.
Intelligence Still Needs System Supervision.
235 — Inference Timeout
Define: How long can one inference run before it becomes invalid?
AI Has Deadlines Too.
236 — Stale Prediction
If image is already:
500 ms old
then a perfect prediction may be operationally useless.
Correct Intelligence at the Wrong Time Is Wrong Product Behavior.
237 — Timestamp AI Output
A prediction should be associated with: The Input Time.
Especially in moving/control systems.
238 — Sensor Synchronization
Multimodal AI may require:
aligned in time.
AI Fusion Depends on Time Integrity.
239 — Missing Modality
What if:
camera unavailable
microphone fails?
Multimodal AI Needs Degraded-Mode Behavior.
240 — Quality Input Flags
Model may need to know: sensor data is currently unreliable.
AI Should Not Treat Every Tensor as Equally Trustworthy.
241 — Production AI Architecture Review
A mature review asks:
Product
What decision is AI making?
Dataset
Does training data represent production reality?
Model
Is accuracy adequate?
Hardware
Can it fit into memory/power/thermal?
Runtime
Does accelerator support every critical operator?
Timing
Does the whole pipeline meet the deadline?
Reliability
What happens when AI is uncertain or unavailable?
Manufacturing
Will sensor variation change accuracy?
Lifecycle
How will the model be updated and monitored?
Review the Product Intelligence — Not Only the Model.
242 — AI Hardware Selection Matrix
Compare candidate platforms on:
Area | Question |
Compute | Actual model latency? |
Memory | Peak RAM requirement? |
Acceleration | Supported operators? |
Power | Energy per inference? |
Thermal | Sustained performance? |
Runtime | Supported deployment framework? |
Lifecycle | Long-term silicon/software support? |
Choose the Hardware Using the Workload.
243 — Model Candidate Matrix
Compare:
Model | Accuracy | Recall | Latency | RAM | Flash/Storage | Power |
Model Selection Should Be Quantitative.
244 — Accuracy / Latency Pareto Frontier
One model may maximize accuracy.
Another minimizes latency.
Neither dominates completely.
Edge AI Optimization Is Usually a Pareto Problem.
245 — MLPerf
Neutral benchmarks can help compare platforms.
MLPerf Tiny v1.4 now provides standardized workloads and accuracy/energy methodology across a wide range of low-power architectures.
Benchmarks Are Useful — but Your Product Workload Is the Final Benchmark.
246 — Vendor Benchmark Warning
Don't compare: Model A @ INT8
against: Model B @ FP16
and call it: Hardware Comparison.
Benchmark Conditions Must Match.
247 — Accuracy Target Matters
One platform may be faster because:
It Uses a Lower-Accuracy Configuration.
Compare equal-quality targets where possible.
This is one of the reasons MLPerf uses defined quality targets.
248 — Model Compilation Benchmark
Benchmark the exact:
Model
Compiler
Runtime
Hardware
combination.
Edge AI Performance Is a Stack Result.
249 — Compiler Upgrade Regression
New compiler:
but changes numerical behavior.
Toolchain Updates Require Revalidation.
250 — Runtime Upgrade Regression
Same principle.
AI Runtime Is Product Infrastructure.
251 — ONNX Lifecycle
ONNX does not maintain an LTS branch; its official release management states each minor release is superseded by the next.
Model-Format Version Strategy Matters for Long-Lived Products.
252 — Freeze the Production Toolchain
Once qualified:
record:
ONNX Version
Compiler
Runtime
SDK
Driver
Production Intelligence Must Be Reproducible.
253 — AI Debug Build
Development firmware/platform may expose:
intermediate tensors
timing
accelerator profiling.
EVT AI Needs Observability.
254 — Release Build
Production can remove unnecessary debugging while retaining:
failure telemetry.
Optimize Without Becoming Blind.
255 — EVT Edge AI
EVT asks:
Can the Complete Sensor-to-Inference Architecture Work?
Validate:
sensors
preprocessing
accelerator
model
timing.
256 — EVT Dataset Feedback
Prototype measurements may reveal:
Training Data Was Wrong.
That is useful.
Hardware EVT Can Improve the Dataset.
257 — DVT Edge AI
DVT asks: Does AI remain accurate and performant in the final product environment?
Validate:
enclosure
thermal
lighting
acoustics
operating corners
software workload.
258 — DVT Should Use Multiple Units
One prototype cannot reveal: Manufacturing Distribution.
259 — DVT Field-Like Data
Collect data from realistic operation.
Final Validation Should Resemble the Customer Environment.
260 — PVT Edge AI
PVT asks: Can production repeatedly create hardware that preserves AI performance?
Validate:
calibration
sensor assembly
model loading
production tests
traceability.
261 — Production Variation Study
Correlate:
Sensor Calibration
Optical Alignment
Board Revision
with:
AI Output.
Manufacturing Data Can Become AI Quality Data.
262 — Field AI Monitoring
After shipment:
monitor selected:
confidence
alarm rates
resource use
model version.
Model Release Is the Beginning of AI Lifecycle.
263 — Fleet Model Cohorts
Compare: Model v4 vs. Model v5 on similar hardware.
Field Deployment Can Provide Evidence of Model Improvement.
264 — Hardware Cohort Analysis
Compare model performance across:
or sensor vendors.
AI Can Reveal Hardware Variation.
265 — This Is Where 365PCB Becomes Unusually Interesting
A pure AI company may see: Accuracy dropped 4%.
A hardware company may see: Sensor passed electrical test.
A manufacturer may see: Assembly passed.
But an integrated ODM can ask:
Dataset?
Model?
Quantization?
Sensor Calibration?
Lens Alignment?
Power Noise?
Thermal Throttling?
PCB Revision?
Component Lot?
AI Performance Is a Physical Product Property.
266 — Field Data to Manufacturing Feedback
Imagine: False Detection Rate
correlates with: Camera Module Lot B.
Now AI telemetry becomes: Manufacturing Evidence.
This is extremely powerful for365PCB's long-term technology direction.
267 — The Factory Can Learn From the Model Too
We previously said: The Factory Should Learn From the Fleet.
Now we can go one step further: The Factory Can Learn From the Intelligence Running Inside the Fleet.
268 — AI + Digital Thread
Potential chain: Sensor Lot → PCB Revision → Calibration → Model Version → Inference Performance → Field Result
AI Can Extend Product Traceability Into Behavioral Quality.
269 — What Does World-Class Edge AI & Embedded AI Engineering Look Like?
At the highest level: Product Requirement → AI Use-Case Definition → Sensor Architecture → Data Collection → Ground Truth → Dataset Versioning → Model Selection → Training → Accuracy / Error Analysis → Compression → Quantization → Distillation / Pruning where appropriate → Model Export → ONNX / LiteRT / ExecuTorch / Target Toolchain → Hardware Compiler → CPU / DSP / GPU / NPU Mapping → Memory Planning → Zero-Copy Data Path → Preprocessing → Inference → Postprocessing → Decision Logic → Timing Validation → Power Measurement → Thermal Validation → Uncertainty / Fallback → Security → Model Versioning → Signed Deployment → Model OTA → EVT → DVT → PVT → Production Calibration → Field Monitoring → Drift Detection → Retraining → Fleet Deployment → Reliable Product Intelligence
That is the difference between: Running a Neural Network
and: Engineering an Intelligent Product.
Typical Edge AI & Embedded AI Development Deliverables
Depending on project requirements, a 365PCB ODM AI program may include:
AI Product Requirements
AI Use-Case Definition
AI Feasibility Analysis
Classical Algorithm vs ML Trade Study
Sensor / AI Co-Design
Sensor Data Architecture
Dataset Requirements
Data Acquisition Plan
Data Collection Firmware Inputs
Dataset Construction
Dataset Versioning
Ground-Truth Definition
Annotation Guidelines
Dataset Quality Review
Train / Validation / Test Strategy
Device-Level Dataset Split
Production Variation Dataset Inputs
Environmental Dataset Inputs
Data Augmentation
Synthetic-Data Inputs
Preprocessing Architecture
Feature Engineering
DSP + AI Hybrid Architecture
Classification Model Development
Regression Model Development
Object Detection
Segmentation
Audio AI
Keyword Spotting
Anomaly Detection
Time-Series AI
Predictive-Maintenance Inputs
Sensor-Fusion AI
TinyML
MCU AI
Edge Vision
Edge Audio
Multimodal AI Inputs
On-Device LLM Feasibility
On-Device GenAI Inputs
Model Architecture Selection
Model Training
Transfer Learning
Model Fine-Tuning
Model Benchmarking
Confusion-Matrix Analysis
Precision / Recall Analysis
Threshold Optimization
Confidence Calibration
Out-of-Distribution Inputs
AI Fallback Architecture
Model Compression
Post-Training Quantization
Quantization-Aware Training
INT8 Optimization
INT4 Feasibility Inputs
FP16 / Mixed-Precision Inputs
Pruning
Structured Sparsity Inputs
Knowledge Distillation
Model Export
ONNX Integration
LiteRT Integration
ExecuTorch Integration
CMSIS-NN Integration
Tensor Runtime Integration
Target NPU Compiler Integration
Operator Compatibility Analysis
Graph Partitioning Analysis
Accelerator Fallback Analysis
CPU / DSP / GPU / NPU Architecture
Hardware Accelerator Selection
Model / Hardware Benchmarking
MLPerf-Based Comparative Inputs
Memory Footprint Analysis
Activation-Memory Analysis
Tensor Memory Planning
Static Memory Allocation
AI Runtime Integration
Zero-Copy Architecture
DMA Buffer Integration
Camera / ISP / AI Pipeline
Audio / DSP / AI Pipeline
Sensor / AI Pipeline
Preprocessing Optimization
Postprocessing Optimization
Asynchronous Inference
Multi-Model Scheduling
Accelerator Resource Management
Real-Time Inference Inputs
Worst-Case Latency Analysis
Frame / Sample Timing
RTOS AI Integration
Embedded Linux AI Integration
CPU Affinity Inputs
Inference Profiling
Throughput Analysis
Latency Histogram
Power Measurement
Energy-per-Inference Analysis
Low-Power AI Architecture
Always-On AI
Cascaded AI Architecture
Thermal Characterization
Sustained AI Performance
Throttling Analysis
PDN / AI Power Inputs
DDR / Memory-Bandwidth Inputs
Edge / Cloud AI Partition
Offline AI Architecture
Model Security Inputs
Model Integrity
Model Signing
Model Authentication
Secure Model Deployment
Model Confidentiality Inputs
Adversarial-Robustness Inputs
AI Privacy Inputs
Model Version Control
Hardware / Model Compatibility
Model Release Manifest
Model Hash / Traceability
Model OTA
Model Rollback
Canary Model Deployment
Shadow Model Evaluation
Model Drift Monitoring Inputs
Data Drift Inputs
Field AI Telemetry
AI Health Monitoring
Model Cohort Analysis
Dataset Feedback Loop
Retraining Pipeline Inputs
MLOps / Edge MLOps Inputs
AI Toolchain Configuration
AI Build Reproducibility Inputs
Compiler / Runtime Version Control
Reference Model vs Deployed Model Correlation
Numerical Correlation
Quantization Regression Testing
Recorded-Data Replay
Hardware-in-the-Loop AI Testing
Live Sensor Validation
Multi-Unit AI Validation
Camera Calibration Inputs
Sensor Calibration Inputs
Production Calibration
End-of-Line AI Functional Test
Production Model Programming
AI Product Configuration Traceability
EVT AI Validation
DVT AI Validation
PVT AI Validation
Environmental AI Validation
Long-Duration AI Testing
AI Field Failure Analysis
Model Lifecycle Maintenance
Edge AI Architecture Documentation
The actual engineering depth should follow:
AI Use Case + Sensor + Dataset + Accuracy Requirement + Latency + RAM + Storage + Power + Thermal + Compute Architecture + Product Lifetime.
Bring Us the Intelligence Problem — Not Just the Model
You can begin with:
Product Requirement
Sensor Data
Existing Dataset
Existing Model
PyTorch / TensorFlow Model
ONNX Model
Target MCU / SoC
NPU
Memory Limit
Power Requirement
Latency Requirement
Existing AI Accuracy Problem
Existing Edge AI Prototype
or simply:
Tell Us What the Product Needs to Understand — and What Hardware It Must Understand It On.
365PCB can help translate:
Physical Signal → Dataset → Model → Embedded Hardware → Inference → Product Decision → Production.
Don't Just Put AI on the Device.
Define the Product Decision.
Understand the Sensor.
Build the Right Dataset.
Validate the Ground Truth.
Choose the Right Model.
Compress It Intelligently.
Quantize It Carefully.
Map It to the Right Compute Engine.
Control the Memory.
Reduce Data Movement.
Measure End-to-End Latency.
Measure Energy per Inference.
Validate Sustained Thermal Performance.
Design the Fallback.
Protect the Model.
Version the Model.
Test It on Real Production Hardware.
Monitor It in the Field.
Feed What You Learn Back Into the Next Model — and the Next Product.
365PCB Edge AI & Embedded AI Development connects:
Sensors + Analog + PCB + MCU / SoC + RTOS / Linux + DSP + GPU / NPU + AI Models + Power + Thermal + Manufacturing + Cloud
English
