How do you deploy ML to edge / mobile?

medium

Answer

$(1) Model conversion: TFLite (Android), CoreML (iOS), ONNX Runtime Mobile, PyTorch Mobile.$
$(2) Quantization aggressive: int8 / int4 for size.$
$(3) Pruning + distillation.$
$(4) Hardware-specific: NNAPI, Metal, DSP, NPU (Qualcomm Hexagon, Apple Neural Engine).$
$(5) On-device fine-tuning (federated learning).$
$(6) Model size 5-50 MB typical.$
$(7) Offline capability.$
$(8) Careful with battery / thermal.$
$Use cases: face detection, keyboard suggestion, translation.$
$Cloud fallback for hard cases.$

Check yourself — multiple choice

Random
Convert TFLite/CoreML/ONNX Mobile/PyTorch Mobile + aggressive int8/int4 + prune/distill + HW-specific (NNAPI/Metal/DSP/NPU) + on-device tune (federated) + 5-50MB + offline + battery/thermal aware; cloud fallback hard cases
Just cloud
Not real

Edge: convert + quant + HW-specific + battery aware + cloud fallback.

#serving#infrastructure

215 interview questions in this topic.