One such company addressing the issue for the connected home is Veego. They claim to have AI inference that runs on home routers and modems in order to identify performance issues. “The typical router has much lower performance than many connected devices,” said Denis Sirov, Co-founder and CTO, Veego. “We developed processes that provide inference on routers while utilizing less than 0.75% of CPU performance.”
One way they are able to do that is training a robust model in the datacenter, then using part of the runtime code to identify the devices in a specific household. That lets the company narrow the necessary processing to a far smaller dataset than the entire set of options.