Edge AI for Beginners

They are really embracing ai! I can feel them all around even. Above me. Below me.

Always cool to see SLM support from a big company, albeit for inference

MS GitHub seems to be featuring a lot of beginners courses all at the same time. Wonder if they're just pumping them out with AI at this point.

It seems this is focused on on-device computation - as distinct from, say, Cloudflare's definition of the "edge" as a smart CDN with an ability to run arbitrary code and AI models in geographically distributed data centers (https://workers.cloudflare.com/).

Per Microsoft's definition in https://github.com/microsoft/edgeai-for-beginners/blob/main/...:

> EdgeAI represents a paradigm shift in artificial intelligence deployment, bringing AI capabilities directly to edge devices rather than relying solely on cloud-based processing. This approach enables AI models to run locally on devices with limited computational resources, providing real-time inference capabilities without requiring constant internet connectivity.

(This isn't necessarily just Microsoft's definition - https://www.redhat.com/en/topics/edge-computing/what-is-edge... from 2023 defines edge computing as on-device as well, and is cited in https://en.wikipedia.org/wiki/Edge_computing#cite_note-35)

I suppose that the definition "edge is anything except a central data center" is consistent between these two approaches, and there's overlap in needing reliable ways to deploy code to less-trusted/less-centrally-controlled environments... but it certainly muddies the techniques involved.

At this rate of term overloading, the next thing you know we'll be using the word "edgy" to describe teenagers or something...

It's funny that they used AI to translate into other languages, because the Arabic cover image is just gibberish.

Not comfortable with the phrase edge ai.

The very first sentence:

> Welcome to EdgeAI for Beginners – your comprehensive...

Em dash and the word "comprehensive", nearly 100% proof the document was written by AI.

I use AI daily for my job, so I am not against its use, but recently if I detect some prose is written by AI it's hard for me to finish it. The written word is supposed to be a window into someone's thoughts, and it feels almost like a broken social contract to substitute an AI's "thoughts" here instead.

AI generated prose should be labeled as such, it's the decent thing to do.

What are the best Small Language Models (SLMs) these days?

TL;DR

This is a course on how to use Microsoft compute to maximise their profits

This is far from what I expected. There is not much related to quantization, pruning, common architectures, precision or benchmarking. For those interested in this topic, I would recommend content from MIT HAN Lab.

Hmm, why do they ask to fork it first and then clone the original repo?

Isn’t edge AI just a way to deploy AI to meet product requirements? What is special about this course? Is Microsoft trying to sell this as a service? If so what is the revenue model and hardware used?

I remember when we bought and installed, among the first in the world, the AWS Outpost, sold as an "edge" (of in between cloud and on prem) infrastructure product. Same term has been previously (ab)used also in the security space, at - again - the confluence between cloud and on-prem. And then - yet one more time - the "edge" was a closer data center for localized delivered cloud services.

Is this a Craigslist post from 2007? or just Slopai seconds?

This looks like AI slop to me. The first two modules repeat the benefits of Edge AI five times. The "Practical Implementation Guide" https://github.com/microsoft/edgeai-for-beginners/blob/main/... ends with a Pre-course checklist. The whole article is just mini-examples without enough context to understand anything.

Thank you Microsoft, my llm phishing agents have never been more profitable. Scamming w/ "AI" is the future my friends!

This was made by AI

One of the most common uses for edge AI not listed in this course is computer vision. You similarly want real-time inference for processing video. Another open source project that makes it easy to use SOTA vision models on the edge is inference: https://github.com/roboflow/inference

I would say this is a poor beginners guide for quantization/compression, it's mostly an API guide for tf/keras quantization APIs it doesn't tell the beginner why or when or which layers (and why) they should apply it to.

But the modules that compare the different model families are quite good. As are the remaining modules that are "How to deploy to $platform 101", including microsoft's, of course ;)

Not that I have a better resource at hand for quantization/compression _for beginners_, and I am probably a bad judge for how beginner friendly Song Han's TinyML course was...

I clicked hoping the models would be available in the “Edge” browser.