0:29
Hi everyone, welcome back to C-Shop Kooner live show
0:53
As always, I'm your host, Stephen Simon, and I will be back with another episode of Ask Me Anything show
0:58
can you imagine episode number 33 i'm kind of getting very delayed with this show i don't do
1:02
much often because we have been doing so many other live shows and a partner conference and
1:08
events so i'm gonna busy with that but we are back again with ama uh it's it's been a while
1:13
almost like after two weeks we're coming to the show so welcome everyone welcome back to
1:17
c-sharp corner live show um if you're someone who's joining us for the very first time i'll
1:22
do a quick reminder that i i always say that we stream eight days in a week because we do so many
1:29
so every monday we do azure ai show that's both hosted by me where we talk about the uh azure ai
1:36
ecosystem uh we talk about different cognitive services and all that michael has to provide
1:42
then on wednesdays we talk we feature our c-sharp on our mvp show where we feature the
1:47
the community leaders on how they started their MVP journey, followed by technical session
1:54
Then on Thursday, we do Ask Me Anything, as it is today, Thursday. Then on Fridays, we do Growth Mindset shows
2:00
That's at 12 a.m. Eastern. We changed the time. I think we are at season two, where we're talking about how you can go ahead and build better softwares
2:08
That's hosted by Mahesh Chan. Then on Saturdays, we have Rocking the Code World with .NET Dave
2:14
I believe he is at episode number 22. He has interviewed some of the really great speakers from the .hunting system, Matt Stolson
2:21
Caitlin Dolat, Kimber Hibben, Scott Hunter. I think the only guy who is missing in his list is Scott Hansel, but I think he should
2:28
be getting it very soon. So apart from that, we have a few announcements that tomorrow we have the IoT conference that
2:36
starts at around 8 a.m., I think 7 a.m. Actually, 7 a.m. Eastern Time, New York
2:41
So definitely go ahead and check out. We have speakers. the really excited section that i'm uh excited is about the project 15 uh from microsoft uh
2:50
the founder itself she's gonna come and talk about how the project actually started at the
2:54
insights and how you can also go ahead and contribute to that open source project followed
2:59
by that on the saturday we have the global ai student conference i'm just talking to hank he
3:05
said that we have over 1200 stations on our platform and uh apart from that i just forgot
3:11
to tell you a very interesting thing that in tomorrow's conference we are giving away microsoft
3:15
surface laptop 4. can you imagine that i someone give it to me i need it so they're giving away
3:21
microsoft surface laptop 4. you just have to be very active in the conference shared social media
3:26
uh yeah that's how that's how simple the criteria is so that's all in the in the this week uh in
3:34
the coming in the coming days we have conference on azure we have conferences on dotnet okay hi
3:40
everyone uh hi sir thank thanks for joining thanks for your time uh and many more exciting events so
3:46
stay tuned at the c sharp dot live at c sharp dot live we add all our live shows and watching
3:53
conferences so just hit that platform and you can find all the upcoming uh live shows so hi welcome
3:59
once again everyone please drop your location in the comments it's always interesting to know from
4:03
which location you're joining you never know well your neighbors might be turning you can go ahead
4:07
and make friends in the live chat okay so having said that in today's in today's amy right i'm
4:13
really excited to host daniel that and daniel is going to talk about the ml.net right so i think
4:21
ml.net was released almost a couple of years by if i'm not wrong and it is it is still new but i
4:27
believe it has matured a lot so it's pretty interesting topic to go ahead and cover today
4:32
and before i go and bring our guest let me go ahead and read quick bio for daniel daniel is
4:40
a cn software developer with over 15 years of experience and he's a big fan of c sharp
4:45
since his inception inception in 2008 2001 so it means when it started he actually started working
4:51
with dotnet he's an international speaker in the field of dotted technologies and he's spoken in
4:56
over 40 conferences in eu and usa in the last four years he's an organizer of apex walks conference
5:02
That's great. That would be great for us to talk about on what they're doing in the coming days
5:07
And he has taught over 700 students and he's also a Microsoft MVP
5:12
So without any further ado, let's go ahead and bring our guest, Daniel Kostia
5:17
Hi, Daniel. Welcome to the C-Shop Corner Live Show. Hi, Simon. It's a pleasure to be here
5:23
I remember with pleasure C-Shop Corner was like a lighthouse for me for more than 15 years, long before Stack Overflow
5:36
And yeah, it was like an extension of help from Video Studio or something, but anyway, very useful during all this time
5:50
And thank you for inviting me. Yeah, thanks, Daniel. Actually, thanks, Daniel, for accepting the invitation. It's always great to have you back. I think I'm hosting for the second time, so I'm really excited. And yeah, actually, C Sharp Corner is a pretty old website. As I read all the information about us, it started probably when I was born. That's how this started
6:14
So I think it was known as .net heaven. So they kind of changed this to C Sharp Corner or something like that
6:21
Yeah, it's but thanks to all your readers and contributors who look for the platform
6:25
I think that that's how you have grown in the past two decades. So Daniel, how are you doing
6:31
Actually, by the way, where are you joining us today from? I'm joining from Netherlands
6:37
Bahaag is the city. I'm originally from Romania, but now I'm living here for the second year
6:44
and I'm planning to stay a few more years. I have a job here now
6:49
Okay. How's the weather? Is it like cold? Is it like... Well, it's getting warmer
6:58
but yet we are waiting for the real spring because here it's very pleasant
7:06
The flowers are everywhere and it's getting green and it's a pleasure, yeah
7:13
Okay. That's great. Lucky you. Here in India, we have 45 degrees Celsius
7:17
That's close to 100 degrees. Wow. It's very hard. So Daniel while I was reading your bio I just read that you are the organizer and the co of the ApexConf and that I not prone to re t for the Apex box right
7:36
Well, we had two editions, but then the pandemic hit us like everywhere else
7:45
And for now, it's kind of on hold. We can see if we will continue or not
7:54
But nevertheless, for me, it was an opportunity to meet big speakers and to get together the community
8:06
And now it's like pretty fine to do this online. Everyone are migrating online
8:15
So I'm fine since I can talk about technology. I can listen to such wonderful events
8:25
It's fine. OK. Yeah. So I think people have moved online. But I think people are definitely very desperate
8:31
about the offline events. People really need it. Yeah. So with that, Daniel, as you say today
8:37
we're going to talk about ML.net. But actually, when I go ahead and do a quick Google search
8:42
for ML.net, it tells me that it was written only a couple years back and i see you you have been working in this field from back in dot from 2001 so how did
8:51
you actually go and get started and working with this ml.net ecosystem with ml.net yeah yep i have
8:59
prepared a cool topic for an event related to iot how we can deploy and run a dotnet core application
9:12
on an iot device a raspberry pi and because the electronics was an older passion which i didn't
9:25
developed enough iot was like wow another chance to to meet electronics and
9:35
and electronic components and sensors and LEDs. And, of course, when I start to play with the reading data from the sensors
9:47
very soon I got lots of data. And the larger the data, the bigger the chaos
9:58
So kind of naturally, I have realized, okay, I have to get some meaning from this data, some information, and machine learning was the
10:10
next natural step. And of course, I couldn't miss ML.net because it was released, the beta released
10:20
was in the same time, and I have joined these large and beautiful areas
10:27
okay so that's great i think so you started actually with iot use her that there's a lot
10:33
of gear that's coming in so i would definitely i would be divided to feature makes a lot of data
10:37
and she said hey this data can be used to go and yze it so actually that's how you started with
10:42
the mld.net right so that that's great so that's great so maybe let's go ahead and then get started
10:47
and actually in the group we could talk about what is actually ml.net at the very first place
10:52
what is ml.net first I would like to add I started with ml.net and no data science knowledge
11:00
and that's very important because I'm supposing most of our audience it's developers so yeah it's
11:11
important well what is ml.net I will tell you a few things which is more
11:21
relevant well first it's a framework because you can follow some directions
11:29
and use some libraries in order to build a model and to consume a model so it's
11:38
kind of complete environment to do machine learning even more you can do deep learning
11:45
with ml.net so it's not vision it's not limited to machine learning but primarily this is
11:56
for developers it's code first and that means you can code every step there and build from scratch
12:09
a model and then as well you can consume it and it's a cross-platform and it's very good news
12:17
because maybe it's not an is not news it's a fact which is a very necessary
12:24
nowadays because we want to run this on different platforms for example to
12:33
mention only one of them the arm CPUs are more and more popular now with Apple
12:41
doing this switch towards M1 CPUs. I think this was a last
12:48
if you need more reasons to invest in ML.net as a development team
12:58
this was maybe the last and very important reason. Apple migrating to ARM CPUs
13:11
and of course it's for example I didn't run ML.net only for
13:23
only for Windows I usually run for Linux so it's indeed a cross-platform and
13:32
another thing it's on-premise what that means you can run in your on your
13:41
machine you don't need a cloud you don't need to send the data over the internet it's running on
13:48
the edge so you can create your model on your machine and consume it maybe you're on the top
13:57
of the mountain maybe you are on on a iot device you can do all the stuff there and for now it's
14:09
doing inference for deep learning models but we are expecting soon to be able to do the training
14:18
for deep learning as well like a neural network this is a ML.net but I will get into some details
14:33
um so okay that makes sense uh you did say that ml.net is open source and you did say that you
14:41
don't don't run just on windows and you primarily run on linux right so uh so when i think of ml.net
14:48
or anyone who comes from a c sharp background they have they don't have to go ahead and worry about
14:52
learning python or maybe our programming language they can use their existing c sharp and dot net knowledge to go ahead and start building machine learning models right yeah well there are six millions dotnet developers in the world
15:07
so we cannot expect all of them to learn python only to consume um nml.net uh and machine learning
15:19
model. So it's natural to look around in this large .NET ecosystem and find something to build
15:31
from scratch a machine learning model. And this is ML.NET. We have some other machine learning
15:37
services like Azure ML or cognitive services, but this is code first for developers and it's
15:47
on-premises. That's great. On-premises definitely makes sense because some of the organizations don't would like to go and put their all the training data to the cloud, right
15:57
So you can just do it on your system. That definitely is a benefit. We have one question
16:03
coming up from Jose, if I say it right. He says, could C-sharp and ML.net be faster than Python
16:10
python is interpreted and c sharp is g g it compliant can you say again please
16:17
can you can you read on the screen i've added it on the screen oh okay let me
16:28
okay um could c sharp and then i'll only be faster oh okay good question
16:34
by design it's faster because it's compiled and there is a white paper somewhere if you search for
16:44
ml.net white paper it will direct you right to to the document it's way faster than python and
16:53
than other frameworks because it's yeah it's compiled okay so i think uh pp he would one would usually think of it this way is because we usually use
17:11
primer python and r in for the data science right so this question comes obvious that hey will it be
17:17
as hostile as not okay so you said yeah it would uh okay let's take one more questions from arjun
17:23
he says could you please guide me how we can use cpu while training model yeah there is support for
17:29
training using gpu you have to install additional nuget packages but indeed this is supported
17:39
and this is uh important especially for uh deep learning scenarios machine learning it's not not
17:48
very um computing uh extensive task not not like deep learning you can train machine learning in
17:57
seconds or minutes depending on the data of course but when we are talking about the learning it's
18:04
days or weeks yeah definitely it takes a lot i mean even for the for the training right you
18:11
were doing in a local system you need a really i think a good content especially yeah else if you
18:17
if you run it on the cloud it's going to cost you a lot yes so uh with that you did mention
18:24
channel about the NuGet packages, right? You can go ahead and install some of the NuGet packages
18:29
right? So people who come from the Python or R, what they would usually do is, for instance
18:35
they have this amazing library of pip. At least if I talk about Python, they have this pip
18:41
that allows you to go ahead and add n number of Python libraries, maybe for visualizations, maybe
18:47
for exploratory data ysis. That is really, really helpful to go ahead and even start
18:53
deciding which algorithm to use even if we train it right so uh how does that work in the ml.net
18:59
so one has to is there a good uh packages available in the new package system or someone has to play
19:05
for somewhere how does that work yeah um i will get maybe into more details later the demo part but
19:13
ml.net it's more it's modular so it has the core in Microsoft ML NuGet package and then some
19:26
trainers are in some other packages because you don't need all the time all the trainers
19:32
for example like GBM my GBM it's in additional NuGet package fast three it's in additional
19:40
you get package that it's modular so you can keep adding more for example if you need to work with
19:50
onyx or tensorflow you have to add another additional nuget package okay this is keeping it
19:58
very flexible agile okay yeah so this so we should compare it with like other python or r
20:07
but I think it is mature enough to go ahead and provide you all the libraries and the factors
20:13
that you may need to go ahead and get started with your ML. I have a quick question that I just feel
20:20
it right. So, okay. So, I think, Daniel, before I ask that, do you have anything to go ahead and
20:28
tell? I would just ask any questions that I feel. Anything you want to cover, follow along
20:32
Okay, so do you want me to get into some... Please stop me and ask questions anytime
20:39
because we are doing this for the audience. Yeah. It's not scripted, right
20:47
It's all going on. Yeah. Okay. I will switch back to the slides now
20:53
Yes. All right. well it's not easy to start with machine learning I I I saw that myself it was so hard I I it was
21:10
like okay I want to do that I understand a little enough to get it started but something it's not
21:20
right I I couldn't find the scenarios I couldn't find okay I know in theory how I can use it but
21:31
now in production or in real scenario how I where I can put my machine learning so why is that we
21:43
are so into the algorithmic programming we don't even realize it. I have an
21:49
interesting hypothesis for you. Has machine learning more human-like behavior than traditional coding? Think a little. Actually it has. Let me give you
22:04
an example. Writing an algorithm is like writing down a cooking recipe. So the
22:11
The recipe has ingredients, that's input data, and steps you have to follow to get the food done
22:21
That's output data. Well but you or someone else maybe from Stack Overflow or from C Corner did the recipe before
22:35
And the machine is only running it. Is the machine intelligent for running new recipe
22:44
Is this making the machine more human? No. Machine learning, on the other side, is inventing a recipe
22:51
Failing, succeeding, repeating until the recipe is good enough to eat it
22:59
This is making the machine learning more human than coding. We put our intelligence, aka algorithms, in the code, and that's pretty rigid
23:11
Why not to rely on the machine learning to create intelligence? Well, another question, is your algorithm adaptable by its nature
23:24
No, it's not. You're writing there and it stays. It's very rigid
23:29
It's frozen into your binaries. But machine learning, it's adaptable to data
23:37
We, as developers, have to be prepared for this new paradigm. I mean, machine learning
23:44
Because in 10 or 20 years, it will change the way we code
23:49
But until then, we continue to write our own recipes ourselves. So again, why it's so hard to get into machine learning
24:02
Well, first, it's the scenario to define your problem. Because when we have a hammer, all the things around us look like nails
24:14
This is machine learning. We have to identify valid scenarios for our problem
24:22
Because machine learning is like an extension for traditional programming. It's not replacing it
24:30
Then it's data. We have to get data. And we need data
24:36
When? Yesterday. We need data because no data, no party. we cannot think about machine learning if we don't have data we have to have the
24:49
data in advance I like to say the data is the engine not the fuel think about it
25:00
and then it's training to build the training pipeline it's another hard
25:07
part and remember that we are not married supposedly we are not data
25:13
scientists so yes building the training pipeline it's a challenge but maybe in
25:21
this order scenario data and then training pipeline well let's focus on
25:34
scenario a little ytically speaking we all know what to do with small data
25:40
it's easy to comprehend small data it's easy to see patterns but with large data
25:47
we are lost without instruments our brain is built like that for example
25:52
images we are not seeing images like an array of pixels that's the ytical
26:02
approach we are seeing it in two two dimensional three dimensional arrangements images our band has
26:11
instruments to see the eyes and to hear a picture a movie or a song not to read the arrays of the
26:21
bytes in it and i like to like to make ogies for example machine learning is like intuition
26:29
What is intuition? For me it's like finding the patterns in data. For example, you are in a jungle
26:38
and you suddenly see with the corner of your eye a shadow falling on you. What are you doing
26:46
yzing running well i think having the data you have like danger of collision unexpected
26:57
aggressive yellow and black well 60 60 probability it's a tiger so you better run 60 percent is much
27:10
it's a little well it's good enough to run and to do the ysis later it's the same with the
27:19
machine learning we are not using machine learning for precise 100% precise results your brain just
27:29
runs a machine learning model with a significant score of danger when you are deciding to run
27:38
because maybe it's a tiger it actually yes you are running your own machine learning model now
27:47
this is a wonderful use case where the machine learning is very useful anticipating guessing
27:55
so first yes we have to to position to place this paradigm in the right place machine learning it's
28:05
not like a universal panacea for everything so why it's it's it's so hard
28:15
to identify the scenarios for machine learning because we are used to think
28:22
to think out the algorithms and to put the computers to work this is a
28:28
developer life right we are we are thinking in our mind and then we are
28:34
writing the steps we are not putting the machine to think we have to change that
28:41
in order to to use efficiently machine learning as developers we have to
28:49
retrain our brain in that direction so for that we need an easy tool an
28:58
accessible and easy tool like like ml.net if we are dotnet developers and we
29:06
need data talking about data let's look at these this is it's an artificial
29:14
scenario I I have created for my demos so I can experience with them these
29:23
observations are labeled so we have data like temperature luminosity infrared distance this is data from sensors and for this data I make a prediction the
29:39
prediction it's the the label one of these categories flashlight or lighter
29:46
or daylight or infrared. So my set of readings falls into one of these categories
29:59
Actually, the source it's generating in reality the source is generating the data for the sensors let me
30:12
exemplify we have a data set with readings from an iot device and we have we want to predict the
30:20
category so giving 20 degrees Celsius 90% luminosity zero infrared and the
30:31
distance it's 100 centimeters obviously it's a it's a cold light so very
30:38
probably it's a flashlight not a lighter not else so as I said before the data is
30:50
the engine not the fuel we need more data we need enough data how how much
30:57
data is enough well enough it's when it's solving your problem so more data
31:09
never hurts as as far as it's very data but data is the most important thing for
31:19
for machine learning machine learning model is it is as good as its data the
31:27
machine learning model cannot be better than its data because data is creating
31:34
the machine learning model that means a bad data set drives to a bad model even
31:43
more it's more important to have good data set than having the most performant algorithm
31:58
why is that because a bad data set will result in bad machine learning model so you work for nothing
32:04
so if you think to skip data preparation do not be surprised you have to pay the price later
32:16
for that reason I have called this in ogy with technical debt I have called this
32:23
data preparation debt easy to remember and then it's training traditional versus machine learning well traditional programming is is all about
32:41
dropping some input data into an algorithm and get the output data let me show you this
32:50
in the right side this is traditional you have input drop it along with the algorithm and you
32:59
get output data. Training on the other side it's relying on input and output
33:05
data and ml.net in our case it's figuring out a function an algorithm
33:14
good enough to solve our problem but be very careful and I give you an example
33:22
let's suppose you have a line to measure distance and you know nothing about
33:36
summary or product arithmetical operation some or product and we are trying to figure out why it's this two and two resulting in four and you are
33:54
trying and you're starting to measure to see how long is this piece maybe it's
34:03
1.99 the other one maybe it's 2.2 but it's close to for something well it seems
34:12
to be after many measurements it seems to be the operations between these two
34:18
it's a sum but if you are resuming to little data like adding only the the
34:31
pieces which are ytically something like two and two you might end up you
34:40
need their product because 2 plus 2 is the same like 2 by 2 but if you have
34:47
enough data you will realize soon you will need plus or product because 3 by
34:54
3 it's not 6 but 9 so this is maybe a trivial and naive example of how you can
35:04
be tricked but little data you have to have enough data in order to to get the
35:12
valid model for your problem then the last part in the life cycle when you are
35:25
trying to consume a model you find yourself in the same traditional
35:31
scenario when you have some input data drop the data into the algorithm aka
35:39
machine learning model and you get the output data so this is nothing new that's
35:46
why the largest part in machine learning is the training part let's take a look
35:55
this diagram I have created for a demo and for the presentations we have the
36:04
training pipeline which is a pipe of transformers a transformer is a component
36:14
that takes the data does some more concrete and it turns the new transform
36:20
data a transformer and the transformers are by design lazy in order to accommodate large pipes
36:29
in order to accommodate accommodate large data manipulation then an estimator it's an object
36:40
that learns from data that the result of the learning of such learning again is a
36:50
transformer so the estimators are eager they are not lazy they are eager for
36:57
example fit method which triggers the training it's an estimator it's it
37:05
triggers the learning and then the prediction it's something which we apply it's like applying a transformation on a row of data but the prediction time
37:23
to get some score some prediction these are the instruments which we are using in the
37:35
demo in a bit let me go to the demo but not before telling you the prediction so
37:45
since we have the model even if we just created it or we are loading from a
37:50
previously created model we are creating the prediction engine dropping some into
37:56
data predict get out data but fortunately for this training part we
38:09
have the model builder what's more the builder well to find the right trainer
38:14
we can ask a data scientist we can get inspired by some my tech over floors
38:27
is sharp corner or we can use model builder what model builder is able to
38:35
detect which is the best trainer for a specific scenario each scenario has a
38:42
catalog associated catalog with a collection of trainers so the model bidder is trying these trainers and it's measuring the quality of the
38:55
generated model builders in a specific time frame the best score wins sometimes
39:06
you don't need the fastest sometimes you need the most precise so it's up to you which one it's
39:14
better for you so again I'm not a data scientist I'm trying to nail down the world of data science
39:23
from a development perspective don't get me wrong I'm not saying data scientist is not needed anymore more
39:32
What I'm saying, you don't have to be a data scientist to get it started
39:37
You have model builder for that. You can later validate for production
39:45
with a data scientist. And let's do some demo. I will create a completely new project, a console application
40:00
But frankly, I will not use this application. It's just a dummy project in order to get to the machine learning part
40:22
Simon, if there are questions, please let me know. Yeah, Daniel, there are a few questions
40:30
Let me know you want to take it. You can take it now or maybe after the demo
40:34
No, I will take them now. OK. I will switch to the so I can see the question myself. OK
40:44
OK, just. OK, you're back, right? So OK. So the first question is coming from Arjun, right
40:50
He's asking that this is a good question, that is it possible to work with TensorFlow
40:57
and ML.NET at the same time? Can we just bring them together
41:04
TensorFlow as a model, it's compatible. I think ML.NET is able to consume TensorFlow models
41:14
But if you plan to train from scratch, you have another framework developed by a Microsoft team
41:22
and it's called TensorFlow.net. If you search for TensorFlow.net, you'll get it there
41:28
And that's even more, it's compatible by naming convention with the Python library
41:37
So for someone who used Python before, it's easy to migrate to .net
41:44
Even more, even more, ML.net, deep learning part, part it's built upon it's built upon tensorflow.net so they are it's a
41:57
dependency okay yeah so they're both the options are that you can go ahead and
42:02
either consume it or maybe just build on your own use this tensorflow.net
42:06
library that's out there right okay so yeah we'll take one more questions okay
42:11
okay okay just yeah okay now you can follow with them can we edit the code
42:17
generated by model builder yes okay let me show you this this is exactly what I'm going to demo
42:23
I will zoom in so we have to right click on the project hit add and then machine learning
42:33
all you have to have it's the data so these are the scenarios which are covered by ml.net I will
42:45
not get to me the details details because we don't have time so I will choose multi classification which is the first one for now I can train with CPU
42:55
but for some scenarios GPU is available as well so I will hit next because I
43:01
want to load the data I have a data you know as I said before no data no party
43:10
but I had the data with about 1,000 readings so I'm selecting which column I
43:16
want to predict and this is the source you know the light of the
43:22
flashlight one of those categories and I will hit advanced data options because
43:29
maybe I want to fine-tune the the type of the features involved in the model
43:38
builder what's detected temperatures a numerical field which is correct correct correct all of these are correct but for PR it's zero or one because it's
43:51
detecting if it's a human presence or not so I leave it categorical but for
43:56
humidity I know it's numerical but maybe I have two little values and it's
44:02
mistakenly assumed it's a categorical but it's not it's numerical and I don't
44:08
need the this field created at because it will not help me get a better model
44:15
daytime fields are better for forecasting not necessarily for such scenarios like multi classification so having these data I will hit next I will
44:29
put for example 20 seconds time for training is it enough or not there is a
44:39
help here it will advise you how long it's advisable to to train but make sure
44:46
it's it's enough you can try with the larger intervals so now what's happening
44:53
here model builder it trying the trainers from a category and okay I will give 30
45:05
seconds because obviously for me because I know already we didn't have enough
45:13
time to detect too many trainers we can see below the micro accuracy the macro accuracy and the
45:29
duration just a few seconds and we are close to get what we need okay
45:43
Okay, next step, we can even test to make a prediction for some randomly generated, actually, I think it's not randomly, it's the first row in our data set
45:57
And then I hit add to solution. In this way, I'm answering the last question
46:07
If we are able to modify the generated code. Yes, we are able to modify by hand after this is generated
46:18
Even more, I didn't install yet, but we are able to persist these steps in a json file and we can keep track in a history
46:36
the results of building of experiencing with this but i will show you what we got here i will
46:47
create this console application generated because this is part it's consuming like
46:57
actually this part it's predicting data but not but not before having the model built
47:04
model input it's generated by the ml.net and it's the corresponding one-on-one C-sharp
47:20
object for our head in the csd file we can see here for example PIR its string but the other
47:31
which are numbers numerical or float and the most important thing in the
47:39
consume models there is not right there model builder this is the training pipeline this is the training
47:59
pipeline we can see here the source it's transformed from string to number and
48:06
then the PR which is categorical feature it's transformed as well into number and
48:15
then all of them now being numbers are concatenated in a composite feature this
48:23
feature it's sent to the trainer here to the light GBM trainer and then this is
48:34
a mapped back to a predicted value and voila we have the training pipeline but
48:47
now this is a something which is a kind of autopilot which is not the best we
48:58
can do and for that I want to show quickly in a few minutes what we can do
49:07
to get a better model and for that I will introduce to you introduce you
49:17
very quickly to Jupyter with .NET kernel. Maybe you're familiar if you are working
49:28
with Python, maybe you're familiar with Jupyter, but now Jupyter is supporting
49:33
.NET and it's supporting C-sharp, it's supporting F-sharp, PowerShell, HTML, JavaScript sequel and we are expecting more so what I have here we can copy and
49:51
paste from our generating generated source code we can paste here the
50:00
pipeline I have cleaned cleaned up a little but what I want to show you we
50:08
can run all these and measure the quality so on autopilot mode the model
50:20
builder it's building a model with this accuracy 91.64 but if we want to go an
50:30
extra mile we should do that if we want a better model we should try to make it
50:40
better and we have some instruments grace to Jupiter notebook we can use
50:47
this in a very visual way so I will run this I will go test over this for
50:59
example we can check how are these how are our data distributed by categories we don't want
51:06
unbalanced data if we have on highly unbalanced data we have to do something related I have another
51:14
presentation which is addressing actually six months ago I had something focused on data
51:23
preparation so there if you can you you can search on YouTube and check how I
51:32
got into details yzing and preparing the data but what I want to see here
51:40
it's we can yze data and decide if we want to keep all the features or we
51:49
simply don't need them all because obviously our impulse it's to to give to
51:57
the trainer pipeline all the data we have but obviously not all the data is
52:03
bringing value so we have to decide which data is relevant which is not
52:09
because the not relevant data we want to remove or something else having too many
52:19
features has a direct impact on the dimensionality of our model so reducing this dimensionality results in a faster model
52:36
The performance is important. Many times we need real-time predictions or near real-time predictions
52:46
so this is important but yzing by features correlation metrics we can see
52:54
which features are having direct impact on another features for example if two
53:02
features are very similar they are variating together they are rising in
53:12
the same time or lowering in the same time very probably they are redundant we
53:19
don't want to keep them both so we can remove one of them for example we have
53:23
to temperature sensor very close in the same system so maybe maybe we we have
53:32
that for redundancy but we don't need to measure both because it will
53:41
increase the dimensionality of our problem then another thing we can do we
53:48
can measure the permutation future importance why what we can see here it's
53:54
the temperature luminosity and infrared have the the biggest impact on our model
54:02
this distance a little but definitely PIR and humidity are not bringing value
54:10
even more if this value are less than zero it's negative we have here like a
54:20
lucky guess so our data it's doing lucky guesses definitely it's not something we
54:29
want to keep so let's remove these features from the lower part and see if
54:37
if our quality of the model increase or not okay i will load again and take a look i keep only temperature
54:49
luminosity and infrared and we will measure again and now it's almost 93 so it increased from 9164
55:02
to 93 it doesn't look much for our case but it can be very important
55:10
improvement and sometimes a few percents are very important i will not go further with this demo because it's not the
55:22
the scope of this talk i wanted to exemplify how we can improve
55:30
but we don't need to we can to have a faster jump start we can rely on model builder
55:39
and my advice it's just start it just start to play it's uh something it's a skill you want to add
55:48
to your skills and it's something complementary we will we cannot avoid the future it will hit us
55:58
sooner or later and we want to be prepared i have some resources which i recommend but meanwhile
56:08
i want to get some other questions so simon if you have something just let me know please
56:16
yeah that was great daniel and and i really love the entire even step by step very go ahead and
56:21
they talk about it and also i did like that he mentioned that chip is now available almost
56:26
everywhere it's not just in python or c plus plus it's also in the sql they call it more like a data
56:31
studio and the amazing part it even inside the data studio you can go ahead and run python version
56:36
it's also in the dot net and i believe it's coming everyone is going and actually adopting
56:41
this jubbit notebook which is pretty handy exporting export as html or pdf that that's
56:47
very easy so uh we have one question uh i think we have a few questions actually uh let's take maybe
56:54
from the top as you're running a little over time. Sitaraman is asking, can ML.NET be extended to run models
57:02
from other source like RunwayML or generative deep learning models like CycleGAN from other sources
57:11
Yes, let me tell you how. ML.NET is working with native. It has its own native format
57:21
but in addition to that it's able to consume for now to consume uh tensorflow models that's um
57:32
in protobuf format or it's dot tf and it's able to consume onyx models well onyx format
57:40
it's a portable it's supported developed by microsoft uh and google supported by amazon and
57:47
or bigger players. So it's kind of becoming a standard. So as far as you can export your model
57:56
in whatever framework you like, if you are able to export it in Onix or TensorFlow
58:02
at least in theory, because maybe there are still some incompatibilities. I cannot get into detail, it's too much material
58:12
But in theory, yes, it's able. And there are a few examples over the internet
58:19
and on the official websites where YOLO model developed with, I don't know, it's carrots maybe. Yeah
58:31
Yeah, it's consumed with ML.net to make predictions. And it's inception model from Intensive Flow
58:41
Inception model is another popular model, which is able to do image recognition
58:47
But there are more, there are more. And I recently played with the ArcFace
58:54
which is face recognition. But as I said before, ML.NET, it's still evolving
59:03
and it's not covering yet all the data types or the all the parameters which are available in the original framework I'm hoping it will be
59:20
compatible with everything soon and at that point we'll be able to consume any tensorflow model yeah
59:29
I think I think it's only been two years and and the Microsoft has been investing a lot in this I
59:33
and I see community growing a lot. And in any of the major conferences
59:38
I see Microsoft coming up and giving us some more, if not majors and minor places
59:42
And yeah, so I think with that, it is also one question that definitely answers
59:45
is that you did mention what YOLO's, so he asked, can we run YOLO on this
59:49
and get object location on this? I think pretty much same approach
59:53
There is a Tiny YOLO version two, which is the official version
59:59
Sample but if you take a look on my PowerPoint there is the Rubik's code blog and one of the
1:00:10
latest articles it's consuming your version 4 which is detecting 80 classes instead of 20
1:00:20
But it's not limited to these so The answer is yes. Yeah with a few versions
1:00:31
but it may be anyone who can figure out how to set the hyper parameters for any
1:00:41
YOLO models and other models Okay, then you didn't mention what you tried this in your blog
1:00:47
so maybe if you could give your blog link in the private chat i can put in the comments
1:00:51
that people can also go ahead and see it later on uh now before and why do you do that uh this is
1:00:57
a pretty interesting question and i think i know we are already over time uh static world would ask
1:01:02
can we use it in unity 3d and the reason i feel he's asking is that in the unity development
1:01:07
you actually use c sharp probably it was c sharp javascript and pool and i think bull is gone so
1:01:14
I'm not sure about if JavaScript is still there or not, but we use definitely C-Sharp in Unity
1:01:20
So you think we could somehow tweak in and use ML models in Unity
1:01:26
Any other application that allows C-Sharp development? ML on Net is built on .NET standard, so it's compatible with anything
1:01:40
And I could see some WPF samples consuming, training and consuming. I can tell Unity 3D it's compatible as well
1:01:55
but I have this I didn see it in work Yeah yeah It something to go ahead and explore maybe right so yeah because uh yeah but people write like c sharp for the unity uh i know let let take uh maybe last
1:02:13
final one then we'll wrap it up uh we'll just uh could you talk about uh it's called scisharp or
1:02:18
c sharp i don't know scisharp i think we have given the link in the chats of the github so i
1:02:23
I think you could go ahead and maybe learn a lot more about that from there
1:02:27
Yeah, C Sharp, it's an umbrella for more technologies, which are dealing with porting Python libraries into .NET
1:02:40
I didn't port directly with these libraries, but the good news, they are supported in C Sharp
1:02:49
so you don't have to import Python code into your solution in order to use it I don't know how much
1:02:59
of it it's comparable but I know it's it's a very nice job in this it's getting mature yeah and it's
1:03:10
it's it's in in in development okay so then I think I think that that's how we would wrap right
1:03:17
I think there are only three minutes over time. Before we wrap, any final thing you want to say or something about all people who joined or something
1:03:27
If you could switch again on my last slide on PowerPoint. Yeah
1:03:31
You just move it to your slide and then add it to the screen. Oh, sorry
1:03:37
That's right. Yep, there you go. Okay, let me conclude then. ML.NET, it's not aiming to replace existing frameworks like TensorFlow, Keras in Python or whatever language
1:03:50
But considering the AI is going to be adopted by the majority of the applications as developers it good we have an in alternative for doing data science machine learning and myself I prefer this code first approach and the fact i can develop in my daily language like c sharp and uh in addition ml it easy
1:04:25
to learn it's fast it runs on premise on different platforms it's open source you can contribute
1:04:34
do not refrain you from contributing I have contributed myself and it was the the first
1:04:42
contributions I was very shy starting but because being a work in progress you definitely definitely
1:04:52
can find something to contribute on different levels and for the future the bless the blazer
1:05:03
has support for inference or prediction for now but for training it will be
1:05:12
supporting we will support it very soon it was not easy because the JavaScript
1:05:18
limitation to single thread model and then very soon we will get support for
1:05:27
very soon means hopefully when dotnet 6 at most but not at 66 dotnet 6 will be
1:05:33
released it's my feeling we will have arm CPU support for training and for
1:05:43
prediction text personally I'm very excited because the Raspberry Pi and and the apple n1 actually i decided to buy an apple n1 um with arm cpu for more reasons but
1:05:58
this is maybe the most important i'm a geek yeah yeah yeah daniel you are definitely a geek and
1:06:06
all your slides are really amazing uh one final question i keep on saying final question Daniel but one apart from all this you know ML data science AI C sharp dot net community stuff speaking in live shows and conferences What are your hobbies What is something that you do apart from all this well
1:06:27
I have a little daughter so I cannot go outside for my runnings or stuff
1:06:36
so I have to be very careful in my spare time well I'm
1:06:45
programming Raspberry Pi I had this guy with me wow that's great
1:06:52
this is a camera proximity, proximity sensors, luminosity, temperature LEDs and I'm playing with this
1:07:03
sometimes but mostly I'm preparing for the next talk and the next talk and this
1:07:11
is taking most of my time. Wow that's great I love that you put the ultrasound
1:07:18
at the very right place. Because I want to feel some familiarity. I humanized my bot
1:07:33
That's amazing. I wasn't expecting that to come. So thanks, Diane. That was an amazing session
1:07:39
So really lovely. And I believe there were quite a few questions. So thank you everyone who joined
1:07:43
us today uh you can always go back and watch the session on c sharp dot live or c sharp corner
1:07:48
youtube channel everything is available on demand and it's all archived so uh thank you so much
1:07:53
everyone yeah thanks really loved it yeah thanks for that awesome gadget people love
1:08:01
thank you again for having me and uh uh i wish you a very good um uh day and thank you very much
1:08:09
all right thank you everyone thanks everyone joining we'll see you next thursday same time
1:08:13
and see you soon bye bye see you soon bye bye