Hello, my name is Mrs. Holborow, and welcome to Computing.

I'm so pleased you've decided to join me for the lesson today.

In today's lesson, we'll be training and testing a model to ensure the model's outputs are expected.

We'll also explore the characteristics of poor-quality data.

Welcome to today's lesson from the unit "Machine learning using the micro:bit." This lesson is called "Training and testing a model." And by the end of today's lesson, you'll be able to train a machine learning model and test the model's predictions.

Shall we make a start? We will be exploring these keywords in today's lesson.

Training data, training data, a set of examples used to teach a machine learning model how to recognise patterns.

Prediction, prediction, the output that the model generates based on the patterns determined from the training data.

Confidence, confidence, a measure of how likely the output of a model is correct.

Bias, bias, error in a machine learning model that causes the consistent output of certain types of outcomes.

Outlier, outlier, a data point that is significantly different from the rest of the data.

Look out for these keywords in today's lesson.

Today's lesson is split into three parts.

We'll start by recognising that models use patterns to make predictions.

We'll then test a model's outputs are expected.

And then we'll finish up by identifying some characteristics of poor-quality data.

Let's make a start by recognising models use patterns to make predictions.

There are lots of different types of machine learning models.

An AI classifier is a machine learning algorithm that organises data into groups.

It then labels each group as a class.

A classifier learns by finding patterns in its training data.

The software analyses patterns between samples, individual pieces of data, and then groups them into classes.

So you can see here, we have two classes represented by two different colours, Class A, which is blue, and Class B, which is red.

And you can see, the classes have been grouped together.

By recognising patterns within each class and between classes, the classifier can build a model and predict which class new data belongs to by using boundaries or rules that separate them.

Time to check your understanding.

True or false? A classifier analyses patterns between samples.

Pause the video whilst you have a think.

Did you select true? Well done.

Samples with similar patterns can be grouped into a class.

True or false? A classifier analyses patterns between classes.

Pause the video whilst you have a think.

Did you select true? Well done.

The classifier is created by analysing the patterns of data in each class.

Once the classifier has been created, new items of data are analysed by the classifier for the classifier to make a prediction of which class the new data belongs to.

Okay, we're moving on to our first task of today's lesson, and you've done a fantastic job so far, so well done.

I'd like you to describe how new data is classified.

You could use these keywords, boundaries, predict, and class.

Pause the video whilst you answer the question.

How did you get on? Let's have a look at a sample answer together.

A classifier uses boundaries in the model to predict which class new data should belong to.

Remember, if you want to revisit your answer and add any extra detail, you can pause the video and do that now.

Okay, we're moving on to the second part of today's lesson, where we're going to test a model's outputs are expected.

Okay, for this part of the lesson, we're going to need to go to oak.

link/createai.

Here, we're going to pick up where we left off, so you can either open your last session or you can continue the session you saved last lesson.

Note, if you do not have an existing session, then you can use the data sample provided as an additional resource.

To train your model, click the blue "Train model" button.

Test your model works as expected.

Note, you need to make sure that you have your micro:bit connected here.

Perform one of your movements well and see which class your model predicts, or estimates, the action matches.

Does it think it's a good movement? Perform one of your movements badly and see which class your model predicts, or estimates, the action matches.

You will see the confidence, or certainty, scores for your actions.

So, I've recorded 10 samples for both my good and bad class.

I'm then going to click on the blue button which says "Train model." I then click the button that says "Start training" and it will start training my model.

Once my model has trained, it'll take me to a slightly different screen.

You can see, I still have the classes of Good and Bad.

This time, they've been labelled in different colours to help me recognise them.

Now I can start practising my movements and see the estimated action.

The estimated action is down in the bottom right-hand corner.

So at the moment, it's unknown.

Let's start moving the micro:bit and see what it says.

Okay, so there's a good shot, another good shot, another good shot.

This time, if I move it around a bit, okay, getting a bit of unknown.

Still good.

Oh, there's a bad.

Getting some bad shots there.

So it's making some predictions, but sometimes these predictions aren't consistently good.

Ask someone else to test your model.

Did they get the same or different confidence scores? This time, I'm going to ask somebody else to test my model.

So I'm going to ask my daughter, who's much smaller than me, and see if the estimated actions are as good.

Let's try it now.

Okay, so we're still getting some good movements here.

A few bad, and we're not following the serve pattern, but a few mistaken good ones as well.

So, my model is working fairly well for a different person, but perhaps it could be refined a bit.

Why is it important to test your model with other people? Maybe pause the video here whilst you have a think.

Izzy says, "Not everyone moves the same way.

A good movement for one person may be a bad movement for someone else." That's a really good point, Izzy.

Lucas says, "Having diverse data samples reduces the risk of introducing data bias into a model." That's excellent, Lucas, and well done for using the keyword bias in your answer.

Lucas has a question, though, "Why is it important to reduce data bias?" Jacob says, "Data bias can lead to incorrect conclusions or unfair decisions." Time to check your understanding.

What did the system use as training data? A, labelled samples of movements, B, class names, or C, the total time for all samples to play back.

Pause the video whilst you have a think.

Did you select A? Well done.

The system used labelled samples of movements as training data.

How did you test that the model worked as expected? A, checked all the values matched the training data, B, saw the expected movement on the graph, or C, checked the confidence of the prediction.

Pause the video whilst you have a think.

Did you select C? Well done.

To test that the model worked as expected, you checked the confidence of the prediction.

Okay, we're moving on to our second task of today's lesson.

I'd like you to explain why the model's prediction may be inaccurate.

Pause the video whilst you have a think.

How did you get on? Great work.

Let's have a look at a sample answer together.

The input may not match the patterns found in the training data.

Okay, we're moving on to the final part of today's lesson, where we're going to identify characteristics of poor-quality data.

You have 10 samples.

Adding more training data can help the system build a more accurate model as more data is available for comparison to find patterns.

Look for some outliers in your training data.

Lucas says, "Some of my samples look different to others." Do you have some samples that look different to others in your data? Andeep says, "An outlier is a sample that significantly differs from the other samples.

Did you start an action late? Was one action not as accurate as the others?" What happens when there is no movement? Izzy says, "My model still predicts, but the prediction is uncertain." Jacob says, "My model is still certain that my movement fits a class." The model classifies every input as one of the existing classes, even if the data doesn't match either class well.

Many models output probabilities that must sum to 100%.

The highest probability class is chosen, even if it's only slightly better than the alternative.

Time to check your understanding.

Which of these can lead to poor-quality predictions? A, samples all of the same length, B, lots of samples, or C, limited number of classes.

Pause the video whilst you have a think.

Did you select C? Well done.

A limited number of classes can lead to poor-quality predictions.

Which one of these is a characteristic of poor-quality data? A, predictions change with new data, B, some samples are outliers, or C, confidence scores are high.

Pause the video whilst you have a think.

Did you select B? Great work.

A characteristic of poor-quality data is that some samples are outliers.

Okay, we're moving on to our final set of tasks for today's lesson, and you've done a fantastic job, so well done.

I'd like you to explain why the model may make a prediction when no movement is detected.

Pause the video whilst you have a think.

How did you get on? Let's have a look at a sample answer together.

The model can only choose from the available classes, and at the moment, we only have good and bad movements.

Therefore, the system predicts the closest match, even if it is a poor fit.

Okay, we've come to the end of today's lesson, and you've done a great job, so well done.

Let's summarise what we've learned.

A classifier's predictions are only as accurate as the training data allows.

Classifiers can still make predictions with a high confidence score even if the training data is poor.

Causes of poor training data include outliers, limited training data, and poorly labelled data.

It's important to test a classifier to make sure the outputs are expected.

I hope you've enjoyed today's lesson, and I hope you'll join me again soon, bye.