Multi-Class Deep Learning Model for CIFAR-10 Object Recognition Using Keras Take 2

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The CIFAR-10 dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.

INTRODUCTION: The CIFAR-10 is a labeled subset of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The CIFAR-10 dataset consists of 60,000 32×32 color images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images.

The dataset is divided into five training batches and one test batch, each with 10,000 images. The test batch contains exactly 1,000 randomly selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5,000 images from each class.

In iteration Take1, we constructed a simple VGG convolutional model with 1 VGG block to classify the images. This model serves as the baseline for the future iterations of modeling.

For this iteration, we will construct a few more VGG convolutional models with 2 and 3 VGG blocks to classify the images. The additional models will enable us to choose a final baseline model before applying other performance-enhancing techniques.

ANALYSIS: In iteration Take1, the performance of the Take1 model with the default parameters achieved an accuracy score of 66.39% on the validation dataset after training for 50 epochs. After tuning the hyperparameters, the Take1 model with the best hyperparameters processed the training dataset with an accuracy of 100.00%. The same model, however, processed the test dataset with an accuracy of only 67.01%. We will need to explore other modeling approaches to make a better model that reduces over-fitting.

In this iteration, the performance of the VGG-1 model with the default parameters achieved an accuracy score of 66.69% on the validation dataset after training for 50 epochs. The VGG-2 model achieved an accuracy score of 71.35% on the validation dataset after training for 50 epochs. The VGG-3 model achieved an accuracy score of 73.81% on the validation dataset after training for 50 epochs. The additional VGG blocks helped the model, but we still need to explore other modeling approaches to make a better model that reduces over-fitting.

CONCLUSION: For this dataset, the model built using Keras and TensorFlow achieved a satisfactory result and should be considered for future modeling activities.

Dataset Used: The CIFAR-10 Dataset

Dataset ML Model: Multi-class classification with numerical attributes

Dataset Reference: https://www.cs.toronto.edu/~kriz/cifar.html

One potential source of performance benchmarks: https://machinelearningmastery.com/how-to-develop-a-cnn-from-scratch-for-cifar-10-photo-classification/

The HTML formatted report can be found here on GitHub.

Multi-Class Deep Learning Model for CIFAR-10 Object Recognition Using Keras Take 1

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The CIFAR-10 dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.

INTRODUCTION: The CIFAR-10 is a labeled subset of the 80 million tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The CIFAR-10 dataset consists of 60,000 32×32 color images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images.

The dataset is divided into five training batches and one test batch, each with 10,000 images. The test batch contains exactly 1,000 randomly selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5,000 images from each class.

For this iteration, we will construct a simple VGG convolutional model with 1 VGG block to classify the images. This model will serve as the baseline for the future iterations of modeling.

ANALYSIS: In this iteration, the performance of the Take1 model with the default parameters achieved an accuracy score of 66.39% on the validation dataset after training for 50 epochs. After tuning the hyperparameters, the Take1 model with the best hyperparameters processed the training dataset with an accuracy of 100.00%. The same model, however, processed the test dataset with an accuracy of only 67.01%. We will need to explore other modeling approaches to make a better model that reduces over-fitting.

CONCLUSION: For this dataset, the model built using Keras and TensorFlow achieved a satisfactory result and should be considered for future modeling activities.

Dataset Used: The CIFAR-10 Dataset

Dataset ML Model: Multi-class classification with numerical attributes

Dataset Reference: https://www.cs.toronto.edu/~kriz/cifar.html

One potential source of performance benchmarks: https://machinelearningmastery.com/how-to-develop-a-cnn-from-scratch-for-cifar-10-photo-classification/

The HTML formatted report can be found here on GitHub.

Bob Lewis on IT Projects, Part 7

In the book, There’s No Such Thing as an IT Project: A Handbook for Intentional Business Change, Bob Lewis and co-author Dave Kaiser analyzed and discussed the new ways of thinking about IT and business management.

These are some of my takeaways from reading the book.

In the “The Seven Change Disciplines” chapter, Bob and Dave discuss the seven disciplines organizations need to master to make the intentional change both real and sustained.

Leadership: Leadership is the art of getting others to follow the leader’s vision.

Business Design: Business design is about creating a concrete answer that can realize the leader’s vision. The design should be complete enough to articulate the vision in the terms where everyone involved can understand both the change itself and their role in it.

Technical Architecture Management: Technical architecture establishes the design and engineering guidelines needed so that the collection of applications, data repositories, and underlying infrastructure assemble logically and efficiently. The goal of technical architecture is to support the organization’s processes and practices.

Application Development / Application Integration and Configuration: The key point to remember is this. When it comes to achieving intentional business changes, the goal of IT is to avoid being the bottleneck.

Organizational Change Management: It would be wrong to assume people naturally resist change – people naturally resist change they expect will be bad for them. As a leader of the organization and to the extent possible, the leader should design every business change, so it leaves employees better off than they were before the change happened.

Implementation Logistics: Start every business change implementation with a well-chosen pilot. The pilot affects relatively few people but otherwise includes most of the complexity of the actual rollout.

Project Management: Projects are how change happens, so a solid project management discipline must be part of any effort that manages an intentional business change.

To make it all work, Bob and Dave believed that mastery in isolation is not enough. The seven disciplines must come together as an integrated whole, whose practitioners actively collaborate to make intentional change happen.

So, what can be done to address “The Seven Change Disciplines” opportunities and challenges? Fortunately, Bob and Dave have some solid suggestions laid out at the end of Chapter Seven. I highly recommend the book.

落敗的風格

(從我一個尊敬的作家,賽斯·高汀

這數學不得不令人信服。您將會輸掉大多數參加的比賽。這怎麼可能是其他方式呢?在一百,一千或十億人競爭的情況下,只有一個人獲勝。

這意味著您將根據落敗的方式而不是獲勝的方式來觀察和衡量您自己。

大部分人所知取勝的方法通常是完全配合,為法官提供他們想要的東西,像其他所有人一樣訓練,但要加倍努力。

但是落敗的風格是去創造可能性。去更有創新力。去做慷慨的工作來值得他人一提。

如果您將要落敗(很可能會落敗),那為什麼不落敗的有風格些?

Web Scraping of O’Reilly Software Architecture Conference 2019 Berlin Using Python

SUMMARY: The purpose of this project is to practice web scraping by extracting specific pieces of information from a website. The web scraping Python code leverages the BeautifulSoup module.

INTRODUCTION: The Software Architecture Conference covers the full range of topics in the software architecture discipline. Those topics include leadership and business skills, product management, and domain-driven design. This web scraping script will automatically traverse through the entire web page and collect all links to the PDF and PPTX documents. The script will also download the documents as part of the scraping process. The Python script ran in the Google Colaboratory environment and can be adapted to run in any Python environment without the Colab-specific configuration.

Starting URLs: https://conferences.oreilly.com/software-architecture/sa-eu-2019/public/schedule/proceedings

The source code and HTML output can be found here on GitHub.

Multi-Class Deep Learning Model for MNIST Handwritten Digits Using Keras Take 2

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The MNIST dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.

INTRODUCTION: The MNIST problem is a dataset developed by Yann LeCun, Corinna Cortes, and Christopher Burges for evaluating machine learning models on the handwritten digit classification problem. The dataset was constructed from many scanned document datasets available from the National Institute of Standards and Technology (NIST). The MNIST handwritten digit classification problem has become a standard dataset used in computer vision and deep learning.

Images of digits were taken from a variety of scanned documents, normalized in size and centered. Each image is a 28 by 28-pixel square (784 pixels total). A standard split of the dataset is used to evaluate and compare models, where 60,000 images are used to train a model, and a separate set of 10,000 images are used to test it. It is a digit recognition task, so there are ten classes (0 to 9) to predict.

For the Take1 iteration, we constructed a simple dense-layer model to classify the images. This model serves as the baseline for future iterations.

For this iteration, we will convert the dense-layer model into a convolutional model to examine whether we can improve its performance.

ANALYSIS: For the Take1 iteration, the performance of the Take1 model with the default parameters achieved an accuracy score of 98.05% on the validation dataset after training for five epochs. After tuning the hyperparameters, the Take1 model processed the training dataset with an accuracy rate of 98.82%. The same model processed the test dataset with an accuracy of 98.17%. If we were to train the model much further than three epochs, we could run into the over-fitting issue.

For this iteration, the performance of the Take2 model with the default parameters achieved an accuracy score of 98.77% on the validation dataset after training for five epochs. After tuning the hyperparameters, the Take2 model processed the training dataset with an accuracy rate of 99.46%. The same model processed the test dataset with an accuracy of 98.61%.

Adding the convolutional layers improved the model’s prediction performance slightly. The increasing difference between training and validation accuracy after each epoch indicated that we potentially have a high variance problem. We will need more data to train the model before we can bring up the prediction accuracy of the validation dataset.

CONCLUSION: For this dataset, the model built using Keras and TensorFlow achieved a satisfactory result and should be considered for future modeling activities.

Dataset Used: MNIST Handwritten Digits Dataset

Dataset ML Model: Multi-class classification with numerical attributes

Dataset Reference: https://www.tensorflow.org/datasets/catalog/mnist

One potential source of performance benchmarks: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/

The HTML formatted report can be found here on GitHub.

Time Series Model for Monthly Rainfall Coppermine Using Python

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a time series prediction model and document the end-to-end steps using a template. The Monthly Rain Coppermine dataset is a time series situation where we are trying to forecast future outcomes based on past data points.

INTRODUCTION: The problem is to forecast the monthly rainfall. The dataset describes a time-series of rainfall (in millimeters) over 44 years (1933-1976), and there are 528 observations. We used the first 80% of the observations for training and testing various models while holding back the remaining observations for validating the final model.

ANALYSIS: The baseline prediction (or persistence) for the dataset resulted in an RMSE of 17.17. After performing a grid search for the most optimal ARIMA parameters, the final ARIMA non-seasonal order was (0, 0, 2) with the seasonal order being (1, 0, 1, 12). Furthermore, the chosen model processed the validation data with an RMSE of 10.73, which was better than the baseline model as expected.

CONCLUSION: For this dataset, the chosen ARIMA model achieved a satisfactory result and should be considered for further modeling.

Dataset Used: Monthly Rain Coppermine 1933 through 1976

Dataset ML Model: Time series forecast with numerical attributes

Dataset Reference: Rob Hyndman and Yangzhuoran Yang (2018). tsdl: Time Series Data Library. v0.1.0. https://pkg.yangzhuoranyang./tsdl/.

The HTML formatted report can be found here on GitHub.

Multi-Class Deep Learning Model for MNIST Handwritten Digits Using Keras Take 1

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The MNIST dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.

INTRODUCTION: The MNIST problem is a dataset developed by Yann LeCun, Corinna Cortes, and Christopher Burges for evaluating machine learning models on the handwritten digit classification problems. The dataset was constructed from many scanned document datasets available from the National Institute of Standards and Technology (NIST). The MNIST handwritten digit classification problem has become a standard dataset used in computer vision and deep learning.

Images of digits were taken from a variety of scanned documents, normalized in size and centered. Each image is a 28 by 28-pixel square (784 pixels total). A standard split of the dataset is used to evaluate and compare models, where 60,000 images are used to train a model, and a separate set of 10,000 images are used to test it. It is a digit recognition task, so there are ten classes (0 to 9) to predict.

For this iteration, we will construct a simple dense deep-learning model to classify the images. This model will serve as the baseline for future iterations.

ANALYSIS: For this iteration, the performance of the Take1 model with the default parameters achieved an accuracy score of 98.05% on the validation dataset after training for ten epochs. After tuning the hyperparameters, the Take1 model processed the training dataset with an accuracy rate of 98.82%. The same model processed the test dataset with an accuracy of 98.17%. If we were to train the model much further than three epochs, we could run into the over-fitting issue.

CONCLUSION: For this dataset, the model built using Keras and TensorFlow achieved a satisfactory result and should be considered for future modeling activities.

Dataset Used: MNIST Handwritten Digits Dataset

Dataset ML Model: Multi-class classification with numerical attributes

Dataset Reference: https://www.tensorflow.org/datasets/catalog/mnist

One potential source of performance benchmarks: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-from-scratch-for-mnist-handwritten-digit-classification/

The HTML formatted report can be found here on GitHub.