December 2020 – Merely Do It!

Multi-Class Image Classification Model for Kannada Handwritten Digits Using TensorFlow Take 3

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a predictive model using a TensorFlow convolutional neural network (CNN) and document the end-to-end steps using a template. The Kannada Handwritten Digits dataset is a multi-class classification situation where we attempt to predict one of several (more than two) possible outcomes.

INTRODUCTION: Kannada is a language spoken predominantly by the people of Karnataka in southwestern India. The language has roughly 45 million native speakers and is written using the Kannada script. This modeling project uses the same format as the MNIST dataset regarding how the data is structured. All details of the dataset curation have been captured in the paper titled: Prabhu, Vinay Uday. “Kannada-MNIST: A new handwritten digits dataset for the Kannada language.” arXiv preprint arXiv:1908.01242 (2019)

From iteration Take1, we constructed a simple three-layer CNN neural network to use as the baseline model for analyzing this dataset.

From iteration Take2, we constructed a four-layer CNN neural network to use as the baseline model for analyzing this dataset.

In this Take3 iteration, we will construct a five-layer CNN neural network to use as the baseline model for analyzing this dataset.

ANALYSIS: From iteration Take1, the baseline model’s performance achieved an average accuracy score of 99.48% after ten epochs using the training dataset. After trying out different hyperparameters sets, the best model processed Kaggle’s public test dataset with an accuracy score of 97.90%. Furthermore, the final model processed Kaggle’s unseen test dataset with an accuracy measurement of 97.74%.

From iteration Take2, the four-layer model’s performance achieved an average accuracy score of 99.49% after ten epochs using the training dataset. After trying out different hyperparameters sets, the best model processed Kaggle’s public test dataset with an accuracy score of 98.24%. Furthermore, the final model processed Kaggle’s unseen test dataset with an accuracy measurement of 97.80%.

In this Take3 iteration, the five-layer model’s performance achieved an average accuracy score of 99.52% after ten epochs using the training dataset. After trying out different hyperparameters sets, the best model processed Kaggle’s public test dataset with an accuracy score of 98.08%. Furthermore, the final model processed Kaggle’s unseen test dataset with an accuracy measurement of 97.74%.

CONCLUSION: In this iteration, the TensorFlow CNN model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

Dataset Used: Kannada Handwritten Digits Dataset

Dataset ML Model: Multi-class image classification with numerical attributes

Dataset Reference: https://www.kaggle.com/c/Kannada-MNIST/overview

One potential source of performance benchmarks: https://www.kaggle.com/c/Kannada-MNIST/leaderboard

The HTML formatted report can be found here on GitHub.

Algorithmic Trading Model for Simple Trend-Following Strategy Using Python Take 1

NOTE: This script is for learning purposes only and does not constitute a recommendation for buying or selling any stock mentioned in this script.

SUMMARY: This project aims to construct and test an algorithmic trading model and document the end-to-end steps using a template.

INTRODUCTION: This algorithmic trading model examines a simple trend-following strategy for a stock. The model buys a stock when the price reaches the highest price for the last X number of days. The model will exit the position when the stock price crosses below the mean of the same window size.

In this Take1 iteration, we will set up the models using one fixed window size for each model. The window size will vary from 10 to 50 trading days at a 5-day increment.

ANALYSIS: From this iteration, we analyzed the stock prices for Apple Inc. (AAPL) between January 1, 2019, and December 24, 2020. The trading model produced a profit of 81.49 dollars per share. The buy-and-hold approach yielded a gain of 92.60 dollars per share.

CONCLUSION: For the stock of AAPL during the modeling time frame, the trading strategy did not produce a better return than the buy-and-hold approach. We should consider modeling this stock further by experimenting with more variations of the strategy.

Dataset ML Model: Time series analysis with numerical attributes

Dataset Used: Quandl

The HTML formatted report can be found here on GitHub.

Multi-Class Image Classification Model for Kannada Handwritten Digits Using TensorFlow Take 2

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

From iteration Take1, we constructed a simple three-layer CNN neural network to use as the baseline model for analyzing this dataset.

In this Take2 iteration, we will construct a four-layer CNN neural network to use as the baseline model for analyzing this dataset.

In this Take2 iteration, the four-layer model’s performance achieved an average accuracy score of 99.49% after ten epochs using the training dataset. After trying out different hyperparameters sets, the best model processed Kaggle’s public test dataset with an accuracy score of 98.24%. Furthermore, the final model processed Kaggle’s unseen test dataset with an accuracy measurement of 97.80%.

CONCLUSION: In this iteration, the TensorFlow CNN model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

Dataset Used: Kannada Handwritten Digits Dataset

Dataset ML Model: Multi-class image classification with numerical attributes

Dataset Reference: https://www.kaggle.com/c/Kannada-MNIST/overview

One potential source of performance benchmarks: https://www.kaggle.com/c/Kannada-MNIST/leaderboard

The HTML formatted report can be found here on GitHub.

Multi-Class Image Classification Model for Kannada Handwritten Digits Using TensorFlow Take 1

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

In this Take1 iteration, we will construct a simple three-layer CNN neural network to use as the baseline model for analyzing this dataset.

ANALYSIS: In this Take1 iteration, the baseline model’s performance achieved an average accuracy score of 99.48% after ten epochs using the training dataset. After trying out different hyperparameters sets, the best model processed Kaggle’s public test dataset with an accuracy score of 97.90%. Furthermore, the final model processed Kaggle’s unseen test dataset with an accuracy measurement of 97.74%.

CONCLUSION: In this iteration, the TensorFlow CNN model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

Dataset Used: Kannada Handwritten Digits Dataset

Dataset ML Model: Multi-class image classification with numerical attributes

Dataset Reference: https://www.kaggle.com/c/Kannada-MNIST/overview

One potential source of performance benchmarks: https://www.kaggle.com/c/Kannada-MNIST/leaderboard

The HTML formatted report can be found here on GitHub.

Annie Duke on Thinking in Bets, Part 9

In her book, Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts, Annie Duke draws on examples from business, sports, politics, and poker to share tools anyone can use to embrace uncertainty and make better decisions.

In the chapter, “Adventures in Mental Time Travel,” Annie Duke discusses the technique of recruiting out past-self and future-self to help our present-self in the decision-making process. These are some of my favorite concepts and takeaways from reading the book.

“Reconnaissance: mapping the future”

For us to make better decisions, we need to perform reconnaissance on the future. If a decision is a bet on a particular future based on our beliefs, we should consider what those possible futures might look like before we place a bet.

Scenario planning is a prominent example of scouting out the future. We should always be asking ourselves the questions, “What are the possible futures? What are the probabilities of those possible futures?”

By doing the hard work of reconnaissance, we can position ourselves better with the following benefits. First, we remind ourselves that the future is uncertain and to have a realistic view of the world. Second, we will be better prepared for how we are going to respond to different outcomes. Reconnaissance can make us more agile.

Third, anticipating the range of outcomes can keep us from unproductive acts of regret when a particular future happens. Finally, we are less likely to fall prey to resulting or hindsight bias by mapping out the potential futures and possibilities.

“Backcasting: working backward from a positive future”

There are two ways of advance thinking: standing at the beginning and looking forward or standing at the end and looking backward. The second approach is much more effective.

Forecasting the future from where we stand can be daunting. From the vantage point of the present, it is hard to see past the next step. We get a sense that the present the immediate future loom large, with anything beyond that fuzzy and losing focus.

When we identify the goal and work backward from the journey’s end to “remember” how we got there, research has shown that we do better. Imaging a successful future and backcasting is a useful approach for identifying necessary steps for reaching our goals.

“Premortems: working backward from a negative future”

Like backcasting, working backward from the end can also help us to imagine an unfavorable future. Backcasting and premortem complement each other as we must always account for both positive and negative outcomes.

Despite the popular wisdom that we achieve success through positive visualization, it turns out that incorporating negative visualization makes us more likely to achieve our goals. A premortem is a healthy form of organized skepticism. The approach gives us permission and freedom for dissent. Think of premortem as acting as our own red team.

Imagine both positive and negative futures allow us to plan and prepare for a wider variety of challenges. Both the likelihood of positive and negative futures must add up to 100% since the positive space of backcasting and the negative space of premortem must still fit into one finite continuum.

Once we decide and one of those possible futures happens, we cannot discard all that work, even if it included work on ends that did not occur. Forgetting about an unrealized future can be dangerous to good decision-making.

你在拖延嗎？

（從我一個尊敬的作家，賽斯·高汀）

我有片木牌上有幾個字。

答案通常是“是的”。

如果您在等待一項不可避免的延遲，那麼您就不是在拖延。如果您在客戶會注意到的方式上使事情變得更好，那麼您就不是在拖延。如果您發現現在的空間給您帶來歡樂並維持著您，那麼您就不是在拖延。

但是…

如果您退縮並尋找原因，然後將該原因替換為另一個原因，那麼您可能就是在拖延。

NLP Model for Movie Review Sentiment Analysis Using TensorFlow Take 5

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a text classification model using a neural network and document the end-to-end steps using a template. The Movie Review Sentiment Analysis dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

Additional Notes: This script is a replication, with some small modifications, of Dr. Jason Brownlee’s blog post, Deep Convolutional Neural Network for Sentiment Analysis. I plan to leverage Dr. Brownlee’s tutorial and build a TensorFlow-based text classification notebook template for future modeling of similar datasets.

INTRODUCTION: The Movie Review Data is a collection of movie reviews retrieved from the imdb.com website in the early 2000s by Bo Pang and Lillian Lee. The reviews were collected and made available as part of their research on natural language processing. The dataset comprises 1,000 positive and 1,000 negative movie reviews drawn from an archive of the rec.arts.movies.reviews newsgroup hosted at IMDB. The authors refer to this dataset as the ‘polarity dataset.’

We will use the last 100 positive reviews and the last 100 negative reviews as a test set (200 reviews) and the remaining 1,800 reviews as the training dataset. This is a 90% train, 10% split of the data.

In iteration Take1, we constructed the necessary code modules to handle the tasks of loading text, cleaning text, and vocabulary development.

In iteration Take2, we constructed a bag-of-words model and analyzed it with a simple multi-layer perceptron network.

In iteration Take3, we fine-tuned the model using different Tokenizer modes in the Keras API.

In iteration Take4, we will fine-tune the model using word embedding to fit a deep learning model.

In this Take5 iteration, we will fine-tune the model using the Word2Vec algorithm for developing a standalone word embedding mechanism.

ANALYSIS: From iteration Take2, the baseline model’s performance achieved an average accuracy score of 85.81% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 91.50%.

From iteration Take3, the best model’s performance (using the frequency Tokenizer mode) achieved an average accuracy score of 86.06% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 86.50%.

From iteration Take4, the best model’s performance (using the word embedding with Keras) achieved an average accuracy score of 84.17% after ten epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 88.50%.

In this Take5 iteration, the best model’s performance (using the Word2Vec word embedding algorithm) achieved an average accuracy score of 52.61% after ten epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 49.00%.

CONCLUSION: In this iteration, the Word2Vec word embedding algorithm appeared to be unsuitable for modeling this dataset. We should consider experimenting with other word embedding techniques for further modeling.

Dataset Used: Movie Review Sentiment Analysis Dataset

Dataset ML Model: Binary class text classification with text-oriented features

Dataset Reference: https://www.cs.cornell.edu/home/llee/papers/cutsent.pdf and http://www.cs.cornell.edu/people/pabo/movie-review-data/review_polarity.tar.gz

One potential source of performance benchmarks: https://machinelearningmastery.com/deep-learning-bag-of-words-model-sentiment-analysis/

The HTML formatted report can be found here on GitHub.

Binary Image Classification Deep Learning Model for Casting Quality Inspection Using TensorFlow Take 5

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a predictive model using a TensorFlow convolutional neural network (CNN) and document the end-to-end steps using a template. The Casting Quality Inspection dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

INTRODUCTION: This dataset is about the casting process during the manufacturing of a product. A casting defect is an undesired irregularity in a metal casting process, and the significant problem is that most organizations carry out the inspection process manually. It is a very time-consuming process and prone to human errors. These photos are a top view of the submersible pump impeller, and the dataset contains 7348 image files. All images are the size of 300×300 pixels grey-scaled images with image augmentation.

From iteration Take1, we trained a few three-layer CNN models and evaluated them using 715 300×300 pixels test images.

From iteration Take2, we trained a few three-layer CNN models and evaluated them using 1300 512×512 pixels test images.

From iteration Take3, we trained a VGG-16 CNN model and evaluated it using the 1300 512×512 pixels test images.

From iteration Take4, we trained an InceptionV3 CNN model and evaluated it using the 1300 512×512 pixels test images.

In this Take5 iteration, we will train an ResNet50V2 CNN model and evaluate it using the 1300 512×512 pixels test images.

ANALYSIS: From iteration Take1, the baseline model’s performance achieved an accuracy score of 99.72% after 15 epochs using the training dataset. After tuning the hyperparameters, the best model processed the validation/test dataset with an accuracy score of 99.86%.

From iteration Take2, the baseline model’s performance achieved an accuracy score of 72.62% after 15 epochs using the training dataset. After tuning the hyperparameters, the best model processed the validation/test dataset with an accuracy score of 72.62%.

From iteration Take3, the VGG16 model’s performance achieved an accuracy score of 100% after 20 epochs using the training dataset. The same model processed the validation/test dataset with an accuracy score of 73.92%.

From iteration Take4, the InceptionV3 model’s performance achieved an accuracy score of 99.80% after 20 epochs using the training dataset. The same model processed the validation/test dataset with an accuracy score of 79.84%.

In this Take5 iteration, the ResNet50V2 model’s performance achieved an accuracy score of 99.91% after 20 epochs using the training dataset. The same model processed the validation/test dataset with an accuracy score of 76.23%.

CONCLUSION: In this iteration, the ResNet50V2 CNN model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

CONCLUSION: In this iteration, the VGG16 CNN model appeared to be suitable for modeling this dataset. We should consider experimenting with TensorFlow for further modeling.

Dataset Used: Casting Quality Inspection Dataset

Dataset ML Model: Binary image classification with numerical attributes

Dataset Reference: https://www.kaggle.com/ravirajsinh45/real-life-industrial-dataset-of-casting-product

One potential source of performance benchmarks: https://www.kaggle.com/ravirajsinh45/real-life-industrial-dataset-of-casting-product/notebooks

The HTML formatted report can be found here on GitHub.