Annie Duke on How to Decide, Part 5

In her book, How to Decide: Simple Tools for Making Better Choices, Annie Duke discusses how to train our brains to combat our own bias and help ourselves make more confident and better decisions.

These are some of my favorite concepts and takeaways from reading the book.

Chapter 5 “Taking Dead Aim at the Future: The Power of Precision”

In this chapter, Annie Duke discusses how imprecise we can be when we try to factor probability into decision-making. She offers the following recommendations:

  • We should avoid relying too much on natural language terms that express likelihoods, such as “very likely” and “unlikely.” They can be useful occasionally, but they are still blunt instruments for analyzing and communicating probability.
  • To improve our decision-making, we need to find ways to improve our estimates continually. If we hide behind the safety of a general, vague term for probability, there is very little motivation to check our information and learn more.
  • General terms that express likelihood also mean very different things to different people. Using those ambiguous terms can lead to confusion and miscommunication with people we want to engage for help.
  • What we need to do is to be more precise by expressing probabilities as percentages. Using specific numbers can help us uncover information that can correct our beliefs’ inaccuracies and broaden our knowledge.
  • In addition to making a precise (bull’s-eye) estimate, we should also include a range, with a lower and upper bound, around that estimate to express our uncertainty. This notion of using range is similar to the concept of the confidence interval in statistics.
  • Annie also suggests using the “shock test” to gauge whether our upper and lower bounds are reasonable. In other words, would we be shocked if the correct answers ended up outside that range? Our goal should be to have approximately 90% of our estimates capture the objectively real value.

幸好

(從我一個尊敬的作家,賽斯·高汀

在所有這些混亂之中,有些人還仍然能夠相信。 去信任他人,去信任可能性並且去信任自己。 況且…

我們周圍都是機會。

我們經常還有第二次機會。

雖然還是有待解決的問題。

我們已從中學到了一些東西。

前面還有一個新的進展,是指日可待。

總是有機會讓周圍變得更好。

NLP Model for Sentiment Labelled Sentences Using TensorFlow Take 7

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a text classification model using a neural network and document the end-to-end steps using a template. The Sentiment Labelled Sentences dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

INTRODUCTION: This dataset was created for the research paper ‘From Group to Individual Labels using Deep Features,’ Kotzias et al., KDD 2015. The paper researchers randomly selected 500 positive and 500 negative sentences from a larger dataset of reviews for each website. The researcher also attempted to choose sentences with a positive or negative connotation as the goal was to avoid selecting neutral sentences.

From iteration Take1, we deployed a bag-of-words model to classify the Amazon dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

From iteration Take2, we deployed a word-embedding model to classify the Amazon dataset’s review comments. We also compared the result with the bag-of-word model from the previous iteration.

From iteration Take3, we deployed a bag-of-words model to classify the IMDB dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

From iteration Take4, we deployed a word-embedding model to classify the IMDB dataset’s review comments. We also compared the result with the bag-of-word model from the previous iteration.

From iteration Take5, we deployed a bag-of-words model to classify the Yelp dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

From iteration Take6, we deployed a word-embedding model to classify the Yelp dataset’s review comments. We also compared the result with the bag-of-word model from the previous iteration.

In this Take7 iteration, we will deploy a bag-of-words model to classify the combined dataset from all three sets of review comments. We will also compare the result with the bag-of-word model from the previous iteration.

ANALYSIS: From iteration Take1, the bag-of-words model’s performance achieved an average accuracy score of 77.31% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 71.00%.

From iteration Take2, the word-embedding model’s performance achieved an average accuracy score of 73.25% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 67.00%.

From iteration Take3, the bag-of-words model’s performance achieved an average accuracy score of 77.26% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 68.66%.

From iteration Take4, the word-embedding model’s performance achieved an average accuracy score of 72.84% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 66.00%.

From iteration Take5, the bag-of-words model’s performance achieved an average accuracy score of 75.19% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 72.00%.

From iteration Take6, the word-embedding model’s performance achieved an average accuracy score of 73.44% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 72.50%.

In this Take7 iteration, the bag-of-words model’s performance achieved an average accuracy score of 76.84% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 78.72%.

CONCLUSION: In this modeling iteration, the word-embedding TensorFlow model did as well as the bag-of-words model. Furthermore, we should continue to experiment with both natural language processing techniques for further modeling.

Dataset Used: Sentiment Labelled Sentences

Dataset ML Model: Binary class text classification with text-oriented features

Dataset Reference: https://archive.ics.uci.edu/ml/datasets/Sentiment+Labelled+Sentences

The HTML formatted report can be found here on GitHub.

Binary Image Classification Deep Learning Model for Chest X-Ray Pneumonia Images Using TensorFlow Take 2

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a predictive model using a TensorFlow convolutional neural network (CNN) and document the end-to-end steps using a template. The Chest X-Ray Pneumonia dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

INTRODUCTION: The dataset contains chest X-ray images selected from retrospective cohorts of pediatric patients of one to five years old from Guangzhou Women and Children’s Medical Center, Guangzhou. The image collection is organized into three folders (train, test, val) and contains subfolders for each image category (Pneumonia/Normal). There are 5,863 X-Ray images with various display resolutions in this collection.

From iteration Take1, we trained a simple three-layer CNN model and used the model’s performance as the baseline.

In this Take2 iteration, we will use the same three-layer CNN model from Take1 and apply it to the same photos with higher resolutions (from 640×480 to 1024×768).

ANALYSIS: From iteration Take1, the baseline model’s performance achieved an accuracy score of 100% after 20 epochs using the validation dataset. However, the final model processed the test dataset with an accuracy measurement of only 72.91%.

In this Take2 iteration, the model’s performance achieved an accuracy score of 62.50% after ten epochs using the validation dataset. However, the final model processed the test dataset with an encouraging accuracy measurement of 83.33%.

CONCLUSION: In this iteration, the TensorFlow CNN model appeared to be suitable for modeling this dataset, but we need to experiment with the TensorFlow model to improve its accuracy.

Dataset Used: Chest X-Ray Images (Pneumonia) Dataset

Dataset ML Model: Binary image classification with numerical attributes

Dataset References: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia, https://data.mendeley.com/datasets/rscbjbr9sj/2, and http://www.cell.com/cell/fulltext/S0092-8674(18)30154-5.

One potential source of performance benchmarks: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia

The HTML formatted report can be found here on GitHub.

Algorithmic Trading Model for Trend-Following with Holding Period Strategy Using Python Take 1

NOTE: This script is for learning purposes only and does not constitute a recommendation for buying or selling any stock mentioned in this script.

SUMMARY: This project aims to construct and test an algorithmic trading model and document the end-to-end steps using a template.

INTRODUCTION: This algorithmic trading model examines a simple trend-following strategy for a stock. The model buys a stock when the price reaches the highest price for the last X number of days. The model will exit the position when the stock price crosses below the mean of the same window size.

In addition to the stock price, the models will also use the trading volume indicator to confirm the buy/sell signal further. Finally, the strategy will also incorporate a fixed holding window. The system will exit the position when the holding window reaches the maximum window size.

In this Take1 iteration, we will set up the models using a trend window size for long trades only. The window size varied from 10 to 50 trading days at a 5-day increment. The models will also consider a volume indicator with a varying window size between 10 and 15 days to further confirm the buy/sell signal. We will not limit the holding period by setting the maximum holding period to 99 days for this iteration.

ANALYSIS: In this Take1 iteration, we analyzed the stock prices for Apple Inc. (AAPL) between January 1, 2018, and December 31, 2020. The trading model produced a profit of 95.90 dollars per share. The buy-and-hold approach yielded a gain of 91.54 dollars per share.

CONCLUSION: For the stock of AAPL during the modeling time frame, the trading strategy produced a slightly better return than the buy-and-hold approach. We should consider modeling this stock further by experimenting with more variations of the strategy.

Dataset ML Model: Time series analysis with numerical attributes

Dataset Used: Quandl

The HTML formatted report can be found here on GitHub.

Binary Image Classification Deep Learning Model for Chest X-Ray Pneumonia Images Using TensorFlow Take 1

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a predictive model using a TensorFlow convolutional neural network (CNN) and document the end-to-end steps using a template. The Chest X-Ray Pneumonia dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

INTRODUCTION: The dataset contains chest X-ray images selected from retrospective cohorts of pediatric patients of one to five years old from Guangzhou Women and Children’s Medical Center, Guangzhou. The image collection is organized into three folders (train, test, val) and contains subfolders for each image category (Pneumonia/Normal). There are 5,863 X-Ray images with various display resolutions in this collection.

In this Take1 iteration, we will train a simple three-layer CNN model and use the model’s performance as the baseline.

ANALYSIS: In this Take1 iteration, the baseline model’s performance achieved an accuracy score of 100% after 20 epochs using the validation dataset. However, the final model processed the test dataset with an accuracy measurement of only 72.91%.

CONCLUSION: In this iteration, the TensorFlow CNN model appeared to be suitable for modeling this dataset, but we need to experiment with the TensorFlow model to improve its accuracy.

Dataset Used: Chest X-Ray Images (Pneumonia) Dataset

Dataset ML Model: Binary image classification with numerical attributes

Dataset References: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia, https://data.mendeley.com/datasets/rscbjbr9sj/2, and http://www.cell.com/cell/fulltext/S0092-8674(18)30154-5.

One potential source of performance benchmarks: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia

The HTML formatted report can be found here on GitHub.

NLP Model for Sentiment Labelled Sentences Using TensorFlow Take 6

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery.

SUMMARY: This project aims to construct a text classification model using a neural network and document the end-to-end steps using a template. The Sentiment Labelled Sentences dataset is a binary classification situation where we attempt to predict one of the two possible outcomes.

INTRODUCTION: This dataset was created for the research paper ‘From Group to Individual Labels using Deep Features,’ Kotzias et al., KDD 2015. The paper researchers randomly selected 500 positive and 500 negative sentences from a larger dataset of reviews for each website. The researcher also attempted to choose sentences with a positive or negative connotation as the goal was to avoid selecting neutral sentences.

From iteration Take1, we deployed a bag-of-words model to classify the Amazon dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

From iteration Take2, we deployed a word-embedding model to classify the Amazon dataset’s review comments. We also compared the result with the bag-of-word model from the previous iteration.

From iteration Take3, we deployed a bag-of-words model to classify the IMDB dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

From iteration Take4, we deployed a word-embedding model to classify the IMDB dataset’s review comments. We also compared the result with the bag-of-word model from the previous iteration.

From iteration Take5, we deployed a bag-of-words model to classify the Yelp dataset’s review comments. We also applied various sequence-to-matrix modes to evaluate the model’s performance.

In this Take6 iteration, we will deploy a word-embedding model to classify the Yelp dataset’s review comments. We will also compare the result with the bag-of-word model from the previous iteration.

ANALYSIS: From iteration Take1, the bag-of-words model’s performance achieved an average accuracy score of 77.31% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 71.00%.

From iteration Take2, the word-embedding model’s performance achieved an average accuracy score of 73.25% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 67.00%.

From iteration Take3, the bag-of-words model’s performance achieved an average accuracy score of 77.26% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 68.66%.

From iteration Take4, the word-embedding model’s performance achieved an average accuracy score of 72.84% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 66.00%.

From iteration Take5, the bag-of-words model’s performance achieved an average accuracy score of 75.19% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 72.00%.

In this Take6 iteration, the word-embedding model’s performance achieved an average accuracy score of 73.44% after 25 epochs with ten iterations of cross-validation. Furthermore, the final model processed the test dataset with an accuracy measurement of 72.50%.

CONCLUSION: In this modeling iteration, the word-embedding TensorFlow model did as well as the bag-of-words model. Furthermore, we should continue to experiment with both natural language processing techniques for further modeling.

Dataset Used: Sentiment Labelled Sentences

Dataset ML Model: Binary class text classification with text-oriented features

Dataset Reference: https://archive.ics.uci.edu/ml/datasets/Sentiment+Labelled+Sentences

The HTML formatted report can be found here on GitHub.

Annie Duke on How to Decide, Part 4

In her book, How to Decide: Simple Tools for Making Better Choices, Annie Duke discusses how to train our brains to combat our own bias and help ourselves make more confident and better decisions.

These are some of my favorite concepts and takeaways from reading the book.

Chapter 4 “The Three Ps: Preferences, Payoffs, and Probabilities”

In this chapter, Annie Duke goes into the necessary details of building an effective decision process. She suggests the following six steps:

Step 1 – Identify the reasonable set of possible outcomes.

Step 2 – Identify your preference using the payoff for each outcome – to what degree do you like or dislike each outcome, given your values?

Preferences matter. Annie suggests we identify them upfront. It is not sufficient to merely identify the possible outcomes, and we should be explicit about which outcome we would prefer to see.

Payoff also matters. While we certainly have preferences on the outcomes, we also need to put forth the effort upfront in identifying the size of the payoff. Without diligently considering the payoff’s size, it would be difficult to assess how much we should risk achieving the preferred outcome.

Step 3 – Estimate the likelihood of each outcome unfolding.

To assess whether a decision is optimal or sub-optimal, we need to know the payoff and the likelihood for any given outcome. This also implies that to become a better decision-maker, we need to be willing to make educated guesses on each outcome’s possibility.

Step 4 – Assess the relative likelihood of outcomes you like and dislike for the option under consideration.

Often, we do not prefer all outcomes equally. Combining probabilities with preferences and payoff can help us to evaluate our decision better and minimize the paradox of experience bias. By clearly defining our preference beforehand, it can help us get out from under the shadow of the particular result that we experience.

Step 5 – Repeat Steps 1-4 for other options under consideration.

Step 6 – Compare the options to one another.