Blog

Getting started with the Universal Recommender – Single machine setup

The Universal Recommender I’ll assume if you’re reading this that you’re already at least familiar with Universal Recommender, the most popular ML engine for PredictionIO. If you’re not sure what either of those things are we’ll have a separate post on those in the near future. The creators of Universal Recommender (Pat Ferrel and the team at ActionML) make it super easy to get started trying out their algorithm, loading in some of your own data, training, and getting personalized recommendations back out. You can just spin up an EC2 instance preloaded with everything you need and get off [...]

2023-01-14T00:18:13-08:00March 1st, 2018|0 Comments
Read More

Using Category Encoders library in Scikit-learn

I recently found a relatively new library on github for handling categorical features named categorical_encoding and decided to give it a spin. As a reminder - categorical features are variables in your data that have a finite (ideally small) set of possible values, for example months of the year or hair color. You can't feed these into predictive models as raw text, so some conversion is necessary to prepare these variables to be useable. Typically, you create a new, separate column for each possible value (or alternately depending on the intended model, n-1 values) and each of these new [...]

2023-01-16T16:36:58-08:00December 15th, 2016|1 Comment
Read More

TF-IDF Basics with Pandas and Scikit-Learn

In a previous post we took a look at some basic approaches for preparing text data to be used in predictive models. In this post, well use pandas and scikit learn to turn the product "documents" we prepared into a Tf-idf weight matrix that can be used as the basis of a feature set for modeling. What is Tf-idf? Tf-idf is a very common technique for determining roughly what each document in a set of documents is "about". It cleverly accomplishes this by looking at two simple metrics: tf (term frequency) and idf (inverse document frequency). Term frequency is [...]

2023-01-16T16:33:37-08:00November 18th, 2016|7 Comments
Read More

A Shiny New Python Data Science Sandbox in 30 Minutes Or Less

This post will give beginners a full walkthrough to go from nothing to a fully functional linux/python/pandas/scikit-learn environement with jupyter as a front end. For exploratory work, I really like this stack. My native OS is Windows, but since we're using VMs I would imagine the setup for OS X is very similar and probably won't need any modification (other than steps for configuring the VM). If you have a solid internet connection, we should be able to get this all done in under 30 minutes startiiiinnnnnng NOW... 1. Download an Ubuntu Desktop version of your choice. I like [...]

2017-01-30T11:40:40-08:00June 24th, 2016|5 Comments
Read More

Investigating missing data with missingno

I recently came across a new python package for visualizing missing elements of a data set. The aptly named "missingno" is super useful when you're taking your first look at a new data set and trying to get a feel for what you're working with. Having a sense of the completeness of the data can help inform decisions about how to best handle missing values. In this post, we'll take a quick look at the small and simple Shelter Animal Outcomes data set from one of the current Kaggle competitions. Matrix visualization The first visualization is the "matrix" display. [...]

2023-01-16T18:56:08-08:00May 20th, 2016|0 Comments
Read More

Text Pre-processing Basics with Pandas

In this post, we'll take a look at the data provided in Kaggle's Home Depot Product Search Relevance challenge to demonstrate some techniques that may be helpful in getting started with feature generation for text data. Dealing with text data is considerably different than numerical data, so there are a few basic approaches that are an excellent place to start. As always, before we start creating features we'll need to clean and massage the data! In the Home Depot challenge, we have a few files which provide attributes and descriptions of each of the products on their website. The [...]

2017-01-30T13:47:13-08:00May 10th, 2016|4 Comments
Read More
Go to Top