What is Deep Learning?

dlIn recent years the concept of deep learning has been gaining widespread attention. The media frequently reports on talent acquisitions in this field, such as those by Google and Facebook, and startups which claim to employ deep learning are met with enthusiasm. Gratuitous comparisons with the human brain are frequent. But is this just a trendy buzz word? What exactly is deep learning and how is it relevant to developments in machine intelligence?

For many researchers, deep learning is simply a continuation of the multi-decade advancement in our ability to make use of large scale neural networks. Let’s first take a quick tour of the problems that neural networks and related technologies are trying to solve, and later we will examine the deep learning architectures in greater detail.

Machine learning generally breaks down into two application areas which are closely related: classification and regression. Continue reading

Diaspora of the Machines

130912171352-01-voyager-story-topRecently NASA has been in the news with the successful launch and recovery of the Orion space craft. This was a four hour two orbit test of the new capsule that is intended to support future manned missions beyond the Earth. In addition, there has been a huge growth of the space industry in the last decade including commercial ventures such as Space X and Blue Origin, as well as proposals to mine the asteroids. There is always a tremendous interest in sending people to space, and in fact it seems to be an imperative for the human race to escape potential future disaster scenarios on the Earth by seeking solace among the stars.
Continue reading


I am currently working on improving this blog and putting more projects and documentation on here that might be useful or interesting for people to see. I’ve spent a long time looking for a slightly better theme, but every time I find something I like, it ends up being the case that it won’t install properly or else it has issues on mobile, etc. Most frustrating. I’m trying to make the blog a bit easier to use from mobile devices.

Natural Image Statistics

I’m doing some simple exploration of image statistics on a large database of natural images. The first thing that I tried was computing the histogram of neighboring image pixel intensity differences. Here is the graph for that using a log y axis, for a few pixel separations.

Pixel differences histogramIt is clear that large differences occur much more rarely and that the most probable pixel to pixel spatial change in intensity is zero. However the tails are heavy, so it is nothing like a Gaussian distribution. The adjacent pixel intensity difference log probabilities were fairly well fitted by a function that goes like -|kx|^{0.5} , and the pixels further apart require a smaller exponent.
Continue reading

Check out the new Tree Crafter Web Site

Tree CrafterI created a graphic design app for the iPad last year called Tree Crafter that lets you create tree inspired organic vector art and animations. It is fun to play with and quite relaxing. Also you can create great designs for web graphics or for merchandise with relative ease.

Recently I built a new web site for this app: http://treecrafter.com

If you have an iPad you should give it a go and let me know what you think. I am very keen to get it reviewed, so if you are an app reviewer, I can give you a promo code if you would like to write about it.

Calculating Engine Accepted at Maker Faire

Maker FaireI received the good news that my Relay Calculating Engine has been accepted for showing at the Maker Faire in San Francisco in May. I’m looking forward to taking it down there and showing it to interested people.

Make Magazine also interviewed me about this project and plan to feature the work too. I’m excited to see what they write about it.

My main concern is to finish it and get it all working in time for May. This should not be a problem since I currently have the luxury to be able to work full time on the project and at present, I am about 90% complete.

I hope that you can come along and check it out. I will also be showing some of the other projects on which I have been working. I’ll also be attending the Seattle Mini Maker Faire, but not showing anything off there.

Analyzing the Market – Part 3

BitcoinThis is part 3 of my series of posts on the statistics of financial markets. Part 1 is here.

In previous posts, I have found that working in log prices makes sense and that the double exponential distribution is a good fit to price change data. In this post, I will look at correlations over time in price changes.

Let’s ask a simple question: Does yesterday’s price change predict today’s price change? Continue reading

Analyzing the Market – Part 2

BitcoinThis is part 2 of my series of posts on the statistics of financial markets. Part 1 is here.

I have established that a double exponential distribution fits price movements when they are converted to log prices, at least for bitcoin, Apple, and Dell. (Actually I have checked it on a few other NASDAQ stocks too.)

Once we have a statistical model, we can generate some data to see if it produces results that look like the actual price graph. Below you can see the real 2 month bitcoin price graph, together with two graphs that were obtained by using a model based on the Continue reading

Analyzing the Market – Part 1

BitcoinThis series of blog posts is intended to document some mathematical analysis that I have been doing on the bitcoin price graph and on price histories of securities in the stock market. The purpose is to understand something about the statistics of these price movements, and to learn about the behavior of the stock market in general.

One thing that is useful about bitcoin is that trading is never stopped. Because everything runs 24 hours 7 days per week, there are no artifacts to do with starting and stopping trading on specific exchanges and transitioning between financial Continue reading