This is the final of the series where I will examine the problem that #Canada has with its lack of #funding for #technology startups. If you happen to have missed part I of Canada, and more specifically Toronto, belongs at the top– #Tech (for me) over the last 20 years; you can still view it here, or part II – Canadian brain drain and Toronto tech is too cheap can be viewed here. I then examined (in part III) the lack of VC money, visionary tech leaders and major exits in Canada then finally (in part IV) I looked over why University of Waterloo and University of Toronto have a love affair with the valley.
I will continue to write one post per week of meaningful length. I don’t do much in terms of social media (other than some light twitter) as I find it’s detrimental to my personal goals, so this is where you’ll be able to find most of my ramblings going forward. You may also want to check out my business where I am attempting to throw the real estate community (in Ontario) on its head by applying #Artificial Intelligence techniques to the mass amounts of data that I’ve amassed in aggregate from being a #realty broker.
Below is sort of a table of contents for why the time is only now for Canada:
- Tech (for me) over the last 20 years (this post)
- The Canadian brain drain
- How cheap is too cheap – yes #Toronto, I mean you
- The lack of #venture capital #money in Canada
- The lack of visionary technology leaders in Toronto
- The lack of major exits
- University of Waterloo and University of Toronto students’ obsession with the valley
- The rise of #incubators and #accelerators
- The godfather of AI – Geoffrey #Hinton
- Canada’s Liberal government sets the national focus correctly!
- What the future may hold for us
Incubators and Accelerators
The start of silicon valley was, well, due to silicon itself. It was the rise of the hardware engineer and the beginning of it was market by companies such as Hewlett Packard and Texas Instruments fighting over calculators and reducing the size ever more. Later a full on geek out (and I mean that in the best of ways, as I love to geek out on anything awesome myself; I’ll likely talk about the hacks that I produce at DeepLearni.ng in a future post) at the Homebrew Computer Club led to hobbyists meeting and sharing ideas.
Oddly, this is the what I believe to be the first Incubators of sorts in the valley.
(the first incubator and the idea of incubating) was not (born) in the West Coast, it was actually the East Coast that produced the first incubator (in Batavia, New York)
In a time where there was not much happening in Menlo Park, and definitely a time before internet forums, reddit, stackoverflow, medium, etc… this was the way to share ideas, and share ideas they did. From these clubs came ideas such as Steve Wozniak and Steve Jobs’ Apple I.
Other fine #engineering hubs such as Xerox PARC led to chance meetings of super smart engineers who went onto found their own companies, companies that originally dealt mainly in hardware, but then opened the door for some newly-found wealth (these aforementioned engineers) to want to pay-it-forward and help their brethren.
Fast-forward a few decades and as everyone knows, companies out of the valley have absolutely killed it and caused a tech revolution all over the world. One of the models that is often attributed to the valley is the Incubator model, however it is very likely that the first real idea was internal to companies and was a method to incubate ideas within a company. And, no, this was not in the West Coast, it was actually the East Coast that produced the first incubator (in Batavia, New York), oddly, it was not a planned idea to, but a necessity due to real estate needs (they could not find a master tenant for a large warehouse, so they just sub-divided it into many offices and helped nurture new business ideas).
The difference between Incubating and Accelerating
Most people do conflate both of these concepts. They are somewhat related and you may even find your business being part of both, but it will be at different stages of the business journey. The infographic below does a really good job of getting at the differences.
Looking and feeling like high-tech cities
During the dot com boom, the Medical and Related Sciences Discovery District was formed in downtown Toronto. I distinctly remember this as it was at the tail end of my education at the University of Toronto and I used to come out at the TTC station nearby (College Station) to walk to my engineering classes.
The idea was not solely theirs either, in early 2000, a New York City incubator Safeguard, also opened up shop in Toronto. The concept was built on the backs of bankers seeing the need (and opportunity) during the dot com era to capitalize on the north-east’s talent and they even IPOed. Unfortunately, during the crash of the early 2000s, they shut down.
Fast-forward nearly a decade (just passed the global financial meltdown of 2008) and business #entrepreneurs were once again beginning to flourish and many of the wealthier industrialists and educators (who are never short of money) began opening up their versions of Incubators and Accelerators.
Lately, it seems that every single business and university is jumping on the bandwagon, and much like stocks, when everyone is doing it – you know the field is saturated (or the last fool is in). In the case of entrepreneurs, this is great though, it hopefully helps drive down what these locales might take from you and your company in terms of equity and forces them to be more competitive about what they offer. This list (from medium) is a pretty good one on some of the better ones – the ones you may look to find great mentors, funding opportunities and possibly channel partners while growing (or accelerating) your business.
With the influx of these, entrepreneurs now had a hope (outside of fully bootstrapping and being one of the very few who might make it on their own) to gain space, mentorship, networking and funding for their ideas.
Critical mass of new startups
This has led to a ton of new startups (or burgeoning ideas) popping up all over Canada (and especially in Toronto).
Ever since my first foray into the startup life in 2007, when my friend and I were trying to launch a mobile advertising platform to local retailers based on a person’s location (called iWords), I have noticed more and more players popping up and it’s really inspiring to see. It’s a bit of a shame that there wasn’t this mass of incubators and accelerators at the time as I think that may have tipped the needle in our favour (as we were well ahead of Groupon and other deal aggregator, and very far ahead of all these restaurant delivery apps). What I really love is the fact that at this point in time, there is no reason such as that to fail, there are a ton of resources in Toronto (and in Canada) and as a result there has been a consistently growing ecosystem for entrepreneurs in Toronto.
With that said, Canada got another HUGE boost in its tech worthiness from a professor from the University of Toronto, whose work was so ahead of its time that it was dismissed by his contemporaries.
Geoffrey Hinton
The body of work that Hinton has produced focused on one grandiose idea, that we could get computers to “think” like humans do. For a very long time, as many ahead of their time often are, Hinton was cast aside by his peers. His students and him were in on the joke that they knew something that others didn’t and that as soon as there was enough computation available, as soon as Moore’s Law caught up to their work, there would be massive breakthroughs.
We program them to act like simplified neurons whose output value depends on the total input they receive from other neurons or from the sensors. Each of the input lines to a neuron has an adaptive weight, and the total input is the sum of the activities on the input lines times the weights on those lines. By varying the weights, it is possible to make a neural network respond differently to the input it receives from its sensors.
The main idea of neural nets is to have a rule for how the weights on the input lines to the neurons should change as a function of experience. For example, we show a network an image and ask it to activate neurons that represent the classes of the objects that are present in the image.
Let’s take a quick peak below at a fun application of deep learning itself so that if you do not have experience with the technology, you can have a bit of fun watching this video. There are also a lot of generative demos online that you can search for (here is a decent list: https://github.com/nashory/gans-awesome-applications)
Students
The power that someone like Hinton has is to inspire his students / researchers to be the next crop of amazing inventors. The focus that he has had on creating new things throughout his career has led to some of the best AI companies in the world hiring his Ph.Ds right out from his labs.
[vtftable ]
Name;;;Year of Graduation;;;Doctorate topic;;;What they are doing now;nn;
David Ackley;;;1987;;;Stochastic Iterated Genetic Hillclimbing.;;;http://www.cs.unm.edu/~ackley/;nn;
Mark Derthick;;;1988;;;Mundane Reasoning by Parallel Constraint Satisfaction.;;;http://www.cs.cmu.edu/~mad/;nn;
Richard Szeliski;;;1988;;;Bayesian Modeling of Uncertainty in Low-Level Vision.;;;http://www.research.microsoft.com/~szeliski/;nn;
…;;;…;;;…;;;…;nn;
Richard Zemel;;;1994;;;A Minimum Description Length Framework for Unsupervised Learning.;;;Co-Founder and Director of Research at the new Vector Institute for Artificial Intelligence{;n}Senior Fellow, Canadian Institute for Advanced Research;nn;
Brendan Frey;;;1997;;;Graphical Models for #Machine Learning and Digital Communication.;;;Canada Research Chair in Information Processing and Machine Learning{;n}Fellow of the Canadian Institute for Advanced Research;nn;
Radek Grzeszczuk;;;1998;;;NeuroAnimator: Fast neural network emulation and control of physics-based models.;;;Intel;nn;
Sangeev Oore;;;2002;;;Digital Marionette: Augmenting Kinematics with Physics for Multi-Track Desktop Performance Animation;;;Google Brain{;n}Vector Institute;nn;
Yee Whye Teh;;;2003;;;Bethe Free Energy and Contrastive Divergence Approximations for Undirected Graphical Models;;;Deepmind;nn;
Roland Memisevic;;;2007;;;Non-linear Latent Factor Models for Revealing Structure in High-dimensional Data ;;;Twenty Billion Neurons;nn;
Ruslan Salakhutdinov;;;2009;;;Learning deep generative models. ;;;Microsoft;nn;
Graham Taylor;;;2009;;;Composable, distributed-state models for high-dimensional time-series. ;;;Vector Institute{;n}NextAI;nn;
Andriy Mnih;;;2009;;;Learning distributed representations for language modeling and collaborative filtering ;;;Deepmind;nn;
Vinod Nair;;;2010;;;Visual object recognition using generative models of images. ;;;Yahoo! Labs;nn;
Ilya Sutskever;;;2012;;;Training recurrent neural networks. ;;;OpenAI{;n}Google Brain{;n}DNNResearch;nn;
Abdel-rahman Mohamed;;;2013;;;Deep Neural Network Acoustic Models for Automatic Speech Recognition;;;Amazon Alexa{;n}Microsoft Research Labs{;n}KULeuven{;n}RDI Technologies;nn;
Vlad Mnih;;;2013;;;Machine learning for aerial image labeling. ;;;Google Deepmind;nn;
Navdeep Jaitly;;;2014;;;Exploring Deep Learning Methods for discovering features in speech signals. ;;;Google in the Brain;nn;
George Dahl;;;2015;;;Deep Learning Approaches to Problems in Speech Recognition, Computational Chemistry and Natural Language Processing. ;;;Google on the Brain;nn;
[/vtftable]
As you can see above, and as per Hinton’s personal affiliation with Google itself, most of his (recent) students end up at Google in some capacity. Those that did not actually end up advising, mentoring and leading a lot of really great initiatives in and around Toronto, such as the Vector Institute (which we – DeepLearni.ng –are a proud part of) and NextAI.
If you happen to be an undergrad at U of T, you can always take his intro to Neural Networks course as well – offered here.
Alexnet
Before Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton started using GPUs to train deep neural networks on image classification data sets (and competitions), the best we could do was approximately a 26% error rate. As you can see below, deep learning and the use of GPUs in 2012 revolutionized this competition and the aforementioned scientists paper was a huge hit that year (especially at NIPS).
AI is Canada’s National Focus
Toronto and Canada was now poised to become a world leader in AI. We had all the ingredients to this soup in the cauldron. We had the godfather of AI himself here, we had one of the universities responsible for pumping out an unfair number of neural network Ph.Ds and we all of a sudden saw ourselves with a liberal government who was willing to focus on STEM.
Recently, artificial neural networks inspired by our understanding of how the brain computes have dramatically reduced the performance gap between people and machines, and this seems to me to vindicate the idea of using the brain to provide inspiration.
Trudeau and the Liberal Government
Canada’s federal government isn’t just paying lip service to the priority they’re placing on artificial intelligence. Budget 2017 proposed to provide $125 million to launch a Pan-Canadian Artificial Intelligence Strategy to promote collaboration between Canada’s main centres of expertise in Toronto—Waterloo, Montréal, and Edmonton—and position Canada as a world-leading destination for companies seeking to invest in AI. The lion’s share of that funding, along with $50 million from the Ontario government, helped create the Vector Institute, a recently announced independent research facility for artificial intelligence located in Toronto’s MaRS Discovery District. The Vector Institute engages with universities across Ontario and Canada to attract, train, and keep the world’s best minds in Canada.
“The job market is changing, and instead of resisting in vain, we’re focused on funding research and innovation, like in AI and quantum computing, that’ll help lead the change here in Canada.”
– Justin Trudeau, Prime Minister of Canada
In addition to providing financial support, Canada’s federal government recently introduced changes to immigration policy that will make it easier for companies to bring in technical talent from other countries. A fast-track visa program that offers up permanent residency was introduced in June 2017 with the goal of attracting innovators from across the globe.
Going forward
The US is currently still head and shoulders above the rest of the world with the world’s leading AI companies and tech giants (which is obvious by looking at the most valuable companies in the world). As mentioned above, these are the major winners at the moment in attracting talent from top schools, companies such as the usual FANGs (Facebook, Amazon, Netflix and Google, oh and Apple too).
China plans to be a world leader in AI by 2030 and recently committed $2.5B to a national AI research park in Beijing with a goal of supporting 400 companies.
Further, China’s large population and aggressive AI talent acquisition strategies from its tech giants, China is really setting themselves up to dominate and become the true global superpower (forget just finances USA, AI is coming to eat FinTech’s lunch).
Finally, there’s us – Canada! We are still looking good, we have some of the better schools (although they are currently feeder schools for some of these US companies at times), we have lots and lots of funding coming from the Federal government and a lot of programs aimed at fostering AI (especially in Toronto and Montreal).
We currently have a lot of really cool AI companies such as where I am #CTO (DeepLearni.ng), ElementAI and of course Layer6 which just got acquired by TD Bank. At DeepLearni.ng, we focus on ensuring that the Enterprise is ready for its AI journey with tools, education, platforms and process, which has aided our partners and clients to realize orders of magnitude of ROI against what we do.
Focus on investment and empowerment
My hope here is that we can get critical mass of great startup companies that get big, powerful and look to challenge the Googles, Facebooks, Amazons, et al of the world. Get some very nice exits behind our fellow Canadians and incentivize them to invest back into the system, make it easy for all of us to do this so that we don’t look South (especially to the valley) to invest in companies.
We need great mentors and coaches in Toronto. Vector, NextAI, Creative Destruction Lab, Ryerson DMZ, MaRS, Communitech are all great starts, however I feel like we need tech leaders who have experienced wins behind these rather than the more common academics and business folk that are usually there.
I myself have been meeting with as many founders of startups or burgeoning startups as possible outside of traditional means (i.e. seeing them at pitch meetings) to see if there is anything that I can do to help, any way to positively impact them and any way in which we can all grow together. There needs to be more of this and more willing grey beards in the tech world in Canada.
Teach meaningful courses and ideas at Universities
Another key point that is almost an afterthought but needs to be mentioned, Universities need to get on-board with what works in industry, not just what papers might be published. I have been part of several committees on curriculum and education in the past and keep noticing that these universities and many of the profs themselves do not focus on industry usable knowledge.
It would be nice that all Maths, Sciences and Engineering programs have a compulsory co-op term as part of the curriculum, something that forces students to actually do real work, see what kind of things are truly out there in the real world and then start to choose their path accordingly.
I myself take the whole education piece very seriously and make it my personal mission to empower the co-ops that work for me, give them as much REAL work (not grunt work) as possible and have them make real decisions that will impact our bottom line. I have been told that this is unique, however I feel that it is necessary. As we battle the big, well funded, easily recognizable companies of the world, the only way to be better is to out-do them at something – and for me, that’s the growth of my staff.
That’s it for this part of the series and welcome to 2018 with a long-term forecast of what I think might be possible. I have promised my really good friend that I would shed some light on a topic that I am going to have to heavily research for next week, how Canadian fathers should (do they?) take parental leave and how startups should empower them to do so. See you here next week, right around the same time.