Hardware nerds are hot
Changes in computing mean software companies need hardware-savvy employees
By Sam Blackman, CEO and co-founder, Elemental Technologies
Blackman: Your next hire might need hardware chops. Photo: Elemental Technologies
Whether we knew it or not, we’ve all been relying on something called “Moore’s Law.” Back in the 1960s, Intel (INTC) co-founder Gordon Moore noticed that the number of transistors that could cheaply be placed on an integrated circuit had been doubling every two years.
That meant that central processing units, or CPUs — the chips that drive computer performance — were getting twice as fast in that same time period. That amazing rate of technological change has held up for more than 40 years.
Moore’s Law is why we take it for granted that the cell phone we carry around today is more powerful (and cost us less) than the top-of-the-line desktop computer we bought ten years ago. It is also why we’re not surprised that in less than a decade the Web has changed from a place to look at ugly text pages to a place to watch high-definition TV shows.
But after 40 years, Moore’s Law is slowing down. We’ve finally reached the point where faster processors consume too much power, and manufacturing them to achieve ever-higher frequencies gets to be too expensive. This technological pressure will radically reshape the way we build computers and write software in the years to come.
Going forward, computers will get faster by adding additional processors that work together to solve problems. That’s why we hear more these days about the number of cores in the CPU rather than how fast the processor is in our computer. Giants like Intel and Nvidia (NVDA) are racing to create new “massively parallel solutions,” composed of as many as 240 individual processors designed to work in concert to solve problems.
Unfortunately, writing software that runs well on massively parallel systems is incredibly difficult. Engineers need to figure out how to break big problems down into smaller pieces that individual processors can work on at the same time, how to keep all of the individual processors coordinated with each other, and how to assemble all of the work into a useful output.
At the recent Hot Chips microprocessor design conference in Palo Alto, Calif., John Hennessey, the president of Stanford University, called parallel computing “the hardest problem in computer science.”
To date engineers have only solved a small set of problems using parallel systems, and it’s not for lack of trying. Microsoft (MSFT) and Intel think that figuring out parallel computing is so important that they’ve invested $20 million funding parallel computing research centers at the University of California Berkeley and the University of Illinois at Urbana-Champaign.
Difficult or not, the future of computing is going to be on massively parallel systems. Some savvy companies are already taking advantage of massively parallel systems to trade stocks, search for oil, and offer online video games. At Elemental Technologies, we’re building software to help professionals process video files faster and more economically than ever before.
To build the kind of team that can take advantage of these massively parallel systems, software companies are going to have to rethink the mix of engineers that they are hiring. They will need people with experience in hardware design and low-level “close to the metal” programming. Engineers who understand how these new massively parallel architectures work, and know how to parallelize problems. Today, programmers with these skills are in seriously short supply.
There’s a pool of great engineers who don’t even realize that their future is working for software companies yet, though. They’re the digital hardware engineers who have spent their career working for chip companies and startups working on things like embedded systems and integrated circuits – where the parallel processing paradigm has been in use for years, since that is the way physical devices work. The smartest software companies will snap up as many of these engineers as soon as possible in the next few years and put them to work building software that can take advantage of the computers of the future.
Companies that don’t harness this resource will find themselves disrupted by faster, cheaper, and smarter software from competitors who did.
Blackman is CEO of Elemental Technologies, a Portland, Ore.-based company that develops massively parallel software to help computers convert video more efficiently.
Seeing this article makes me laugh because in college I was punished by my professor almost 20 years ago for not wagging my finger at a classmate's off colour joke.
My punishment was to design a parallel sort algorithm where the returns of the subsorts could be simply sequentially lined up and still be finished and correct. Try laying a British dictionary and American one on top of the other and you realise just how silly the request was. I still pretended to make an effort though but not enough to keep from being docked a letter grade.
I think it would be more fun to design a computer that had each of the 5 senses and one would program these senses operate independently but share their data with a core that was applying statistical models to determine the likelihood of 5 seemingly independent views of the world all implying the same event and then attempting to deduct cause and effect by requesting, say the sight and hearing subprocessors to increase their resolution of the data they are collecting for this certain statistically interesting subset that was implicated. The same could approach could be applied to all data streams. Maybe this is already what is being done or maybe this approach is wrong, but it would be fascinating what such a system would have 'learned' with no preconceived ideals of geometry, colour, music, or the like. It could then be programmed to build different sets and give them names and see what kind of dictionary it created.
No two dictionaries would ever be alike but two such computers could share dictionaries and find matches and then teach each other the differences.
You could then send them searching for conditions that they 'learned' rather than what someone has attempted to tell them via specifically programming it.
Many hardware engineers are already trained to program in a derivative of Ada and think about how to solve problems using parallelism. We call it "VHDL".
In response to David Emery: I completely agree that CS programs at schools like University of Illinois Urbana Champaign, Carnegie Mellon, and a few others are turning out some superstar parallel programmers. But a reasonably high percentage of software engineers who have only been trained in high-level languages like C++ or Java really don't have a deep understanding of the underlying architectures they are programming to because there are so many layers in between. Embedded software engineers have to understand the hardware from day one, including all the concurrency and real-time requirements, which translates quite nicely to programming GPUs. Thanks for your thoughtful comment.
Thanks for your interest! In answer to R Brown's query, that is a good point. I was not intending to single out Microsoft other than to say they have invested some serious research dollars in solving the problem. That said, there are several interesting programming languages that all support massively parallel GPU processors today. NVIDIA released CUDA in late 2006, which is what Elemental uses in products today. Apple has led the development of the OpenCL language, and Microsoft has DirectX 11 Compute Shader. So there are many viable platforms out there, but actually getting optimized performance out of them is quite challenging.
I think the overlooking of Apple's work is because the Intel/Microsoft efforts are a lot larger, more public, and longer-running. Apple tends to be a lot quieter about its projects until they actually get released. Moreover, the Microsoft/Intel effort was a major effort essentially setting up large research centers at major universities.
Also, Grand Central Dispatch took a lot of people by surprise in that it was a lot better than people expected.
I am one of those hardware engeineers who would like to be in the software business. Parallel computing is the future – looking forward to the day I can get hired as these initiatives grow (any thoughts on who is hiring – perhaps msft, aapl, msft) would be much appreciated.
I'm curious why you included Microsoft's unnamed proprietary efforts, yet did not mention Apple's Grand Central Dispatch, a tool to enable software developers to more effectively program multiple core CPUs, and which has been made Open Source, available to all.
As you are the CEO of a company in the field, I can only assume it has to do with your alliances.
Well, there -are- software technologies out there for both special purpose parallelism and general purpose parallelism. One problem the industry has is recognizing what we've already done. As an example, back in the mid-80s, the Ada programming language and its concurrency model was successfully deployed on a variety of shared memory multiprocessor minicomputers, from companies like Sequent and Encore. This included design methodologies, compiler and operating system technologies, etc. Those machines look a lot like today's multi-core processors, including problems like cache consistency, etc. As a result of this experience, substantial updates were made for Ada95.
Schools need to teach concurrency principles, and not just this year's favorite programming language. People need to understand things like consensus algorithms and byzantine failure, as well as race conditions and deadlocks, and then understand how various technologies address these fundamental problems.
So I see this as an education problem for both engineers and managers, to recognize what we do know and apply it better, as well as for the research community in extending that knowledge. I'm not convinced that hardware engineers understand a lot of these technologies better than well trained computer science/software engineers. But I am convinced that the core argument, We have to do better than we're doing now. is correct.
alwasy new that pushing code to the hardware's limits had its use. Now, finally, someone backs it up in research.The old way of proprietary code and meta programming to obscure processes from the hardware is slowly falling apart.The reasons are obvious.The old 40 years old UNIX code is showing its age.Time to re-invent the car, not the wheels on it.Windows is still based of off DOS cludge, and well, I can't speak for embedded or zen systems, I haven't seen the code for them yet.From what I have seen, programmers have taken the lazy way out in recent years.