A number of years ago, we decided it was important to figure out how to filter startups based on likelihood of success. This seemed like a daunting task because it’s highly speculative. If you ask 5 investors to rate 5 startups you are likely to get 6 different opinions most of the time. Everyone approaches the process with their own bias.
AngelMD has been fortunate to partner with some terrific firms working to tackle this challenge from a variety of angles. None of them has thought more deeply, and done more mathematical work on this topic, than ValidEval. The ValidEval platform originated out of a call from the Kauffman Foundation to score startups presenting at pitch events.
The Kaufman Foundation is one of the world’s leading entrepreneurial think-tanks and startup support organizations, so they know a thing or two about pitch events, startups etc. The ValidEval platform has continued to evolve from the initial Kaufman solution and is now used to evaluate startups and grant proposals resulting in billions of dollars of deployed capital and enormous social proof.
When ValidEval is combined with the expertise of the AngelMD Clinical Advisory Board it creates significant insights and unprecedented analysis of startup merit. It also enables important longitudinal analysis to determine whether earlier predictors and signals were in fact accurate. It enables AngelMD to determine which evaluators carry the most weight in terms of both contribution and accuracy. (yes, we evaluate the evaluators ) We remark internally that just because someone is an outlier doesn’t mean they are wrong. ValidEval helps to strip bias from innovation analysis and arrive at objective conclusions.
We provide this context because working with ValidEval has exposed us to some important characteristics tied to potential startup success and failure. We follow the math. Several years back, ValidEval engaged a Harvard-trained economist named Sabrina Howell, PhD. Dr. Howell did some deep analysis of startup performance data and determined that the most determinant characteristic of startup success was the “coachability” of the entrepreneur. The notion of coachability was analyzed in great detail, but essentially was a way to identify entrepreneurs that were willing to iterate and pivot. Those might seem like obvious attributes you want to see from an entrepreneur, but surprisingly the majority of entrepreneurs don’t actually demonstrate this key trait.
An entrepreneur is more likely to believe they have built something to change the world; and if the world doesn’t get it, the world is wrong. We have developed some methods and lines of questioning to tease out whether or not an entrepreneur is likely to demonstrate coachability over time.
Somewhat related to this characteristic is the ability to think on one’s feet. This is more easily identified by asking someone to walk through the solution to a stated problem in a live conversation. While they think you are searching for the right answer to the problem presented, the manner in which they react to the line of questioning, think on their feet and articulate an answer, are far more important than the actual answer.
(Dr. Howell further found that the ValidEval platform was more effective in identifying key predictors of startup outcomes than platforms used by a number of leading institutions including her Alma Mater. While there is never a “right” or “wrong” answer in an evaluation, the key is to develop probability characteristics. For those who watched the tv show Billions, you know they talked probability all the time. In one scene, the star hedge fund character asks one of his lieutenants, “How certain are you?” His answer “I am not uncertain.” They moved forward with that investment.)
When the AngelMD Clinical Advisory Board evaluates startups, we use a structured process to ensure we are weeding out bias and developing a statistically valid sample size of evaluations. Given that we are leveraging physician time and expertise, we don’t want to “overspend” on this valuable resource by engaging more evaluators than will provide an optimal data-set. On the other hand, our methodology will beat any individual investment analysis in almost every way because of its unique design and the mathematically optimized sample size of contributors.
But if you aren’t using our system to do deep analysis, figure out a method to determine “coachability” in the startup you are considering. It may be the difference between a good idea and a good company.
(Next up: we’ll review some key operational attributes to look for in a startup including simplicity.)
