The “Why” of Valuation
Building Valuation Discipline
by Michael Greene and Alex Villacorta
Business school classes often demonstrate concepts like “the wisdom of crowds” or “the Winner’s Curse” by asking students to guess the number of jelly beans in a jar. Miraculously, the average guess of the class is typically quite close to the actual number of beans, while the range of guesses is distributed broadly. Never in question, however, is the precise number of jelly beans that represents the RIGHT answer. There is a finite and knowable amount of jelly beans, and whoever is “closest to the pin” has the best estimate.
One would expect that valuing a home would work similarly. While it may be challenging for an individual appraiser, broker, or automated valuation model (AVM) to nail the value of a home, at least we can know ex post facto who made the closest estimate. It is as easy as determining who was “closest to the pin” when the home trades. But is it?
What makes the definition of accuracy in the valuation space different from counting jelly beans is not only the “what” but the “why.” That is to say that the business objective the valuation supports is highly relevant to how one should measure that valuation’s accuracy.
For example, a mortgage lender wants to know what value for the home is supported by its surrounding market, so that it does not face idiosyncratic risk from an overvalued individual asset relative to its MSA-level risk models. In this case, an accurate point estimate of the price at which a home will transact is less important than the most probable minimum value supported by enough comparable properties to imply a liquid resale market.
A home seller or their agent would like to know what the highest bidder might be willing to pay for their home to formulate a listing and marketing strategy. Especially in times of tight inventory or when in possession of a unique home, this number is likely to be different from the mortgage lender’s number. Similarly, a retail home buyer wants to buy the house of their dreams as cheaply as possible. What matters to that buyer is not the intrinsic value of the home, relative to its neighborhood comps, but figuring out how to bid one penny more than the seller will accept (or competing bids). For these retail market participants, the most probable point-estimate of transaction price is what matters.
An investor would like to know the after-repaired resale value or rental rate to determine whether the property will deliver the returns they expect. This requires not only an understanding of the home’s value today, but also both the average price and market depth of peer assets in the neighborhood with similar hedonic characteristics but an upgraded fit and finish.
Customizing Valuation Discipline to Business Needs
The consumers of home valuations face an increasingly broad array of product choices around which to build their valuation discipline. In the last decade, the availability of cheap computing and storage power has unlocked previously unimaginable datasets for use in automated valuations (AVMs). Richer data has allowed for better attribution of value to an expanded list of property attributes and as a result has led to significantly tighter error bands. This increase in accuracy has been so significant over the last decade that AVMs have evolved from offering limited use cases around free-to-the-consumer marketing to offering credible, underwrite worthy valuations at the asset level. Recently, the industry has focused innovation around clever combinations of man and machine, with a spectrum of options from condition-informed AVMs to hybrid appraisals. The physical constraints imposed by the COVID 19 pandemic have accelerated the adoption of these “cyborg” valuations by realtors, lenders, and investors, funding further vendor innovation. It is no longer sufficient to rely on industry-accepted best practices when choosing how to value homes. Instead, each practitioner must create a valuation discipline that maps their company’s unique “whys” to the “whats” that the market is offering.
As Peter Drucker said, “what gets measured gets managed.” While human providers of appraisals and BPOs are typically measured against client outcomes, such as revision requests, automated or computer-assisted valuations lend themselves to systematic measurement and comparison. In constructing a valuation discipline, this is an enormous advantage to those of us steeped in interpreting these data, but presents dangerous pitfalls to those who are not.
Vendors and Appraisal Management Companies (AMCs) aim to serve as many market participants as possible, each of whom, as described above, wants something different. In competing for business, these vendors aim to achieve good scores against universal benchmarks across national averages. For automated solutions, these may include accuracy (usually measured by median absolute percentage error, or MdAPE), systematic bias, hit rate, or large error risk (measured by percent of valuations within 5, 10, or 20 percent of the ultimate sale price), each of which is more or less important to a different client. In the case of human-powered valuation, revision request frequency and turn time become important considerations, and everyone is sensitive to cost. A client’s “whys” determine which of these benchmarks should define their evaluation process.
Which Model is Best For You
If you are the buyer or seller of an individual home (or their agent) looking for a quick independent source of value to corroborate your comparable market analysis, an “accurate” AVM, as defined by a low national average error rate (MdAPE) is probably less useful to you than a “not wrong” AVM, defined by a low probability of major error (PPE5 or PPE10). After all, what is the point of being right on average if the client is only buying one house?
By contrast, an institutional investor using AVMs to conduct macroeconomic research and backtests may favor an AVM with no systematic bias and a high hit rate, rather than one that has a low probability of major errors. Being right on average is precisely this investor’s goal. Along the same lines, a mortgage lender looking to determine market support for their loan-to-value calculation, as described above, may do best with a set of AVMs that exclude list price from their computation, or those that ascribe a risk score based on the depth and breadth of similar comparables. These models may compare unfavorably to their competition in the dimensions of hit rate or MdAPE, but their structure may offer a more commercially useful answer.
When consumers of valuations have a nuanced understanding of which metrics are most appropriate for their business’s “whys,” they can leverage a wide array of products, tools, and data available to deliver an answer customized to their needs. At ResiShares, for instance, a substantial portion of our team has spent time in the valuation industry prior to their transition to institutional investing. Moving from seller to consumer, the ability to focus attention only on regions, assets, and valuation problems relevant to the business is liberating. Building a thoughtfully designed valuation discipline for internal use can be a source of edge for a fraction of the cost of building even a mediocre valuation product for external sale. As all aspects of the real estate industry gain in scale and automation, taking time to understand how (and why!) is time well spent.
