The quality of those not hired is the most valuable recruiting metric that you have never heard of! It informs you how often your organizations is failing to hire the highest quality applicants.

A few years back I was advising a Fortune 100 firm that had a painfully slow and somewhat arrogant hiring process. To demonstrate the negative impact of their process I had to prove to a skeptical senior manager that they were letting top candidates get away. I asked a manager hiring for an important job to rank, in order of quality, 100 applicants who had been sourced for the role. The chosen rank was discretely written on the back of paper copies of the candidate’s resumes. Months after the role had been filled, the manager was asked if they were satisfied with the hire. He was, and felt quite certain that he had successfully hired a “top 5” candidate. After hearing of his satisfaction I had him look at the initial rank he had provided the candidate who was later hired: 75.

You can imagine his shock when he realized that the hiring process had somehow let every single one of the top-ranked applicants that the firm had prided itself in hiring “every single time” slip away. Clearly the quality of the people who they didn’t hire was significantly higher than the quality of the one that they did.

Selecting HR Metrics Is Unfortunately Not a Scientific Process

Most organizations adopt metrics based on those covered by benchmark reports or that can be easily enabled via their technology providers, instead of determining what they need to discover or prove. As a result, many organizations are burdened with data and reports that offer little in the way of guidance helping them improve their effectiveness. One metric often not fully taken advantage of is quality of hire, which I estimate less than 40% of organizations even attempt to use. Even fewer use the quality of hire derivative, quality of those not hired, because it can very quickly demonstrate how poorly a process performs.

Determine Where Your Recruiting Problems Are Occurring

During an advisory conversation with a recruiting leader at a well-known social networking firm experiencing difficulty achieving hire diversity, I asked “at what step or stage is your recruiting process failing?” I wasn’t surprised when he responded “we don’t actually know, we just know that the overall recruiting process is not producing the results we need.” Like many organizations, this organization lacked well-thought-out metrics that enable both performance reporting and process diagnostics.

Recruiting processes fail because either they do not attract enough top-quality candidates up front, or they fail to accurately identify, assess, and sell those attracted on the job at later stages in the process. Most organizations focus heavily on measuring sourcing effectiveness, but ignore the later stages of the process altogether. One benefit of using a “quality of those not hired” metric is that it focuses exclusively on the back end, where I estimate at least 50% of those organizations not meeting their goals have problems. If you doubt that the problem is post-attraction, ask your favorite agency or executive recruiter what percentage of qualified candidates are lost due to slow or ineffective actions on the part of hiring managers and corporate recruiting processes.

One of the purposes of the quality-of-those-not-hired metric is to force organizations with a high percentage of quality hires slipping away to identify where in their process the talent opts out or gets dropped. There are six post application stages where firms lose top candidates, including:

  1. Resume screening process — the ATS, a recruiter, or a hiring manager mistakenly screens out top applicants.
  2. Telephone screen — top applicants rank poorly on their phone screens or their screen cannot be completed, so they are dropped from consideration.
  3. Interview scheduling — they get frustrated over the number of interviews and dropout or they cannot complete them in time because of scheduling conflicts.
  4. Interview assessment — they voluntarily drop out before the interviews can be completed, or the interview process mistakenly rates them poorly.
  5. The offer process — either the process fails to include most of the top applicants on the list of finalists, or they reject the offer.
  6. Reference checking — even though they are high-quality candidates, they somehow fail the reference/background check.

Since the goal of a good metric is to help you identify what is not working, carefully select and implement at least one metric that can point out failures occurring during the latter stages of your recruiting process.

Focus Only On the Top Candidates

A quality-of-those-not-hired metric can become cumbersome if it attempts to categorize the quality of every applicant who doesn’t get hired. In order to save time and money, narrow your focus to the strategic issue of “what happened to the cream of the crop?” Out of 50 applicants for a single job, there might only be three who were so qualified that a hiring manager would actually regret failing to hire them. I call these individuals “regrettable misses,” and it is these folks that the quality-of-those-not-hired metric aims to highlight.

Action Steps for Developing a “Quality-of-Those-Not-Hired” Metric

If you decide to implement a quality-of-those-not-hired metric, there are several action steps to consider, including:

  • Setting goals — I recommend that you set goals for the use of this metric that include: accurately identifying the top three to five “regrettable” candidates; determining what percentage of top candidates become finalists for the position, and determining what percentage of new hires came from the top candidate list.
  • Select an evaluation range — this metric should focus solely on reporting the progress of “the very top applicants” who senior managers would regret not hiring. To limit the scope of evaluation, preselect what size of candidate slate will be evaluated. For most jobs, three to five top applicants would be a sufficient number to track. You can also use a set percentage of all applicants (i.e. top 10%) to define what you mean by top.
  • Determine when to identify top applicants — identify the top applicants early on in the hiring process so that you will have time to address any issues that emerge before a final hiring decision is made. If you are conducting an audit post hire, you need to make sure that the person doing the initial selection isn’t aware of which individuals were finalists and who was hired.
  • Select methods for identifying top applicants — the best method for identifying the top applicants is to have multiple evaluators select a finalist slate that is then merged to create the sample that will be monitored. An alternative approach involves using the profile matching capabilities of your ATS to produce a listing of top applicants. A third possible list segments applicants who come from high-value benchmark firms.
  • Report the metric in percentages — the best way to report the quality of “those not hired” metric is in percentages. For example: 66% of all finalists came from the top-ranked list, and 47% of the time a top-five-ranked candidate was hired.
  • Identify the stage where top talent slips through — for high priority and mission-critical jobs, after the hiring process is complete, identify at what specific stage in the recruiting process did a top applicant opt out or get dropped from consideration. You can then use that information to improve that stage.
  • Identify cause for top candidate removal from consideration — if a significant number of top candidates opt out or are dropped from consideration without becoming finalists, follow up and find out why. If your process screened them out prematurely, recruiters and hiring managers must be questioned to identify what knockout criteria is being applied. If the candidate dropped out on their own, they need to be questioned to see if their early withdrawal could’ve been prevented.
  • Keep in touch — separate from the process of calculating the metric, the organization should keep in touch with and build a relationship with the high-quality applicants who you regret missing. Building this relationship will help to ensure that they will favorably consider another opportunity with your firm in the future. Develop an alert system so that the star applicants can automatically receive e-mail alerts whenever a relevant job opens up.

Sample “Quality of Those Not Hired” Report

Here is a sample report illustrating what a recruiting leader or hiring manager might see.

Job Family: ASIC Engineer

Report Period: Q3 2010

Hire Volume: 53

Metric Job Family Organization
Percentage of top candidates in finalist pool 53% 52%
Party responsible for removal from consideration
–candidate filtered out by recruiting process 17% 48%
–candidate opted out of recruiting process 83% 52%
Percentage of top candidates who rejected offer 95% 22%
Percentage of hires from top candidate slate 31% 42%
Process stage contributing largest slate loss Interview Scheduling

Recommended Actions

Candidates reported that more often than not interviews would need to be rescheduled because times initially proposed were no longer available upon confirmation. Many candidates reported that it took recruiting coordinators longer than a week to confirm meeting dates and times. Solution:

  1. Allocate dedicated time slots to recruiting activities that cannot be booked by other activities more than 24 hours in advance.
  2. Establish service level agreements that call for manager response to scheduling inquiries within four business hours.

Final Thoughts

If you were a competitive fisherman participating in a pro fishing tournament and you repeatedly landed prize-winning fish, you would be justifiably proud. However, if you repeatedly caught prize contenders but lost them prior to tournament completion, wouldn’t you want to know exactly where and why you kept losing them? That is exactly what the “quality-of-those-not-hired” metric tells you. It reports how often you successfully land a great applicant, but fail to convert them to employee. Your organization can’t attain the highest level of new hire on-the-job performance (quality of hire) if your process allows the highest-quality applicants to be missed.

Without even knowing the name of your organization I can predict that throughout the most recent downturn you let your employee referral program “go,” so to speak. By failing to take advantage of new trends, technologies, and tools, and decreasing efforts to update and keep the program highly visible, your organization has allowed a number of program-performance-degrading-things to occur.

Unlike the previously posted list of program design features that can “kill” an ERP, these factors plague even well-designed programs, rendering them weak and ineffective. While much more likely to occur during economic downturns and periods of reduced or halted hiring, these program degraders can emerge whenever an employee referral program is neglected.

If your program doesn’t seem to wield the power it once did, do a quick mental audit to see if any of these factors may be to blame.

The Top 20 Employee Referral Program Performance Degraders

  1. Loss of a program manager — losing or failing to replace a program manager charged with championing the program and keeping it active can lead to disastrous consequences even for a short period of time. Solution: a resource, hold them accountable, empower them with program metrics, and educate them on key ERP success and failure factors.
  2. Reliance on job announcement spam — unfortunately it has become common practice to spam employees with irrelevant job announcements and generic program communications, both of which overworked employees quickly learn to recognize and set aside. Solution: use a more targeted approach to sending out announcements, decreasing the overall volume, and making sure each contains information relevant and of interest to the recipient.
  3. Repetitive message formats — years of experience have demonstrated that using the same message format over and over will eventually result in employees tuning out the noise, just as you tune out the billboard that rarely changes on your commute to work! Solution: toss out form-messaging templates and craft a real communication that educates, empowers, excites, and calls for action.
  4. Repetitive rewards — program rewards and prizes are intended to excite, but once they become commonplace or stale, they cease to be effective. Solution: periodically change or rotate program incentives using a survey sent to a sample of your employees to determine what would or would not work.
  5. Program suspension — some organizations completely suspend their program when hiring is slow, resulting in an “out of sight, out of mind” mentality among employees with regards to being 24/7 talent scouts. Solution: regardless of requisition volume, never suspend a program. It’s OK to narrow the scope of the jobs covered or temporarily reduce incentives, but the process and mantra to keep employees scouting talent for future hiring needs must never cease.
  6. Not countering “inappropriate now” arguments — when reductions in force occur, it’s not uncommon for arguments against external hiring to emerge on the premise that giving a job to anyone other than those displaced already would damage morale. Solution: it’s unfortunate that reductions in force must occur, but when they do, those shed by the organization are often those in non-core roles or that possess skills no longer as valuable to the organization as their rate of compensation. To assume that a mission-critical or key vacancy could always be filled by the ranks of those laid off is silly. It’s also silly to assume that other organizations would be laying off volumes of talent in previously hard-to-fill areas, reducing the difficulty of recruiting scare talent moving forward. It’s the role of the ERP program manager to counter or prevent such arguments before they get started. In modern agile organizations, hiring can occur in critical business units just as layoffs occur in others. Make managers and employees aware that the development of new products and services (and their future job security) often depend upon access to new skills and technologies. While retraining those displaced is nice, sometimes it is both time and cost prohibitive.
  7. Not maintaining operational responsiveness — without dedicated attention, it is easy for programs with exceptional service standards to slip, resulting in delayed responses to inquiries, slower referral response times, and even complete non-responsiveness. Because response time is the No. 1 success factor for ERPs, service standards should be restored. Solution: re-examine the service standard expectations you have set for your program, and if unable to resource the program adequately to maintain them, reduce the scope of the program temporarily or seek volunteer assistance so that every interaction meets expectations.
  8. Relying on the original business case — business priorities change, unfortunately few HR organizations update their business case for key programs such as the ERP to reflect the changing environment. Without ongoing program positioning, it’s easy for programs to lose their executive champions and for participants to forget all the ways the program benefits them and makes the organization stronger. Solution: the business case for the employee referral program should be reexamined every quarter and changes should made to the program strategy and operating model in accordance with changing needs. Executives, managers, and employees in particular need to be reminded of the important role the program plays in building better teams and improving organizational performance. The communicated business case should include evidence of improved quality of hire, retention rates, diversity rates and promotion rates. It is especially important to update the “performance differential” calculation, which demonstrates that referral hires produce more on-the-job than other sources of hire.
  9. Ignoring new tools and technologies — programs designed even recently might not have taken advantage of newly developed referral tools, approaches, and technologies. In recent years a lot of development in process and technology has occurred to support vacancy prioritization, electronic referral marketing/communications, social network extension, external stakeholder participation and automated program administration. Solution: every quarter reevaluate how technology can empower the world-class process you desire to execute and try out many of the new service/tool offerings among pilot study audiences. With many tool developers adopting agile development methodologies, product offerings are likely to change/evolve frequently.
  10. A lack of employee education — even in great times many organizations failed to provide enough tools, training, and support to help employees uncover great talent within their network, so during tough times it’s no wonder that education efforts all but go away. Without referral events, manager executed referral activities, PDA parties, referral open houses, “Give Me Five” visits, and priming exercises, ERPs become purely reactive and fail to produce the volume of flow needed in the most critical areas. Solution: the referral team must develop quick but compelling presentations/exercise kits for employees and hiring managers (available both online and in person). These tools should explain and clarify new and revised policies, procedures, and expectations, as well as walk employees through simple exercises designed to help them identify possible referrals for current and near term key talent needs. Participant research reveals that a majority of employees are underwhelmed with the amount of how-to guidance their organization provides on identifying possible referrals, networking, dealing with “would you refer me” requests, and how to convert contacts into referrals.
  11. Not interfacing with related HR programs — in recent years many organizations have invested in social media programs, alumni group development, and contingent labor programs, all of which have logical ties to the ERP, but that might not have been used well during the downturn. Solution: put together a team with leaders from each related program to determine where partnerships make sense, where duplicate efforts are underway, and most importantly where resources/tools are underused.
  12. Outdated prioritization — well-designed referral programs prioritize vacancies based on their business impact, and referrals based on the past referral success rate of the referrer. However, the organization’s priorities may have changed. Solution: if you don’t have a prioritization schema, develop one now. If you do have one, work with senior management to adjust the schema based on emerging needs quarterly or as critical incidents emerge. (Note: prioritization does not require that individual referrals be treated any differently during the hiring process.)
  13. Lack of recruiter training — failing to periodically update training of existing recruiters and skipping the training of new recruiters regarding the critical success factors of referral programs can degrade a program from within. Solution: program managers need to be continually educated on the latest benchmark best practices and performance targets leading organizations are adopting and develop periodic training/information sessions for recruiters outlining their evolving role.
  14. Failing to do periodic upgrades — the performance of even the best-designed referral programs degrade quickly when program evolution ceases. Solution: if you are not now or have not been rolling out program enhancements and promotions at least once a quarter, start now. Tie enhancements and promotions to forecasted critical needs and short-term business issues to create natural excuses to communicate about the program and improve visibility.
  15. Not using metrics — great referral programs rely heavily on metrics to continually improve, but when times get tough, metrics often all but disappear. Solution: identify how employee referrals impact business operations and layer program performance metrics into existing finance and operations reports, making the program visible as a performance driver. Do not forget to quantify the dollar impact on revenue of the key quality of hire metrics.
  16. Mergers with other recruiting programs — during tough economic times, it is not uncommon for autonomous referral programs to be combined or merged with other recruiting programs. This lack of identity and control will rapidly degrade program performance. Solution: if you have merged your ERP with other initiatives, make the business case to restore its independence immediately.
  17. Unfounded legal fears — even well-designed referral programs get “nitpicked” on by “overly nervous” lawyers and HR professionals who often present their personal opinion as unbiased professional guidance. Solution: don’t argue with attorneys; instead, partner with them on the premise that it is the corporate counsel’s duty to find a way to do what the organization “needs” to do in a manner that reduces the organizations exposure to risk. You wouldn’t propose writing their legal documents, so they shouldn’t design your programs! Make the business case for key program features and outline the negative impact of foregoing a practice. Risk mitigation is about balancing the possible cost of litigation with the financial benefit of a practice. If you are not armed with ROI projections, real-world data, and best-practice benchmark examples, don’t expect to fend off unfounded legal arguments.
  18. New/alternate ATS — many applicant tracking systems provide an ERP module that becomes more attractive versus operating a separate ERP program with isolated infrastructure during tough times. These modules which allow employees to send an invitation to apply to referrals turn all referrals into online applicants long before they need to be. They also result in a dramatic decrease in the conversion rate of employee-initiated employee referrals. Solution: figure out when it makes sense in your organization for a referral to become an applicant, and structure your technology solutions to empower your process the way you want it versus altering your process to fit the design of an available tool. Hundreds of mashable tools and services exist today that can empower your program as you envision it.
  19. Failing to scale — in tough times organizations merge and get acquired. If your organization has done either, it’s not uncommon for a program designed for a small organization to be ineffective in a larger organization. Solution: evaluate what elements of your program can scale to meet the needs of the newly combined organization and which elements need redesign. Until all program elements can function effectively, consider limiting the scope of the program to that which the existing infrastructure can support at the service level desired.
  20. No globalization — if your organization has become a truly global one, as many have, your ERP must be globalized. Solution: look at all processes, communications, and policies to ensure that cross-border referral of talent is being facilitated, and that all possible scenarios have been planned for. Identify what elements of your global program my require localization (communications, rewards, etc.) and develop a matrix specifying each.

Final Thoughts

Very few things that are easy to do have more of an immediate impact than using a failure analysis “checklist” to conduct a quick assessment of an important HR program. With hiring targets growing, there is no more important recruiting program to assess than your ERP, don’t you agree?