Monitor and evaluate displaced talent and employer outcomes

By integrating a thoughtful approach to monitoring and evaluation from the beginning, organizations building new programs to facilitate displaced talent mobility can:

1. Track progress toward stated program goals 
2. Continuously improve their programs
3. Contribute to a growing evidence base about the impact of displaced talent mobility, and
4. Generate knowledge about displaced talent mobility that can shape the future of this important solution. 

This article offers further information about how organizations can pursue each of these goals. 

Skip ahead to

Track progress towards program goals 

As displaced talent mobility programs are being designed, proposed, and implemented, organizations will often set several key performance indicators. These indicators are often tracked in real time to ensure that projects are unfolding effectively and in line with proposed timelines. 

Some relevant indicators for displaced talent mobility programs include:

Indicators to track awareness of displaced populations about labour mobility as an opportunity:

• # of profiles created on the Talent Catalog or similar expression of interest database
• # of CVs created

Indicators to track employer awareness and commitment: 

• # of employers expressing interest in recruiting displaced talent 
• # of employers that initiate employment by providing a job description
• % of employers that receive a pitch that initiates recruitment
• # of industries reached in pitches 

Indicators to track conversion rate to successful hiring in particular occupations: 

• # of intakes conducted
• # of CVs shortlisted and shared with employers
• # of employers that conduct interview processes
• % of employers that receive CVs that advance to interview
• # of interviews facilitated
• % of CVs shared with employers that result in an interview
• # of employers that make job offers
• # of official job offers extended and #of job offers accepted
• # of job offers extended in a given occupation
• % of interviews that advance to a job offer
• % of employers that advance to interview that make a job offer
• % of job offers that are accepted

Indicators to track direct beneficiaries of displaced talent mobility programs: 

• # of principal applicant visas filed
• # of total visa applications filed (with dependent family members)
• # of visa applications approved
• % of visa applications that are approved 
• # of principal applicants relocated
• # of people relocated (including dependent family members)

Indicators to track retention (high retention is a key indicator of future job matching success): 

• # of people retained on the job at 6 months after start date
• # of people retained on the job at 1 year after start date
• % of people retained on the job at 6 months
• % of people retained on the job at 1 year

Indicators to candidate and family well-being and settlement: 

• % of candidates that report feeling safe and secure in their new home 
• % of candidates who report their spouses as having a high level of satisfaction with their life in the destination country
• % of candidates reporting an increase in their overall quality of life 
• % of candidates reporting positive feelings of their life overall 

Indicators to track employer satisfaction:

• % of employers who rate the candidate(s) technical skill level today as meeting their role requirements 
• % of employers who rate the candidate(s) soft skill level today as meeting their role requirements 
• % employers that say they would refer or recommend other employers hire displaced talent
• % employers that say they are willing to recruit displaced talent again
• % who envision hiring displaced talent being part of their standard recruitment strategy
• # of return employers (initiate another recruitment process after a successful hire)
• % of employers that make a job offer that become repeat employers

Indicators to track protection and durable solutions outcomes

• # of people who secure permanent residence status
• # of people who secure citizenship

The above metrics might be disaggregated by source country, destination country, gender, ethnicity/nationality, industry, or other meaningful categories as appropriate. 

See sharing impact data to drive wider learning and results for examples of how and why these metrics are used.

Continuously improve programs based on analysis of indicators 

When data on indicators is collected regularly, organizations can use that data to conduct real-time learning and make rapid adjustments to their programs. By tracking conversion rates between different steps of the process, organizations can build their institutional knowledge about where they are facing the greatest barriers to success, and develop strategies to address those challenges. 

Organizations can review outcomes on a recurring basis, again paying attention to factors such as source country, destination country, gender, ethnicity/nationality, industry, or others that may affect conversion rates. 

Areas with particularly low conversion rates may point to important opportunities for additional training, intervention, or resources. It may indicate an opportunity for gathering additional feedback from diverse stakeholders. It may also indicate the presence of barriers that may need to be addressed through advocacy to employers, service providers, and/or government agencies.

Continuously improve programs based on feedback from diverse stakeholders 

A unique feature of displaced talent mobility is the breadth of actors engaged in its implementation. Effective monitoring and evaluation exercises should gather input from all affected parties. Important actors that can provide feedback on the strengths and weaknesses of a program include:

• Displaced people (including family members)
• Employers
• Service providers or partner organizations involved in the process
• Government actors or bodies involved in the process
• Staff/ colleagues involved at all stages of the process

Each actor is likely to have a unique set of interests and needs, and each can provide valuable perspective. Each actor may also have different feedback depending on where they are in the process. For example, a displaced person who has registered in the Talent Catalog but not yet been invited for an intake may have key feedback that varies significantly from an individual who has secured a job through the program. 

Tactics for gathering feedback in an ongoing manner include: 

• Incorporating the opportunity to participate in anonymous surveys in all electronic communications 
• Conducting intermittent feedback surveys
• Conducting in-depth interviews with individuals 
• Conducting focus groups 
• Incorporating opportunities for free-form feedback into formal impact survey mechanisms

At certain times throughout the program cycle, it may be useful to invite external evaluators to collect qualitative feedback. This may allow program participants to share feedback they may be uncomfortable sharing directly with program implementers. 

Building an evidence base about the impact of displaced talent mobility

Beyond tracking program metrics, organizations facilitating displaced talent mobility have an opportunity to gather essential data about the quantitative and qualitative impact that these programs have on the lives of displaced people as well as employers. 

Comprehensive and longitudinal participant surveys are an essential tool for capturing impact data. By administering the same survey before and after a displaced person relocates for work, organizations can begin to measure the full scope of the impact on family’s lives, from income to perceived safety and security, access to healthcare, access to education, and more. By surveying after a visa has been granted, at 6 months after arrival, and at 1 year after arrival, and potentially longer intervals, organizations not only measure how a family’s well-being changes over time but can also respond to feedback that arises in real-time. 

Through aggregating and sharing data between labour mobility organizations, lessons learned can be shared. Aggregating data can be used as a tool for advocacy for labour mobility pathways.

Privacy and data protection

When collecting data from beneficiaries of displaced talent mobility programs it is critical to respect the right to privacy and ensure protection of personal data.

Organizations collecting data and conducting monitoring and evaluation activities in relation to labour mobility programming should follow UNHCR’s Guidance on Protection of Personal Data of Persons of Concern. Organizations should have privacy and data protection policies in place to ensure adherence to best practice and relevant laws, and these policies should be publicly available.