Before going over the hard skills needed to measure impact, let us quickly go through what impact measurement is just so we are on the same page.

Impact measurement is causal, meaning, we are trying to identify the drivers of a particular outcome or impact. To hammer this home with an example, consider “A resume building workshop”. The objective of the workshop is to improve the quality of your resume so you receive more interview invites from companies to increase the chances of you getting a job. Measuring impact in this scenario would mean that you need to know if the resume building workshop resulted in you getting more number of interview invites.

With this in mind, let us talk about the hard skills and technologies needed to keep track of impact, number of interviews students get as a result of the workshop, over time.

This is the list of skills and technologies required (This is not an exhaustive list and doesn’t cover complex scenarios such as regression analysis).

1. Ability to track uniquely each participant of the workshop
2. Ability to collect data from these participants at different points in time
3. Ability to join data sets from points 1 and 2 (we’ll talk about this using a simple use case)
4. Ability to derive insights and tracking it over time

Don’t worry we will go over each one of these through an example. This is going to be easy.

Track uniquely each participant of the workshop

Lets continue the example we discussed above.

To measure the impact of resume building workshop on its participants we need to keep track of the participants in someway, because we need to know whether these participants are receiving any interview invites at all?

Lets say we use a simple spread sheet to track the participants. The data looks like so,

Student Name	Email	Gender
John	john@word.com	Male
Tom	tom@gmail.com	Male
Ashley	ashley@yahoo.com	Female

You may collect more information on the participants, for the sake of simplicity we just have 3 (Name, Email and Gender).

The email makes the participants unique. Its mandatory to uniquely identify a participant in some way. You could use an email or a random “id” but uniqueness is important. We’ll explain the why in just a bit, hang in there.

Lets go to skill 2…

Ability to collect data from these participants at different points in time

Now that we have the basic registration information from the participants, we need to collect some data on the outcomes of our workshop.

The Outcome of our workshop is “Multiple interview invites for the participants”.

The data for number of interviews can come from any of the sources feedback surveys, interviews etc

Lets imagine that we are conducting a feedback survey on the participants to gather this data. This survey is being conducted post workshop. The data from the survey may look like so,

Email	Can you give us feedback on the workshop so far and the impact it had on your job search and interview calls?	How many interview calls did you get?
john@word.com	The workshop has been very helpful in knowing how to design impactful resumes and to highlight my strengths. Hopefully I will start getting interview invites soon.	No calls yet
tom@gmail.com	No response	No calls yet
ashley@yahoo.com	The workshop is great and I have been having calls with ,amu industry connections and one of my connections helped me in getting an interview call.	1 call

Now, may be we want to check in on these participants 3 months after the workshop. This helps keep track of outcomes over longer periods. In the 3 -months survey we could ask the same questions or have additional questions.

The data from the survey may look something like this

Email	Can you give us feedback on the workshop so far and the impact it had on your job search and interview calls?	How many interview calls did you get?	Did you get a Job?
john@word.com	The workshop has been very helpful in knowing how to design impactful resumes and to highlight my strengths. Hopefully I will start getting interview invites soon.	2 calls	Yes
tom@gmail.com	Fantastic workshop and I was able to practically apply all the concepts taught in the workshop and even helped me in confidently navigating through the interviews as well.	4 calls	Yes
ashley@yahoo.com	The workshop is great and I have been having calls with ,amu industry connections and one of my connections helped me in getting an interview call.	2 calls	Yes

Now, we have 3 datasets in total containing information on the same participants

1. Participant registration data
2. Post workshop survey
3. 3-months check-in survey

To effectively track change over time we need the ability to bring these datasets together.

This brings us to skill 3….

Ability to join data sets

Why join datasets at all?

Lets continue with our example.

Let us say we want to keep track of

Number of participants with interview invite immediately after the workshop vs 3-months after the workshop

With data sitting in different surveys, it is impossible to answer the question above. This is where ability to join datasets comes into picture.

Organizations handle it in a variety of different ways,

• Ignore the problem (yes you read that right) – this could be a solution to the problem if the problem isn’t that big for you
• Manually handle this on excel or google sheet
• Use lookups etc on Google/Excel
• Use SQLs (our preferred way to join datasets, explaining SQL for this is beyond the scope of this write up)

And this is where identifying the participants uniquely becomes such a crucial thing. Imagine you didn’t have a way to identify the participants uniquely, how would you bring the data together?

Our merged data looks something like this

Email	Can you give us feedback on the workshop so far and the impact it had on your job search and interview calls?	How many interview calls did you get?	Stage
john@word.com		No calls yet	Post-workshop
tom@gmail.com		No calls yet	Post-workshop
ashley@yahoo.com		1 call	Post-workshop
john@word.com		2 calls	3-months check-in
tom@gmail.com		4 calls	3-months check-in
ashley@yahoo.com		2 calls	3-months check-in

With this dataset we can easily answer the question,

Number of participants with interview invite immediately after the workshop vs 3-months after the workshop

This brings us to the final skill, ability to derive insights…

Derive insights and track over time

With the merged data, one could easily keep track of the outcome over-time. Organizations generally need excel or google sheets or could use a more sophisticated BI solutions.

Example of an insight from the merged data

With an ad-hoc excel/google sheet analysis this won’t be a continuous process and every-time there is new data collected you likley would have to repeat this process. It is ok if the number of records are less.

I case where the data is collected more frequently a system setup to continuously collect and track data is extremely helpful.