Description

Abstract—LinkedIn is one of the most popular social networks for professionals. Many of us rely on LinkedIn to expand connections and find new opportunities after graduation. In this project, we are going to study one task of LinkedIn – the searching function. The objective is to make a better interface design that the users can have a more relevant result on the LinkedIn search result page and accomplish the task.
1 QUALITATIVE EVALUATION
In this section, we will evaluate the Wizard of Oz prototype – using the voice signature to identify who you are talking with. Here is a quick recap of what the prototype looks like. As shown in Figure 1, when a user is talking to another person, the interface should be able to use the voice signature to identify the person’s LinkedIn profile and make the suggestion to the user.

Figure 1—Wizard of Oz prototype for identify who you are talking to
1.1 Evaluation Plan – Interview
To perform the qualitative evaluation of the prototype, we will use the interview method. The participants are friends, family and classmates. They are chosen if they are regular LinkedIn users and in the age group both recent graduates and also seasoned professionals. Friends and family are easy to recruit. Classmates will be chosen from other OMSCS group projects. The interview will be conducted remotely except for families using Zoom or other tools. The response
will be noted in the notepad for further aggregated analysis.
1.2 Actual Content of Evaluation
Because this is a new interface that people have no experience with before, we will provide a description of what the interface is capable of. The interview shall be conducted using semi-structured questions to allow interviewees to express concerns about the new interface.
Interview Questions:
1. How often would you like to know the LinkedIn profile of the person you are talking with?
2. How often do you forget your connections and spend much time searching people using different combinations of names and filters?
3. How helpful would it be if you can search for someone without even knowing the name?
4. What would you do if you want to know where your new colleague worked before?
5. Would it be helpful if instead of you initiate the searching, the LinkedIn app can suggest the profile of the person you are talking with?
6. What could be your concern if the LinkedIn search function can help you to identify who you are talking with?
7. (If yes for the above question), do you think if you are allowed to turn off searchability of your own profile can help to mitigate the privacy concern?
8. On which device would you prefer to see this function?
1.3 How the Evaluation will address requirements
Refer to the Table 1 of our requirements, our survey can help to identify gather data about functionality, usability and learnability. We will be able to understand whether the interface is easy to use, little time to understand, and able to accomplish their desired task.
Also, because we are semi-structured questions, the users can help to make suggestions about the existing interface. That will make the interface more user friendly and flatten the learning curve of a new user. Question about device compatibility is asked, but most likely the interface is more useful in the mobile device.
Table 1—Requirements for the LinkedIn Search Function
Functionality Usability Learnability Compatibility Efficiency
Requirements User can find target in the search result It is easy for the user to search It takes little time for user to learn interface The interface works on
both mobile and website The interface can also help expert user to accomplish task efficiently
Evaluation How many
times user can find target in result Interface is rated as easy to use for most users How fast user start to use the
interface Users can accomplish task on both mobile and website Users can finish task faster with shortcuts
2 EMPIRICAL EVALUATION
In this section, we will evaluate the text prototype – using predictive modelling to identify the most relevant result based on the search query and user profile. Here is a quick recap of what the prototype looks like. As shown in Figure
2, when a user clicks the search button, the interface should be able to use AI models to identify the most relevant LinkedIn profiles and rank the search results accordingly.

Figure 2—Use predictive modelling to give the most relevant search results
2.1 Define control and experimental conditions
In this experiment, we will define our hypothesis as below:
Null Hypothesis H0: The new interface does not increase the CTR (Click-ThroughRate) in the search result page. It means the user is not more likely to find what he wants on the search result page.
Alternative Hypothesis H1: The new interface does increase the CTR (ClickThrough-Rate) in the search result page. It means the user is more likely to find what he wants on the search result page.
2.2 Experimental Method
Table 2—Empirical Tests Selection
IV DV Treatments Recommended
Categorical Nominal 2 or more Chi-Squared test
Categorical Ordinal 2 KS test
Categorical Interval or Ratio 2 Student’s t-test
Categorical Ordinal 3 or more Chi-Squared test
Categorical Interval or Ratio 3 or more ANOVA
Categorical Binomial 1 or 2 Binomial test
Interval or Ratio Interval or Ratio 2 Linear Regression
We will use between-subjects analysis – the A/B testing method to verify if the alternative design is significantly better. Because our data is a ratio (CTR), based on Table 1, (Joyner, 2016b) it is better for us to use Student’s Test method. Users are randomly selected into 2 groups (Group A and Group B), Group A will see the original interface, and Group B will see the new interface, which uses predictive modelling to better rank results. If we can see the final p-value < 0.05, that means we can successfully reject the null hypothesis and conclude that the new interface is significantly better at giving users better results.
2.3 What lurking variables might confound the data
According to Joyner (Joyner, 2016c), we have below tips to conduct a successful Empirical Evaluation.
1. Control what you can, document what you can’t
2. Limit your variables
3. Work backwards
4. Script your analyses in advance
5. Pay attention to power
3 PREDICTIVE EVALUATION
In this section, we will evaluate the wireframe prototype – using a photo to do facial recognition and match the LinkedIn profile. Here is a quick recap of what the prototype looks like. As shown in Figure 3, a user is able to upload a photo instead of searching for a name. The interface then should be able to use the facial recognition model to identify the most relevant LinkedIn profiles.

Figure 3—Wireframe of facial recognition in LinkedIn search function
3.1 Selecting Type of Task Analysis
Because this is a new interface for the searching task, we would like to use Cognitive Walkthrough rather than the GOMS model. GOMS model is more suitable to evaluate an interface assuming the users are all expert users. By using the Cognitive Walkthrough, we can possibly identify the advantage of the new interface and potential cognitive barriers.
3.2 Specific Tasks to be Addressed
Cognitive task analysis adopts the predictor view of a human’s role in the system. It generally follows below common sequence: (Joyner, 2016d)
1. Collecting preliminary knowledge
2. Identify knowledge representations
3. Apply focused knowledge elicitation methods
4. Analyze and verify data acquired
5. Format results for the intended application
The goal of the user is to find the correct LinkedIn profile of the photo uploaded. It can be a single photo, a group of photos, or taking a real-time photo. The interface will be able to match that photo to the database and find the corresponding LinkedIn profiles of people in the photo.
There are 3 sub-tasks for the user’s top-level task (goal):
1. Uploading the photo for searching the LinkedIn profile
2. Taking a real-time photo for searching the LinkedIn profile
3. Find the correct profile in the search result page
The operators available in the interface:
1. Clicking the LinkedIn search box
2. Clicking the search by photo button
3. Clicking Photo button
4. Taking a photo using the built-in camera interface
5. Clicking Upload button
6. Clicking “Select Photo” button
7. Uploading photo
8. Scrolling through the search result page
9. Selecting filters in the search result page
10. Clicking profile in the search result page to redirect to the profile page
4 PREPARING TO EXECUTE
In this section, we will select 2 out of 3 evaluations from the last section to execute. According to Joyner, below are the advantages for different methods:
(Joyner, 2016a)
Table 3—Advantages for different evaluation methods
Advantages Qualitative Empirical Predictive
Informs ongoing design decisions Yes – Yes
Investigates the participant’s thought process Yes – Yes
Draws conclusions from actual participants Yes Yes –
Identifies provable advantages – Yes –
Provides generalizable conclusions – Yes –
Does not require any actual users – – Yes
We will choose Qualitative Evaluation (Interview) and Predictive Evaluation
(Cognitive Walkthrough) for our execution. The reason is that it is difficult to conduct an Empirical Evaluation for our interface and collect enough data points to draw a conclusion. Especially, the interface requires an AI model in place for testing, that development is not feasible to deliver within time constraints.
5 REFERENCES
[1] Joyner, David (2016c). 5 Tips: Empirical Evaluation. url: https://classroom. udacity.com/courses/ud400/lessons/9753906972/concepts/97594129830923.
[2] Joyner, David (2016d). Cognitive Task Analysis. url: https://classroom. udacity.com/courses/ud400/lessons/9438503139/concepts/94303133490923.