In the post Make Money as a Search Engine Evaluator – Part 1, we discussed how becoming a Search Engine Evaluator (SEE) could provide additional income working from home.
However, some employers require you to pass an online assessment before offering you a position.
What Happens After Applying?
If your application made the cut, you’ll receive an email containing a link to the exam, a copy of the General Guidelines, and Lionbridge candidates will also get access to an exam preparation portal.
You have seven days to study the General Guidelines and complete all three parts of the test.
Referring to the General Guidelines during the test is allowed, but don’t let that fool you into thinking the assessment will be easy.
The tests have been designed to assess understanding of the General Guidelines, and passing will take more than merely regurgitating them.
My top tip is read the General Guidelines at least twice, and know the gist of how it wants you to deal with specific things, including the relevance of a user’s location, what constitutes a high/low-quality webpage, and how to identify Your Money Your Life (YMYL) pages.
You should follow the rules and concepts presented and not use your personal judgment. Pay particular attention to the practical task examples documented in the General Guidelines.
As mentioned, Lionbridge candidates are given access to an exam preparation portal. You should know that your activity on this portal is noted in conjunction with your exam performance. As such, it’s advisable to go through the webinars and task simulations before starting the exam.
There are three parts to the assessment. Starting with a multiple choice Theory test, followed by a practical test on Page Quality, and then another practical test for assessing Needs Met. You only advance to the next test by passing the previous one.
After completing a test, you should receive an email, confirming your result, within ten minutes of submitting.
The table below shows a suggested schedule, which, if followed, will ensure plenty of time to complete the assessment.
|Day 1-2||Read the General Guidelines and set up your phone for the Needs Met part of the assessment|
|Day 3||Do Test 1 – the multiple choice Theory part of the assessment|
|Day 4||Re-read the Guidelines focussing on the Page Quality section|
|Day 4-5||Do Test 2 – the Page Quality part of the assessment|
|Day 6||Re-read the Needs Met section of the General Guidelines|
|Day 6-7||Do Test 3 – the Needs Met part of the assessment|
Part 1 – Theory Test
This examines your theoretical understanding of rating concepts and your reading comprehension skills. You are allowed to search the General Guidelines (using Ctrl & F) while doing the test.
There are 132 true or false questions over four pages. Take care when reading the questions, as some are worded to confuse you.
At the bottom of your screen, you’ll see two buttons:
- Complete – to submit answers and move onto the next set of questions
- Submit & Stop – to submit answers and have a break
Never leave an incomplete task for more than 1 hour. If you need to stop, complete all the questions on the page and then click the Submit & Stop button.
Once you’ve finished all 132 questions you’ll see this message:
No tasks are currently available. Please try again later.
If you passed, Test 2 – Page Quality would be available shortly after.
Part Two – Page Quality
For this part of the assessment, you are tested on your ability to rate the quality of web pages.
On starting you will be presented with a form that has a link to the web page, a series of questions about the page, and boxes arranged in a grid for you to tick the most appropriate description of the:
- Main content
- Quality of the website information
- Website reputation
- Expertise, authority, and trustworthiness of the author/website
Ticking a particular box highlights it in a certain colour – which corresponds with its quality rating for the relevant category. The ratings are:
- Dark Blue = Lowest rated
- Medium Blue = Low
- Grey = Medium
- Light Green = Medium to High
- Medium Green = High
- Dark Green = Highest
A colour key near the top of the page gives guidance on how to use the colours to assign the overall Page Quality rating.
It says things like, “mostly grey boxes should be assigned an overall rating of Medium,” or “mostly green, and at least one dark green should be given an overall rating of Highest.”
Follow the colour key to decide on the overall quality of the page.
There will be seven web pages to assess, and you should expect 2-3 low-quality web pages, 2-3 medium-quality web pages, and 2 high-quality web page. If your results vary from this, i.e., you have four low-quality pages, then something has gone wrong.
Remember to rate the quality of the web page only, and not the website as a whole. This can be confusing to grasp, as you will need to navigate parts of the website, such as an about us page, to inform your rating decision. But your goal is to focus only on the quality of the web page in the given link.
Also, you can write comments to supplement your review of the webpage. A recommended format is:
- state the purpose of the page
- say what informed your decision to rate the Main Content/website info/reputation/E-A-T as you did
- justify how you gave your overall rating, i.e., mostly greys justify a rating of Medium.
Part 3 – Needs Met
You will assess how well the search results have met the needs of a user query. There are seven individual queries to do. Each query has 8-12 search result blocks that require your assessment.
A result block is a summary of the web page that Google has found for the query. There’ll be a title, a URL, and some wording.
An Example of a Web Search Result Block (WSRB)
There are also Special Content Result Blocks (SCRB) that serve the same function, but they differ in that they may feature pictures, graphics, additional links, or reviews.
An Example of a Special Content Result Block (SCRB)
A user query merely is what someone would type into Google, or speak into their phone. Your job is to rate each result block for how well it meets the user query. The scale runs from “Fails to Meet” to “Fully Meets.”
Each result block has a link to the web page being assessed, which you must click and assess on your phone unless the result block is an SCRB, in which case, you would evaluate the SCRB on your laptop.
Understand the user intent, and use this insight to guide you on how well the web page meets the intent. If you’re unsure about what the user intent is, open a new window and do some additional research on the query. All intents will fit into one of the six categories below.
- Know queries – refer to questions that have no definitive answer and cannot be answered quickly, i.e., how do I get a give up smoking?
- Know simple – are questions that have a short definitive answer, i.e., who is the founder of Amazon?
- Do queries – are when the user wants to achieve something, e.g., buy a pair of shoes.
- Device action query – to use the phone to perform a specific action, e.g., set timer for 10 minutes.
- Website query – to go to a webpage, e.g., ebay.co.uk.
- Visit in person query – to navigate go to a business location.
NOTE: scoring a Fully Meets rating means the result is the perfect response. It is fully satisfying for the query and no other result is needed. With that mind, only know simple, device action, and website queries can ever be Fully Meets. And they are rated Fully Meets if the result is within a Special Content Result Block. The exception to this is website queries, which can be rated Fully Meets even if the result block is not a Special Content Result Block.
When assigning the Needs Met rating, think about the user’s location.
Consider the query [coffee shop]. A result of a coffee shop 30 miles away from the user’s location would be a Fails to Meet, as people don’t usually travel 30 miles for coffee.
This refers to a situation where there are two equally likely intents.
For example, the query [John Lewis] may mean the user wants to:
1) Visit a nearby John Lewis, and requires navigation information, or
2) It could also mean the user wants to go to the John Lewis website.
As the query [John Lewis] lacks the information to make this call, we assume both the website and visit-in-person intent are equally likely. Dual intent result blocks should be rated Highly Meets+.
Be strict with any Your Money Your Life (YMYL) pages. Think about how expert and trustworthy the author is.
Types of Interpretation
Some queries can mean different things to different people, consider if there are dominant, common, or minor interpretations. Results that match with a dominant interpretation should have a higher Needs Met rating than results for minor interpretations.
Factor into your rating how well the page works on your mobile, is it hard to see? Do the buttons work? Is navigating the screen difficult? Websites not optimized for mobile should receive a lower Needs Met rating.
An Example of a Needs Met Task
The E-A-T slider should be dealt with by asking yourself whether this is an authoritative source, and are they trustworthy. There’s no need to do more than this when deciding your E-A-T Rating.
Leapforce candidates receive an email with an overall result on the exam due date whereas Lionbridge candidates get their result a day after the exam due date.
If you’ve passed, Leapforce requires you to sign some electronic forms, and you’ll be working within four days. However, Lionbridge applicants need to complete a training course before being granted access to their platform.
Best of luck. Be sure to leave a comment or get in touch if you have any questions.