How To Pass The Search Engine Evaluator Assessment- Part 2

By |2019-02-18T23:21:25+00:00July 3rd, 2018|Miscellaneous|

In the post Make Money as a Search Engine Evaluator – Part 1, we discussed how becoming a Search Engine Evaluator (SEE) could provide additional income working from home. 

However, some employers require you to pass an online assessment before offering you a position.

 

What Happens After Applying?

If your application made the cut, you’ll receive an email containing a link to the exam, a copy of the General Guidelines, and Lionbridge candidates will also get access to an exam preparation portal. 

You have seven days to study the General Guidelines and complete all three parts of the test. 

Referring to the General Guidelines during the test is allowed, but don’t let that fool you into thinking the assessment will be easy. 

The tests have been designed to assess understanding of the General Guidelines, and passing will take more than merely regurgitating them.

My top tip is read the General Guidelines at least twice, and know the gist of how it wants you to deal with specific things, including the relevance of a user’s location, what constitutes a high/low-quality webpage, and how to identify Your Money Your Life (YMYL) pages.

You should follow the rules and concepts presented and not use your personal judgment.  Pay particular attention to the practical task examples documented in the General Guidelines.

As mentioned, Lionbridge candidates are given access to an exam preparation portal.  You should know that your activity on this portal is noted in conjunction with your exam performance.  As such, it’s advisable to go through the webinars and task simulations before starting the exam.

 

Assessment Format 

There are three parts to the assessment.  Starting with a multiple choice Theory test, followed by a practical test on Page Quality, and then another practical test for assessing Needs Met.  You only advance to the next test by passing the previous one. 

After completing a test, you should receive an email, confirming your result, within ten minutes of submitting.

The table below shows a suggested schedule, which, if followed, will ensure plenty of time to complete the assessment. 

Time Frame Action
Day 1-2 Read the General Guidelines and set up your phone for the Needs Met part of the assessment
Day 3 Do Test 1 – the multiple choice Theory part of the assessment
Day 4 Re-read the Guidelines focussing on the Page Quality section
Day 4-5 Do Test 2 – the Page Quality part of the assessment
Day 6 Re-read the Needs Met section of the General Guidelines
Day 6-7 Do Test 3 – the Needs Met part of the assessment

 

Part 1 – Theory Test

This examines your theoretical understanding of rating concepts and your reading comprehension skills.  You are allowed to search the General Guidelines (using Ctrl & F) while doing the test. 

There are 132 true or false questions over four pages.  Take care when reading the questions, as some are worded to confuse you.

At the bottom of your screen, you’ll see two buttons:

  • Complete – to submit answers and move onto the next set of questions
  • Submit & Stop – to submit answers and have a break

Never leave an incomplete task for more than 1 hour.  If you need to stop, complete all the questions on the page and then click the Submit & Stop button.

Once you’ve finished all 132 questions you’ll see this message:

No tasks are currently available. Please try again later. 

If you passed, Test 2 – Page Quality would be available shortly after.

 

Part Two – Page Quality

For this part of the assessment, you are tested on your ability to rate the quality of web pages.  

On starting you will be presented with a form that has a link to the web page, a series of questions about the page, and boxes arranged in a grid for you to tick the most appropriate description of the:

  • Main content
  • Quality of the website information
  • Website reputation
  • Expertise, authority, and trustworthiness of the author/website

Ticking a particular box highlights it in a certain colour – which corresponds with its quality rating for the relevant category.  The ratings are: 

  • Dark Blue = Lowest rated
  • Medium Blue = Low
  • Grey = Medium
  • Light Green = Medium to High
  • Medium Green = High
  • Dark Green = Highest

A colour key near the top of the page gives guidance on how to use the colours to assign the overall Page Quality rating. 

It says things like, “mostly grey boxes should be assigned an overall rating of Medium,” or “mostly green, and at least one dark green should be given an overall rating of Highest.”   

Follow the colour key to decide on the overall quality of the page.

There will be seven web pages to assess, and you should expect 2-3 low-quality web pages, 2-3 medium-quality web pages, and 2 high-quality web page.  If your results vary from this, i.e., you have four low-quality pages, then something has gone wrong.

 

Assessment Tips

Remember to rate the quality of the web page only, and not the website as a whole.  This can be confusing to grasp, as you will need to navigate parts of the website, such as an about us page, to inform your rating decision.  But your goal is to focus only on the quality of the web page in the given link.

Also, you can write comments to supplement your review of the webpage.  A recommended format is:

  • state the purpose of the page
  • say what informed your decision to rate the Main Content/website info/reputation/E-A-T as you did
  • justify how you gave your overall rating, i.e., mostly greys justify a rating of Medium.

 

Part 3 – Needs Met

You will assess how well the search results have met the needs of a user query.  There are seven individual queries to do.  Each query has 8-12 search result blocks that require your assessment. 

A result block is a summary of the web page that Google has found for the query.  There’ll be a title, a URL, and some wording. 

 

An Example of a Web Search Result Block (WSRB)

 

 

 

 

 

 

 

 

 

There are also Special Content Result Blocks (SCRB) that serve the same function, but they differ in that they may feature pictures, graphics, additional links, or reviews.

 

An Example of a Special Content Result Block (SCRB)

 

 

 

 

 

 

 

A user query merely is what someone would type into Google, or speak into their phone.  Your job is to rate each result block for how well it meets the user query.  The scale runs from “Fails to Meet” to “Fully Meets.”

Each result block has a link to the web page being assessed, which you must click and assess on your phone unless the result block is an SCRB, in which case, you would evaluate the SCRB on your laptop.

 

Assessment Tips

User Intent

Understand the user intent, and use this insight to guide you on how well the web page meets the intent.  If you’re unsure about what the user intent is, open a new window and do some additional research on the query.  All intents will fit into one of the six categories below.

  • Know queries – refer to questions that have no definitive answer and cannot be answered quickly, i.e., how do I get a give up smoking?
  • Know simple – are questions that have a short definitive answer, i.e., who is the founder of Amazon?
  • Do queries – are when the user wants to achieve something, e.g., buy a pair of shoes.
  • Device action query – to use the phone to perform a specific action, e.g., set timer for 10 minutes.
  • Website query – to go to a webpage, e.g., ebay.co.uk.
  • Visit in person query – to navigate go to a business location.

NOTE: scoring a Fully Meets rating means the result is the perfect response.  It is fully satisfying for the query and no other result is needed.  With that mind, only know simple, device action, and website queries can ever be Fully Meets.  And they are rated Fully Meets if the result is within a Special Content Result Block.  The exception to this is website queries, which can be rated Fully Meets even if the result block is not a Special Content Result Block. 

 

User’s Location

When assigning the Needs Met rating, think about the user’s location. 

Consider the query [coffee shop].  A result of a coffee shop 30 miles away from the user’s location would be a Fails to Meet, as people don’t usually travel 30 miles for coffee.

 

Dual Intent

This refers to a situation where there are two equally likely intents. 

For example, the query [John Lewis] may mean the user wants to:

1) Visit a nearby John Lewis, and requires navigation information, or

2) It could also mean the user wants to go to the John Lewis website.

As the query [John Lewis] lacks the information to make this call, we assume both the website and visit-in-person intent are equally likely.  Dual intent result blocks should be rated Highly Meets+.

 

YMYL

Be strict with any Your Money Your Life (YMYL) pages.  Think about how expert and trustworthy the author is.

 

Types of Interpretation

Some queries can mean different things to different people, consider if there are dominant, common, or minor interpretations.  Results that match with a dominant interpretation should have a higher Needs Met rating than results for minor interpretations.

 

Mobile Optimization

Factor into your rating how well the page works on your mobile, is it hard to see?  Do the buttons work?  Is navigating the screen difficult?  Websites not optimized for mobile should receive a lower Needs Met rating.

 

An Example of a Needs Met Task

Most result blocks will have a Needs Met Rating slider and an E-A-T Rating slider to complete. Focus most of your attention on getting the Needs Met rating correct. 

The E-A-T slider should be dealt with by asking yourself whether this is an authoritative source, and are they trustworthy.  There’s no need to do more than this when deciding your E-A-T Rating.

 

What Now?

Leapforce candidates receive an email with an overall result on the exam due date whereas Lionbridge candidates get their result a day after the exam due date.

If you’ve passed, Leapforce requires you to sign some electronic forms, and you’ll be working within four days.  However, Lionbridge applicants need to complete a training course before being granted access to their platform.

Best of luck.  Be sure to leave a comment or get in touch if you have any questions.

About the Author:

Howdy! I'm James. My role is to help you make money online. I'll act as your guide to ensure that you have the necessary tools and knowledge to add additional sources of income to your life.

4 Comments

  1. Alice Pillerová 12/11/2018 at 09:11 - Reply

    Hallo Samuel, thank you for your webpage. Im Alice. Im from the Czech Republic. Im trying to find job online. I can offer up to 20 hours a week and good knowledge of English. I tryed the Falcon project, but it has finished, before I joined. Im not interested in Lionbridge´s Map Analyst position. Do you have any idea, what should I do ? Are there other truthful companies offering online jobs? Thank you for your answer. Alice

  2. Bruno 25/02/2019 at 22:35 - Reply

    Hi Samuel, I failed on second part of the exam and I am a bit confused about one point: one webpage purpose was to share information about a tennis brand. The MC was okay but when I went through the website, I found that they had a shopping section and a bad reputation (don’t deliver). Also no information and customer service available. Due to that I rated it Lowest PQ, because I considered website YMYL although that particular webpage wasn’t. What’s your Opinion?

    • Samuel Wan 26/02/2019 at 19:04 - Reply

      Hi Bruno

      Sorry to hear that, but the good news is you’ll probably get another chance, in fact, I know of people who have had up to 4 chances.

      Based on what you say, you classed the webpage as YMYL even though the webpage in the link was only informational. Remember to focus on the page in the link they give.

      By the way, reputation should be informed by 3rd party sites, like better business bureau, wikipedia, etc., because companies generally don’t publish negative information about themselves on their own website.

      So in summary, yes when it’s a YMYL page, be harsh with the rating, but in this case, the page was not a YMYL page.

      Let me know if you have any more questions.

Leave a Reply

%d bloggers like this: