Tips For Search Engine Evaluators

By |2018-10-07T19:36:43+00:00August 13th, 2018|Miscellaneous|

As a newly employed Search Engine Evaluator you may feel uneasy about what lies ahead.  So, to help you settle in, this post will outline some key points to ensure success in your new role.


Types of tasks

The type of tasks you do in any given work window varies depending on what the Google engineers release.  The role involves dealing with many types of tasks, including Page Quality, Needs Met, Side by Side, as well as ExperimentalExperimental tasks can include:

  • Assessing speech quality
  • Checking recipes for accuracy
  • Judging image satisfaction
  • Having conversations with the Google Assistant

But given the developmental nature of technology, Experimental tasks can cover just about anything.  And occasionally you’ll come across one that you haven’t seen before.  It goes without saying that taking your time to understand the requirements will pay dividends, even if that means over-running your time allocation.

Side by Side

The purpose of Side by Side tasks is to give your input on Google algorithm changes. 

Just like a standard Needs Met task, you are required to assess how well each result block meets the user’s needs, but it differs in that the result blocks are grouped into left and right sets. 

Your job is to decide which set is overall better for a user, the extent to which it is better, i.e. the same, slightly better, better, or much better, and to give your comments.

Side by Side ratings form part of your quality review each month, meaning the accuracy and quality of your Side by Side work determines your month-end quality score.

Page Quality

Page Quality is about rating a webpage according to how good it is.  This involves describing the purpose of the page, checking for porn and whether it loads and if the page is in a foreign language.  You’ll also need to make a judgment on whether the webpage is malicious, harmful, or lacking in purpose.

Your Money Your Life (YMYL) pages have the potential to negatively impact someone’s life, so being able to identify them is important.  YMYL pages should be rated more strictly than a non-YMYL page.  For example, a YMYL shopping page with no contact information on the website is judged as low quality.  Whereas a non-YMYL page with no contact information, such as a family photo page, would not be rated as harshly.

As with Side by Side ratings, Page Quality tasks also form part of your monthly review, so it’s important to take your time and evaluate Page Quality tasks to the best of your ability.

Needs Met

Needs Met involves assessing how well a search result meets the needs of a user.  Doing this relies on correctly interpreting the likely user intent, and using this to assess the usefulness of each search result.

Only know simple queries, website queries, and action queries, where the result is in a Special Content Result Block (SCRB), can be Fully Meets.

If a result block is an excellent fit for the query but is not within an SCRB, the maximum it can score is Highly Meets.


Practice makes…

Once you have access to the task portal you can do live tasks.  However, in order to maximize your first quality score, it’s a good idea to do simulation tasks and the training modules before starting the live tasks. 

Investing this time not only improves your confidence but doing the simulation tasks will also upgrade your status in the system.  This means you’ll have access to more Experimental tasks, and so can potentially work more hours.


Quality score

Your performance on Side by Side and Page Quality tasks will determine your quality score.

After 3 months, a rater is expected to score 85%.  Should your quality score consistently fall below expectation, your account will be flagged and access to tasks will be restricted.

If this happens, you’ll receive notification from the vendor, who will likely give you the chance to improve your score.  On being notified, it would be wise to spend time going through the training modules.

Time spent reading, self-educating or taking breaks between tasks is not billable.


Recording your time

You are paid according to the number of tasks you do times by the expected time to complete each task.

It’s recommended you use a handwritten tally chart to record the types of task you do, and the expected time.  Simply transfer this information into your online timesheet at the end of the day.

3rd party automatic timekeeping apps do exist.  They work as add-ons to the platform, but vendors do not approve given the confidential nature of the work.

At the end of the calendar month, your timesheet will be reviewed by looking at the time you billed versus the actual time spent doing the tasks.  If you were quicker than the expected time, you will be asked to amend your timesheet to reflect this.  There is no penalty for taking longer than the expected time.



Task availability

Generally, vendors offer around 2 hours of work each day.  But you should remember this is not guaranteed, and in reality, work schedules are inconsistent.  For this reason, it’s important not to rely on Search Engine Evaluation as your only source of income.

Tasks are allocated on a first come first serve basis. That being so, be prepared to start work as soon as the notifications come.

Leapforce workers have a task availability tool, which can help plan time by identifying periods of high task availability.  Unfortunately, there is no equivalent for Lionbridge workers. 


Account restriction

A quality review is implemented if your quality score falls consistently below expectation.  Being under review will restrict the number of tasks available to you.

Should this happen, it takes a proactive approach to get unrestricted.  Start by asking for written feedback on areas for improvement.  Whilst waiting for the feedback, make use of your time by going to the Community Portal.  Here you can view your past work and do simulation tasks.

More often than not, it’s the Side by Side tasks that are the most troublesome.  Ask yourself which side has the most useful results for a user and why.

Once you have your feedback, reply with a plan on the steps you’ll take to improve your performance.

About the Author:

Howdy! I'm James. My role is to help you make money online. I'll act as your guide to ensure that you have the necessary tools and knowledge to add additional sources of income to your life.

Leave a Reply

%d bloggers like this: