Search engines often possess the kind of elusive charm reserved only for the subjects of mythical folklore. This can prove to be problematic for business owners, who need to have a basic understanding of how this sophisticated technology works, and how they can use it and manipulate it to the benefit of their business. In this article we’ll briefly explain how search engines work, and how you can use this knowledge to your advantage as a business owner.
Search engines are all slightly different
The first thing to understand about how search engines work is that they all operate slightly differently. They each have their own software and programs that run on various formulas (known as algorithms) which determine the relevancy, speed and nature of search results. In essence, however, all search engines perform three main tasks. Firstly, they do something known as ‘crawling’. This involves assessing and analysing online content available to them. They then categorise this content (this is known as indexing). Finally they decide which content is going to be relevant and useful to searchers (this is known as ranking).
What are ‘bots’, and how do they work?
Search engines use computer programs known as ‘bots’ (short for robots) that crawl the internet examining and assessing content – including web pages, images and videos. Bots are also known as crawlers, or spiders. These sophisticated systems relentlessly hop from page to page following links – constantly visiting new pages and revisiting old ones in order to find fresh content to index. The way in which these computer programs work is little known – a closely-guarded secret only shared in the smallest of circles of upper echelons at the likes of Google and competitors such as Bing.
What is indexing – and why do I need to know about it?
After crawling comes indexing. During the second part of the process the huge list of content amassed by the bots is maintained and consistently added to, growing as new content emerges on the web. This immense index serves as a source of information for all search results arise from – this is the list used when searches are generated. However it’s important to note that not everything bots find makes the cut and finds its way onto their giant index. Search engines have specific criteria – and any pages or content not meeting this these standards will not be indexed. Bots are on the lookout for certain bad practices employed by businesses (sometimes innocently) more often than you may think – inadvertently harming their chances of being found, noticed and engaged with online.
How can I be sure that I’m indexed?
Following best practice guidelines for SEO is very important – both from a search perspective and user viewpoint. Non-indexing is most often down to bad practice – such as directly copying content (either on your own site, or from another), cramming in keywords at every opportunity and also failing to follow development protocol, like ensuring your site is responsive and accessible. Why is this the case? These measures are in place to make sure that search results are as relevant as possible, and bots prioritise only one page of duplicate content, the one they deem to possess the highest authority. Google also places penalties upon (or can completely remove) content from its index that it deems to violate the ‘rules’ or engage in bad practice.
What determines my website’s level of ranking?
Once crawling and indexing has been completed, ranking takes place. This is the order in which search results are displayed when you enter your key terms into a search engine. When you hit ‘search’, it compares the words and phrases you’ve used with the results in its index. There may be thousands – millions of results generated. So how are these sorted in order of relevance and importance? This is largely what sets Google apart from its competitor search engines – and is the most secretive part of their ‘formula’. Although following SEO good practice and employing a team of experts is undoubtedly the place to start, few people know exactly how the rankings are generated. If you’re interested in boosting your profile online, you can chat with us today or check out our related articles here.
Follow our blog for the next instalment of our Beginner’s Guide to the Web series.