You have a website. Great! But do people find you? And if so, how? (Use analytics!)
A large portion of traffic to many website come from a referral by search engines. How do search engines know what to return when you do a search? That’s the trillion dollar question for many web developers and site owners.
Google and Bing — the major players in the search engine field — each uses their own algorithms. (Bing now provides the search results for Yahoo!) Yahoo! started as a web directory (Yahoo is an acronym for “Yet Another Hierarchical Officious Oracle” and started as “Jerry and David’s Guide to the World Wide Web” after founders Jerry Yang and David Filo). The web quickly grew too big to manually catalog and place into directories, which is where Google came in. At this point, most search engines looked at keyword (the search term) density, which can be easily manipulated. Larry Page and Sergey Brin developed a method for using backlinks (links from one website to another) as votes of sorts. The more links to a website, the more popular that website was deemed to be. This in essence was the start of Google and why Google dominated. Google has since added many other factors to their algorithm (estimated at over 200) such as authority, context, page load speed, original content and more. You can find a comprehensive list here.
The factors each search engine uses are proprietary, though some have stated what some of the factors are. But if the factors are known, can web owners manipulate the website to rank higher? Yes and it even has a name: Black Hat SEO. When backlinks were a highly weighted factor, people created link farms — pages with nothing but links. Then Google created the concept of authority — is the page legit and who is linking to the page? The SEO field became a game of cat and mouse.
So, the search engines are consequently and frequently tweaking their algorithms to prevent abuse. Google recently announced the roll-out of Penguin 3.0, the latest algorithm which filters out poor quality sites. Changes are summarized here.
My take: look at what Google (or Bing) is trying to do. People use Google (or Bing or Yahoo!) because the results provided are relevant to the user’s search. If you use good content on your site, you will attract the users you seek. If you start playing games to attract customers (bad backlinks, keyword stuffing, unnatural copy), you will forever be playing cat and mouse with the search engines. And you may get banned, which they do to egregious violators. Really, just use good practices (such as proper page titles), make normal copy and if you have good content, the search engines will find out and rank you highly. They want to. And you won’t have to worry when Google rolls out the next version of Penguin or Panda.