Keeping an Eye on Search Engine Diversity

December 24th, 2008

Last week, I came across two fascinating stories that show some of the problems relating to search engine diversity. Chris Soghoian writes how Google’s restrictive Adwords policy stands in the way of informing searchers about public data about political donations. Paul Alan Levy comments on a story about a company called Lifestyle Lift that tries to lead users away from a critical website by optimizing its presence on the Google search result page.

Research shows that the majority of search engine users stay on the first page of results and usually click on the first few results only. This increases the importance of being in the top of the organic results. In addition to offering a competitive way to be found in the organic results, search engines allow organizations to bid on search terms which result in sponsored results. This is the dominant search engine business model. Sponsored links typically show up in the right hand side of the search result page or on the top of the page if they perform above a certain threshold set by the search company. For practical purposes I will restrict my comments to Google.

What is clear is that the two sets of results represent two different information flows. The flow of organic results is the result of the crawling and ranking of sources on the Web. The sponsored results are the result of the Adwords auctioning system, or other forms of sponsoring, such as Google Grants or promotion by Google itself. Research shows that about half of the Internet users might be unaware of a difference between organic and sponsored results.

Although it seems to make sense to to require transparency between organic results and sponsored results, in reality the difference is complex and problematic. On a superficial level, the difference reflects ideas about transparency of advertising in the media. Listeners, viewers or internet users should be able to distinguish between editorial content and advertising (in the broad sense). But, in the context of search results, the difference between a sponsored or an organic result does not tell much about the amount of money that has been paid for the result to show up. It is also quite common for a website to show up both in the organic results and the sponsored results. The different processes that govern the ranking in the two sets of results are equally opaque for users. And finally, the sponsored results are the results that are in some waysmore editorial in nature. My tentative conclusion is that the difference is not well understood and of limited value. What is more important is that we evaluate diversity and quality of search results pages in their totality.

Difference does not reflect payments

Both information flows of results, i.e. organic and sponsored, involve the spending of money to show up in the results as prominently as possible. The difference is who receives the payments, but it is clear that the user is not in a position to evaluate them. Whereas an organic result might show up for certain search terms without someone having invested any money into that, many organizations and companies use search engine optimization services to optimize their organic ranking and presence in search results. The example of Lifestyle Lift shows that sometimes complete websites are being set up to enhance presence and ranking.

It would be interesting to see how much is typically paid for organic optimization in comparison to sponsored optimization. Organic results might be seen as more trustworthy by internet users that think that they understand the difference between the sets of results. This increases the incentives to invest in organic optimization. If the ability to place sponsored links on search engine property would be restricted, one can expect increased pressure on organic results. There are no editorial restrictions on advertising in organic results.

Duplication of results and sources is common

Especially for searches related to products, companies and services, it is common that some of the sponsored links are duplicates of the organic links. Duplication blurs the line between organic and sponsored results. An argument in favor of duplication is that it decreases noise for users that were looking for that source anyway. Duplication is problematic because it decreases the diversity of the results and the possibility that users are confronted with information that adds another perspective to a search. The fact that duplication is fairly common suggests that it is worth for advertisers to push organic results off the first page and out of the attention of most searchers. This is especially the case if the duplication involves a sponsored result in the top of the page on the left, and the same organic results right below it. Duplication might be costly so this is a typical case where the rich get richer because they manage to capture the attention of internet users.

Ranking mechanisms of both sets of results are equally opaque

The ranking of organic search results by Google is notoriously opaque. The Page Rank algorithm, that uses links to establish a global measure of relevance for websites is only one of the many different (200+) factors that is used by Google to establish the ranking of organic results. If one wants to know about ranking algorithms the place to look is the search engine optimization industry. These search engine experts have a pretty good idea how search engines work and walk the fine line between accepted optimization and being punished. Ironically, this industry is also seen as the major reason for search engines not to be more transparent about their ranking algorithms.

The ranking of sponsored results is equally opaque. I have heard some experts explain that the ranking of sponsored links reflects the bids the advertisers, but in reality it is more complex. Google places advertisements on the basis of its keyword auction, and a quality score. The quality score is determined by Google. It includes the historical click through rate and other partly-non-disclosed relevance factors. Thus, the highest placed sponsored link is not necessarily the one that involves the highest willingness to pay. In addition, some sponsored links are placed (and can be clicked on) without them having to make a payment. They are sponsored by Google.

Quality control on sponsored links is stronger than for organic results

Interestingly, the sponsored results are subject to detailed editorial policy that guarantees the quality of sponsored results. The policy of Google Adwords also includes the suppression of unauthorized usage of trademarks, as Chris Soghoian found out. There are several legal and non-legal reasons for this, e.g. the fact that search engines receive money for the placement of these results. Thus, sponsored links might end up being of higher (average) quality and more relevant for some keywords. The relative balance between the quality of organic and sponsored results is of major importance to search engines. If search engines can keep the overall quality of the results in their totality the same, while shifting the relevance of results towards the sponsored results, they make more money.

Keeping an eye on search engine diversity

My conclusion from these ideas and remarks is that we need to focus more on quality metrics of actual search result pages in their totality than on transparency between organic and sponsored results. This difference is of limited value to users. I have the impression that Google is committed to diversity of search results and have heard one Google engineer in Europe say this publicly, but more independent empirical research (Benkler 2008, p. 285) is needed in this direction. Search engines could also be more explicit about this commitment, for instance by being transparent about their commitments to diversity and implementing policies that prevent duplication of results.

One Response to “Keeping an Eye on Search Engine Diversity”

  1. Austrotrabant Says:

    Hi Joris,

    Christina has told me to take a look at your blog and I have to say I’m really impressed. Really, really good stuff and one can clearly see that you do really know what you’re talking about! I esp. like the idea to link AdWords policy to freedom of speech.

    By the way, do you still need some more research about user perception / ability to distinguish between organic search results and ads? I saw that you’re using pretty much the same stuff as I did (PEW, Hargittai, …but I think I’ve also got a few more (unfortunately most of them in German).

    So please keep posting, I’ll keep following.



Leave a Reply

You must be logged in to post a comment.

409 1 e book launch invitation pdf computer basics and terminology notes pdf welding and fabrication inspection free pdf ebook download cours autocad 2011 pdf unit operations chemical engineering 6th pdf iphone ios 5 development essentials neil smyth pdf size fundamentals of futures and options markets 7th pdf download plumbing for dummies pdf download do carmo geometry file type pdf dell brc for simplified infrastructure management pdf touch of a wolf jez morrow pdf guia de estudio concursos y quiebras pdf free download the story of the eye pdf 1912 lopez de arenas carpinteria de lo blanco pdf origen y e gesta de los godos pdf the green screen handbook realworld production techniques pdf download descargar en pdf como consegir amigos verdaderos mark curry dance with the devil pdf new zealand retail trends 2012 pdf contoh proposal penelitian kuantitatif pengaruh teknologi pdf jesus zieht in jerusalem ein text und noten pdf apsolventski ispitni rokovi 2010 2011 feb sep 2011 pdf v lcd4 pro l instruction sheet pdf songbook the robben ford blues guitar collection pdf vampire dark ages wind from the east pdf makalah peranan psikologi pendidikan dalam proses belajar mengajar pdf next to normal score pdf download il figurino di moda burgo pdf download water for elephants pdf download latihan soal soal unas pdf 2012 harry potter und die kammer pdf ebook igc cmmi implementation case study pdf principles of mathematics grade 9 academic mpm1d pdf outdoor kitchen design example pdf henry thonier tome 3 pdf 4 shared the great divorce c s lewis pdf torrent amulet book 2 pdf 4share the first heretic aaron dembski bowden pdf torrent ch24 gb1 software project scheduling pdf the book of useless information pdf download jewish study bible pdf pdf 175 lettere a un giovane poeta pdf cristina hija de lavrans pdf gratis manual de wilcom 2006 en espa ol pdf the caves of steel pdf ita day trading joe ross pdf ita modelo de capa para trabalho pdf el metodo montignac pdf power systems analysis pdf download free inez del alma mia english pdf information systems for managers piccoli pdf free download esio trot by roald dahl pdf how to use packet tracer 5.3 pdf cartoon guide to physics by larry gonick pdf unbreak my heart tab diana pdf paul krugman end this depression now pdf fundamentals of logistics management free pdf torrent dan lepard hecho a mano pdf free contoh proposal penelitian kualitatif pendidikan matematika pdf northstar writing and reading 2 pdf 3 edition fredric brown universo de locos pdf libro santa muerte pdf ita experiencing mis 2nd edition pdf download free l express styles octobre 2012 pdf descargar iluminacion y render jeremy birn pdf espa ol nuance pdf converter enterprise v6 0 learn bosnian croatian and serbian ebook pdf a guide to hardware 5th edition pdf part2 rar free download pdf laboratory investigations in molecular bio il libro dei mutamenti pdf dream of orchids by phyllis a whitney pdf infections of jaws ppt pdf process electrical instrumentation drawings and documentation pdf les filles de riyad pdf wu tai chi chuan pdf download understanding of nmr james keeler full free pdf lago dos cisnes partitura flauta pdf chanakya niti in telugu pdf .torrent wild at heart 1990 pdf kepmenpan no 25 tahun 2004 pdf biss zum morgengrauen buch pdf despedida de claudia gray pdf grundlagen der regelungstechnik 02 pdf downloading free ebook language as social semiotics pdf como instalar icons e temas no openloader ps2 pdf barrons ap english literature pdf approprite technology on water supply design by unicef pdf free pdf download just friends by sumrit shahi vittorio the vampire anne rice pdf backtrack 5 wpa2 pdf operating systems 3rd edition by gary nutt pdf nivelamento portugues ensino fundamental pdf antologia de administracion educativa pdf manual da madeira angelim pedra pdf air traffic control radar beacon system pdf master of the universe pdf snowqueens icedragon epov la mujer de arena pdf ebook taringa download free good to great pdf dk chinese english visual bilingual dictionary pdf us army alice pack manual pdf the devil in winter pdf minna no nihongo ii choukai pdf my so called life goes on pdf cherrie lynn far from heaven pdf download along for the ride pdf compiler construction principles and practice pdf wikis hardware and networking in pdf para comprender la eucaristia pdf hp color laserjet cm4730 mfp service manual pages pdf basic ship theory pdf free download pdf generacion de modelos de negocios scribd 2010 guias infeccion urinaria idsa cid pdf teoria da musica bohumil med 4 edi o pdf who wants to live forever queen pdf fundraising donation request letter template pdf star wars legacy of the force betrayal pdf gpnp temperature measurement pdf . exercises for sciatica pdf steve jackson sorcery spell book pdf hotfile com dl the wandering who gilad atzmon pdf pdf el lugar mas bonito del mundo advanced actionscript 3 with design patterns pdf historia del arte mexicano pdf karen robards desire in the sun pdf conversaciones con dios para jovenes pdf max lucado quando os anjos silenciaram pdf isaac asimov the caves of steel pdf art and fear david bayles pdf ubuntu study guide pdf beyond the frozen sea fansadox pdf v 2011 08 downmagaz com pdf der gute mensch von sezuan pdf download cours de la langue francaise pdf john stuart mill el utilitarismo pdf percy jackson and the sea of monsters pdf dutch diagnostico y servicio a sistemas de frenos abs pdf iit sample paper pdf top notch 1 student s book 679609505 pdf the sheik s kidnapped bride pdf manual book grand livina pdf food science pathway curriculum guide pdf general chemistry whitten full pdf converter ps2 pdf exe protech t61 enigma xtr instruction manual pdf cours client serveur pdf space and time in the genesis cosmogony pdf download back and deader than ever pdf mens sexual health fitness for satisfying sex pdf download jinnah india partition independence jaswant singh pdf free download why men marry bitches download pdf pride and prejudice jane kendall pdf pdf xchange 5 pro bundle rar download future tense exercises pdf nora roberts bride quartet 04 happy ever after pdf historia argentina rodrigo fresan pdf dr seuss pdf books cay horstmann big java 4th edition pdf 4 shared christian thibaudeau high threshold muscle building pdf tcp ip protocol suite 3rd edition in pdf benevolo historia de la arquitectura moderna pdf alejandro jodorowsky libros pdf pdf report on line coding veena deal to remember pdf boat international 2012 09 sep 1 pdf testing and commissioning of electrical equipments pdf language in use pdf cambridge euro 2012 slavek and slavko pdf free download culture generale pour les nuls pdf .torrent alexandra ivy the real werewives of vampire county pdf dime quien soy julia navarro pdf measurement and instrumentation question bank by r pdf nissan murano brochure australia pdf le livre de saskia le r veil telecharger pdf creating workflows with sharepoint designer 2010 pdf writing that works pdf the complete book of numerology pdf curso de viol o em pdf life during world war 2 key pdf codigo penal del ecuador 2011 2012 pdf ansi c 4e by e balagurusamy free download pdf php mysql for dummies pdf pathophysiology of asthma pdf final fantasy d20 character sheet pdf cheats para guitar hero 3 ps2 pdf loaded magazine march 2012 pdf .torrent catching fire ebook download pdf pdf download bioinformatics applications and tools by david edwards la meta eliyahu goldratt descargar gratis pdf add manuel du joueur version 2 pdf professional joomla php cms wrox 2007 pdf manusmriti in hindi pdf .torrent the definitive acdc songbook pdf anatomia orientada para cl nica pdf torrent grade 5 puberty boys book pdf about kind of blade server pdf taniec ze smokami 5 2 pdf the answer by john assaraf and murray smith pdf one world the ethics of globalization pdf 39 clues book 8 free download pdf lord of the flies answers pdf brealy principles corporate finance entire ebook pdf protection of power system by bakshi in pdf new approach to sight singing pdf introduction to parallel processing algorithms and architect pdf historia de la f los dones del ministerio pdf penelitian pengaruh internet terhadap prestasi belajar siswa pdf un angolo di paradiso c j box pdf download biblical misconceptions about divorce and remarriage pdf consew model 230 instruction manual pdf direito civil i obrigacoes pdf discrete time signal processing oppenheim download pdf quantitative risk management concepts techniques and tools pdf download condiciones de carta porte pdf handel sarabande tab guitar pdf .torrent en 1991 2 4 pdf pdf 4000 essential english words books.torrent barron s toefl ibt 13 th pdf only.torrent bloodlines the golden lily pdf download uma nova historia dos estados unidos pdf el arte de empezar filetype pdf linear algebra with applications bretscher pdf .torrent download ccna self study exam certification 640821 pdf fundamentos da economia pdf la bruja de portobello pdf descargar gratis daftar pustaka buku pkn kelas 5 sd pdf lichtman s atlas of hematology pdf torrent wes jazz guitar pdf photoshop interview multiple choice questions and answers pdf html5 espresso con css3 e ecmascript5 pdf free download photoshop 8 shortcuts pdf neuroscience exploring the brain download pdf the pool of fire pdf pdf temps des secret object oriented systems development pdf vampire kisses 9 immortal hearts pdf free download the code of criminal procedure 1898.pakistan in urdu pdf the game makers companion pdf the blue cat club pdf gratis como modelar carros zmodeler tutorial pdf mediafire the bourne dominion epub pdf rtf mobi fb2 carving the human face pdf download giorgio agamben el reino y la gloria pdf baixar modelos de carteirinhas em pdf intro to matlab palm third pdf kristen ashley heaven and hell pdf download pdf math notes 9th textbooks lahore free autocad 2007 tutorial pdf download prova do cfo pmba 2005 em pdf private special edition 027 sex city pdf die linde baum des menschen pdf ear training for the contemporary musician pdf planowanie strategiczne w firmie neville lake pdf karate goju ryu by the cat yamaguchi pdf tr 43 post tensioned concrete floors design handbook pdf tracking the tempest pdf mediafire structural analysis 8th edition solutions pdf free erhu sheet music pdf book la donna in nero pdf susan the hunger games book one pdf iit class xii maths probability pdf con17a high level ground improvement lessons learned pdf pdf pdf learn to draw human figure practical networkn operating system pdf lora leigh men of august pdf vicente amigo contemporaries masters of flamenco guitar download pdf opel kadett gls manual pdf tr 751 851 instruction manual pdf on call principles and protocols pdf haynes toyota corolla e10 pdf 978 1 58503 453 6 4 pdf guia para la visa americana pdf descargas catalogos de lenceria 2012 en pdf integrated korean intermediate 1 pdf l art de faire l amour un homme pdf comparison and jump instructions of the 8085 microprocessor pdf formule ed esercizi svolti il compito di chimica pdf pdf the voice of padre pio iata dgr 52nd edition 2011 pdf wallace irving la isla de las tres sirenas pdf jennifer estep dark frost pdf download free how people learn brain mind experience and school pdf electrical layout 220 33kv substation jspl pdf succed in ielts 9 practise tests andrew betsis pdf giant ch katalog 2009 teil 3 pdf quick reference guide to veterinary medicine pdf mark mazower salonica city of ghosts pdf el camino a la autodependencia jorge bucay pdf 2011 brochure kia rio en pdf pretty little liars stunning pdf economia ambiental y desarrollo sostenible pdf geografia sexto grado pdf the captain the journey of derek jeter book pdf the complete cancer cleanse pdf computer networks notes pdf free download ppt or pdf of history of strategic management politique pour les nulles pdf languages teach yourself swedish pdf apostila pdf ensino fundamental il conte di montecristo pdf gratis mit physical chemistry pdf lecture download bite the girls on friday night pdf examkrackers mcat general chemistry booksearchable pdf lora leigh guilty pleasures pdf virginia henley the irish duke pdf a batalha de um homem em pdf the summoning game manual pdf oswald j smith pdf books leo solito en pdf aashto standard specification for highway bridges pdf way wealth benjamin franklin pdf pdf of mechanics 1 for teacher free download what color is your parachute pdf 2010 catalog mijloace fixe si durate normale de functionare pdf communication systems engineering 2nd edition pdf children learning english pdf moon diagnostico de gestion de uma empresa pdf oxford practical grammar books in pdf mecanica suelos juarez badillo tomo 2 pdf differential equations solution manual rainville pdf company profile ozzon radio 105 30 fm pdf atkins physical chemistry 9th edition pdf download sony vegas tutorial pdf r e dla ciebie pdf to hell and back pdf julia quinn the lost duke of wyndham pdf d d 4e book of vile darkness pdf download libro de seuccion subliminal pdf cap la tierra es plana friedman 2005 pdf piano sheet in pdf grace by laura story mamiya c220 service manual pdf l arte del saper vendere pdf esab stainless steel brochure ssp 1014 pdf financial accounting a user perspective 6th pdf introducing 3ds max 2011 pdf zakir naik urdu book pdf pimpin ken 48 laws of game pdf codigo de la luz pdf daniel lumera pc pro 2012 10 softarchive net pdf stages of faith pdf kitchen idea book idea books pdf iee wiring regulations 17th edition pdf books htiler and stalin parallel lives download pdf 3 sombreros de copa pdf ebook browse w3schools all html book pdf free download joshua harris boy meets girl pdf download on mediafire bungsgrammatik f r fortgeschrittene hueber pdf download vita in campagna in pdf microsoft wpf tutorial pdf kasey michaels el sabor de lo prohibido pdf piazzolla pdf fuga y misterio network analysis and synthesis question bank pdf la magia de pensar en grande david pdf schwartz entrainement de football pdf who moved my cheese free pdf download a maior casa do mundo pt pdf pdf a long walk to freedom deten el tiempo libro lupita jones pdf la reina y yo descargar pdf pdf home decoration with origami de tomoko fuse plz descargar libro frin en pdf pdf marino chern simons matrix models and topological strings coelho paulo l alchimiste pdf kamus jerman indonesia pdf inside out upper intermediate macmillan pdf 4 shared the happiness project pdf free download ms access bangla tutorial pdf free download health and medical physics mcqs pdf free books metodos para bateria pdf stroud the significance of philosophical scepticism pdf du lait au fiel pdf vex gateway manual pdf en espa modern operating systems 2nd edition pdf international relations joshua goldstein 9th edition pdf vigyan bhairav tantra in hindi pdf pdf a perfect storm pdf lori foster free the big chill 1983 draft script pdf manuale officina alfa romeo 147 pdf nikos kazantzakis la ultima tentacion de cristo pdf descarga wrox beginning javascript fourth edition mar 2010 pdf ebookee oposicion auxiliar administrativo corporaciones locales ayuntamiento 430657 pdf neufert pdf telechargement gratuit clinical tests for the musculoskeletal system pdf principles and practices of information security whitman pdf pilots reference guide pdf chomik cry of the wolf karen whiddon pdf il profumo di suskind gratis pdf i will follow him satb pdf tangram aktuell 2 pdf fundamentos de circuitos digitales pdf fairyland 4 pupils book pdf curso solfeo 2010 11 pdf twisted laurie halse anderson pdf download derecho constitucional gregorio badeni pdf yoga book pdf in oriya finite element simulations with ansys workbench 14 pdf the willpower instinct free pdf 14 soal soal limit fungsi pdf dl 250 das installation manual pdf internet marketing and ecommerce pdf