Why Google should, and shouldn’t, give up its secrets

Google’s mission to “organise the world’s information” is predicated on one very important thing. Secrecy. Secrecy which may about to fall apart in the UK High Court.

The Register are reporting that a UK High Court judge has ordered Google to allow the court’s chosen SEO expert to inspect their search engine algorithms or remove them from evidence in its defence against accusations that it unfairly demoted the listings of price search rival “FoundEm”.

Google are now faced with a tough decision – if it withdraws the evidence it may lose the case, weaken its defence in this and potentially any future court case or hand over the crown jewels – the secrets to how it ranks webpages in search results.

The costs of losing the case will be millions of pounds of damages plus the very real risk that once Google loses one court case, it will lose more – the precedent will be set that Google has acted unfairly in its stewardship of one of the world’s most important algorithms.

The cost of winning it could be losing control of the search market.

Google rose to prominence because of all the competing search engines that once existed (remember Altavista, Lycos, Ask Jeeves…) it was the best one. Google gave you want you wanted – good, accurate answers. The quality of the results from Google was simply better, in the eyes of most users, and Google soon became the dominant and default search engine.

At the same time as Google was winning the market, the importance of internet searching grew and the average user’s preference for picking the first answer they saw was understood. Thus, search engine optimisation was born; the quasi-mystical pseudo-science of understanding what could make a page appear higher in the list of results for a given phrase and deliver that holy grail of the web – free eyeballs on your content and free customers for your products.

Whilst in the early days Google might have been a little more transparent about how they ranked pages, they soon learnt the importance of keeping their ranking process secret from both their competitors and from their customers.

Google keep the way in which they rank pages secret. Yes, they provide guidelines for webmasters and they announce certain key algorithm changes but the vast majority of their rules and “ranking factors” are unknown. What they do reveal is nebulous, sometimes contradictory, and always the subject of debate amongst digital marketers and SEO experts.

In a 2013 survey, nearly 60% of respondents to a survey on Search Engine Roundtable even felt that Google lied to them about how it works. That may seem like some time ago, but Google has nothing to increase transparency since then. If anything, they have become more opaque.

When asked to describe how the Google algorithm works, Google Vice-President Vint Cerf said it was impossible.

Second, this is an incredibly complex algorithm – there are hundreds of variables that go into it, trying to figure out what is quality information and those variables change whenever we experiment to try to improve the quality of the algorithm. To make this really clear, we’re starting to introduce neural network mechanisms for machine learning. We’re talking about systems that are almost impossible to understand. We can see their reaction to the training but if you open up the box, it’s a box with a million dials in it that have all been set to some amount. Nobody really understands exactly how those dials got set because of the way training algorithms work. If somebody says to you, “I opened up the box and here is dial number 7,222 and it’s set to 0.08 – what happens if we change it to 0.10?” Nobody knows the answer to that. 

It’s understandable why people ask to be shown the algorithm. The problem is, it’s not clear that anyone including us completely understands the algorithm and how it works. All we can do is to keep testing it to see what its responses are and adjusting it so that it improves the quality of the result.

Vint Cerf –

I think that’s a great description of artificial intelligence as we have it today. The “million dials” described by Cerf are influenced by the input of thousands of human “search quality raters”. Despite the detailed guidelines they are given, the input of these humans will be, inevitably, human – and therefore unpredictable.

The truth may be that Google doesn’t know entirely how Google works.

But there remains a compelling case for secrecy, even if that is the case.

Why Google has to keep its secrets

There isn’t an SEO who hasn’t dreamed of finding a super-secret Google instruction manual on some dark corner of the Internet. Like the holy grail, the lost treasure of the Sierra Madre, or the contents of the brief case in Pulp Fiction, the mystical properties of such an item beggar belief.

Imagine being able to place any webpage you want at the top of the search engine results for any search phrase you want, with one hundred percent accuracy. Imagine what you could do…

Now imagine what someone else could do.

With great power comes great responsibility

Google’s position as the dominant search engine has brought it immense wealth and power. But, like Spiderman’s ability to swing from a web, this also comes with great responsibility.

If you Google “Am I having a heart attack?”, it’s Google’s algorithm that picks which page you are sent to first. If you Google “how to save my marriage”, they pick the top search result for that too. People ask Google life-altering questions every single day. The consequences of pages no longer being ranked by merit (or by “Expertise, Authority, and Trust” as Google describe it) but based on the ability of the page’s owner (or their agents) to manipulate the Google index could be catastrophic.

Of course, that’s exactly how things are today…

Why Google can’t keep its algorithm secret any longer

Google is a private organisation and whilst it may be bound by the laws of the countries in which it operates, it is fundamentally answerable to nobody but itself and its share holders. It has no requirement to do the “right” thing, only the most profitable thing.

Up until 2018, Google’s official code of conduct contained the phrase “Don’t be Evil”. In 2018 this was revised to “Do the right thing”. The question is – right by who?

If search engine optimisation experts and software were unable to influence the position of a webpage in Google’s results, the search engine optimisation industry would have collapsed long ago. Instead, it is bigger than ever. Google gets increasingly complex, increasingly vague, whilst also being increasingly important.

So, we are already in a position where the position of a page in the search engine results is influenced by the page owner’s ability to optimise their page for Google’s algorithm.

How sure are you that the first result is the best result?

What happens if Google loses control? What if it already did?

Donald Trump is President of the USA. The United Kingdom is leaving the EU. You may be happy about these things, you may not be. Those who aren’t have laid a large portion of blame for voters “getting it wrong” at the feet of Google, Facebook, Twitter, and other technology companies who they feel failed to control the spread of fakes new stories on their platforms.

People ask Google life-altering questions every day, and that includes looking for information on who, or what, they should vote for.

If Google has lost control of what appears where in the index, if it can’t separate fact from fiction, can it continue to operate without any form of oversight?

Somebody somewhere needs to be looking at Google’s algorithms to ensure that they are telling the truth about how the search engine works. But, once the genie is out of the bottle, it seems inevitable that this information will leak and once it does, the integrity of the Google index will be permanently damaged.

We may never, ever, be able to trust the index again.

The Big Problem is… the alternative might be worse

Despite its flaws, I doubt there are many Internet users who would like to go back to a pre-Google world. Google searches have become a part of our daily lives; through our computers, our phones, our voice-activated assistants, and through services that use Google behind the scenes for information.

The only close competitor, in my opinion, is Duck Duck Go. Duck Duck Go produces quality results and has a much stronger stance on user privacy than Google. Today, however, it lacks the resources to take on Google’s search volumes or marketing budgets. While Google remains the biggest player in search it will attract the biggest marketing spend – which empowers it to remain on top.

When you take into account what it would take for a competitor to unseat Google, even if Google found itself in a weakened position as the result of restrictions imposed by governments, the only current viable competitor is Microsoft’s Bing. And nobody really wants to use Bing. Think of it like a map – if you want to get some A to B, would you buy the fourth best map on the market?

Equally, there is the very difficult question of involving governments in the organisation of the world’s information.

Whilst Google might be immensely powerful, unimaginably powerful for most of us, it is still bound by law. It is not a nation, not a government, and it has to turn up in court just like the rest of us if somebody thinks it broke the rules. It just turns up with really, really good lawyers.

A very wise friend of mine once told me “Never assume you will always live under a benevolent government”. What he meant by this was that you should never hand over rights, or powers, that you might need to protect yourself at some point in the future. The idea of a state controlled Internet search engine fills me with dread, it should fill you with dread too.

Not only is it questionable if a government, any government, could deliver such a project (statistics show that only 10-15% of government IT projects are ever deemed “successful”) but it is highly questionable what their algorithm would even do. How would a government rank webpages? What would it prioritise? And how do you rate your government in terms of Expertise, Authority, and Trust?

No, the truth is that whilst where we are with Google, and search engines in general, may not be a good place… it’s least worst of all possible bad places.

You can read the original article at The Register here:

Leave a Reply