Mittwoch, 17. Juni 2020

Merging and advanced filtering of GSC and GA data with Search Console Helper

<tl;dr>

Search Console Helper merges GSC and GA data and applies multiple condition filters

It is not an advertising!
It is a kind recommendation to all of my colleagues - the tool is of an outstanding nature, i was waiting for it since i'm doing SEO!

Search Console Helper allows, among others:
  • To work with up to 5.000 daily data rows,
  • To work with 24 months of GSC data (not maximally 16, like GSC allows)
  • Filter GSC data with multiple conditions for keywords and urls. There are including and excluding filters, supporting the kind of regular expressions,
  • One-click merging of GSC and Google Analytics data,
  • To export data on any way you need,
  • and much more unique stuff you can't imagine ;)
</tl;dr>

Let us look, what happens with the Google's Search Console, previously Google Webmaster Tools, and on the market around of it.

Bottlenecks of the Google Search Console and some tools to the help

Google Search Console is a tool, any webmaster and SEO works everyday. All of us know about bottlenecks of it. I name just some of them, which i personally find as the most annoying:
  • Sampling. Mega sampling. mega fucking sampling. Just try to filter your brand keywords out, than filter only your brand keywords, amd than add metrics of non-brand and brand keywords - you get never ever the results before filtering.
  • Only 1.000 data rows,
  • And the worst of the most annoying bottlenecks - filtering only one entity at the time. Only one keyword, only one URL. It was possible though an URL hack to filter the previous version of GSC with multiple filter conditions. The current version doesn't allow this.

Ways to get rid of the GSC bottlenecks

There are some ways to workaround and to (partly) get rid of these bottlenecks. I.e.:
  • There is a german tool, coded in Cologne, which calculates the sampling away. But, it has own bottlenecks, like the absence of an API, slow processing and the lack of multiple filter conditions - and it is pretty expensive for this.
  • There is a finnish tool - not a tool, but an extension for the Google Spreadsheets. It makes possible to avoid GSC sampling, to filter with multiple conditions and to export from GSC maximally 5.000 rows for the certain date. Pretty good one and not very expensive - but there are still only 5.000 data rows. And - there is no any visialization, you should plot your data at your own, with own graphing possibilities of Google Spreadsheets.
  • Another way is not for everybody. Google Search Console has its own API - it can be queried manually, though Postman and the like, or automatically - it has querying libraries for different languages. But - somebody must code it.

Our use case for Google Search Console, actually the stress test of it 

My colleague and me were digging into suspiciously similar trends of search engine visibility of some websites. Finally we came to idea, the ups and the downs of the visibility could be anyhow explained with the ranking factor of Google, we name as "non-brand relevance".
After we formed this theory, we began to gather data to get the final explaination. To look into clearly separated brand- and non-brand data 

we needed to filter around 4.000 brand keywords in and out, and visualize trends of all GSC metrics for both brand and non-brand clusters in the time frame of the last 16 months.
For the smartest among yourselves: no, usage of the built-in GSC filter "Queries containing" wouldn't do the job. No, because of the nature of brand of this certain website. The site we were dealing with, is an insurance company, and the brand keywords are not only those including the brand name of the company, but of the products too. This means, product names are branded too. I.e. an insurance product "liability insurance", offered by this company, could be named as "liabesturance", and this artificial word as the search keyword could have an own search volume. That's why the cluster of brand keyword was so big.

Based on the daily stats this dataset was understandably higher than an Excel can tolerate.

Having these circumstances we firstly stopped and treaded water. The ways to go seemed to be:
  • to code an own solution (scrapped: lack of time an money),
  • to use the finnisch tool (the only way promised to get any result anytime soon). This way wasn't reliable, because we were missing the ability to forcast, how many time we'll need to go this way to its end.
The solution seemed finally to recede into the distance...

Every time, when i'm out of ideas, have no clue, how to do something, i begin to post a question into Stack Overflow. And, in many cases, i found an answer while writing a question - but for some unknown reason not earlier.

Exactly so was this case too. While i was writing a new post, i've came to idea to look for a tool, which maybe would match our needs. I thought, if somebody would build a tool, which makes working with GSC smarter it
  • would let users avoid known GSC bottlenecks,
  • could be found with keywords "search console tool".
And look - on the place 43 i've got what we needed - http://searchconsolehelper.com/, yes, this is and will be our tool of choice to work with the GSC. This is btw. the lesson for all those thinking there is no life after page 1.

Backed by Search Console Helper we accomplished our case

I describe only tasks we personally accomplished with Search Console Helper (alias SCH). But, anybody doing webmastering and/or SEO  can be sure: this tool can much more, than your daily business with GSC.

Well, these were tasks we did:
  • Importing 16 months of data (to make comparison with GSC possible),
  • Filtered data with multiple condition include and exclude filter for keywords and URLs,
  • Got data from Google Analytics certain metrics and merged it with the GSC data,
  • Got help from the tool owner via Teamviewer...
In fact, filtering 4.000 keywords from 16 months of data took its time. I normally do this with Excel, formula and exclude list - but in this case it was smarter to let Search Console Helper work, because of immediate visualization.

Finally, using Search Console Helper we were succesful with accomplishing of our task. 

SearchConsoleHelper - wanna some GUI pics?

Well this is the main window:
search console helper: main window with filters

Do you see asterisks at the beginning and the end of keywords? These are the wildcards: you puteach filter expression in its own line, using wildcards you catch all occurences of it.

And on three further pics you'll seeanother outstanding filter possibilities, which work directly from the main window:
Search Console Helper: filter condition setup
Next pic shows contain entity setup - what can be filtered:
Search Console Helper: filter entity setup

This pic shows, which conditions can be applied to the filer:
Search Console Helper: filter conditions

And so looks your GSC data merged with the data from Google Analytics - easy-peasy done - in this case all data from GSC (query, url, clicks, impressions, CTR, position) and the bounce rate:

Search Console Helper: merge data from GSC and GA in one click

FYI: SearchConsoleHelper is an installable programm - i like such tools more than online SaaS because of more stability and much effectiver usage of CPU and RAM.

I could endless bend you ears about this absolutely unique and outstanding tool, but i should slowly shut up and come to an end. 

Well, what i mainly got with Search Console Helper:
  • Google Search Console, Google Analytics and Google PageSpeed Insights Test in one box,
  • merging of GSC and GA data,
  • applying multiple filters with multiple filtering conditions to Google Search Console data.
I'm doing SEO since couple of years - this combination of features offers no tool on the market. I suppose, any SEO or SEO-focused webmaster would sell their souls extensively use such tool many times a day. If so - then run for it: Search Console Helperhttps://searchconsolehelper.com/
I definitely do. 

BTW. developers of Search Console Helper are hidden behind https://www.woax-it.com/ - a software development boutique from Austria. At the daytime they develop CMS for small- and medium-sized businesses.
Yandex.Metrica