Monitoring your move to SSL with the Toolbox

Google has already started to reward the encryption of websites via the HTTPS protocol in 2014. Over the past few years many website operators have switched to HTTPS and the development in the UK shows substantial growth.

Towards the end of 2016 about 30% of the Top 100 results on google.co.uk are HTTPS pages:

A move to HTTPS takes much more than simply flipping a switch. You have to pay attention to many technical aspects and we would like to show you how to follow and monitor your website’s switch to HTTPS with this SISTRIX Toolbox tutorial.

The monitoring is carried out via a report and an Optimizer project.

The SSL Report

For starters, we open the domain we want to monitor in the Toolbox and check if there are more than one subdomains, which we have to keep an eye out for. To do so, please look at SEO -> Hostnames to check if there are other hostnames besides the www. or non-www host.

Hostname evaluation for the domain

In our example, we decide that we only want to monitor the www.sistrix.de host.

Total Overview

Now we can jump back to the domain overview page and click on the cogwheel in the top right hand corner of the Visibility Index graph, where we choose the “Compare Data in Chart” option.

Box option "Compare in chart" within the Visibilty Index graph

Here we type in both the HTTP and HTTPS paths for the www.sistrix.de host. Any time you want to compare directories, it is important that the entry starts with the protocol (http:// or https://) and is then followed by the rest of the URL, which ends with a terminating /.

How to type in a directory and end with a terminating /

We are now shown the Visibility development for both protocols in one graph. Next, we use the cogwheel, once again, to open the options and select “Add to Report”. As report we choose “Create new Report” and will then open it in a new tab, when the Toolbox asks what we want to do next.

Box option "Add to report"

In the report itself, we add a meaningful title to the report by clicking on the cogwheel right next to “My Report”. Afterwards, we limit the graph to show only the last 1.5 years by using the box-option. If the switch to SSL of your domain is only just approaching, you can also chose to limit the graph to the last 2 months, for better clarity. It is recommended to add an appropriate title to the graph, while you are in the options.

Box option for the data boxes in a report

We have now created an overview of the overall health of the domain in the desktop search.

However, the search behaviour has been shifting further and further towards mobile devices, over the past few years. To take this into account, we now re-create the same evaluation using the mobile Visibility. To do so, we go back to the tab with the graph comparison and open the box-option and click on “Compare Data in Chart” again. We can now choose the mobile Visibility version in the drop-down menu and keep the paths intact. We now get the graph for the mobile Visibility Index.

Comparing the mobile Visibility in one graph

We add this graph to our report (we can now simply click on the report name in the report-selection), adjust the time axis accordingly and give the box a title.

Detail View

Our next step is to find the most important directories for the domain and add them to our report. You can either check your analytics data or click on “SEO -> Directories” in the Toolbox, after you typed in the domain. If the switch to SSL has already taken place, you can display the directory evaluation for an earlier time by clicking on “Select another date” in the graph’s box-options and selecting a date before the switch took place.

SEO -> directories with an alternative date

In our example, we will choose to keep track of the development of the three directories /frag-sistrix/, /news/ and /tutorials/. You are, of course, free to add more directories to your report.

In order to add the graphs to our report, we proceed exactly as we did for the overall view and create a comparison graph for each directory for desktop and mobile.

Directory evluation for the /frag-sistrix/ directory in the report, for desktop and mobile

If there are many directories that you want to monitor, you can collect all the desktop and mobile evaluations together in one single graph.

Comparison between the desktop and mobile graphs for HTTP and HTTPS, all in one

The SSL report: conclusion

We can now decide if we want to add some additional text elements to the report. This makes sense if several stakeholders are supposed to access the report, where you can use the free-text fields to briefly explain what is shown and what to look out for. You will find much more information on the report design options in our tutorial “PDF-Reports: Configuration & Adjustments“.

Once we have set up the report according to your wishes, we want it to be automatically send to us, once a week. This also has the nice side-effect that the reports will be stored in the archive, which makes it easy to track changes for a specific week, any time we like.

You can download the example report for sistrix.de here.

The ranking URL check

After your switch to HTTPS, Google will have to crawl the respective HTTP pages once again in order to find the 301-redirects to the HTTPS version of each page. A simple way to check how far Google is with this, is to look at which protocol version Google will return in the search results.

After a certain transition phase, which may take up to half a year depending on the size of the page, Google should only deliver HTTPS pages in the search results. We can use this to our advantage, by going to “SEO -> URLs” for our domain (in our case sistrix.de) and activating the predefined “Without SSL” filter. This gives us a list of all the URLs for our domain where Google is still not showing the HTTPS version in the search results.

Vorgefertigter "Ohne SSL" Filter unter SEO -> URLs

At the start of the switch, this list will likely be quite long. Over time, it should become shorter and ideally, after no more than 6 months, it should not show any results at all.

The technical examination in the Optimizer

If you have not created an Optimizer project for the domain to be monitored, it is worth looking into our tutorial on how to create an Optimizer project.

In the project itself, we can use the expert mode and the available filters to examine two important technical conditions.

HTTP Pages without redirects

Ideally, after switching to HTTPS, there should be no HTTP pages on your domain which return a status code 200 (OK). All HTTP pages should refer to their HTTPS counterpart by using a 301-redirect.

To verify this, simply go to the expert mode in the Optimizer and create a URL filter for “http:” and a status code filter for “not 301”.

Optimizer expert mode filter for the crawled URLs

This will return a list of all the URLs (found during the crawl) on the domain which use http: in the URL and output a status code other than 301. In this list you will now find all URLs which have not been redirected correctly (or which return a 404 or 200 status code and are therefore not redirected at all) and need to be checked.

Internal links with redirects

When it comes to the internal links on your domain, there is no justifiable reason for one or more redirects to exist, at all. You have full control over your internal links and can save both the user and Google the time a redirect takes.

In the expert mode, please switch to “Links” in the upper right hand corner and you will find a table of all internal and external links, found during the crawl. Then add these two filters:

Optimizer expert mode filter for the crawled Links

By using the type filter you define that you only want to see INTernal links and the "not 200" status code filter ensures that you get all the other possible status codes (like 301, 302, 307, 404, 410 or even 418). If you get any results here, change the internal links to point directly at the new version of the page.

Conclusion

You can use the SEO module and the Optimizer to monitor and follow your domain’s switch from HTTP to HTTPS, while the automatic reporting allows for a weekly overview of the domain’s Visibility on Google.