There are literally more than dozens of SEO tools so we can’t list them all. Instead we decided to pick our favourites that we use every day as part of our workload.
Screaming Frog has been the primary go to for SEO consultants since neolithic times. Word on the ice says that it was even used to hunt Mammoths, but we’re here to talk about how it can supercharge almost everything in SEO and beyond (yes beyond).
Screaming Frog primarily runs locally from your machine, though it can be run in the cloud if you have the resources to host it there (#fun).
The trend towards purely cloud based crawlers has, as with many industries, started to take over software development. Indeed, for large sites that need regular monitoring that emulates Googlebot and its Web Rendering Service, cloud based tools are really important.
Nevertheless, nothing beats having something at your fingertips that you can configure to your heart’s content and doesn’t charge you credits for every URL the tool requests.
Screaming Frog really is a toolbox allowing you to.
- Scrape using Xpath and Regex
- Search a site for specific terms
- Identify redirect and canonical chains
- Sniff out error pages
- Give you an idea of word count
- Identify backlinks from a list of URLs
- Easily set up custom Exclude and Include rules using regex
..and much more, not least of which gathering all of the the meta data and peripheral information you could possibly expect from an SEO analysis tool. Seer Interactive’s guide to using Screaming Frog is still very useful despite being a bit out of date.
Playing with the settings you can run small crawls on lists of URLs and get the results very fast, which is something you really can’t do with cloud based software while you’re in the middle of a meeting with your client 😉
Cost: £149.00 Per Year for a single licence
Google’s Search Console is a great tool that you should definitely be using. There are some significant limitations with the interface in the ‘Performance’ report. The main ones are.
- Limitation of queries shown (capped at 1,000 for export).
- Inability to correlate queries with pages.
- Inadequate filtering options.
With the sheets ad-on, you can bypass all of these issues and get the full benefits of the API. You can also enable backup cycles so that you can automate your reporting and warehouse your historic Search Console data in a Google sheet.
N.B. This is not a perfect add-on, as there are instances where it will time out and struggle with large data sets. Therefore it’s wise to create smaller data sets and aggregate them elsewhere.
Cost: Free 🙂
Deep Crawl was the first enterprise level, cloud based crawler that went to market. Though it has rivals, it’s probably fair to say that it’s still market leading and tends to come out with original features faster than other Googlebot emulators.
Log file analysis is now something that Deep Crawl supports and it also integrates well with larger SEO platforms such as Conductor.
Apart from storage and processing overhead, one of the advantages of using Deep Crawl and its cousins is that it can be set to crawl your sites regularly, creating very useful trended views, allowing you to monitor your technical changes over time.
Cost: Light – £100 annually – up to 10,000 URLs. Light Plus – £440 annually – up to 40,000 URLs
Many SEOs are migrating away from Excel in an attempt to automate analysis and the use of SQL databases. ‘Data lakes’ are on the rise (though at SEO Eskimo we prefer to use ‘data pantries’, especially in the winter).
Still Excel is such a great SEO tool and knowing your way around it is an important part of being a marketeer let alone and SEO. Most tools export via .csv or .xlsx formats so it’s natural that we should be able to manipulate data directly in a traditional spreadsheet.
Cost: £59.99 Per Year for Office 365 Personal
Talking about Excel, there’s a great plugin for SEO, conveniently called, SEO Tools for Excel. If Excel is your bag, this is a great way to conveniently pull in a lot of metrics via APIs straight into Excel.
Our favourite feature is ‘=Regexismatch’, allowing you to return TRUE or FALSE statements against large set of data using regular expressions #cool.
Cost: £103.43 a year (including VAT)
Sublime Text is one of many code editors out there. In fact there are so many it would be unfair to say that it’s the best. At the same time it’s very flexible and a great starting point for code editing and programmatic editing and has some nice themes & plugins.
Cost: Try it for free or make a donation to keep their developers well stocked in seal blubber.
Ahrefs is our favourite off-site analysis tool for a number of reasons. It’s rivals have great features (think Majestics Trust and Citation Flow metrics), but the interface and ready to go reports make Ahrefs a winner in our book.
You can quickly and easily run reports at any level of a site, including any URL prefix. So for example, if I just wanted to see the backlink profile of pages under the http://bbc.co.uk/programmes/ folder, rather than the whole site or just that page, I could select that option (as well as all of the others).
Another great report, especially useful for competitor analysis is the ‘Best by Links’ report. This report ranks pages on a given domain by the strength and quality of the links that point to them, allowing you to easily zone in one the most linked content on a given site. This is the kind of data you’d need to pivot from raw reports using other tools and can be applied to just ‘Live’ or all links ever found (as can most of the other reports).
Cost: 4 pricing plans at $99, $179, $399, and $999 per month
SemRush is an old favourite in the industry and represents great value for the huge range of features it includes.
Its ‘keyword universe’ is one of the largest, meaning that its ability to track and evaluate competitors’ visibility with a significant degree of accuracy.
You can also create projects that allow you to track your own keywords that pertain to your own project.
There are also great data sets for the analysis of paid search and display campaigns, making it a one stop shop for digital marketing.
Cost: Pro – $99.95. Guru – $199.95. Business – $399.95
Advanced web ranking (AWR) used to be another piece of software that ran on your local machine. Unlike Screaming Frog it was designed to scrape Google’s results to get a view on where you and your competitors rank on the web.
Using desktop machines to scrape Google became somewhat impossible, as Google made it harder and harder to scrape its SEPRs. Today checking your rankings really does need to be done in the cloud and AWR Cloud is a great way to track keywords that you care about and monitor them over time. It represents great value for money, though it doesn’t have the bells and whistles that tools such as SEMRush have.
Cost: Starter ($49/month), Pro ($99/month), Agency ($199/month), and Enterprise ($499/month)