Predicting rankings with machine learning, our experiment from 2017

The question behind the experiment

Finding keywords is easy, assessing them is not. For our example website, a niche shop on raw feeding for dogs (BARF), SEMrush returned more than 1,100 keywords it already ranked for. With the competitors' keywords, there were more than 3,600. For a larger website that is not many, yet it is still too many to check by hand.

We wanted to know for which of these keywords a top 10 position could be reached with little effort. To do this, we trained a machine learning model that predicts the position after an on-page optimization. We first presented the experiment at SEO Day 2017.

Method in five steps

1. Think about how you would do it by hand

By hand you would look at search volume and CPC. High values suggest a lot of competition. If a very similar competitor ranks in the top 10, that is probably achievable for your own page too. If your own page is already well optimized but ranks on page two or three, more on-page work brings little. If there is no matching page or it is poorly optimized, the potential is greater. This is exactly the data the model gets.

2. Collect and convert the data

Via the SEMrush API we retrieved all keywords of the example site and its competitors. We then crawled every ranking page and checked whether the keyword phrase appears in the title, content, URL path or domain. The model only processes numbers. That is why we also encoded domains and keywords as numbers.

3. Train the model

We split the data into 80% training data and 20% test data. The algorithm was an MLPClassifier, a neural network from scikit-learn. In our tests a neural network worked best for this task. Training can take minutes or hours. This simplified example shows the principle.

4. Predict positions for optimized pages

For the keyword “barf fleisch kaufen” (buy BARF meat), the phrase was missing from the shop page's title and URL. It appeared partly in the content and in the domain. The model predicted position 20. In the test data, the page was at position 18. We then entered the same data with the phrase in the title and content. The model predicted position 1.

5. Evaluate all keywords

The same ran for all 3,600 keywords. For terms unrelated to the shop, such as “kreuzberg”, “durchfall” (diarrhea) or “ziege” (goat), the model gave no prediction. That speaks for its quality, because a good ranking is not to be expected for such terms either. Filtered to predicted positions below 10, around 600 mostly relevant keywords remained.

Result

Instead of checking 3,600 keywords by hand, this produced a list of 600 candidates for new or optimized pages. Some of them were misses, and some forecasts were too optimistic. As a starting point for the on-page work, the list did its job. Whether the pages actually ranked that way after optimization is something we did not measure as part of the experiment.

Limitations of the experiment

• Only a few on-page features, mainly the position of the phrase in title and content. External links and analyses such as TF*IDF were missing.
• One niche website with 3,600 keywords. More ranking data, more competitors and more keywords should improve the prediction.
• We did not systematically optimize parameters and algorithm.
• A classifier treats every position as a separate class. A regression that knows the distances between positions would be an obvious comparison (our assessment, not tested).

What has changed since?

The method is nine years old. Three things have changed considerably since then.

• Google understands language better. According to Google, systems such as RankBrain, neural matching and BERT now work on connecting words with concepts. Pages can therefore rank even without the exact phrase (Google documentation as of 10 December 2025). A feature such as “all words in the title” therefore explains rankings less than in 2017.
• The results page is fuller. According to Google, AI Overviews and AI Mode build on the same ranking systems. A position on its own, however, says less about clicks than it used to when there is an AI overview above it.
• Tools are more accessible. scikit-learn with the MLPClassifier still exists. The code of our experiment is publicly available in the repository searchVIU Labs on GitHub, together with an example that predicts the CTR per position from Search Console data.

The basic idea remains useful. If you want to prioritize keywords by potential, combine competitive data with what is on your pages and let a model pre-sort the list.

Common mistakes when rebuilding it

• Test and training data overlap, and the model appears more accurate than it is.
• The prediction is read as a promise. It is a ranking for checking by hand.
• Keywords unrelated to the offer stay in the list because nobody looks through the results.

Frequently asked questions