How to increase your search traffic in Rails with programmatic SEO

Feb 08, 2024

I taught myself the basics of SEO and programmatic SEO for a Rails website that promotes a range of products. This is how I did it.

The site displays thousands of products and I knew there should be a way to programmatically expose the product terms to drive more traffic by making relevant products easy to find.

It took a little bit for me to join the dots and build out the programmatic SEO capability. Here are all the moving parts.

What is Search Engine Optimization?

Lets step back and note SEO is the process of improving a website’s visibility in search engines to get more traffic. A web page will only rank well if the search engine indexes it AND thinks it’s the best result for the search.

The focus of this post is more the nitty gritty of how to create SEO-optimized webpages in Rails.

I’ll cover the tech side with SEO-friendly / human readable URLS, metatags and sitemaps

I WONT cover content or keyword research (which are probably the more important parts to rank well 😅).

Ive found https://ahrefs.com/seo is a really good beginners guide to SEO in general so start there if youre looking for more general SEO knowledge!

For now I’m assuming

Youll have a bunch of terms you want to be indexed on google for (e.g. someone searching for cheapest huggies nappies at coles in australia)
Youll have the data available to drive your all your programmatic keywords

What is programmatic SEO?

Programmatic SEO to me is using structured data and coding to create a large number of landing pages to target topics that people are searching for in Google.

Google will treat the following as 5 separate pages but as we’ll dig into its really the same Rails controller being used to search and retrieve variations of my product data re-using the same layout each time

How to implement search and filtering and generate SEO-friendly URLs in Rails

Heres a website that promotes a range of products. The website URLs in the interaction below are all SEO-friendly URLS

On load we see https://www.justnappies.io/ with ~1700 products
We click the ‘amazon’ filter and the page refreshes with the updated URL and 383 products at https://www.justnappies.io/cheapest-nappies-at-amazon-in-australia
We click the swim filter and now see 16 products available at https://www.justnappies.io/cheapest-swim-nappies-at-amazon-in-australia

We can see the URL is pretty readable and and has a few repeatable keywords sourced from the underlying product data

cheapest {type} nappies at {store} in australia

You can imagine {store} can be [amazon, coles, woolworths, aldi] and so on. You can also filter by brand, product size.

So whats the deal with the data?

Having access to a lot of structured, queryable data is key to enabling programmatic SEO

All my product data is structured and stored in my Postgres database so that I can query it easily. As an aside, initially my product data was NOT consistent or structured and it was super painful to use e.g. how do you present/query shoe size 1 vs shoe SIZE 1 vs shoe size one etc consistently and accurately

I spent a bunch of time cleaning and standardizing the data and the effort was easily worth it for simple querying from my rails controller

Searching using the Ransack gem

I’m using the Ransack gem which lets you easily add search forms to your Rails app. The appeal of Ransack was I dont need to add any additional dependencies (elasticsearch etc) i.e. I can just use my Postgres database as is

Lets dig a bit deeper with what is actually happening in Rails in the above gif.

When clicking the ‘amazon’ filter we are submitting a html form with the following request URL

https://justnappies.io/?q%5Bs%5D=price_per_cup_cents+asc&q%5Bstore_name_in%5D%5B%5D=amazon

Our Rails controller gives these params to Ransack to retrieve the subset of products linked to amazon

Our Rails controller ALSO does the following

The Rails controller converts the ransack params back into a seo friendly url and returns this SEO-friendly URL back to the browser
1. The browser then triggers a stimulus controller to modify session history stack programmatically to display the SEO-friendly URL
2. The browser page’s meta tags are also updated with the SEO friendly terms

Heres the code for the controller

Heres the relevant code for the index.html webpage that our controller populates. We pass the seo_friendly_url variable to a stimulus controller url_controller.js

The stimulus controller url_controller.js uses the history.pushState method which allows developers to modify the browsers session history programmatically. The javascript is telling the browser to set the browser URL to https://www.justnappies.io/cheapest-nappies-at-amazon-in-australia. Easy when you know how!

At this point we have implemented displaying a human readable SEO-friendly URL in the address bar

If someone google searches “cheapest tooshies nappies in australia” I want my link to appear on the google search results page and when they click the link then the correct products are displayed

But how do we give this URL to google to index? And when google indexes and displays this URL how do we make sure our Rails app can route it correctly?

I’ll start with routing first

How do we enable Rails to route our SEO-friendly URLS correctly?

My website has the following URLs

https://www.justnappies.io/cheapest-tooshies-nappies-in-australia

https://www.justnappies.io/cheapest-huggies-nappies-at-coles-in-australia

If you click on the tooshies link you’ll end up looking at a product page full of tooshies at all stores

If you click on the huggies link youll end up looking at a page full of huggies at coles only. Makes sense.

To do this I need to update my Rails routes.rb file to cater for all variants of terms in my URLs and route them all to the same dashboard_controller

cheapest {brand} nappies {sizes} at {stores} in australia
cheapest {brand} nappies in australia

Heres my routes.rb file with some of the route variants

You should place the most complex routes first to avoid unintentional matches

Now we can update the same rails controller to handle receiving an seo-friendly url and then converting its params to Ransack params for querying

Great! Now when we click the link https://www.justnappies.io/cheapest-huggies-nappies-at-coles-in-australia in our browser we see the correct products displayed on our webpage

Now we need to tell the world about it - we need to create a sitemap

Using a Sitemap to list all the pages on your website

A sitemap is a file that contains all the URLs on your site that you care about, helping search engines find and crawl them. My websites one is https://justnappies.io/sitemap.xml

I use the sitemap_generator gem to generate my sitemap file

When using sitemap_generator I have a sitemap.rb that looks something like this, it c covers all the urls for all variants of my products.

I can then run via ruby config/sitemap.rb to generate the sitemap.xml and push to google for indexing

The above sitemap.rb is hardcoded urls - you could even generate your sitemap entries dynamically by reading your products from your database and extracting their fields. This could be automated to run daily if your product catalogue was growing

Google Indexing API

Waiting for google to read your sitemap.xml and eventually crawl is the default approach - Ive found it can be (very) slow which is annoying if you want to see results immediately.

You can use paid tools to speed up the indexing. Ive used https://tagparrot.com/ in the past for automated, quick, bulk indexing, it does exactly what it says it will.

Using MetaTags to make your Rails application SEO-friendly

I use the meta-tags gem to make my rails app SEO-friendly

I followed this handy tutorial when implementing metatags - https://dev.to/junko911/make-your-rails-application-seo-friendly-by-setting-meta-tags-6o5

My application.html.erb contains the following addition to enable meta tags

I explicitly set my metatags variables in my controller, after calculating my seo friendly url I then push it into the meta tags fields too. It then magically gets injected into my webpages title and meta tags!

If we inspect the content of this web page we can see the meta tags title and description are set to “cheapest huggies nappies at coles in australia’ as expected

Wrap Up

Ive covered a bunch of ground and each of the points could easily be a blog post on its own. Creating SEO-optimized webpages in Rails is a simple way to expose your content keywords to drive more people to your website!

Felix’s Substack

Discussion about this post