3 Programmatic SEO Approach To Try For Your Website

⭐️What is an SEO Competitor Analysis

An SEO competitor analysis is a process of evaluating and analyzing the SEO strategies and performance of your competitors to gain insights and identify opportunities to improve your own search engine optimization efforts through programmatic SEO approach. It involves studying the websites and online presence of your competitors to understand their organic search rankings, keyword targeting, content strategy, backlink profile, and overall SEO tactics.

How to perform an SEO competitor analysis using Python

Set up a custom search engine

Create a new search engine

https://programmablesearchengine.google.com/controlpanel/create

Enter a Name for your search engine

New search engine has been created

Here is a link to the Programmable search engine homepage and within a few clicks, you should be able to set up your own programmable search engine.

https://programmablesearchengine.google.com/controlpanel/all

Then go Customise

Go to Google Developer Console and click the Get a Key button.

https://developers.google.com/custom-search/v1/overview#api_key

Select your search engine and go Next.

Competitor Analysis – SERP heatmap Master.ipynb

https://colab.research.google.com/drive/1SqVPj7nYynXFHX8alYkAIp-Pm2ArUNQY?usp=sharing#scrollTo=6d0c64b0

Run and Wait.

Add Keywords and your Search Engine ID & API Key

Run the serp_goog function

View the results of the serp_rankings output

5. Run & Export Data to CSV

7. Run the SERP Heatmap Function

8. Call or Invoke the serp_heatmap function

9. View your SERP Heatmap Data Visualization

Cookie Audit Introduction:

It’s essential today to understand the presence and purpose of cookies on a website. The data privacy landscape is continuing to evolve and there are various laws and regulations in place throughout the world, where cookie compliance is almost always included in some way.

1) Select ‘Cookies’ For Extraction

Open up the SEO Spider, go to ‘Config > Spider > Extraction’ and select ‘Cookies’ under ‘URL Details’.

This means the SEO Spider will now store all cookies discovered.

Please note, when you choose to store cookies, the auto exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued.

This means it will affect your analytics reporting, unless you choose to exclude any tracking scripts from firing by using the exclude configuration (‘Config > Exclude’).

2) Enable JavaScript Rendering

Click ‘Config > Spider > Rendering’ and select ‘JavaScript’. This means the SEO Spider will open each web page in a headless Chrome browser behind the scenes.

This is an important step, as it allows cookies that are loaded using JavaScript or pixel image tags to be discovered.

The window size is automatically set to Googlebot Smartphone, but this can be adjusted to desktop if there are any differences in the way the site issues cookies.

3) Set The User-Agent To Chrome

To replicate a regular user, switch the user-agent to a browser, such as Chrome via ‘Config > User-agent’.

Some websites conditionally set cookies based upon user-agent and crawling as the ‘Screaming Frog SEO Spider’ may not always give a true picture otherwise.

4) Ignore robots.txt

Click ‘Configuration > robots.txt > Settings’ and select ‘Ignore robots.txt’ or ‘Ignore robots.txt but report status’.

https://www.screamingfrog.co.uk/wp-content/uploads/2020/12/Ignore-robots.txt.png

Cookies can be loaded from URLs which are available to users but not bots, such as pages or resources blocked via robots.txt. Therefore it’s important we allow all resources to be loaded. Any user set Include or Exclude functions can also affect this.

5) Crawl the Website

Open up the SEO Spider, type or copy in the website you wish to crawl in the ‘Enter URL to spider’ box and hit ‘Start’.

Wait until the crawl finishes and reaches 100%, but you can also view some details in real-time.

6) View Number of Cookies For Each URL

In the ‘Internal’ tab, there’s a ‘cookies’ column which displays the number of cookies discovered for each URL.

You will need to scroll over to the right to see it. This will help you identify where cookies are being discovered in the crawl.

7) View The Cookies Tab

Click on a URL in the top window, then the lower ‘Cookies’ tab to populate the lower window pane with more details on cookies discovered for each URL.

You can click on the above image to view a larger version. You’re able to see granular cookie data for every URL. You can also highlight multiple URLs at a time and view them together (the ‘address’ column on the right shows which URL each is on).

The columns listed in the Cookies tab include:

Cookie Name – The name of the cookie.
Cookie Value – The cookie value.
Domain – The domain that issued the cookie. These can be either first or third party.
Expiration Time – The cookie expiry time.
Secure – Details of the cookie secure attribute. True means the ‘secure’ attribute is present.
HttpOnly – Details of the cookie HttpOnly attribute. True means the ‘HttpOnly’ attribute is present.
Address – The URL the cookie was set on.

8) View Aggregated Cookie Summary Report

Export an aggregated summary of cookies discovered by clicking ‘Reports > Cookies > Cookie Summary’.

This shows an aggregated view of unique cookies discovered during a crawl, considering their name, domain, expiry, secure and HttpOnly values. The number of URLs each unique cookie was issued on will also be displayed. The cookie value itself is discounted in this aggregation (as they are unique!).

⭐️How To Calculate Keyword Density

Purpose: The purpose of keyword density calculation is to determine the prominence of a specific keyword or phrase within a given text or webpage. Keyword density was historically used as a metric to gauge the relevance and importance of a keyword within the content. However, it is important to note that keyword density is not as significant a factor in search engine rankings as it once was.

While keyword density is still considered by some as a basic SEO (Search Engine Optimization) factor, it is generally recommended to focus on creating high-quality, relevant content that is valuable to users, rather than obsessing over keyword density. It is important to strike a balance between using keywords naturally and ensuring that the content is readable and provides a good user experience.

⭐️Keyword Density Formula in Classic Way:

Keyword Density = ( KR / ( TW – ( KR x ( NWK – 1 ) ) ) ) x 100

KR = how many times key-phrases are repeated

NWK = number of words in key-phrases

TW = total words in the analysed text

Number of Key Phrases are Repeated:

Here the keyword “ashwagandha’s anti-inflammatory properties” has repeated for 8 times

So, Number of Key Phrases are Repeated: 8

Number Of words In Key-phrases:

Here 4 words are present in the keyword

So, Number Of words In Key-phrases: 4

Total Words In The Analysed Text:

Here the word number of the web page needs to be counted.

The word number of the blog is 1187.

So, Total Words In The Analysed Text: 1187

Calculation Of Keyword Density:

Keyword Density = ( KR / ( TW – ( KR x ( NWK – 1 ) ) ) ) x 100

KR = 8

NWK = 4

TW = 1187

So,

( 8 / ( 1187 – ( 8 x ( 4 – 1 ) ) ) ) x 100

=( 8 / ( 1187 – ( 8 x 3 ) ) ) x 100

=( 8 / ( 1187 – 24 ) ) x 100

=( 8 / 1163) x 100

=0.687876 ~ 0.7

So, the Keyword density is 0.7(approx)

The ideal score are as follows:

If the keyword density is between 0.5 – 1.5 = Good
If the keyword density is 1.5 – 2 = Need To reduce keyword numbers
If the keyword density is >2 = Keyword Stuffing

Now let us compare keyword density:

The score for this blog is 0.7(approx) which is within the zone of 0.5 – 1.5.Hence the keyword density of this current blog is 0.7, so it is Good.