3 Programmatic SEO Approach To Try For Your SEO Campaign

3 Programmatic SEO Approach To Try For Your SEO Campaign

SUPERCHARGE YOUR ONLINE VISIBILITY! CONTACT US AND LET’S ACHIEVE EXCELLENCE TOGETHER!

    ⭐️What is an SEO Competitor Analysis

    An SEO competitor analysis is a process of evaluating and analyzing the SEO strategies and performance of your competitors to gain insights and identify opportunities to improve your own search engine optimization efforts through programmatic SEO approach. It involves studying the websites and online presence of your competitors to understand their organic search rankings, keyword targeting, content strategy, backlink profile, and overall SEO tactics.

    programmatic seo approach

    How to perform an SEO competitor analysis using Python

    Set up a custom search engine

    1. Create a new search engine

    https://programmablesearchengine.google.com/controlpanel/create

    Enter a Name for your search engine

    1. New search engine has been created

    Here is a link to the Programmable search engine homepage and within a few clicks, you should be able to set up your own programmable search engine. 

    https://programmablesearchengine.google.com/controlpanel/all

    Then go Customise

    1. Go to Google Developer Console and click the Get a Key button.

    https://developers.google.com/custom-search/v1/overview#api_key

    Select your search engine and go Next.

    1. Competitor Analysis – SERP heatmap Master.ipynb

    https://colab.research.google.com/drive/1SqVPj7nYynXFHX8alYkAIp-Pm2ArUNQY?usp=sharing#scrollTo=6d0c64b0

    Run and Wait.

     Add Keywords and your Search Engine ID & API Key

    Run the serp_goog function

    View the results of the serp_rankings output

     5.  Run & Export Data to CSV

    7. Run the SERP Heatmap Function

    8. Call or Invoke the serp_heatmap function

    9. View your SERP Heatmap Data Visualization

    Cookie Audit Introduction:

    It’s essential today to understand the presence and purpose of cookies on a website. The data privacy landscape is continuing to evolve and there are various laws and regulations in place throughout the world, where cookie compliance is almost always included in some way.

    1) Select ‘Cookies’ For Extraction

    Open up the SEO Spider, go to ‘Config > Spider > Extraction’ and select ‘Cookies’ under ‘URL Details’.

    This means the SEO Spider will now store all cookies discovered.

    Please note, when you choose to store cookies, the auto exclusion performed by the SEO Spider for Google Analytics tracking tags is disabled to provide an accurate view of all cookies issued.

    This means it will affect your analytics reporting, unless you choose to exclude any tracking scripts from firing by using the exclude configuration (‘Config > Exclude’).

    2) Enable JavaScript Rendering

    Click ‘Config > Spider > Rendering’ and select ‘JavaScript’. This means the SEO Spider will open each web page in a headless Chrome browser behind the scenes.

    This is an important step, as it allows cookies that are loaded using JavaScript or pixel image tags to be discovered.

    The window size is automatically set to Googlebot Smartphone, but this can be adjusted to desktop if there are any differences in the way the site issues cookies.

    3) Set The User-Agent To Chrome

    To replicate a regular user, switch the user-agent to a browser, such as Chrome via ‘Config > User-agent’.

    Some websites conditionally set cookies based upon user-agent and crawling as the ‘Screaming Frog SEO Spider’ may not always give a true picture otherwise.

    4) Ignore robots.txt

    Click ‘Configuration > robots.txt > Settings’ and select ‘Ignore robots.txt’ or ‘Ignore robots.txt but report status’.

    https://www.screamingfrog.co.uk/wp-content/uploads/2020/12/Ignore-robots.txt.png

    Cookies can be loaded from URLs which are available to users but not bots, such as pages or resources blocked via robots.txt. Therefore it’s important we allow all resources to be loaded. Any user set Include or Exclude functions can also affect this.

    5) Crawl the Website

    Open up the SEO Spider, type or copy in the website you wish to crawl in the ‘Enter URL to spider’ box and hit ‘Start’.

    Wait until the crawl finishes and reaches 100%, but you can also view some details in real-time.

    6) View Number of Cookies For Each URL

    In the ‘Internal’ tab, there’s a ‘cookies’ column which displays the number of cookies discovered for each URL.

    You will need to scroll over to the right to see it. This will help you identify where cookies are being discovered in the crawl.

    7) View The Cookies Tab

    Click on a URL in the top window, then the lower ‘Cookies’ tab to populate the lower window pane with more details on cookies discovered for each URL.

    You can click on the above image to view a larger version. You’re able to see granular cookie data for every URL. You can also highlight multiple URLs at a time and view them together (the ‘address’ column on the right shows which URL each is on).

    The columns listed in the Cookies tab include:

    • Cookie Name – The name of the cookie.
    • Cookie Value – The cookie value.
    • Domain – The domain that issued the cookie. These can be either first or third party.
    • Expiration Time – The cookie expiry time.
    • Secure – Details of the cookie secure attribute. True means the ‘secure’ attribute is present.
    • HttpOnly – Details of the cookie HttpOnly attribute. True means the ‘HttpOnly’ attribute is present.
    • Address – The URL the cookie was set on.

    8) View Aggregated Cookie Summary Report

    Export an aggregated summary of cookies discovered by clicking ‘Reports > Cookies > Cookie Summary’.

    This shows an aggregated view of unique cookies discovered during a crawl, considering their name, domain, expiry, secure and HttpOnly values. The number of URLs each unique cookie was issued on will also be displayed. The cookie value itself is discounted in this aggregation (as they are unique!).

    ⭐️How To Calculate Keyword Density

    Purpose: The purpose of keyword density calculation is to determine the prominence of a specific keyword or phrase within a given text or webpage. Keyword density was historically used as a metric to gauge the relevance and importance of a keyword within the content. However, it is important to note that keyword density is not as significant a factor in search engine rankings as it once was.

    While keyword density is still considered by some as a basic SEO (Search Engine Optimization) factor, it is generally recommended to focus on creating high-quality, relevant content that is valuable to users, rather than obsessing over keyword density. It is important to strike a balance between using keywords naturally and ensuring that the content is readable and provides a good user experience.

    ⭐️Keyword Density Formula in Classic Way:

    Keyword Density = ( KR / ( TW – ( KR x ( NWK – 1 ) ) ) ) x 100

    KR = how many times key-phrases are repeated

    NWK = number of words in key-phrases

    TW = total words in the analysed text

    Number of Key Phrases are Repeated:

    Here the keyword “ashwagandha’s anti-inflammatory properties” has repeated for 8 times

    So, Number of Key Phrases are Repeated: 8

    Number Of words In Key-phrases:

    Here 4 words are present in the keyword

    So, Number Of words In Key-phrases: 4

    Total Words In The Analysed Text:

    Here the word number of the web page needs to be counted. 

    The word number of the blog is 1187.

    So, Total Words In The Analysed Text: 1187

    Calculation Of Keyword Density:

    Keyword Density = ( KR / ( TW – ( KR x ( NWK – 1 ) ) ) ) x 100

    KR = 8

    NWK = 4

    TW = 1187

    So,

    ( 8 / ( 1187 – ( 8 x ( 4 – 1 ) ) ) ) x 100

    =( 8 / ( 1187 – ( 8 x 3 ) ) ) x 100

    =( 8 / ( 1187 – 24 ) ) x 100

    =( 8 / 1163) x 100

    =0.687876 ~ 0.7

    So, the Keyword density is 0.7(approx)

    The ideal score are as follows:

    1. If the keyword density is between 0.5 – 1.5 = Good
    2. If the keyword density is 1.5 – 2 = Need To reduce keyword numbers
    3. If the keyword density is >2 = Keyword Stuffing

    Now let us compare keyword density:

    The score for this blog is 0.7(approx) which is within the zone of 0.5 – 1.5.Hence the keyword density of this current blog is 0.7, so it is Good.

    FAQ

    Programmatic SEO involves using automated or systematic methods to optimize a website, including competitor analysis, cookie audits, and keyword density calculations to improve search engine visibility efficiently.

    An SEO competitor analysis evaluates your competitors’ websites to understand their keywords, content strategy, backlink profile, and overall SEO tactics. This helps identify opportunities to improve your own SEO.

    You can set up a custom Google Programmable Search Engine, obtain an API key, run Python scripts (like SERP rankings and heatmap functions), and export the results to CSV for insights on competitor performance.

    A cookie audit involves analyzing all cookies on a website to understand their purpose, compliance with privacy laws, and how they affect analytics and user tracking. Tools like Screaming Frog SEO Spider can help perform this audit.

    Steps include enabling cookie extraction, rendering JavaScript, setting the user-agent to a browser, ignoring robots.txt, crawling the website, and reviewing cookies for each URL. You can then generate an aggregated cookie summary report.

    Keyword density measures how often a keyword or phrase appears on a page relative to the total words. It helps evaluate relevance, though high-quality content and user experience are now more important than strict keyword counts.

    Keyword density = (Number of times the key phrase appears / (Total words – ((Number of times key phrase appears) × (Words in key phrase – 1)))) × 100. Ideal keyword density is typically between 0.5% and 1.5%.

    While not a major ranking factor anymore, keyword density provides insight into content relevance and ensures keywords are used naturally without overstuffing or underusing them.

    Python, Google Programmable Search Engine, SEO Spider (Screaming Frog), and analytics tools for keyword and SERP analysis are commonly used for programmatic SEO.

    They provide actionable insights, automate data collection, identify gaps and opportunities, ensure privacy compliance, and optimize content for both search engines and users, boosting online visibility efficiently.

    Summary of the Page - RAG-Ready Highlights

    Below are concise, structured insights summarizing the key principles, entities, and technologies discussed on this page.

    The content outlines a programmatic SEO approach starting with SEO competitor analysis, which assesses competitors’ keywords, rankings, content structure, and backlink profiles to uncover optimisation opportunities. Using Python and Google’s Programmable Search Engine, users can automate SERP evaluations by generating custom search engines, retrieving API keys, and running a Colab-based SERP Heatmap notebook. The workflow helps visualise competitor positioning, rank distribution, and keyword gaps, enabling data-driven SEO strategy development.

    A second programmatic SEO technique involves performing an in-depth cookie audit using Screaming Frog SEO Spider. By enabling cookie extraction, JavaScript rendering, custom user-agents, and ignoring robots.txt, users can detect cookies issued across all URLs—including those set by scripts or blocked resources. The tool surfaces data like cookie name, domain, expiry, security attributes, and URL source. Exporting the aggregated cookie summary helps identify compliance issues, third-party trackers, and consistency gaps across the site.

    The final method focuses on keyword density analysis to evaluate how often a keyword appears relative to total word count. Using the classic formula, the example computes a density of 0.7—within the ideal 0.5–1.5 range. Although keyword density is no longer a major ranking factor, it remains useful for detecting keyword stuffing, ensuring natural language flow, and maintaining content readability. The goal is balanced optimisation rather than artificially inflating keyword frequency.

    Tuhin Banik - Author

    Tuhin Banik

    Thatware | Founder & CEO

    Tuhin is recognized across the globe for his vision to revolutionize digital transformation industry with the help of cutting-edge technology. He won bronze for India at the Stevie Awards USA as well as winning the India Business Awards, India Technology Award, Top 100 influential tech leaders from Analytics Insights, Clutch Global Front runner in digital marketing, founder of the fastest growing company in Asia by The CEO Magazine and is a TEDx speaker and BrightonSEO speaker.

    Leave a Reply

    Your email address will not be published. Required fields are marked *