SUPERCHARGE YOUR ONLINE VISIBILITY! CONTACT US AND LET’S ACHIEVE EXCELLENCE TOGETHER!
Artificial intelligence systems rarely work with only one possible answer.
They usually operate with candidates.
A retrieval system may find:
- 50 documents
- 20 passages
- 10 entities
- 8 products
- 6 possible answers
- 5 citations
- 4 recommendations
- 3 actions
The system then has to decide:
Which candidate should appear first?
This is a ranking problem.
AI systems increasingly require ranking at multiple stages:
- document retrieval
- passage retrieval
- entity selection
- source selection
- citation selection
- answer selection
- recommendation ordering
- tool selection
- action prioritisation
- product matching
- expert identification
- knowledge-source preference
A useful AI architecture therefore needs more than knowledge.
It needs a mechanism for determining which available option should receive the highest priority.
That is the role of ai-ranking-model.json.
In simple terms:
ai-ranking-model.json defines the machine-readable criteria used to score, compare and prioritise candidate information, entities, answers, sources or actions.
The file can potentially describe:
- ranking domains
- ranking candidates
- scoring signals
- signal weights
- evidence quality
- entity relevance
- topical relevance
- source authority
- freshness
- semantic similarity
- context fit
- citation support
- confidence
- penalties
- exclusions
- tie-breaking rules
- ranking thresholds
- reranking stages
- diversity rules
- ranking explanations
- evaluation metrics
- governance
The objective is not simply to create another list of SEO ranking factors.
The objective is to create a structured framework for AI prioritisation.
What Is ai-ranking-model.json?
ai-ranking-model.json is a proposed machine-readable JSON framework for representing how an AI system ranks multiple candidate results.
Those candidates may include:
- documents
- passages
- webpages
- knowledge objects
- entities
- answer primitives
- citations
- products
- services
- recommendations
- actions
- tools
A simple ranking rule might look like this:
{
“rankingModelId”: “ranking:answer-source:001”,
“candidateType”: “source”,
“signals”: [
{
“name”: “topicalRelevance”,
“weight”: 0.40
},
{
“name”: “sourceAuthority”,
“weight”: 0.30
},
{
“name”: “freshness”,
“weight”: 0.20
},
{
“name”: “citationSupport”,
“weight”: 0.10
}
]
}
The model describes how candidate sources may be evaluated relative to one another.
It does not itself contain all knowledge.
It defines the prioritisation logic applied to knowledge.
Why AI Systems Need Ranking Models
Retrieval without ranking can produce noise.
Suppose a user asks:
What is Generative Engine Optimization?
A retrieval system finds:
- an official definition page
- an old blog post
- a service page
- a glossary entry
- a third-party article
- an unrelated page mentioning GEO once
All six technically match the query.
But they should not necessarily receive equal priority.
The ranking layer needs to ask:
- Which source most directly answers the query?
- Which source is canonical?
- Which source is current?
- Which source has stronger evidence?
- Which candidate matches the exact entity?
- Which candidate is relevant to the user’s context?
Without ranking, AI retrieval may surface technically related but weak information.
A Ranking Model converts candidate retrieval into ordered relevance.
Candidates
↓
Scoring
↓
Filtering
↓
Reranking
↓
Final Priority
Retrieval Is Not Ranking
The distinction between retrieval and ranking is important.
Retrieval
Answers:
Which candidates might be relevant?
Example:
Retrieve all documents related to GEO.
Ranking
Answers:
Which of those candidates is most useful for this specific question?
Example:
Rank canonical GEO definition above a passing mention in an unrelated article.
A system may retrieve 100 candidates but only send the top five to an LLM.
The Ranking Model determines which five survive.
Ranking Within the ThatWare AI Stack
A machine-readable AI infrastructure may include multiple layers.
entity-registry.json
Defines:
What entities exist?
knowledge-graph.json
Defines:
How are those entities connected?
brand-memory.json
Defines:
What should the system remember?
context-engine.json
Defines:
What matters in this particular situation?
ai-answer-primitives.json
Defines:
What reusable knowledge can contribute to an answer?
reasoning-map.json
Defines:
Which reasoning relationships connect evidence and conclusions?
ai-ranking-model.json
Defines:
Which available candidate should receive the greatest priority?
It is ranking-first.
ai-decision-layer.json
Defines:
Given the ranked evidence, what decision should follow?
ai-feedback-loop.json
Defines:
Did the ranking, decision or answer perform well, and what should improve?
Together:
IDENTITY
↓
KNOWLEDGE
↓
CONTEXT
↓
RETRIEVAL
↓
RANKING
↓
ANSWER
↓
REASONING
↓
DECISION
↓
ACTION
↓
FEEDBACK
The Ranking Model sits between candidate discovery and final selection.
Ranking Happens More Than Once
A sophisticated AI system may rank candidates at several stages.
Stage 1: Document Ranking
Which documents should be retrieved?
Stage 2: Passage Ranking
Which sections of those documents matter most?
Stage 3: Entity Ranking
Which entity best matches the query?
Stage 4: Evidence Ranking
Which evidence should support the answer?
Stage 5: Answer Primitive Ranking
Which reusable answer unit best matches the intent?
Stage 6: Citation Ranking
Which source should be cited?
Stage 7: Recommendation Ranking
Which option should be shown first?
This can be represented as:
Query
↓
Documents Ranked
↓
Passages Ranked
↓
Entities Ranked
↓
Evidence Ranked
↓
Answer Components Ranked
↓
Final Response
Candidate Types
A Ranking Model should clearly define what it ranks.
Possible candidate types include:
document
passage
entity
source
citation
answer_primitive
product
service
recommendation
tool
action
Example:
{
“candidateType”: “answer_primitive”
}
Ranking signals should vary by candidate type.
For example, freshness may be critical for pricing information but less important for historical facts.
Core Ranking Signals
A machine-readable ranking framework may use many signals.
The most useful depend on the use case.
Semantic Relevance
Semantic relevance measures how closely the candidate meaning matches the question.
Example:
{
“signal”: “semanticRelevance”,
“description”: “Measures semantic similarity between the query and candidate.”
}
This may be calculated through:
- embeddings
- semantic encoders
- cross-encoders
- LLM rerankers
- lexical-semantic combinations
Topical Relevance
A document may contain the right words but still be about the wrong topic.
Topical relevance asks:
Is this candidate fundamentally about the subject?
Example:
A page titled:
Complete Guide to GEO
may deserve higher topical relevance than:
20 Marketing Trends Including GEO
for a direct GEO definition query.
Entity Match
Entity match verifies whether the candidate refers to the correct entity.
Example:
{
“signal”: “entityMatch”,
“requiredEntity”: “entity:concept:generative-engine-optimization”
}
This reduces ambiguity.
Canonicality
A canonical source can receive additional preference.
Example:
Official product page
>
third-party summary
for factual product information.
A ranking system may contain:
{
“signal”: “canonicalSource”,
“boost”: true
}
Canonical does not automatically mean correct.
But it can be an important ranking signal.
Authority
Authority evaluates the credibility of a source for the topic.
Potential indicators include:
- primary research
- official documentation
- subject-matter expertise
- publisher reputation
- cited evidence
- authorship
- editorial standards
Authority should not be reduced to superficial domain metrics alone.
Evidence Strength
A candidate containing explicit supporting evidence may rank higher than an unsupported assertion.
Example:
{
“signal”: “evidenceStrength”,
“values”: [
“primary”,
“supported”,
“secondary”,
“unsupported”
]
}
Freshness
Freshness matters when information changes.
High-freshness categories may include:
- prices
- schedules
- regulations
- product features
- service availability
- leadership
- statistics
Example:
{
“signal”: “freshness”,
“maximumAgeDays”: 30
}
Context Fit
The same candidate may rank differently depending on context.
For example:
Query:
Best SEO service
User A:
Local restaurant
User B:
Global ecommerce company
The same service should not necessarily rank first.
Context fit may consider:
- company size
- industry
- geography
- user intent
- lifecycle stage
- budget
- technical maturity
Intent Match
A ranking system should identify search intent.
Examples:
informational
commercial
transactional
navigational
comparison
troubleshooting
research
A definition page may rank highly for informational intent.
A pricing page may rank highly for commercial intent.
Source Type
Certain queries require preferred source types.
Example:
For:
Current product pricing
preferred sources may be:
- official pricing page
- official documentation
- verified sales documentation
- third-party comparison
The ranking framework can encode that preference.
Citation Support
A candidate source may mention a topic without supporting the actual claim.
Citation support evaluates:
Does the candidate directly justify the statement being made?
This becomes especially important for generative answers.
Specificity
Specific information can outrank generic information.
For example:
AI SEO services for enterprise SaaS
may be more relevant than:
Digital marketing services
when the query is specific.
Completeness
Some candidates answer only part of the question.
A completeness score may consider how much of the information need is covered.
Confidence
Some internal systems may generate confidence values.
However, confidence should have a defined methodology.
Avoid unexplained fields such as:
{
“confidence”: 0.99
}
unless the score has a meaningful calculation.
Ranking Penalties
Ranking does not only involve positive signals.
Weak candidates may receive penalties.
Examples:
- outdated information
- duplicate content
- missing evidence
- wrong entity
- source conflict
- unsupported claim
- low relevance
- incomplete answer
- policy violation
Example:
{
“penalties”: [
{
“condition”: “stale”,
“value”: -0.25
},
{
“condition”: “unsupported_claim”,
“value”: -0.40
}
]
}
Hard Exclusions vs Ranking Penalties
A candidate may be weak or completely ineligible.
Penalty
The candidate remains eligible but ranks lower.
Example:
Article is slightly outdated.
Exclusion
The candidate should not be considered.
Example:
Wrong entity.
Structured example:
{
“exclusionRules”: [
“wrong_entity”,
“invalid_source”,
“prohibited_content”
]
}
Ranking Weights
A ranking model may assign weights.
Example:
{
“weights”: {
“semanticRelevance”: 0.35,
“entityMatch”: 0.20,
“authority”: 0.20,
“freshness”: 0.15,
“citationSupport”: 0.10
}
}
But numbers should not be added simply to make the model look sophisticated.
Weights should ideally be:
- tested
- explainable
- validated
- adjusted through evaluation
Dynamic Weighting
Different queries may require different weighting.
Example:
Historical Query
Who founded the organisation?
Freshness may matter relatively little.
Price Query
What does the product cost today?
Freshness becomes critical.
A dynamic model might contain:
{
“dynamicWeights”: {
“pricing”: {
“freshness”: 0.35
},
“historical”: {
“freshness”: 0.05
}
}
}
Ranking Thresholds
Candidates below a certain quality level may be rejected.
Example:
{
“minimumScore”: 0.65
}
If no candidate passes:
Do not fabricate an answer.
The system can instead:
- retrieve more sources
- broaden the search
- ask for clarification
- abstain
Top-K Selection
AI systems often select a limited number of candidates.
Example:
{
“selection”: {
“topK”: 5
}
}
This controls how many results continue to the next stage.
Too few candidates may reduce recall.
Too many may introduce irrelevant context.
Reranking
Initial retrieval may be fast but approximate.
A second ranking stage can improve precision.
Example architecture:
Vector Search
↓
Top 100 Candidates
↓
Cross-Encoder Reranker
↓
Top 20
↓
LLM Reranker
↓
Top 5
This is called reranking.
A Ranking Model can define these stages.
Lexical and Semantic Hybrid Ranking
Exact terminology still matters.
A hybrid ranking system may combine:
BM25 / keyword match
+
semantic embeddings
+
entity match
+
authority
Example:
{
“retrievalMode”: “hybrid”,
“signals”: [
“lexicalMatch”,
“semanticMatch”,
“entityMatch”
]
}
Hybrid systems often provide better coverage than relying on only one retrieval method.
Source Ranking
When multiple sources support a claim, the system can prioritise.
Example:
Primary Source
↓
Official Documentation
↓
Research Institution
↓
Trusted Secondary Source
↓
General Secondary Source
The precise hierarchy should vary by topic.
Citation Ranking
Citation ranking may consider:
- claim support
- source authority
- canonicality
- freshness
- accessibility
- specificity
Example:
{
“rankingModelId”: “ranking:citation:001”,
“signals”: [
“claimSupport”,
“primarySource”,
“freshness”,
“specificity”
]
}
Entity Ranking
A query may match several entities.
Example:
Apple
Potential entities:
- Apple Inc.
- apple fruit
- Apple Records
- Apple Bank
Entity ranking can consider:
- surrounding context
- user history
- topic
- geographic context
- semantic similarity
Example:
{
“candidateType”: “entity”,
“signals”: [
“contextMatch”,
“semanticMatch”,
“entityPopularity”,
“canonicalMatch”
]
}
Answer Primitive Ranking
Suppose the system contains:
primitive:geo:definition
primitive:geo:benefit
primitive:geo:limitation
primitive:geo:process
Question:
What is GEO?
The ranking model should prefer:
primitive:geo:definition
Question:
Does GEO guarantee ChatGPT rankings?
The model should prefer:
primitive:geo:limitation
This can be represented through intent matching.
Recommendation Ranking
Recommendation ranking is different from document ranking.
The goal is not simply:
Which product mentions the query most often?
The system should evaluate suitability.
Signals might include:
- use-case fit
- eligibility
- budget fit
- availability
- user constraints
- evidence
- risk
Example:
{
“candidateType”: “service”,
“signals”: [
“goalFit”,
“businessSizeFit”,
“technicalFit”,
“availability”
]
}
Tool Ranking for AI Agents
Agents may have access to multiple tools.
Example:
Web search
Database search
CRM
Calendar
Knowledge base
The Ranking Model can help determine which tool should be tried first.
Example:
{
“candidateType”: “tool”,
“signals”: [
“taskCompatibility”,
“dataAuthority”,
“cost”,
“latency”,
“risk”
]
}
Action Ranking
An AI agent may identify multiple next actions.
Example:
Retrieve more evidence
Ask the user
Escalate to human
Generate recommendation
The Ranking Model can order possible actions before the Decision Layer makes the final selection.
Ranking vs Decision
These layers are related but distinct.
Ranking Model
Answers:
Which option should receive the highest priority?
Decision Layer
Answers:
What should actually happen?
For example:
Ranking:
1. GEO
2. LLM SEO
3. Traditional SEO
Decision:
Recommend GEO and LLM SEO.
The Ranking Model orders.
The Decision Layer acts.
Ranking vs Reasoning
Reasoning explains relationships.
Ranking evaluates priority.
Example:
Reasoning Map:
AI Visibility
↓ related to
Entity Authority
Citation Support
Retrievability
Ranking Model:
Candidate A has stronger entity authority.
Candidate B has fresher citations.
Candidate C has better semantic relevance.
The model combines those signals to rank candidates.
Ranking vs Feedback
Ranking determines priority.
Feedback evaluates whether the priority was correct.
Example:
Ranking Model chooses Source A.
↓
Answer performs poorly.
↓
Feedback Loop identifies Source B was more accurate.
↓
Ranking Model adjusted.
This creates ranking optimisation.
Recommended File Location
A public resource could be located at:
Alternative:
or:
However, detailed ranking weights may contain proprietary logic.
A public version may therefore describe principles while keeping operational ranking models private.
Recommended MIME Type
Serve as:
application/json
Recommended response:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Recommended Top-Level Structure
A mature file may contain:
{
“metadata”: {},
“organization”: {},
“rankingDomains”: [],
“candidateTypes”: [],
“signalDefinitions”: {},
“rankingModels”: [],
“penalties”: {},
“exclusionRules”: {},
“reranking”: {},
“tieBreaking”: {},
“diversityPolicy”: {},
“evaluation”: {},
“relatedResources”: {},
“governance”: {}
}
Field-by-Field Explanation
metadata
Example:
{
“metadata”: {
“version”: “2026.1”,
“fileType”: “ai-ranking-model”,
“generatedAt”: “2026-09-03”,
“lastUpdated”: “2026-09-03”,
“publisher”: “ThatWare LLP”,
“language”: “en”,
“canonicalUrl”: “https://thatware.co/ai-ranking-model.json”
}
}
rankingModelId
Every ranking model should have a stable identifier.
Example:
{
“rankingModelId”: “ranking:answer-primitive:definition”
}
Possible pattern:
ranking:<candidate-type>:<purpose>
Examples:
ranking:document:retrieval
ranking:entity:disambiguation
ranking:citation:selection
ranking:service:recommendation
rankingDomain
Defines where the model applies.
Example:
{
“rankingDomain”: “answer_generation”
}
Domains may include:
retrieval
citation
recommendation
entity_resolution
agent_action
tool_selection
candidateType
Example:
{
“candidateType”: “document”
}
signals
Signals determine ranking quality.
Example:
{
“signals”: [
{
“name”: “semanticRelevance”,
“weight”: 0.35
},
{
“name”: “authority”,
“weight”: 0.25
}
]
}
signalDirection
Some signals should increase ranking.
Others reduce ranking.
Example:
{
“signal”: “freshness”,
“direction”: “positive”
}
Example:
{
“signal”: “staleness”,
“direction”: “negative”
}
normalization
Different signals may use different scales.
Example:
semantic similarity = 0 to 1
authority = 0 to 100
freshness = days old
Normalization converts them into comparable ranges.
Example:
{
“normalization”: “min_max”
}
penalties
Example:
{
“penalties”: [
{
“condition”: “duplicate”,
“penalty”: 0.15
},
{
“condition”: “stale”,
“penalty”: 0.30
}
]
}
exclusions
Example:
{
“excludeWhen”: [
“wrong_entity”,
“invalid_canonical”,
“policy_violation”
]
}
thresholds
Example:
{
“thresholds”: {
“minimumCandidateScore”: 0.60
}
}
reranking
Example:
{
“reranking”: {
“enabled”: true,
“initialTopK”: 50,
“finalTopK”: 5
}
}
tieBreaking
When candidates receive similar scores, additional rules may apply.
Example:
{
“tieBreaking”: [
“primarySource”,
“freshness”,
“canonicalUrl”
]
}
diversityPolicy
Ranking only highly similar candidates can reduce answer diversity.
A diversity policy may prevent:
5 nearly identical passages
from occupying all available context.
Example:
{
“diversityPolicy”: {
“deduplicateSemanticOverlap”: true,
“maximumCandidatesPerSource”: 2
}
}
explanation
Ranking should ideally be explainable.
Example:
{
“rankingExplanation”: {
“candidate”: “source:A”,
“reasons”: [
“highest semantic relevance”,
“canonical source”,
“current documentation”
]
}
}
Complete Example ai-ranking-model.json
{
“metadata”: {
“version”: “2026.1”,
“fileType”: “ai-ranking-model”,
“generatedAt”: “2026-09-03”,
“lastUpdated”: “2026-09-03”,
“publisher”: “ThatWare LLP”,
“language”: “en”,
“canonicalUrl”: “https://thatware.co/ai-ranking-model.json”,
“description”: “Machine-readable framework for ranking documents, entities, answer primitives, citations, recommendations and other AI candidates.”
},
“organization”: {
“id”: “entity:organization:thatware”,
“name”: “ThatWare”,
“legalName”: “ThatWare LLP”,
“url”: “https://thatware.co/”
},
“rankingDomains”: [
“retrieval”,
“entity_resolution”,
“answer_generation”,
“citation”,
“recommendation”
],
“signalDefinitions”: {
“semanticRelevance”: {
“type”: “positive”,
“description”: “Semantic similarity between the query and candidate.”
},
“entityMatch”: {
“type”: “positive”,
“description”: “Degree to which the candidate refers to the correct canonical entity.”
},
“authority”: {
“type”: “positive”,
“description”: “Evidence-backed credibility of the source for the relevant subject.”
},
“freshness”: {
“type”: “positive”,
“description”: “Recency relative to the information type.”
},
“citationSupport”: {
“type”: “positive”,
“description”: “Degree to which the candidate supports the intended claim.”
},
“staleness”: {
“type”: “negative”
},
“entityConflict”: {
“type”: “negative”
}
},
“rankingModels”: [
{
“rankingModelId”: “ranking:document:retrieval”,
“candidateType”: “document”,
“signals”: [
{
“name”: “semanticRelevance”,
“weight”: 0.35
},
{
“name”: “entityMatch”,
“weight”: 0.25
},
{
“name”: “authority”,
“weight”: 0.20
},
{
“name”: “freshness”,
“weight”: 0.20
}
],
“minimumScore”: 0.60,
“reranking”: {
“enabled”: true,
“initialTopK”: 50,
“finalTopK”: 10
}
},
{
“rankingModelId”: “ranking:answer-primitive”,
“candidateType”: “answer_primitive”,
“signals”: [
{
“name”: “intentMatch”,
“weight”: 0.40
},
{
“name”: “semanticRelevance”,
“weight”: 0.30
},
{
“name”: “entityMatch”,
“weight”: 0.20
},
{
“name”: “freshness”,
“weight”: 0.10
}
]
},
{
“rankingModelId”: “ranking:citation”,
“candidateType”: “citation”,
“signals”: [
{
“name”: “citationSupport”,
“weight”: 0.40
},
{
“name”: “authority”,
“weight”: 0.30
},
{
“name”: “canonicalSource”,
“weight”: 0.20
},
{
“name”: “freshness”,
“weight”: 0.10
}
]
}
],
“globalExclusionRules”: [
“wrong_entity”,
“invalid_source”,
“unsupported_claim”,
“policy_violation”
],
“tieBreaking”: [
“primary_source”,
“canonical_source”,
“freshness”
],
“diversityPolicy”: {
“deduplicateNearDuplicates”: true,
“maximumCandidatesPerSource”: 2
},
“evaluation”: {
“metrics”: [
“precision_at_k”,
“recall_at_k”,
“mean_reciprocal_rank”,
“ndcg”,
“citation_support_rate”,
“human_relevance_score”
]
},
“relatedResources”: {
“entityRegistry”: “https://thatware.co/entity-registry.json”,
“knowledgeGraph”: “https://thatware.co/knowledge-graph.json”,
“answerPrimitives”: “https://thatware.co/ai-answer-primitives.json”,
“decisionLayer”: “https://thatware.co/ai-decision-layer.json”,
“feedbackLoop”: “https://thatware.co/ai-feedback-loop.json”,
“contextEngine”: “https://thatware.co/context-engine.json”
},
“governance”: {
“versionWeights”: true,
“testRankingChanges”: true,
“monitorRankingBias”: true,
“allowRollback”: true
}
}
AI Ranking Model and RAG
RAG depends heavily on ranking.
A basic RAG pipeline may retrieve documents based on vector similarity.
But semantic similarity alone may not be enough.
A more advanced pipeline may use:
Query
↓
Vector Retrieval
↓
Keyword Retrieval
↓
Entity Matching
↓
Candidate Merge
↓
Ranking Model
↓
Reranker
↓
Top Evidence
↓
LLM
The Ranking Model determines what enters the context window.
That can materially affect answer quality.
First-Stage Retrieval vs Second-Stage Ranking
Fast retrieval systems often optimise for recall.
The goal is:
Find as many potentially relevant candidates as possible.
Second-stage ranking optimises for precision.
The goal is:
Put the best candidates first.
Architecture:
Stage 1
Broad Retrieval
↓ 100 Candidates
Stage 2
Detailed Reranking
↓ 10 Candidates
Stage 3
Final Evidence Selection
Ranking Metrics
Ranking quality requires measurement.
Several information-retrieval metrics may be useful.
Precision@K
Measures how many of the top K results are relevant.
Example:
Top 10 results
8 relevant
Precision@10 = 0.8
Recall@K
Measures how much relevant information was successfully retrieved.
Mean Reciprocal Rank
Measures how highly the first correct result appears.
If the correct result is first, performance is strong.
NDCG
Normalized Discounted Cumulative Gain evaluates whether highly relevant items appear near the top.
Human Relevance Rating
Human reviewers can score ranking quality.
Metrics should match the actual use case.
Ranking Evaluation Dataset
A Ranking Model requires test queries.
Example:
{
“query”: “What is Generative Engine Optimization?”,
“expectedTopCandidate”: “primitive:geo:definition”
}
Another:
{
“query”: “What are the limitations of GEO?”,
“expectedTopCandidate”: “primitive:geo:limitation”
}
These can become ranking benchmarks.
Feedback-Driven Ranking Improvement
The Feedback Loop can identify ranking mistakes.
Example:
Query
↓
Source B ranked #1
↓
Human reviewer says Source A is superior
↓
Ranking error recorded
↓
Signal weights reviewed
↓
Model retested
This creates:
Ranking
↓
Evaluation
↓
Adjustment
↓
New Ranking
Ranking Bias
Ranking models can introduce bias.
Examples:
- always favouring organisation-owned sources
- favouring large brands regardless of relevance
- ignoring smaller authoritative experts
- overvaluing popularity
- underweighting primary evidence
A governance framework should evaluate these effects.
Example:
{
“biasMonitoring”: {
“enabled”: true,
“reviewSourceConcentration”: true
}
}
Brand-Owned Sources Should Not Automatically Win
A company may be tempted to create:
If ThatWare source exists
→ always rank first
That would undermine reliability.
A better principle is:
Prefer canonical first-party information when the query concerns that organisation’s own factual information, provided the source is current and relevant.
For independent evaluation or comparative questions, external evidence may deserve greater weight.
Ranking and Commercial Bias
Recommendation ranking should avoid turning the framework into a sales engine.
Example of poor logic:
Most expensive package
→ rank first
Better ranking factors include:
- actual user fit
- availability
- needs
- evidence
- constraints
Temporal Ranking
Freshness should be query-sensitive.
Example:
Who founded ThatWare?
Historical source from several years ago may remain valid.
But:
What services does ThatWare offer today?
recent information becomes much more important.
This can be encoded using content classes.
Query Classification Before Ranking
Ranking improves when the system understands the question first.
Possible classification:
Definition
Comparison
Current Fact
Historical Fact
Recommendation
Troubleshooting
Commercial
Research
Different ranking models can then be selected.
Query
↓
Intent Classification
↓
Choose Ranking Model
↓
Rank Candidates
Dynamic Ranking Models
A sophisticated system may not use one universal ranking formula.
It can select models dynamically.
Example:
{
“routing”: {
“definition”: “ranking:definition”,
“pricing”: “ranking:freshness-sensitive”,
“research”: “ranking:evidence-first”,
“recommendation”: “ranking:user-fit”
}
}
This is often more sensible than one formula for every query.
Ranking Explainability
For enterprise systems, reviewers may need to know:
Why did Candidate A outrank Candidate B?
Example:
{
“candidate”: “source:A”,
“scoreBreakdown”: {
“semanticRelevance”: 0.92,
“authority”: 0.90,
“freshness”: 0.85
}
}
Again, score values should only be surfaced when meaningful.
An explanation could instead be categorical:
Candidate A ranked first because it was the canonical source, directly answered the query and contained current verified information.
Diversity and Redundancy
Suppose top retrieval results are:
Page A paragraph 1
Page A paragraph 2
Page A paragraph 3
Page A paragraph 4
Page A paragraph 5
This may reduce evidence diversity.
The model might instead select:
Canonical Page
Research Source
Supporting Documentation
Relevant FAQ
depending on the task.
Diversity should not override relevance, but it can improve robustness.
Ranking and Contradictory Evidence
Ranking models should not silently hide meaningful disagreement.
Suppose:
Source A says 30 days.
Source B says 60 days.
The system should not necessarily rank one and suppress the other without review.
A contradiction detector may trigger:
Conflict
↓
Decision Layer
↓
Retrieve Additional Evidence
or
Display Uncertainty
Rank Fusion
Multiple retrieval methods may produce different rankings.
Example:
Keyword Search Ranking
Vector Search Ranking
Knowledge Graph Ranking
Rank fusion combines them.
Methods may include:
- weighted fusion
- reciprocal rank fusion
- learned ranking
A JSON model may document:
{
“fusionMethod”: “reciprocal_rank_fusion”
}
Learning-to-Rank
Traditional rules use manually defined weights.
Learning-to-rank systems learn ranking functions from labelled examples.
Training data may contain:
Query
Candidate A → highly relevant
Candidate B → relevant
Candidate C → irrelevant
A learned model can identify which signals best predict relevance.
A public JSON file should not expose proprietary model internals unless intentionally disclosed.
LLM-Based Reranking
Large language models can also rerank candidates.
Prompt:
Given this user question and these candidate passages, rank the passages by usefulness and factual support.
The Ranking Model can define:
- candidate limits
- required evidence
- output format
- exclusion rules
LLM-based reranking can be powerful but may be slower and more expensive than simpler methods.
Ranking for AI Search Visibility
For GEO and AEO analysis, ranking models can also help study why certain brands or sources become strong candidates.
Potential dimensions may include:
- entity relevance
- topical authority
- structured information
- evidence
- citations
- freshness
- semantic coverage
- source consistency
This does not mean public LLM ranking algorithms are known.
An internal model can instead be used as an analytical framework.
Important Distinction: Internal Ranking Model vs External AI Ranking
A ThatWare ranking model can model:
How we believe candidate quality should be evaluated.
It cannot truthfully claim:
This is exactly how ChatGPT ranks brands.
Public AI systems use proprietary architectures that are not fully exposed.
Therefore ai-ranking-model.json should remain clearly positioned as:
- internal machine-readable infrastructure
- research framework
- RAG ranking architecture
- AI visibility analysis framework
not as a leaked or guaranteed third-party algorithm.
Common Mistakes to Avoid
Mistake 1: Calling It Google’s AI Ranking Algorithm
Do not imply that a custom ranking model represents Google’s actual ranking formula.
Mistake 2: Using One Ranking Formula for Everything
Pricing, research and historical queries may require different priorities.
Mistake 3: Overweighting Keyword Matching
Exact words do not guarantee relevance.
Mistake 4: Ignoring Entity Identity
Wrong-entity matches can appear highly semantically similar.
Mistake 5: Ignoring Freshness
Current questions require current evidence.
Mistake 6: Ranking Unsupported Claims Highly
Evidence quality should matter.
Mistake 7: Arbitrary Weights
Do not invent mathematical precision without testing.
Mistake 8: No Minimum Threshold
If all candidates are weak, the system should not force one to rank first.
Mistake 9: No Diversity Control
Duplicate passages can dominate context.
Mistake 10: Always Ranking First-Party Sources First
First-party information is important for organisational facts but should not automatically override stronger independent evidence in every context.
Mistake 11: No Feedback Loop
Ranking quality should be measured and corrected.
Mistake 12: No Version Control
Changing ranking weights can materially change system behaviour.
Mistake 13: Hiding Contradictory Evidence
Material conflicts should be surfaced rather than silently suppressed.
Mistake 14: Treating Ranking as Decision-Making
Ranking orders candidates.
Decision logic determines the final action.
Implementation Process
Step 1: Identify Candidate Types
Determine what needs ranking.
Examples:
documents
entities
answer primitives
citations
services
Step 2: Define Ranking Goals
Ask:
What does “better” mean for this ranking?
For retrieval, it may mean relevance.
For citations, it may mean claim support.
For recommendations, it may mean user fit.
Step 3: Define Signals
Select meaningful ranking factors.
Step 4: Define Exclusions
Identify candidates that should never be ranked.
Step 5: Create Initial Weights
Only where weighting is useful.
Step 6: Build Evaluation Queries
Create known query-result expectations.
Step 7: Test Ranking
Measure:
- precision
- recall
- top-result quality
- human relevance
Step 8: Add Reranking
Use more sophisticated ranking for top candidates where useful.
Step 9: Add Tie-Breaking
Define what happens when scores are similar.
Step 10: Add Diversity Controls
Prevent unnecessary duplication.
Step 11: Connect the Feedback Loop
Capture poor ranking outcomes.
Step 12: Version Changes
Track modifications.
Step 13: Monitor Bias
Review whether certain source types or entities are disproportionately favoured.
Step 14: Add Rollback
Allow ranking changes to be reversed.
Validation Checklist
Before deploying ai-ranking-model.json, verify:
- JSON is syntactically valid.
- Metadata is present.
- Model version is identified.
- Ranking model IDs are unique.
- Candidate types are explicit.
- Ranking domains are defined.
- Signals are clearly described.
- Signal direction is known.
- Weights are justified where used.
- Dynamic weighting is available where appropriate.
- Exclusion rules are explicit.
- Minimum thresholds exist where needed.
- Tie-breaking is defined.
- Duplicate candidates are controlled.
- Freshness is query-sensitive.
- Entity identity is validated.
- Primary evidence can receive appropriate priority.
- Unsupported claims receive penalties.
- Contradictory evidence is handled.
- Ranking outputs can be evaluated.
- Regression tests exist.
- Human evaluation is available.
- Ranking changes are versioned.
- Rollback is supported.
- Commercial bias is monitored.
- Public files do not expose sensitive proprietary ranking logic unintentionally.
- The framework is not presented as the actual proprietary ranking algorithm of Google or another public AI system.
Example Connection With ai-answer-primitives.json
Question
↓
Retrieve Relevant Primitives
↓
Ranking Model
↓
Rank:
1. Definition Primitive
2. Related Explanation
3. Benefit Primitive
↓
Select Best Context
Example Connection With ai-decision-layer.json
Rank Candidate Services
↓
1. GEO
2. LLM SEO
3. SEO
↓
Decision Layer
↓
Recommend GEO + LLM SEO
Example Connection With ai-feedback-loop.json
Ranking Model
↓
Source A ranked first
↓
Human says Source A outdated
↓
Feedback Loop
↓
Freshness weighting reviewed
↓
Model updated
Example Connection With context-engine.json
User Context
↓
Enterprise
US Market
AI Visibility Goal
↓
Context Engine
↓
Ranking Model
↓
Candidates ranked according to enterprise AI-search relevance
Example ai.txt Reference
# AI Ranking Resources
Ranking Model:
Decision Layer:
Feedback Loop:
This documents the resource.
It does not require external AI systems to consume it.
Example llms.txt Reference
## AI Ranking Model
Machine-readable ranking principles for retrieval, evidence selection, citations and AI recommendations:
Strategic Value of an AI Ranking Layer
The web traditionally focuses on ranking webpages.
AI systems expand ranking beyond webpages.
They increasingly need to rank:
Entities
Facts
Evidence
Sources
Passages
Answers
Products
Recommendations
Actions
This creates a new architectural requirement.
A website may contain excellent information, but AI systems still need to decide:
Which part matters most right now?
The Ranking Model formalises that prioritisation process.
From Search Ranking to AI Candidate Ranking
Traditional SEO often asks:
How does my webpage rank?
AI systems create a broader question:
How does my information rank among all candidate knowledge available to the model?
That may include:
- page ranking
- passage ranking
- entity ranking
- citation ranking
- answer ranking
- recommendation ranking
The unit of competition is no longer always a webpage.
Sometimes it is a fact.
Sometimes it is a source.
Sometimes it is an entity.
Sometimes it is an answer fragment.
Ranking as a Bridge Between Retrieval and Reasoning
Retrieval creates possibilities.
Ranking determines priority.
Reasoning interprets the highest-quality evidence.
Decision logic determines the next action.
This creates:
Retrieve
↓
Rank
↓
Reason
↓
Decide
↓
Act
The better the ranking layer performs, the better the downstream system can potentially perform.
Final Summary
ai-ranking-model.json is a proposed machine-readable architecture for defining how AI systems prioritise candidate information before answer generation, recommendation or action.
It can describe:
- candidate types
- ranking domains
- relevance signals
- entity match
- authority
- evidence quality
- freshness
- context fit
- intent match
- signal weights
- penalties
- exclusions
- thresholds
- reranking
- tie-breaking
- diversity
- evaluation
- governance
Its role within the broader AI stack can be summarised as follows:
Entity Registry
Defines what something is.
Knowledge Graph
Defines how things are related.
Brand Memory
Defines what should be remembered.
Context Engine
Defines what matters right now.
AI Ranking Model
Defines which candidate deserves the highest priority.
AI Answer Primitives
Define what reusable knowledge can be said.
Reasoning Map
Defines how evidence and ideas are connected.
AI Decision Layer
Defines what should happen.
AI Feedback Loop
Determines whether the result was good and what should improve.
The Ranking Model is therefore the bridge between:
Available Information
and:
Selected Information
Its purpose is not to manufacture rankings.
Its purpose is to make prioritisation structured, explainable, testable and governable.
As AI search, RAG, semantic retrieval and autonomous agents become more sophisticated, organisations increasingly need to understand not only whether information can be retrieved, but why one piece of information should be selected above another.
That is the role ai-ranking-model.json is designed to support.
