Full graph →

Search Engine Optimization

SEO for a content site is not a trick — it is the discipline of making your content legible to a crawler: reachable, unambiguous, and worth ranking. A search engine has to find a page, fetch it, render it, decide which of several near-identical URLs is the real one, judge the content, and finally present a title and URL that a human chooses to click. Each of those steps is a lever you control, and they are not equally powerful. Most SEO advice fixates on the weakest lever (keywords in URLs) while ignoring the strong ones (crawlable hierarchy, canonicalization, sitemaps, per-page metadata).

Provenance: the symbology URL scheme redesign (gitea sfi/symbology, milestone "URL scheme & SEO", issues #136–#141). The site's original "reddit-inspired" routes — /f/0001402057-26-000011, /d/<accession>/<content-hash> — were compact and shareable but told neither crawlers nor humans what a page contained, and the filing pages had no URL relationship to their company at all.

The pipeline: crawl, index, rank, click

Crawling is discovery: the engine follows links and sitemap entries to enumerate your URLs, budgeted per site. Indexing is the decision to store and serve a page. Ranking is where a page appears for a query. Click-through is the human choosing your result over the one above it. The distinction matters because interventions target different stages — a sitemap improves crawling, noindex controls indexing, content quality dominates ranking, and titles/URLs drive click-through. Diagnosing "we don't show up in Google" starts with locating which stage is failing (Google Search Console reports each separately).

Two structural prerequisites come before any of the levers below. First, the content must be server-rendered: crawlers execute JavaScript inconsistently and at lower priority, so a client-only SPA is a soft failure mode. SvelteKit's SSR satisfies this by default. Second, the site needs an internal link hierarchy a crawler can walk: every canonical page reachable by following links from the root. A page that only exists behind a search box or a POST is invisible.

URLs: identity vs. display

The honest assessment: keywords in the URL are a minor ranking signal. What a readable URL actually buys is click-through on a results page — /companies/CDW/10-K/2026/business-description versus /d/0001402057-26-000011/6e57c5cb1c69 — plus shareability and a hierarchy that mirrors your internal linking. Design URLs for humans and let the crawler benefit incidentally.

The structural insight from the symbology redesign is that URLs split into two classes:

  • Canonical content pages carry human-meaningful segments: entity names, document types, periods. These are the pages you want indexed, so every segment should tell a reader what they'll find. Hang them off one spine (/companies/[ticker]/...) so the URL tree and the link hierarchy coincide.
  • Permalinks carry stable identifiers: content hashes, accession numbers, UUIDs. Their job is to never break, not to rank. It is correct — not a compromise — for a provenance viewer like /s/<content-hash> to stay hash-addressed; content-addressed identity is the feature.

The tension inside the first class is that human-meaningful names are mutable. A stock ticker reads well in a URL but companies rename; the stable identifier (EDGAR's CIK, an internal ID) is unreadable. The resolution is to build URLs from the display name, track its history, and redirect old names to current ones — accepting that you now own a rename-handling problem the opaque identifier never had. Whichever way you choose, keep the stable-identifier route alive forever as a redirect: inbound links are an asset you never voluntarily break.

Migrations between schemes use permanent redirects — 301, or 308 when the method must be preserved. Permanent status codes transfer accumulated link equity to the new URL; a 302/307 tells the engine the move is temporary and it should keep the old URL indexed. Redirect shims are cheap and should be immortal.

Canonicalization: one page, one URL

Search engines dedupe aggressively, and duplicate content is easier to create than it looks: http vs https, trailing slash variants, query parameters, and any URL scheme that accepts aliases. Symbology's synthesis viewer resolves content-hash prefixes, so /s/468a, /s/468a05d8, and /s/468a05d8b3f8 all render the same page — an unbounded family of aliases for every piece of content.

The fix is <link rel="canonical" href="..."> in the head of every content page, naming the one true URL for that content. The engine consolidates signals from all aliases onto the canonical and indexes only it. Rules of thumb:

  • Every indexable page declares a canonical, even when it points at itself — it costs nothing and preempts parameter/alias duplication you didn't anticipate.
  • The canonical must be absolute and must be the URL your internal links use. A canonical that disagrees with your own link graph sends mixed signals.
  • Canonical is a hint; redirects are a directive. Where you can redirect an alias, do that instead. Canonical is for aliases that must keep rendering (the hash-prefix case).

Sitemaps: telling the crawler what exists

A sitemap is an XML enumeration of your canonical URLs, declared in robots.txt, that decouples discovery from link-walking. For a catalog-shaped site (thousands of company/filing/document pages), it is the difference between the crawler finding your long tail in days versus months — and <lastmod> timestamps let it re-crawl only what changed.

Serve it dynamically from the database rather than generating files; the sitemap is a view of the catalog:

// src/routes/sitemap.xml/+server.ts (SvelteKit)
export const GET: RequestHandler = async ({ setHeaders }) => {
	setHeaders({
		'Content-Type': 'application/xml',
		'Cache-Control': 'public, max-age=3600' // full-catalog query; don't serve it per-crawl
	});
	const pages = await getCanonicalPages(); // canonical URLs + updated_at, nothing else
	const urls = pages
		.map((p) => `<url><loc>${site}${p.path}</loc><lastmod>${p.lastmod}</lastmod></url>`)
		.join('');
	return new Response(
		`<?xml version="1.0" encoding="UTF-8"?>` +
			`<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls}</urlset>`
	);
};

Include only canonical, indexable pages — no redirect shims, no auth-gated routes, no noindex pages. A sitemap entry is an assertion that the URL is worth indexing; polluting it with URLs that redirect or refuse indexing erodes the crawler's trust in the file. Skip <priority> and <changefreq>; major engines ignore them. One file holds 50,000 URLs / 50 MB uncompressed; past that you split into per-section files under a sitemap index, so structure the emitter by section from the start even while a single file suffices.

Per-page metadata: what the engine ranks and displays

The <title> and <meta name="description"> are the strongest per-page levers you control directly. The title is a genuine ranking input and the headline of your search result; the description rarely affects rank but is your ad copy on the results page. Both should be generated per-entity from real data — "CDW 10-K FY2026 — Business Description | Symbology", with a description drawn from the section's actual summary — never a site-wide constant. A catalog site that templates these well gets thousands of long-tail landing pages for free.

Controlling indexing is the other half. <meta name="robots" content="noindex"> removes a page from the index while leaving it usable. Reach for it on pages that would compete with your canonical tree: hash-addressed permalinks whose content duplicates a readable page, filtered/sorted views, thin generated pages. The bias should be to launch ambiguous page classes as noindex and open them up later — flipping the switch is instant, while evicting thousands of thin indexed URLs takes months.

Things That Go Wrong

Blocking in robots.txt to de-index a page. robots.txt stops crawling, not indexing — a disallowed URL with inbound links still gets indexed as a bare URL with no snippet. Worse, because the crawler can't fetch the page, it never sees a noindex tag you put there. To de-index: allow the crawl, serve noindex.

Temporary redirects on permanent moves. A 302 during a URL migration strands link equity on the dead scheme and keeps old URLs in the index indefinitely. Migrations use 301/308.

The canonical/link-graph contradiction. Pages declare canonical URL A while every internal link points at alias B. The engine resolves the conflict unpredictably. Canonicals, sitemap entries, and internal links must agree — they are three statements of the same fact.

Alias explosion without canonicals. Prefix-matched IDs, case-insensitive slugs, and tolerated query params each mint unlimited URLs per page. Harmless for users, but the crawl budget spreads across the aliases and the engine picks a "real" one for you — possibly the ugly variant.

Redirect chains. Scheme migration two redirects through scheme migration one. Each hop costs crawl budget and engines cap the hops they'll follow (Google: ~10). Point old shims directly at the current canonical when a second migration lands.

References