Why catalog image links fail

A product feed can pass basic spreadsheet checks while hundreds of its image links are unusable. A URL may point to a deleted file, a redirected landing page, an expired signed asset, or a server that refuses automated requests. The failure is easy to miss because product titles, prices, and stock values remain intact. Customers then encounter blank thumbnails, while marketplace importers may reject listings or silently substitute placeholders. A reliable image audit checks more than whether a cell contains something resembling a URL. It evaluates the network response, confirms that the returned resource is an image, records uncertain results, and preserves enough context for a human to fix the correct product. This tutorial develops a practical Python workflow for checking catalog images without downloading every full-resolution asset.

Start with a useful input file

Export the catalog as UTF-8 CSV with a stable product identifier, image URL, and ideally an image role such as primary or gallery. Keep SKU values as strings because leading zeros may be significant. A simple input might contain sku, image_url, and image_role. If one product has several images, store one image per row or expand separate image columns before auditing. Do not overwrite the source file: preserve a dated copy and write results to a new location. For broader catalog cleanup, the principles behind product data cleansing are useful because image validity is only one part of feed quality. Decide whether URLs are public, authenticated, or time-limited before testing. Those distinctions determine what a failed request actually means.

Understand what an HTTP response proves

A successful HTTP status does not guarantee a usable product image. Some sites return a branded error page with status 200, while others redirect missing images to a generic placeholder. A 404 usually means the resource is missing at that path, and a 410 explicitly indicates that it is gone. A 403 may reflect permissions, anti-bot protection, or a private asset rather than a broken public image. A 429 indicates rate limiting; retrying aggressively makes the situation worse. Server-side 5xx errors can be temporary. The content type matters too: text/html is not the expected response for a JPEG or WebP file. Treat network errors and ambiguous statuses as separate categories rather than marking everything broken.

Choose a safe request strategy

HTTP HEAD requests ask for headers without a response body and can be efficient for large catalogs. However, some servers do not implement HEAD correctly, even when GET works. A practical checker tries HEAD first, then uses a small streamed GET request when HEAD returns an unsupported or suspicious result. The streamed request should close promptly instead of downloading the whole image. Use reasonable connection and read timeouts so one slow host cannot freeze the batch. Follow redirects, but capture the final URL because a redirect to a home page or login screen may indicate an invalid asset. Identify your checker clearly, respect website access rules, and avoid running large audits against servers you do not control without permission.

Install the Python dependencies

The example below uses requests for HTTP and Python’s built-in csv, pathlib, and time modules. Install requests in a virtual environment with python -m pip install requests. The script expects a file named product_images.csv in the current working directory and writes image_audit.csv. This keeps the example portable and avoids introducing a large data-processing dependency. If you need to normalize multiple supplier files first, see the related guide to combining CSV files with different columns. Run the audit against a small sample before expanding to the entire feed. That first pass reveals unusual authentication, redirect, or rate-limit behavior.

Build a reusable image checker

The following code distinguishes confirmed failures from uncertain responses. It accepts only HTTP or HTTPS URLs, makes a bounded request, checks whether the final content type starts with image/, and records the final destination. A HEAD response with a suspicious status or content type is rechecked using streamed GET. The code does not read the image body, which limits bandwidth. It is deliberately conservative: an authentication or server failure remains a review item rather than a definite broken link.

Python example

import csv
import time
from pathlib import Path
from urllib.parse import urlparse
import requests

INPUT = Path("product_images.csv")
OUTPUT = Path("image_audit.csv")
TIMEOUT = (5, 12)

def inspect(url, session):
    if urlparse(url).scheme not in ("http", "https"):
        return "invalid_url", "", "", ""
    try:
        response = session.head(url, allow_redirects=True, timeout=TIMEOUT)
        if response.status_code in (403, 405, 429) or (
            response.status_code == 200 and
            not response.headers.get("Content-Type", "").lower().startswith("image/")
        ):
            response.close()
            response = session.get(
                url, allow_redirects=True, timeout=TIMEOUT, stream=True
            )
        with response:
            status = response.status_code
            kind = response.headers.get("Content-Type", "").split(";")[0].lower()
            final = response.url
            if status in (404, 410):
                result = "missing"
            elif status == 429:
                result = "rate_limited"
            elif status in (401, 403):
                result = "restricted"
            elif status >= 500:
                result = "server_error"
            elif status == 200 and kind.startswith("image/"):
                result = "reachable_image"
            else:
                result = "review"
            return result, str(status), kind, final
    except requests.RequestException:
        return "network_error", "", "", ""

with requests.Session() as session, INPUT.open(
    newline="", encoding="utf-8-sig"
) as source, OUTPUT.open("w", newline="", encoding="utf-8") as target:
    reader = csv.DictReader(source)
    fields = list(reader.fieldnames or []) + [
        "audit_result", "http_status", "content_type", "final_url"
    ]
    writer = csv.DictWriter(target, fieldnames=fields)
    writer.writeheader()
    for row in reader:
        result = inspect(row.get("image_url", "").strip(), session)
        row.update(zip(fields[-4:], result))
        writer.writerow(row)
        time.sleep(0.5)

Interpret results before changing data

The output deliberately calls successful responses reachable_image rather than valid_image. A reachable image can still be a tiny tracking pixel, an irrelevant placeholder, or the wrong product photograph. Review missing results first, because 404 and 410 are stronger signals of a genuinely broken resource. Investigate invalid_url records for malformed schemes, accidental spaces, or missing protocols. Restricted and rate_limited results need a different response: verify permissions, use an approved API or authenticated export, or schedule a slower retry. A network_error may reflect DNS, TLS, connectivity, or timeout problems. Never replace an image solely because a single automated request failed.

Avoid false positives from anti-bot controls

Some content delivery networks serve images normally to browsers but reject unfamiliar automated clients. Others use short-lived tokens embedded in URLs. When a script reports 403, compare the same link in an ordinary browser and check whether access depends on cookies, a referrer, or authorization. Do not attempt to bypass access restrictions. Instead, request an authorized feed or use the supplier’s official asset API. If a URL expires after an hour, checking yesterday’s exported link says little about the current catalog. Record the export timestamp and the expected validity period. The goal is to assess customer-facing assets accurately, not to manufacture a green audit report by defeating security controls.

Control traffic and retry carefully

Even a modest catalog can contain tens of thousands of images. Sending requests too quickly may overload a small image server or trigger rate limits. Start with sequential requests, a delay between checks, and a small sample. For production use, add a bounded retry policy for transient timeouts, 429, and 5xx responses, using exponential backoff and honoring Retry-After when supplied. Avoid retrying 404 or 410 automatically. Group URLs by host so that one supplier’s limits do not affect every other source. The related guide on handling HTTP 429 errors in Python explains why respectful pacing is more reliable than repeated immediate requests.

Check dimensions and visual relevance separately

HTTP headers cannot prove that an image meets marketplace specifications. After identifying reachable resources, perform a second, selective audit for dimensions, file size, format, transparency, and image content. A 120-by-120 thumbnail may technically load but look poor in a zoomable product gallery. A file labeled image/jpeg might also contain an unexpected format or corrupt bytes. For files you are authorized to download, Pillow can inspect decoded dimensions and catch damaged files. Visual relevance requires another layer: compare a sample of photographs against SKUs, variants, and color attributes. Automated image similarity can assist, but human review remains important for subtle mismatches.

Detect placeholders and duplicate images

A supplier may return the same “image unavailable” graphic for many different missing products, with HTTP 200 and a valid image content type. To detect this, collect hashes of authorized downloaded images and count repeated hashes across unrelated SKUs. Exact hashes identify byte-for-byte duplicates, while perceptual hashes can flag resized or recompressed versions. Neither method proves a problem on its own: multiple variants may legitimately share a packaging photograph. Inspect suspicious clusters and maintain an allowlist for intentional shared assets. This is especially important when the feed combines different suppliers or marketplaces, where placeholder graphics and naming conventions vary.

Create a repair queue instead of deleting rows

A useful audit ends with a prioritized correction list. Include SKU, original URL, final URL, failure category, supplier, and last-checked date. Assign confirmed missing primary images the highest priority because they can block product publication. Put restricted, rate-limited, and transient errors into a separate verification queue. Record the replacement URL, who approved it, and whether the asset matches the product. Keep the old URL for traceability rather than silently replacing values. Teams maintaining large catalogs can connect this workflow to product data enrichment, where missing assets are resolved alongside incomplete product attributes.

Validate replacements before publishing

After repairing links, rerun the checker against changed records and compare results with the previous audit. A corrected URL should return an image resource and, where relevant, pass dimension and visual checks. Check whether redirects are stable and whether a content delivery network is still serving a cached error. Verify a small sample in the actual storefront or marketplace preview, not just in Python. Document cases where a feed requires signed URLs or a separate image upload process. Those assets may be perfectly valid in the intended integration but unsuitable as permanent public links. The final validation must match the destination system’s rules.

Conclusion

Broken image URLs are a catalog-quality problem, not merely a networking problem. A dependable audit distinguishes missing resources from blocked requests, temporary outages, redirects, and valid but incorrect images. Start with a conservative HTTP check, preserve source records, pace requests responsibly, and use separate visual checks for dimensions and product relevance. Most importantly, turn the findings into a documented repair queue rather than automatically removing uncertain images. That approach produces cleaner product feeds while avoiding unnecessary changes to assets that were never actually broken.