fetch_error_codes.RdThe closed set of `error_code` values that [collect_content()] and [pie_cat()] can return, with the pipeline stage each belongs to and whether it is worth retrying.
fetch_error_codes()A tibble with columns `code`, `stage`, `retryable` and `description`.
These strings are shared verbatim with the Python package `piedomains`, so results from the two can be aggregated together.
[fetch_report()] to summarise an actual run, [source_vintage()] for the provenance of the static label sources.
fetch_error_codes()
#> # A tibble: 18 × 4
#> code stage retryable description
#> <chr> <chr> <lgl> <chr>
#> 1 invalid_domain validate FALSE Not a fetchable http(s) host.
#> 2 robots_blocked validate FALSE Disallowed by the host's robots.txt.
#> 3 private_address validate FALSE Host resolves to a non-global addre…
#> 4 dns_error fetch TRUE Host did not resolve.
#> 5 connection_error fetch TRUE Connection refused or reset.
#> 6 timeout fetch TRUE Request exceeded the timeout.
#> 7 http_error fetch TRUE Non-2xx status.
#> 8 content_type_rejected fetch FALSE Response was not HTML.
#> 9 content_too_large fetch FALSE Body exceeded the byte cap.
#> 10 bot_blocked fetch FALSE An anti-bot interstitial was served.
#> 11 no_archive_snapshot fetch FALSE No archived capture for the request…
#> 12 too_many_redirects fetch FALSE Redirect chain exceeded max_redirec…
#> 13 empty_text process FALSE No text could be extracted.
#> 14 thin_content process FALSE Below the token floor for an honest…
#> 15 service_unavailable infer TRUE Classification service asleep or do…
#> 16 rate_limited infer TRUE Classification service rate-limited…
#> 17 service_error infer TRUE Classification service returned an …
#> 18 unknown NA FALSE Unclassified failure.
# which failures are worth trying again?
subset(fetch_error_codes(), retryable)
#> # A tibble: 7 × 4
#> code stage retryable description
#> <chr> <chr> <lgl> <chr>
#> 1 dns_error fetch TRUE Host did not resolve.
#> 2 connection_error fetch TRUE Connection refused or reset.
#> 3 timeout fetch TRUE Request exceeded the timeout.
#> 4 http_error fetch TRUE Non-2xx status.
#> 5 service_unavailable infer TRUE Classification service asleep or down.
#> 6 rate_limited infer TRUE Classification service rate-limited us.
#> 7 service_error infer TRUE Classification service returned an error.