Fetches pages with collect_content() and passes their original HTML to piedomains
for extraction, tokenization, calibration and classification. The Python package must
be version 0.15.0 or newer. This uses its text classifier, not its image or fusion models.
pie_cat(domains = NULL, pages = NULL, threshold = 0.1, model = NULL, ...)Character vector of domains or URLs. Supply either domains or pages.
Data frame from collect_content(), including its retained html column, or
previous piedomains results.
Previously extracted text alone is insufficient to reproduce model preprocessing.
Failed fetches are retained. Failed inference can be retried on the same pages.
Finite probability in [0,1]. Include every category at or above this
value from the full model distribution, always retaining the predicted category.
Optional Hugging Face model ID or local checkpoint directory. Overrides
PIEDOMAINS_TEXT_MODEL for this call. The pinned default model is reused within a session.
Passed to collect_content() when pages is absent.
A tibble following classification_results. Source-specific data are
retained in details; all source labels are retained in labels.
A current fetch does not guarantee a correct current label. The shipped checkpoint
has known retail/drugs confusion; see its model card. Archive and cached content retain
their collection dates. details preserves the page and its fetch_* collection outcomes
separately from
the terminal classification status, stage, error_code and retryable columns.
Install reticulate and declare Python requirements before initializing Python:
reticulate::py_require("piedomains==0.15.0", python_version = ">=3.12").
Set RETICULATE_PYTHON to select an existing Python environment.
Model weights download from Hugging Face on first use. The Python package pins its
default checkpoint; PIEDOMAINS_TEXT_MODEL can select another checkpoint or local path.
Results report the resolved model, revision when available, Python package version,
extractor, cleaning mode and calibration temperature.
if (FALSE) { # \dontrun{
pages <- collect_content(c("cnn.com", "wikipedia.org"))
pie_cat(pages = pages)
} # }