TrawlerPolicyData

Posted on by Darth Autocrat

GoogleApi.ContentWarehouse.V1.Model.TrawlerPolicyData

Table of Contents ▼

Jump to a specific part of the page:

Description

Trawler can add a policy label to a FetchReply. The two main cases are: - "spam" label added for specific spammer IPs listed in trawler_site_info, which most crawls auto-reject. - "roboted:useragent" (e.g. "roboted:googlebot") if InfoOnlyUserAgents field is set in FetchParams

Attributes List

This module has the following attributes (case-insensitive ascending order):

View Attributes

Attributes

  1. ExtraData (type: integer(), default: nil)
    - in roboted case, the RobotsInfo
  2. Label (type: String.t, default: nil)
    - "spam" or "roboted:googlebot"

Type

@type t() :: %GoogleApi.ContentWarehouse.V1.Model.TrawlerPolicyData{
ExtraData: integer() | nil,
Label: String.t() | nil
}

Function

@spec decode(struct(), keyword()) :: struct()

Data sourced from HexDocs : GoogleApi.ContentWarehouse.V1.Model.TrawlerPolicyData