Fetcher -> FetchClient FetchReplyData is the metadata for a reply from a FetchRequest. For metadata + document body, FetchReply is further below. NOTE: FetchReplyData (and FetchReply) is the output interface from Multiverse. Teams outside Multiverse/Trawler should not create fake FetchReplies. Trawler: When adding new fields here, it is recommended that at least the following be rebuilt and pushed: – cron_fetcher_index mapreduces: so that UrlReplyIndex, etc. retain the new fields – tlookup, tlookup_server: want to be able to return the new fields – logviewer, fetchutil: annoying to get back ‘tag88:’ in results ————————– Next Tag: 125 ———————–
Author: Darth Emmett
TrawlerFetchReplyDataDeliveryReport
Depending on which Data Acquisition API the client uses, the fetched content can be delivered to the client in the RPC, via Goops, or copied to the client’s storage system (possibly after transformation). In the latter case, DeliveryReport will contain info about the delivery status, such as whether we have permission error, whether the destination storage is out of quota, etc).
TrawlerFetchReplyDataFetchStatsClientStateStats
Records stats about state changes on the client side if there’re any. For example, there’s a state of cache lookup when using private-cache client library.
TrawlerFetchReplyDataHTTPHeader
header : value
StorageGraphBfgAuthorityFeedbackMetadata
go/entity-authority NB: currently all facts associated with Authority Feedback are externally traceable to the requests from which they originate.
StorageGraphBfgLivegraphProvenanceMetadata
Metadata on source assertions that isn’t part of the user-visible Triple payload, and that doesn’t really represent data provenance, but that’s used to affect the way Livegraph and possibly other horizontal KG infra systems process the triple. Read: fields below really shouldn’t be part of the cross-system Triple proto at all. But because Triple is used both as an internal and an external KG API, we at least want to "hide" those fields that ought to be purely part of the internal source <-> LG contract. Next id: 6
StorageGraphBfgLmsPolicyMetadata
Metadata fields for LMS. See go/lms-online-restrictions for details.
StorageGraphBfgPublicInformationMetadata
StorageGraphBfgSpiiCertification
A fact about potentially sensitive personal info (http://what/SPII) can be "certified" iff it meets specific requirements. See go/kg-spii-certification for details.
StorageGraphBfgUmpPolicyMetadata
Metadata fields for UMP. A piece of data satisfies UMP policy if: (data access region is in the regions_allowed
list) AND (data access time >= availability_start_timestamp) AND (data access time < availability_end_timestamp)