Free phishing feed

Free phishing data files

Our published phishing findings and flagged brand lookalikes, in one complete file each hour, from 12 hours after we publish or flag them. Each record names the independent sources that listed the hostname, a name under it, or an address it resolved to.

Format: Gzip NDJSON, one record per line, with a JSON manifest and three plain hostname lists
Updates: Every hour, and within minutes of a correction
Delay: At least 12 hours after we publish or flag a finding
Findings: Published or flagged in the last 7 days
Licence: Free for non-commercial use, with credit. Terms and attribution

Files

File	Contents
`https://domainspoof.org/data/manifest.json`	Read this first. It names the current snapshot and hostname lists, with their SHA-256, size and record count, the delay, the counts by kind, the licence and each source’s credit.
`https://domainspoof.org/data/LICENSE.txt`	The data licence.
`https://domainspoof.org/data/eligible/<time>.ndjson.gz`	The current snapshot: every record, one per line, ordered by id. Its name is in the manifest.
`https://domainspoof.org/data/hostnames-resolving.txt`	Hostnames whose latest DNS check had an address: one per line, sorted, after a # header with the credit, licence, delay and generated time.
`https://domainspoof.org/data/hostnames-brands.txt`	Hostnames that name a brand we track (every lookalike does): one per line, sorted, after a # header with the credit, licence, delay and generated time.
`https://domainspoof.org/data/hostnames-all.txt`	Every hostname in the data files: one per line, sorted, after a # header with the credit, licence, delay and generated time.
`https://domainspoof.org/data/lists/<time>/hostnames-<list>.txt`	The same lists under their own names, as the manifest’s lists[] names them, with their SHA-256.

Read the files

Fetch the manifest.
Fetch the snapshot its snapshot.key names, and check its SHA-256.
Replace your copy with it. Each snapshot is complete.

curl -s https://domainspoof.org/data/manifest.json -o manifest.json
key=$(jq -r .snapshot.key manifest.json)
curl -s https://domainspoof.org/data/$key | gunzip | jq -r .hostname | head

A snapshot answers 404 once a newer manifest names another one. Read the manifest again and fetch the new one.

When findings come and go

The files update after each hourly publish and after each correction. A finding enters at least 12 hours after it qualifies for the files. If our record of a publish is incomplete, those 12 hours start again for whatever the file does not show yet. So the files start empty, and findings join from 12 hours on. A finding stays while it qualifies, up to 7 days after publication. A flagged lookalike enters at least 12 hours after a listing first flags it, and stays up to 7 days after its first flag. When a source, address or brand leaves a finding, or a correction withdraws it, the next snapshot shows the change. A finding that leaves and returns waits at least 12 hours again.

Record fields

Field	Meaning
`id`	The finding’s stable id, one per kind and hostname.
`revision`	The revision of our published record this one comes from. The files can drop a source, address or brand and keep the same revision.
`kind`	reported_domain, spamhaus_drop, phishing_infrastructure or listed_lookalike (a flagged registered lookalike of a brand’s domain).
`hostname` `registrable`	The certificate hostname, or the lookalike’s domain, and its registrable domain.
`sources[]`	Reported findings: each independent source with its status (listed, delisted, aged_out or unknown), the day that status began (since) and under, the listed parent domain, when there is one. Lookalikes: each reporting source that lists the lookalike, a name under it (host) or a domain it is under (under), with flagged_at, when we first saw that listing.
`sources[].listed_from` `observed_from` `listed_to`	The current listing period: its start by the source’s own date where it gives one, our first observation of it, and its end once closed.
`sources[].brief` `periods` `history[]`	Whether the period closed within two of the source’s poll intervals, how many periods the source has had for the host, and the last five, each with its close_reason.
`addresses[]`	Address findings: each listed address the hostname resolved to (list, ip). Lookalikes: each listed address that flags it (list, ip, flagged_at).
`lookalike_of[]` `found_by` `conditions`	Lookalikes: the brand and official domain it spells, and how (spelling); what found it registered (dns_sweep: our DNS sweeps; zone_file: the daily new registrations in registry zone files); for zone_file, where the zone-data conditions are. Lookalikes have no revision, first_published, state or evidence_through.
`brands[]`	The brand ids the hostname is attributed to, as on domainspoof.com/brands/<id>.
`first_observed`	Our first sighting, in a certificate or a list. A lookalike: the earliest first flag among its listings in the file.
`first_published`	When we first published the finding; null for one published before records began (baseline: true).
`evidence_through`	The day of the newest qualifying evidence, advanced in steps of at least 30 days.
`state` `state_since`	eligible, or not_resolving after two failed DNS checks, and since when.
`dns` `dns_checked_on`	Our latest DNS check, current rather than delayed: resolved (an address), no_address, not_found (the name does not exist) or unknown (an inconclusive check, or none yet), and its UTC day.

Dates are ISO 8601 UTC; source dates and evidence_through are the UTC midnight of their day. Records carry no certificate data, no source’s raw records and no URLs.

Feeds

For new findings as they enter, read the free phishing feed in JSON or RSS. Real-time findings are on domainspoof.com.

Licence

The files are free for non-commercial use with credit. Commercial redistribution needs our agreement. Read LICENSE.txt and the terms and each source’s credit line.