Tutorials

Scrape contact details from a website via API

Two lookups read public web pages for you. The contacts scraper returns the email addresses, phone numbers and social profile links a site publishes, and the website scraper returns one page as clean text with its links. This tutorial calls both, shows what the responses look like, and spells out what happens when a site refuses, times out or simply has nothing to find.

By LookValid engineering. Published 2026-10-19.

Two endpoints, two jobs

The two scraping lookups side by side
PropertyContacts scraperWebsite scraper
EndpointPOST /v1/website-contacts-scraperPOST /v1/website-scraper
Inputdomain or url, optional max_pages (1 to 10, default 5), optional two-letter countryurl, optional format of markdown, text or html
ReturnsEmails, phones in E.164, social profiles, pages scannedTitle, description, headings, text, internal and external links, word count
Price10 credits, only when contacts are found10 credits, only for a successful scrape

The two scraping lookups side by side

The contacts scraper is a small crawl. It starts at the home page, stays on the same site, and only follows pages that look like contact pages: contact, about, team and similar. It never reads more than max_pages pages.

Call the contacts scraper

curl -X POST https://YOUR_API_HOST/v1/website-contacts-scraper \
  -H "Authorization: Bearer $LOOKVALID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain":"python.org","max_pages":5}'

Request: find the public contact details of a domain

This is a captured response from our own test run, trimmed of envelope fields. Each email and phone also carries a source_url naming the page it came from; this site published none, so those lists are empty.

{
  "found": true,
  "data": {
    "emails": [],
    "phones": [],
    "social_profiles": [
      { "platform": "linkedin", "handle": "company/python-software-foundation",
        "url": "https://www.linkedin.com/company/python-software-foundation" },
      { "platform": "x", "handle": "ThePSF", "url": "https://x.com/ThePSF" }
    ],
    "pages_scanned": [
      "https://www.python.org/",
      "https://www.python.org/about/",
      "https://www.python.org/about/apps/",
      "https://www.python.org/about/quotes/",
      "https://www.python.org/about/gettingstarted/"
    ],
    "scanned_at": "2026-10-18T02:06:05.039Z"
  },
  "sources": ["https://www.python.org"]
}

Captured response

Without an API key the same lookup is available as a free tool, limited to 3 pages per call.

Call the website scraper

curl -X POST https://YOUR_API_HOST/v1/website-scraper \
  -H "Authorization: Bearer $LOOKVALID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","format":"markdown"}'

Request: read one page as markdown

{
  "found": true,
  "data": {
    "final_url": "https://example.com/",
    "status": 200,
    "title": "Example Domain",
    "description": null,
    "canonical": null,
    "language": "en",
    "headings": [],
    "format": "markdown",
    "content": "This domain is for use in documentation examples without needing permission. ...",
    "content_truncated": false,
    "links": { "internal": [], "external": [] },
    "images": 0,
    "word_count": 25,
    "fetched_at": "2026-10-18T02:06:04.110Z"
  },
  "sources": ["https://example.com"]
}

Captured response, with the page text shortened

status is the HTTP status of the page itself and final_url is where redirects ended. Text longer than a fixed cap is cut, and content_truncated tells you when that happened.

When a site blocks you or has nothing

A page that cannot be read is not an error in the usual sense, and none of these cases cost credits. Both endpoints fetch through one guarded path that follows at most five redirects, caps the body size, and refuses addresses on private networks. The outcomes below are what the code returns.

What each outcome looks like
SituationResponseCharged
The start page answers 403, 404 or another non-success status200 with found: false and the status in reasonNo
The page is not HTML, the name does not resolve, or redirects run out200 with found: falseNo
The pages were read but hold no emails, phones or profiles200 with found: false and the page count in reasonNo
The site answers 5xx, 408, 425 or 429, or is too slow503 provider_unavailable, retryableNo
The URL points at a private or reserved address400 invalid_inputNo

What each outcome looks like

The split between the first and fourth rows is deliberate. A 4xx page is a stable answer, so it is cached like any result. A 429 or 5xx is usually temporary, so reporting it as a miss would hide a site that is up again a minute later. HTTP defines 429 as a signal that the sender made too many requests, and a server may add a Retry-After header saying how long to wait 2. Treat our 503 the same way: wait, then retry.

On robots.txt: the crawl does not use the file as a gate. It reads the paths and sitemaps the file lists to discover contact pages. The Robots Exclusion Protocol itself states that its rules are not a form of access authorization 1. If you want to see what a site publishes there, the site files checker shows it.

const response = await fetch('https://YOUR_API_HOST/v1/website-contacts-scraper', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.LOOKVALID_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ domain: 'python.org', max_pages: 5 }),
});
const body = await response.json();
​
if (response.status === 503) {
  // The site was slow or answered with a temporary status. Nothing was charged.
  console.log('retry later with a pause');
} else if (response.status === 400) {
  console.log('bad input', body.error.code, body.error.fix);
} else if (!body.found) {
  console.log('nothing to read:', body.reason); // free, and cached
} else {
  console.log(body.data.emails, body.data.phones, body.data.social_profiles);
}

Branch on every outcome

What it costs and how to keep it low

  • Success billing: a found result costs the credits in the table above, and every miss, refusal and timeout is free.
  • Repeats are free. Results are cached per account for 7 days, so asking for the same site with the same max_pages and country again costs nothing.
  • The cache key ignores www. and the scheme, so python.org and https://www.python.org count as one site.
  • Use max_pages: 3 for a quick pass and raise it only for sites whose contact details sit deeper.

Run the contacts call above against a domain you care about, then see the contacts scraper page for the full field list, or the quickstart to create a key.

Sources

  1. RFC 9309: Robots Exclusion Protocol, read 2026-10-19
  2. RFC 6585: Additional HTTP Status Codes (429 Too Many Requests), read 2026-10-19