Tutorials
Scrape contact details from a website via API
Two lookups read public web pages for you. The contacts scraper returns the email addresses, phone numbers and social profile links a site publishes, and the website scraper returns one page as clean text with its links. This tutorial calls both, shows what the responses look like, and spells out what happens when a site refuses, times out or simply has nothing to find.
Two endpoints, two jobs
| Property | Contacts scraper | Website scraper |
|---|---|---|
| Endpoint | POST /v1/website-contacts-scraper | POST /v1/website-scraper |
| Input | domain or url, optional max_pages (1 to 10, default 5), optional two-letter country | url, optional format of markdown, text or html |
| Returns | Emails, phones in E.164, social profiles, pages scanned | Title, description, headings, text, internal and external links, word count |
| Price | 10 credits, only when contacts are found | 10 credits, only for a successful scrape |
The two scraping lookups side by side
The contacts scraper is a small crawl. It starts at the home page, stays on the same site, and only follows pages that look like contact pages: contact, about, team and similar. It never reads more than max_pages pages.
Call the contacts scraper
curl -X POST https://YOUR_API_HOST/v1/website-contacts-scraper \
-H "Authorization: Bearer $LOOKVALID_API_KEY" \
-H "Content-Type: application/json" \
-d '{"domain":"python.org","max_pages":5}'
Request: find the public contact details of a domain
This is a captured response from our own test run, trimmed of envelope fields. Each email and phone also carries a source_url naming the page it came from; this site published none, so those lists are empty.
{
"found": true,
"data": {
"emails": [],
"phones": [],
"social_profiles": [
{ "platform": "linkedin", "handle": "company/python-software-foundation",
"url": "https://www.linkedin.com/company/python-software-foundation" },
{ "platform": "x", "handle": "ThePSF", "url": "https://x.com/ThePSF" }
],
"pages_scanned": [
"https://www.python.org/",
"https://www.python.org/about/",
"https://www.python.org/about/apps/",
"https://www.python.org/about/quotes/",
"https://www.python.org/about/gettingstarted/"
],
"scanned_at": "2026-10-18T02:06:05.039Z"
},
"sources": ["https://www.python.org"]
}
Captured response
Without an API key the same lookup is available as a free tool, limited to 3 pages per call.
Call the website scraper
curl -X POST https://YOUR_API_HOST/v1/website-scraper \
-H "Authorization: Bearer $LOOKVALID_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","format":"markdown"}'
Request: read one page as markdown
{
"found": true,
"data": {
"final_url": "https://example.com/",
"status": 200,
"title": "Example Domain",
"description": null,
"canonical": null,
"language": "en",
"headings": [],
"format": "markdown",
"content": "This domain is for use in documentation examples without needing permission. ...",
"content_truncated": false,
"links": { "internal": [], "external": [] },
"images": 0,
"word_count": 25,
"fetched_at": "2026-10-18T02:06:04.110Z"
},
"sources": ["https://example.com"]
}
Captured response, with the page text shortened
status is the HTTP status of the page itself and final_url is where redirects ended. Text longer than a fixed cap is cut, and content_truncated tells you when that happened.
When a site blocks you or has nothing
A page that cannot be read is not an error in the usual sense, and none of these cases cost credits. Both endpoints fetch through one guarded path that follows at most five redirects, caps the body size, and refuses addresses on private networks. The outcomes below are what the code returns.
| Situation | Response | Charged |
|---|---|---|
| The start page answers 403, 404 or another non-success status | 200 with found: false and the status in reason | No |
| The page is not HTML, the name does not resolve, or redirects run out | 200 with found: false | No |
| The pages were read but hold no emails, phones or profiles | 200 with found: false and the page count in reason | No |
| The site answers 5xx, 408, 425 or 429, or is too slow | 503 provider_unavailable, retryable | No |
| The URL points at a private or reserved address | 400 invalid_input | No |
What each outcome looks like
The split between the first and fourth rows is deliberate. A 4xx page is a stable answer, so it is cached like any result. A 429 or 5xx is usually temporary, so reporting it as a miss would hide a site that is up again a minute later. HTTP defines 429 as a signal that the sender made too many requests, and a server may add a Retry-After header saying how long to wait 2. Treat our 503 the same way: wait, then retry.
On robots.txt: the crawl does not use the file as a gate. It reads the paths and sitemaps the file lists to discover contact pages. The Robots Exclusion Protocol itself states that its rules are not a form of access authorization 1. If you want to see what a site publishes there, the site files checker shows it.
const response = await fetch('https://YOUR_API_HOST/v1/website-contacts-scraper', {
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.LOOKVALID_API_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({ domain: 'python.org', max_pages: 5 }),
});
const body = await response.json();
if (response.status === 503) {
// The site was slow or answered with a temporary status. Nothing was charged.
console.log('retry later with a pause');
} else if (response.status === 400) {
console.log('bad input', body.error.code, body.error.fix);
} else if (!body.found) {
console.log('nothing to read:', body.reason); // free, and cached
} else {
console.log(body.data.emails, body.data.phones, body.data.social_profiles);
}
Branch on every outcome
What it costs and how to keep it low
- Success billing: a found result costs the credits in the table above, and every miss, refusal and timeout is free.
- Repeats are free. Results are cached per account for 7 days, so asking for the same site with the same
max_pagesandcountryagain costs nothing. - The cache key ignores
www.and the scheme, sopython.organdhttps://www.python.orgcount as one site. - Use
max_pages: 3for a quick pass and raise it only for sites whose contact details sit deeper.
Run the contacts call above against a domain you care about, then see the contacts scraper page for the full field list, or the quickstart to create a key.
Sources
- RFC 9309: Robots Exclusion Protocol, read 2026-10-19
- RFC 6585: Additional HTTP Status Codes (429 Too Many Requests), read 2026-10-19