Leetcode #1242: Web Crawler Multithreaded
In this guide, we solve Leetcode #1242 Web Crawler Multithreaded in Python and focus on the core idea that makes the solution efficient.
You will see the intuition, the step-by-step method, and a clean Python implementation you can use in interviews.
Problem Statement
Given a URL startUrl and an interface HtmlParser, implement a Multi-threaded web crawler to crawl all links that are under the same hostname as startUrl. Return all URLs obtained by your web crawler in any order.
Quick Facts
- Difficulty: Medium
- Premium: Yes
- Tags: Depth-First Search, Breadth-First Search, Concurrency
Intuition
We need to explore a structure deeply before backing up, which suits DFS.
DFS keeps local context on the call stack and is easy to implement recursively.
Approach
Define a recursive DFS that carries the necessary state.
Combine child results as the recursion unwinds.
Steps:
- Define a recursive DFS with state.
- Visit children and combine results.
- Return the final aggregation.
Example
interface HtmlParser {
// Return a list of all urls from a webpage of given url.
// This is a blocking call, that means it will do HTTP request and return when this request is finished.
public List<String> getUrls(String url);
}
Python Solution
Complexity
The time complexity is O(V+E). The space complexity is O(V).
Edge Cases and Pitfalls
Watch for boundary values, empty inputs, and duplicate values where applicable. If the problem involves ordering or constraints, confirm the invariant is preserved at every step.
Summary
This Python solution focuses on the essential structure of the problem and keeps the implementation interview-friendly while meeting the constraints.