hakrawler
Auth/lab ref: Fast Go web crawler for discovering URLs, endpoints, and JavaScript files. For crawling web applications to build a URL inventory before fuzzing or during OSINT on web infrastructure.
v2026.11.13-21cd5684 installs+4 in 7 days39 StarsMITsynced 10 hours ago
Install
No NotHoly CLI yet? Install it first.
$ curl -fsSL https://notholy.com/install.sh | sh$ notholy skills install aeondave/hakrawlerWrites to ~/.notholy/skills after checking the signature and the SHA-256.
SignatureSigned, ed25519
Packagef87f9855…8f8ffd41
f87f9855fc18…41638f8ffd412 files · 2.6 KBScanLow1 finding
- Links to outside domains
Sourcegithub.com/AeonDave/malskill/tree/main/offensive-tools/recon/hakrawler@21cd568
On this page
Fast Go web crawler — discover URLs, JS files, forms, and endpoints.
Quick Start
go install github.com/hakluke/hakrawler@latest
# Crawl a domain
echo https://target.com | hakrawler
# Depth 3, include subdomains
echo https://target.com | hakrawler -d 3 -subs
# Output as JSON
echo https://target.com | hakrawler -json
# Pipe multiple domains
cat domains.txt | hakrawler -d 2Core Flags
| Flag | Purpose |
|---|---|
-d N |
Depth (default: 1) |
-subs |
Include subdomains |
-u |
Unique URLs only |
-insecure |
Skip TLS verification |
-t N |
Threads |
-timeout N |
Timeout per request (s) |
-H "K:V" |
Custom header |
-json |
JSON output |
-scope REGEX |
Limit to URL pattern |
-plain |
Print plain text (no color) |
-proxy <url> |
HTTP/SOCKS5 proxy |
-cookie <str> |
Cookie string |
-dr |
Disable following redirects |
-w N |
Wait N ms between requests |
Common Workflows
Build URL inventory for fuzzing:
echo https://target.com | hakrawler -d 3 -u | tee urls.txt
# Feed to ffuf
ffuf -w urls.txt:URL -u URL -mc 200Discover JS files:
echo https://target.com | hakrawler -d 2 | grep "\.js$"Combine with httpx for live check:
cat domains.txt | hakrawler | httpx -silent -mc 200Scope-limited crawl (stay in scope):
echo https://target.com | hakrawler -d 3 -scope ".*\.target\.com.*" -uExtract API endpoints:
echo https://target.com | hakrawler -d 3 -u | \
grep -E "(/api/|/v[0-9]+/|\.json|\.xml)" | sort -uForm action discovery:
echo https://target.com | hakrawler -json | \
jq -r 'select(.type=="form") | .source'Multi-target via live hosts:
subfinder -d target.com -silent | \
httpx -silent | \
hakrawler -d 2 -u | sort -u > all_urls.txtResources
| File | When to load |
|---|---|
references/crawl-tips.md |
Scope filtering, JS analysis, pipeline patterns, JS secret extraction |
Files in this version
references/13.7 KB
- crawl-tips.md3.7 KB
- SKILL.md2.3 KB
Permissions
| Permission | Declared | Found by the scan |
|---|---|---|
| Reads files | Yes | Yes |
| Writes files | Yes | Yes |
| Runs shell commands | Yes | Yes |
| Uses the network | Yes | Yes |
| Uses connectors | No | No |
| Drives a browser | Yes | Yes |
Scan findings
Reviewed on Nov 13, 2026
- InfoLink · Links to outside domains
A clean scan lowers the risk but does not prove a package is safe. Read what you install.
| Version | Published | Commit | SHA-256 | Signature | Permission change |
|---|---|---|---|---|---|
| v2026.11.13-21cd568 | Nov 13, 2026 | 21cd568 | f87f9855 | Signed | – |
More from AeonDave
All from this publisherMore in Web and bug bounty
See the categoryAsk anything your scope allows.
Create an account, add credits and ask the first question in the chat or the terminal.
- 1/3
Create the account
Individual or company. You can switch to a company later without losing balance or keys.
IndividualCompany - 2/3
Add credits
By card. Pay per token, no subscription.
$5.00minimum deposit - 3/3
Ask
In the chat, or install the CLI and run notholy login in your terminal.
$ notholy login