Scraping specific sneakers' bid price list in StockX:
- How to work:
- Go to https://stockx.com/buy/air-jordan-1-retro-high-og-defiant-couture.
- Wait to complete loading the confirmation page.
- Click the confirm button.
- Wait to complete loading the size list page.
- Retrieve the sizes and prices.
Set the project root path to GOPATH.
mkdir chromedp-example
cd chromedp-example
export GOPATH=$(pwd)
go get -d github.com/yoheimuta/chromedp-example # ignore a `no Go files` error.
cd src/github.com/yoheimuta/chromedp-exampleRun your chrome headless-shell.
docker pull chromedp/headless-shell
docker run -d -p 9222:9222 --rm --name headless-shell chromedp/headless-shellThere are 3 main.go which use different logic to demonstrate how to utilize the chromedp.
In fact, each code differs only repository injected in main.go.
# normal implementation.
go run cmd/stockx/main.go
# faster implementation. This uses the cookie to skip the confirmation page.
go run cmd/faststockx/main.go
# parallel support implementation. This is more performant.
## NOTE: This uses a local chrome app.
go run cmd/parallelstockx/main.gogo test -v -count 1 -timeout 240s -race ./...