How to specify crawl time for specific URLs from a list of URLs

[basedygt]

i know we can use -ct argument to specify maximum crawl time but this options sets the crawl time for entire list of URLs.

I want to know how to set -ct for each URL instead of all URLs since i can't find any built in options inside the tool for it nor any workaround like below command worked:

cat example.com/subs.txt | \
  while read subdomain; do \
    ./katana -u "$subdomain" -ct 10m -o "$domain"/katana/"$subdomain".txt -fs $subdomain; \
  done

The reason why this is a handy feature as many websites root domain has endless contents and the scraper fails to scrape it's subdomain automatically due to this issue.

[ehsandeep]

@basedygt I guess the above workaround is way to go, as -ct is supported for whole execution and not based on input url.

[dogancanbakir]

This discussion closed automatically due to inactivity. Feel free to reopen or start new if still relevant.