Follow

Q&A: Does Shape Defense allow Good Bots (like Google) to crawl pages?

Question

Does Shape Defense allow Good Bots (like Google) to crawl pages?

 

Environment

  • Silverline Shape Defense
  • Bot defense
  • Bot protection
  • Anti-bot

 

Answer

Does Shape Defense allow Good Bots (like Google) to crawl pages?

  • Silverline Shape Defense does not seek to protect an entire website against automated traffic (good or bad bots).
  • Silverline Shape Defense seeks to protect the high-value endpoints, like the “login” or “password reset” page – as those endpoints should not be processing ‘Good Bots.’
  • In that way, good bots will be allowed to crawl pages, for indexing, etc.
  • If you want to prevent web scraping by malicious bots, see Q&A: How does Silverline Web Scraping Protection Work?

Which endpoints should you be defining for protection?  

  • Focus on your high value endpoints (login/password challenges, forms, inventory and pricing) that are also being protected from WAF and DDoS attacks
  • If unsure, you can contact the SOC.

 

Related Content

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request