Think of it like a sign on your gate saying who’s welcome where. Polite visitors obey it.
In plain terms
A small file on your website that tells crawlers which parts they may visit. Well-behaved crawlers follow it; it isn’t a lock.
Why it matters for a small business
One wrong line in robots.txt can hide your whole site from search engines or AI crawlers. It takes a minute to look: open yourbusiness.com/robots.txt in a browser.