# To add a comment to the file, start the line with the # character. # User-Agent is used to target a particular web crawler. # Any rules declared below it will apply to that User-Agent. # To hide a file or folder from the User-Agent, type the word 'Disallow' followed by a semi-colon. Sitemap: https://www.fnc.co.uk/sitemap-fnc-co-uk.xml Sitemap: https://www.fncaustralia.com.au/sitemap-fncaustralia-com-au.xml User-Agent: * Disallow: /bin/ Disallow: /config/ Disallow: /umbraco/ Disallow: /views/ User-agent: Semrushbot-SA Disallow: / User-agent: Semrushbot-SI Disallow: / User-agent: SemrushBot Disallow: /