User-agent: * Disallow: / # Certain social media sites are whitelisted to allow crawlers to access page markup when links to /images are shared. User-agent: facebookexternalhit Allow: /imglib User-agent: FacebookBot Allow: /imglib