Robots.txt
5-15 minIntermediate
Magento 2 provides a dynamic robots.txt generator.
Prerequisites
- Admin Panel
Easy Recommended
Admin Generator (Recommended)
Configuring the virtual file.
1
Configure
1
Content > Design > Configuration > Edit Global Store
2
Expand 'Search Engine Robots'
3
Here you can add custom instructions to the 'Edit custom instruction of robots.txt file' field
4
This is safer than a physical file
Verification Checklist
- Visit /robots.txt