Generate llms.txt files from Screaming Frog website crawls

• Generate llms.txt files for SEO optimization
• Automate content discovery for language models
• Filter and structure website data for better AI training

User submits a form with website details and Screaming Frog export Data is extracted from the provided CSV file Key fields are set for each URL, including title and description URLs are filtered based on status, indexability, and content type An optional text classifier can refine the selection of URLs The llms.txt file is generated for download or upload

1. 1
   Import the workflow into n8n
2. 2
   Fill in the form with the website name and description
3. 3
   Upload the internal_html.csv file from Screaming Frog
4. 4
   Run the workflow to generate the llms.txt file