Add llms.txt to your website
Table of Contents
What is llms.txt? #
The llms.txt file helps artificial intelligence (AI) systems better understand your website by providing a clear outline of your site’s structure and content, typically using Markdown syntax. It lists important page URLs and describes their content, making it easier for AI to interpret your site.
Having an accurate llms.txt is important—without it, AI tools may struggle to navigate your website or recognize the relationships between different pages. This can decrease your site’s visibility in AI-driven search results.
While llms.txt is similar to robots.txt in that both assist web crawlers, they serve distinct roles. robots.txt controls crawler access, whereas llms.txt focuses on describing your site’s content for AI understanding.
How to Create & Generate Content for llms.txt #
The first step is to create a new file named llms.txt in the root directory of your website.
To generate content for llms.txt, simply head over to https://llmstxt.firecrawl.dev/.
Click on Enter Firecrawl API key to enter your Firecrawl API key.
If you don’t have one, go to the https://firecrawl.dev website, create a free account, and find your API key.
Once you get the API key, paste it into the llms.txt generator website and hit save.
Here, you can enter your website URL, and it will automatically generate the content for you. You can also edit the content as per your requirements.
Now, just enter the URL of the website you want to generate the llms.txt for, and click on Generate.
This will automatically generate the content for your llms.txt file, so you can simply copy and paste it into your llms.txt file.
After generating the content, simply copy it and paste it into your llms.txt file.
Once done, deploy your website and you will find your llms.txt file at https://yourwebsite.com/llms.txt.
Here’s an example: https://hextaui.com/llms.txt
Adding llms.txt to your website is a straightforward process that can significantly enhance how AI systems understand and interact with your content. By following the steps outlined above, you can ensure that your site is better equipped for AI-driven applications and search results.
For more information on llms.txt, you can refer to the following resources: