Create llms.txt file for Mirascope documentation

[betterthanever2]

Description

As Mirascope documentation grows, it becomes harder to navigate and keep up to. There is this thing called llms.txt. They suggest

A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.

Here's the relevant part of the FastHTML documentation that explains it a little more. In short, it's a page in Markdown format that contains the entire documentation and that you can feed to an AI agent, and then ask it (them?) questions about it.

Would be really nice to have.

[willbakst]

I'll look into this!

[willbakst]

Circling back here as I continue to research. One thing I need to figure out is how to do this with mkdocs without impacting the site (e.g. Google really does not like duplicate content, so I need to ensure I do things like remove the .md urls from the sitemap etc.).

Generally, I think including something here to make it easier to port our docs into an LLM is worth doing, so I'll continue to think through this and provide updates as I have them.