added llms.txt, llms-full.txt, copy markdown button

[Benanna2019]

Add LLMs Support✨

I have found that having llms.txt and llms-full.txt support and the ability to copy the markdown for a given page quickly allows for quickly working with LLMs to create/debug architect applications

To maintain a high standard of quality in our releases, before merging every pull request we ask that you've completed the following:

Here are the changes:

• added copy markdown button - just quick copy of docs page if someone is trying to debug or get something going quickly

• added llms.txt route - high level overview of architect for llms

• added llms-full.txt route - complete site breakdown into markdown as that can be referenced by llms as documentation for architect.

Checks:

• Forked the repo and created your branch from main
• Made sure tests pass (run npm it from the repo root)
• Updated relevant documentation internal to this repo (e.g. readme.md, help docs, inline docs & comments, etc.)
• [ ] Linked to any related issues, PRs, etc. below that may relate to, consume, or necessitate these changes

[CLAassistant]

All committers have signed the CLA.

[filmaj]

Thanks for the PR, this is interesting. I had to read up on https://llmstxt.org to understand the purpose of this.

I have an immediate concern and that is reading the file system on every HTTP request to llms-full.txt. I don't think we get crazy traffic but still. At a minimum, I'd want to cache the read and compiled text into a closure variable at the top of the handler, so that the doc files are only read on every cold boot of a Lambda rather than on every request.

My other concern is the design of the 'copy markdown' button 😬 I don't have any specific suggestion or recommendation on how to fix that, sorry 😞 It feels in the way for me. I wonder if there's a more subtle treatment we could do 🤔

[brianleroux]

Agree about cache; should be easy to add a cache-control header in the response for those new routes. 24 hours would be fine. We don't change frequently! Could we change the copy button to have no text? I think users recognize that symbol

[Benanna2019]

Agree about cache; should be easy to add a cache-control header in the response for those new routes. 24 hours would be fine. We don't change frequently! Could we change the copy button to have no text? I think users recognize that symbol

These are great suggestions. I think this is the benefit of contributing for me is there are a lot of things I'll learn by working with you all that I think would help me just generally be better at the craft.

I'll update this with these changes. Thanks!

[filmaj]

As for the CI failures, if you rebase with latest main, those should be resolved.

[Benanna2019]

@filmaj and @brianleroux what needs to happen here for this to get merged? I added the requested updates just didn't want to leave it hanging and/or if I missed something wanted to check