Automatic Generation?

I also had the same confusion. But there is a configuration page here /admin/content/llms-txt where you can use various tokens to create the structure you need. Also you can create sections here /admin/content/llms-txt/sections that will be automatically added to llms.txt file. For example you can create a section for Main Menu and use the token [llms_txt_markdown_menu:main] inside (you might need the token_filter module). But you can also add the token directly on configuration page. You can use any token, so this makes it generic.