large body of text, multiple files, inconsistent format. llms seem to be hit or miss when it comes to chunking. is there a application that I don't know about that can make it happen? the text is academic medical articles with tons of content. I want to chunk it for embedding purposes
