Because this needs to be a regular thing, come forward warriors and announce the feats you have achieved! For the glory of the Empire!
Because this needs to be a regular thing, come forward warriors and announce the feats you have achieved! For the glory of the Empire!
Set a local large language model loose on transcripts of some work meetings to generate summaries and lists of action points. For the glory of the empire!
That’s actually pretty cool. How does one go about doing that? Also, I feel like I should follow this with the obligatory, GLORY TO YOU AND YOUR HOUSE!
It’s a bit technical, unfortunately, at least the way that I do it. I installed the Whisper and wrote a Python script that uses it to create transcripts of audio recordings, storing them in JSON files. Then when that’s done I fire up KoboldCPP with the Command-R model, with a context set large enough to hold roughly two hours of transcript - I rarely have anything longer than that to process. I’ve got an RTX 4090 with plenty of RAM and VRAM so I can handle a fairly large model. Then I run a second Python script that uses KoboldCPP’s API to process the transcripts in various ways - getting a paragraph-long summary, getting a list of action items, and so forth.
I wouldn’t be surprised if there aren’t better tools out there now specifically for this kind of task, this is just what I cobbled together over the years as these things came out. So might be worth looking around for something like that.
For the glory of the Empire, of course.
Thanks for the breakdown. That’s really cool. QaPla!