Running AI models locally on a Mac for writing stories

[joptimus]

Hi guys,

I've been using perchance AI story generator for a while and usually it's great. Sometimes when I accidentally close my browser, all non saved progress is lost. And also sometimes, generation is slow or won't happen at all.

Therefore I'd like to run these models locally. Any tips for software or LLM models? I'm on a Mac.
Thanks!

 

[Ticklemang]

Ollama is going to be your best (only?) bet. It is going to be really slow and the models you’ll be able to run are going to be way shittier.

I would recommend just using openrouter (saves everything in cookies so you can’t do it in incognito). If you’re computer savvy then set up OpenWebUI and hook it up to openrouter and use that. I run an OpenWebUI container on my home server so I can easily access it. Everything is saved and it comes with account management out of the box too.