Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • Sabata@ani.social
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    19 hours ago

    Running decencored Qwen3.6-27b and a 9b Gemma for RAG and scrapes on Ollama with a mostly vibe coded discord bot. Just got it to run tools and scrape and post news on a schedule. The first model I can run locally that’s smart enough to be useful. May give Jan a try for the back end after reading that other guys rant.

    Mostly use it for stupid questions I could have googled and to brag to friends.