Has anyone tried in organization to use self hosted llm models for agentic programming?
Im curious if it makes any sense. My organization spends fortune on tokens from US companies. I want to recommend something… I think that will be cheaper to use it on own machines instead…
There are a few of the chinese open models that are okay for coding, but in terms of functionality they’re extremely basic. You can make them work, but if people are used to the big corpo models it’s going to be hard to get them to switch to what is basically a chatbot, and the open-source tools to give them much needed QoL functionality are pretty rough right now.
For sure worth looking into self hosting but it’s going to take quite a bit of convincing to get people to shift over, I fear.