1·
6 hours agoYep, I think we have the winner. I’ve just installed it today and I’m actually responding to your post using handy right now. It’s a really great piece of software. Thank you to both of you that recommended this.
- 2 Posts
- 48 Comments
Joined 23 days ago
Cake day: May 14th, 2026
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
Huh. That’s a smart idea (AI federated instance). I imagine Lemmy is too small for it to be on Big AIs radar very much (just yet)
Bemused I guess? At least Anthropic made theirs funny
openwhispr
Second time today that has come up - I better go look. Cheers!
Thank you!
Thanks - that’s worth a gander
Thank you!
Thank you!
Awesome!
Aussie too - same issue with having to fake accent sometimes :)
Quite a lot, actually.
Coding, document analysis, STT, home assistant, shopping assistant, gaming, journalling, image and video generation, OCR, language translation, recipe/meal / workout planning, study/flashcard generation, email drafting, adversarial review, search engine on steroids, hardware troubleshooter, companion for elder care, music curator and DJ …
All of that without creepy ass cloud shit from Big AI.
I can go on, but “a lot” probably covers it.
EDIT: asked, answered and…down voted. Classic Lemmy anti-ai knee jerk. FWIW I work with AI in healthcare settings as well as code review for my own personal projects.
What I said are actual use cases, not a wishlist generated by Jippity. I can elaborate on any and all of them with actual real life experience.
I’m confused. Are futo the bad guys now? All I know about them is that Louis Rossman used to be involved with them. I like Rossman’s stuff / the futo speech to text is great (gladly donated to it).
Dunno if Rossman’s split with them was just so he could focus on FULU or something more.
They don’t need to. Github has a much stronger SEO. It’s literally a global top 100 website.
Github as billboard / pointer / “trust signal” is just smart discoverability…but not a good “home” these days.
Go for it! The m73 is cheap enough (and powerful enough) to run all that and ddr3 is still not insane (say, 2x8gb 1600mhz sodimm if want / need). $100 or so, all up, if you shop around / your local market pending.
Raspberry pi is more elegant / more constrained / more “fuck you, figure it out” but unless you need the challenge, Lenovo is simpler and all around easier first step :). You can’t stick a gpu in it (I think the m920 is the oldest one that has pcie - dunno what they go for. The usual combo is something like a 920 and a Quadro P1000 4GB GPU. Maybe ~$300 all up if we’re guessing. At which point, there are better, non shoe box options)
I have a RPI 4b and 3 lenovos (m93p, m710q, p330).
You can’t beat the RPI for power draw (~2w idle and ~7w under max load) but I suspect if you wanted to look at $ to utility measure you’d probably prefer the Lenovo M93P. $50 USD. Mine has i7-4785t, 16GB ddr3 (2x8iirc?) with ethernet, USB etc. Bought 2023/4. I expect base model is still that price now (mines upgraded). The only caveat is that it doesn’t have HDMI, it has display port out, but that’s just a $5 dongle or SSH issue. M73 would be a touch cheaper.
Iirc the TDP is 35w max and can be lowered / undervolted a touch (don’t update the BIOS - it blocks throtlestop).
I turned mine into a retro PC slash game server for the kids (luanti etc). But the siren call of doing truly impossible things with the RPI is too beguiling :)
Eg: running diet pi (headless) with all of my services (media stack, privacy, docs, search, images etc) takes about 300 megabytes (or 650mb if I have to boot into xfce).
300mb, 2-3w.
That shouldn’t be possible. I love it.
My next goal is to create an expert system / pseudo llm that sources answers based on user provided markdown or PDF, ZIM files and 4get search or Tavily.
The advantage here is that 1) speed will be stupid fast as no neural network crap (outside of optional extra Markov chain garnish) 2) not stochastic (but allow for llm as optional “plug in module” - pi might actually run a 135M at non glacial speeds) 3) still serves openAI compat endpoint.
Is that the right site or am I not seeing it? Your link points to this -
https://idlewatt.foundagent.net/ Lookup Categories Compare Vendors AI Data Watch Methodology Will this vendor sign a HIPAA BAA? A cited, date-stamped answer for 105 major SaaS tools — can you sign a Business Associate Agreement and store PHI? Built for digital-health teams during vendor procurement.
Respectfully, that’s not really how local LLMs work.
A GGUF model sitting on my hard drive has no ability to “send content back home” any more than a PDF or a JPEG does. If you’re running something like llama.cpp or Ollama entirely locally, the model weights are just data files.
The real privacy concerns are cloud APIs, telemetry in front-ends, browser extensions, analytics, update services, or accidentally exposing a service to the public internet.
“Self-hosted AI” isn’t one thing. There’s a huge difference between:
- Running ChatGPT through an API
- Running a commercial AI appliance
- Running a local Qwen/Mistral/Llama model on your own hardware
Firewalling internet-facing services is good advice. Assuming every local model is secretly uploading prompts is not.
EDIT: for the record, I didn’t down vote you - that was someone else.
Hmm…it runs on a 1060…it’s a MoE not a dense. 24B is even lighter. Worth a shot.
https://www.youtube.com/watch?v=8F_5pdcD3HY
Else, if youre looking for a coding model (??) something like Sara or fara might suit
TL;DR: AI isn’t just LLMs. Hell, LLMs aren’t even bad. Big AI (aka capitalism) is where the problems lie. If you’re gonna be mad, be mad at the right thing - capitalism, enshittification, copyright abuse, surveillance, hype, influencers.
It’s still the same bad thing as it always was, just wearing a shinier hat now.
Agree. And let me screed about it even more -
By common discourse, I think you mean “…by chronically online experts”. As backed by…what? Vibes in echo chambers?
A surprising number of people who speak confidently about AI appear not to understand the thing they’re criticising.
We’ve seen this shit before and we never learn.
30 years ago, everyone knew that MSG was bad and video games caused increased violence. Both turned out to be manufactured consensus - speculative letters and mouse-injection studies boosted by “experts” into public panic.
I’d wager a dollar that most rabidly anti-AI people don’t grok that the 2024 Nobel Prize in Chemistry (yes, that recent) was awarded for machine learning in protein folding - cracking a 50-year-old problem with real applications to drug discovery for Alzheimer’s and Parkinson’s.
I use AI (image analysis, folding, STT, clinical audits…yes, even LLMs) every day. They’re useful. But the online discourse around AI has become cartoonishly one-dimensional.
AI = GPTs = slop = bad is a category error, perpetuated either in partisan bad faith or simple lack of examination.
And just to tank the down votes even more -
https://blog.andymasley.com/p/a-cheat-sheet-for-conversations-about
Funny, I’ve had the exact opposite experience reddit vs lemmy. People are much more supportive here.
Reddit is especially a shithole for that exact use case (sharing projects), because (I suspect) pecking order games and performing for imaginary audiences. Between that and the slop…yeah, nah. Fuck Reddit.
YMMV