So I've been walking around with a bunch of business ideas in my head for the last couple decades. I finally decided to actually start taking these ideas out of my head and putting them out into the world. If you're curious you can see some of the other early work on my project list.
What is friendlai.xyz?
Friendlai (pronounced just like "Friendly". Sorry, naming things is hard) is an AI platform built to take advantage of spare compute capacity from modern macs and powerful gaming GPUs. It's built on top of open source models where prompts and requests get farmed out to a community network of nodes. This means your queries won't end up in a data center, destroying the planet and drinking all of our water.
Why does it cost money?
There's still costs associated though. The orchestration server has costs. We need to incentivize folks to actually run these nodes and pay for the extra power consumption (tiny) and wear and tear on their machines (never actually proven), so you still have to buy credits.
I've tried to keep the credits as cheap as possible. The amount of credits consumed is based on the size/complexity of the model, which hopefully brings more powerful machines into the network.
Is this finished?
I'd still consider this a beta / experiment. Today is day 1 of being truly live. I haven't attracted a ton of nodes or users yet. I don't yet know the problems I'm going to face (privacy, security, speed) or how it's going to match user expectations.
I think this platform would be really great as a replacement to the (admittedly amazing) frontier models. It's also theoretically really great for large bulk workloads where you don't need a frontier model and cost ends up being a top consideration. (We have a standard API)
Depending on how folks use it, what their reactions are, I'll definitely keep expanding it. It already solves a bunch of problems for me, and it's one of my favorite sites for accessing open models more easily.
What's it built in?
- The latest version of Angular with SSR of course
- AppHosting
- MariaDB SQL
- Stripe for Payments
- Nodes are TypeScript but compiled into real apps with Bun
- Ollama under the hood for powering all of the open model queries
What next?
Based on the first day feedback, I think some users are expecting frontier-model features, like connectors, RAG (web-search agumentation for recent / novel information), etc. I started working on these and they should be available soon, but they have hard costs (I'm not a search engine), which will have to be deducted from credits.
Give it a try and give me feedback! And please stay Friendly <3
