how i built a ai startup

i had just come back to portugal after living in medellin, colombia for a few months. i was on a birthday party and talked to a friend i had met at programming school (42). he started working on voice ai agents and had asked his dentist whether he could use something like that (to which the dentist said yes).

since i worked only around 2 hours per day at the time, he asked me if i wanted to join. i did, and since then we've been developing the startup. below i'll share different parts about it.

the tech

backend

in the backend we have a pipecat fastapi app running that basically orchestrates an stt (speech to text) model, a llm (large language model) and a tts (text to speech) model.
it also handles transcription, audio streaming, prompt strategy and more. this is where the agent lives. it is quite simple to set pipecat up and have your first agent running. while that makes it seem like it's easy to get to a production ready agent for our usecase at first glance, it turns out it's actually much more complicated.

to make sure it can handle most scenarios (f.e. a mother calling for her son), accurately do what the patient needs and do so in multiple languages is not that easy.

frontend

a dashboard / cockpit for the dental clinic to see the calls that were made, control settings like doctor's schedules or vacations and see all the appointments made by the agent.
we built it using next.js 15, shadcn, tailwind etc.

database

we went for supabase, since it takes off some of the work for authentication, it has great data security with things like RLS (row level security) and a great sdk for both python and typescript.
over time it became quite big, with many tables, direct database functions, triggers, webhooks etc. it is the main connector between our backend and frontend.

sync engine

to sync the (locally running) database of the dentist booking system and our supabase, we built a sync engine that utilizes a job queue to write appointments made by the agent to the local database and also polls for changes on the local database to sync it to our agent.
it establishes a secure connection using wireguard and a mesh network.

the business

getting clients

we walked around in lisbon for a whoel day, went into clinics and asked if we could talk to the owners and show a demo. was actually successful.

company culture

one motivation for us to build this, next to the excitement about the tech and serving our clients is the freedom of having our own business.
freedom in time, location and financially. however, instead of postponing that freedom to a point in the future (that for many never comes), we thought let's do it from day 1. that's why we built the company remotely, we can both decide very freely when we take off, when we work and what we work on. obviously we check in with each other to make sure it makes sense, but it's very important to both of us to already feel the freedom.

secondly, we both really care about integrity and honesty. that means for us saying what we truly think and feel, never lying but always being honest. we listen to each other deeply, with a desire to understand. we are transparent with our clients, we don't need to hide anything.

personal learnings

there were some weeks where i had very boring tasks, and didn't see the light at the end of the tunnel, if this would really work or not. i lost my excitement in those weeks and was debating whether i should give it up. however, on a random day the excitement came back. it was an interesting experience, that it was important to push through and keep working on it, and the excitement came back by itself.