# An AI Personal Assistant

This document is where we will compile ideas and thoughts about how we will make a personal assistant, using current AI technology, to be useful to us personally.

## Technology
The intent is for this to be natural and easy fit into our lives. We do **not** want to have to use extra apps or jumps through hoops. Interacting with this assistant should be done in iMessages, or WhatsApp, or standard AI assistants.

We are going to try to use Zapier and connect to it directly. Whether through a custom Telegram bot and MCP servers in Claude.

### June 18, 2025
I did some research, and a good UI could be WhatsApp. They support an [API integration](https://developers.facebook.com/docs/whatsapp/cloud-api/guides/send-messages). We can stand up an REST API server in Python that will listen to webhooks as well manage the integrations with the LLM.


The CRON Task Runner in this schema is what allows the agent to initiate conversations unprompted by scheduling workloads to happen at an arbitrary time not tied to a user initiating the conversation.

The [WhatsApp API pricing](https://business.whatsapp.com/products/platform-pricing?country=North%20America&currency=Dollars%20(USD)&category=Utility) is tricky but reasonable. According to them, it is free to respond to messages that the users initiates within a 24 hours window. However, if the agent initiates the message, it costs `$0.004` per message.

If we assume that 50 messages per day will be exchanged and from those 25% are agent-initiated, this is the cost breakdown:
13 x `$0.004` x 30 = **$1.56/mo/user**

Of course there are other costs like using the LLM, database storage, etc... I don't want to do the math now but from experience this is what we can expect:
LLM costs = **$10/mo/user**
Database & server costs = **$20/mo**
Zapier costs = around **$50/mo**

The size of our server and database is more than enough to support +1000 users so this is generous enough to allow us two to just use it.

[Zapier](https://zapier.com/pricing) is the tightest bottleneck it terms of cost. It allows our agent to access data from other apps (email, calendars, etc...) so it's important to have. Nonetheless, there is high a chance we can defer paying for Zapier and get by on the free plan but it will depend on our usage which is TBD once we have a prototype in hand.

In order to give the EA the ability to send voice messages, we can use [ElevenLabs](https://elevenlabs.io/app/speech-synthesis/text-to-speech) where Verdi already has an account. My plan supports up to 60,000 characters of audio per month and additional characters cost `$0.15/1K chars` once that limit is hit. If messages are on average less than a tweet's length (140 chars) then it would fall within usage limits. See math 25 msgs/day x 30 days x 140chars = 105K chars.
We would thus pay an additional $0.15 x 4 = `$0.60/user/mo`.

## Architectural Pillars
- Agent CRON checks every 10min for a due action/task
- Agent remembers things about the user
- Agent can self-manage tasks list that are time-interval or event bound (V.8, V.9, V.4, R.1, R.3, R.6, R.8)
- Agent is multimodal and can listen/send audio messages (V.1)
- Agent can use ChatGPT for Deep Research and simple searches for finding information (V.2, V7, R.2, R.4, R.9, R.10)
- Agent can access gated websites through my account for research like FB marketplace or LinkedIn. (V.6, V.7, V.10, V.3)
- Agent can perform long-running tasks (V.10, R.9)
- Agent can read or take actions in my applications like Notion, Gmail, Calendar (V.3, V.5, V.4, R.7)
- Agent can organize info learned from users dynamically according to requirements (R.1)

## The Use Cases

The things we are looking to have our assistants accomplish for us.

### Richard's
1. ( Getting overwhelmed with too many ideas/losing creative thoughts ) < Organize all my ideas into themes, prioritize execution, and remind me when to revisit paused ones. Quick-capture voice or type ideas via Imessage and auto saved files. >
2. ( Never waste Time in traffic again ) < I'm about to leave for 9AM Meeting/work, my assistant check traffic and says "leave in 12 mins to avoid a jam on Main St." >
3. ( Your life, beautifully remembered ) < Weekly check-ins and quarterly highlight reels, automatically crafted by my assistant. ( It's Sunday evening, my assistant send a nudge: Ready to log your week? I jot down who I met, what I watched, where i visited, and my goals I hit. -- Then at the end of the quarter, my assistant surprises me with a visual Stroy, a personalized highlight reel with Ai generated art and reflections from everything i did. -- like my own private documentary.) >
4. ( Deals that knows you ) < I have a date night planned for Friday, my assistant checks my calendar and date and time, and says 3-course deal at your top-rated Italian place - want to reserve? \ my assistant knows that I want a new watch, a laptop, or headphones from a specific band or store -- my assistant send me nudge saying there is a deal for this watch i want, it fits my budget of 800 dollars.>
5. ( Plan a $100 budget date night ) < I have data to my assistant that I would like to spend 100 dollars on my next date night, given time and date -- my assistant Automatically plan a date night with activities of my interests, with events happening on the day of, ( eg. live music, festivals, F1 car racing, salsa dancing downtown etc.) All under my budjet. >
6. ( Getting bored with routines) < Experience something new Fridays -- my assistant suggest variations in my workouts, playlist, Food, hobbies, clothings, events etc.>
7. ( spend less time searching for that one email ) < My assistant becomes a memeory recall tool (eg. I can say when did verdi send me his resume, or last email. search for a specific email etc.) >
8. ( Train my brain ) < My assistant Train my brian, one challenge at a time. (eg. Daily Wonderlic-style tests to boost my cognitive skill, track my progress, and gamify my personal growth) -- Find the missing number: 11, 22, 33, __ 55. A) 44, B) 43, C) 42, D) 41 >
9. ( Plan a $2,000 budget trip ) < With specific details, I asked my assistant to planned my a 2,000 dollars budget Trip -- my assistant Auto plan my trip by building a dashboard, of the best travel destination to my liking, best deal of hotels, flights. uber rides, and prefer activites. All under my budget. >
10. ( Not knowing when cool new places open near you ) < my assistant auto-track new stores openings, coffee shops, event or markets based on my intrests and zip code. A new form of discovery. >

### Verdi's

- [x] I can communicate with it primarily through voice messages
- [x] It can regularly search for networking events on the internet for me and inform me of the ones that would be good for me to attend
- [x] 3. It regularly looks at the list of the people in my personal CRM and checks their social media for updates and adds it here
- [x] 4. If I have a desired cadence for staying in touch with these people (semi-yearly, quarterly), the assistant will either reach out via text, or put it on my calendar to give them a 15min call
- [x] 5. The assistant regularly checks my email inbox and unsubscribes from the marketing emails I don't want. If it's unsure it will compile a list and propose to me if I agree with unsubscribing with those
- [x] 6. I can ask my assistant to find things I want to buy and it is able to look on various places: Google, Amazon, resell website like FB Marketplace, etc...
- [x] 7. I can ask my assistant to do deep dives on potential targets I want to get in touch and build relationships with (ie: Hiring Managers, startup founders, etc...) and it will do deep research to build a profile
- [x] 8. It Can follow up with me unprompted and ask about things on a daily/weekly basis. For exaample:
1. how am I doing on my leetcode, have I done them yet?
2. Send motivational messages in the morning alongside a quick update with my goals for the day
3. Can pester me about the tasks on my calendar or todo-list and hold me accountable to them
- [x] 9. Continuously learns about me to be useful. At a set interval, asks me questions to get to know me and stores in its DB so it knows about my life (useful when sending auto-updates to people)
- [x] 10. Can help plan meals and grocery for the week. If I tell it we're missing something, it will go open the grocery pickup app, find it and add it to the cart. If it struggles or runs into issues doing this, it will commmunicate back to me the problem: it's out of stock, or couldnt connect to the app, etc...