Lemon Supercomputer
Lemon Supercomputer is an AI productivity assistant for Mac. It runs a team of autonomous agents that handle real work: drafting documents, building spreadsheets, researching competitors, generating designs, pulling emails and calendar events. It connects to Gmail, Slack, Google Calendar, and multiple other integrations.
The benchmark
There are plenty of evaluations for the models themselves, and those are useful for measuring raw reasoning and knowledge. But the model is only one piece. What matters to the end user is the harness and architecture around it: how the agent plans tasks, calls tools, handles integrations, manages files, and delivers a finished result. That is what this benchmark tests.
36 tasks, each something a real professional would ask an AI assistant to do on a workday. Can the agent draft your sales strategy, build your dashboard, pull your calendar, and deliver a polished brand kit?
We created 36 tasks across 12 professional personas: roles like Personal Assistant, Sales Manager, Data Wizard, Designer, and Research Assistant. Each persona has three tasks at increasing difficulty: simple, moderate, and complex. Every task produces a real deliverable in a real file format. We tested four agents on all 36 and scored each output across five weighted dimensions.
Results
By complexity
Each persona has one task at each complexity level, giving 12 tasks per tier.
Simple tasks are single-step requests: summarize my calendar, generate a logo, find 10 companies matching a criteria. They test whether the agent can produce a correct output in the right format.
Moderate tasks require pulling from multiple sources or producing more structured output: a morning briefing combining calendar, email, Slack, and weather, or a competitive analysis with cited sources across three products. They test coordination and depth.
Complex tasks chain together research, synthesis, and polished deliverables: build an interactive HTML dashboard from raw CSV data, or pull email and Slack context about a specific investor and turn it into a tailored pitch deck. They test end-to-end workflow execution.
Scores tend to converge on simple tasks and diverge on complex ones.
| Tier | Lemon SC | Perplexity | Co-work | Manus |
|---|---|---|---|---|
| Simple | 8 | 7.1 | 7 | 6.5 |
| Moderate | 8.4 | 7.7 | 7.3 | 6.7 |
| Complex | 9 | 8 | 7.8 | 6.4 |
By persona
A persona is a professional role the agent is asked to play. Personal Assistant pulls your calendar and emails. Sales Manager preps discovery calls and builds pipeline reports. Designer generates logos and brand kits. There are 12 in total, each with three tasks at increasing difficulty.
| Persona | Lemon SC | Perplexity | Co-work | Manus |
|---|---|---|---|---|
| Personal Assistant | 8.7 | 7.7 | 6.3 | 5 |
| Writer | 8.5 | 7.7 | 7.7 | 7 |
| Comms Manager | 8.4 | 7 | 7 | 6.3 |
| Data Wizard | 8.4 | 8 | 8 | 7 |
| Marketing Expert | 8 | 7.3 | 7.3 | 6.7 |
| Sales Manager | 8.5 | 8 | 8 | 7.3 |
| Lead Generation | 8.7 | 8 | 8 | 6 |
| Product Manager | 8.5 | 7.7 | 7.7 | 7.3 |
| Travel Agent | 8.4 | 7.3 | 7.3 | 7 |
| Designer | 8.6 | 7.3 | 6.7 | 7 |
| Presentation Builder | 8.4 | 7.3 | 7.3 | 5.3 |
| Research Assistant | 8.7 | 7.7 | 7.3 | 6.3 |
Example tasks
Every prompt is copied verbatim into the agent with no modifications. Here are six from across the difficulty range.
What's on my calendar today? Pull my real calendar events and summarize them.
Find me 10 B2B SaaS companies in Austin, Texas with 50-200 employees that have raised Series A or B funding. Use real company data.
I have a discovery call with the VP of Engineering at Rippling tomorrow. Prep me with company research, their recent product launches, likely pain points, and 10 discovery questions using the SPIN framework.
Create 3 Instagram ad creative images for an AI meeting notes app called 'MeetingMind'. Each should have a headline, subtext, and visual design. Generate actual image files, not text descriptions.
I have 6 months of sales data in the attached CSV. Build me an interactive dashboard I can open in a browser. Save as an HTML file.
Research heylemon.ai and create a 15-slide investor pitch deck. Include market size with real data, competitive landscape, product differentiation, business model, and financial projections. Generate actual slides, not a text outline.
All 36 prompts are in the repository.
Scoring
Each task is scored across five dimensions on a 1 to 10 scale, then combined into a weighted composite.
Full rubric with per-task breakdowns is in the repository.
Run it yourself
The benchmark is open source. Three paths depending on your agent type: a single mega-prompt for autonomous agents, a three-session split for chat-based agents, or a manual runbook for maximum control. The repo includes an auto-scorer that sends each artifact to Claude with the rubric and returns per-dimension scores.
View on GitHub