How to Audit an AI Agent Before Your Customers Do
We audited six companies’ AI support agents. The costly mistakes weren’t the viral ones. Here’s how to test yours against your own policies, regularly like pen testing.
Blog
AI & Evals
Evals, prompt optimization, synthetic research and the economics of working with LLMs — from 2022 to today.
7 min read
Escape the Permanent Underclass
AI spending ranges from $0 to $20,000/month — and the gap is growing. Explore the "permanent underclass theory," why Silicon Valley fears being priced out of AI, and what economics says about whether human labor really becomes worthless.
Read more
9 min read
The Pairwise Gold Triad: Hill Climbing with Subjective Evals
The Pairwise Gold Triad is an eval pattern for subjective AI tasks. Compare challenger outputs against an incumbent using gold references and a fixed LLM judge—then hill climb with autoresearch.
Read more
10 min read
Lessons Learned Generating 5,000 AI Personas
Discover hard-won lessons from scaling AI persona research to 5,000 simulated consumers. Learn practical tips for managing costs, ensuring statistical accuracy, and avoiding common pitfalls when using LLMs for market research.
Read more
11 min read
Bridging the NCO Gap: The Human Backbone of the Post-AGI Economy
AI excels at strategy and execution but fails at creative problem-solving. Explore why humans will remain essential as "NCOs" in the post-AGI economy, bridging the gap between AI planning and AI action through adaptive intelligence and real-world judgment.
Read more
26 min read
Prompt Engineering: From Words to Art and Copy
AI art is here, and prompt engineering is the job of the future. Take my prompt template and learn how to find all the art styles, quality boosters and magic words to get the generative art you need.
Read more
9 min read
Bait Hook Reward Payload
How to (actually) write engaging LinkedIn posts with ChatGPTOne of the things people get super excited about with ChatGPT is it’s ability to write content for you.
Read more
15 min read
Forcing AI to watch 100s of hours of video and tell you what you missed
Ever wished you could watch hundreds of hours of video in minutes? We did it with AI. Dive into our journey of analyzing countless gaming streams, the hiccups we hit, and what this means for how we'll learn from videos in the future. Get ready for a world where no visual content goes unseen.
Read more
5 min read
How to write a book on AI when everything is changing
Prompt engineering is a fast moving topic, so to write a book on it, we had to zoom out and find transferable and timeless principles that worked with GPT-3, still work with GPT-4, and will work in the future with GPT-5.
Read more
9 min read
Vision Prompting Guide
Vision models are underrated, and in may ways are the future of LLMs. Learning how to prompt them now will pay off dividends in the future.
Read more
10 min read
You're the secret sauce: writing original content with AI
Most people using ChatGPT or AI writing tools for content marketing have it exactly backwards. You have to provide the secret sauce, and use AI as your ghostwriter.
Read more
8 min read
Prompt Optimization
It's one thing to discover a prompt that works through trial and error, but prompt engineering is a whole other discipline with a lot more scale and robustness required. You need to be A/B testing your prompt by running them 20+ times in order to really understand what affects performance.
Read more
5 min read
Winning the “AI wrapper” game
How do you win the AI wrapper game and level up as an AI-powered company? There's a standard path to get there that you can follow, which will increase quality and (eventually) decrease cost.
Read more
5 min read
ChatGPT in GSheets
If you've ever wanted to use the power of ChatGPT in Google Sheets, now you have a template which gives you access in a simple Custom Function.
Read more
11 min read
Can ChatGPT Make You Laugh?
ChatGPT can tell jokes if you know how to ask. I use prompt engineering to make ChatGPT funny again.
Read more
4 min read
ChatGPT doesn't respect word count
ChatGPT can't count words, so stop trying to ask it. This is why prompt testing is important, because without it you wouldn't know what part of your prompts are performing.
Read more
6 min read
AI Writing Style - Prompt Experiment
How do you get the AI to write like you? Here are my results of A/B testing multiple approaches, with the winning prompt that performed 20% better!
Read more
4 min read
GPT3() Function in GSheets
GPT-3 is revolutionary but relatively inaccessible if you don't know how to code! Using GPT3 function in GSheets will make it easier to integrate into your workflow.
Read more
11 min read
AIO: Artificial Intelligence Optimization
AIs are growing exponentially and as a result consumer behavior is changing. There's an opportunity for forward-thinking marketers to invest in AIO or Artificial Intelligence Optimization and ride the wave as brands scramble to get into the AI training set.
Read more
6 min read
Building your own AI assistant like Iron Man
Everybody wants an AI assistant like Iron Man. For the first time it's becoming possible to actually build that, with completely open-source components.
Read more