10 Questions to Ask Every AI Vendor
Separate hype from substance with these questions that AI vendors don't want you to ask.
The vendor problem
AI vendors are optimized to sell, not to help you make good decisions. They'll show you the best possible demo, cite the most impressive case study, and gloss over the hard parts.
Your job is to see through the pitch.
The questions
1. What's your error rate, and how do you measure it?
If they can't answer this clearly, they don't understand their own system.
2. Can I talk to a customer who's been live for more than a year?
Anyone can be happy at month three. Month fifteen is where the truth lives.
3. What happens when the model is wrong?
How do users know? What's the recovery process? Who's liable?
4. How does the model get updated?
Models drift. Data changes. How does the system stay current?
5. What data do you need, and what do you do with it?
Where does it go? Who has access? What happens when the contract ends?
6. What's the realistic timeline to production value?
Not the pilot. Not the demo. Real, measurable business impact.
7. What does your pricing look like at 10x scale?
AI costs can explode with volume. Know the trajectory before you commit.
8. What's your AI actually doing vs. what's rules?
Some "AI" products are 90% rules with a thin AI veneer. That's not necessarily bad, but you should know.
9. What use cases are you NOT good for?
If they say "we're good for everything," that's a red flag.
10. Can I run a blind test against your competitor?
Confident vendors welcome this. Nervous ones don't.
Build vs Buy in AI: A Framework for Decision Making
When should you build custom AI solutions and when should you buy off-the-shelf? Here's how to think through the decision.
AI Strategy Starts with Questions, Not Answers
Before you can build an AI strategy, you need to know what questions to ask. Here are the five that matter most.
Start your learning journey
Personalized AI fluency for executives. Daily lessons delivered to your inbox.