AI Vendor Vetting Checklist

The checklist

1. 1. Vendor Profile and Stability

   • Company name, headquarters, and legal entity.
   • Years in business, and specifically in AI.
   • References or case studies from businesses similar to yours.
   • Financial stability and funding history, including any pending mergers or acquisitions.
   • Support structure and escalation path: is it 24/7? Regional? Who do you actually reach?
   • Company size, and whether the team is full-time, contract, onshore, or offshore.
   • Whether the company is positioned to keep growing independently, or is likely to be acquired or sold.
   • Who the investors are, and what happens to your data if the company is acquired or shuts down.

2. 2. How Your Data Actually Moves

   • Is the underlying AI model proprietary, open-source, or licensed from a third party, like OpenAI or Google?
   • Does this tool send your data to any outside AI model or API to generate a response?
   • If so, is that data anonymized or masked before it's sent?
   • Are those outside calls logged, so there's a record of what was sent and when?
   • What specific categories of your data will this vendor receive, process, or store?
   • Can the vendor provide a simple explanation or diagram of where your data goes, from entry to storage?
   • Can you, or your IT provider, restrict or turn off calls to outside AI models if needed?
   • Are model updates documented, with a way to roll back a bad update?

3. 3. Data Security and Privacy

   • Encryption in place both at rest and in transit.
   • Role-based access controls and multifactor authentication.
   • Options for where your data is physically stored, if that matters for your industry.
   • Clear, written data retention and deletion policies.
   • A documented breach notification process, and a track record you can actually check.
   • A direct answer to whether your data is used to train the vendor's models.
   • Regular third-party security audits or penetration testing.
   • A written data processing agreement.

4. 4. Incident Response and Business Continuity

   • A documented incident response plan.
   • Clear definitions of what counts as an event, an incident, and a breach.
   • Defined notification timelines if something goes wrong.
   • Recovery time commitments and data restoration guarantees.
   • A business continuity and disaster recovery plan on the vendor's side, not just yours.
   • A clear owner for handling issues like a model error or unexpected AI behavior.

5. 5. Compliance and Legal

   • Compliance with regulations relevant to your industry: HIPAA, GDPR, CCPA, SOC 2, ISO 27001, or others.
   • Clear, written terms on who owns your data.
   • Clarity on who is liable if the vendor mishandles your data.
   • Your right to audit how your data is handled.
   • Indemnification if a mistake by their AI causes damage to your business or a client.
   • Disclosure of any subcontractors or additional vendors involved in delivering the service.

6. 6. Responsible Use and Oversight

   • How the vendor tests for and addresses bias in its outputs.
   • Whether you can get an understandable explanation for how the AI reached a given output.
   • Whether a person reviews or can override AI decisions before they reach a customer.
   • A way for you or your customers to challenge or appeal an AI-driven decision.
   • Published principles or guidelines the vendor holds itself to.

7. 7. Fit With How You Actually Work

   • Compatibility with the software you already use.
   • Available documentation and support for setup.
   • Training and onboarding support included, or priced separately.
   • Service level agreements covering uptime and support response time.
   • Clear communication when something changes or the product is updated.

8. 8. How the Vendor Actually Makes Money

   • What the vendor's main revenue source actually is: subscriptions, usage fees, or something involving your data.
   • Whether the vendor uses or sells customer data as part of its business model, and whether you can opt out.
   • Whether pricing changes with usage, and what happens if you exceed a limit.

9. 9. Cost, Contract, and Exit Terms

   • Contract length and renewal terms, including whether it auto-renews.
   • Total cost, not just the license fee: implementation, training, and support.
   • What you're actually expected to gain, in terms you could measure in three months.
   • What happens to your data if you leave: can you export it, and will it actually be deleted.
   • Whether you'll have a real point of contact, not just a support ticket queue.

10. 10. Reputation and Transparency

    • Public transparency reports or disclosures about how the AI is used.
    • Any past public incidents involving data handling or misuse.
    • Independent reviews, not just testimonials the vendor selected.
    • A way to report ethical concerns or misuse.
    • Any pending legal disputes or regulatory investigations.

11. 11. Where the Product Is Headed

    • Whether the vendor shares a product roadmap and updates it regularly.
    • Whether their direction actually lines up with what your business needs.
    • How they handle feature requests, and whether customers get any input.