make informed generative ai technology decisions
Evaluating GenAI Tech Partners
Select the right generative AI technology partners.
When it comes to generative AI technologies, you’re not just choosing a vendor. You’re selecting a strategic partner. How can you be certain that the companies you’re considering will meet more than just your technical requirements? Use this questionnaire from our AI Pathfinder™ toolkit to guide your evaluations and get your technology partnerships off to a good start.
As a marketer, you have countless options for generative AI tools and systems. You might start by looking at your current marketing technology stack. For example, major providers like Adobe, Salesforce, and HubSpot are rapidly adding generative features to their marketing and creative clouds. At the same time, you’ll probably consider at least a few of the many AI-native startups that aim to become the go-to solution for specific use cases. It’s a complicated space that gets more crowded by the day.
Choosing the right technology partners for your near-term pilots and long-term programs can feel overwhelming — but it doesn’t need to be. At the end of the day, evaluating a generative AI system isn’t much different from evaluating any other system in your martech stack. You just need to know what to look for.
The right questions will help you decide which tools are right for your use cases, your people, your workflows, and your organization’s requirements for safety, security, confidentiality, and performance.
Here are the areas we explore when evaluating GenAI technologies for ourselves or for our clients. Remember, these questions should inspire inquiry, not lock you into a rigid interview. Customize your questionnaire. Gather the information that matters most for you and your organization.
15 Areas to Explore
- Core capabilities: What is your GenAI system’s primary function, and how does it align with our marketing use cases? How can your GenAI features and functionality add value to our marketing programs and workflows? If your solution also uses traditional, predictive AI algorithms, how do these work together with your GenAI to deliver outputs or outcomes?
- Foundation models: Which GenAI foundation model(s) do you use in your system or applications? Are they third party, open source, or proprietary? How do you customize or fine-tune those models? To what extent can they be fine-tuned or prompt-tuned with our proprietary data?
- Model training and maintenance:What’s your approach to model training, retraining, fine-tuning, and maintenance? How often will the models need to be updated or retrained?
- Performance and accuracy: How do you measure the performance and accuracy of your generative AI models or applications? Can you provide any benchmarks or metrics to demonstrate their effectiveness?
- Transparency and explainability: How transparent and explainable are your models? Can you provide insights into the decision-making process behind the generated output?
- Customization and scalability: How customizable and scalable are your generative AI solutions? Can they be tailored to fit our specific marketing use case(s), needs, and objectives? Can your models be trained or fine-tuned with our proprietary data?
- Data requirements: Does your model or application come pre-trained? Does it use our data? Both? What types and volumes of data are required for your model or application? What are the data quality and preprocessing requirements?
- Integration and compatibility: How easily can your generative AI solutions be integrated with our existing marketing technology stack? Are they compatible with our existing tools and data infrastructure?
- Data privacy and security: What measures do you have in place to ensure data privacy and security? Are your solutions compliant with relevant data protection rules (e.g., GDPR, CCPA) and emerging AI regulations (e.g., EU AI Act)?
- Ethical considerations: How do you address ethical considerations such as bias, fairness, copyright, intellectual property rights, and potential misuse of your models and/or applications?
- Support and collaboration: What kind of support and collaboration can we expect during the implementation and integration process? Do you offer ongoing support, training, and access to experts to help us maximize the value of your technology?
- Learning curve and training: How difficult will it be for our marketers to learn to use your GenAI system(s)? What training and support does your company provide?
- Pricing and ROI: What’s your pricing structure? What does it cost to get started? What are the long-term costs for maintaining, updating, and using and scaling your GenAI? Can you provide a clear ROI estimate for our specific use cases?
- Deployment timeframe: How long will it take to integrate your GenAI system or tools into our existing marketing operations, and what are the steps involved?
- Service level agreements (SLAs): What kind of customer support is available? What are the terms of your service level agreements regarding uptime, performance, and support?
