9 Common AI Questions Clients Ask Us

1. Should you use a large language model (LLM) or traditional machine learning (ML)?
2. How do you measure ROI on a generative AI project?
3. Should you be worried about data security and compliance with AI?
4. How fast can you deploy AI and see value?
5. How do you estimate token costs for AI projects?
6. How do you choose the right AI model for your project?
7. Do you have to use your cloud vendor’s model for your AI project?
8. Which cloud vendor is right for AI integration?
9. Do you need a RAG pipeline (Retrieval-Augmented Generation)?

Answers to these questions are not just technical choices; they’re strategic moves. Get them right, and your AI initiative delivers real, lasting impact. Get them wrong, and your momentum could stall.

In the sections ahead, we break down these questions and guide you from curiosity to confidence in the next phase of your AI implementation.

Should you use a large language model (LLM) or traditional machine learning (ML)?

This is one of the most critical early decisions in any AI journey. It affects how you approach your data, what problems you can solve, and how much your solution will cost. In short, we like to break it up this way:

• Use an LLM when your project requires natural language understanding, summarization, classification, content generation, or knowledge extraction.
• Use traditional ML when working with structured data, like spreadsheets or databases, and need predictive models (e.g., sales forecasting or churn prediction).

Pro tip: Many successful enterprise solutions combine both. They use LLMs to handle unstructured user input and ML to handle structured, backend decision-making.

How do you measure ROI on a generative AI project?

You can’t manage what you don’t measure. And without proving the business impact, AI initiatives quickly lose support. However, ROI will take time, so it’s best to take a bird’s-eye view of your entire project and (depending on how you implement your AI) track results across these three pillars:

• Efficiency gains: Are you saving time and/or reducing labor costs through automation?
• Revenue impact: Is AI driving more conversions, increasing upsell potential, or enabling new offerings?
• Risk reduction: Has AI helped reduce human error, improve compliance, or respond to customers faster?

Avoid hype metrics that say little about real-world value, such as flashy numbers like model size or benchmark accuracy. Use real outcomes to gain buy-in and to justify scaling. Track the metrics related to the problem you want AI to solve. Measuring the issue first gives you a solid foundation to build from.

Should you be worried about data security and compliance with AI?

Yes, and you should plan for it early. Trust drives adoption, and that trust depends on how well you protect your data.

We recommend:

• Preventing training misuse: Make sure your data isn’t being used to train third-party models. Enterprise agreements can help with this.
• Securing your infrastructure: Use private endpoints, encryption, audit logging, and strict access control.
• Consider on-premises: For highly sensitive environments, deploying models on-premises offers full control.

Bottom line: Own your data and protect your systems. Prioritizing security early can also help you avoid unexpected compliance headaches down the line.

How do you estimate token costs for AI projects?

Generative AI models charge based on tokens the model reads and generates. If you’re not careful, costs can escalate fast.

Watch for:

• Model size: Model size can vary depending on the project. For reasoning models, there’s typically a “reasoning effort” setting, like low, medium, or high, which affects how many tokens the model uses. Higher effort means more tokens and less predictability in output length. We usually control the “max completion tokens” setting to manage this. This sets an upper limit and helps prevent the model from going off the rails, but it can also cut off longer responses. So, the right choice depends on the project’s goals and prompt length.
• Prompt length: Bigger queries and responses = more tokens = higher costs.
• Usage volume: Frequent usage across teams can multiply your monthly bill.

To pick the right model and understand your likely costs, test early. Run simulations during a pilot phase to forecast and manage budget expectations.

Pro tip: Work with an agency that’s been through this before. A seasoned consulting partner can help you choose the right model, steer you away from wasteful usage, and guide you toward cost-effective options; including newer models or fine-tuning strategies that fit your budget.