There’s a Reason that AI Talk at (Procure)Tech Events is All Talk and No Substance

A recent rant on LinkedIn noted that, at DPW (and other events), there was “Tons of talk about using AI, not so much about how to procure it, scope it, contract it, measure it, commercial agreements, which suppliers are best for which usage.”

Well, there’s a simple reason for that. And it goes as follows.

It’s the tech-du-jour. More specifically, the hype-du-jour. As a result you can forget any advice on how to:

–> Procure It

No one really knows who has what, or what it actually does, what it’s really worth, how to properly cost it, how to compare offerings and offers. So how can they tell you how to procure it?

–> Scope It

According to the hype it’s your new employee that does everything for every one, despite the fact that it hallucinates more times per second than an LSD junkie does in a lifetime. Without knowing what it does or where it goes, they can’t tell you how to scope it!

–> Contract It / Commercial Agreements

Even the vendors wrapping someone else’s model don’t really know what it does, what they can guarantee, what they can’t, or who is really liable (although courts are starting to make those decisions). So no one knows how to contract it.

–> Measure It

Whereas we have tried-and-true mathematically sound measures for traditional deep neural networks where you can get accuracy ranges with confidence ranges, when it comes to LLMs, no one has a f*ck1ng clue how to measure them. Random tests by random humans judged by random people with random definitions of accuracy is not a measure. And I’d have more confidence in a decision made by a Koala. At least it’s cute!

–> Assign It

If we can’t even measure what it does, do you think we can measure how well the vendor pushing it really understands it? Definitely NOT!

And that, in a nutshell, is why there’s a lot of hot air and nothing of actual substance in all the AI discussions. Which you should avoid anyway. Because you don’t want tech with a 6% success rate [McKinsey, MIT], which is half the general success rate of new tech installations (now that tech failure rates have reached an all time of 88% [Bain]).

You cannot export-control math!

A truly brilliant observation by Mr. Stephen Klein in a recent post on how China May Be The Only One With Mythos because they may have saved its responses (and thus figured out how to replicate it).

(Gen-) AI LLMs are just mega math models. Really big mega math models with probabilities being computed on top of probabilities being computed on top of probabilities in force-feedback loops that reinforce its computations (which could be brilliant deductions or hallucinations that equal the acid high of the most LSD addicted junkie on the planet).

And since the outputs are dependent on the equations that define the model and the training data, if someone can recreate the training data they can reverse engineer the equations from the output if they are sufficiently adept at mathematics, or at least a close proximity.

Which means that blocking off access to those who can evaluate and improve the model will not help if those you don’t want to have the model already have it.

But this isn’t a post about the blocking of AI models (because I personally think that’s great), but a post about what happens if you try to hide your capabilities behind “proprietary algorithms based on math” assuming that no on else can recreate it if you don’t talk about it and that the IP alone justifies an unreasonable price for your product or valuation for your company.

Every country has mathematical geniuses, and more than one can come up with the next iteration of a mathematical theory at about the same time. Maybe only one gets remembered (Newton vs. Leibniz), but it doesn’t mean they didn’t both invent the same concepts at about the same time (in calculus).

And the more you trump it up, the more it entices someone to tear it down. (And figure out how you built it.)

On its own, the math alone is not the advantage you think it is. There are a lot of free scientific papers with great math. In fact, more than enough to create your own Claude, DeepSeek, Gemini, Grok, etc. But most people can’t because it’s not just the equations, it’s the parameters, the training data, and the implementation.

With regards to the implementation, just because you have server racks that can do trillions of calculations per second, that doesn’t mean you can code inefficiently. For example, an average token output by Claude requires billions of operations, and an average question posed to Claude will require 1,000 to 10,000 tokens to answer, for 1 trillion to 1 quadrillion calculations for an output. A poorly designed model could require 10, 100, or 10,000 times that.

The same goes for classical machine learning or optimization algorithms. Good implementations will require billions of calculations. Bad, trillions to quadrillions to quintillions. Responses go from real-time to hours to days to the computations never end.

Then there is the training data. You can’t judge a model implementation unless you have good training sets that are representative of real-world problems. Optimizing for theoretical problems that don’t exist in the real world isn’t helpful and may, in fact, lead to a worse solution that not even testing it at all!

The real differentiator is the expertise both in the implementation of solutions based on math and deep knowledge of the domain the solution is for. That can’t be recreated by mathematical ability alone and requires experience. And that can be export controlled (and represents the real value).

What Data Do You Need For Successful Procurement?

In a comment to a recent post over on Linked in, Mr. Buckingham asked Do you think that data driven decisions are clearly the correct thing to do, but that they tend to maintain the status quo and can be restrictive to innovation?

I couldn’t leave this one alone and responded that:

“They only maintain the status quo IF the data collected and constraints created are limited to those that support the status quo … which, sadly, they usually are …

As the Sourcing Optimization Grand Master Paul Martyn recently pointed out — the real context never gets mentioned, stakeholders just add “necessary” constraints, costs, and weightings that they know will heavily favour the incumbent

And as the Sourcing Simplifier Garry Mansell regularly points out, organizations fail because they only include the best lagging indicators in board presentations to ensure they can keep doing the same old, same old

But if you instead collect [only] external data on the market, and not internal data that supports the status quo, the tendency will be to focus heavily on innovation and change.

Data is the way to go, but it has to be evenly distributed across internal and external so you can get the full picture.”

Seemed like this is something that should be elaborated on.

For example, let’s say Procurement is trying to gauge the effectiveness of their category sourcing in a key category. They could demonstrate this through internal or external metrics. Internally they could show the average price per unit decreased year-over-year by a significant percentage. Externally, they could retrieve the average market price for the primary products and show they are paying less. If they only ever use one of these metrics, and it stays good, as Mr. Buckingham notes, it maintains the status quo, even if it shouldn’t. In order to truly gauge effectiveness it has to, at the very least, measure both. Just reducing costs doesn’t mean you are getting the best price, and just beating the market doesn’t either. In some categories, market quotes / GPO rates / etc. are never the best price, which is dependent on what you actually need, who, and where, you are getting it from.

Digging deeper, if the specs are over engineered, you restrict to a suppliers in a certain geography, or only deal with incumbents, you’re not getting the best price. So you not only need to take internal and external measures and benchmarks, but ensure you are taking the right internal and external measures and benchmarks.

And then, if there are hidden costs (from increased risk, revised shipping / order lead times, quality reductions, etc.), take this into account as well. (Which is why you need strategic sourcing decision optimization with what-if scenarios, as we’ve been telling you for the past 20 years.)

Only when you identify the right data and collect the right data can you make the right decision, which might reinforce the status quo, might tell you do do something completely different, or tell you to split your bets and do both!

The reality is, you should always make data-driven decisions, but you can only do so if you are collecting the right data for your needs.

Software Acquisition Insider Tips 2026 Part VI

It’s been 17 years since SI published its first major series on generic insider tips back in 2009 where we gave you a lot of advice that more-or-less still stands today if you want to safely acquire software. In our preamble, we overviewed what those 11 pieces of advice were then, and summarized the 6 major difference that affect how you apply that advice today so you can continue to make the right decisions when acquiring software in the age of AI Hype and exaggerated I2O claims. In the last four parts we addressed the first eight pieces of advice and how they have evolved over the years. Today, we conclude.

Separate software from service

Seventeen years ago we wrote:

Many software products aren’t really software products at all. In other words, there is some incantation that has to be performed by the software vendor, in the form of services, configuration, or other magic, on a regular basis, to keep the software running. In this case, you haven’t bought software, you’ve bought software plus services. What’s even worse is that you’ve single-sourced it. If the vendor goes broke, or can’t deliver, you have no options.

This is still a reality with many products, and, even worse, in the age of AI-based “agentic” offerings, they only keep working if the vendor is constantly monitoring, maintaining, upgrading, retraining, correcting exceptions and errors behind the scenes that you don’t see, etc. There’s more hidden services than ever. That, combined with the escalating token costs, is why they have to sell you on “outcomes” because they can’t charge SaaS fees and survive. (And that’s why outcomes is a dirty word. Just remember freedom of speech means that they can say anything they want, even if it’s not true, but that you have the right to ascertain the truth and demand that word never be used again in their RFP responses! [As well as “AI”.])

Good software is very affordable today. If it’s too expensive, it’s either relying too much on experimental AI that doesn’t work, stuffed full of features you don’t need, or hiding services behind the scenes you don’t want to be paying for.

Monitor the market

This isn’t 20 years ago where there were very few price benchmarks beyond what you could collect from an RFP and a few general price ranges from your favourite consulting firm that were wider than an average football field, this is now where there are dozens of platforms that monitor SaaS spending and a few big consultancies that specialize in not only monitoring and pricing how low a vendor will go, but breaking apart their combined bids into individual SKUS because they have hundreds of data points to compare against as they have been collecting that data for years as they negotiated on behalf of their clients. If you do the research, you should know exactly how much each vendor will charge, on average, for the solution you are looking at for an organization of your size, what the price trends are, when they make the best deals, and what the best pressure points are. If you don’t, you shouldn’t be making major 6, 7, and even 8 (when you consider lifetime) figure software acquisitions. Period. Market monitoring is a must!

Skip the mind games

Abruptly end a meeting mid-stream to make a point? Make the salesman sweat at quarter-end to squeeze a few extra discount points? Scream emotionally that they are price gouging and you’re going to not only report them to the Better Business Bureau but tell all your friends in the local Procurement Association? Threaten to use Klod or Chat J’ai Pété to build it yourself? Lie and say you can make do with your current system for another quarter if you can’t come to a deal or just select any random competitor at DPW?

Sure you could use these techniques to try to get a better deal using an antagonistic wild west negotiation philosophy, but at the end of the day, it will cost you more than you save because even if the rep caves (because the rep knows if he doesn’t close something, he’s the next rep walked out the door by the investors to make a point), how incentivized is the rep, and the vendor as a whole, going to be in helping you to succeed? Especially if their profit margin is close to 0 in a best case situation? Answer: not very. They will do the absolute minimum to meet their contractual requirement, and then take the phone off the hook, tell the chatbot AI to infinitely loop you when you try to contact support, and, when there is an actual bug, make sure you are last in the queue to get serviced, waiting until the last minute of the SLA to do it

Focus on the value you are getting, a price point based on real market data and intelligence that they should be able to match (and make a fair margin while actively supporting you to meet the ROI metrics they promise), and delayed payments until the module is actually live and users actually onboarded before you start paying for it. (In other words, if you buy six modules, but only two are available day one, two more won’t be available for 90 days, and two more for 180 days, you don’t pay the full license fee until all modules are live AND all users set up. And implementation/integration payments are tied to milestone completion.) That’s way more effective.

Now, as always, there are a lot more tips and advice we could give, but these are the biggies. If you want more details, dig deep in the archives. Or, you can contact <font=black>the doctor for an engagement.

Software Acquisition Insider Tips 2026 Part V

It’s been 17 years since SI published its first major series on generic insider tips back in 2009 where we gave you a lot of advice that more-or-less still stands today if you want to safely acquire software. In our preamble, we overviewed what those 11 pieces of advice were then, and summarized the 6 major difference that affect how you apply that advice today so you can continue to make the right decisions when acquiring software in the age of AI Hype and exaggerated I2O claims. In the last three parts we addressed the first six pieces of advice and how they have evolved over the years. Today, we continue.

Draft a real unbiased RFP

Properly Define the Scope

This was the first tip we gave you 17 years ago, and the first tip we repeat today. As per our Successful Vendor Selection Series, solution selection is a 7 step methodology, and the vendor assessment process, which contains the RFP process as a subprocess, is step 5. That’s because you can’t properly define the scope until you:

  1. identify what your real need is
  2. holistically identify what a solution needs to do
  3. determine your organizational maturity (and the complexity of a solution you can actually support)

Only then can you define the scope and begin to construct a good RFP.

Focus on Functions and Requirements, Not Features

What does it have to do? Why? What processes is it supporting? What user groups? What are their primary and secondary use cases? What level of configuration do you need? What integration does the system need to support.

And definitely don’t use a “FREE RFP”. Those are all vendor generated and vendor sponsored (even if being offered by an “independent” third party) and designed to make one specific vendor look good against peers (and focus on the features they have, not necessarily the features you need). Always have been (since Procuri sponsored the FreeRFP site for Procurement 20 years ago that got them enough attention to be acquired by Ariba). Besides, most applications will be stuffed with features you don’t need and never use. How many (Microsoft) Word features do you use regularly? 30? Maybe 50 if you’re a “power” user. How many features does (Microsoft) Word have? Ten times that (and maybe more). It’s the same with enterprise apps … especially from providers that aren’t providing anything differentiated or actually improving on the core offering and core value … lots of features, few functions, and fewer still that provide value.

Draft the RFP BEFORE You Look at Any Solutions

Or any vendor material — it’s all marketing propaganda anyway! Otherwise, you will be tempted to write the RFP using the terminology and viewpoint biased by the vendors you researched / liked the most from their public marketing and demo materials. This will give them an unfair advantage, that they will take full advantage of, and that can prevent you from getting the right solution.

DRAFT THE RFP BY HAND

DO NOT USE GEN-AI! Sure it can whip up a good sounding 20 page RFP in 20 seconds on a powerful enough server, but sounding good (because it’s grammar and vocabulary is better and bigger than yours) is not being good. It can leave out key requirements. Forget critical specifications. For standards, get the requirements wrong. For integrations, get the versions and extent of requirements wrong. For services, get mandatory SLAs wrong. And so on. Good RFPs are drafted by hand. You can use semantic AI tools to improve the grammar and spelling, and then use Gen-AI to make suggestions as to where you want more detail, clarity, etc, but good RFPs are drafted by hand! (Now, this is for software acquisition and strategic purchases. For tactical commodity tail-spend where it’s just a puffed up RFQ, it’s not as critical you start by hand.)

Define ALL Of Your Evaluation Criteria Before You Send It Out

As well as any core requirements for the vendor / platform that can’t be waived. If there are a number, do a quick up-front RFI where you collect all the financial, legal, compliance, platform, and core function requirements and then don’t even bother sending the RFP to any vendors who don’t make the cut. If the RFI eliminates too many, determine which requirements are really absolute vs. which are strongly recommended (and weighted highly) in the evaluation.

You need to know what you’re looking for and how you are evaluating BEFORE you read the first word of any vendor response.

… as Well As Your Metrics for Measuring Implementation and Adoption Success

A successful implementation is not one where the vendor creates a new instance, connects to your ERP, and sucks in the data. A successful implementation is one that is adopted, utilized, and generates a return on the chosen metrics. That’s why the last step of the seven step process is post-implementation monitoring, advisory, and training. Until you reach the target metrics, the implementation, and vendor, ain’t done. So your prospects need to know up front what’s expected of them, what is required in the SLA, how they will be measured, and what milestones they need to meet.

… streamlined for performance, not wokeness

An RFP defines your solution requirements, not your organizational philosophy. Furthermore, the best vendor could be headquartered half a world away and their operational requirements and societal expectations could be completely different from yours. Plus, your personal preferences in terms of staffing are yours, not theirs, and you should not try to enforce it … especially when doing so could be illegal in your home country or theirs. If you want the best solution, you need to let them do what they need to do to hire the best people for the job. All you can specify is any laws and regulations you are subject to and what your CSR philosophy is. Let the vendor fill in the blanks beyond any absolutes.