- 90% more affordable than Amazon S3. No egress fees
- 15 Edge locations worldwide
The world of enterprise software, SaaS, and cloud platforms has gone through a Cambrian explosion in the last half-decade, as the number of solutions available nearly doubles every two to three years. Meanwhile, the locus for decision-making has been pushed deeper into the organization, and the number of constituents impacted by each decision has also increased when choosing the right software vendor for the use case. Therefore, it’s more challenging than ever to know when the organization is engaged in rash decision-making or whether it has succumbed to analysis paralysis. Organizational stress arises as, in many cases, it is unclear what the right questions are to ask, let alone the right solutions.
In the images below of the marketing technology (2022 MartechMap) and cloud-native landscapes (2023 CNCF map), you’ll see that there are hundreds or thousands of options to choose from, regardless of your general area of tooling selection.
The challenge for the enterprise is how to brave this new world of seemingly infinite options while the amount of organizational resources committed to the tech stack and potential business consequences continue to grow.
So, software selection projects now produce more complex decisions with more impact. The enterprise must up its decision-making game or risk falling further behind its competition or facing outright failure on business goals.
In this article, we’ll address the following key topics:
Ultimately, this will help you with software selection, whether it be:
We'll take a broader definition of a software vendor in this article because the lines between infrastructure, platforms, and off-the-shelf and software-as-a-service (SaaS) are becoming increasingly blurred in the tech stack. It’s a broad unit of measure that can be expanded to include implementation strategies and specific vendors. For example, we’ll consider comparing two software suites that accomplish the same thing with multiple vendors.
There will always be idiosyncrasies associated with any vendor or software vendor category. Generally, the rules for running a good software vendor selection process are applicable more or less evenly across these different parts of the tech stack. This guide will help you whether you are:
It's a challenging process because software comparisons include things that are difficult to compare, that are constantly changing (e.g., new releases, variations in performance, flexible pricing), and that require balancing stakeholder interests, managing political and ego-driven influences, optimizing for both cost and quality and the ease of software implementation.
The philosophy embraced in this article has been derived from our general experience and the experience of Taloflow's many hundreds of customers and users. In the end, however, the organization has to iterate to a process that best fits the organizational needs concerning culture, capabilities, and stakeholders. You’ll usually arrive at a unique implementation of what’s discussed in this article.
In most cases, and despite the advent of AI copilots in the last few years, decisions in software selection projects are still largely made by humans. As such, they suffer from all the same biases and failings of any human decision. Ultimately, we are only good at looking at complex things objectively and arriving at conclusions that consider risk, diverse points of view, multiple perspectives, and cascading temporal impacts if we have a firm process. In other words, don’t “wing it”.
Here’s some further reading on our weaknesses when it comes to complex decisions:
Companies that make good software vendor selections tend to have five common approaches that they use in the software evaluation process:
Companies that get it wrong are prone to making errors in their technology and software vendor selection process, and this has serious consequences, whether it be a drop in stock price or no avocado to sell (see Mission Produce faces operational issues after botched ERP implementation | Supply Chain Dive) or the large failure encompassed in this quote:
"The U.S. Air Force has decided to scrap a major ERP (enterprise resource planning) software project after spending $1 billion, concluding that finishing it would cost far too much more money for too little gain. Dubbed the Expeditionary Combat Support System (ECSS), the project has racked up $1.03 billion in costs since 2005, 'and has not yielded any significant military capability,' an Air Force spokesman said in a statement. 'We estimate it would require an additional $1.1B for about a quarter of the original scope to continue and fielding would not be until 2020. The Air Force has concluded the ECSS program is no longer a viable option for meeting the FY17 Financial Improvement and Audit Readiness (FIAR) statutory requirement. Therefore, we are canceling the program and moving forward with other options in order to meet both requirements.”
Bad software selections have big consequences.
A leading cause of wasted resources and unfulfilled expectations, software vendor selection errors occur when an organization selects a service or software vendor based on specific criteria but later realizes that the chosen vendor needs to adequately meet the business goals.
Software vendor rejection errors happen when an organization rejects a software vendor, believing that an alternative option better aligns with its objectives. However, the rejected vendor might have been more suitable, resulting in missed opportunities for improved outcomes.
Lack of variance errors occur when the software selection process does not include a large enough pool of possible vendors and software solutions.
Inefficiency errors occur when an organization expends excessive resources on the software selection process, such as time (these can take a long time!), money, and staffing. The value derived from the decision does not justify the high costs incurred, resulting in inefficient resource allocation.
Risk assessment errors comprise a failure to assess the risks associated with a software decision accurately. Organizations may need to be more accurate in determining the potential risks, leading to suboptimal decision-making and unforeseen consequences.
Long-term consequences errors relate to underestimating or overestimating the long-term consequences of software decisions while focusing on short-term costs or benefits. Balancing short-term gains with long-term considerations is essential for effective decision-making.
The goal of the structured selection process is to ensure:
From our team’s decades of combined experience refining cloud and software vendor selection, here are some of the most essential elements of the process:
Please note that the term “requirements” will be used broadly here. Here’s what it is:
Objectives are always subject to constraints. Whether it be market-driven specifications that render some possible solutions impractical, the company's balance sheet and income statement, or technical debt. Requirements gathering should be the systematic way the enterprise goes about identifying both the objectives and the constraints:
The following are the requirements gathering steps of enterprises with good technology vendor selection processes.
The organization should be able to define the problem and create a requirements list before it looks for a solution.
If the enterprise fails to meet the five 9s of uptime guarantee, it will lose customers and be subject to penalties.
Never be down for more than 15 minutes.
So often, companies start the process of searching for new software from the bottom - “I need an APM or observability tool!” leaving critical requirements and possibilities off the table.
Once the objectives are defined, then you can create clear requirements lists. We are using the plural here intentionally because if multiple solutions are available, you might end up with different requirements documents.
If you’re interested in getting vendor selection checklists and software requirements templates for various categories, we have a trove of those here:
and more coming soon...
At the start of the software evaluation process, many possible solutions exist to discern the objectives.
However, most potential solutions will not fit well given the circumstances and won’t produce the requisite objectives, whether it be for a bad-fit pricing model or a poor user experience.
The requirements perform two different functions:
Increase sales opportunities in Europe by decreasing the response time of our application, thereby improving the overall user experience.
In the above scenario, we’d first only consider vendors who house data in the US. Then, a second pass of only vendors that can return a response in under 900ms. The requirements are winning threshold functions.
Constraints should be viewed as either:
Suppose we remove the constraint by changing heuristics or another piece of software. In that case, it is an objective, and you should expand the problem solution set to include the potential removal of the constraint. For example, one might add the objective “Set up EU company to handle EU data so that data location rules no longer constrain us.”
Using the same objective above. Let's add an important requirement: “Must have an easy-to-use console interface.”
Let's further define what software features make up the Requirement:
When looking for new software, I can now do a software comparison with questions like “Are vendor A’s colors brighter than vendor B’s?” which precipitates having some basis for making that comparison. No basis is required for applying the constraint.
For the reasons above, at Taloflow, we prefer using hierarchical requirements or, in our case, a “Software Requirement to Software Feature” array. Hence, there is an ability to fine-tune the comparison between those requirements, which are not constraints, and the features that make them up.
One of the implications of using a problem-first approach in software procurement is that you could select problems that don’t neatly fit each other. In such cases, we recommend multiple requirements specifications instead of trying to combine them into one multi-propose requirement document.
We recommend that most enterprises use a requirements-gathering tool to help lessen the organizational stress of the software procurement process, including creating requirements lists. There are several in the marketplace, including:
Of course, AI can be leveraged in the requirements list creation process. Used correctly, these tools should simplify the process.
Once you understand the requirements and objectives, you can develop a plan to provide input into the decision-making process. We recommend letting the requirements docs inform your decision about what tests to run and what vendor information is needed. It's common to look for the following types of data in the decision-making process, often collected by different areas of the organization.
Providing technical data is usually time-consuming and involves an engineering budget. For this reason, we believe that it’s the best way for a technical evaluation to occur after there has been some winnowing of the possible solution set down to a few vendors based on criteria other than the technical evaluation.
We have found that most of our clients use one of two methods to gather technical data:
They hire an expert or reach out to another enterprise that has used the software in a situation very similar to the situation proposed by the client. Here, it is really important to get apples-to-apples software comparisons.
They deploy a test suite, often with a level of abstraction, to gain some basic understanding of the underlying technology. This process is usually associated with a dev team.
Utilizing industry-standard benchmarks and performance metrics to assess the technical capabilities of the software.
Developing prototypes or conducting PoCs to test the software in a controlled environment.
Employing automated tools for stress, usability, and integration testing.
The finance department usually conducts financial analysis taking into account the pricing models, often in collaboration with project managers and IT leaders.
Specialized business analysts or financial consultants play a key role in some of these business processes.
Calculating all costs associated with the software over its lifecycle, including acquisition, operation, and maintenance costs.
Estimating the financial return expected from the investment in the software.
Advanced financial metrics to evaluate the profitability of the investment.
The goal of vendor evaluation is to assess whether or not the vendor will be able to provide the service in the future and how the vendor makes the service available (including things like support) in a way comparable that’s to others.
The vendor evaluation often requires a cross-departmental software selection team involving IT, procurement, legal, and sometimes external consultants.
Evaluating the vendor's alignment with the company’s long-term strategic goals and values.
Assessing the vendor's commitment to innovation and their research and development track record for their software solutions.
Examining the vendor’s track record, market presence, and customer feedback.
Analyzing the vendor’s financial stability and longevity in the market.
Risk management involves collecting or generating sufficient information to determine the company's level of risk (see risk categories above) in a structured way. Risk is often one of the more challenging things to measure because understanding uncertainty is not something the human brain is particularly good at (see Klineman, Thinking, Fast and Slow).
These are some of the methods we have seen companies use to gather risk information:
This method consists of holding a meeting or posting a thread asking developers what they’re most concerned about and generating a list of downside risks coupled with an estimation of their likelihood.
This method consists of looking at similar past decisions, performing a post-mortem, and using this as an abstraction as to what risks exist in the current solution.
Using probabilistic models to understand possible outcomes' range and probabilities.
Testing how the software and vendor would perform under extreme conditions or future scenarios.
Maintaining a risk register to track potential risks and their mitigation strategies, along with regular monitoring.
Including contract clauses that protect against specific risks, such as non-performance, data breaches, or sudden cost increases.
Risk management is often a collaborative effort involving IT, legal, finance, and operations teams, each contributing their expertise to identify and mitigate risks over the course of the software selection project.
Combining these sophisticated and traditional techniques ensures a comprehensive evaluation and decision-making process. This approach allows companies to make informed choices that balance technical capabilities, financial viability, vendor reliability, and risk management effectively.
After thorough preparation with a robust requirements document and comprehensive testing and vendor analysis, the software selection project enters a critical phase. This phase requires the organization to navigate the maze of conflicting interests, technical specifications, financial constraints, and strategic implications.
There are many different ways to approach this part of the process. The key challenges are:
Here are some ways organizations approach decision-making. They don’t often use these terms but follow the methods more or less. It is important to note that even if informal, most companies are performing one or more of the following types of analysis:
Below we list the top vendor selection software and tools. However, you should reference our full Vendor Selection Software Guide.
In conclusion, software vendor selection is a necessary process that can have a significant impact on an organization's success, mainly because the risks associated with a bad decision are so high, including:
Selecting the right software solution has obvious benefits, especially if you can do it right the first time. In many ways, running a thorough process for software vendor selection will ensure that:
If you’d like to use the best-in-class vendor selection software to help you get the job done, look no further than Taloflow to get a free tooling report tailored to your exact use case in minutes.
Get a detailed requirements table and filter solutions for your exact use case using our platform.
