If you have ever experienced a project going haywire with a dev shop, you might have wondered whether you should have known better. Were there tell-tale signs that you missed? Were there red flags that you ignored?
Finding a competent dev shop to build your product is hard because the stakes are so high: your startup’s future is on the line, you will be spending substantial sums of cash, and you just don't know if the dev shop will deliver. You do your due diligence which means reading their self-selected case studies, checking out online customer reviews, and looking through their advertised portfolio. But even several five star reviews don’t guarantee your satisfaction. Every experience is unique. We all have radically different standards.
The best option is to sample the work first. Not just hearing about what they have done with similar clients or seeing the polished product from past work. You should experience their work firsthand but on a smaller but still valuable project. You should hire them for a feasibility assessment.
The feasibility risk is one of Marty Cagan’s four major risks in product development. It’s the one risk that a dev shop should tackle outright, while also lending support to the usability risk. Feasibility is the only risk that requires deep technical expertise, as it tells you what can be created, how much it will cost, and how long it will take.
Customers often contact a dev shop to hand off feature specs and expect for engineering to bring the product to life, but a dev shop can’t realistically assess cost, scope, or time without investing a lot of effort into discovery. While a dev shop might quote you a rough estimate in order to win your business, without a deep dive into the technical implications of the product idea, it’s all guesswork.
Therefore the dev shop has three options:
Spend a ton of unbilled time on researching the customer’s needs.
The dev shop is either desperate if they’re willing to do anything to get some work, disingenuous if they're going to pretend that they will put effort into the review, or bad at business if they don’t value their own time and skills enough to actually get paid. If it's the last case, they might be decent engineers, but you might also suffer from headaches of dealing with a poorly run business that doesn’t manage their time or resources well.
Do a cursory review of the documentation and pull an estimate out of thin air.
They’re dishonest and it’s a bad sign.
Tell the customer the hourly fee (i.e. time & materials) and then offer to get to work.
They don’t want to BS you. They can’t give you an estimate, let alone a fixed price per deliverable, without spending significant time assessing the situation and they won’t do that for free. With an hourly arrangement, you’re not locked into the engagement and can end the relationship if things aren’t going well.
All three options, however, are focused on getting straight into development and none of them give you a preview of what working with the dev shop will be like. Some customers prefer to pay a fixed price per deliverable. Most customers at least want an estimate even if they’re going to pay per hour. But neither is possible without the dev shop doing research and discovery before development.
In his book Inspired, Marty Cagan goes through several techniques for assessing feasibility of a product or feature idea. The whole basis of Cagan’s philosophy is to get the product manager, product designer, and lead engineer together from the outset as they tackle the risks of value, usability, feasibility, and business viability. Cagan believes that the top down approach of forming a product roadmap and passing along specs to the engineering team for the build is a recipe for disaster. You want engineering sitting at the table way before any commitments are made.
If you want to work with a dev shop, they should have the technical understanding and experience to fill that role. They need to be able to translate your vision and desired outcomes into a scope, timeline, and resources. They should be able to explain to you the trade-offs of any technical decision and help you focus on prioritizing.
So what can the feasibility assessment reveal about the dev shop?
There are expensive US-based dev shops, cheap offshore vendors, and mid-priced nearshore agencies. You can find solid devs and good tech talent all over the world and cheap doesn’t always mean bad, just as expensive doesn’t necessarily mean great. However, any good developer, designer, or product manager needs to have skills in critical thinking, problem solving, and creativity.
A lot of fly-by-night dev shops turn their profit by hiring junior developers or just bad developers who slap together spaghetti code from a few snippets on Stack Overflow, producing a house of cards that will inevitably come crashing down from bugs and dependencies. When you’re aware of technical debt and opt for it as a trade-off, it’s one thing, but no good comes out of writing bad code out of ignorance.
Whether junior or mid or senior level, you want a developer who thinks before writing code and doesn’t blindly copy other people’s solutions. A feasibility assessment requires the same analytical qualities that any decent developer should have whether debugging, debating solutions, or reading API documentation. If the dev shop can’t lead you through determining the technical feasibility risks of your product idea, be wary of the technical talent that they claim to have.
Some dev shops don’t offer feasibility assessments which should be a warning sign against working with them. A feasibility assessment is in the best interest of both the startup and the dev shop because it is a discovery process that helps get everyone on the same page. Other than not having the time, there really isn’t a reason a dev shop should not be interested in doing a feasibility assessment.
Another issue is particularly relevant with offshore dev shops. Running feasibility assessments involves engineers taking the lead in doing research and asking questions. Cultural nuances can come into play here and communication can suffer. Also, engineers are communicating with you in their non-native language. Due to cultural differences and a language barrier, the dev shop might not direct the conversation and you will pay the price for it. You're better off realizing this during the smaller, lower-cost feasibility assessment than during the larger, more costly development process.
You might balk at the idea of spending money on a feasibility assessment. You might be inclined to head immediately into development with the dev shop in order to save money and time. But the cost of a feasibility assessment is a drop in the bucket compared to software development costs and it will arm you with valuable info about your product idea and the dev shop itself.
