Thoughts on Rewrites (Estimation)

Estimating rewrites is hard, and frequently developers got a tendency to underestimate such a task – as, in their mind, they know exactly how the system should work, as they already built it once. This holds only if the rewrite is 1-to-1 rewrite – meaning you are constructing a similar solution from both an architectural, deployment and functional perspective. But if that is the case, why then even bother rewriting?

Moving from an on-premise single solution to a super-scalable cloud based solution requires a complete new architecture and equally important, new deployment scenarios. The latter needs to factor in continuous deployment such that it will be possible to deploy functional updates at a quicker pace than that for on-premise solutions, as well as taking into consideration that updates shouldn’t break customer specific configuration or functionality.

These elements, along with others, spills into the architecture – that needs to be more loosely coupled, utilizing a decoupled messaging strategic allowing to deploy functional instance clusters – by using a service oriented approach, like MSA – Micro Service Architecture, and utilizing patterns like, say, CQRS. For all applications there will always be a certain requirement for customer specific functionality or configuration, which in a super scalable solution shouldn’t be accomplished by code changes, but by providing facilities in the underlying platform for extending the solution by means of metadata, configuration, rules engines or built-in scripting capabilities. For the latter, it is imperative that such a construct can be isolated within its own cluster to avoid scripts to impact the base system.

Another element of rewriting is the functional capabilities of the solution going forward. When faced with the opportunity/challenge of a rewrite – it is equally important to reconsider/rethink the functionality both at the micro-level (features) and at the macro-level (solution). It is an opportunity to solve previously shortcomings as well as focus on new capabilities that may not even exist in current offering.

The above leads to the rule of rewrites: DON’T. If required, rethink and build new, with a sharp focus on the value creation a new solution would bring to the customer.

Focusing on constructing a flexible architecture and internal processes allowing for continuously deployment initially, will pave the way for the ability to provide functionality to customers earlier in the development phase, than can be accomplished in an on-premise model, as you control the update cycles – and are not at the mercy of the customer’s ability or willingness to deploy updates. Moreover, it is crucial to align platform and functional development activities such that visible progress can be made available to the stakeholders, infusing confidence that the project is on track. With regards to the latter, then having tangible deliverables throughout the development phase will increase the success of the project significantly, not only because stakeholders tends to get nervous without visible progress, but equally important you will be able to receive customer feedback on a continuously basis, allowing you to either adjust or enhance features quickly and redeploy for feasibility testing.

The net is, that a transition from a traditional on-premise solution to a cloud based solution – in most cases – will take equal time as it took to develop the solution you are trying to cloud enable. The question then becomes, how to calculate the original effort. It is not that straight forward to calculate the total effort in man month invested in building a solution over a multi-year time frame, but it is a good mental exercise trying to do so – as it will give you a better idea of the total scope. In most cases people get surprised about how long it actually took to build a solution.

The next element of the top-level estimation process is somewhat controversially, as it builds on research into programmer productivity by Boehm conducted in 1995. Boehm found that a programmer in a medium complex system, effectively, produces 900 lines of code per month (LOC/pm). Note that this number is an average over the lifetime of a system. Initially the number is higher; however, as the complexity of the system increases the LOC/pm decreases. The LOC/pm can be used as a sanity check against the total estimated time from above.

The third element of the top-level estimation is to create a team estimate, where the development team themselves gives a high-level estimate.

Having these numbers will allow you to get a rough idea on the total effort required, and some numbers on which you can base your reasoning when presenting the effort and investment required to stakeholders.

Example

To illustrate I will use an examples from two previous projects.

Case 1

The team had estimated a total effort of 324 man month. The estimated historical effort came to 900 man month. The LOC/pm gave 920 man month. In this particularly case the actual time ended up being 870 man month. Here the historical and LOC/pm matched very well, but the team estimate was 37% of the actual.

Case 2

In another project the team estimate was 180 man month. Estimated historical effort came to 800 man month. LOC/pm gave 1300. Here the actual time was 1080 man month. In this situation the team estimate was 16% of the actual.

In both of the above cases, the hardest part was not to convince stakeholders about the effort required, but to get the development teams to accept the initial estimation based on a combination of historical effort and LOC/pm. Personally I find that intriguing as it illustrates that developers are inherently optimistic when it comes to estimation. Not only at the micro-level – implementing specific features – but also at the macro-level – top-level solution estimation.

Will Windows 8 bring MS into the tablet/phone game?

Recent I had a discussion about the contents of the following article: http://gizmodo.com/5839665/windows-8-slate-hands-on-its-fantastic-but-dont-sell-your-ipad, where I labeled a post: “Eventually the tablet OS market will be like the desktop OS market. MS against Apple also on tablets.” which caused an interesting debate. So I thought I would write why I believe this will be so.

I would say that on the tablet market Android isn’t imposing any real threat to anything, maybe apart from themselves. Regardless of ice cream, soft ice or whatever sweet project name Google came up with this time – guess it requires a lot of sweets to swallow Android, it won’t bring them into the tablet market. If you’ve followed stories on non-iPad tablets, then they’re being returned at an alarming rate. There’s nobody delivering Android tablets, not I’ve been able to find, that are keen on informing about number of sold units – normally something you would be very keen on doing if you were selling a lot of units. Should give you a hint to the traction the Android based tablet devices.

One thing that history has taught us is that big corporation becomes very innovative when their core business and primary revenue generator comes under pressure, and feel themselves against the wall. That’s the point in time, they either bet their business (as Apple did) or reinvent themselves (as IBM did). Not sure MSFT is really against the wall, yet. But they definitely feel under pressure, and are probably concerned about their primary revenue generators being Office and Windows. Will they succeed, only time can tell – but it’s definitely naive to count MSFT out of the game on tablets and phones.

On the subject of Android on phones, then there’s not just 1 Android, there’s uncountable number of Android implementations out there. Every phone and tablet vendor using Android got their own version of Android. This fragmentation causes problems with the apps developed for Android, and you can have apps that works on a device from Samsung, but doesn’t work on a device from HTC. People keeps talking about the success of the Android platform and there’s a bunch of charts showing that there’s more Android phones than, but that covers all flavors of Android on all vendors of devices. If you compare the manufactures of phones, then Apple is the single biggest vendor of phones. Followed by Samsung and Nokia. Note that Samsung includes non-smartphone and non-android phones, goes for Nokia as well. So how successful is Android really when it comes to it? And also, how to you measure whether or not something or someone is successful?

Google buying Motorola could backfire on Android, as the current device manufactures will be concerned, that Motorola will get an unfair market advantage when it comes to delivering new versions of Android. There’s already been speculations about Samsung looking at WP7 and/or buying WebOS and/or promoting Bada. I bet you, that the Softies in Redmond and employees at Apple was clapping their hands when Google announced the acquisition. As much as Apple and MSFT (probably) hates each other, they do agree on disliking google. And together they do make a somewhat scary opponent. MSFT is probably already working hard to convince Samsung and HTC that WP7 is a better bet than Android, and by acquiring Motorola, Google just made the argument easier.

There’s been a lot of criticism of MSFT not building a complete new OS for tablets, or at least uses the WP7 OS from the phone. To some extent I do share that concern, as Windows is a huge OS. The interesting thing, though, is that Apple is trying to merge MacOSX and iOS into one OS, that work started with LION. So you could argue, that Apple and MSFT is doing similar things, however, Apple used a convergence strategy – having 2 distinct OSes and converge them, whereas MSFT used a duplicate strategy, duplicate your OS to multiply devices – and slim it. By the end of the day, both Apple and MSFT are trying to do the same thing, having one OS to ease development of apps. For Apple that would allow users to use iOS based apps on Macintosh, allowing Apple to sell more computers. For MSFT it’ll allow users to use the apps they already know on tablets. Same Same, but different starting point and approach.

How will this end? Well, only time can tell. But it’s imperative when we try to project or anticipate the future that we stay non-biased and objective; otherwise, we risk making critical strategic decisions based on personal preferences, which doesn’t always come out successful.

Essence of being a successful Development Manager?

The above question was sent to me the other day from a former colleague and friend. He was preparing a presentation for heads of product management and product development, and not being an engineering manager himself he asked for input on the subject of “Essence of of being a successful Development Manager”

I usually refer to a development manager as an engineering manager, as developing software is much more than simply developing the software itself. When you develop/design software today, you need to take all kind of operational aspects into account (http://bit.ly/fW0OvT).

Technical

Anyway, so what’s required to be a successful engineering manager. Though question. And maybe not. One thing I’ve realized after working in software engineering for more than 20+ years (wow getting old), and have had a number of different managers, is that the single most important and an absolutely unquestionable qualification for any engineering manager is that they have a background in software engineering. That they actually have written code, tried to find a subtle bug in a multi-threaded application, that they’ve been through the process of reading thousands of lines of code to locate a memory leak (not using profilers). Have dealt with double-pointers (Macintosh handlers), cried at night because a data structure wouldn’t align at word-boundary – all that nitty-gritty stuff that makes us all sweat. I don’t purposely disqualify people that haven’t developed in any low-level programming languages like C and Assembler; however, I strongly believe that even if you’re of the automatic GC generation it still worth your while to understand and gain experience with fundamental aspects of the computer, like how memory operates.

Why is this so important, that an engineering manager actually knows how to write software? For same reasons that a lead of a bridge construction team is a bridge construction engineer, how else can he make critical decisions? Actually how can anyone be trusted to make decisions about any professional field, unless they got the t-shirt? How can any non-software engineer provide any assistance, help or guidance to the team unless he or she knows what it’s all about? I don’t know, and based on personal experience, they can’t!

There’s more to having software engineers running development teams than just being able to facilitate the process of developing the software. It’s also about understanding where in the process the team is, how far we are, and more important (sometimes) how much are we’re off course. Only people with engineering background can read another engineer, can understand where they are, and truly appreciate the work being done. And BTW engineers don’t really trust other professions, that’s just a fact. So lot’s of good reasons to why the single most important qualification of any engineering manager is that the manager in fact him- or herself is an engineer.

So a good engineering manager knows his stuff, knows how to write code, know how to design architecture, knows how to make a reasonable data model and knows how to execute. He can ask the right questions and provide the right technical answers and guidelines to his team, and he’s capable of writing code himself. Actually, any first level engineering manager should spend 25% of their time writing code. Way way better for the project than spending 25% of your time making MS Project plans, that doesn’t tell you anything anyway. Being in the code on a daily basis will tell you precisely where the project is and where it’s heading. If you’re too remote from the code you lose oversight! The project starts slipping, you’ll have delays and the team suffers, the company suffers, customers suffers and the engineering manager suffers – because he’s not a “real” engineering manager.

Able to say a resounding NO

This is actually very important qualification as well. We as engineers don’t like to say the “NO – it cannot be done”. Why is this? Personally I find it very hard to say NO. I feel that if I can’t do it, I somehow failed. It’s like a challenge, and engineers/nerds/geeks loves challenges, hence anything/everything is possible. And it probably is; however, not on a schedule.

So what most typically happens is that we say something like: “Well, it’s probably possible if we add additional resources and delay the deadline”. Nothing wrong with that, right? We don’t really say NO, and we take on the challenge. Wrong, it’s dead wrong. Reason being that the receiving party of your kind-of-NO isn’t an engineer, probably someone in the business, and here’s what they pick-up from your kind-of-NO statement: “Well, it’s ..noise.. possible ..more noise…”. See these people only pick up what the want to hear, everything else gets lost in the translation from engineering-lease to sales-lease.

If you’ve come to the conclusion that it’s impossible within the given framework (resources and time) then, to my knowledge, there’s only one approach that works – and that is to spell it out loud and clearly by starting your answer with the NO, and whatever comes after shouldn’t in any form or shape take the power out of the NO. The receiving party will most likely eye-ball you for a while, and then start the actual negotiation on how to accomplish it. But I tell, you’ll never get to that part if you don’t start out saying NO.

Actually, NO is probably the single most important weapon you have as an engineering manager when there’s a risk of feature creep.

Understands Requirements

The third, IMHO, important qualification is the ability to understand what the business is asking for, what’re the requirements, being able to speak with marketing, sales and product management to gather the necessary input needed to be able to understand what’s expected to be developed. Take that knowledge and turn around and turn it into something engineers understand and can build product based on.

This one, actually ties nicely in with the first qualification, as it requires deep technical understanding to discuss functional requirement with the field, and at the same time being able to envision how the product or a feature eventually could end up being coded. Also, allow the engineering manager to get a sense for what’s possible and in what time frame, without always have to disturb the development team asking for help.

#programming #software #management