The hidden costs of PaaS & microservice engineering innovation

[tl;dr The leap from monolithic application development into the world of PaaS and microservices highlights the need for consistent collaboration, disciplined development and a strong vision in order to ensure sustainable business value.]

The pace of innovation in the PaaS and microservice space is increasing rapidly. This, coupled with increasing pressure on ‘traditional’ organisations to deliver more value more quickly from IT investments, is causing a flurry of interest in PaaS enabling technologies such as Cloud Foundry (favoured by the likes of IBM and Pivotal), OpenShift (favoured by RedHat), Azure (Microsoft), Heroku (SalesForce), AWS, Google Application Engine, etc.

A key characteristic of all these PaaS solutions is that they are ‘devops’ enabled – i.e., it is possible to automate both code and infrastructure deployment, enabling the way to have highly automated operational processes for applications built on these platforms.

For large organisations, or organisations that prefer to control their infrastructure (because of, for example, regulatory constraints), PaaS solutions that can be run in a private datacenter rather than the public cloud are preferable, as this a future option to deploy to external clouds if needed/appropriate.

These PaaS environments are feature-rich and aim to provide a lot of the building blocks needed to build enterprise applications. But other framework initiatives, such as Spring Boot, DropWizard and Vert.X aim to make it easier to build PaaS-based applications.

Combined, all of these promise to provide a dramatic increase in developer productivity: the marginal cost of developing, deploying and operating a complete application will drop significantly.

Due to the low capital investment required to build new applications, it becomes ever more feasible to move from a heavy-weight, planning intensive approach to IT investment to a more agile approach where a complete application can be built, iterated and validated (or not) in the time it takes to create a traditional requirements document.

However, this also has massive implications, as – left unchecked – the drift towards entropy will increase over time, and organisations could be severely challenged to effectively manage and generate value from the sheer number of applications and services that can be created on such platforms. So an eye on managing complexity should be in place from the very beginning.

Many of the above platforms aim to make it as easy as possible for developers to get going quickly: this is a laudable goal, and if more of the complexity can be pushed into the PaaS, then that can only be good. The consequence of this approach is that developers have less control over the evolution of key aspects of the PaaS, and this could cause unexpected issues as PaaS upgrades conflict with application lifecycles, etc. In essence, it could be quite difficult to isolate applications from some PaaS changes. How these frameworks help developers cope with such changes is something to closely monitor, as these platforms are not yet mature enough to have gone through a major upgrade with a significant number of deployed applications.

The relative benefit/complexity trade-off between established microservice frameworks such as OSGi and easier to use solutions such as described above needs to be tested in practice. Specifically, OSGi’s more robust dependency model may prove more useful in enterprise environments than environments which have a ‘move fast and break things’ approach to application development, especially if OSGi-based PaaS solutions such as JBoss Fuse on OpenShift and Paremus ServiceFabric gain more popular use.

So: all well and good from the technology side. But even if the pros and cons of the different engineering approaches are evaluated and a perfect PaaS solution emerges, that doesn’t mean Microservice Nirvana can be achieved.

A recent article on the challenges of building successful micro-service applications, coupled with a presentation by Lisa van Gelder at a recent Agile meetup in New York City, has emphasised that even given the right enabling technologies, deploying microservices is a major challenge – but if done right, the rewards are well worth it.

Specifically, there are a number of factors that impact the success of a large scale or enterprise microservice based strategy, including but not limited to:

  • Shared ownership of services
  • Setting cross-team goals
  • Performing scrum of scrums
  • Identifying swim lanes – isolating content failure & eventually consistent data
  • Provision of Circuit breakers & Timeouts (anti-fragile)
  • Service discoverability & clear ownership
  • Testing against stubs; customer driven contracts
  • Running fake transactions in production
  • SLOs and priorities
  • Shared understanding of what happens when something goes wrong
  • Focus on Mean time to repair (recover) rather than mean-time-to-failure
  • Use of common interfaces: deployment, health check, logging, monitoring
  • Tracing a users journey through the application
  • Collecting logs
  • Providing monitoring dashboards
  • Standardising common metric names

Some of these can be technically provided by the chosen PaaS, but a lot is based around the best practices consistently applied within and across development teams. In fact, it is quite hard to capture these key success factors in traditional architectural views – something that needs to be considered when architecting large-scale microservice solutions.

In summary, the leap from monolithic application development into the world of PaaS and microservices highlights the need for consistent collaboration, disciplined development and a strong vision in order to ensure sustainable business value.
Advertisements
The hidden costs of PaaS & microservice engineering innovation

Strategic Theme # 3/5: Continuous Delivery/DevOps

[tl;dr This post discusses why state-of-the-art technologies and techniques which brings together agile software delivery with operational considerations are strategically key to business success.]

Continuous Delivery is a subject that makes many operations folks break out in a cold sweat..the concept of being able to deploy change as soon as it is ready rather than on fixed deployment schedules is surely a recipe for disaster.

In most cases, it is: the act of taking a traditional monolithic application build, and moving it from test to UAT to production is often a very painful, labour intensive process, with many gates and controls. Because applications typically aren’t build with an operational perspective in mind, most of the creative coding goes into the business functionality, and not into the so-called ‘non-functional’ requirements such as ‘operability’.

The net result is that it’s a lot of work to deploy change, and hence it has to be planned well in advance.

The solution is automation: automation of building, testing, configuration and deployment.

Such automation requires significant investment, and so is generally only worth the effort when a significant number of teams can benefit from the automation. This in turn depends on standards: there is little point in building a highly automated toolchain for Java when most folks want to do C++, for example.

But the investment pays off, in the ability to respond rapidly to business needs, to quickly make change and be confident that nothing critical breaks, and to provide operations folks with key tools to manage and configure running applications without involving the core development teams.

Continuous delivery may only be an aspiration: it may not be necessary to deploy several times a day, even if you can. But the act of preparing for it makes the whole process much more efficient and massively reduces change-related risk.

The seminal book on this topic, ‘Continuous Delivery: Reliable Software Releases Through Build, Test and Deployment Automation’ by Jez Humble and David Farley covers all the topics very well, and dovetail nicely with related strategic themes of Enterprise Modularity and Lean Enterprise.

The question of Standards always raises its head: in my mind, Standards are in basic conflict with Innovation..although sensible standards can provide the constraints that allow genuine innovation to thrive (as, in general, innovation is a supply-side response to one or more constraints in meeting demand).

When it comes to solution standards (i.e., specific languages, libraries, tools, infrastructure, etc), one of the key drivers for identifying standards should be standards that allow automated toolchains to be built, according to Continuous Delivery principles. This generally means specific languages, specific code repositories, specific package dependency management, specific build repositories and tools, specific test frameworks, configuration engines, deployment tools, etc, etc.

In short, to achieve continuous delivery, firm technical standards need to be in place for the most frequently used architectures. For projects deviating from the standards, it must be understood that additional investment will be required to enable continuous delivery.

As each domain will have its own unique non-functional requirements, the level of toolchain automation appropriate needs to be agreed as part of the domain’s target architecture – and with it, the relevant technical standards the domain wishes to adopt. Short-cuts to deliver change which do not use the automated toolchain process needs to go through an exception process: technical debt in toolchain automation is usually not a sustainable option.

Strategic Theme # 3/5: Continuous Delivery/DevOps

DevOps & the role of Architecture in Achieving Business Success

[TL;DR] Creating a positive re-inforcing effect between enterprise architecture and DevOps will provide firms with a future digital competitive advantage.

I finished reading an interesting book recently:

The Phoenix Project: A Novel About IT, DevOps and Helping Your Business Win (Gene Kim, Kevin Behr, & George Spafford)

Apart from achieving the improbable outcome of turning a story about day-to-day IT into a page-turner thriller (relatively speaking..), it identified that many challenges in IT are of its own making – irrespective of the underlying technology or architecture. Successfully applying lean techniques, such as Kanban, can often dramatically improve IT throughput and productivity, and should be the first course of action to address a perceived lack of resources/capacity in the IT organisation, ahead of looking at tools or architecture.

This approach is likely to be particularly effective in organisations in which IT teams seem to be constantly reacting to unplanned change-related events.

As Kanban techniques are applied, constraints are better understood, and the benefit of specific enhancements to tools and architecture should become more evident to everyone. This forms part of the feedback loop into the software design/development process.

This whole process can be supported by improving the process by which demand into IT (and specific teams within IT) is identified and prioritised: the feedback loop from DevOps continuous improvement activities form part of the product/architecture roadmap which is shared with the business stakeholders.

The application of these techniques dovetails very nicely with advancements in how technology can be commissioned today: specifically, infrastructure-as-a-service (virtualised servers); elastic anti-fragile cloud environments, the development of specialised DevOps tools such as Puppet, Ansible, Chef, etc; the rise of lightweight VMs/containers such as Docker and Mesos, microservices as a unit of deployment, automated testing tools, open modular architecture standards such as OSGi, orchestration technologies such as Resource Oriented Computing, sophisticated monitoring tools using big data and machine learning techniques such as Splunk, etc, etc.

This means there are many paths to improving DevOps productivity. This presents its own technical challenges, but is mostly down to finding the right talent and deploying them productively.

An equal challenge relates to enteprise architecture and scaled agile planning and delivery techniques, which serves to ensure that the right demand gets to the right teams, and that the scope of each team’s work does not grow beyond an architecturally sustainable boundary – as well as ensuring that teams are conscious of and directly support domains that touch on their boundary.

This implies more sophisticated planning, and a conscious effort to continually be prepared to refactor teams and architectures in order to maintain an architecturally agile environment over time.

The identification and definition of appropriate boundaries must start at the business level (for delivering IT value), then to the programme/project level (which are the agents of strategic change), and finally to the enterprise level (for managing complexity). There is an art to identifying the right boundaries (appropriate for a given context), but the Roger Sessions approach to managing complexity (as described in this book) describes a sensible approach.

The establishment of boundaries (domains) allows information architecture standards and processes to be readily mapped  to systems and processes, and to have these formalised through flows exposed by those domains. This in the end makes data standards compliance much easier, as the (exposed) implementation architecture reflects the desired information architecture.

How does this help the DevOps agenda? In general, Agile and DevOps methods work best when the business context is understood, and the scope of the team(s) work is understood in the context of the whole business, other projects/programs, and for the organisation as a whole. This allows the team to innovate freely within that context, and to know where to go if there is any doubt.

DevOps is principally about people collaborating, and (enterprise) architecture is in the end the same. Success in DevOps can only scale to provide firm-wide benefits with a practical approach to enterprise architecture that is sensitive to people’s needs, and which acts as a positive reinforcement of DevOps activities.

In the end, complexity management through enterprise architecture and change agility through dev-ops go hand-in-hand. However, most folks haven’t figured out exactly how this works in practice, just yet…for now, these disciplines remain unconnected islands of activity, but firms that connect them successfully stand to benefit enormously.

DevOps & the role of Architecture in Achieving Business Success

Digital Evolution Architecture

[tl;dr The Digital Evolution Architect drives incremental technology portfolio investment towards achievable digital objectives.]

Enterprise architecture (EA) has been around for quite a few years. It has mostly been discovered by firms which recognised the rising cost of technology and the need to optimise their technology footprint. They recognised that unfettered technology innovation across departments and teams may serve individual fiefdoms in the short term, but does the whole firm no good in the medium and long term.

But those were in the days when (arguably) the biggest competitive concern of organisations was their peers: as long as you did technology better, and more cost effectively, than your peers, then you were golden. So the smarter organisations improved their investment management processes through the introduction of enterprise architecture. Portfolio construction and investment management was not only based on what the firm wanted to do, but how it did it.

All well and good. For those industries that managed to get by so far without enterprise architecture, the drivers for change today are much more insidious. It is no longer your peers that are challenging your status quo: it is smaller, nimbler businesses; it is more demanding user expectations; it is enterprise-grade technology at pay-as-you-go prices; it is digital business processes that massively compress the time it takes to transact; it is regulators that want to know you are in control of those same processes; it is data as a strategic asset, to be protected and used as such.

These digital drivers require firms to adopt a formal digital agenda. This in turn will require firms to rethink how they manage their technology investments, and how they drive those investments through to successful execution.

One thing is clear: firms cannot go digital overnight. Firms have built up substantial value over the last 20 years in their post-mainframe technology portfolios. They support tried-and-tested processes. Now those processes are under stress, and it is not clear which (or any) of these systems (or firms) will have a place in the highly digital environment we can expect to have in the next 5-10 years.

Firms that recognise the challenges have multiple efforts underway within their IT function – including, but not limited to:

  • Reducing the cost of infrastructure through virtualisation and cloud adoption
  • Consolidation & modernisation of application portfolios
  • Investment in mobile development capabilities
  • Investment in ‘big data’ initiatives
  • Increased focus on security management
  • Improved SDLC and operational processes (DevOps)
  • etc

The problem is, all these initiatives tend to take place in their own silos, and end up being disconnected from the ‘big picture’ – if, that is, there is in fact a ‘big picture’ in the first place. Bringing all these disparate efforts together into a cohesive set of capabilities that allow a firm to thrive in a highly digital environment is non-trivial, but must be an explicit goal of any firm that has established a formal digital agenda.

To do this requires the bringing together of some well-established skills and disciplines that most firms do not have, or are still developing capabilities in:

  • strategy development,
  • investment governance,
  • enterprise architecture,
  • (scaled) agile development, and
  • ‘true’ service-oriented architecture.

(‘True’ service-oriented architecture means designing reusable services from the ground-up, rather than just putting a service interface on top of legacy, monolithic, non-cloud architectures. ‘Micro services’ may be an acceptable term for this form of SOA.)

It also requires a recognition that a multi-tiered IT organisation is a necessity – i.e., that the traditional IT organisation cannot retain control over the entire technology spend of a firm, but instead should focus on those capabilities of cross-business significance (beyond obvious functions like HR, Finance, etc), and enable business (revenue)-aligned IT folks to build on top of these capabilities.

A digital architecture cannot be built over-night: the key skill is in understanding what the business needs to do now (i.e., in the timeframe of the next budgeting period), and in continually evolving the architecture to where it needs to be in order to be ‘digital’ through the projects that are commissioned and executed, even as specific business needs will be constantly changing and adapting.

This means that strategies for cloud adoption, application portfolio consolidation and modernisation, mobile app development, big data, security, ‘devops’ etc all need to be implemented incrementally as part of this over-arching digital evolution approach.

The digital evolution architect (DEA) is the role or function that supports the transition, putting legacy applications on a clear evolutionary path, ensuring that progress does not leave its own legacy, and that the overall cost of change is on a fixed downward trend, even in the face of continuous innovation in technology. It is the role described in the ‘Opportunities and Solutions” part of the TOGAF method for enterprise architecture, albeit with a very explicit digital mandate. As such, it requires close collaboration with folks involved in business & data architecture, solution (information systems) architecture and technology architecture. But the DEA is primarily concerned about evolution towards coherent goals, not about deciding what the optimal target state is, and hence cares as much about portfolio management as about architecture.

For larger firms, the enterprise architecture function must identify which DEA’s are needed where, and how they should collaborate. But DEA’s should be accountable to the head of Digital Transformation, if a firm has one. (If a firm doesn’t have or plan to have such a function, then don’t expect it to be competitive or even around in 10 years, regardless of how big it is today.)

This blog will elaborate more on the skills and capabilities of the digital evolution architecture concept over the coming weeks.

 

 

 

 

 

Digital Evolution Architecture

The long journey for legacy application evolution

The CTO of Sungard recently authored this article on Sungard’s technical strategy for its large and diverse portfolio of financial applications:

http://www.bankingtech.com/247292/how-to-build-a-better-sungard/

The technical strategy is shockingly simple on the surface:

  • All products are to provide hosted solutions, and
  • All new products must use HTML5 technology

This (one assumes) reflects the changing nature of how Sungard would like to support its clients – i.e., moving from bespoke shrink-wrapped applications deployed and run in client environments, to providing hosted services that are more accessible and cheaper to deploy and operate.

Executing this strategy will have, I believe, some interesting side effects, as it is quite difficult to meet those to primary objectives without addressing a number of others along the way. For example, the development of key capabilities such as:

  • Elasticity
  • Dev-Ops
  • Mobile
  • Shared Services
  • Messaging/Integration

Elasticity

Simply put, hosted services require elasticity if cost effectiveness is to be achieved. At the very least, the capability to quickly add server capacity to a hosted service is needed, but – especially when multiple hosted services are offered – the ability to reallocate capacity to different services at different times is especially key.

For legacy applications, this can be a challenge to do: most legacy applications are designed to work with a known (ahead of time) configuration of servers, and significant configuration and/or coding work is needed when this changes. There is also a reliance on underlying platforms (e.g., application servers) to themselves be able to move to a truly elastic environment. In the end, some product architectures may be able to do no more than allow products to host a replica of the client’s installation on their data centres, shifting the operational cost from the client to the provider. I suspect that it won’t be long before these products become commercially unviable.

Dev-Ops

Dev-ops is a must-have in a hosted environment – i.e., effective, efficient processes that support robust change from programmer check-in to operational deployment. All SaaS-based firms understand the value of this, and while it’s been talked about a lot in ‘traditional’ organisations, the cultural and organisational changes needed to make it happen are often lagging.

For traditional software vendors in particular, this is a new skill: why worry about dev-ops when all you need to do is publish a new version of your software for your client to configure, test and deploy – probably no more than on a quarterly basis, or even much longer. So a whole new set of capabilities need to be built-up to support a strategy like this.

Mobile

HTML5 is ready-made for mobile. By adopting HTML5 as a standard, application technology stacks will be mobile-ready, across the range of devices, enabling the creation of many products optimised for specific clients or client user groups.

In general, it should not be necessary to develop applications specifically for a particular mobile platform: the kinds of functionality that need to be provided will (in the case of financial services) likely be operational in nature – i.e., workflows or dashboards, etc.

Applications specifically designed to support trading activities may need to be custom-built, but even these should be easier to built on a technology stack that supports HTML5 vs more traditional 2-tier fat client & database architectures.

Traditional 3-tier architectures are obviously easier to migrate to HTML5, but in a large portfolio of (legacy) applications, these may be in a relative minority.

Shared Services

In order to build many more innovative products, new products need to build upon the capabilities of older or existing products – but without creating ‘drag’ (i.e., software complexity) that can slow down and eventually kill the ability to innovate.

The way to do this is with shared services, which all teams recognise and evolve, and who build their products on top of their shared capabilities. Shared services mean better data governance and architectural oversight, to ensure the most appropriate services are identified, and to ensure the design of good APIs and reusable software. Shared services pave the way for applications to consolidate functionality at a portfolio level.

Messaging/Integration

On the face of it, the Sungard strategy doesn’t say anything about messaging or integration: after all, HTML5 is a client GUI technology, and hosted services are (in principle) primarily about infrastructure.

And one could certainly meet those stated objectives without addressing messaging/integration. However, once you start to have hosted services, you can start to interact with clients from a business development perspective on a ‘whole capability’ basis – i.e., offer solutions based on the range of hosted services offered. This will require a degree of interoperability between services predicated primarily on message passing of some sort.

On top of that, with hosted services, connection points into client environments need to be more formally managed: it is untenable (from a cost/efficiency perspective) to have custom interface points for every client, so some standards will be needed.

Messaging/Integration is closely related to shared services (they both are concerned with APIs and message content etc), but are more loosely coupled, as the client should not (to the fullest extent possible) know or care about the technical implementation of the hosted services, and instead focus on message characteristics and behaviour as it relates to their internal platforms.

Conclusion

The two objectives set by Sungard are a reflection of what is both meaningful and achievable for a firm with a portfolio of its size and complexity. Products that choose to adopt the directives to the letter will eventually find they have no strategic future.

Product owners, on the other hand, that understand where the CTO is really trying to get to, will thrive and survive in an environment where legacy financial applications – feature-rich as they are – are at risk of being outed by new internet-based firms that are able to leverage the best of the latest technologies and practices to scale and take market share very quickly from traditional vendors.

Portfolio owners on the buy-side (i.e., financial service firms) should take note. A similar simply stated strategy may be all that is needed to get the ball rolling.

The long journey for legacy application evolution