The role of microservices in Enterprise Architectural planning

Microservices as an architectural construct is gaining a lot of popularity, and the term is slowly but surely finding its place among the lexicon of developers and software architects. The precise definition of ‘microservices’ varies (it is not a formal concept in computer science, as to my knowledge it is not grounded in any formal mathematical abstraction) but the basic concept of a microservice being a reusable lightweight architectural component doing one thing and doing it extremely well is a reasonable baseline.

What do ‘Microservices’ mean to organisations used to understanding their world in terms of Applications? What about all the money some organisations have already spent on Service Oriented Architecture (SOA)? Have all the Services now been supplanted by Microservices? And does all this stuff need to be managed in a traditional command-and-control way, or is it possible they could self-manage?

The answers to these questions are far from reaching a consensus, but here folllows an approach that may help begin to help architects and planners rationalise all these concepts, taken from a pragmatic enterprise perspective.

Note that the definitions below do not focus on technologies so much as scenarios.

Applications

Applications are designed to meet the needs of a specific set of users for a specific set of related tasks or activities. Applications should not be shared in an enterprise sense. This means applications must not expose any re-usable services.

However, application functionality may depend on services exposed by many domains, and applications should use enterprise services to the fullest extent possible. To that extent, applications must focus on orchestrating rather than implementing services and microservices, even though it is generally through the act of building applications that services and microservices are identified and created.

Within a given domain, an Application should make extensive use of the microservices available in that domain.

Applications require a high degree of governance: i.e., for a given user base, the intent to build and maintain an application for a given purpose needs to be explicitly funded – and the domain that that application belongs to must be unambiguous. (But noting that the functionality provided by an application may leverage the capabilities of multiple domains.)

Services

Services represent the exposed interfaces (in a technical sense) of organizational units or functional domains. Services represent how that domain wants information or services to be consumed and/or published with respect to other domains.

Services represent a contract between that domain and other domains, and reflect the commitment the domain has to the efficient operation of the enterprise as a whole. Services may orchestrate other services (cross-domain) or microservices (intra-domain). The relationship between applications and services is asymmetric: if a service has a dependency on an application, then the architecture needs reviewing.

Services require a high degree of governance: i.e., the commitments/obligations that a domain has on other domains (and vice-versa) needs to be formalised and not left to individual projects to work out. It means some form of Service Design methodology, including data governance, is required – most likely in parallel with application development.

Microservices

Within a domain, many microservices may be created to support how that domain builds or deploys functionality. Microservices are optimised for the domain’s performance, and not necessarily for the enterprise, and do not depend upon microservices in other domains.

Applications that are local to a domain may use local microservices directly, but microservices should be invisible to applications or services in other domains.

However, some microservices may be exposed as Services by the domain where there is seen to be value in doing so. This imposes an external obligation on a domain, which are enforced through planning and resourcing.

Microservices require a low degree of governance: the architectural strategy for a domain will define the extent to which microservices form part of the solution architecture for a domain. Microservices developers choose to make their services available and discoverable in ways appropriate to their solution architecture.

In fact, evidence of a successful microservices based approach to application development is an indicator of a healthy technical strategy. Rather than require microservices to be formally planned, the existence of microservices in a given domain should be a hint as to what services could or should be formally exposed by the domain to support cross-divisional goals.

It is quite unlikely that meaningful (resilient, robust, scaleable) Services can be identified and deployed without an underlying microservices strategy. Therefore, the role of a domain architect should be to enable a culture of microservice development, knowing that such a culture will directly support strategic enterprise goals.

Domains

A domain consists of a collection of applications, services and microservices, underpinning a business capability. It is an architectural partition of the organisation, which can be reflected in the firm’s organisational structure. It encapsulates processes, data and technology.

In particular, domains represent business capabilities and are clusters of business and technology – or, equivalently, business function and technology.

The organisational structure as it relates to domains must reflect the strategy of the firm; without this, any attempt at developing a technology strategy will likely fail to achieve the intended outcome.

So, the organisational structure is key, as without appropriate accountability and a forward-looking vision, the above concepts will not usefully survive ad-hoc organisational changes.

While organisational hierarchies may change, what a business actually does will not change very often (except as part of major transformation efforts). And while different individuals may take different responsibilities for different domains (or sets of domains), if an organisation change materially impacts the architecture, then architectural boundaries and roadmaps need to change accordingly so that they re-align with the new organisational goals.

Summary

Partitioning an organisation’s capabilities is a necessary prerequisite to developing a technology strategy. A technology strategy can be expressed in terms of domains, applications, services and (the presence of) microservices.

The key is to not be overly prescriptive, and to allow innovation and creativity within and across domains, but providing sufficient constraints to encourage innovation while simultaneously keeping a lid on complexity. The presence of microservices and a viable microservice strategy within a domain can be used as a good proxy for enterprise technology strategy alignment.

The role of microservices in Enterprise Architectural planning

Mining the gold in the legacy application hills

One recurring challenge that CIOs & CTOs have is talent retention: in the face of continuing and accelerating innovation in technology, and continued lowering of barriers to entry for those who want to get involved in technology, finding and retaining people who are professionally satisfied with looking after so-called ‘legacy’ technology is increasingly difficult.

The most visible impact is inexorable rising cost of change, which CIOs usually deal with in the traditional way: by outsourcing or offshoring the work to locations where similar talent can be found, but cheaper.

But this tends not to solve the problem: rather, the ‘hard’ bits of the legacy applications are ignored in favour of writing new code that the current maintainers can understand. Fear, uncertainty and doubt then creeps in when any of the ‘legacy’ software needs to change. The end result is increased complexity and no sustainable reduction in cost of change.

Only a few years ago, mainframes were considered to be legacy applications. Today, applications written 7+ years ago are the new legacy applications in most organisations, but ostensibly any application whose original architects and maintainers have moved on are now ‘legacy’.

With all the focus on new, exciting technologies, what technical strategy should be adopted to maximise valuable existing functionality, retain and engage development talent, and allow legacy architectures to fit seamlessly and sustainably into new, innovative (cloud/mobile-based) architectures?

Legacy applications have a significant advantage over new applications: they reflect what is effective and useful from a business perspective (even if their implementations do not reflect what is efficient). With the benefit of hindsight, it is possible to look at these applications through a modular lens, and define architectural boundaries of change that can be enforced through technology.

Such an approach can then lead to potential opportunities for decomposing complex monolithic applications, optimising infrastructure usage, deploying components onto anti-fragile infrastructure (elastic fabrics), and finally identifying potentially reusable business services and components that can form building blocks for new applications using innovative technologies.

In particular, it allows complexity to be managed, useful working software to be maximised, and talent to be retained, through providing a route to working with new, in-demand technologies.

In defining a technical strategy supporting a enterprise-wide digital strategy, this is a feasible, cost manageable way to evolve the large portfolio of legacy applications that many large organisations have, both from the bottom up, as well as the top down.

The broad approach is well captured in this presentation here from a global financial services firm, and the overall architectural approach is well described in this book, from 2010.

In the last few years, there have been significant advances in certain enabling technologies (around modularity, APIs, IaaS/PaaS, containers, HTML, etc) and increased focus on enabling architectural disciplines (like business, data, and solution architecture) that are making such a strategy feasible as part of a medium/long-term plan to address IT complexity.

In short, if firms do not have some strategy for dealing with their legacy technology, they will be unable to compete in the increasingly digital economy that is impacting almost every industry out there.

Mining the gold in the legacy application hills