Achieving Agile at Scale

[tl;dr Scaling agile at the enterprise level will need rethinking how portfolio management and enterprise architecture are done to ensure success.]

Agility,as a concept, is gaining increasing attention within large organisations. The idea that business functions – and in particular IT – can respond quickly and iteratively to business needs is an appealing one.

The reasons why agility is getting attention are easy to spot: larger firms are getting more and more obviously unagile – i.e., the ability of business functions to respond to business needs in a timely and sustainable manner is getting progressively worse, even as a rapidly evolving competitive and technology-led commercial environment is demanding more agility.

Couple that with the heavy cost of failing to meet ever increasing regulatory compliance obligations, and ‘agile’ seems a very good idea indeed.

Agile is a great idea, but when implemented at scale (in large enterprise organisations), it can actually reduce enterprise agility, rather than increase it, unless great care is taken.

This is partly because Agile’s origins come from developing web applications: in these scenarios, there is usually a clear customer, a clear goal (to the extent that the team exists in the first place), and relatively tight timelines that favour short or non-existent analysis/design phases. Agile is perfect for these scenarios.

Let’s call this scenario ‘local agile’. It is quite easy to see a situation where every team, in response to the question, ‘are you doing agile?’, for teams to say ‘Yes, we do!’. So if every team is doing ‘local agile’, does that mean your organisation is now ‘agile’?

The answer is No. Getting every team to adopt agile practices is a necessary but insufficient step towards achieving enterprise agility. In particular, two key factors needs to be addressed before true a firm can be said to be ‘agile’ at the enterprise level. These are:

  1. The process by which teams are created and funded, and
  2. Enterprise awareness

Creating & Funding (Agile) Teams

Historically, teams are usually created as a result of projects being initiated: the project passes investment justification criteria, the project is initiated and a team is put in place, led by the project manager. Also, this process was owned entirely by the IT organisation, irrespective of which other organisations were stakeholders in the project.

At this point, IT’s main consideration is, will the project be delivered on time and on budget? The business sponsor’s main consideration is, will it give us what we need when we need it? And the enterprise’s consideration (which is often ignored) is who is accountable for ensuring that the IT implementation delivers value to the enterprise. (In this sense, the ‘enterprise’ could be either a major business line with full P&L responsibility for all activities performed in support of their business, or the whole organisation, including shared enterprise functions).

Delivering ‘value’ is principally about ensuring that  on-going or operational processes, roles and responsibilities are adjusted to maximise the benefits of a new technology implementation – which could include organisational change, marketing, customer engagement, etc.

However, delivering ‘value’ is not always correlated to one IT implementation; value can be derived from leveraging multiple IT capabilities in concert. Given the complexity of large organisations, it is often neither desirable or feasible to have a single IT partner be responsible for all the IT elements that collectively deliver business value.

On this basis, it is evident that how businesses plan and structure their portfolio of IT investments needs to change dramatically. In particular,

  1. The business value agenda is outcome focused and explicit about which IT capabilities are required to enable it, and
  2. IT investment is focused around the capability investment lifecycle that IT is responsible for stewarding.

In particular ‘capabilities’ (or IT products or services) have a lifecycle: this affects the investment and expectations around those capabilities. And some capabilities need to be more ‘agile’ than others – some must be agile to be useful, whereas for others, stability may be the over-riding priority, and therefore their lack of agility must be made explicit – so agile teams can plan around that.

Enterprise Awareness

‘Locally’ agile teams are a step in the right direction – particularly if the business stakeholders all agree they are seeing the value from that agility. But often this comes at the expense of enterprise awareness. In short, agility in the strict business sense can often only deliver results by ignoring some stakeholders interests. So ‘locally’ agile teams may feel they must minimise their interactions with other teams – particularly if those teams are not themselves agile.

If we assume that teams have been created through a process as described in the previous section, it becomes more obvious where the team sits in relation to its obligations to other teams. Teams can then make appropriate compromises to their architecture, planning and agile SDLC to allow for those obligations.

If the team was created through ‘traditional’ planning processes, then it becomes a lot harder to figure out what ‘enterprise awareness’ is appropriate (except perhaps or IT-imposed standards or gates, which only contributes indirectly to business value).

Most public agile success stories describe very well how they achieved success up to  – but not including – the point at which architecture becomes an issue. Architecture, in this sense, refers to either parts of the solution architecture which can no longer be delivered via one or two members of an agile team, or those parts of the business value chain that cannot be entirely delivered via the agile team on its own.

However, there are success stores (e.g., Spotify) that show how ‘enterprise awareness’ can be achieved without limiting agility. For many organisations, transitioning from existing organisation structures to new ‘agile-ready’ structures will be a major challenge, and far harder than simply having teams ‘adopt agile’.

Conclusions

With the increased attention on Agile, there is fortunately increased attention on scaling agile. Methodologies like Disciplined Agile Development (DaD) and LargE Scale Scrum (LeSS), coupled with portfolio concepts like Scaled Agile Framework (SAFe) propose ways in which Agile can scale beyond the team and up to enterprise level, without losing the key benefits of the agile approach.

All scaled agile methodologies call for changes in how Portfolio Management and Enterprise Architecture are typically done within an organisation, as doing these activities right are key to the success of adopting Agile at scale.

Advertisements
Achieving Agile at Scale

Digital Evolution Architecture

[tl;dr The Digital Evolution Architect drives incremental technology portfolio investment towards achievable digital objectives.]

Enterprise architecture (EA) has been around for quite a few years. It has mostly been discovered by firms which recognised the rising cost of technology and the need to optimise their technology footprint. They recognised that unfettered technology innovation across departments and teams may serve individual fiefdoms in the short term, but does the whole firm no good in the medium and long term.

But those were in the days when (arguably) the biggest competitive concern of organisations was their peers: as long as you did technology better, and more cost effectively, than your peers, then you were golden. So the smarter organisations improved their investment management processes through the introduction of enterprise architecture. Portfolio construction and investment management was not only based on what the firm wanted to do, but how it did it.

All well and good. For those industries that managed to get by so far without enterprise architecture, the drivers for change today are much more insidious. It is no longer your peers that are challenging your status quo: it is smaller, nimbler businesses; it is more demanding user expectations; it is enterprise-grade technology at pay-as-you-go prices; it is digital business processes that massively compress the time it takes to transact; it is regulators that want to know you are in control of those same processes; it is data as a strategic asset, to be protected and used as such.

These digital drivers require firms to adopt a formal digital agenda. This in turn will require firms to rethink how they manage their technology investments, and how they drive those investments through to successful execution.

One thing is clear: firms cannot go digital overnight. Firms have built up substantial value over the last 20 years in their post-mainframe technology portfolios. They support tried-and-tested processes. Now those processes are under stress, and it is not clear which (or any) of these systems (or firms) will have a place in the highly digital environment we can expect to have in the next 5-10 years.

Firms that recognise the challenges have multiple efforts underway within their IT function – including, but not limited to:

  • Reducing the cost of infrastructure through virtualisation and cloud adoption
  • Consolidation & modernisation of application portfolios
  • Investment in mobile development capabilities
  • Investment in ‘big data’ initiatives
  • Increased focus on security management
  • Improved SDLC and operational processes (DevOps)
  • etc

The problem is, all these initiatives tend to take place in their own silos, and end up being disconnected from the ‘big picture’ – if, that is, there is in fact a ‘big picture’ in the first place. Bringing all these disparate efforts together into a cohesive set of capabilities that allow a firm to thrive in a highly digital environment is non-trivial, but must be an explicit goal of any firm that has established a formal digital agenda.

To do this requires the bringing together of some well-established skills and disciplines that most firms do not have, or are still developing capabilities in:

  • strategy development,
  • investment governance,
  • enterprise architecture,
  • (scaled) agile development, and
  • ‘true’ service-oriented architecture.

(‘True’ service-oriented architecture means designing reusable services from the ground-up, rather than just putting a service interface on top of legacy, monolithic, non-cloud architectures. ‘Micro services’ may be an acceptable term for this form of SOA.)

It also requires a recognition that a multi-tiered IT organisation is a necessity – i.e., that the traditional IT organisation cannot retain control over the entire technology spend of a firm, but instead should focus on those capabilities of cross-business significance (beyond obvious functions like HR, Finance, etc), and enable business (revenue)-aligned IT folks to build on top of these capabilities.

A digital architecture cannot be built over-night: the key skill is in understanding what the business needs to do now (i.e., in the timeframe of the next budgeting period), and in continually evolving the architecture to where it needs to be in order to be ‘digital’ through the projects that are commissioned and executed, even as specific business needs will be constantly changing and adapting.

This means that strategies for cloud adoption, application portfolio consolidation and modernisation, mobile app development, big data, security, ‘devops’ etc all need to be implemented incrementally as part of this over-arching digital evolution approach.

The digital evolution architect (DEA) is the role or function that supports the transition, putting legacy applications on a clear evolutionary path, ensuring that progress does not leave its own legacy, and that the overall cost of change is on a fixed downward trend, even in the face of continuous innovation in technology. It is the role described in the ‘Opportunities and Solutions” part of the TOGAF method for enterprise architecture, albeit with a very explicit digital mandate. As such, it requires close collaboration with folks involved in business & data architecture, solution (information systems) architecture and technology architecture. But the DEA is primarily concerned about evolution towards coherent goals, not about deciding what the optimal target state is, and hence cares as much about portfolio management as about architecture.

For larger firms, the enterprise architecture function must identify which DEA’s are needed where, and how they should collaborate. But DEA’s should be accountable to the head of Digital Transformation, if a firm has one. (If a firm doesn’t have or plan to have such a function, then don’t expect it to be competitive or even around in 10 years, regardless of how big it is today.)

This blog will elaborate more on the skills and capabilities of the digital evolution architecture concept over the coming weeks.

 

 

 

 

 

Digital Evolution Architecture