API · API Gateway · Microservices · Uncategorized

Conway’s law in API integration


At a couple of consulting engagements, I observed how Conway’s law is still relevant and used it to design better integration between software components.

This blog post maps how organisation of software development teams in organisations doing API development can lead to a smoother/faster-to-market product and how this relates to a well know observation made by Melvin Conway.

Conway’s Law

  • “Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.” – Melvin Conway [1]
  • “If you have four groups working on a compiler, you’ll get a 4-pass compiler” – Eric S Raymond [2]

API development teams and modules

The observation was the API modules were divided into 3 “modules” – service providers, service consumers and the API gateway providers aka the “new middleware guys”.

3 API Teams

Side Note: This pattern can also be seen in Enterprise Application Integration projects where backend applications rely on a middleware to send/receive data in a canonical format with each other and a portal …

SOALayers

 

Business impact

The impact of have a 3 team organisation was observed during the software development lifecycle and maintenance.

During the development process, there were 3 modules created – the core service, the gateway component and the consumer component (Mobile apps, Web UI etc). This led to fragmented testing, duplicated tests across teams,  general confusion regarding service availability monitoring and support responsibility in non-prod environments. Often core service developer would refuse to communicate with end users, relying on the middleware/gateway team to be the first-point-of-contact.

Similarly operational support was fragmented – the gateway enabling team had to monitor the core services, alert the backend teams and relay information across multiple teams leading to bottleneck.

The business impact were therefore delayed projects, frustrated testers and higher resolution times for incidents.

 

Software Cohesion

We use the concept of Software Cohesion to understand why having multiple teams, and different modules of software, have such an impact on the SDLC and operation of an API.

Cohesion is nothing but how closely components of a software are related to each other, a formal definition is as follows:

“cohesion measures the strength of relationship between pieces of functionality within a given module”  [3]

In an API development context, the 3 components – core service, gateway and consumer code need to be analysed for cohesiveness.

 

Cohesion.png

The simplest argument would be to have a single module, one team building everything and be done with it! However this presents a new challenge since there may be more than one client (Mobile, Web, 3rd party – B2B) consuming the core service.

We believe that there should be at-the-most two modules – Service and Consumer. The service team would therefore own the gateway component, own the end-to-end testing of the API and be the first point of contact during development and support (You build it, you own it!)

It is important to therefore understand and determine which components of a software are related – there is cohesion vs which are not. Doing so can help decouple systems while maintaining integrity within related components.

So what’s the fix?

Recognise which components are closely related and which components can be decoupled. Decoupling systems that are closely related – by direct action or by accident can lead to communication issues, maintenance issues and chaos in a project.

In an API development lifecycle, we found the best practise is to allow service providers to own and provision the API gateway components (instead of a separate team of middleware specialists)

APIBestPractise.png

This empowers the API developers and allows them to communicate/test with the consumers directly while giving them ownership and responsibility of the overall solution.

Conclusion

We observed certain pain-points emerged during API development process due to ownership of software components and traced the root to organisation of teams per Conway’s law.

A study of the systems and their dependencies showed that getting core service teams to own the entire API stack (from the gateway down) helped significantly reduce friction during the SDLC and improve operational support.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s