Make components reusable and NOT necessarily the orchestration layers in your software platform

Making things reusable in software engineering is very important. That’s why all the general purpose programming languages have methods/functions and they also have object oriented capabilities. Using these capabilities you can implement different patterns. Yes, different languages implement these patterns differently, but my point is that they are all achievable.

Generally speaking junior programmers write code that is simple, but it may not be as reusable as we would like it to be. On the other hand, when programmers gain some experience and learn these different programming patterns, they may get over-excited and start applying these patterns without figuring out first what problem they are trying to solve. They may kind of look at the list of patterns and say: “Oh, this one would be cool”. It is totally fine to use a known programming pattern, but how you arrive at using one decides if you are doing it for the right reasons. Here is what I mean.

First, understand what problem you are trying to solve.

Then figure out the solution for this problem without being disrupted by available patterns. Focus strictly on the solution and work it out on paper or a piece of napkin.

Now that you have a solution to your problem, you can look at the list of known programming patterns at your disposal and most likely one those patterns would be the one that exactly fits the solution. However, there could be a chance that you don’t have a known pattern or it may mean that you need to use a combination of two patterns with slight modifications.

You may be wondering where I am going with this. I am basically trying to get to my next point which is the main subject of this article. I want to talk about making your individual components in your platform reusable and NOT necessarily the actual orchestration layers that you may have.

Let’s assume that you have the following components:

• Component A
• Component B
• Component C
• Component D
• Component E

You need to develop each one of these components in such a way that they are reusable so that the orchestration layer can invoke those components without needing to learn about the internals of your components. For the sake of this example, I will assume that those components are just methods/functions. Then let’s assume that the orchestration layer looks as following:

Orchestration Layer:

MethodA();
Async call to MethodB();
MethodC();
Wait for MethodB() response
MethodD();
MethodE();

Then what could happen is that you try to make this orchestration layer reusable by creating a template for it. When you do that, you could create a workflow or some type of state-map that ties all these calls and you have some fancy configuration that allows you to connect state to state and you have a mechanism to incorporate a new step in the state-map without writing any code.

I understand this conceptually, but my question to you is: Why? What do you gain by doing this? What problem are trying to solve? Are you solving a problem at the root?

A lot of times the response is: We need to do it because we want to introduce new steps in this step-factory (state-map) without needing to push any code.

Yes, there are valid reasons to introduce the step-factory pattern (state-map) in your design, but most of the time you don’t have a good reason and I will explain why.

You maybe introducing this step-factory design for wrong reasons. It could be that in your organization it is very hard to deploy a code/binary change to production due to a lot of processes. So you ended up changing your design (even for a simple case as above) and introducing the step-factory to get around the process so that you can deploy a config change quickly to production. First, you are complicating your design and implementation by introducing this without good reasons. Second, you need skilled developers to make code changes within it and you are actually adding more risk.

Let me use the following analogy with the car engine and other cars parts under the hood of your car. Under the hood in the engine bay, you have the following:

• engine
• battery
• alternator
• water pump
• brake cylinders
• air intake and air filter
• radiator
• fan
• …etc

You can picture those car parts as the individual components that I described above in your software platform. The engine bay itself would be the orchestration layer. Everything in this engine bay (orchestration layer) is connected a specific way and it works great. If something goes wrong, you can go inside the individual parts (components) and you can replace the full part or the insides of that part. For example, the air filter could be dirty and you just open the air filter box (component) and put a new air filter. So this setup with this engine bay (orchestration layer) and these parts (components) is great if all you are doing is changing the insides of a part/component or replacing the whole part/component with another one that fits in (using the same interface). So by changing these parts, you are pretty much changing just the configuration within your step-factory. If that is all you are doing, that is perfectly fine and this engine bay / orchestration layer / workflow is perfectly good for you.

However, the world of software engineering (especially microservices world) is not as static as the engine bay of your car. You typically have a need to put things together using different permutations or you have a need to orchestrate in a different way. That’s where you need to step back and ask yourself if you really need to turn your orchestration layer into some template/step-factory/state-map when you are actually constantly changing what that orchestration looks like? This is where you realize that the complexity of your state-map/workflow via step-factory pattern is actually complicating things and it is slowing you down, and it is adding the unnecessary risk. There is nothing wrong with making a code change in your orchestration layer in order to meet a requirement. Tell yourself, it is ok. It is not 100% configurable, but I know that making it 100% configurable would slow me down because of added complexity.

As for my example above about using the configurability to get around your process, you really need to think about solving that problem at the root. That company probably introduced the process to slow developers down because they were manually deploying things to production and breaking things. So the typical reaction from your Change Control team is to tighten the SDLC process. What you are missing is the continuous integration and automated deployments that increase the confidence, success and reduce the risk. That is your solution for that problem and your Change Control team will welcome that because it is repeatable and predictable.

In conclusion, please think about the complexity of your code. Make your individual components modular/reusable and use them in your code. After you use them enough in your orchestration layer, you can determine if you want to keep your orchestration layer simple, or you need to build a template (i.e. step factory or workflow) for it. Think about the maintenance here and your use cases. If you are constantly changing the flow of your orchestration layer, then the complexity you are introducing with the workflows/step-factory is very unnecessary. Then stick with the basics, and it is ok to change code in order to adjust the orchestration layer. It is ok, but streamline your deployments throughout the pre-prod environments and into production to make this experience very smooth.

Thank you for reading.

Almir Mustafic