Battling complexity in large web applications

Nowadays, companies are hardly ever satisfied with a mere web-presence. No, websites have become true applications in their own right. Instead of being a by-product, most businesses acknowledge that the web offers them a nice channel to do business. However, the demands from the business keeps growing as each company want to be better than its competitors.

The increasing demands from the business have a direct influence on the applications complexity. Each time, especially when joining external projects, I am amazed by the way complexity is dealt with in most applications. “Building software isn’t rocket science”, but sometimes we do seem to make it just as complex.

In this article, I provide some insight in different forms of complexity, and how software development teams can design their application in order to deal with the ever growing demands from the business.

At the dawn of each software development project, the sky is blue and the grass is green. That’s the type of project most developers like to do. However, as time progresses and features are added and changed (they are hardly ever removed, for some reason), the developers become entangled by their own code. Concepts –in which lots of time was invested to get them defined- become muddled and intentions of methods and services become vague. Eventually, the application will evolve into a Big Ball of Mud.

Causes of complexity

The degree in which the project team deal with complexity significantly influences the speed at which a codebase evolves into a ball of mud. There are numerous sources for complexity, that can be separated into three categories.

It is important to distinguish the different causes of complexity

Project Environment

The first source is the project environment itself. This mainly includes the availability of resources (e.g. people, money, knowledge) and the management style of the project manager and customer. Some project managers truly believe that when a developers estimates a piece of work at 1 week, that the sentence “I’ll give you 3 days” will actually do good to the project on the long term. Although an interesting topic, it is not the type of complexity I want to deal with in this post.

Technical Complexity

Another aspect influencing complexity is the technical environment of the project. What are the frameworks and platforms used in the development and deployment of the software. As with any technology, it is important to know the powers and limitations of each framework. If a framework requires expert knowledge to interact with it, then try to abstract that knowledge away into separate modules.

Every development team contains a couple of developers that is very well capable of solving these technical solutions. However, once solved and abstracted away, this technical complexity is mitigated and hardly a risk to the project. Of course there are extremes, where project heavily rely on bleeding edge technology. On those projects, you’re likely to keep a high level of technical complexity in the project. But no matter what, it should be clearly separated from the other type, discussed in the next paragraph.

Business Complexity

The third aspect is the one I will be focusing on in this post: the business domain. It is the sum of the knowledge, rules and activities of your customer that makes them survive. Solving the problems in this domain is often highly complex, but not unsolvable.

The majority of the business complexity is composed of the business rules. In contrast to the technological environment, the business environment is continuously in motion. And for some reason, this always means the addition of new rules and the changing of old ones, hardly ever removing them. The result of this process is an ever-growing application with increasing complexity.

The next sections cover a few guidelines that help you manage complexity and maintain it in acceptable ranges. In the end, people have to be able to understand the code.

Application Layering

The use of layers in software engineering is a concept I encounter in almost any project I join. The boundaries and definitions of these layers, though, are often fuzzy or undefined. The result is that, as the project progresses, different types of concerns leak into layers of the application where those concerns do not belong.

When correctly defined, layers can help developers make a clear separation between different types of complexity. Solve technical issues in another layer (e.g. the infrastructure layer) than the business complexity (e.g. application or service layer). Define clear boundaries between these layers and make sure each developer understands and respects them.

Use a model

The real world as a whole is useless when it comes to solving problems in software engineering. If you’re in the theatre ticket selling business, you don’t care that seats come in different styles, different colors and that some types of seats have legs. In other words, you don’t care what a seat is in the real world. I don’t expect to be telling new stuff here, but for some reason, I see it go wrong many times.

A common mistake I encounter in programs is that there is no real model at all. Well, there is a modules in the project called model, and it contains classes with getters and setters. Technically, it is a model, but is has been simplified to the extent that it is pretty much useless in solving the business problems. These problems are then solved in the service layer. But with the additions of new rules, this layer grows out of proportions with in some cases hundreds of methods, some of which are quite-the-same-but-not-really.

Another pitfall is the desire to make their model similar to the real world. The real world is not relevant to the application. An application needs to solve problems. The real world might just not be the ideal place to do that. Instead, the business itself should drive the modeling choices. And if two parts of a business (e.g. divisions) have different views on the same actual thing, consider creating a different model for each one of them.

A third consideration often forgotten in the development process is that not every part of the application should have to use the same model. A model is used to solve (complex) problems. Each part of an application solves different problems in the domain. A logical conclusion is that these parts do not necessarily ask for the same model. A good indicator for the possibility of splitting the model is that a single word is used in different contexts, and has a different meaning in each context. The word “Flight” will mean different things to the ground crew and the cabin crew. They will all have different intents when using the concept of a Flight. Combining all these intents into a single (shared) domain concept will most likely clutter your
domain and make it harder to understand.

Watch your language

Ideally, the names of the concepts in the model reflect the actual words used in the business. If a new concept is added to the model, that name should be introduced in the business as well. Eric Evans calls this the “Ubiquitous Language”, the language used by all people and software involved in the domain the software runs in.

Why is this important? It helps new developers on a project. If they are familiar with the terminology, they will most likely understand the application right away. If they don’t, they will pick up the jargon faster and be able to talk to domain experts earlier.

It will also help in finding the locations where changes need to be made. If the model is correct (i.e. fits the business nicely), the language the domain expert uses will pinpoint you to the location where changes need to be made.

Modularity

Where simple applications will contain just a few concepts (where each concept is implemented as one or few classes), large applications may contain literally hundreds or thousands of concepts. A human being is simply not able to maintain a good overview of that amount of concepts.

Therefore, separate related concepts into modules. Within a module, these concepts may have tight relationships. In other words, the classes representing these concepts will have dependencies towards each other. Across modules, dependencies should be minimized. I often encounter dependencies between modules that don’t really need to be there. Removing them decreases application complexity and increases its maintainability. Especially “Entity”  to “Entity” relationships should be avoided as much as possible.

For example, let’s say we have an Order that references a Product, both in different modules as different departments manage the lifecycle of these objects. What happens if the Product price changes? All orders containing that product (and most of them have probably been handled already) will change as well. The solutions is simple, add a property to the order that states the price at which the product was sold. What if other characteristics change? The name for example? Remember, we don’t care about the real world , where products don’t just change name, we care about the world the way the business wants it. Instead of referencing the Product entity as a whole, perhaps it is sufficient to copy the relevant data from the Product into the Order at the time the Order is created. Or perhaps the entity reference is only maintained for as long as the Order is being processed. This allows you to delete Product instances without losing data in the Orders module.

The choices you can make in keeping nicely separated modules heavily depend on the way the model has been built. The better the model is, the more flexibility you have in making choices that help your application.

Conclusion

In this post, I have briefly touched a few types of complexity encountered in software projects. Don’t deal with multiple types of complexity in one location (i.e. layer) in your application. Make a clear separation between these to prevent technical choices from interfering with your business logic and vice versa.

When it comes to business logic, try to make a model that fits the way the business likes to work. Don’t try to make it look like the real world. That world has proven to be less than ideal when it comes to solving complex problems.

Use the Ubiquitous Language to define the terms and concepts in your model. This helps new developers to learn the language faster and get a better understanding of the application.

Finally, when the number of concepts expands beyond human capability of maintaining an overview of them, separate concepts into modules. Choose your modularity carefully. Try to keep dependencies tight inside the module, while limiting the dependencies between modules.