Technology Grafting

Created on 2021-12-23 09:48

Published on 2021-12-23 17:01

Wikipedia defines grafting as: "a horticultural technique whereby tissues of plants are joined to continue their growth together". Grafting is often used for producing new varieties of fruits on older trunks that are more resilient or adapted to a climate.

When working with legacy software we can see modernization as grafting. The rootstock is the older technology - proven and battle-tested while the scion is a newer technology that will enhance the rootstock's qualities. The technological grafting needs two things: interfaces/protocols and encapsulation.

The interface layer ensures that the data can be understood by both scion and rootstock. If the rootstock uses some very specific encodings (EBCDIC, various other locale-specific charsets) then the translation between them must be consistent in both directions. Things seem to be easy with textual protocols (although there might be also some issues) than with binary proprietary protocols. If a fast bidirectional translation is not possible (very different protocols) then a middleware might be necessary but in this case, it would be best that this middleware

On the other hand, encapsulation permits the scion to maintain all its properties and continue working as if it would be still attached to its original trunk. When grafting a pear branch on an apple tree the pear branch will continue to function as pear and will not hinder the apple trunk. The same behavior should be present when grafting software components. However, this strong encapsulation requires the open interfaces discussed before.

One of the most interesting technological graftings I have seen lately is in the form of modernizing older web applications using micro frontends packed as web components. The idea is not a novel one, web components are one of the ways of composing micro frontends, but this has massive implications for all the functional and nonfunctional aspects of the application. Imagine old PHP or Struts code bases getting an instant makeup with nice web components that reduce the server load and break monolithic applications into simpler parts. The web components isolate the new developments from the old code base and will permit the evolution of the older code base.

Grafting techniques and patterns might prove very useful in all kinds of modernization scenarios as there are a lot of use cases for this. The use cases can range from, as I wrote above, UI modernization to more interesting ones like adapting 50 years old systems to web or IoT. If the results of grafting are successful then this might help the rootstock also evolve (like Autocad's migration to the web). Grafting permits the progressive evolution of the software without disruptions in base technologies used at a minimal extra cost. The complicated problem is to determine where is the best place to insert the scion and how to make it behave on the same security and functional constraints as the rest of the application.

Necessary formalism

Created on 2021-12-19 14:39

Published on 2021-12-19 16:02

Most of us have started with a small application that solved a business need. Then we had some success with it therefore we added functionalities, improved UI, made it multi tenant and so forth. In the meanwhile the code has grown and every "if" condition we added made it more complex.

We added "ifs" because we had to respond to changes created by customer requests, changing laws and policies, access to different markets. Some of us were smart and resorted early to design patterns but some of us continued with the old way of adding ifs... In the end application is huge and hard to debug because it's codebase no longer reflects it's initial purpose. There is no "isomorphism" between the problem that application solved and the implementation of the application. Even applications that were well engineered have this issue as the changes they accumulated over time are not described in the original universe where the application was spawned, in the business part, but rather the engineering created approximations in the implementation domain that are maintained. To make a forced comparison the situation is similar to getting the image of a forest in autumn simply by creating a huge palette of colours without thinking of what we are trying to represent with them.

So moving along this line, how do we keep applications from going adrift? How do we keep in sync the reality where applications emerged with their implemented behaviour? Here formalism appears. Describing application behaviour in a formal way, close to what people understand is the key. This formalism have many names ranging from "Domain Driven Design" to DSLs but all focus on capturing the domain information on a higher level, not directly in code. Code is generally a byproduct of those, formal descriptions are needed for any DSL and low-code solution because, quoting Markus Volter: "models are semantically rich for their domains" whilst code is not semantically rich, just behaviouraly rich.

One of the challenges I am thinking about is: how do we retrofit formalism into application so that we make them more maintainable? There is no definitive answer but especially for legacy application refactoring to models is a key. We should start small, refactor user interface behaviour for example to state charts (in form of Redux or XState). In the meanwhile the refactored interactions could be again reorganized both server and client side into some business process descriptions (it would be ideal to use a dedicated DSL for those in this stage - but this is generally hard, so refactoring to state charts is a good beginning). Iterating over this a couple of time we will reach better models on the domain and clearer actions defined on the set of the models - hence we get a crude formal definition of the domain. At this point the process, described probably in a specific language can be understood again by business people therefore they can evolve it while it is still generating lower level formalisms (as the statecharts) that developers or specific tools can generate code from.

The new developments nowadays have the chance of starting clean as the current tooling is quite advanced (Jetbrains MPS, Eclipse Sirius). New projects can start from DSLs or from BPMN visual representations and generate rich models that can be visually manipulated in low code solutions. Moreover a repository of well described models and processes can be shared inside a company so people can reuse the high level abstractions, manipulate them in their own code and getting interoperability and common behaviour for free. Also testing is positively impacted as tests clarity increases because models carry more semantics inside. What would be easier to test: 500 lines of written code or 12-15 lines of DSL or visual representation? Where are the errors simpler to catch?

Well crafted formal definitions reduce the implementation and design burden for the implementation team as they shunt most of the repetitive design and boilerplate tasks in an application (rich models already contain the logic that otherwise has to be expressed and maintained in code). Continuous effort of expressing behaviours in high level, formal languages and refactoring older code towards the same seems a way of increasing maintainability and quality. DSLs, modeling and low-code are in fact facets of the same base problem - how to capture and communicate succinctly, correctly and in a non ambiguous way domain knowledge both for people and for machines.

PS: There are better approximations of a process definitions than Amazon's StepFunctions or BPMN. To quote a friend - they are like GoTo programming and do not incorporate transactional aspects.