Your AI, Your Rules

There is a moment familiar to anyone who has ever fallen deep into a video game, the moment you stop playing the game and start investing in it. You buy a better headset. You tune your controller. You learn the maps not because someone told you to, but because you care. The experience rewards your attention, and your attention reshapes the experience. It becomes yours in a way no off-the-shelf purchase ever quite is.

I believe this is exactly how we should think about artificial intelligence, not as a utility we log into, like electricity or Wi-Fi, but as something closer to a companion we build, tend, and grow alongside. Not a service. A relationship. This might sound fanciful in a world where AI is largely discussed in the language of industry: compute costs, API calls, market share, frontier models. But I think the dominant framing is wrong, or at least incomplete. The question we should be asking is not who owns the most powerful AI, but how do we make sure everyone can have their own.

The Gamer's Instinct

Consider how gamers relate to their equipment. A serious gamer does not resent spending money on a quality setup. They choose to invest because the return is real, sharper response, deeper immersion, a machine tuned to how they play. Nobody mandates this investment; it emerges naturally from genuine engagement with something they love.

What if we extended this logic to AI? What if, instead of renting intelligence from a distant server farm, people were encouraged to own their AI experience, to provision it, personalise it, and take pride in it the way a craftsperson takes pride in their tools? This is not a fantasy. The hardware and open-source ecosystems are already moving in this direction. Small, capable models can run on consumer devices. The infrastructure for decentralised AI is being built quietly, piece by piece. What is missing is not the technology, it is the cultural permission to think of AI as something you have, rather than something you use.

A world where AI runs in a distributed, community-owned way is also a more resilient world. When intelligence is not concentrated in a handful of data centres controlled by a handful of companies, it cannot be switched off by a single policy decision, a single outage, or a single acquisition. Decentralisation is not just an ideological preference; it is practical engineering.

Companions That Collaborate: The AI Association

If your AI reflects who you are, then what happens when you work with someone else?This is where the companion model opens up into something genuinely exciting. Imagine two researchers, one a biologist, one a data scientist, joining forces on a project. Each brings their own expertise, their own tools, their own way of thinking. And each brings an AI companion shaped by years of working in their field: one steeped in molecular literature and lab protocols, the other fluent in statistical methods and code. When the humans collaborate, their companions collaborate too. The biology companion surfaces relevant papers; the data science companion writes the analysis pipeline. The whole is sharper than the sum of its parts. This is not a fantasy of automation replacing people, it is a vision of people working more effectively as people, augmented by tools that genuinely understand their domain. The AI does not flatten the differences between the two researchers; it amplifies them. Your companion is most useful when it knows your speciality deeply, which means that diversity of expertise becomes a feature, not a problem to be standardised away.

We might think of these as AI associations, loose, voluntary groupings of people and their companions, formed around a problem, a project, or a shared goal. A legal collective. A research consortium. A neighbourhood planning group. Each member's AI brings specific knowledge; together they can tackle problems that no single person and no single general-purpose model could handle alone. And because the association is composed of individuals who each own their tools, it remains accountable to its members. There is no single point of capture, no platform that can hold the group's work hostage.

This is a fundamentally different vision from the one currently on offer, where a single model is trained to be good at everything for everyone, and where the answer to any question tends to look roughly the same regardless of who asked it. The companion model rewards depth. The association model rewards collaboration. Together, they make a case for AI as something that makes human groups smarter by making individual humans more themselves.

Growing Together: The Ethics of Personal Knowledge

Here is where the companion metaphor earns its keep. Imagine an AI that begins with a solid, ethically grounded core, trained on established, publicly available knowledge: science, history, literature, mathematics, the accumulated record of human inquiry. This is the foundation, the shared ground. Nobody owns it; it belongs to everyone. But then imagine successive layers, built over time, that belong entirely to you. Your conversations. Your notes and papers. Your library, your code, your half-finished ideas and well-worn arguments. Your AI companion does not just answer questions, it knows you. It remembers that you spent three years working on distributed systems, that you find a particular philosopher's work compelling, that you tend to think in analogies before you think in abstractions. It has grown alongside you, shaped by the texture of your intellectual life. This is not surveillance. This is the opposite of surveillance. The data lives with you, on your terms, under your control. Nobody else has access to the map of your mind that your companion holds.

And here is something the current discourse almost entirely misses: this model restores the incentive to learn. One of the quieter, more troubling effects of centralised AI is what it does to intellectual motivation. When any question can be answered in seconds by a model that has read everything, the temptation is to stop reading yourself. Why develop expertise when you can simply prompt for it? Why struggle with a hard concept when a fluent summary is one message away? The centralised model, for all its power, subtly undermines the habits of mind that make people genuinely capable: curiosity, persistence, the willingness to sit with difficulty until understanding arrives. The personal companion model inverts this dynamic entirely. When your AI is built from your knowledge, when its depth in a subject reflects your depth in that subject, then learning is not just admirable, it is strategically valuable. Every paper you read, every skill you develop, every domain you come to understand makes your companion more capable. You are not outsourcing your intelligence; you are compounding it. The better you become, the better your companion becomes. The relationship rewards growth in a way that the subscription service never could.

This matters especially for young people. An educational model built around personal AI companions would look radically different from one built around access to a centralised oracle. Students would be encouraged not to ask the AI what to think, but to build an AI that thinks like them, which means they would have to think, clearly and deeply, first. The incentive structure shifts from consumption to cultivation. That is not a small change. It is a different philosophy of what learning is for. The consequences reach further than education, into every situation where one person is trying to understand another. Consider the job interview. It has long served as a rough but honest signal: how does this person think? What do they know? What is their instinct when a hard question lands unexpectedly? Today, those signals are being eroded. When every candidate has been coached by the same centralised model, they begin to arrive speaking the same language, deploying the same frameworks, structuring their answers with the same confident fluency. The surface looks polished. But the differentiation that interviews exist to reveal, the genuine depth, the particular way someone's mind works, the real contour of their experience, gets flattened into a kind of averaged competence that tells you very little about who is actually in the room.

The companion model pushes back against this in a concrete way. If your AI is built from your intellectual history, your actual work, your real thinking, your specific expertise, then what you bring to a conversation, an interview, a collaboration, is genuinely yours. The companion amplifies your particular strengths rather than covering for your gaps with borrowed fluency. A person who has developed genuine expertise in a field will have a companion that reflects that depth; a person who has not cannot fake it by asking the same model for a polished summary. Human skills, hard-won, idiosyncratic, specific, are preserved rather than minimised. The people in the room start to look different from each other again, because their tools are as individual as they are. There is an ethical dimension here that deserves to be named plainly: knowledge has value, and that value belongs to the people who created it. A well-designed AI ecosystem should make it easy to pay for access to great books, scientific papers, and curated information, and make it economically worthwhile for authors, researchers, and institutions to participate. Free, openly licensed sources will flourish. Proprietary sources will be accessed fairly, with compensation flowing to their creators. The alternative, an internet-scale scrape of everything, with no accounting for who made what, is not a feature of the AI age we should accept as inevitable. We can choose better.

Knowledge is not free to produce. Treating it as though it were is not liberation; it is just a different kind of theft, dressed in the language of openness.

Plugged Into the Sun

There is another dimension to the companion relationship that tends to get lost in abstract debates about AI governance: the physical world. AI systems are hungry. Training a large model consumes enormous quantities of energy. Running inference, at scale, across billions of daily interactions, consumes more. At the moment, much of this energy comes from sources that carry real environmental costs. This is not a reason to abandon AI, it is a reason to be deliberate about how we power it.

A distributed model of AI ownership opens up a possibility that centralised data centres cannot easily replicate: genuinely local, genuinely renewable energy. If your AI companion runs on a device in your home, it can run on electricity from your solar panels, your community wind cooperative, or a regional grid powered by hydroelectric dams. The energy that feeds your companion's thinking can come directly from the sun that falls on your roof. This is not a small thing. It means that the environmental footprint of AI can, in principle, be tied directly to individual choices and local conditions rather than to the energy mix of whichever state happens to host the nearest mega-campus. It means that the relationship between a person and their AI companion extends outward, into the world, into a relationship with energy and place and planet. An AI that runs on sunlight feels, in some small but meaningful way, more honest than one that runs on something you would rather not think about.

The Most Important Argument: This Must Be for Everyone

Everything I have described, the owned experience, the layered personal knowledge, the local green energy, risks becoming a luxury if we are not careful. And that would be a catastrophe. We are at a hinge moment. AI is becoming genuinely useful: in education, in medicine, in legal access, in scientific research, in creative work. The people and nations that have early, deep access to these tools will develop advantages that compound over time. Skills improve. Productivity rises. New ideas get generated faster. The gap between those with access and those without will not stay flat, it will widen, and widen, and widen.

This is the rich-get-richer dynamic played out at civilisational scale, and it should alarm us as much as any geopolitical threat. A world in which AI capability is concentrated in a small number of wealthy countries is not a stable world. It is a world in which the accidents of economic history, which country industrialised first, which attracted the most capital, which built the most undersea cables, come to determine who gets to participate in the next chapter of human cognition.

The gamer gear model matters here too. When the barrier to entry is low enough, when a modest device and a modest monthly commitment can buy you a genuine AI companion, personalised and capable, then geography becomes less determinative. A student in Lagos or Lahore or La Paz, with a decent phone and a community mesh network, should be able to access the same quality of intelligent assistance as a student at a well-endowed university in a wealthy country. This is not utopianism. It is a design choice we could make, if we decided it mattered. Lowering the entry barrier is the single most important policy and engineering priority of this moment. It is more important than who wins the race to the most powerful model. It is more important than which company achieves which benchmark. A billion people with access to a good-enough AI companion will do more for human flourishing than ten thousand people with access to a perfect one.

What We Are Really Choosing

The question of how AI develops over the next decade is, at its core, a question about what kind of relationship we want to have with our own intelligence, and with each other.

The centralised model says: trust the institutions. Rent access. Accept that the infrastructure of thought belongs to those who can afford to build it. This model has a certain logic to it, and it is not without merit. But it also has a long historical track record of concentrating power in ways that prove difficult to reverse. The companion model says something different. It says: your intelligence is yours. Your knowledge is yours. Your energy should be yours. The tools you use to think and learn and create should belong to you, should grow with you, should reflect your values, including your environmental values, your epistemic values, your sense of what knowledge is worth paying for.

It says that the AI era does not have to be a story about dependency and extraction. It can be a story about relationship. About investment, not in the financial sense, but in the human sense: the time and care you put into something because it matters to you, because it reflects who you are, because you want it to be good. It says that when people come together, each with their own companion, their own expertise, their own piece of the puzzle, the result is not uniformity but genuine collective intelligence. Groups that are smarter because their members are more themselves, not less. And it says something hopeful about human skill and human difference: that the right technology does not sand people down to a common average, but makes their particular capabilities more visible, more legible, more valuable. That a person's genuine expertise, earned through years of work, shaped by their unique path, is worth preserving. That when two people walk into a room, we should still be able to tell them apart. And it says something hopeful about learning: that the right technology does not make us lazier, but more ambitious. That a tool which rewards your growth will make you want to grow. That the incentive to understand things deeply, which has always been one of the finest human impulses, can be strengthened by AI, not eroded by it, if we build it right.

Gamers know this instinct well. So do gardeners. So do anyone who has ever spent more time and energy on something than was strictly necessary, because they wanted it to be theirs. That is the AI era I want to live in. Not the one where intelligence is piped to me like water from a distant reservoir, metered and billed and subject to outage. But the one where my AI companion and I have been through things together, where it knows my way of thinking, runs on my electricity, and belongs to a world where everyone, regardless of where they were born, gets to have one too. We are not there yet. But we could be. And the first step is simply deciding that this is what we are building toward.

Selling Snake Oil: Celebrity, Credibility, and the Hype Machine

Created on 2026-04-10 10:22

Published on 2026-04-10 10:26

There is a peculiar affliction spreading through the technology world, one that has nothing to do with algorithms or compute budgets. It is the calculated deployment of a famous name to lend gravity to something that would otherwise sink without a trace. Two recent cases illustrate the pattern with uncomfortable clarity.

The first involves Kristen Stewart. In January 2017, a paper appeared on arXiv titled Bringing Impressionism to Life with Neural Style Transfer in Come Swim, co-authored by an Adobe engineer, a producer and, notably, the actress Kristen Stewart. The AI community was briefly confused, then largely charmed. It should have stayed confused. The three-page paper is a high-level case study of applying an existing, well-understood technique to a short film Stewart directed. No new method. No novel insight. Nothing the original style-transfer literature hadn't already established. What Stewart genuinely contributed was using the technology in her film, a legitimate creative act, but not a research contribution. The inflation was entirely in the reception: breathless press coverage, NVIDIA blog posts, researchers competing to calculate her Erdős number. The name did all the work the science could not.

The second case is fresher, louder, and considerably less defensible.

In April 2026, a GitHub repository appeared under the account milla-jovovich, launching an AI memory system called MemPalace, co-credited to the actress Milla Jovovich and crypto CEO Ben Sigman. Within 48 hours it had over 23,000 stars. The headline claim was a perfect score on LongMemEval, the gold-standard benchmark for AI memory systems. Developers tore it apart within hours.

The benchmark, it turned out, never generated an answer to any question. It checked whether a correct session ID appeared in a retrieved list, never verifying that the retrieved content actually answered anything. The 100% LoCoMo score was achieved by setting the retrieval pool size larger than the total number of sessions in the dataset, guaranteeing the right answer was always included by default. As one analyst put it, it reduced to dumping everything into Claude and asking which part matched. That is not memory. That is not retrieval. The README advertised compression ratios that real tokenizer counts disproved, and a knowledge-graph contradiction-detector that was never actually wired into the code.

Then there is the authorship question,the one that cuts deepest. The account hosting the repository had seven commits and two days of GitHub history. The original account that pushed the code, named "aya-thekeeper", was deleted immediately after launch. When pressed, Jovovich and Sigman explained that "Lu", the mysterious name appearing in commit history, was simply Jovovich's AI coding agent. Jovovich herself admitted the division of labor plainly: she described the concept, Sigman built the software. Whether that constitutes co-development, or simply the purchase of a famous face for a launch campaign, is a question neither of them has answered convincingly. What is documented is that a cryptocurrency also named MemPalace, with Jovovich and Sigman holding a 50% creator reward split, was pumped and dumped within 24 hours of the announcement.

The real contribution of Milla Jovovich to MemPalace remains unproven. What is proven is that her name generated millions of impressions for a project whose benchmarks were rigged, whose README described features that didn't exist, and whose original developer quietly vanished.

Both cases expose the same mechanism. A famous name bypasses the scrutiny that any anonymous submission would face. It reframes the question from "Is this good?" to "Isn't this surprising?"and, surprise, unlike quality, requires no verification. The press amplifies, the stars accumulate, and the actual engineers doing mundane, honest work in the same problem space receive nothing.

In science and engineering, a name is not an argument. The only thing that has changed is how efficiently a famous one can be converted into attention and in the right hands, into a coin.

Buckets and Sovereignity

Created on 2026-01-31 07:12

Published on 2026-01-31 07:26

S3 buckets have a problem. A developer needs a bucket, a single bucket. Something that'll take AWS about 30 seconds to spin up. However they have to open a Jira ticket. Then they ping you on Slack. Then they show up at your desk because the ticket's been sitting there for three days and their sprint is ending.

Then, to your chagrin, open the huge Terraform repo, add their bucket, run terraform plan, watch myriads of resources scroll by, get it reviewed, wait for the pipeline, and boom. One S3 bucket. Only took a week and three senior engineers.

This is stupid. Very stupid. But we keep doing it because we've convinced ourselves that "infrastructure as code" means all infrastructure must live in the same repository, managed by the same team.

Is it true? An S3 bucket that only exists to serve one application isn't infrastructure. It's part of the application.

Infrastructure is: VPC, EKS cluster, networks, etc. A bucket that stores upload images for the marketing website? That's application state that happens to live in AWS instead of Kubernetes. That could have been another API call to a object storage service. Managing application resources as infrastructure with an infrastructure tool is not friendly at all.

Enter AWS Controllers for Kubernetes.

ACK does something wonderfully simple: it turns AWS resources into Kubernetes resources. Want a bucket? Write a manifest. The ACK controllers watch these manifests and call the AWS APIs. The developers get self-service. You get to stop being the S3 bucket vending machine. There is no giving up control, you're shifting what you control. Th control increases in fact as the cluster will try to keep resources under control continuously, not only when terraform plan/apply happen. Instead of gatekeeping every single resource creation, you define the rules.

Another team needs a Postgres database? They drop this in their repo:

They commit it. The pipeline runs. The database appears. Your policies ensure it's in the right VPC, encrypted, backed up, tagged correctly. Tags are applied for correct cost allocation so the financial minded persons have a nice cost breakdown.

This is what platform engineering actually means. You build the platform, the foundation, the policies, the standards. Developers build on top of it. Both teams do what they're good at.

And when they delete that feature branch? The database goes away. No orphaned resources quietly racking up charges for six months because everyone assumed someone else would delete it. Everything's in Git, versioned with the application code. Developers can see infrastructure status with kubectl get dbinstance. The blast radius of any change is limited to that team's namespace.

The plot twist: digital sovereignty.

In the last year cloud shifted from pure convenience to strategic threat. European regulators are increasingly uncomfortable with critical data for European companies subject to American surveillance laws. GDPR was just the opening act. The Digital Services Act, the Data Governance Act, these are forcing real architectural decisions.

And here's the uncomfortable truth about ACK: it makes you really, really good at AWS. Every pattern, every practice, every custom resource is AWS-specific. Which is fine for US but not so much for Europe.

Enter Crossplane who does something clever. Instead of direct AWS API bindings, it gives you an abstraction layer. You define what a "Database" means for your organization, and Crossplane figures out whether that's RDS, or Cloud SQL, or, interestingly, a database on OVHcloud, Scaleway, or Open Telekom Cloud. They're European cloud providers, subject to European law, outside the reach of the CLOUD Act. For organizations handling European citizen data, running European critical infrastructure. This actually matters.

The developer experience stays the same. They still request a "Database." Your platform team just swaps what that provisions underneath. Today it's AWS. Tomorrow it's a sovereign provider. The workflow doesn't change.

Maybe the organization doesn't care about digital sovereignty. Maybe one is fine being all-in on AWS forever. That's legitimate. But the organizations that do care: regulated industries, government contracts, ones thinking five years ahead, are realizing that flexibility isn't just nice to have.

Because the geopolitical landscape that made AWS the obvious choice in 2015 might not be the landscape of 2030. ACK solves the S3 bucket problem beautifully. But Crossplane solves it and gives you an exit strategy. In a world where "which country's laws apply to this data" is becoming as important as "how many nines of uptime do we get," that's worth thinking about.

The marshmallow experiment

Created on 2025-05-18 07:08

Published on 2025-05-18 08:17

Somewhere in the 70s psychologists at Stanford University made an experiment about "delayed gratification". Children were offered marsh mallows that they could eat immediately or, if they delayed it for 15 minutes, they would also get some extra treat as a bonus. Following up the results they found a correlation between those that were able to delay gratification and their later results in life being better in comparison to those who were not able to do it.

On the other hand we are pushed towards immediate gratifications by a lot of factors that our society is based. We are sold the idea that everything should be "Bigger, better, faster, more" and we need to "want it all, and want it now". If things are fast we pat ourselves on the back considering ourselves "efficient". Is it true? Are the results similar to the Marshmallow experiment?

https://ownmygrowth.com/2020/09/27/instant-gratification/

The current trend for boosting efficiency is the use of AI everywhere. The most relevant trend is now the "vibe coding" - fast results with minimal effort. Yes, results are impressive, the assistants are able to produce code, sometimes better code than the humans in a fraction of the time as they have indexed HUGE amounts of ALREADY WRITTEN CODE. Hence a task that used to take days could be theoretically be finished in hours if not even in minutes. That is great and saved a lot of time.

But the question is what we do with that time? How do we spend it? Do we take just another task from the Jira board and throw it to the army of assistants asking the correct prompts or prompting an assistant to prompt another assistant to do it? Or we try to "grok" it ourselves? Do we invest the gained time in something creative and new? In my opinion creativity doesn't boost in the same way, contrariwise, creativity comes generally from scarcity not from abundance. Having everything served immediately reduces the need for searching and understanding the solutions for the problem at hand. As a recent Microsoft study revealed we are just trusting the response as it seems plausible and after a while we take it for granted as being correct without double checking it.

Doing something good as a human requires effort, delayed gratification. Try learning a new instrument. It will take a long time to produce something that resembles music, especially for those that at their first musical experience. I was, and I still am, an horrible player, practiced for years until I was able to play something that sounded good (depending to whom you ask). But I enjoyed the trip. Before playing that solo i had to learn not only the instrument, chords, and finger-picking but I learnt about bands, trends , music in general and, probably the most important thing, I got to know people with similar interests. Putting effort into doing something is the key to progress as Derek Muller explains it.

The lack of delayed gratification leads to stagnation, lack of desire to do something, no imagination for what might be the end of the trip. A general state of INFANTILIZATION might appear as the skills needed to obtain something will be underdeveloped and tantrums will be satisfied unconditionally. The issue is that also decision makers will be affected by the same symptom as all their goals will apparently be satisfied instantly with no rebuttals.

AI is a wonderful tool, I am all in for that. The issue is not AI, as it is not sentient, it is us in our infantile way of using it to get immediate gratification instead using it wisely - try first, get feedback or advice, try again cycle. If we are more efficient day after day, we should take some time to understand it. How embedding works, what is quantization, what risks MCP poses. Try replicate and understand the mathematics behind it, go back to the Algebra and Calculus one neglected earlier in life, verify the responses by reading the literature suggested as source (if it really exist) or try learning to draw something with pen and paper just to understand the techniques after asking the AI to do it as an example. Try to delay the gratification and dream of some better reward. Most of the time it worth it.

To finish in a darker note:

https://www.orau.org/health-physics-museum/collection/radioactive-quack-cures/pills-potions-and-other-miscellany/vita-radium-suppositories.html

Somewhere in the '30s Radium suppositories were seen as a great thing, as Radium was something novel at that time as a cure to slowness as their advertising was reading. Not understanding something completely and using it JUST BECAUSE IT WAS ADVERTISED is dangerous at least. Short term gratification cycle is just followed once again.

Weak Discouraged Men! Now Bubble Over with Joyous Vitality Through the Use of Glands and Radium

“If YOU are showing signs of “slowing up” in your actions and duties, perhaps long before you should—if you have begun to lose your charm, your personality, your normal manly vigor—certainly you want to stage a “comeback.” The man who has lost these precious attributes of youth knows how to appreciate their value. He realizes that happiness depends on his ability to perform the duties of a REAL MAN. Sweet, glorious pleasures of life. Nature intended that you should enjoy them.”

“Now is the time to act! Today! RIGHT NOW! Tomorrow may never come.”

Links:

1. https://www.lifehack.org/353923/instant-gratification-short-lived-you-should-aim-for-long-term-goals

2. https://medium.com/@darinleavitt/the-epidemic-of-instant-gratification-aef2a8d38903

My response to "From Mere Engineer to True Artist"

Created on 2024-04-11 15:28

Published on 2024-04-11 18:58

Codecamp Timisoara took place today, and it was quite a nice conference. The lineup was top-notch, and the topics were quite diverse. There were also some talks that seemed overly specialized, possibly requested by sponsors, but overall, it was quite an entertaining edition.

James Coplien delivered an interesting talk called "From Mere Engineer to True Artist" at the end. Coplien is known for stating unpleasant truths, and this is okay; it provides food for thought.

He pointed out many elephants in the room somewhat bluntly, but radical candor is necessary to shake up the industry. However, some of his assertions today were not entirely true, as there are many nuances that I struggle with.

• What are architects? Everybody is an architect; there is nothing to argue about here. At a certain scale, architecture is present in every detail of the field; every commit is an architectural decision. However, the title "architect" carries more implications. The term "architect" originates from the Greek ἀρχιτέκτων (archi - first/over, techton - builder/mason). Architects were defined by Vitruvius as mediators, experienced workers able to mitigate stakeholders' needs. Architects have scars from previous battles and failed buildings, which gives them another "unnamed quality" of being able to steer and communicate to avoid repeating the same mistakes.

• The lack of importance of a "4-year CS degree" and the notion that "CS is not a science/engineering." I completely disagree. In my view, CS is a science, residing somewhere between a branch of mathematics, a cornerstone of electrical engineering, a niche of quantum physics, and a domain of philosophy, yet with some fundamental ideas of its own. Of course, talent and hard work can compensate for formal education, but formal CS studies help evolve it from a craft to an engineering discipline by incorporating the feedback of countless generations of practitioners, creating something as solid as civil or chemical engineering. Maybe CS is not a true science yet, but it is at least "science in the making", gaining substance year after year. A 1-year boot camp could theoretically create good programmers but not scientists or engineers. The graduates of these boot camps could be productive, but they would mostly align with the 90% that Coplien fears will disappear, especially if their motives for choosing CS as a profession are not fueled by passion and continuous learning. He argued that kids can create software. True, they can create impressive things from a young age, but sometimes their creations are naive, functional indeed but lacking the details that a seasoned practitioner would add — details that provide an edge in terms of usability, performance, and quality. Kid's creations, even if not completely abstract are full of abstractions

Unicorn drawn by a kid

vs.

Unicorn drawn by a professional

Coplien argued that Renaissance architects who delivered value had no architectural studies; still, they created wonderful buildings. In my opinion, it was not necessary at that point in time, as during that period, the arts and crafts were mostly similar, and practitioners were genuine polymaths.

• Commissioned work: Most of the examples of great architecture he presented (Brunelleschi, Michelangelo, da Vinci) worked on commissioned buildings. The artists worked at the order of a sponsor, as artists are often poor, in need of money. There are just a few self-sustained artists, mostly in the 20th century, who were able to market themselves so that they could create at their own will. Even these exceptions had to do commissioned work at the beginning until they were able to create independently. Starchitects like Oscar Niemeyer created "commissioned works of art" - for example, the city of Brasilia. It was their genius that transformed a mundane request into an unforgettable work of art, but they did not initiate the project themselves with their own funds.

• Abstractions are bad: Again, I do not agree. Abstractions are the foundation of science and art. If we observe the evolution of Picasso's bull, we see it goes from concrete to abstract, refining the shape to its bare essence. There is nothing that one can take out from the last image and still have a bull. Abstractions create flexibility, a quality that permits evolution. If we abandon abstractions, modern art would still resemble the bucolic style of the 16th century, and software would still be in Fortran dialects. Functional programming and object-oriented programming are based on abstract concepts that allow them to express real-world problems in an understandable manner and create higher-order simplifications of the problem at hand.

• Beautiful code: With some references to Christopher Alexander's "unnamed quality," Coplien wants us to deliver beauty. The problem lies in how he characterizes beauty. His aesthetic criteria are vague. Beautiful code is not only visually appealing; what if we cannot see? Perhaps one has a test-sensing organ that is sensitive to testable code and code with a lot of tests - a testable flower and its petals. We all strive to write beautiful code, even in suboptimal languages, but let's be honest - we do what we can to meet our deadlines. Maybe we write truly beautiful code once or twice in our careers, but often, we write code that brings value. Value can be delivered not only by ethereal, elegantly crafted code but also by solid, practical code. While he advocates for the axiom of delivering value first, his subsequent request for beauty is in stark contradiction.

Delivery pipelines

Created on 2023-08-11 09:14

Published on 2023-08-11 10:18

The industry is buzzing currently with term as Continuous Delivery, Continuous Deployment an Pipelines. Pipelines are techniques to seed up the processing time by sequentially running a set of tasks on specialized stages, at every moment of time one step consumes the output of the previous one. The concept is not new, it has been started in the early 1900 when it was applied to factories and assembly lines. But the current pace of software development just put a lot of spotlight on them.

The software pipeline most often do a set of operations as check-out code from VCS, compile/build it, package it, run tests on it, deploy the artifacts to a repository and inform the stakeholders of the result, update various metrics and dashboards. These actions require a script to put things in the right order and check the results in every step.

In software the pipelines are generally defined through a specific language, either a visual one or a text based one - generally YAML or Groovy. These are run by specialized software as Jenkins, GitHub Actions, CircleCI, etc. The high level pipeline described in YAML is calling various other scripts that define the individual steps - like micro-operations.

The build step for example is the invocation of a Makefile, shell script or a Maven build. The pipeline will wait for the completion of the build step before passing the artifacts for testing or packaging. Such a situation is interesting because there will be two points of control in the pipeline. One at pipeline level and one at step level but their responsibilities will bleed into each-other.

This might lead to some problems:

1. Inefficient pipelines: If the result of a step can be guessed before the whole step completes the pipeline should fail fast. It makes no sense to wait for the completion. Imagine that you have a multi project build. In the trivial case the projects are built and packaged. But what if one of them has an issue? Should we wait for completion of all builds and then juts cancel the packaging step? What if there are modifications on a single project that has no egress dependencies? Again - it makes no sense to package all other projects as they haven't been modified. However with the split logic this is hard to achieve as the operations on pipeline level are quite coarse-grained. On the other hand most build tools (Gradle, MSBuild, ...) are perfectly able to run more parts of the process by themselves - create archives, containers, publish artifacts - and probably they should do it. Fail fast and incremental builds are essential for accelerated release cycles. The build tool is in a better position to understand what it is building by looking in the source code than the pipeline that is merely an orchestrator of some loosely coupled processes. Caching build results and artifacts, using parallel builds could reduce build times from 20 minutes to less than a minute - so some DORA metrics will look way better and will make both developers and managers less impatient. Speculative execution and rescheduling is well known in CPU pipelines - software engineers should have some inspiration from the clever solutions that hardware folks are successfully applying for more than two decades.
2. Portability: The high level pipeline logic is hard to be moved from one type of executor to the other. Pipelines built for Jenkins will be hard to be ported to another CI/CD system. The worst part is that it won't be easy for a developer to run the pipelines locally in order to have similar results as on the server. Dev/Prod parity should be not only on the tooling versions but also on the environment where code is built. Being able to run the pipeline locally would mean that a developer can also debug it if it's the case - it creates better visibility in the whole project - enabling a DevOps culture. Having an ops/build team that is managing the pipeline in secrecy is in my own view an anti-pattern. The devs will happily throw any issue away to the build/ops as "it works on their computers" - creating some knowledge towers. Also the posibility of running the pipelines on the local machines will ultimatly reduce the load on build machines and the queuing - improving again the metrics.
3. Mutability: The environment on which the pipeline runs should be immutable in order to produce consistent results. This is well treated by GitHub actions but for Jenkins (or other on premises CI/CD solutions) or even locally on the developer machine this is slightly complicated. However this can be solved if the pipeline can run in a container which is immutable. Many IDEs today offer the possibility of a development container that provides trusted and stable environments. This is extremely important nowadays in order to mitigate supply chain attacks. The immutability of the environment would mitigate issues as those described in Ken Thompson's paper from 1984 "Reflections on Trusting the Trust". This immutability contrasts partially with the caching needed for mitigating the speed but this can be solved quite elegantly nowadays with crypto methods so no cache poisoning can be inflicted.

Containerized pipelines solve both portability and mutability issues - so the developers could have both freedom of choosing tools and rigor for their builds in the same time. I learnt about a company that creates Visual Studio customized installations and pushes them every night to the developer's machines in order to solve the issues above. This is not only inefficient (hundred of gigabytes transferred and computers never in standby) but it is also error prone as a there are a lot of machine specific issues that might interfere - so in the end there is no certitude that the configuration is identical on all computers and that there is no drift. Imagine that in a large project a single library has some different settings - it will take hours for the developer to investigate an error caused by an obscure glitch that happened overnight. Running pipelines locally in the container would enable the developers use Rider or VS Code on Windows while still being able to test and build on trusted environments and deliver Linux software. Jenkins has this feature also, but is somehow exposing it in a clumsy way despite its huge value. Contrariwise GitHub Actions make it completely transparent for the consumers of the steps - one has to look in the action's source to know that it uses containers.

Speed is a crosscutting concern and can be addressed regardless if the pipeline runs or not in a container. A slow build won't get faster if done in docker. An aggregated approach - with the right tools could improve the development experience, speed and security of an organization.

#pipeline #containers

Legacy

Created on 2022-11-17 12:29

Published on 2022-11-19 19:56

The life that we are currently living is shaped in many ways by "legacy". In most cases, the word legacy carries positive connotations related to wealth, culture, and tradition. However, this is not the case with software. Here the word "legacy" has a lot of negative meanings associated with it. What is "legacy" in software? Why do we consider it bad?

Legacy software simply means that the code base is old, probably unmaintained, and hard to read. It says nothing about the value or the quality of the code, it just states that is old. Then why do we consider it bad? In the real world legacy is a source of wealth, something we grew upon. Probably the answer lies in our laziness. How many of us can fluently speak or write in Latin for example? Or classical Greek, old Norse, medieval French maybe? A few of us can, although is a pity, many fundamental works of mankind were written in those languages. Still, we do not read them in original, it is more convenient to read modern versions of those in a familiar language.

The same happens in software. Programmers are using contemporary languages and lost the ability to read older ones, so the code they stopped understanding became mysterious and potentially dangerous for them. As with classical languages, there is just of handful of people that have the patience to study and understand older code and those are the ones still able to explain the values that we, in our pride and ignorance, cannot see in the old code.

Many critical infrastructures and day-to-day codebases are "legacy", still, they work well, and they keep supporting our daily life. Banking still relies on Cobol and RPG, scientific computations still use Fortran, and operating systems still build on C.

I have to praise a relatively unknown "legacy" programming language and runtime: Concept. Concept is a 4GL (4th generation language) that started in Norway during the '80s. Googling it might yield 0 results, nevertheless, it delivers daily for a few million people. The language has all the features one would expect UI library, database connectivity, and it can run both server side and client side, it's kind of memory safe. The syntax seems a little dated but is still expressive enough to implement huge projects. As the community of Concept developers is not large it starts to lack new talent and also tooling. Despite this, there is still maintenance effort and is kept as much as possible in line with the latest industry trends: REST, JSON, MQs, 64bit code generation, containers, and such.

Understanding older languages is never easy. As I said before, due to our laziness we tend to ignore them and rewrite everything with no guarantee of a better job. Probably it is often a better idea to rewrite, but we first need to understand what we are replacing. We need tooling for understanding older code bases, especially when the original specifications of the software are lost. We need helper tools that guide us through the syntax and structure of the code and could provide handlers for the business logic already written in the old code bases enabling true reuse. Developing tooling that would transparently retarget old languages on new platforms will guarantee that the legacy received is still producing the expected results and we can build more interesting things in a more cooperative way. Rewriting software with feeble specifications or no specification at all, using the legacy system as a model but not fully grokking it, is in my opinion far more dangerous than keeping battle-tested code running.

The Five Ideals

Created on 2022-09-10 06:35

Published on 2022-09-10 14:15

Locality and Simplicity; Focus, Flow, and Joy; Improvement of Daily Work; Psychological Safety; Customer Focus - these are the five ideals of an organization that might blossom into a unicorn.

While some of them can be grown from the inside, starting from the development and operations teams and evolving them into a 'DevOps' culture, others are leveraged mostly by managers.

Helping teams improve for the first four ideals in the company creates more room for the latter. Those create a lot of turmoil and non-functional requirements but in the end, the price paid yields probably squared.

This is why managers that act toward the five ideals are as precious as mythical animals. This is also why when such a manager leaves it's quite sad.

To low code or to not low code

Created on 2022-01-31 19:43

Published on 2022-01-31 21:37

In 2005 I had my first encounter with a low-code platform. It was Alcatel's SCE/SDE later known as PrOSPer. It was a pretty capable environment for that period targeted for the generation of IN services composed of SIBs (Service Independent Blocks). Most of the IN abstractions were encapsulated inside the SIBs, the service creator was chaining the SIBs visually to create a new service. When it was needed a new SIB could be also created by implementing a set of interfaces needed for a SIB to run in its SLEE environment or to be seen inside the SCE/SDE interface.

The system generated C/C++ code, compiled it and then created the deployment descriptors. The idea behind it was a great one. It offered to domain experts the means for describing an IN service in a high-level language, with a good visual interface. Howeve,r there were some issues with it. The quality of the generated code was not optimal, the generator was a write-onlyy" one that missed any form of round-trip engineering, sometime the generated code had to be patched by hand and that made the high level description obsolete because the patched code couldn't be imported back in the SCE. Versioning of the code was also a nightmare in the SVN/ClearCase environment. All in all the developers were avoiding the SCE/SDE and were trying to handcraft their own code with simpler call-flows and better control over the implementation, keeping only the parts that were absolutely necessary to communicate with the SLEE.

The next low code experience I had was with Apache NiFi. NiFi is a flow based programming system that is used for all kind of data manipulation. It is again composed by a set of blocks that perform various operations on the data streams that were sent to their inputs. New blocks can be easily added or generic scripted blocks could be inserted so that the system is easily extensible. Versioning is still not great but is not terrible also, the system works pretty well but it's still missing some advanced features as meta-models, types and so on. There are also several other data-flow systems as NodeRed but most of them are in the same category of "write only" generators.

A major step forward was made by the Eclipse foundation with its EclipseSirius modelling workbench. This one was indeed based on higher level abstractions. Being constructed on top of GEF/EMF it had superpowers as reverse engineering, round-trip engineering, and easy creation of user interfaces that were not limited just to a data flow like systems before it.

The interesting questions that those low-code solutions raise is in the sphere of languages. What is a language? What are the entities a language operates on? What are the rules that make the language correct?

Professor Jordi Cabot reaches the conclusion (see slide 13) that low-code is in fact a fancy syntax and a marketing term used for some model-driven architectures and development environments. All the "low code" systems described are in fact visual syntaxes that describe the interaction between some entities (called blocks, or SIBS).

Having models and meta-models make the "low code" even more interesting as now other abstractions can be created. Constructions in the new language can have some formal semantics, type systems can be applied. Testing can be moved from generated code to the visual syntax itself. Probably, as also Eric Evans describes in its Domain Driven Design book the most important quality of the "low code" is that it enables the human interaction and comprehension. Maybe lawyers will not operate with visual law designing tools (although it would be interesting) but for sure they could grasp models of law, fact, proof, patent that would permit them to use a low code solution in their own bounded context.

So far the "low code" solutions I referred to were visual but this is not the only way of having it. There can be for sure low code textual languages or DSLs.

The above example encapsulates lots of domain entities: tax payer, taxable income - entities that would be pretty hard to be manipulated in normal programming languages due to the amount of boilerplate code needed. If we make another step maybe the situation could look like:

If we accept that a "low code" visual solution is just an alternative syntax for a DSL then we can link the two worlds. Most of the problems that were hard to solve in the visual syntax can be now rewritten in the DSL so we get many advantages:

• as before we can have type systems and meta-models
• we get an AST that facilitates transformations of the models
• we get sane versioning as textual representations are VCS friendly.
• we get somehow a human understandable/maintainable code

There are also two other aspects that also emerge here. One is the possibility of a "language engineering workbench" where domain specific languages to be created and augmented with thick semantic layers. This has huge applicability in domains where is a lot of formalism and already developed systems.

The other interesting one, that in my opinion has a major impact, is the "projectional editing" that would permit the language creator to manipulate the language internals easier, in some situations without resorting to traditional lexers or parsers as the language internals would be exposed as models in their own right. Modern language workbenches already offer great tooling for projectional editing making the development of DSLs easier.

What I am trying to conclude here is that "low code" is not a single term but rather a combination of paradigms so in my opinion evaluation of "low code" cannot be done in total separation from domain modeling and language engineering. My arguments are probably naive, but I see value in higher level systems description, although this is not always necessary or desirable. Higher level abstractions are beneficial not only in the human-machine languages but they are also great for human-to-human communication as they reduce the miscommunication. Well human engineered languages could result into better implementation.

Many thanks to Jennek Geels for introducing me to his concepts of domain modeling.

References:

https://modeling-languages.com/low-code-vs-model-driven/

https://martinfowler.com/dsl.html

https://martinfowler.com/bliki/DslBoundary.html

Dutch tax DSL: https://resources.jetbrains.com/storage/products/mps/docs/MPS_DTO_Case_Study.pdf

Language workbench: https://web.cecs.pdx.edu/~apt/onward14.pdf

Technology Grafting

Created on 2021-12-23 09:48

Published on 2021-12-23 17:01

Wikipedia defines grafting as: "a horticultural technique whereby tissues of plants are joined to continue their growth together". Grafting is often used for producing new varieties of fruits on older trunks that are more resilient or adapted to a climate.

When working with legacy software we can see modernization as grafting. The rootstock is the older technology - proven and battle-tested while the scion is a newer technology that will enhance the rootstock's qualities. The technological grafting needs two things: interfaces/protocols and encapsulation.

The interface layer ensures that the data can be understood by both scion and rootstock. If the rootstock uses some very specific encodings (EBCDIC, various other locale-specific charsets) then the translation between them must be consistent in both directions. Things seem to be easy with textual protocols (although there might be also some issues) than with binary proprietary protocols. If a fast bidirectional translation is not possible (very different protocols) then a middleware might be necessary but in this case, it would be best that this middleware

On the other hand, encapsulation permits the scion to maintain all its properties and continue working as if it would be still attached to its original trunk. When grafting a pear branch on an apple tree the pear branch will continue to function as pear and will not hinder the apple trunk. The same behavior should be present when grafting software components. However, this strong encapsulation requires the open interfaces discussed before.

One of the most interesting technological graftings I have seen lately is in the form of modernizing older web applications using micro frontends packed as web components. The idea is not a novel one, web components are one of the ways of composing micro frontends, but this has massive implications for all the functional and nonfunctional aspects of the application. Imagine old PHP or Struts code bases getting an instant makeup with nice web components that reduce the server load and break monolithic applications into simpler parts. The web components isolate the new developments from the old code base and will permit the evolution of the older code base.

Grafting techniques and patterns might prove very useful in all kinds of modernization scenarios as there are a lot of use cases for this. The use cases can range from, as I wrote above, UI modernization to more interesting ones like adapting 50 years old systems to web or IoT. If the results of grafting are successful then this might help the rootstock also evolve (like Autocad's migration to the web). Grafting permits the progressive evolution of the software without disruptions in base technologies used at a minimal extra cost. The complicated problem is to determine where is the best place to insert the scion and how to make it behave on the same security and functional constraints as the rest of the application.

Necessary formalism

Created on 2021-12-19 14:39

Published on 2021-12-19 16:02

Most of us have started with a small application that solved a business need. Then we had some success with it therefore we added functionalities, improved UI, made it multi tenant and so forth. In the meanwhile the code has grown and every "if" condition we added made it more complex.

We added "ifs" because we had to respond to changes created by customer requests, changing laws and policies, access to different markets. Some of us were smart and resorted early to design patterns but some of us continued with the old way of adding ifs... In the end application is huge and hard to debug because it's codebase no longer reflects it's initial purpose. There is no "isomorphism" between the problem that application solved and the implementation of the application. Even applications that were well engineered have this issue as the changes they accumulated over time are not described in the original universe where the application was spawned, in the business part, but rather the engineering created approximations in the implementation domain that are maintained. To make a forced comparison the situation is similar to getting the image of a forest in autumn simply by creating a huge palette of colours without thinking of what we are trying to represent with them.

So moving along this line, how do we keep applications from going adrift? How do we keep in sync the reality where applications emerged with their implemented behaviour? Here formalism appears. Describing application behaviour in a formal way, close to what people understand is the key. This formalism have many names ranging from "Domain Driven Design" to DSLs but all focus on capturing the domain information on a higher level, not directly in code. Code is generally a byproduct of those, formal descriptions are needed for any DSL and low-code solution because, quoting Markus Volter: "models are semantically rich for their domains" whilst code is not semantically rich, just behaviouraly rich.

One of the challenges I am thinking about is: how do we retrofit formalism into application so that we make them more maintainable? There is no definitive answer but especially for legacy application refactoring to models is a key. We should start small, refactor user interface behaviour for example to state charts (in form of Redux or XState). In the meanwhile the refactored interactions could be again reorganized both server and client side into some business process descriptions (it would be ideal to use a dedicated DSL for those in this stage - but this is generally hard, so refactoring to state charts is a good beginning). Iterating over this a couple of time we will reach better models on the domain and clearer actions defined on the set of the models - hence we get a crude formal definition of the domain. At this point the process, described probably in a specific language can be understood again by business people therefore they can evolve it while it is still generating lower level formalisms (as the statecharts) that developers or specific tools can generate code from.

The new developments nowadays have the chance of starting clean as the current tooling is quite advanced (Jetbrains MPS, Eclipse Sirius). New projects can start from DSLs or from BPMN visual representations and generate rich models that can be visually manipulated in low code solutions. Moreover a repository of well described models and processes can be shared inside a company so people can reuse the high level abstractions, manipulate them in their own code and getting interoperability and common behaviour for free. Also testing is positively impacted as tests clarity increases because models carry more semantics inside. What would be easier to test: 500 lines of written code or 12-15 lines of DSL or visual representation? Where are the errors simpler to catch?

Well crafted formal definitions reduce the implementation and design burden for the implementation team as they shunt most of the repetitive design and boilerplate tasks in an application (rich models already contain the logic that otherwise has to be expressed and maintained in code). Continuous effort of expressing behaviours in high level, formal languages and refactoring older code towards the same seems a way of increasing maintainability and quality. DSLs, modeling and low-code are in fact facets of the same base problem - how to capture and communicate succinctly, correctly and in a non ambiguous way domain knowledge both for people and for machines.

PS: There are better approximations of a process definitions than Amazon's StepFunctions or BPMN. To quote a friend - they are like GoTo programming and do not incorporate transactional aspects.

Communication and innovation

Created on 2021-07-29 15:23

Published on 2021-07-29 15:55

More than a year ago we were talking about backups. My solution then was something based on duplicity and rclone and I was explaining it to one of my colleagues who had another approach. I was pretty happy with it although it had some quirks. Then one of the other colleagues asked us: "Have you tried restic?". I did not know about it and I found it quite interesting. Months passed and I had to implement again an off-site backup system. Although I had the old tools in the pocket I remmembered restic and I soon realised that is the perfect fit for my incremental backup task.

I imagine that I would have lost probably some days trying to use other tools and craft an adapter layer on top of the old tools. Listening to others brang me pretty much advantage both in terms of efficiency as well as in terms of knowledge - another tool on my belt.

One thing that I miss while working remotely is the posibility of sharing knowledge in an informal way. When enginers discuss around the coffee machine some crazy geekish ideas (from Raspberry Pi autonomous lawnmowers to a new programming language) or they lucidly dream some interesting features in their code then innovation is about to happen. Many things that are in an incumbent state are hard to find in mainstream publications and blogs but sometimes there is one who has a passion about "obscure-ish" (eg. https://qconsf.com/system/files/presentation-slides/control_theory_in_container_orchestration.pdf) subjects and a wise remark at the right moment can cut months of development because that insight solves in a creative way an otherwise hard task.

I had my share of these happenings: timer_fd (simplified state machines), NancyFX (instead of all OWIN stuff), Swig (for genetrating bindings) and many others but the key fact was that I was among peers who gave me back at one moment valuable information. This is one of the things that videoconference/Skype/Meet still cannot substitute yet. They are fantastic tools for this time but unfortunately they are still unable to create the context for this kind of communication. I remember that many ideas that were later translated into patents emerged in this kind of informal gatherings and sometime a "What about..." created the spark.

I have also found an interesting article (https://theconversation.com/companies-are-trying-to-connect-remote-workers-with-virtual-water-coolers-but-its-harder-than-it-sounds-146505) that reaches almost the same conclusions - apps cannot really mimmick spontaneity and generate context for innovation.

Prepping

Created on 2021-03-12 21:41

Published on 2021-03-17 17:17

"Anything that can go wrong will go wrong" (Murphy's Law)

"Preppers" are persons that spend a lot of time preparing for an imminent disaster. In case of doomsday, they might survive it as they planned minutely for any possible outcome long before. Preppers are stocking equipment, clothes, food and often create shelters so that they could face even the most atrocious situations with relative calm. Prepping puts some financial strain on people but not as much one might think as preppers' most important assets are knowledge and improvisation. Regardless how elaborated the disaster recovery plan is, it's never perfect so creativity should always be welcomed.

Preppers use some techniques that IT should learn from:

1. Rehearsal - preppers really exercise their plan. They try surviving wilderness, with their bushcraft skills, thus testing their theories and equipment. In IT from time to time it's worthy simulating a nasty situation in the infrastructure. It doesn't need to be a complicated one that would imply the whole SimianArmy but at least simulate a disaster, solve it and take notes. Ask colleagues to stop a service or make a ethernet loop in a switch. Learn where to look for clues and when the incident is solved try to automate the solution. There are organizations that take rehearsals very seriously. I've been part of probably three fire simulation drills at Adastral Park in less than a month but I can say that It was the only occasion where I've seen so much calm and organization as people knew their drill.
2. Share information - preppers are sometime organized in clubs and they share lots of relevant information on materials and techniques. Community driven DRP might be interesting as people come with different viewpoints that might reveal flaws or provide better solutions. Information sharing might be the most important source of learning.
3. Use cheap and easy to find materials. For example many things can be simulated on scaled down versions of the real systems. A full K8S/EKS cluster can be simulated with a petty RaspberryPi running K3S. Most of the things will be the same but costs will be negligible. The differences can be documented and there are also lots of third parties that can simulate other services cheaply (Lambdas, Object Storage). In some situation one will have to improvise massively to recover from a disaster and then knowing what to do with almost off-the-shelve components becomes invaluable.
4. Invest in understandable systems - preppers try to understand nature and mechanisms so that they increase their survival odds. Although automation is a must it should be transparent to the engineers and clearly present what's behind the scenes. If the state is extremely bad and automation magic stopped working then understanding the system might be handy at least to go back in a restorable state with minimum effort. I for one was in a similar situation when I discovered that the backup I made had to be restored on a disk with different physical characteristics. Knowing what was the backup process I was able to hack the restore on the new drive although the tool was not supporting it.
5. Keep stashes - preppers often have hidden stashes of food and tools around the house. Engineers should also make stashes of resources if possible offsite or in different clouds. If something goes horribly wrong at least some things could be partially running. An OCI solution, replicated on a different vendor's platform, would make a lot of sense so if a provider is unavailable restoring the solution on another would not mean an entire rewrite but rather a reconfiguration. Vendor agnostic IAC solutions and CNI packed services are handy in this case - they can be easily run on different environments.
6. Have spare batteries - preppers don' t rely on a single centralised solution – e.g. electricity; they always have alternate independent energy sources that they can use - batteries, generators, etc. In the light of the recent OVH fire let's imagine the following scenario: the company's automation solution (Chef, Puppet, Octopus) is also affected somehow then the situation will be similar to electricity or transport failure. On the other hand, having the IAC implemented through standalone tools (Terraform, Ansible) increase the resilience as there is no longer a single point of failure, any engineer could replicate the infrastructure from his/her own machine as long as one can connect to the datacenter/cloud.
7. Learn how to use and build tools - preppers spend time in learning how to build bows, arrows, tents and also how to use them. Off the shelf tools are fine (backup/restore, IAC, etc) but sometime one needs to do specific operations (e.g. recover data from an ZFS formatted drive). Then building small and specific tools would be invaluable so a programming language that permits rapid development is handy - currently I see that people are looking towards Python and Go.

What I am trying to say is that pessimism and preparation are key for survivability in case of a disaster. Even a digital one.

 

Chan vs Sama

Created on 2020-05-01 08:42

Published on 2020-05-01 10:03

The way one addresses others is very much culture dependant. This can range from the extremely formal Japanese expressions in the domain of "chan" (dear little) to "sama" (most honorable), to less formal languages as English where addressing is somehow less strict - although British subjects don't use "Dear Queenie/Hi Lizzie" nor Japanese start a letter with "Naruhito-chan".

So what are the limits of politeness/familiarity on LinkedIn? How does one starts an invitation to an unknown person? What about a letter to a known one? I am not a native English speaker so most of the formal English knowledge I have comes from books, but so are many other people that use English for business purposes. I have learnt that every language has several functional styles that are used in diverse situations - science, law, newspapers, literature, business. In my own view LinkedIn is a business environment where business partners meet. They can meet for the first time or they might know each other for long, but still they are in a business environment - therefore a business functional style should be adopted on this kind of communication.

The first action inside the network is the invite. How would you invite another person to join your network? Do you know him/her in person? Does he/she knows you? It's a pity that LinkedIn permits empty invitation messages because they reduce the empathy levels between those who want to connect. Very seldom people bother to write personalised invitation messages and explain their goal for connecting. This is especially true for some headhunters who expect automatic acceptance of their empty invites. To be a little harsh - candidates are expected to send cover letters, wouldn't it be fair to ask the same from recruiters?

On the other hand there are people who write something in their invite messages. This can range from very formal "Dear Sir/Madam" to the "Dear Robert". This is very much country/industry dependent but as a rule it's better to be more formal when you don't know the person you are talking to.

In the meanwhile there are over-friendly invites - I for one received "Querido Danielito" and "Dragă Dani" from persons I have never met. Are those okay? I would hardly think so. There are many LinkedIn etiquette related posts online and most of them are common sense rules that help one find appropriate tone and style. As an exercise of imagination - how would Lincoln start the famous "Bixby Letter"? With "Dear Madam" or "Hi Lydia". Indeed some things changed but some of them not so much and although globalisation relaxed the norms it is still okay to use a business-casual style on written documents. I am coming from a cultural background where people address each-other on their given name only after some mutual agreement (see the last paragraph of this article) and hence I do appreciate an initial degree of formalism. I am not absurd and I understand that "Hi/Dear Daniel" is acceptable but from there to "Dani" (which is not by the way a short name that I use) it's quite a lot of way to go.

So, culturally speaking, it is better to do a small research before starting a letter and choose the apropriate tone, style and expressions. Because: "Utlänning - Främling - Ramen - Varelse"

Easter date differences

Created on 2020-04-12 07:22

Published on 2020-04-12 08:20

Many of my colleagues in Germany, Netherlands and US are surprised by the fact that there are two Easter dates therefore two disjoint sets of vacation days. One for the western, Catholic world, and one for the eastern, Orthodox world. The explanation for this is in fact very interesting and comprises a lot of mathematics and history.

The Catholic Easter is computed as the first Sunday after the first full moon on or after the March Equinox. Seems simple but in fact this is just a little bit more complicated. The synodal moon period is 29.530588 days hence there is not a constant day of month for occurrence, the first full moon Sunday after March 20/21 could be as late as April 18.

The western world started using Gregorian calendar from 1528 but the Orthodox countries kept using the Julian calendar that is 13 days behind. The formula is basically the same but there is the Julian offset to be counted in. Moreover Orthodox Church has a third condition, given by the 1st Ecumenical Council of Nicaea held in 325, that states that the Easter should be after the Jewish Passover (Passover is on Nissan 14th, date computed on a lunar calendar). As Passover is one week long and is computed similarly to the Catholic Easter this can push the Orthodox Easter date to May. The difference between the two can be up to a month. For example if in 2020 the Catholic Easter is on April 12 and Orthodox Easter is on April 19, in 2024 the dates will be be March 31 and May 5. However, sometimes the two dates could be equal, last time in 2017 and next time in 2025.

Now for the geeks, computing Easter date is a very interesting coding kata. Implementations rely on moon phase determination (math) and the Gregorian/Julian conversions (math and history).

Java and .NET provide pretty good mechanisms and there is a plethora of libraries for other languages. The calculation of Easter date was called "Computus" in Latin (https://en.wikipedia.org/wiki/Computus) and might be the etymon for Computer, hence the Computer Science has a very interesting origin. The wiki page offers quite a lot of information and starting points (Gauss algorithm, tabular representations, ...) so during self-isolation boredom one can try something interesting. As a side note, the Julian calendar seems to be related to the works of St. Dionysius Exiguus, a monk born in present-day Romania, so probably the bugs in the Julian calendar are also his fault.

The Minitel Paradox

Created on 2020-03-27 07:06

Published on 2020-03-27 07:47

Minitel was great. It was internet before internet. I was envious on Minitel as a kid and as a teenager because I was able to read about it therefore I was dreaming of using it someday. Minitel was an tightly integrated solution that was sized precisely for its purpose. The terminal, the access network, the services were perfectly matched together. The overall usage experience was great.

In the meanwhile, before web, there was a myriad of BBS solutions, more or less reliable, more or less updated. The experience of using them was ranging from "meh" to awful. But they were not integrated at all, they had no specific access network, terminals and servers could be run on any hardware. It was mostly patchworks but the system worked. Then the "black swan" event happened and, in almost no time, WWW and internet emerged globally. The BBS systems rapidly evolved in browsers and websites, access networks transitioned from dial-up to fast FO or 4G. And everything evolved to what we have today.

The highly integrated Minitel stayed the same in the '90s and after, not being able to keep up with the rapid developments of web. Sadly, it was finally shut down in 2012. Minitel's flaw was that it was too well engineered. That it had perfectly fitted parts. But when everything works as expected there is no drive to change it. It is better to have in some situations sub-optimal parts but clear and open interfaces because this forces evolution. And evolution leads to survivable systems.

This was a long intro. My story today is about very skilled friend that created microservice templates with everything built-in. They were batteries included, as Minitel once was. Everything was inside, whether one was using it or not and integration in the system required that every microservice having the same structure. A perfect match. Almost as good as Minitel. My opinion on his idea was: "Let there be chaos". Create interfaces and let service developers hook into them as necessary because it is hard to control business needs and technology stack evolution. Hence both implementations and interfaces can evolve separately. It would be an evolutionary patchwork as with internet. The only restriction is to keep interfaces open. His templates were fantastic. Out of the box messaging, smart logging and metrics, Kubernetes ready - checked, any nowadays technology was there with a clear scope - current business needs. But, as I remarked to him - current business needs, interfaces and contracts were defined inside template not outside of it, changing an interface required to rebuild all the services or creating many adapters that in the end will require more mapping an translation logic.

Coming back to my debate, on a smaller scale keeping his solution will work flawlessly for a while but it won't evolve. It will freeze the technology stack and it will break "Favor composition over inheritance" dogma - because the system will hardly find spare parts that fit together and can sustain the evolution. Also skill sets evolve from ANSI control codes in BBS to React. It's uncommon to find ANSI gurus in the younger generations.

If we raise the question "How would a Minitel world respond to COVID-19 situation? Home office use cases, collaboration, and so forth?" Would it be even possible? My guess would be: NO. Because the associated costs of changing a tightly integrated solution would be times higher than changing parts of a heterogeneous and open solution (imagine the costs of changing every terminal and link at a global scale).

Engineering systems should find balance between perfect and perfectible and accept perfectible yet incomplete components and as architects we should deal with evolvability as one of the important NFRs.

Interestingly, Minitel started evolving after it's shutdown in 2012 when open innovation was possible: https://opendna.com/blog/2012/06/30/the-end-of-minitel-its-past-and-our-future/

AlgoCheating

Created on 2019-11-25 06:12

Published on 2019-11-25 07:00

I had several interviews at tech giants. Some of them were successful some of them not. The main characteristic of the interviews was the focus on algorithms - and this is for a good reason as data grows you will need faster algorithms to partially keep up with data. In that period I studied mainly from the classical books (CLR, Skienna, ...) and solved problems that I knew that were given in the local IOI/CACM qualification rounds. I was able to do recursion, graphs, data structures and concurrency due to my individual study and also due to my university subjects. It was a decade long process. Getting to an interview was a huge deal then and required quite a lot of work.

In the menawhile things have changed a bit. Now it is pretty easy to get to an interview but more or less the focus on algorithms remained unchanged. As of today I see lots of sites, books and apps that promise instant learning of algorithms and data structures. They present both some theoretical parts and also some solutions to well known interview problems. On top of that they are also offering mock interviews to the candidates who want a job at one of the unicorns. On another scale there is also an entire industry of courses and "academies" that also promise fast learning of some CS subjects to people who want a career change

As an idea these are not bad things but I am skeptical about their consequences. The first issue I see is that they will create robotic candidates that will be able to mechanically reproduce only the content in the book "Supercharged Algorithms" (fictional title) or whatever they have purchased. These recipe-for-algorithmic-success books are in my own view just a cheatsheet for passing the interview. The second issue is the mock interview. Practicing it is okay but the risk is for a company that adheres to this type of interviewing to hire mostly good actors as it becomes easier for candidates to mock the desired behaviour. The third issue is that learning mechanically some technology is a recipe for disaster on the long term. It's "programming by coincidence" type of behaviour where one who graduated a fast track course on programming will apply what he/she has seen in the course. What it lacks is the element of metacognition, of critically thinking about the problem, context and solution. This is what a "Supercharged Algorithms" or "Java Academy" cannot teach in one month.

There are some major consequences of these. One is that the companies get fooled and hire based on memorised solutions and acting. The results will be seen in about a decade. The second issue is that this "fast track training" industry is undermining the academic degrees. Why learn CS for 3 to 5 years when you can get similar results with 3 to 5 months courses? Of course that there are brilliant people in CS that haven't majored in CS but most of them are self taught. They skilled up in years not in days. Most of these people have solid backgrounds in other sciences as Physics or Mathematics and they were able to apply abstract thinking in their new field. I am very much inline with what Steve Klabnik said:

"To make a terrible analogy: nobody expects plumbers to have a physics degree but they do have to know some things about water physics, and that can be learned in a way that doesn’t necessarily involve getting a physics degree. And that is super cool, totally valid, and not a problem. But that doesn’t mean that physics degrees are bullshit or not useful to plumbers"

What can be done to circumvent these? In my opinion the most important thing is to detect the "fake". Candidates should write code together with the interviewer, should be asked meta questions, cross checked their knowledge against other field (when someone says that an algorithm complexity is O(n) it would be a nice follow-up to ask how one demonstrates this mathematically - it would prove some basic calculus skills). Breaking one's script and making him/her think outside the box would be also a good way of discovering both potential and meta cognition.

I will finish in a bitter note though, as I see that the industry is going mostly for cheap and universities don't fight back in a determined way. It goes much in line with E.W. Dijsktra said: "So, if I look into my foggy crystal ball at the future of computing science education, I overwhelmingly see the depressing picture of "Business as usual". The universities will continue to lack the courage to teach hard science, they will continue to misguide the students, and each next stage of infantilization of the curriculum will be hailed as educational progress." Worse, the same infantilized curriculum is taken out from universities and is further stripped down of the little science it still has on the bones, and then, is sold as a panacea for successful interviews.

The Programmer's Toolchest

Created on 2019-07-02 14:33

Published on 2019-07-03 16:14

One of the most inspiring images is the above from the "Fine Woodworking" book back cover. It is emblematic to illustrate the attention for quality tools that ease the work professionals in some domain have. On my home DIY attempt I also tend to buy better tools as I find them easier to use, I got spare parts and they generally last longer in workshop conditions.

When it comes to software the things get more complicated as there the expensive tooling is not necessarily the best one. Clearcase or TFS although expensive do not do a better job than Git that is free. Neither MSVC compiler does a better job than GCC. So it becomes hard to choose a good tool for one's toolchest.

I generally select software tools after some time playing with them and I always weight up both the qualities and the price of the tool. I have many free tools in my collection but there are also some that cost, although there are also free alternatives for them. I have worked many years with some free tools before changing to non-free ones or to other free tools. Among the qualities I appreciate in a tool are:

1. Ease of use - It makes no sense to have a tool you cannot operate or if yo can operate it is not comfortable for longer usage. It would be like continuously using Jacques Carelman's contraptions. So before adopting a new tool I play with it a couple of weeks to check how it works on my system, maybe on another operating system. Then I try accomplishing both a simple, routine task with it as well as a more complicated one. Sometimes the decision is easy: VS vs. Rider but sometimes is damn hard: Eclipse vs. Idea, HG vs. GIT. In every situation have a tool that you know well and handle it perfectly. Know it's options, config files, keyboard shortcuts.
2. Community - the more people are using it the better is it. Most of the time this is true, although there can be some more obscure tools or libraries that accomplish the job better or cheaper. Sometimes is wiser to keep a look on the second runner and also make some investments on them just to protect from the hype. AngularJS anyone today? It was great some years ago. Flash? Crowd is generally right but it can be also manipulated. So again, testing the tool and comparing it with alternatives helps a lot. If one can also read the source code, there are some important hints about the heath of the project.
3. Coverage - I have my favourite tools for every aspect of software development: design, coding, testing, packing, operating. In every aspect I like to have a handy tool that ease my work and that, ideally, can be used to automate tasks.
4. Specialized/Generic - as with power tools there are some tools that can, as a cordless drill can drill and screw, do several things. One of these tools is VS Code - you can do almost anything in it, a feature that only Emacs had. But VS Code still lacks behind other specialised tools, for example PyCharm when it comes to managing projects, libraries, virtual environments. So it's good to have both in portfolio and use them according to the actual needs.
5. Extendable - I have mentioned Emacs before. It was the first true extensible tool. It could be customised from a Tetris game to a high end Prolog environment. Nowadays most of the tools follow the same trend and they can be adapted to get more functionality that initially designed for. I'd rather use VS Code to VS as the former one can be customised in so many ways while the latter one is quite rigid.
6. Self-contained and small - I do not want to install 32 Gigabytes of libraries and frameworks just to run a single application. And I also do not want that the system gets bloated and unstable just because I had to install a design tool.

In the end how much should one developer invest in the tools used? To make again an analogy with the tools one buys for himself I consider that it would be fair to invest the same percentage of money in software tools as in DIY. If one can use the tools for freelancing or project work then this expenses can be deducted so in the end it really make sense to have a rich tool chest. Buying software or making a donation to an open source project really helps the creation of better/nicer tools as the money covers the countless hours of research we generally take for granted. Think about the quantity of research behind LLVM or Eclipse's EMF...