Taking Decisions Like a Court

In 2022 I wrote about taking decisions and the steps which come before you select an option: timing, purpose, stakeholders, requirements, options and their assessment. This article looks at the same topic from the other end. It’s about how you check a decision and write it down, so that other people can follow it and, if needed, challenge it in an orderly way.

For this I borrow from the way lawyers work. Courts and public authorities have refined a method for this over a long time, and much of it carries over to architecture boards, steering committees and any other group which decides on behalf of others. You’ll get a model in six stages, some rules for the meeting itself, a fast track for decisions which can be reversed, and the places where the comparison ends.

The legal concepts come from German law. Other legal systems have similar tools, but this article does not compare them. All examples are constructed scenarios, and most of them follow one decision about data and AI, which I’ll introduce below.

Why Look at Courts?

You might wonder what an enterprise architect can learn from judges. The point is this: a court has to decide every case which is brought before it, also when some facts remain unclear, and in many cases a higher court can review the decision afterwards. An architecture board is in a similar position. It cannot wait for complete information, and its decisions are questioned later, sometimes by people who weren’t part of the discussion.

Before we dig into the stages, let’s have a look at two ideas from legal theory which shape the whole model.

The Method Checks the Decision

In 1970 the German legal scholar Josef Esser described how a judge’s preliminary understanding of a case shapes the result and even the choice of method. A common reading of his work: judges often start with a sense of the right result, and the legal method serves to check whether this sense holds up. Lawyers therefore distinguish between how a decision is found and how it’s justified.

For our purposes this means that intuition is allowed. An experienced architect will often have a favorite option after the first meeting, and that’s fine, as long as the favorite has to pass a structured check before it becomes a decision.

Defensible and Properly Reached

When a German administrative court reviews a decision which left the authority room for discretion, it checks whether the authority used its discretion without errors. Whether another result would have been better is left to the authority. The sociologist Niklas Luhmann went one step further in 1969 and described how decisions gain acceptance through the procedure which produced them (Legitimation durch Verfahren).

Let’s say the weather forecast gives a 70 percent chance of rain, so you take your umbrella, and it stays dry all day. The umbrella turned out to be unnecessary, yet your decision was sound, as you looked at the forecast and drew a sensible conclusion from it, which is all you can do without knowing the weather in advance.

The same holds for architecture decisions. If someone says that an architecture board “always decides wrong”, the word “wrong” rarely fits, as you can seldom judge the quality of a result before it has played out. You can judge the procedure and the reasons, though, and that’s where the model below helps.

The Example

The example which runs through the stages looks like this: a company which sells machine parts to other businesses has 120 sales representatives. The sales department wants an AI assistant which prepares customer visits. It should answer questions such as “What has this customer ordered in the last two years, and which complaints are still open?” from the data in the CRM system. Two options are on the table. Option A is a prototype which an internal team has built: it reads the CRM data through the company’s central data platform and sends it to a hosted large language model, and 20 sales representatives have used it for two months. Option B is the AI assistant which the CRM vendor offers as part of its license, and which reads the CRM directly. Now the sales department asks the architecture board to approve an assistant for everyone.

The Model in Six Stages

The stages follow the structure of a legal examination. First come admissibility and the question, then the criteria and the facts, and last the assessment and the reasons:

1. Admissibility: Is this ours to decide, and is it due now?
2. The question: Who wants what from whom, and on what basis?
3. Criteria: Which criteria apply, and in which order?
4. Facts: Which facts are disputed and relevant?
5. Assessment: Does each option meet each criterion?
6. Decision and reasons: What is decided, why, and when is it reopened?

Two more parts complete the model: safeguards for the meeting, which apply at every stage, and a fast track for decisions which can be reversed. This is what the whole model looks like:

Stage 1: Admissibility

Before a court looks at the substance of a case, it checks whether the case is admissible. Is the court responsible, does the claimant have a legitimate interest in a decision, and were form and deadlines respected? In general terms: first clarify who may decide and whether a decision is needed now, and only then talk about the content. Quite a few decisions which later count as wrong were taken by the wrong group or at the wrong time.

In our example, the architecture board can decide on the platform and on the integration with the CRM, but it cannot decide alone. The CRM holds names and contact details of people at customer companies, so the data protection officer needs to be involved. In Germany, the works council has a right of co-determination when a company introduces a technical system which is suitable for monitoring the behavior or performance of employees (section 87 of the Works Constitution Act), and an assistant which logs who asked what will usually fall into this category.

The timing is worth a look, too, as there are two decisions to take. The pilot license for 20 users runs out at the end of March, so the board has to decide by then whether the pilot continues for another three months. The rollout to all 120 sales representatives is planned for the sales kickoff in July, so this decision is due by June. The first one is a candidate for the fast track, which I describe further below. The stages below follow the second one.

Stage 2: The Question

German law students learn to start every civil law case with the question “Who wants what from whom, and on what basis?” (Wer will was von wem woraus?). The answer forces you to name the parties, the request and the rule which might support it.

For a board this means: write down the operative part of the decision before the discussion starts. What would be decided, which options are there, and on which basis? Without a written motion, a discussion tends to drift. One person talks about the model vendor and the next about hallucinations, and both may have a point, but the meeting does not get closer to a decision.

In our example, the sales department wants the architecture board to allow the assistant to read CRM data for all sales representatives, based on the company’s AI policy. A first draft of the operative part could look like this:

The AI assistant for sales goes live for all 120 sales representatives on 1 July. It reads the CRM data of the customers assigned to the user.

With this sentence on the table, the board can discuss each part of it, and as we’ll see in stage 5, the draft does not survive unchanged.

Stage 3: Criteria and Their Order

Law has a hierarchy of norms, from the constitution down to local regulations, and rules for collisions. The higher norm beats the lower one (lex superior), the more specific norm beats the general one (lex specialis), and the newer norm beats the older one (lex posterior). Where a norm is unclear, lawyers interpret it by its wording, its context, its history and its purpose. If the wording reaches further than the purpose, they may even apply the norm more narrowly than its words suggest, which is called teleological reduction.

For a decision this means that you set the criteria and their order before you assess the options. Otherwise you will be tempted to choose the criteria which fit the result you’d like to have.

For architecture principles it means that you also need a rule for the case in which two principles collide. TOGAF asks for part of this, as the rationale of each principle should describe situations in which it takes precedence over another one. A collision rule for the whole set saves you from negotiating each pair of principles on its own.

In our example, two principles collide: “buy before build” and “all access to business data goes through the central data platform”. Option B fits the first one, but it reads the CRM directly and bypasses the data platform. A collision rule such as “laws and contracts first, then data principles, then sourcing principles” settles this conflict before anyone has rated an option, and it puts the data platform principle first.

Stage 4: Facts

I think this is the stage which carries over best. German civil judges learn a technique called Relationstechnik, and one of its rules is that the court only takes evidence on facts which are both disputed and relevant to the decision. Everything else can remain open. Before any analysis, the question to ask is therefore: would a different answer change the decision?

Imagine the board spends two meetings on the question which of the language models behind options A and B scores better in a public benchmark. If both models meet the quality bar the sales department has set, and the decision depends on data access, the benchmark will not change the outcome and can remain open. Whether the assistant respects the permissions of the CRM is a different matter. A sales representative shouldn’t see the customers of another region. The data platform team says that option A takes over the permissions of the CRM, and the security team has doubts. This fact is disputed and relevant, so it’s worth a test with real user accounts.

Courts also scale the standard of proof to what is at stake. Normally a judge needs to be fully convinced. In urgent proceedings it is enough to show that a claim is credible (Glaubhaftmachung), and for the amount of damages the court may estimate (section 287 of the Code of Civil Procedure). For the pilot with 20 users, the statement of the data platform team may do. For the rollout to 120 users, the board needs the test with real accounts. The monthly costs can be estimated from the usage in the pilot, as nobody needs them to the euro to take this decision.

A court also cannot end a case with “we don’t know”. If a fact can’t be cleared up, the burden of proof decides who loses. A board benefits from the same rule, agreed before the discussion starts: whoever wants to deviate from the standard carries the burden. Let’s say the sales department prefers option B, as it comes with the CRM license. The standard is the data platform, so the sales department has to show why the platform doesn’t work for the assistant, and if it cannot, the standard applies. The uncertainty is then settled by the agreed default and does not lead to another postponement.

Stage 5: Assessment

In the assessment, lawyers check whether the facts meet the conditions of a rule, which they call subsumption, and they weigh interests where the rule leaves room for it. Three habits from this stage are worth taking over.

The first one is to check one condition after the other, also when the result looks obvious. It’s the best protection against arranging the facts until they fit the result you’d like.

Let’s say the company’s AI policy says that a use case needs a review by the data protection officer if it processes personal data and if its results are used for decisions about individual people. In the meeting, everyone agrees that the sales assistant deals with companies, so the policy doesn’t apply. Checking the first condition on its own gives a different picture: the CRM holds names, phone numbers and email addresses of contact persons, and the free-text notes contain remarks about them. So the first condition is met. The second one isn’t, as the assistant prepares visits and does not decide anything about the contact persons. The policy does not require a review, yet the check has brought the notes into view, and they turn up again in the next step.

The second habit is the proportionality test, which German courts apply in a fixed order. A measure has to pursue a legitimate aim, it has to be suitable to reach it, and it has to be necessary, which means that no milder means would be equally effective. Only then is it weighed for appropriateness. The first three steps are cheap yes-or-no filters, and the expensive weighing comes last. Many weak options already fail at “necessary”.

In our example, better prepared customer visits are a legitimate aim, and the assistant is suitable to reach it. Does it need to read the free-text notes, though? If the sales department confirms that order history and open complaints are what a representative needs for a visit, reading the notes is not necessary. The notes drop out before anyone has to weigh privacy against convenience. Data protection people will recognize the principle of data minimization in this step.

The third habit is a self-check which comes from the review of discretion. German administrative law knows three typical errors, and each of them has a counterpart in a board meeting:

• The authority didn’t use its discretion at all. In a board, an option is taken because it’s the default, e.g. the CRM vendor’s assistant because it’s already part of the license, and nobody has weighed it.
• The authority went beyond its mandate. In a board, the architecture board settles how the assistant’s logs may be evaluated, which is a question for the works council.
• The authority was guided by considerations which have nothing to do with the purpose, or it overlooked relevant interests. In a board, the team which maintains the CRM data wasn’t asked, although the assistant’s answers depend on the quality of that data.

Lawyers also know the case in which discretion shrinks to zero, because only one decision remains lawful. If the framework contract with a large customer forbids processing its data with AI services, there’s no choice left for this customer’s data. The board should say so and exclude the data, without weighing options it does not have.

Stage 6: Decision, Reasons and Finality

A court decision separates the operative part from the reasons. The operative part is short and can be enforced as it stands. The reasons explain how the court got there, written so that a higher court can check them.

For architecture decisions, the reasons are what an architecture decision record (ADR) contains. Michael Nygard’s widely used template from 2011 has five sections: title, context, decision, status and consequences. It has no place for the assumptions on which the decision rests. Longer templates, such as the one by Jeff Tyree and Art Akerman from 2005, do list assumptions. What I’d add is to treat them as conditions: if one of them no longer holds, the decision is reviewed. German contract law has a term for this, the basis of the transaction (Geschäftsgrundlage). If it falls away, a contract can be adjusted.

In our example, the operative part and the conditions for a review could look like this:

The AI assistant for sales (option A) goes live for all 120 sales representatives on 1 July. It reads the order history and the open complaints of the customers assigned to the user through the data platform. It does not read the free-text notes.

The decision is reviewed if the costs exceed 30 euros per user and month or if the permission model of the CRM changes.

Decisions of German authorities become final after a while, and a closed procedure is only reopened for defined reasons, such as a change in the facts or new evidence. For a board this means that a decision stands until one of its assumptions fails or a procedural error comes to light. This stops the same question from coming back in every meeting, without making decisions last forever.

Courts also have to deal with their own earlier decisions. At the Federal Court of Justice, Germany’s highest court for civil and criminal cases, a panel which wants to depart from another panel’s ruling on a point of law has to refer the question to the Grand Panel (section 132 of the Courts Constitution Act). The deviation is possible, but it needs a procedure and reasons. If another department in our company wants an assistant on HR data next year, it can take a different route than the sales assistant, provided it explains why the reasons of the earlier decision do not apply.

Procedural Safeguards

Some rules from court procedure apply to all six stages, and they carry over to boards and meetings almost unchanged.

Everyone affected is heard before the decision is taken. Informing them afterwards is communication, and it comes too late to change anything. In our example, this means the sales department, the data protection officer, the works council and the team which maintains the CRM data.

A judge who might be biased steps back. In a board, whoever has a personal stake in the result says so or doesn’t vote. The architect who built the prototype has done nothing wrong, but the board should know about it before the vote.

In German collegial courts, one judge acts as rapporteur and prepares a written proposal with reasons (Votum), and the deliberation starts from this text. A board works the same way if one person prepares a written proposal along the six stages. A language model can help with the first draft, as long as the rapporteur checks every reason in it.

The Courts Constitution Act also sets the order of voting, in its section 197. The rapporteur, if one is appointed, votes first, the judges follow from the most junior to the most senior, and the presiding judge votes last. This protects against a common effect in meetings, where people adjust their view to the opinion of the most senior person once it’s known.

At the German Federal Constitutional Court, a judge can attach a dissenting opinion to the decision, while other German courts keep their deliberations secret. For a board, the constitutional court is the better model: write the dissent into the record, so that it’s at hand if one of the assumptions fails later.

The Fast Track

Six stages and five safeguards would turn into a recipe for bureaucracy if you applied them to every decision. Courts have an answer for this in interim relief. The court only takes a summary look at the case, it’s enough to show that a claim is credible, and the court weighs the consequences of granting or refusing the request. Decisions which can be reversed get the fast track, and only those which can’t be undone get the full procedure.

You may know Jeff Bezos’ distinction between one-way doors and two-way doors from his 2015 letter to Amazon’s shareholders. The law adds a sharper criterion: interim relief must not anticipate the main decision (Vorwegnahme der Hauptsache). The question is whether a provisional decision creates facts which cannot be undone in practice.

The pilot extension in our example shows the difference. The board can switch the pilot off at any time, so it looks like a two-way door. The customer data which went to the provider during the pilot can’t be called back, though. Whether it’s deleted depends on the contract and on the provider keeping to it. And after five months of use, 20 sales representatives have changed how they prepare their visits, which makes switching off harder than the license suggests. So the fast track fits the pilot extension, provided the board limits which data goes to the provider while the main decision is open.

Where the Transfer Ends

The model works best for decisions which apply goals that are already set, and it is weakest where the goals still need to be set. There are three reasons for this.

The law provides the norms, while a company often has to define its goals first. In this situation the board is a legislator more than a judge, and stage 3 becomes the main problem.

Courts mostly judge what happened in the past, while companies decide on forecasts. The law does know forecasts, e.g. when the police assess a danger or when the legislator estimates the effects of a law, but courts review them with more restraint. This suggests that the method carries less weight for questions about the future.

The law aims at legitimacy and equal treatment, and speed comes second. If you adopt the model, take the fast track seriously.

Conclusion

Neither a court nor an architecture board can promise that a decision turns out well. What both can do is reach decisions in a way which others can check: the right group at the right time, a written question, criteria in a fixed order, only the facts which matter, one condition after the other, and reasons which name the assumptions that would reopen the decision. For decisions which can be reversed, the fast track keeps the effort small, as long as you check whether the provisional decision creates facts of its own.

The steps which come before all this, from timing to the assessment of options, are in my 2022 post on taking decisions.