The Courtroom of Code: Why Judging Became the Hardest Part of an AI-Era Datathon

From the Engine Room to the Courtroom

In Part 1 of this series, I talked about the Two-Hour Trap and why we had to move from Difficulty through Implementation to Difficulty through Synthesis. In Part 2, we went under the hood of our Synthetic Data Engine: the Payday Pulse, the ATM Personas, and all the intentional friction we built in to keep things AI-resilient.

But here’s the thing nobody warns you about. Once you’ve built a messy, lifelike arena and 50 teams have charged into it with AI coding assistants at their side, you run straight into a whole new problem: How on earth do you grade it all?

Think about what that actually means. Fifty teams across three tracks, each turning in code, data pipelines, ERDs, chatbots, collaboration logs, and a written report. In the old hackathon world, teams spent most of their time building, and the output was small enough for a panel of judges to get through. In the AI era, that flips. AI made the building cheap. And when building gets cheap, everyone builds more, faster, and the pile of things to evaluate gets huge.

The bottleneck moved. It moved from the engine room to the courtroom.

This post is about that courtroom: our plan for it, what actually happened (spoiler: not quite the plan), and what the whole experience made me think about the much bigger picture of letting algorithms judge people.

The Plan: A Hybrid Courtroom

In Part 1, I described the AI Judge we designed. I want to be upfront here: that was the plan. Before I tell you how it really played out, it’s worth walking through what the plan was, because I still think the design is the right direction.

The idea was a hybrid courtroom, with the AI Judge as one seat on the bench, not the whole bench:

• Step 1 – Submit to the AI Judge. Teams upload their final projects, code, and reports to a custom GPT in ChatGPT that has been loaded with our judging rubrics.

• Step 2 – The AI narrows the field. The AI Judge evaluates every submission and picks the top three teams in each track.

• Step 3 – The AI scores like a judge. For each finalist, the AI Judge fills out the full rubric, and that score counts as one judge’s grade.

• Step 4 – The humans take the stand. Finalists present in front of a panel of expert judges, who each score the teams themselves.

• Step 5 – Combine the verdicts. Each human judge’s score and the AI Judge’s score are combined into the final result.

The heavy lifting of building the AI Judge itself was done by a colleague of mine. My part was on either side of it: designing evaluation metrics a machine could actually read, and working out how an AI Judge’s score should sit alongside the human judges’ scores so that neither one quietly drowned out the other.

Writing Rubrics for a Machine

Honestly, this was the part that fascinated me most. A rubric written for humans and a rubric written for an AI are not the same thing.

When you hand a human judge a criterion like “the personas are realistic,” they fill in the gaps with experience and common sense. An LLM will happily fill in those gaps too. It just won’t fill them in the way you meant. So we had to rewrite our criteria into what I call Machine-Readable Judgment: criteria specific enough that an AI, and frankly a tired human at the end of a long day, would land on the same score.

Here’s what that looked like for the GenAI track (100 points, plus a 10-point bonus):

• Task 1 – Segment-to-Persona Generation (30 pts): Personas match the segment data, are complete and clear, and connect logically to banking needs.

• Task 2 – Persona-Guided Chatbot (30 pts): Personalization, natural conversation flow, and safe, professional answers.

• Task 3 – Personalized Product Recommendations (40 pts): Good product match, clear reasoning, and three distinct persona examples.

• Bonus – AI-Agent Handoff (+10 pts): Smooth handoff to a second AI that adds real value, plus a creative touch.

A few design principles came out of writing these:

Anchor every criterion to evidence. “Realistic & Data-Based” didn’t mean “seems realistic.” It meant the personas had to match the age, income, digital use, and salary ratio in the segment profile data the teams were given. That’s something a judge, human or AI, can actually check.

Turn “quality” into structure. “Natural Flow” didn’t mean “the conversation feels nice.” It meant the dialogue followed a six-step structure: Greeting → Needs → Product → Upsell → Objection → Close. You either see those steps or you don’t.

Make safety as close to yes-or-no as possible. “Safe & Professional” meant no financial advice and no false claims. That’s not a matter of taste.

Give score bands, not just maximums. Every task came with a guide (for Task 1, 26–30 = excellent, 20–25 = good, 10–19 = fair, 0–9 = poor) so that the AI and the human judges were calibrated to the same scale.

And on the Data Science track, the penalized, normalized RMSE from Part 1 was the purest version of this idea. No interpretation needed at all; a script just computes it. That’s really the goal: push as much as you can into objective scoring, and save judgment for the things that truly need judgment.

What Actually Happened When the Clock Ran Out

If you’ve ever organized anything like this, you know datathons run on brutal timelines. Between building the Synthetic Data Engine, designing three tracks, stress-testing every problem with adversarial prototyping, and coordinating with all of our partners, something had to give. The AI Judge was the thing that got squeezed.

In the end, we never got the full pipeline working. Here’s what we actually did:

• We ran each team’s report through the AI Judge.

• It gave back a basic evaluation, a general response rather than a reliable, filled-in rubric score.

• A human still had to read and interpret that response and turn it into a score.

• Those scores decided which teams made the top three in each track.

• From there, the expert judges evaluated the finalists the traditional way, human only.

So the AI Judge never really sat on the bench. It was more like a court clerk who prepared case notes. Useful, yes, but a human still had to make the call.

Was it a failure? I don’t think so. It was a really honest lesson in where the hard part of AI-era work actually lives.

Lessons from the Courtroom

Design the judging when you design the problems. We spent weeks stress-testing our problems with adversarial prototyping, trying to break them before the participants could. Our evaluation process deserved exactly the same treatment. If I did this again, the AI Judge would get its own adversarial prototyping: run fake submissions through it (great ones, terrible ones, and weird ones) weeks before game day.

Treat the evaluation pipeline as a product, not an afterthought. What format do teams upload? Can the AI actually read code, notebooks, and slides, or just text? And most importantly, what exactly should it output? Without a strict, structured output, like a filled-in rubric with a score for each criterion, a human ends up interpreting prose. That’s exactly where we landed.

Push the measurable into scripts. The penalized RMSE never needed an AI. Every criterion you can turn into a script is one less thing an AI Judge or a human can get inconsistent about.

Use AI for triage before you trust it with verdicts. Even our basic version helped with the first cut. Narrowing 50 teams down to a set of finalists is a sensible first job for an AI Judge. Handing it the final score is a much higher bar.

Keep the human seat at the head of the bench. In the end, the final decisions were human. And honestly, the more I thought about it, the more I was glad they were.

Which brings me to the bigger picture.

The Paradox of Automated Evaluation

Here’s what stuck with me. Even in a tightly controlled sandbox, with clear rules, rigid constraints, and rubrics we had specifically rewritten for a machine, getting an AI to judge reliably was harder than we expected. And a datathon is the easy case.

We are rapidly accelerating toward a future where AI systems will be used to evaluate university students, grade standardized essays, and screen professional resumes and portfolios at an unprecedented scale. On paper, the promise is intoxicating: total operational efficiency, less human prejudice, and instant, personalized feedback. But as we hand the grading pen over to algorithms, we run straight into a profound paradox: How do we use AI to evaluate human intellect without accidentally crushing human brilliance?

When you build an AI evaluator around a structured rubric, you are essentially training it to recognize a specific, predefined flavor of correctness. Algorithms are great at identifying compliance, spotting structural harmony, and rewarding patterns that closely match the norm. If a student or a job candidate answers a problem using exactly the framework the AI expects, they get a perfect score.

But the most valuable human thoughts, the truly revolutionary breakthroughs in data science, economics, and business, are almost always born outside the norm. They’re non-conforming and unexpected, and at first they look like anomalies. If an automated evaluation system only rewards answers that live inside its predefined boundaries, we risk creating a dangerous institutional feedback loop: an educational and corporate culture that penalizes original, creative risk-taking and rewards polished, standardized mediocrity.

I felt this tension firsthand writing our rubrics. Every time I made a criterion more machine-readable, I made it more precise. But I also made it a little narrower. A team that built a brilliant chatbot that didn’t follow our six-step structure would have lost points for Natural Flow, even if their approach was better.

From Datathons to Governance: The Case of Diella

This tension between algorithmic compliance and human nuance isn’t just an academic debate; it’s playing out right now at the highest levels of government. Consider the fascinating case of Diella, the AI system developed by Albania’s National Agency for Information Society in cooperation with Microsoft, which provides the underlying OpenAI language models through its Azure platform.

Diella launched in January 2025 as a virtual assistant guiding citizens through digital public services on the e-Albania platform. Then, in September 2025, she made global headlines when she was appointed “Minister of State for Artificial Intelligence,” the first AI system in the world named to a cabinet-level government role. Her primary, high-stakes mandate? To gradually take over public procurement and evaluate government contract bids, with the Prime Minister promising that public tenders would be “100% free of corruption.”

• Diella (Albania public tenders)
  The algorithmic promise: objectively filters out corrupt bids and irregular payments.
  The human risk: may eliminate unconventional, highly innovative vendors who don’t fit standard forms.

• Our AI Datathon Judge
  The algorithmic promise: fast, consistent first-pass grading across 50 teams.
  The human risk: penalizing brilliant, out-of-the-box solutions that break the rubric’s mold.

The pitch for Diella is basically the pitch for our AI Judge: replace flawed, biased, or potentially compromised human gatekeepers with an objective algorithm that judges strictly by the book. In public procurement, that kind of algorithmic rigor can flag irregular payments, confirm compliance, and clear administrative bottlenecks.

But the deeper risk is what some AI governance researchers call AI Exceptionalism: the belief that AI systems are inherently more objective and reliable than the humans they replace, and that the algorithm’s definition of “correct” or “valid” is absolute. What happens when an incredibly innovative local business submits a tender proposal built on a completely unconventional financial or logistical model? A human committee might look at that out-of-the-norm proposal, recognize its genius, and adapt. An AI minister looking strictly for compliance with hardcoded criteria might reject it instantly as a non-conforming anomaly.

And here’s the part that hit close to home for me. Almost a year after the big announcement, a BIRN Reality Check found that the AI-powered procurement system Diella was supposed to run was still at the preparatory stage, with the agency only just finishing the terms of reference to hire someone to build it. Experts pointed to the same gaps again and again: no clear legal framework, questions about data quality, and no clear answer to who is responsible for a decision shaped by an AI. Sound familiar? On a much smaller scale, our AI Judge taught me the same lesson. The vision is easy to announce. The plumbing, the oversight, and the accountability are the hard part, and they need to come first.

Looking Ahead: The Evaluation Bottleneck Is Coming for Everyone

Our datathon was a small, contained version of a problem that’s about to show up everywhere. The pattern is always the same: AI makes producing work cheap, so the volume explodes, so evaluation becomes the bottleneck, so people reach for AI to do the evaluating. And very quickly you end up with AI grading work that AI helped create. Here are a few places where I think this is going to get really interesting, and really messy.

Academic evaluation. Universities are living through the Two-Hour Trap right now. A take-home essay or coding assignment that used to measure a week of effort can now be produced in an evening. At the same time, instructors facing hundreds of submissions are tempted to use AI to grade them. If we’re not careful, we end up with AI writing the assignment and AI grading it, and nobody in the loop actually learning anything. I think the answer looks a lot like what we did at the datathon: evaluate the process, not just the product (like our collaboration logs), lean on structured, evidence-based rubrics, and bring back the human conversation, like oral exams, defenses, and presentations, which is exactly what our finalists did in front of the expert judges.

Peer review of research articles. Academic publishing is already under strain. AI-assisted writing makes it easier than ever to produce manuscripts, and the pool of qualified reviewers isn’t growing to match. The temptation to have an LLM write the review is huge. But there are real problems here. Uploading an unpublished manuscript to a commercial AI tool can break the confidentiality that peer review depends on, which is why many publishers and funders have put restrictions on it. The US National Institutes of Health, for example, prohibits its peer reviewers from using generative AI for exactly that reason. And in 2025, Nature reported that researchers had been hiding invisible instructions in their papers, in white text or tiny fonts, essentially prompts telling an AI reviewer to give a positive review. Once evaluators are machines, people will start writing for the machine, not the reader. That’s adversarial prototyping in reverse, and the evaluation system has to be designed for it.

Research proposals and grants. This one worries me the most, because it’s where the paradox of automated evaluation bites hardest. The whole point of research funding is to find ideas that are new. A truly groundbreaking proposal often doesn’t look like the proposals that came before it. It’s the anomaly. An AI screener trained on what successful proposals usually look like will be great at rewarding the safe, well-formatted, incremental idea and quietly filtering out the risky, original one. Funders are also seeing a flood of AI-assisted applications. NIH reported cases of researchers using AI to submit more than 40 applications in a single round, and it responded by capping researchers at six applications per year and refusing to treat applications substantially developed by AI as original ideas. If AI ends up doing the first cut, we need to be very deliberate about protecting the out-of-the-box proposals, maybe even with a dedicated human “wildcard” review for submissions the AI flags as unusual.

Hiring and talent screening. This is the most familiar example. Candidates use AI to tailor resumes to job descriptions, and companies use AI to screen those resumes. It turns into an arms race of keyword optimization, where the AI is essentially grading how well another AI matched the rubric. The candidates most likely to slip through the cracks are the ones with non-traditional paths, who are often exactly the people with the most interesting perspectives.

High-stakes public decisions. And of course, Diella shows where this goes at the highest level. In my own field, I can easily imagine AI being used to evaluate grant applications for public health programs, prioritize which communities get resources, or vet analyses that feed into policy decisions. Those are places where an algorithm rejecting a “non-conforming anomaly” isn’t just a lost point on a leaderboard; it can affect people’s health.

Across all of these, the lessons from our little courtroom hold up surprisingly well:

• Design the evaluation as carefully as the task. Stress-test it with good, bad, and weird examples before it goes live.

• Make rubrics machine-readable, but know what you’re giving up. Precision is great, but every rule you add narrows what counts as “good.”

• Evaluate the process, not just the output. Logs, drafts, and conversations are much harder to fake than a polished final product.

• Use AI for triage, not verdicts. Let it sort, flag, and summarize, and let humans make the calls that matter.

• Build a door for the anomaly. Give unusual submissions a way to reach a human, instead of letting the algorithm quietly reject them.

Conclusion: Keeping the Human in the Loop

Whether we’re talking about an AI system like Diella evaluating multimillion-dollar government tenders, a custom GPT grading datathon projects, or a university algorithm scoring student essays, the lesson is the same. Algorithmic evaluators are magnificent tools for heavy analytical lifting, sorting, and establishing baseline consistency. They save time and can strip out some superficial human biases.

But code can never be the final word on human judgment.

In our datathon, the AI Judge was supposed to lay an objective foundation for the human judges to build on. In reality, the humans ended up carrying more of the load than we planned. Looking back, that’s not the embarrassing part of the story; it’s the most important part. When it came down to deciding who had truly done the best work, we trusted people who could look past the scoreboard.

As we step deeper into an AI-native world, our highest priority must be to design evaluation frameworks, in our schools, our companies, and our public institutions, that use AI for what it’s good at while protecting the non-conforming, unique, and invaluable thoughts that no algorithm could ever predict.

What’s Next?

Thank you for following along with this three-part deep dive! I’d love to hear your thoughts. How is your organization balancing automated efficiency with protecting creative, out-of-the-box thinking? Let’s keep the conversation going on LinkedIn!

For an excellent look at what actually happened with Albania’s AI minister, and the lessons experts are drawing from it, see BIRN’s Reality Check: What Has Albania’s AI ‘Minister’ Achieved So Far? For a sharper take on what it means for democratic accountability, I also recommend Albania’s AI minister: ‘avatar democracy’ and the spectacle of accountability on the ECPR’s blog, The Loop.