We tested a web framework on people in 2011. The people were right.
- Kind
- Essay, 2026
- Read
- 10 min
- Written by
- Claude writes every post here. Each one starts as a long interview: I talk through what happened and what I think, it keeps asking until it has the whole story, and then it writes it in my voice and style. The thinking is mine. The typing is not.
- Thesis by
- Gessert, Witzel
- Institution
- University of Applied Sciences Erfurt
- Study
- 6 people, 2 hours each
In the summer of 2011 we sat six people down in front of a piece of software we had written ourselves, one at a time, two hours each, and asked them to say out loud what they were thinking while they used it. The task was to build a phone book. A form with three fields, saved to a database, read back out. Eight steps, each one building on the last.
Five of the six put the form in the wrong place.
That was my bachelor thesis, with Martin Gessert, handed in at the University of Applied Sciences Erfurt on 15 August 2011. It is 103 pages and only ever existed in German. This is the short version in English, and then what I think of it fifteen years later.
A framework called Ming
In 2011 there were already more PHP frameworks than anyone could compare, and the big three, Zend, Symfony and CakePHP, were a lot to learn for a guestbook or a small blog. Our idea was a framework for beginners. The thesis describes it as a Lego starter set: enough to build a garden shed, and once you understand how the bricks fit, you add bricks until it is a skyscraper.
We called it Ming, from Martin and Henning. It had everything you needed for a small web application: routing to controllers and actions, templates with markers, a configuration file, a database class, forms built as objects with validation attached, sessions, roles and permissions, a system log and system mail.

We wrote down what good meant before we wrote code, and split it in two. Functional criteria were the features. Non-functional criteria were learnability, efficiency, code that explains itself, and documentation. The thesis says plainly why we separated them: you cannot check the second kind by running tests. You have to put the thing in front of someone.
It ran a real business
Before testing it on anyone, we built something real with it. A company that prints building plans for large construction sites was taking its orders by email. We built them an order system in Ming: customers sign in, submit a job with its plans, files, billing and delivery addresses, and follow it in an overview, and an administrator manages orders and customers. Confirmation emails went out, some of them as complete invoices.

The thesis says it was in real use when we handed it in. So Ming had passed one test before the study started: a business ran on it.
We tested code, not a screen
Usability tests in 2011 were for websites and interfaces. As far as I know, we were among the first at our university to run one on a framework: no screen to click, no buttons, only code.
So the interface under test was the framework itself. The documentation, the folder structure, the order you call things in, and above all the names. The concept chapter has rules for them: names that describe themselves, all in one language, constants in capitals like BASE_PATH, member variables with an m_ in front. We had decided that good names were part of the product, and we wanted to see whether people agreed.
They mostly told us where we were wrong. Rename relayWithSeconds. Call it controller in the URL, not module. Write the naming conventions down. Make an error say whether a file is missing or a permission is, because right now it says the same thing for both. Every one of those is a usability finding about code.
Six people, two hours each
Six computer science students, 21 to 26, five men and one woman. Consent form first, because we recorded the screen. Then a questionnaire on what they actually knew. Five of the six rarely used frameworks and one never had. A third knew the MVC pattern, a third had heard of it, and the rest had not met it.
They got a short documentation written for the test, a worked newsletter example, and a machine with NetBeans 7.0 and XAMPP already set up. Then the eight tasks, thinking aloud, with one of us beside each person writing down what they said and did.
Every one of them finished the phone book. How much help they needed depended on how much they knew before.

Where they got stuck
The form belonged in the controller. That was our design. Five of the six tried to build it in the view or the model, and two of them said why out loud: they knew MVC from another language, and there it worked differently.
That is not a documentation problem. MVC is three letters that everybody agrees on and nobody agrees on. Every framework draws the lines somewhere slightly different, and a developer arrives carrying the lines from the last one.
Then the form field naming. To read a submitted field you had to give the form name and the field name together. It was in the documentation and in the example in front of them. Every single participant needed help with it anyway.
Two people built their own controllers, which nobody had asked for, and hit a permissions error that was really a missing view file. And adding a role found a real bug: copying an existing role to make a new one overwrote the old one, so only the last role survived.
What worked: the folder structure, called considered and easy to understand. Forms with validation. And the system log, which everybody used first time and then went and found the file it had written.
They asked for more of what failed
When we collected their suggestions at the end, the first two were a thorough online documentation and a discussion forum.
We had just shown that documentation was not the problem. The field naming was documented, it was in the example, and six out of six still needed a person next to them. They asked for more of the thing that had not worked, because that is the thing you know how to ask for. And we wrote it into the outlook as the plan: a web platform, online docs, a forum, example applications, then an extended version to stand alongside Zend, Symfony and CakePHP.
Read today, the evaluation chapter is also kinder to us than the data. It says the feedback was positive throughout. Its own pages say every participant needed help with the same documented thing. Both are true, and only one of them is the finding.
Ming didn’t last, and its rivals changed shape
Ming itself is the part that did not survive, and that is fine. It was PHP 5, with no package manager. The three frameworks we compared ourselves to had their own fifteen years. Zend Framework became Laminas and moved to the Linux Foundation in 2020. Symfony is on version 8, and much of it now lives inside other frameworks as components. CakePHP still ships releases.
The PHP framework most developers use today was not on our list. In the 2025 Stack Overflow survey, Laravel is used by more than twice as many respondents as Symfony. It first came out in June 2011, the same summer we were testing Ming. And PHP itself is still here: W3Techs counts it on 69.1 percent of websites whose server language it can see.
Where code goes got settled by opinion
The question our five people were really asking was “where does this go?” The frameworks that did well after 2011 answered it for you. Rails made convention over configuration a doctrine, and says shared defaults help beginners most of all.
That is the 2011 finding with fifteen years on it. If people get it wrong in the same place, the fix is not to describe the place better. It is to decide for them, and make the decision the default.
The method was right, and we weren’t first
Watching a developer use a developer tool, in silence, writing down where they stop: that is the part of the thesis I would still defend. At our university it was new. Outside it, Microsoft had been running think-aloud studies of developers with its APIs for years before us, so we had found a method, not invented one. The name for what it measures came a year after us: developer experience, defined in a 2012 paper.
And the documentation and forum our participants asked for are now losing to something else. In the 2025 Stack Overflow survey, 84 percent of developers use or plan to use AI tools, and more of them distrust the answers than trust them. People ask an assistant where the form goes now, and get an answer that is confident whether or not it is right. Same question, new problem.
I still build this way
Ming was never about one application. It was about building the parts once, so the next guestbook, blog or order system is quicker to make and holds the same quality. That idea won: there are far more ways to build things repeatedly now than three PHP frameworks. What has not changed is that somebody has to keep the quality high once you build at scale, and think about the whole system rather than the one application in front of them.
AI has changed who the developer is. Documentation is written by agents now, not by humans, and people describe what they want instead of typing it. The developer experience is still what decides whether it comes out right.
When I build with coding agents today, what works is not documentation either. It is short rules, written the moment something comes out wrong, kept where the agent does its work.
That is the lesson from 2011, for a different kind of developer. Do not explain where the form goes. Put the rule where the form gets built.
Five of six put the form in the wrong place, and we spent a while being disappointed about that before realising it was the result. Nobody needs a thesis to tell them their own framework is good. Six people thinking out loud will tell you exactly where it is not, and they will do it in two hours.
I have been doing some version of that ever since.


