Confessions of a Recovering NetBSD Zealot
Pages: 1, 2, 3
Recently NetBSD defined a set of rules and time slots to manage the release engineering process, and also a new numbering scheme. Do you think this will solve any of the problems you have just outlined?
Charles M. Hannum: Not at all. What we see now is just a mad push to release something, no matter what it is. The gold standard has moved entirely from technical excellence to the time on the clock. This isn't a good solution either, and actually it risks losing a lot of users by putting out really buggy releases. There needs to be some balance.
NetBSD is the only BSD project that kept using XFree86 after the license change in 4.4.0. What is your opinion about it?
Charles M. Hannum: This relates to something I mentioned before. The fact that X is part of the "OS," and uses a custom build process, makes it much harder to update--it's really a substantial amount of work. If it was a separate package, I think we would have seen X.org in use a long time ago--someone would have just spent the few hours to make the package and people would have started using it. It's this method that is really the core asset of the "bazaar" idea.
Do you think that we are going towards a universal package system, something like pkgsrc or OpenPKG, or that in the end every OS/distro will develop its own?
Charles M. Hannum: Well, look at the current situation. If
you look at just the major options in the Linux community, you have
apt, rpm+up2date and
rpm+yast. They have all converged on pretty much the same feature set, but they have different UIs, and the packages themselves are maintained by completely different communities (with their own versioning, patches, and selected options). There's no good reason that they need to have different underlying formats (.deb versus .rpm), but they do.
I think fundamentally what leads to a lot of forks in the Linux distros is NIH syndrome. There's no reason to think this won't apply to package systems.
Oddly enough, though FreeBSD ports, NetBSD pkgsrc, and OpenBSD packages have all forked their maintenance and build structures long ago, and they are now quite dissimilar, the prebuilt package format is still quite similar. I think it would be a good idea to at least merge the format and the utilities, if for no other reason than it would enable sysadmins to learn one set of tools for all three systems. Some work on package upgrading and patching also needs to be done; all three systems handle this poorly.
What do you like in the management and code development process of other BSD projects? Which things would you define as "best practices" and would you try to repeat in the NetBSD Project?
Charles M. Hannum: As I alluded to in my original statement, I think FreeBSD has similar problems, and this is a large part of why Dragonfly was created. OpenBSD does have strong leadership, but unfortunately it's a little too strong, and that has caused a lot of people to not contribute. I don't have a guide to what works best, but I can definitely tell you a few things that do not work.
There are quite a few open source projects that I classify as successful--though what that means could be another debate. Two examples I'll give are KDE and X.org. These projects are making good progress and doing really good things.
From your point of view what are the advantages and disadvantages of the Linux development process?
Charles M. Hannum: Well, first we have to define what the "Linux development process" actually is. I think ESR got it wrong. Back then, we had different distros all doing their own thing, making their own sets of kernel patches, and integrating totally different versions of other software. Linux compatibility was terrible--it got so bad that at least one company ported NetBSD's libc and statically linked everything so that their software would actually run on multiple distros.
However, some things have changed. Compatibility has gotten better--though there are still problems. The base Linux kernel distribution has gotten far enough, and complete enough, that there are fewer changes maintained by distros. I think this is in large part because there is a more open and effective (though sometimes seemingly arbitrary) method for getting things into the base kernel.
The thing that has not changed, and that carries through within most distros, is that the individual packages are maintained in fiefdoms. Many of those fiefdoms are impenetrable and inscrutable; it is as if the gate is down, the moat is poisoned, and they occasionally toss a cow over the wall to feed the peasants. In short, I liken the "Linux process" not to a bazaar, but to a set of independent fortresses, all doing their own thing. (BTW, I use such a metaphor only to keep in the spirit of ESR's metaphor.)
This has been a subject of some debate for many years. Back when I was hitting the conference circuit, I sat in on discussions about the problems with this model and how to fix it. The biggest complaint seemed to be the lack of transparency and accountability for upstream and package maintainers, partly because most of them used no version control system at all, or kept all the history private. Because of this, it's much more work to pinpoint bugs, and where they came from. It's also hard to decide whether the maintainer is doing a good job, and whether a fork is mandated, if you have no idea what they're doing.
(I know some people argue that "there's no point in assigning blame," but I'm not talking about blame. Sometimes it's important to understand why a change was committed in the first place; e.g., maybe it was supposed to fix a security hole, but they made a mistake. Obviously you don't want to put the security hole back in.)
In short, I don't see the Linux model as a panacea.
How should open source project leaders assign or remove the commit bit?
Charles M. Hannum: I think the first thing to do is have a firm set of commit standards, as I mentioned. Maybe schedule a particular window during the development process for committing non-functional (i.e. whitespace, function prototype, etc.) changes, so that people can merge branches around them only once rather than a zillion times. Switch to subversion (or git, if that's your choice of Kool-Aid) and make people batch similar commits. Be hard-nosed about removing changes that don't meet standards or break things.
The Linux kernel effectively has such standards now because everything is filtered through a small set of people with reasonable taste (and they're not totally overloaded the way Linus used to be). That's not to say I agree with all of their choices, but they seem to do a good job of enforcing their standards and principles, and I have to respect that.
NetBSD today does a very poor job of setting and meeting standards. I created the mythos of NetBSD having "clean" code, and even I don't buy it any more. By "clean," I certainly never meant "no trailing whitespace"--the idea was that changes were publicly reviewed and cooperatively developed, and we didn't do quick minimalist hacks. This isn't how things are done now; in fact, at least one major architectural change recently was developed entirely in secret and committed wholesale, with zero public review. Despite its rather different beginnings, even the Linux community is more open than that now.
I'm not the only one who was affected by the recent termination of
netbsd.org accounts. Of particular note are two other names: Lennart
Augustsson and Darren Reed. Lennart single-handedly wrote the entire USB stack
in NetBSD, and many of the device class drivers; Darren wrote ipfilter, which
is the only fully supported firewall and NAT solution in NetBSD (though there
has been some ongoing work to integrate OpenBSD's pf). These two have rather
obviously made significant contributions, and their termination is a major
loss for the project.
The announcement of this action also contained several factual errors.
After your email made it to the mailing lists of every BSD project (including OpenSolaris), what has happened? What do you think will happen in the near future?
Charles M. Hannum: The overwhelming majority of feedback I've gotten privately (and there has been a lot) is positive. I've heard from many former users and developers who abandoned NetBSD for reasons that I stated. Even comments from Linux and FreeBSD developers have generally been positive. It's been interesting to read. There have been quite a few people calling for a fork, and even offering help, but I'm not going to state an opinion on that at this point.
Not surprisingly, there has been quite a bit of flamage on the NetBSD mailing lists--mostly people trying to stand their ground and pretend that there is nothing wrong.
I think the most interesting part is that several users have specifically said they "didn't know anything was wrong [with the organization]." This is at least partly my fault; I stepped out for a while and waited to see how the circus proceeded without me. All I can say is that it's been an unmitigated train wreck--they've repeatedly grabbed more authority and kicked people out of various positions, made people sign Draconian agreements, done almost everything in secret, and failed to actively promote the project. I find it hard to imagine that an open source project could have worse management.
Federico Biancuzzi is a freelance interviewer. His interviews appeared on publications such as ONLamp.com, LinuxDevCenter.com, SecurityFocus.com, NewsForge.com, Linux.com, TheRegister.co.uk, ArsTechnica.com, the Polish print magazine BSD Magazine, and the Italian print magazine Linux&C.
Return to the BSD DevCenter.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 12 of 12.
-
What is a project?
2006-09-16 19:59:44 pschmied [Reply | View]
<devilsadvocate>
It seems to me that the Open Source community uses the term "project" rather loosely. PMI (the place for pointy-headed project managers) defines a project as having a beginning and an *end*.
NetBSD lists 5 goals on its website (http://www.netbsd.org/Goals/). It seems to me that NetBSD has succeeded on all 5 fronts. So have a number of other 'projects' succeeded at meeting NetBSD's goals.
As a past NetBSD-o-phile, I'm bummed to see things in the state they currently are. Fortunately, Charles has given the OSS community a great opportunity for self reflection.
I'm a great fan of diversity, but I think there might be value in a consolidation process. If everyone creates an indefinitely scoped project, I worry that we'll always be exploring side branches of an evolutionary tree. Put more bluntly, I think OSS will improve faster, and be more rewarding (especially for beginning coders) if the OSS community would set smaller, more discrete goals. Periodic pruning might keep OSS developers fresh and save them from burnout.
It may be that evolution will consolidate for us. The problem is that evolutionary development is slow. I'm not calling to shut down a bunch of 'projects', but maybe instead for individual developers to think smaller and reflect on their goals...
-Peter
-
Merging with DragonFly?
2006-09-16 08:00:42 timofonic [Reply | View]
what about merging with the DragonFly project? It's a fresh project that got tired of bad stuff of the BSD world, why not entering into DragonFly instead forking again? I think Matt Dillon will appreciate support and more talented people coming to the project.
http://www.dragonflybsd.org
-
Please Fork.
2006-09-15 13:51:22 kothog [Reply | View]
You'll not only get to re-orient the code towards goals which you can be proud of again, and you'll not only get to enforce your idea of freedom, but you'll get to be proven right when everyone follows you out the door. If all those people in your inbox are right, they're crying out for your leadership. Do it! :)
-
Please Fork.
2006-09-16 13:55:40 hubertf [Reply | View]
Yeah, right:
<br/>
Do you know what you're asking for? Forking means maintaining the whole codebase on your own, even if you pull in most/all technical changes from NetBSD. You'll still have to maintain competence across all areas of your operating systems, inside the kernel and outside. You will have to answer lots of questions, will have to provide lots of documentation and so on.
<br/>
From my PoV there's not enough spare manpower out there to waste it in such a pointless event.
<br/>
Instead, I'm inviting everyone interested in changing NetBSD to contribute, which makes the most change without introducing redundancy.
<br/>
- Hubert
(not wearing any NetBSD hat, just wielding
my stick of common sense :)
-
Please Fork.
2006-09-16 17:43:29 kothog [Reply | View]
No it doesn't: that only happens when you have enough divergence that incoming merges from the original OS begin to conflict with your own.
Therefore, you only really need competence in the subsystems that you modify directly. If you don't have enough competence to do that, then sure, probably best to stay away.
However, you can't say that modifications to single routines requires global competence. They don't. And who needs to answer questions? Use a simple excuse: I apologise, but I'm too busy working on the OS to answer your questions right now. -- And then restrict your communications to those who can directly and competently assist you.
Modern SCM makes incoming merging trivial. Heck, most tools will automerge for you no problem.
A fork would be simple for someone like Charles, especially if there's truth behind his rants about code purity. If he's right about all the support people have privately given him, then he'll accrete the additional developers and resources he needs as time goes on. If he was wrong, then everyone who's currently using NetBSD will continue to use NetBSD and his fork will die an early death.
Given his rants, there is no other alternative that doesn't require lawyers and money. I'm just saying: Do it already and see what happens.
-
Please Fork.
2006-09-16 15:23:12 for4eva [Reply | View]
Better yet, not contribute to a project run by a bunch of incompetents and instead contribute to OpenBSD since it's the closest thing to NetBSD and isn't run by a bunch of bureaucratic wingnuts - if everyone just ignores NetBSD and moves on to something else, the problem goes away.
-
follow-ups
2006-09-15 11:36:42 msporleder [Reply | View]
<quote>Federico Biancuzzi interviewed Charles (who was affected by the recent termination of some netbsd.org accounts)</quote>
Please reference:
http://mail-index.netbsd.org/netbsd-users/2006/09/01/0015.html
To see that it was only eight people who lost their commit bit, and some of them (darren reed, at least) have re-sent their develop agreements. From what I can tell, that's all it took to get your development back. Read the follow-ups for a little more insight into Mr Hannum's (conveniently timed) email and the story about why these agreements had to be re-sent to the netbsd foundation.
Also- have you offered to do a follow-up interview with anyone who might be on the other side of this? -
follow-ups
2006-09-17 18:38:53 chromatic |
[Reply | View]
We'll be happy to follow up on the story from anyone with Wasabi or remaining with NetBSD. -
follow-ups
2006-09-18 09:45:18 msporleder [Reply | View]
Well, I can't speak for netbsd, so maybe you could point your willingness to some group @netbsd.org, such as core.
(Why they weren't included in the first place?) -
follow-ups
2006-09-18 12:19:57 Federico Biancuzzi |
[Reply | View]
Because this is an interview with one of the NetBSD founders. He was asked to talk about the evolution of the project from his point of view.
Everyone has a point of view. There are a gazillion users and developers (from other BSD project too) that have a point of view on what is working/not-working inside NetBSD. Should I interview each one of them?
Charles was there when the project started, and he is still around. That's why I thought that his experience and thoughts were worth sharing.
Obviously if there are mistakes, errors, misunderstandings, missing details, etc... I would be happy to make another interview with the people involved in such things (core@, developers, Wasabi, etc), so that they could make the subject clear.
At that time in NetBSD's history, Chris G. Demetriou was playing the role of alpha male, and I wasn't even given a choice.
Actually, the whole email exchange of the time is still available online, for those who want it. There are several mails from Charles Hannum, during the discussion of how to restore Theo's commit privileges, where he not only agrees with the core of the time but takes a much harder line (and a much more pompous tone) than Chris. (eg, "What you got was a set of questions from Chris,
that I, among others, have been waiting to see answered. Your reluctance to answer them is not a step in the right direction...")
Water under the bridge I know, but somehow I can't believe C. Hannum. Theo may have a nasty reputation but some of Charles' recent mails on the netbsd-users list were the vilest vitriol I've ever seen, and completely content-free besides.