The legal status of emulation programming took a very precarious turn when Blizzard Entertainment and its parent company, media conglomerate Vivendi Universal, filed suit against the volunteer developers of the BNETD Project on April 5, 2002.
Developed under open source, BNETD is server software which emulates most of the functionality of Blizzard's online game network, Battle.net. Irvine, Calif.-based Blizzard is the developer and publisher of the enormously successful StarCraft, Warcraft and Diablo role-playing game franchises, and its Battle.net service enables fans of these games to play with others on the Internet.
BNETD's origins date back to 1998. Created by Mark Baysinger, who at the time was a student at the University of California, San Diego, BNETD was then known as StarHack and was meant to emulate enough of Battle.net so that fans could play StarCraft online on their own servers. The project went into the open source model, and several contributors over the years have enhanced the BNETD code so that other Battle.net-enabled games, like Warcraft and Diablo, can be used with it.
Today, the project is under the aegis of Ross Combs, a 27-year-old computer security programmer/analyst from Austin, Texas. Rob Crittenden, a 34-year-old programmer in Lithicum, Md., is the second lead developer of BNETD behind Combs.
The number of presently active contributors consists of one administrator (Combs) and three coders, but, additionally, Combs says, "There have been 25 to 30 people who have made nontrivial programming, bug investigation, or administrative contributions. If you count small bug reports and suggestions, then that number would be two or three times larger."
The BNETD developers say that BNETD was made as an alternative to Battle.net's oftentimes slow and buggy service. It was also meant to enable friends to play Battle.net-enabled games with each other on a private network, without having to deal with abusive strangers on Battle.net. Written in C and available for the Linux, Win32, BSD and Solaris platforms, the BNETD code, they claim, has been developed over the years through legal, reverse-engineering means. (Since the filing of Blizzard's suit, the code has been removed by the developers from the BNETD site until this matter is resolved.)
|
Related Reading 
Physics for Game Developers |
In its suit, Blizzard claims, among its list of grievances, that BNETD infringes upon its Battle.net trademark, violates its copyrights since use of BNETD constitutes an unauthorized public performance of its games, and -- most damning of all -- that BNETD was created through code stolen from Battle.net.
The ISP hosting the BNETD site, Combs, and 50 "John Does" (presumably, anyone who has contributed significantly to BNETD's development) are listed in the suit. The stakes here -- regarding legal reverse engineering, creating emulation technology, and encouraging or discouraging contributions to open source projects -- are high enough that the Electronic Frontier Foundation (EFF) decided to take up the defending of BNETD and its developers.
The overall facts of BNETD's predicament have made news in the open source community, been analyzed in the legal community, and been covered in other media (full disclosure: I myself wrote such a piece).
But for O'Reilly Network, Combs and Crittenden wanted to speak candidly about the technical aspects of BNETD's development and, to help clear the legal air about how they reverse engineered the inner workings of Battle.net.
Howard Wen: Up until the moment when you chose to temporarily remove the BNETD code from the project's Web site, what was the then-current progress with it?
Rob Crittenden: BNETD has support for all of the major features of Battle.net, including:
To a large extent, from the user's perspective, it is difficult to distinguish [BNETD] from Battle.net in functionality.
Ross Combs: One major user-visible feature that we recently added is an internal mail system, so players can send messages to friends who are not logged in.
We also reworked the "ipban" code, which allows administrators to ban IPs and subnets from connecting to the server. In the past, the list of IPs to ban was kept in a configuration file. This was not optimal because BNETD administrators would need shell access to the server and permission to kill -HUP the BNETD process. Now the administrators can add and delete entries from the chat interface at runtime.
We also added the /reply command which makes replying to "whispers" much easier.
There are always more changes below the surface which improve the server but which aren't always directly visible to the average user. I improved the hashing algorithm for accounts to have a better distribution, which should help speed up servers with many accounts.
There is also some work that seems to be never-ending. One of those items is keeping up with Blizzard's recent patches. The patches often involve slight protocol changes or the introduction of new "packets," which require new code on the server side. Since we can't be sure of the actual purpose for these packets, we have to either guess what they do or we have to blindly mimic what Battle.net does. Sometimes the purpose of the packets becomes obvious due to other protocol changes, or because we see enough examples to figure out the pattern. A further complication is that we have to maintain compatibility with older clients, which may send different packets or send them in a different order.
We are trying to work toward a new, major revision 0.6. The last serious bug holding up that release is the handling of the status field of the STARTGAME4 packet. This field tells us if the game is open for additional players, full, started, or if the game has completed. The server sometimes gets this wrong, which results in people trying to join games that have already started.
Wen: Currently, what are BNETD's limitations, compared to Battle.net?
Crittenden: The biggest limitation is scalability. Some ISPs host game servers that have tens of thousands of users, but we use flat files for user accounts, which is a real limitation. I wrote a GDBM back-end for [BNETD] long ago, but that is now deprecated. (It never really worked very well.) We have a design for database-based user storage, but there has been no coding on it yet.
Wen: Has the plan always been to develop BNETD into a 100-percent-compatible version of Battle.net?
Combs: The original goal was to make a personal StarCraft server for people to play a game or two with their friends. That amount of functionality was achieved early on. After that point, the goal changed and our new efforts went toward implementing all the user-visible features of Battle.net. We still haven't completed that task.
Of course, we don't always stick with the way Blizzard does things [with Battle.net]. For example, we implemented online help for chat commands before it was available on Battle.net. Unlike Battle.net, our users also see the server-side configuration, so much of our work goes toward making the server configuration simple and flexible. It is possible to designate certain accounts to have the ability to connect with "bots" without giving all users access. It is also possible to keep some users from ever being selected as channel operators. We've even added support for IRC and telnet clients.
Our goal these days is to provide the best set of features for both players and server administrators. If Blizzard comes up with new things which support that goal, then we will also implement them.
Wen: What's the technique you used to figure out how Battle.net works, in order to make BNETD compatible with the Battle.net clients (the games)?
Crittenden: I used a Windows NT-based network scanner to capture packets between my client and the real Battle.net server. Then I'd examine the packet dumps to see how they did it, then re-implement it in [the BNETD] server. It was a real iterative process that consisted of:
Some of the packets were easy to figure out; others we guessed at.
Combs: I use tcpdump to log packets. When I try to analyze an unknown protocol, I follow these steps:
Of course, the first thing that was noticed is that the client connects to port 6112 on the server. Registry entries in the client select which server to connect to. When the client connects to the server, it sends a single byte (control-a). The meaning of that byte wasn't obvious at first but we later found that it is selecting the type of connection. There were two other types of connections, each with different formats: file connections and bot connections.
The server replies by sending two identical UDP packets back to the client. We originally thought these were to test the connection latency, but they later turned out to be testing for UDP connectivity (looking for dropped packets due to a firewall or network misconfiguration).
After that, all the traffic has a nice fixed-record format (framing). We like to call these records "packets" even though they are just part of a TCP connection stream. These packets all have the same 32-bit header. The first two bytes are a packet type, and the next two are the packet length in bytes. Strangely, the packet types all start with "0xff." Maybe the packet type is just one byte and the other byte is a value to check for invalid packets.
We have implemented about 100 of these packets. There are no doubt a few currently in use on Battle.net that we don't support, and a few used at one time which Battle.net no longer uses. In any case, we were lucky because the basic protocol was pretty simple. We found a few twists and turns later because there are some additional connection types which the server has to implement which don't use the same format (file downloads are an example).
As things progressed, we were often able to test with clients connected directly to BNETD servers, and try different game types and different buttons on the screen. The protocol was regular enough that we could often guess the proper response packet. Of course, this lead to some bad guesses, which would cause client-side lockups or crashes. An early example was that the client would freeze if the ladder button was pressed.
Once all the functionality was working with StarCraft and Brood War, we added Diablo support. We used a recently patched Diablo install and the work went much faster than expected. It only took two hours, since there were few protocol differences.
Warcraft II (Battle.net Edition) support was even easier than Diablo. Adding Diablo II support was somewhat more difficult because Blizzard was making a lot of new changes on Battle.net at the time. They added a kind of "sub account" which [required] changes to be made all over the BNETD source.
|
Wen: What have you learned interesting technically about Battle.net through developing BNETD?
Crittenden: Fortunately for us, they have a pretty nice network protocol. It consists of a fixed-sized header that contains the command to execute and the length of the remaining data for that command. This makes it very easy to add new command support to the server; you just need to figure out what the command is trying to do (start game, list games, log in, etc.) and then code appropriately.
The protocol itself isn't encrypted at all, just individual packets within them (like a modified SHA-1 hash on the password). Just looking at a packet dump of a login provides great insight into how the protocol works -- it's fairly obvious.
Combs: The protocol wasn't designed to be portable from the beginning. The packet structures are all Intel-sized, packed, and little-endian. That's not exactly bad, but I really suspect they were just send()ing and recv()ing raw structures.
They also use things which we call "tags." They are a lot like the Macintosh resource types in that they appear to be strings, but they are always four characters and aren't NUL terminated. I suspect they are just constants stored in an integer since some compilers support an initialization syntax like:
int mytag = 'abcd';
/* equivalent to ('a'<<24) + ('b'<<16) + ('c'<<8) + ('d') */Which is, of course, non-portable in several ways, and it ends up making the string look backwards if [you read it] a character at a time on little-endian systems. For example, the StarCraft client identifies itself as "RATS" ("STAR" backwards).
We also found several of the more mysterious values the client sends are just copies of values from Windows, specifically the 64-bit timestamps on files and the language/locale information.
Wen: Describe the biggest technical challenges you have dealt with in developing BNETD.
Combs: The first big task was adding persistent storage for accounts. I wanted to make the storage easy to parse and edit with tools or by hand. I settled on a flat-file key, value format like such: "key"="value." Many people have criticized this decision (they would prefer a binary format and/or a database), but I believe it has been more beneficial than not.
64-bit timestamp values are used in several places. I had never seen them before, and I had a lot of difficulty writing a conversion routine that obtained the correct values down to the second. After writing a linear regression routine and feeding in a whole bunch of sample points (yes, I did need more than two since rounding was involved), I figured out the offset and slope.
Probably the most daunting task was figuring out any part of the protocol that involved encryption. Thankfully, the server works without supporting any of those packet types. But that meant going without passwords on the player accounts. Not having passwords was OK for LAN parties and systems behind firewalls, but some people wanted to allow logins from the Internet. Once we implemented account profiles, it became even more important so that players couldn't destroy each other's ratings.
Thankfully, the hash size was the same as SHA1 and [we were] sent an example hashing function. The hashed password was sent in the plain to the server where it was stored for later logins. We figured out that the login hash used the session key and a random value (actually a timestamp), plus that hashed password, and then hashed it again.
The server performs the same operation and compares the results. It's not the greatest scheme (knowledge of single-hashed password is the same value as knowing the password), but it was good enough for a game server. There was some further complication because the hash is performed in an endian-dependent way and it doesn't use the standard initialization or padding.
I spoke about the packet format a bit above. Even though it was very tempting to just import and export raw structures, we tried to do this in a portable way [and] mostly succeeded. I created a set of types and functions called bn_type which implements these:
[These] are stored as arrays of unsigned chars as they would be seen "on the wire." However, we rely on the compiler not adding padding between these elements in the packet structures. We use gcc extensions to keep gcc from doing this, but everyone uses gcc. I don't know of any compiler that adds padding to arrays of unsigned chars, so we are safe until we run into that.
These types are used to build up structures which match the packets sent and received by the server. There is a union of all these packet types, and a character array which allows the network code to read and write them without conversion.
Crittenden: Their ladder support took quite a bit of time as well. They use an extended Elo rating system. This computes a probability to win, and then assigns points based upon that probability and the outcome of the game. The number of possible points decreases as a player advances. There are some pretty complicated formulas for calculating these probabilities, and as the number of players in a game increases, so does the complexity of the calculations. Ross worked a way to figure out the various permutations for 2-5 players, and we quickly added support for that.
While the packets are fairly straightforward, there were endian issues to deal with, and fetching the data out of the packet was a bit tricky. Ross devised a clever system of function calls to fetch 1, 2, 4 or 8 bytes of data from the "packet" at a time, which made programming much simpler. We built upon this base and made a bunch of "helper" functions to fetch string contacts, integers, whatever.
Wen: What other technical challenges do you anticipate will come up for BNETD?
Crittenden: It appears that Blizzard is actively patching their games to not work on non-Battle.net servers. This is really a shame, and I doubt there is much on the server side we can do to prevent this.
Combs: One area of BNETD which has never been completed is our server-interconnection protocol. This was originally designed to hook servers together into a kind of cluster. This is much more difficult to write than a stand-alone server, since it will involve locking and cache coherency. Currently, it allows sharing of games and channels, but there are some bugs.
I suspect Blizzard will be moving toward more encryption and more heavily involved servers. Currently, most of the logic is on the client side. Moving things to the server allows Blizzard more control over the environment.
Wen: Do you have legal advice for others taking part in open source projects that deal with reverse engineering?
Combs: The current environment is not just the fault of special interests. We, the citizens of the U.S., have allowed it to get to this point. We should make an effort to stay informed and get involved. Anyone implementing a clone or work-alike should remember the DMCA. Try to think of every possible way that any interested party might claim you are bypassing copy or access controls. If there is any question, you should contact the EFF.
When reverse engineering, stick with 'exposed' interfaces and communications. If you decide you need to de-compile code or inspect others' implementations, it's best to talk with a lawyer so that proper 'clean room' techniques can be used.
Wen: What other important legal lessons can you share?
Combs: The first is to clearly and prominently state your goals and intentions on the front page of your site. Too many people made assumptions about our group. Most of the assumptions could have been easily dispelled. Let me list a few here: We are not a new group; we have been around since 1998. We do not encourage or condone piracy. [Editor's note: Unlike Battle.net, the BNETD server software does not check whether a game client connected to it is a legal or pirated copy.] We don't write hacks and/or cracks for use on our servers or on Battle.net. We don't modify any client-side executables or libraries.
Another lesson is to get legal advice sooner rather than later. I learned that the DMCA is more than a theoretical problem. While reading about Eric Corley, Dmitry Sklyarov, and others, I didn't realize that I might be in a similar situation someday.
Crittenden: We probably should have talked to a lawyer years ago to get an opinion on whether what we were doing was legal or not. I'm not sure how much it would have helped in this case. But it might have gotten us into a conversation with Blizzard much earlier, and perhaps in a less confrontational way.
Howard Wen is a freelance writer who has contributed frequently to O'Reilly Network and written for Salon.com, Playboy.com, and Wired, among others.
Return to the ONLamp.com.
Copyright © 2009 O'Reilly Media, Inc.