Simulations 101

A simulation is a statistical model used to make informed predictions. Since simulations seem to have a mystical aura these days, due to all this climate stuff, it’s in everybody’s best interest to understand that no, they aren’t magic, and yes, they’re actually quite simple.

You need four things. You need data, from which you can create a model. You need simulated input data to feed the model, and that will produce simulated output data. Because the output are simulated, you need to run the simulation model repeatedly (these are called iterations), and the more iterations you run, the more reliable the output data are. Because you have multiple iterations and therefore, multiple output data, they are interpretated statistically to produce a single output.

Sound complicated? Well, simulations can be nightmarishly complex, but the general concept is actually pretty simple.

Let’s say you are a new freshman at Some State University, and your finances are tight. You did not purchase a parking sticker at registration because you are mathematically savvy and you wanted to determine whether it would be cheaper to buy the sticker or pay parking tickets. Let’s say you know that there is a 30% chance that if you park illegally in the lot outside your classroom building, you will get a ticket (we’ll ignore how you’d get that information). A parking sticker would cost $140 per semester, and each parking ticket would set you back $25. There are 15 weeks in the semester, and after looking at your schedule, you have determined that you would have to park in seven different lots every week (that’s 105 times a semester, and each time, you have a 30% chance of being ticketed).

The 30% chance of being ticketed is the probability you have extracted from the data (again, for the purposes of this, we’ll ignore how you got it). Let me show you how simple this is.

Imagine a roulette wheel with 100 pockets, numbered 1 through 100. Get a piece of paper and a pencil, and write Ticket Y and Ticket N on it. Spin the roulette wheel, toss the ball onto it, and wait for it to land in a pocket. If the number of the pocket is 1-30, make a hash mark under Ticket Y; otherwise, mark Ticket N. Now, because you are going to do this 105 times throughout the semester, repeat this process 105 times.

You have just completed one iteration of the simulation. The more iterations you do, the more reliable your results will be, so do 99 more iterations (by the way, do you see why these are known as Monte Carlo simulations?)

When you have finished all 100 iterations, average the Y and N hashes for all of the iterations (we do other things too, like look at the standard error and so forth, but that’s for another time). Now, multiply the average number under Y, multiply it by $25, and compare it to the cost of a parking sticker.

Using the roulette wheel is simulated input data. It isn’t real, because it’s not really parking in those lots. But it produces a random number, and since you know that the probability of getting a ticket is 0.3, you can determine, based on the simulated data, whether you get ticketed or not. So you can create a simulation model to determine whether it will be cheaper to buy a sticker or pay the parking tickets.

Okay, sure, you can look at the probability and the rest of the data and figure out that it’s going to be cheaper to buy the sticker. But that was merely a very simple model meant only to explain exactly what a simulation is. A simulation can be as complex as we need it to be. For example, weather affects the chance of being ticketed (meter maids don’t like being out in the rain and snow any more than you do). So if the weather is bad, the probability of being ticketed decreases. Again, as long as we know the probabilities, we can easily create a simulation. Also, lots are policed more at the beginnings of semesters (to catch the new students) and in the final two weeks (studying for and taking those final exams). If you have the probabilities, you can create the simulation. Staffing is tight, so lots are policed in shifts throughout the week, so the probability of being ticketed in a particular lot depends on the day of the week and the time. But again, as long as you have the data and can extract the probabilities, you can create the simulation.

This is what we call a manual simulation, where we use a raw probability to calculate the outcome, and for all but the simplest problems, isn’t very sophisticated. But we can use other software packages (the @Risk add-in for Excel, for example) which uses the distribution of past data instead of probabilites extracted from it to create highly sophisticated simulation models.

A simulation is only as good as the input data and the model. If you got, say, the probability of being ticketed wrong, your simulation output would give you an incorrect prediction. Likewise, if you set up your model wrong and got one of the calculations incorrect, you would get an incorrect prediction. Keep that in mind as you read about what this or that simulation predicts.

I learned something today. System restore.

It’s working.

Now, because I get a lot of this idiotic, unhinged stuff in comments and you never see them because I flush them down the digital toilet where they belong, this time, I’m going to approve it.

So without further discussion, I give you our insane and illiterate TALLYBAN!

Off to Best Buy. Why? Well, I thought a slightly larger font would be easier to read on my desktop, so I changed it. Now, if I boot, I get the BSOD, unless I boot in safe mode — and if I do, of course, I can’t change the monitor settings.

I called Best Buy and they said bring it in. So I am.

It looks to me like a setting that could cause a fatal system crash probably shouldn’t be available at all. But maybe that’s just me.

The world’s all time greatest keyboard: The IBM Model M. I hate this keyboard (my notebook). I have an old clicky Dell keyboard on my desktop (the one that came with it is still in the box). I picked up the Dell keyboard at the university warehouse, and I like the touch, but it has that damned Windows key on it, and I hate that thing. I have about four keyboards in boxes that have never been taken out. Keep your squishy keyboards.

Megan McArdle is requesting recommendations for good introductions to Sci-Fi, for female readers. I agree with one of her commenters:

Orson Scott Card, Ender’s Game. The most beloved SF novel of the last 30 years. If you don’t like it, well you probably just won’t like SF, period.
Robert Heinlein, Future History series and Starship Troopers. Classic, core SF
Isaac Asimov, the Foundation Series. Again, classic SF
Arthur C Clarke, Childhood’s End, The City and the Stars, The Deep Range.

But it’s summer, the time of year for something you can really get your teeth into, and anyway, I’m really not very good at recommending books for beginners, as it were — although I highly recommend the above to anyone. So for those who want a really good read, and not just a good narrative, here are my Sci-Fi suggestions.

Dune, Frank Herbert. (Note: Herbert is a one-hit wonder. I don’t recommend anything else he wrote. His first two sequels were, as far as Sci-Fi goes, entertaining, but came nowhere close to the first novel. Reading his other books does give you additional appreciation for Dune, if only because you wonder how he managed to write such a novel.)

Dune is not a book you can whizz through. In fact, I’d suggest that you read it, then immediately start over and read it again. You are plunged into an alien, yet familiar, feudal world far in the future, where everything is political, and the politics are extremely complex. The first time through, you may not understand all of the motivations. That’s fine. A novel this complex can’t be absorbed in a cursory reading (although it’s also a large novel).

Dune melds the scientific with the deeply spiritual, in a novel of massive scope for any literary genre. It’s unfortunate that Herbert only had one great novel in him. His writing style is not dense. It’s the content that gives Dune its complexity — and once you’ve read it, you will understand why any attempt to film it is doomed before it begins.

Book of the New Sun, Gene Wolfe.

Wolfe is the most literary Sci-Fi author, more so than many who write literature. This is a long novel, which used to be released as four books; it is now released as two: Shadow and Claw, and Sword and Citadel. Like Herbert, Wolfe plunges you into a complex world with no explanation of what’s going on, or what anything is. Unlike Herbert, Wolfe will send you running for the dictionary.

Wolfe writes extraordinarily beautiful prose, and New Sun is a book whose prose you will savor. Herbert’s world is feudal. Wolfe’s world is dark and medieval, even in ways barbaric, yet full of wonder.

As you read, it will slowly dawn on you what Wofe described some hundred pages ago, and you’ll say, “Oh, that’swhat that is! I get it!” When you begin, all is mysterious. When you finish, all is clear, and you will wonder why it seemed mysterious in the first place.

Beautifully written, beautifully executed, and beautifully crafted, Book of the New Sun belongs on every bookshelf. You couldn’t read it quickly even if you wanted to, and you won’t want to. It’s a book to be enjoyed as it very slowly gives up its secrets.

But if a rollicking good time is what you’re after, try Cities in Flight by James Blish, or The Mote in God’s Eye by Niven and Pournelle.

HBO is replaying the John Adams mini-series starting tonight, and since this was one of the things on the hard drive of the dead DirecTiVO, I just set the box to record them.

Then I realized that today is the day we get the HD dish installed. So once that happens, I’ll set the box to record off HBO HD.

Except.

If I record something from an HD channel, then burn it to DVD, can it be played on a regular TV? Enquiring minds need to know.

Check this out: WP-SpamFree has blocked 25,320 spam comments.

And I installed it less than a month ago.

I use Akismet too, but it doesn’t catch nearly as many, and I have to scroll through the Akismet queue. So even though it doesn’t get posted, I have to deal with it. WP-SpamFree just gets rid of it.

as in Firefox has crashed one time since I logged on at 6 am this morning.

Firefox has only crashed on me four times so far today.

Okay, twice. Two of those times I got sick of watching it take up all of my CPU and RAM while it froze, and after several minutes, shut down the process. I always have a task manager running. It’s so I can kill Firefox. So what happened to that lean, mean, fast browser?

and Firefox crashed. Ka-boom.

Firefox: The new Netscape, unstable, bloated, and slow as hell.

Sturgeon’s Law, ironically (or appropriately, depending on how you look at it) was proposed by Theodore Sturgeon, a Sci-Fi writer, after years of defending the genre from critics. There are two corollaries (we’ll be primarily concerned with the better known second corollary):

1. Nothing is always absolutely so
2. Ninety percent of everything is crap

The problem Sci-Fi writers and fans have always had, the problem Sturgeon ignored, is this: If you’re going to play the game, you have to play by the rules. You can’t create your own rules, then complain when they don’t let you play.

If Sci-Fi is going to be literature, then like every other genre of literature, it must revolve around the characters and the plot.

I’m not knocking Sci-Fi. I’ve been a Sci-Fi geek for decades. But something has happened over the last twenty years that has made Sturgeon’s Law inapplicable, since far more than 90% of Sci-Fi these days is crap — and here, I’m referring primarily to filmed Sci-Fi, either television or movies.

Well, somethings have happened. The first was Gene Roddenberry (yes, I know, I’ve ranted about the Star Trek franchise before, but this is not a Roddenberry rant, and I’ll try to constrain myself, okay?)

It’s not about the tecnhology, stupid

This is one of the two ways in which Roddenberry degraded Sci-Fi. In the Roddenberry universe, there’s a gadget that does everything. In addition to phasers and breaking the speed of light, we have transporters, replicators, holodecks, time travel, you name it, there is no technological barrier some group on the franchise has not broken.

Phasers are fine, I suppose, although ray guns are just this side of silly, and in Sci-Fi, you pretty much have to have dispensed with Einstein. Holodecks, while extremely annoying because they encourage all kinds of idiotic plots, are, with a sufficient amount of computing power, feasible (well, provided that the dimensions of the created environment do not exceed those of the physical space, as they so often did in the franchise, you know, like when they went mountain climbing on the holodeck).

Transporters and replicators, however, well, don’t get me started. I said I wouldn’t rant, and I won’t. They violate the probability principle, discussed below. But back to the point.

The problem with having a neat techno-gizmo-gadget that does anything you want it to do is that the technology becomes a deus ex machina, as it did in the Roddenberry franchise shows. Scotty was always reconfiguring the dilithium crystals to solve whatever problem had arisen, and then there was the tricorder, which seemed to be sort of an all-purpose magical box.

A great Sci-Fi author, Arthur C. Clarke once said, “Any sufficiently advanced technology is indistinguishable from magic,” but he was referring to perception, not reality. Technology is always distinguishable from magic in one fundamental way: Technology is constrained by scientific laws, and magic is not. Roddenberry’s world did not have technology; it had magic devices presented as if they were technology. When they were constrained by laws, those laws were conveniently forgotten or changed at whim, as the writers needed, hence “reconfiguring the dilithium crystals,” or “sending a gravimetric burst” or pick your favorite example of magic-as-technology every other episode.

It is a necessary precondition of Sci-Fi that the author must create a world constrained by laws. The author (or writer) with artistic integrity creates his world and consistently subjects everyone and everything in it to those laws. The hack writer creates his world, then dispenses with or changes the laws whenever it suits him. Roddenberry and his writers were, by that criterion, hacks. That they used (mostly, but not entirely) actual scientific concepts which they then perverted at will in order to get characters out of situations makes the Star Trek franchise bad Sci-Fi.

It’s probability, not possibility

Sci-Fi authors are far too imaginative, or perhaps imaginative in inappropriate ways. Because we must suspend disbelief to read (or watch) and enjoy Sci-Fi, the world and the scientific laws which govern it may be improbable. The events and the plot, however, cannot cross the improbability line, lest they become absurd.

Yesterday, I saw a particularly ridiculous episode of Voyager in which they transported “photonic matter” onto the ship from a star. The “photonic matter” (except that it had suddenly become “photonic energy,” but why would anyone in Sci-Fi worry about the distinction between matter and energy?) had somehow leaked into the holodeck system, where one of the characters was running a Beowulf program. The “photonic whatever” (matter? energy? do the writers know there is a difference?) turned out to be a (wait for it) “photonic lifeform” and became Grendel in the holodeck program.

Where do I start with this drivel — and it is drivel, from beginning to end. And what was I talking about, since I am mentally ranting, even though I’m not ranting here . . . oh yes.

Yes, it’s possible that “photonic matter” could have 1) developed into some kind of living being, and 2) evolved intelligence, but then, it’s also possible that those grains of salt in your shaker are actually nanodevices the Illuminati use to track your every movement for some nefarious purpose. Nearly anything is possible.

The operating principle is not possibility, but probability. So while a “photonic lifeform” is technically possible, it’s immeasurably improbable. It is so improbable that it’s ridiculous. But let’s forget that, and say it’s likely that “photonic matter” would be alive and intelligent.

Is it possible that this “photonic lifeform” would morph into Grendel on the holodeck? Well sure, but it’s possible that we’re being controlled by reptilian aliens masquerading as human beings. It may be possible, but it’s not very probable, is it? No? So why did these writers produce this nonsense? (Answer: Because they’re hacks.)

Put a moratorium on clichés

I said above that Sci-Fi authors are imaginative in the wrong ways, and here’s why. Sci-Fi is plagued by tired themes that have been beaten to death over the last fifty years and need to be retired for at least the next hundred. “There are things man was not meant to know/do!” is one that’s been with us since the beginning of the genre, and is so old and dead that it needs to be discarded. Yet authors seem incapable of coming up with anything but these dead, tired, whipped to death themes. They just dress them up in new ways, and that’s not imaginative, at least not in the right way.

However, there are two of these themes that almost ensure bad Sci-Fi: Utopianism, and it’s antithesis, dystopianism.

Like any other theme, these were both interesting when they first appeared. But by the time Ryker was spouting such idiotic lines as, “We no longer enslave animals for food production” on television, utopianism had become a parody of itself (hence, the idiocy of the line). Mad Max was cool, but by the time we got Waterworld, dystopianism had become farcical.

Of all the dead clichés in Sci-Fi, these two annoy me more than any others, because they’re both so inherently ludicrous, because they almost always overwhelm the story, and because they nearly always lead the writer into abject stupidity. Neither is original, and neither is interesting. Both need to go. Permanently.

Fantasy is not Sci-Fi

Back in the 60s and 70s, there was a war going on between the hard-core Sci-Fi folks and the squishy, not-very-scientific speculative fiction folks. I fall firmly into the former camp, although I did think at the time that some of my fellows were anal to the point of being silly.

That war no longer rages as it did then, and the speculative fiction folks seem to have won. So at Barnes and Noble, Sci-Fi and Fantasy are shelved together, and the Sci-Fi Channel is more likely to be airing a werewolf movie than a space flick. And crucially, nobody seems to notice.

This is important because I believe that the victory of the squishy, not-very-interested-in-science speculative fiction folks is the reason that shows revolving around magic, such as the Roddenberry franchise, are presented as if they were Sci-Fi. I also believe this victory is the reason that possibility rules, and nobody much cares about probability.

Understand that I’m not knocking Fantasy. But it isn’t Sci-Fi. Magic isn’t science, even when it’s presented as if it were.

Plenty of highly educated science geeks love speculative fiction, and don’t much care that it is presented as if it were Sci-Fi, but they are not the general populace. Back in the 60s when the hardcore Sci-Fi v. squishy not-so-sci speculative fiction war was at its hottest, we lived in a heavily science-oriented society.

That’s no longer true. We now live in a society in which science is under siege from all sides, and people now believe in “alternative medicine,” crystals, auras, chakras, chi-energy, aromatherapy, feng shui, accupuncture, “organic food” and “body toxins,” oh, the list never ends. The fundamental irony is that all of these forms of magic, just like the Star Trek franchise, use para-scientific babble to legitimize their magic.

The language of science, then, is still highly valued, but science itself, not so much.

And these are not correspondent in some way to religion. Magic has supplanted science to a large degree precisely because in order for the magic to be legitimate, it must somehow be presented as if it were science.

I’m not implying causation here. Whether there is any, I cannot say. But “magic-as-science” predominates both in our society and Sci-Fi.

But waving crystals to heal somebody is magic, not science, just as the Roddenberry franchise is Fantasy, not Sci-Fi.

Good Sci-Fi

Both Battlestar Galactica and Blade Runner are not only good, but great Sci-Fi, for mostly the same reasons.

Neither falls prey to the traps that the Star Trek franchise exemplifies. In BSG, there are only two identifiable areas in which the world is more technologically advanced than we are: Space travel (like I said, you pretty much have to dispense with relativity to have Sci-Fi) and artificial intelligence (the people created the cylons, recall). There are no ray guns — the weapons shoot bullets. There are no transporters or replicators or holodecks. The technology in BSG is so minimal that it cannot detract from the characters and the story.

Few, if any, of the events in the plot cross the probability line. There is a spiritual streak in the show, undeniably (the show is supposed to be heavily Mormon influenced, much as The Magic Flute is Masonic, although I don’t know enough about the LDS to comment on that), so we have Six, Boomer, and Roslin sharing the same dream, for example, but — and here is the crucial point — it isn’t portrayed as if it were science. The dream is a dream, and nobody knows why or how they’re all sharing it. It isn’t “reconfiguring the dilithium crystals,” because it isn’t presented as if it were science.

The same is true of Blade Runner. We get hints of lots and lots of advanced technology, particularly in the scenery, but very little of it is part of the story. It never intrudes, much less takes over, as it does in every episode of the Roddenberry franchise. None of the events is so improbable that it cannot be believed.

Most importantly, both BSG and Blade Runner are about the characters and the story, and never contradict their own laws. Never is magic presented as if it were science, and in both, science really has very little to do with the story. And that’s because good Sci-Fi, like any other genre of literature, is about the characters and the story.

If we’re talking about literary Sci-Fi, books or short stories and not movies or television, then great Sci-Fi authors abound. Isaac Asimov, Arthur C. Clarke, Robert Heinlein, Orson Scott Card, Larry Niven, Jerry Pournelle, the list goes on and on. And so do great novels, at least one of which was written by a one-hit wonder (Dune). In fact, my favorite Sci-Fi novel, er, novels, er, series isn’t, as you might imagine, prototypical hardcore Sci-Fi. The science is minimal, but it follows all of the rules (or I would find it incredibly annoying). I refer to the Book of the New Sun, by Gene Wolfe. The Foundation Trilogy, I, Robot, Ender’s Game, Stranger in a Strange Land, Childhood’s End, Cities in Flight, again, the list goes on and on.

The problem with Sci-Fi on film is, I think, that writers are far too tempted to substitute gizmoness and special effects for substance, which is an even larger problem than in novels, because of the time limits involved — see I, Robot, the recent movie, as an example, and I Am Legend as an all too rare counterexample. Here, again, BSG excels, as the writers pack the maximum amount of substance into each episode, with only a few exceptions. Then, there are the Sci-Fi novels that are so complex that they cannot be successfully filmed — and speaking of, I read the other day that they’re going to film Dune again, a project that’s doomed before it begins — but that’s another topic for another day.

We need more shows and movies that revolve around the characters and plot, with less technowizardry and fewer special effects. I’m not holding my breath waiting for either.

As an addendum, unlike all of the Star Trek geeks, I rather liked Enterprise, precisely because they didn’t have a magic gadget for everything, and weren’t able to use gadgets as deus ex machina devices. I thought Enterprise jumped the shark when the writers introduced the time travel nonsense, pulling the show back down into the same, tired, not-very-sci Sci-Fi camp as the other shows.

Life imitates Heinlein. Verra kewl.

Starship Troopers, anyone?

World Wide Telescope (warning: do not click the link if you plan to get anything done in the near future).

So I get up, fire up the browser and go to gmail, and the first news link at the top is this:

Update: A Florida-based firm advertising cheap, Mac-compatible systems running on standardized hardware has created a swirl of controversy this week–and drawn an angry response from one of the principal hackers involved in developing the software involved.

Aside from the legal ramifications of selling “white-box” systems running Apple’s Mac OS X, questions have arisen over the exact nature of Psystar, the firm advertising the systems; and even whether it exists.

Yeah, let’s get that out of the way before we get to the drooling. This Mac clone company may be a scam. So maybe you’d better not order one until we know one way or another. As for the legal ramifications, Apple lifted BSD and didn’t pay a cent for it, and now, Apple and all of the AppleIdiots are giving Apple credit for their OS — which isn’t their OS, it’s BSD, and they don’t deserve any credit for it at all. So I have zero sympathy for anybody “stealing” Apple’s stolen OS. And by the way, the same is true for Apple’s pre-BSD OS and their click-the-icon idiot GUI, which they lifted from Xerox, and didn’t pay one cent for. Not. One. Cent. Apple’s OS has never been theirs. It’s always been somebody else’s, and they have never paid a dime for it.

God, I hate Apple.

Anyway, on to the slobbering:

Apple allowed a handful of companies to manufacture Mac clones in the mid-1990s, but Steve Jobs put an end to the practice when he resumed his chief executive duties in 1997.

There we go. And that’s exactly why Apple has only a tiny market, and will never have more than a tiny market. Steve Jobs is an idiot. But that’s not really the idiocy I was going to mention. This is:

If the Open Computer does in fact exist, Apple might not be too happy about it–but neither will the hackers who have developed the “OSx86″ software Psystar claims to use. Thus far, Apple has made no concerted effort to stamp out the OSx86 project, which consists of various patches allowing Mac OS X to run on commodity hardware, no doubt in part because it has been used only on a small scale and is relatively difficult to implement.

Psystar is proposing, however, to use OSx86-developed tools as the basis for its systems. On its Web site, the firm specifically mentions PC EFI V8, an emulator for the Extensible Firmware Interface (EFI) that lies between OSX and the Intel firmware.

Netkas, the programmer who led the PC EFI effort, retorted angrily to Psystar on his blog. “This is a violation of my authorship rights on PC EFI V8,” he wrote. The patch’s license forbids any “redistribution… for direct or indirect commercial purposes”, Netkas wrote.

Before I start, let’s get one thing out of the way. I have absolutely no problem with open source. None. But this is where open source jumps into the moron pool. In a nutshell, here’s what’s going on.

Open source developers wrote patches so Apple’s OS — which isn’t Apple’s OS, and has never been Apple’s OS — would run on x86 machines, that is, Intel boxes (PCs). It was an open source project, so the developers chose to produce these patches and chose not to make any money from their work.

So far, so good.

The idiocy sets in when these developers who choose to work for free have fits because somebody else uses their product to make money. Sorry, idiots, but them’s the breaks. You chose to write code for free. I’m sorry if Joe has more sense than you do, but you produced the code; he’s going to make some money. That’s what America is all about. If you don’t like it, don’t write code for free — or don’t release source code.

See, the reason these idiots get their panties in a wad is because they’re a bunch of bed-wetting socialists. They’re the morons who have idiotic posters like “INFORMATION WANTS TO BE FREEEEEEEEE!!! in their bedrooms in their parents’ homes (because they don’t have jobs). They don’t think anybody should make money off anything. They think they should get a big paycheck for sitting on their lazy asses, drinking Red Bull, and doing nothing.

Can you tell I have no sympathy for these mouthbreathers?

Understand that if these developers were raising hell because their copyrights were violated and wanted money, I’d be behind them, one hundred percent. But they’re only trying to keep other people from making money, so they can jump right off the rock into the sewer where they belong.

while you can, because Apple will no doubt sue. Jobs et a. have never been smart enough to figure out that the open architecture of the IBM and DOS led to more IBM apps, and was the reason Apple was sent permanently to the fringe.

The idiots at Best Buy did not slave the hard drive from the dead computer, and there’s data on it I need, so I got an external2internal kit. Plug the ATA and power in, stick the USB cable and power cable in the back, and bingo. I had an older one, so I’ve used these before, but it was a pre-USB serial port and I’m not sure this computer even has a serial port, plus it would be really slow.

The directions specifically say to jumper the drive as master. I thought that didn’t make any sense, but I made sure it was still master, hooked it up, closed the kit, and plugged it in.

Vista saw the device, installed the driver, but nothing in My Computer (except it’s not My Computer anymore, but Computer, one of the few things I like about Vista — not as juvenile or cutesy). Been there, done that with flash drives. Windows sometimes doesn’t assign drive letters to USB storage devices. No problem. So after I found it (they’ve moved things around, of course), I opened up disk management.

Like I said, been there, done that. I should have been able to find the drive with no letter next to it, right-click on it, click on change/assign drive letter, and that would be it. Well, the drive shows up without a drive letter, but the partition says MBR healthy primary, and if I right-click on it, all of the options EXCEPT delete partition are greyed out. I don’t want to delete the partition: I want to get to the data on the drive.

So there’s the problem. Vista sees the drive, but won’t let me do anything but delete the damned partition and I want the data on the drive. I’m more than willing to delete the partition and format the drive after I get the data off, but I want the data first.

Oh, and did I mention that I got the BSOD yesterday? On a computer that’s not even three weeks old?

Here’s a useful idea for Firefox (or an add-on): Instead of idiotic crap like “Turn everything on your screen psychedelic!” how about something that loads NO external data, you know, like ad.doublecick.net, google syndication, nothing, just the data from the domain, how about that?

No, I guess that makes too much sense. You idiots are far too into reproducing the disaster that was Netscape (a browser that died a very well-deserved death).

Yanked the hard drive, stuck it in the kit, and plugged it in. The computer picked up the device, but it takes a ridiculously long time for USB drives to show up in the explorer window, so I’m waiting.

So that new computer from Best Buy. They were supposed to slave the hard drive from my dead one so I could get to my files and stuff, but they didn’t. The guy told me there wasn’t room for it.

Crap.

This thing has more than enough room. But I have an internal2external drive kit, so I can pull the hard drive, plug it into the kit, then plug it into my USB port. However. I have no idea where my geeky screwdrivers are — you know, the teeny tiny screwdrivers you need for inside the case. They could be anywhere. I haven’t seen them since we moved. So I guess it’s off to Wal-Mart to buy a nice, cheap little screwdriver.

I finally solved the problem (without the help of MS customer support), and got MS Office 2007 installed on my new desktop (I installed Photoshop this morning, and I’ll install SPSS later today). Playing around with Excel reminded me of the reason I love Excel 2007, which in turn reminded me of, well, keep reading.

Months ago, I saw several dishonest statements on edublogs (sorry, it’s been months ago, and I have neither the time nor the inclination to find links at the moment), stating that zeroes are not valid scores. This is not dishonest in itself, but the threads upon which I commented were those in which the author claimed that zeroes were not statistically valid scores.

That is false.

Zeroes are often statistically invalid for calculating descriptive statistics for the class, such as means or standard deviations. But that does not imply that a zero is an invalid measure of a student’s performance. Note that I pointed this out in comments on these blogs, and got no reply. I assume, therefore, that the statements were made not out of ignorance, but dishonesty.

How does a student get a zero on an exam or assignment? Theoretically, a student might put his name in the exam, then get a zero because he didn’t know any of the answers, although the probability of this decreases as the number of questions increases (that is, it’s almost impossible on, say, a 100-question exam, but entirely possible on a 10-question quiz or assignment).

Assuming that each question has four distractors, and therefore that the probability of randomly getting any question correct is 0.25, the probability of getting a zero on a 10-question quiz is 0.056, or 5.6%; the probability of doing the same on a 100-question exam is 3.2072*10^-13, or 0.00000000000032, or 0.000000000032%.

A student could put his name on an exam, quiz, or assignment and answer nothing, that is, turn in a blank. But how frequently does this happen? How stupid can a student be to ensure a zero, when he could randomly answer, and get a somewhat higher score?

Or a student could not show up to take the exam or quiz, or not turn in the assignment. We’ll return to this scenario in a moment.

If you are calculating descriptive stats on a 100-question exam, and if you have zeroes from students who took the exam, but against all odds, managed to get zeroes (as I said, the probability of this is microscopically small), the zeroes are valid, and should be included in the calculation. Why? Because the students who got zeroes took the exam. Therefore, when calculating descriptive stats for the class, that is, answering the question, “How did the class do on the exam?” requires that you include the zeroes.

This, by the way, has never happened to me, in many years of teaching, grading, and calculating stats. The odds are far, far too small.

The same is true if instead of a 100-question exam, you are calculating class stats for a 10-point quiz or assignment (I have had this happen, quite often, because the probability of getting a zero is much, much higher).

To sum up: If you are calculating performance, and the measure of performance for some students is zero, those zeroes are statistically valid. Leaving them out will artificially inflate your class means.

But we have that other scenario, the one I said I’d address, where Johnny got a zero because he didn’t take the exam (or turn in the assignment). What about that?

Are you going to let Johnny make up the exam? If you are not, then his zero should be excluded from the scores when you calculate class stats, because he did not take the exam. Including his zero will artificially lower your class mean since he did not participate in taking the exam.

However, if you are going to let Johnny make up the exam, the question becomes when you let him make it up. If he takes the exam the day after the class took it, say, then include his score (whatever it may be) when you calculate class stats. But if you let him go a week or two, or worse, longer, before he makes up the exam, do not include his score when calculating class stats. By giving him all of that additional time, you make his score a different measure than those of the rest of the class. You cannot compare his performance on the exam to the performance of the rest of the class.

That leads us, of course, to the question of making up exams or accepting late assignments. This, I suspect, was the agenda of those edubloggers who falsely claimed that zeroes are not statistically valid scores, particularly since all were proponents of laissez-fâire grading policies.

If you work in the primary or secondary schools, your grading policy may very well be dictated from above, and you have no choice. But ignoring that, I hold that, at least in the secondary schools, such mushy gooey laissez-fâire grading policies are destructive.

Note that there are very good reasons for not showing up to take an exam, or not handing in an assignment on the due date. Grandmothers really do pass away. Students really do have religious holy days (well, at least some). It is only reasonable to allow students with valid reasons to make up exams or turn in late assignments. I refer here specifically to students who do not have valid reasons for showing up to take the exam (and “my alarm clock didn’t go off” is not a valid reason).

You teach students bad lessons that must be unlearned, with a great deal of pain for those students, when you let Johnny make up the exam. You teach Johnny that scheduling means nothing, that he may come and go as he likes, and do his work or not as he likes, without consequence. Johnny will not remember you kindly later in life when he fails his classes at the university, or is fired from his job because of the lesson you taught him.

Just as bad, perhaps worse, is that you teach the students who are responsible enough to have shown up for the exam that you have no regard or respect for them. You do not care that they are responsible and take education seriously, while Johnny does not. And if you’re sending that message, then you have no right to complain about students not taking education seriously, do you. You do not take it seriously, so why should they?

If you set your grading policies, and if you teach in the secondary schools or above, then there is no excuse for laissez-fâire grading policies, where you allow any student to turn in any assignment at any time he likes, unless none of your assignments is due on a specific date. You have no right to hold Johnny and the rest of the class to two different standards. It’s called fairness.

Of course, we always got a list of every possible religious holy day from every imaginable religion on the planet every semester, far too many to avoid scheduling exams or due dates on holy days. So we set a policy: If you cannot take an exam or turn in an assignment because of religious observance, tell your professor and make alternative arrangements before the date of the exam or due date, and we will happily accomodate you. Come afterwards and claim you couldn’t take the exam because you were at Good Friday services, and you get a zero. For “acts of God,” we only required documentation of some kind.

Still, I always got a few students who didn’t take the exam, and one or two who just disappeared, usually early in the semester, and didn’t drop the class. Those zeroes are invalid, and cannot be included in calculating class stats.

That leads us to Excel 2007. Because I always had zeroes that had to be exlcluded, I could never use the AVERAGE() function, and instead had to use SUM(range)/COUNTIF(range,”>0″). But Excel 2007 now has the AVERAGEIF() function, more than enough reason to upgrade. (Unfortunately, I don’t believe there is a STDEVIF() function.)

But back to zero scores. Yes, they are in many cases, valid scores. A zero is certainly a valid measure of how a student performed if he couldn’t be bothered to take the exam or do the assignment. In other words, a zero is a valid score for assessing that student’s performance. That student chose the zero when he didn’t take the exam. That it may not accurately reflect his knowledge is irrelevant, since by choosing not to take the exam, he made his knowledge irrelevant. Pandering to such irresponsibility undermines the educational mission, both with the irresponsible dolts and with the responsible students, and it undermines your creditiblity as an instructor.

By the way, there’s a rather entertaining article about multiple choice questions and probability here, if that sort of thing turns your crank. And if you’re curious, no, I have never used guessing penalties (you know, where you subtract a value from the score for each incorrect answer), but I did have an otherwise abominable professor in grad school who dealt with random guessing on tests by using paired T-F questions of the following format:

Statement A.
Statement B.

A. Both statements are true
B. Both statements are false
C. The first statement is true and the second statement is false
D. The first statement is false and the second statement is true

I thought it ingenious.

God knows I don’t want to sound like my grandfather, particularly today, but as you know, I just got a new computer. Do you remember when computers came with manuals, you know, with real information in them? You bought a TRS-80, and you got a CP/M manual. You bought a PC, and you got a PC-DOS (or IBM-DOS, if it was an IBM box), later, an MS-DOS manual. With either, you got a BASIC manual. Remember?

Today, computers come with a diagrams-only setup sheet, and for those who have a 100-word vocabulary, a miniscule setup book, with diagrams and a handful of basic sentences. They come with no information. None at all.

It’s the MacIntosh Moron factor. You don’t need to know anything — just click on the happy face! Steve Jobs has done more to dumb down this country than any other single human being. Sure, Gates helped when he jumped on the GUI Idiot Box train, but it was Jobs and the MacIntosh computer for idiots.

God, how I despise Steve Jobs and Apple. When Jobs dies, there will be a special circle in Hell just for him.

If you use Wordpress, you’ve no doubt noticed that it seems to hate tables. Put the code in, and Wordpress pushes it way down on the page. The reason is a bug in the Wordpress engine that inserts paragraph and break tags in your post. I found a solution (but sorry, I don’t remember where, or I’d credit it).

Insert the following code at the bottom of your stylesheet:

.table br, .table p {
display: none;
}

You’ve created a class named “table” (you can call it anything you like). Now, immediately before the table tag, insert:

<div class=”table”>

and immediately after the closing table tag:

</div>

Chalk it down to not being awake. The internet connection suddenly died. I rebooted the cable modem, and it booted up just fine, but no connection. I waited, then repeated. Same result. So I called tech support.

No outage. So he says, “Do you have a router?” and before he went any further, I felt like an idiot. I rebooted the router, and bingo! that was the problem.

Feel free to jeer. I deserve it.

The iTaser: It’s an MP3 player! It’s a taser! Hat tip: Uncle.

So how’s about that version 15.0.1, other than the idiotic installation? Well, it’s not much different. Improved ODBC drivers and a data exchange engine (same for Informix and Oracle back-ends). Very little difference in the interface. Oddly enough, it seems to run faster, nice when most upgrades run slower because of feature creep (like Firefox).

One of the features I really hated was the graph wizard. Well, it’s just as annoying, and I still hate it. You don’t have to use the wizard to create a graph, fortunately.

There was some talk a few years ago in the department about moving from Excel to SPSS for stats. I came out strongly against the idea. SPSS is a superior (and powerful) stats package, and although it’s not hard to learn or use, provided you know what you’re doing, it’s way more power than students need, it’s not the standard package (Excel is), and it would have cost a lot of money to switch.

Having said that, if you’re going to do serious statistical analyses, consider buying a copy of SPSS or SAS (I’ve never used SAS, but one of these days, I may buy a copy, since the company is right here). You can do more with Excel than most realize (using functions, instead of just the data analysis toolpack), but Excel is a spreadsheet, and isn’t designed to do serious statistical analysis.

And yeah, yeah, I’m working on that stats rewrite.

<rant>

Used to be Microsoft Office was the only software you couldn’t update (you’ve always had to uninstall the current version, then install the new one), and I always put that down to its being Microsoft software. Now, you can’t upgrade anything.

Worse, you can’t just install software. Oh no. I just threw things across the room trying to install SPSS 15.0.1. You have a serial number for the disk and a registration number, and you have to enter both, but no, you then have to get connected to spss.com so they can register your copy even though you’ve already registered the copy.

Of course, the server was down, and I ended up with a trial version. I don’t want a trial version. So I waited, and tried again.

Nope. See, I had a trial version that wasn’t a trial version installed, so I couldn’t register it, even though I’d already entered the registration number, so I had to uninstall it, and go through the whole process again.

And then it opens a browser window and tries to get you to create an account. That means answering all kinds of idiotic, intrusive questions, as well as having to uncheck crap like “Yes, I want you to send me at least forty email messages a day!”

What crap. And in August, I’m going to have to go through this whole thing yet again.

<rant />

to tell you this? Gizmodo says Don’t buy an iPhone.

There is about a 1:157,692,177,645 chance that I will buy one, because:

1. I hate cell phones
2. I hate catch-all gadgets. I want to watch TV on the TV, thanks.
3. It’s cool, and I hate cool.
4. It’s an Apple product, and there a few things I despise more than Apple.

It looks like there’s a nameserver somewhere down. If you get an error trying to reach a site, that’s probably what it is.

I’m thinking seriously of migrating the sidebar entirely to widgets — and I’m a bit puzzled why a theme designer would include archives, posts, etc. code in the sidebar.php file in a widget-friendly theme, when all of them (and more) are available as widgets. The problem with using both is that all of the widgets have to go together (if I’m mistaken, please tell me). The RSS feeds, currently at the bottom of the sidebar, are widgets. If I wanted to use the calendar widget, it would have to go at the bottom with the RSS feed widgets, instead of where the archives are now.

We’ll see. I’m just learning about widgets because until I changed hosting providers, I was stuck with wordpress 1.5.2.

Looks like it, thanks to the folks at Dreamhost. Let’s test it and make sure.

I’m having permalink problems. I had to switch to the default (/?p=xxxx) to get categories and single articles to work. My pages, however, don’t. I’m working on figuring this out, but if anybody has an idea what’s going on, feel free to give me a holler. I get “input not specified” when I try to access one of my pages.

I now really, really, really detest Firefox. I want to either downgrade to 1.5.x, or move to another browser, one that’s supposed to be what Firefox used to be: Fast and stable, not a CPU hog, and with no intrusive “Your files have all downloaded!” popups or automatic updates.

So far today, Firefox has frozen seven times — you know, when there’s supposedly a “script” running on the page. On one of those pages, there was nothing but text. On another, there was no damned script. I know, because I authored that page.

I have had to close and rerun Firefox three times today because it was eating up 100% of my CPU.

And don’t point me to any of those lame about:config hacks. I shouldn’t have to edit a damned thing to make the application stable. Firefox has become Netscape, with feature creep. It’s big and bloated and unstable, and I hate it.

Can I downgrade? If not, what else is out there?

I’m really missing Lynx right about now.

Since we’re going to see Underdog here in about an hour, I decided to shut the desktop down, unplug it (and everything else, except the wireless router), and let it stay unplugged until we get back, just to see if letting it stay unplugged for a few hours will bring both hubs back to life. I know that’s like hoping waving a crystal around over the machine will fix it, but I might as well try.

This started about a week ago when suddenly, my desktop decided my external hard drive needed to be formatted. I tried running chkdsk, but chkdsk crashed. I tried again. Chkdsk crashed again.

I hooked the drive up to my notebook, pulled up a command line, and tried to run chkdsk. It crashed, just like it did on my desktop.

So I took it upstairs to the old 486 running Win2K. And again, chkdsk crashed.

When I hooked it up to the newest notebook running Vista, chkdsk ran just fine. So other than getting a new drive to back up the data and then retiring the drive, I thought that was that.

Uh, no.

I got the new drive, and started backing up data. When I got up yesterday, I saw that after copying about a tenth of the directories and files, chkdsk had started returning “path not found” errors. So I decided to check the drive, and that’s where things started to get really strange.

When I right-clicked on the drive in My Computer and pulled up the properties screen, it reported that 26% of the drive space was free, so obviously, XP knew there were data on the disk. And the directories showed up in the foldier view sidebar on the left. But if I tried to change to one of those directories in the same My Computer window, I got “0 files.” I pulled up a command line and tried dir. Same thing. 0 files.

Then I got the bright idea of trying it on the other machines. Well, this turned out to be a bad idea. Every one gave me a “USB device has malfunctioned” message — which I also got on the desktop when I tried to plug it back in.

And of course you know this is the drive that has all my really important stuff on it, right? If it had nothing but crap on it, it would be running just fine.

That was yesterday. When I got up today, my desktop had one of those obnoxious yellow balloons saying “USB device has malfunctioned,” and note that the buggy drive had not been plugged in. No, this time, the USB device was one of my two USB hubs, so the computer didn’t see any of the connected peripherals. Note that this was a brand new hub I bought and hooked up yesterday. Brand. New. Hub.

I unplugged it, then plugged it back in. Same thing. I rebooted, and this time, my machine returned the same error for BOTH of my hubs. I googled, and found somebody who’d had the same problem and claimed that shutting down, then unplugging the machine for a half hour, and rebooting solved the problem. I decided to try that.

It worked for the old hub, but not the brand new one. I’d had a similar problem before, when one of my peripherals was buggy, so I shut down again, unplugged all of the peripherals from the hub, rebooted, then plugged in the hub with nothing attached, and again, got “USB device has malfunctioned.”

Granted, my desktop has had weird USB issues for a while. If I cold boot with the hubs plugged in, the boot hangs, so I have to unplug them until the XP screen pops up, then plug them in. And my desktop refuses to recognize my flash drive, though my notebook has no trouble with it.

So I have two problems: My hard drive (and data), and that damned USB hub problem. It seems that the only solution to the first problem is to take the drive (and the new one) to Best Buy and have them transfer the files, which will cost $159 (plus tax, of course), and right at the moment, I really don’t have $159 to spend. However, at least there’s a possible solution. I have no idea how to tackle the USB hub problem, or what to do about it, other than spending yet more money on another hub, just to see if it works. I suppose I could take the computer AND the drives to Best Buy, but only God knows how much that would cost.

And as if that weren’t enough, early this morning I decided to start the laser disc to DVD backup. Huh-uh. The DVDR doesn’t “see” the input from the laser disc, even though there’s no reason it shouldn’t. I looked in the manuals. Everything was set up fine, and I did everything I was supposed to.

I was too damned frustrated after the hard drive and USB hub trouble to fight with it.

Below the fold (you’ve been warned).

I got my ELS data from NCES. I put the DVD in my drive and ran the installation program, and it set up a little query application. Fine so far. You run the application and get a list of data columns that mostly make sense (level of parental education). You go through the list, select the ones you want, then export the data.

The query application is fine. It’s when you open the exported file that everything gets bizarre.

In the query application, the column names make sense. But in the exported data, well, SCHID is okay for “School ID,” but EX3_4XN_6R just does not, in any way, map onto “math score.” Nor would any reasonable human being look at M4C_9X_3-2 and think, “reading score.” Sure, you can change them in Access or Excel or SPSS or SAS, but if you’re exporting, say, 15 columns of data, you’re in a world of hurt if you forget to write them down from the query application window before you export the data.

Where do they find these idiots who decide to represent “math score” with EX3_4XN_6R? My guess is that lousy database managers who got fired for incompetence in the private sector go to work for the Department of Education.

Is that cynical?

There are lots more problems, but if I get started, I’ll never stop . . .

What is it with technology that is deliberately annoying? Why do OS developers believe that I want the computer to do things on its own–without my telling it to? And even things that should operate on their own in the background, like anti-virus software, why does mine make me go through a little NEXT, NEXT, NEXT, FINISH game every day after it downloads an update and runs? Why does it have to ask me if I want it to clean or delete infected email? Why wouldn’t I? And if I didn’t want to, why would I be running anti-virus software at all? And somebody please explain why computers by default need to play stupid little ditties every time you open or close a window, or boot the machine, or do nearly anything at all. What the hell makes these software developers think I want to hear their stupid little tunes?

Why does the computer feel it necessary to ask me if I’m sure I want to do something every time I tell it to do something? If I weren’t sure, I wouldn’t have told the computer to delete the damned file.

I despise the user-friendly tech culture–and that is fundamentally why I despise Apple and the Mac idiot box. That’s where all this “point and click” “let me do everything for you!” BS started.

But it’s not just computers. Do they make telephones that don’t beep obnoxiously when there’s a message? Yes, I know there’s a message. I don’t particularly want to listen to it now because I’m busy, but if I don’t that damned beeping will drive me off the deep end.

This “user-friendly” crap drives me nuts.

Thanks to (who else?) Jonah Goldberg, I found this list of rules for robots, inspired by the Japanese rules for robots. A few of my favorites:

Rule #10 - Any robot who begins to behave illogically shall be immediately employed by the government.

Rule #85 - Robots are not allowed to appear on American Idol as contestants, because their ability to modulate their audio processors to sound like Bono gives them an unfair advantage.

Rule #89 - Robots may at no time play the bagpipes.

Rule #111 - Robots who become evil due to conflicted programming, ultra high doses of radiation, or warped artificial intelligence are considered automatically qualified to head programming at one of the major television networks in the U.S. or Britain.

HDW, while reporting the latest Microsoft screw-up with Outlook, says:

If you’re using the MS suite, and you’re thinking of upgrading… I have a word for you. Thunderbird. Download it, use it. A very good e-mail solution.

Before you download Thunderbird, read what I have to say about it. I refuse to use Outlook for reasons I needn’t go into here and when I bought my latest laptop, I downloaded and installed Thunderbird. All I can say is that if migrating email and filters weren’t such a buggy pain in the *ss, I would have stopped using Thunderbird a long time ago.

When I boot my laptop in the mornings, the first thing I want to do is read my email. So I fire up Thunderbird and download my email. So far, so good. I have filters to move certain messages to certain folders, and delete others (you know, like spam), and the next thing I do is click Tools, Run filters on folder.

That’s where the nightmare begins. Daily nightmare, not once in a while nightmare.

Typically, I’ve downloaded 250+ messages, and I can see the messages being moved. I may get to 200 messages, and then it just stops. Every day. I then have to exit Thunderbird, and rerun it so I can again run the filters on my inbox. The second time, I may get to 150 before it just stops, and I go through the whole exit, rerun, tools, run filters on folder process again.

On a good day, I have t