The x86-32 Fedora kernel comes in two flavours. ‘normal’ and ‘PAE’. (There are -debug flavours of both of these two, but ignore those for the purpose of this blog post). The common wisdom surrounding ‘which kernel do I use’ has for the longest time been “How much RAM do you have?”

The kernel offers several configuration options to satisy this question. If you have less than 1G of ram, you don’t need anything special. After 1G, you need to enable CONFIG_HIGHMEM4G. (In actuality, it’s something like 960MB). As the name suggests, this allows you to run kernels that can see up to 4G of RAM. It does this by mapping in/out ‘highmem’ pages (those after the 1G mark) as they are needed. The ‘normal’ kernel for Fedora is HIGHMEM4G.

This sounds like all you’d ever need. Who has more than 4G of RAM ? Even today, typically only servers. However, that mapping/unmapping comes with some overhead, and if you’re using a lot of highmem, you may be better off with the third choice..

Enter PAE (Page Address Extensions). This option is enabled by the kernel config option CONFIG_HIGHMEM64G. Its name tellingly indicates it can support up to 64G of RAM in theory. I say in theory, because there are still a number of really nasty cases that need to be taken care of (Like: the pagetables need to fit in non-highmem memory, and by filling up lowmem with pagetables, you don’t leave much for other purposes). After a point, it just makes more sense to go to x86-64. PAE also gives us the ability to do hardware NX due to it having a wider PTE (page table entry) format.

But there’s another case where PAE is desirable which seems to catch out some people.
If you have exactly 4G of RAM, you may not be able to address it all, even with a CONFIG_HIGHMEM4G kernel.

PCI devices need some physical address space. How much, depends on what PCI devices are present, and how they are configured. This address space needs to live in the lower 4G of memory. So the BIOS carves out a hole in the memory map and says “PCI lives here”. If you had 4G of RAM, you may find the memory map now looks like..

3G RAM : 1G PCI ‘hole’ : 1G RAM

See the problem? To address that top 1G of RAM, we need to use addresses past 4G. And to use those, we need PAE.

This past weekend I’ve been thinking of reviving an idea that has come up countless times. Producing RPM builds of the rawhide kernel for our already released Fedoras. The reason for not doing them so far has come down to bandwidth. (in terms of build system throughput, disk space, mirroring, and people bandwidth).

What I’m toying with doing is some devel kernels for Fedora 11 that are built outside of the Fedora build system. The Fedora kernel team now has enough build bandwidth for x86-[64] that we can actually get builds for those architectures done faster than koji.

Disk space – I’m thinking of just keeping the last 2-3 builds available.

Mirroring – Instead of having these be part of Fedora proper, I think an external repo on something like fedorapeople.org will suffice.

Which just leaves people bandwidth. For the most part the work is going to be just regularly syncing the devel/ branch with a CVS branch of F-11/ For some of this work, some scripting could be done to alleviate some of the pain. Also the frequency at which we push out these builds will determine the pain point. Perhaps every -git isn’t particularly valuable anyway. One build every handful of -git’s should be sufficient for bisecting.

There does remain one additional barrier. Occasionally we introduce something in rawhide builds which just won’t work on F11. For example, the kernel modesetting patches are tied closely to Xorg packages. Sometimes upstream changes require changes in mkinitrd or udev or some other ‘plumbing’. Some of these are regressions, and hopefully by identifying them sooner we can get them reverted/fixed upstream. Sometimes however, things get deprecated, and we need to change these packages. I’m not sure how to cope with this yet in a devel-for-F11 scenario.

One other thing that might be fun to throw into this would be the generation of -vanilla packages. The only reasons we don’t do these as part of the regular kernel builds is the various bandwidth concerns above. The specfile copes with spitting out RPMs with very little work needed. Josh Boyer has been occasionally doing these builds, though there hasn’t been a huge uptake. It’s unclear if this is due to lack of interest, or just a lack of publicity.

Another question to be answered is whether we go the route of enabling debugging in all builds as we do in rawhide, or do separate -debug builds. I’m leaning towards the latter.

I’m not committing anything to this for sure just yet, but it’s something I’ve been giving quite a bit of thought. There are still a bunch of unanswered questions.

I’m not much a book reader. I often tend to start books, and never finish them. Occasionally I’ll pick them back up with the aim of finishing them, but end up starting over, and then abandoning them again at usually the same point. It’s actually been quite a while since I found a book that I couldn’t stop reading until I finished it. Recently, I found one such book: Malcolm Gladwell’s “The tipping point”.

The book explains why some ideas ‘tip’ or become really successful, while others never get off the ground. A lot of the book explained psychological reasons for why people behave the way we do in certain scenarios. From well known theories such as group think, to defining types of people, and the roles they play in the dissemination of ideas. I found Gladwells categorisation of people into connectors/mavens/salesmen intriguing, and spent a while thinking of various people I know and trying to categorise them accordingly. (It’s possible for someone to be in more than one category).

There’s also a lot of anecdotes in there to back up most of his points, coming from some wide and varied scenarios (The studies done by the Sesame Street researchers in order to create the perfect ‘sticky’ educational TV show for example). In some cases they do drag on a bit. In some cases there are multiple examples where one would have sufficed, but overall, it wasn’t tedious whilst hammering home the point.

My only gripe with the book was that there was no mention of the opposite scenario. There’s lots of examples of successful ideas that ‘tipped’, and some ideas that didn’t, but there are no examples or dissection of “Why don’t bad ideas die?”. Some ideas no matter how many times they get shot down seem to bubble back up to the surface every so often. I have some of my own theories why these zombie ideas never go away, but I would have liked to have read the authors take on it..

Anyway, rambling… Good book. Recommended. Wikipedia also has a pretty decent summary of all the points covered in the book.

With the release of Fedora 11 recently, we are now back in the position of looking onwards to the next release. F11 released with a 2.6.29 kernel, and we’re already looking at doing a .30 rebase for it soon. (I was hesitant to type that, because the last time I blogged about doing a rebase, we hit some troubles and ended up skipping the 2.6.28 release for F10). While F11 was stabilising before release, devel/ had continued on, being rebased up to 2.6.30. Yesterday Kyle committed the the first rebase to get us back up to Linus’ tree of the day.

So we’re looking at 2.6.31 for F12. (With various conferences coming up over the next few months, it seems infeasible that .32 will land in time for F12′s release).

Other planned changes? We’ve talked about dropping the exec shield patch that we’ve been carrying since Fedora Core 1.
It’s a pain to have to keep carrying it, and rebasing it, and occasionally fixing it, just to add a poor emulation of a feature that has been in all CPUs for the last five years. The decision isn’t final yet, but it’s something that’s being considered.

Some of the other patches we’ve been carrying (like modesetting drm) are now starting to find their way upstream too, which is obviously a good thing. The only really big thing we’re still carrying that struggles to get upstream is utrace.

A while ago, the following innocuous printk was added to the p4-clockmod driver ..

printk(KERN_WARNING PFX “Unknown p4-clockmod-capable CPU. Please send an e-mail to <cpufreq at vger.kernel.org>\n”);

Which would get printed obviously, if it found a CPU on which it thought it might be able to work on, but would require checking by developers beforehand. Sounds like a great idea on the surface, but this one line turned out to be incredibly annoying.

The problem is that people run old kernels. (Be they self-built old kernels, irregularly updated distros, or even enterprise distros). So even though we removed that printk from the driver in October 2008, we still see the occasional report.

There was a similar situation with ACPI. It asked people to send dmidecode output to a mailing list, and people did so for many months after it stopped being interesting. The ACPI code still has 5 separate places where it asks users to send something to the mailing list, but it only does so in rare cases where we know there’s a BIOS bug. But it still has the effect that if they’re still there in n months time, they’ll live on in RHEL, SLES, Ubuntu LTS etc, even if upstream has stopped caring, or has worked around these bugs in some way.

A better solution to this situation would probably be to replace the ‘send email’ messages with a ‘visit this website http:// ..’ printk. After it stops being interesting, the page could get updated to provide information such as “this is fixed in 2.6.40, you should ask your kernel vendor to remove the printk and apply this patch..”

This announcement is interesting. I actually wondered when olpc was first announced what the appeal was of the geode, and figured it came down to “we get them ridiculously cheap if we buy them in volume”. The geode is a pretty dire processor when it comes to performance. The C7 isn’t exactly much to write home about either, but comparing them both is like night and day.

As well as the noticable performance boost, olpc 1.5 will get for free: SSE, SSE2, a hardware RNG, hardware accelerated encryption, hardware NX (though this would require PAE to be enabled, which may be considered unncessary overhead on an XO), and frequency/voltage scaling.

If they had done this on day 1, I may have been more interested in using one. My one experience with a 1.0 XO was possibly the most miserable computing experience I recall.

As netflix season draws to a close (I put my account on hold when the weather starts getting better), I’ve been trying to get through as many old horror movies that I’ve been meaning to see for a long time as possible. Like 1980′s kind of old.
For the most part, I’ve been kind of disappointed. I’d built up “classics” like
Videodrome, and Eraserhead in my head to be something awesome that I really should see some time.
Perhaps they would have had more effect on me had I seen them several decades ago, but watching them now, I found them meandering, and well, dull. I couldn’t wait for Eraserhead to end. It became an endurance test to see if I could make it through the movie.

By complete contrast, Tetsuo: The Iron Man is possibly one of the greatest movie viewing experiences I’ve had. Wikipedia has a synopsis of the whole film. Even though I had read the synopsis beforehand, it wasn’t really a spoiler. I spent the whole movie asking myself “wtf is going on?”. It sounds awful, but somehow the awfulness wraps around and becomes awesome. There’s lots of really cool stop motion animation, and some of the gory bits are impressive given the miniscule budget that Shinya Tsukamoto had to work with.
The soundtrack by Der Eisenrost was impressive too given this was made in 1988. Listening to it, it’s pretty obvious where lots of early/mid 1990s industrial bands got inspiration.

There’s also a sequel to Tetsuo, which I’ve not seen and netflix don’t have it. So I’m going to have to pick up a copy from Amazon.

Fedora 11 has a shiny new feature, where if text is trying to be displayed in a language for which you have no font, a dialog is helpfully popped up along the lines of “gnome-terminal needs a hindi font, would you like to install one?”

Sounds really useful, but it gets irritating really quickly when you read email in a terminal and you get a lot of spam, so every few emails asks for a new font.

What really amused me though, was seeing someones /quit msg in irssi today.

“Leaving(पुन्हा भेटू)”

I’ve no idea what that says, but it caused a popup. Great, a new way to irritate the hell out of someone on irc!

update:
Apparently, this..

gconftool-2 –type bool -s /apps/gnome-packagekit/enable_font_helper false

will disable this feature.

w.t.f..

“This program automatically looks at what Wi-Fi card you have, it disables any wireless driver currently installed, fetches the correct Windows driver from the Internet and installs it with NDISwrapper.”

or, you could just use Windows.

After seeing a hack attempt on bugzilla.kernel.org’s password file, I got wondering “what if that had succeeded”. After looking at the stored password database in firefox, it turned out that I had been quite lazy and had reused my password for that site on many others. Spent a whole afternoon visiting websites to create unique passwords for every log in.

As my memory isn’t so fantastic, this means I now keep my passwords written down. The downside of this being a single point of failure (If someone gets this file, they have everything — though they’d need my gpg key & password to read it), is outweighed by the fact that as I don’t have to remember them, choosing longer passwords with more weird characters is possible. Though amusingly, quite a few websites don’t like passwords like 0N}nn:/6hai\h5eM, and complain that they aren’t secure enough.