Using ext4 and 2.6.28 kernel on my new laptop

Hoolia! Now on my new laptop (Lenovo ThinkPad T61). I'm using Debian 5.0 (Lenny) with self-built shinning new 2.6.28 kernel and ext4. So far so good.

Next step is to install the latest Intel Graphics drive to benefit from GEM.

Don't use APT::Default-Release in Ubuntu unless you know what you are doing

Setting APT::Default-Release in Ubuntu blocks all future security fixes and updates.

This is related to all versions before Hardy (include). I haven't tested this on Intrepid so I'm not sure about those versions after Hardy.

According to apt_preferences manpage, the target release can be set on the apt-get command line or in the APT configuration file /etc/apt/apt.conf, and "APT::Default-Release "stable";" is given out as an example. This is a very common and popular practice used in Debian community to set the default release and using apt-pin, but doing this in Ubuntu leads to serious security impact with no obvious warning.

After setting APT::Default-Release to "hardy", which is the "Suite" name for main hardy source, no security fixes nor updates would be installed unless their priorities are also set explicitly in apt_preferences. This is because that in Ubuntu's world, security fixes are from "hardy-security" source and other updates are from "hardy-updates" source, which bear different "Suite" from the main source. Setting APT::Default-Release rises the priority of packages from main source to 990, but doesn't cover packages from hardy-security and hardy-updates, so the latter are ignored since their packages now has lower priority (priority 500 only) than those old ones in main source (990).

I set APT::Default-Release to "hardy" on Sep this year until I found this problem today. Removed that setting and I'm surprised to found that I can install 46 security fixes and updates accumulated. Which is pretty sad to me that got known I haven't got security fixes for more than 2 months.

This is a radical deviation from the Debian practice. In Debian all security fixes and updates bear the same "Suite" (etch or lenny) so setting APT::Default-Release to "etch" covers all security fixes and updates.

I think it's unlikely that Ubuntu changes the organization of it's source, so at least a fix to this problem is patching the apt_preferences manpage, alerting people not to use APT::Default-Release like they have used this in Debian and the reason and the following impacts.

I've opened a bug about this: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/295448

Ripping VCD in Linux

I've got some old VCD discs so I've spent some time on studying how to rip them for backing up.

There are two methods available:

1. To make a perfect ripping, use cdrdao read-cd:
   
   cdrdao read-cd --device ATA:1,1,0 --driver generic-mmc-raw --read-raw image.toc
   
   the --device is optional. By using this you got a TOC/BIN file pair that contains all the tracks from VCD which can be used to produce exact 1:1 copies. You can also generate a CUE/BIN pair by passing a filename with .cue extension. Although it is said that mplayer and totem can play the BIN file if proper decoding plugins are installed, I have never been able to do that, which is a big problem for me.
   
   Credit goes to manmath: http://ubuntuforums.org/archive/index.php/t-217997.html


2. For easy playing back, you can extract MPG files by using GNU VCDImager, which is included in many popular distros. Using the vcdxrip program:
   
   vcdxrip -C=/dev/cdrom
   
   By this you got all videos in usable MPG files.

Though you may be able to copy DAT files from VCD to your hard drive directly in Windows, this is not the case in Linux systems, you'll get I/O errors. The reason is that the file system of VCD disc is not valid ISO-9660 and never meant to support direct copying video data as plain files. Windows must be doing some conversion behind the hood.

mplayer and ASS subtitles in UTF-16 with BOM

Tina got a piece of .mkv video with ASS subtitles, which was in UTF-16 and began with BOM (0xFF 0xFE), perhaps generated by using some programs from Windows kingdom. mplayer can't load ASS that begins with BOM and produces the following error message:

Matroska file format detected.
VIDEO:  [avc1]  864x480  24bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
SUB: Could not determine file format
Cannot load subtitles: file.ass

The remedy:

1. Convert the ASS file to UTF-8 (or whatever encoding you are comfortable with). I used iconv: iconv -f utf16 -t utf8 < file.ass > result.ass


2. Play it by using the "-utf8" switch of mplayer: mplayer -ass -utf8 file.mkv. If the ASS contains special characters (such as CJK chars), use a good font by "-font /usr/share/font/cjk.ttf"

Fix nVidia's Proprietary Driver's Poor 2D Performance on Linux

Call me a fool. I bought an XFX GeForce FX 5200 PCI for my workstation (which is actually a Dell SC440 tower server who has no AGP slot). And nVidia's poor proprietary driver is notorious slow of 2D on Linux. The CPU usage of the Xorg process easily goes up to around 70% on my Xeon Core 2 @ 1.86GHz even when scrolling the text in gnome-terminal, not to say reading web pages in Firefox, which slows down everything else. And I've been sticking to a years old pristine GNOME (2.14 shipped with Debian 4.0 Etch) with nothing eye-candy here.

Finally I've found a workaround: http://www.nvnews.net/vbulletin/showpost.php?p=1704073&postcount=1: Update (2008-07-18) worked for me. In courtesy of xsolarwindx.

The improvement: now with this little fix the time used to build an Linux kernel on my workstation is only half as before just because the little baby likes writing its progress onto the terminal and old nVidia hates this so it made the displaying of text even more time-consuming than compiling the multi-million-line kernel itself!

I've been running nVidia's driver 173.14.12 x86_64 on a stock Debian 4.0 Etchnhalf kernel (2.6.24-etchnhalf.1-amd64).

I'll never buy an nVidia unless they released full GPL drivers.

Install ATI's fglrx Catalyst 8.8 Driver (v 8.522) on Linux 2.6.27

The Linux kernel 2.6.27 has changed an API (smp_call_function()) that breaks compilation of ATI's Catalyst 8.8 fglrx driver. You get error like this:

/usr/src/fglrx-8.522/firegl_public.c:3080:50: error: macro "smp_call_function" passed 4 arguments, but takes just 3
/usr/src/fglrx-8.522/firegl_public.c: In function ‘__ke_flush_cache’:
/usr/src/fglrx-8.522/firegl_public.c:3080: error: ‘smp_call_function’ undeclared (first use in this function)
/usr/src/fglrx-8.522/firegl_public.c:3080: error: (Each undeclared identifier is reported only once
/usr/src/fglrx-8.522/firegl_public.c:3080: error: for each function it appears in.)
...

Loïc Minier has made a patch for Catalyst 8.7, and I've adapted it to Catalyst 8.8:

From 42a6390f599294c60a4960b0a6cd4e399b1a81ee Mon Sep 17 00:00:00 2001
From: Yan Li 
Date: Sun, 14 Sep 2008 17:45:48 +0800
Subject: [PATCH] patch for building on 2.6.27

ref: http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43
---
 firegl_public.c |   15 +++++++++++----
 1 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/firegl_public.c b/firegl_public.c
index bdc94b2..d85a567 100644
--- a/firegl_public.c
+++ b/firegl_public.c
@@ -202,6 +202,13 @@
 #define preempt_enable()
 #endif
 
+/* Since 2.6.27 smp_call_function doesn't have a nonatomic/retry argument */
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,27)
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, wait)
+#else
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, retry, wait)
+#endif
+
 // ============================================================
 /* globals */
 
@@ -3077,7 +3084,7 @@ int ATI_API_CALL __ke_flush_cache(void)
 {
 #ifdef __SMP__
     /* write back invalidate all other CPUs (exported by kernel) */
- if (smp_call_function(deferred_flush, NULL, 1, 0) != 0)
+ if (SMP_CALL_FUNCTION(deferred_flush, NULL, 1, 0) != 0)
   panic("timed out waiting for the other CPUs!\n");
 
     /* invalidate this CPU */
@@ -4796,7 +4803,7 @@ struct _agp_memory* ATI_API_CALL __ke_agp_allocate_memory_phys_list(
 void ATI_API_CALL KCL_CallFuncOnOtherCpus(firegl_void_routine_t func_to_call) 
 {
 #ifdef CONFIG_SMP
- smp_call_function( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
+ SMP_CALL_FUNCTION( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
 #endif
 }
 
@@ -4910,7 +4917,7 @@ static int ATI_API_CALL KCL_enable_pat(unsigned int save_orig_pat)
    }
 
 #ifdef CONFIG_SMP
-   if (smp_call_function(KCL_setup_pat, NULL, 0, 1) != 0)
+   if (SMP_CALL_FUNCTION(KCL_setup_pat, NULL, 0, 1) != 0)
        return 0;
 #endif
    KCL_setup_pat(NULL);
@@ -4929,7 +4936,7 @@ static void ATI_API_CALL KCL_disable_pat(void)
     }
 
 #ifdef CONFIG_SMP
-    if (smp_call_function(KCL_restore_pat, NULL, 0, 1) != 0)
+    if (SMP_CALL_FUNCTION(KCL_restore_pat, NULL, 0, 1) != 0)
        return;
 #endif
     KCL_restore_pat(NULL);
-- 
1.5.4.3

I suggest use the --buildpkg function of the ATI driver to build packages for your system. Those packages use dkms to build the kernel module needed. (For Debian/Ubuntu flavor Linux, I suggest use the way as described in https://help.ubuntu.com/community/BinaryDriverHowto/ATI.) Apply the above patch after installed the fglrx-kernel-source.*.deb to /var/lib/dkms/fglrx/8.522/source, then run "sudo dkms build -m fglrx -v 8.522" to build the kernel module.

Reference:
http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43

Generating Gantt from Bugzilla Tasks by Using TaskJuggler

We've been using Bugzilla in various projects to track tasks' status and it turned out to be an excellent tool for collaboration and project management. Gantt is useful in various ways for project status checking and planning. Today I've spent some time to set up automatically generation of Gantt from Bugzilla tasks by using a combination of various tools.

• TaskJuggler: excellent free software on task scheduling and report generation, with beautiful GUI and powerful command-line interface.


• Bugzilla:TaskJugglerIntegration: export Bugzilla tasks to TaskJuggler project file.

The script bztotj.py from Bugzilla:TaskJugglerIntegration is easy to understand, and by changing the SQL in it you can export tasks from Bugzilla by arbitrary filters.

TaskJuggler's task is written in plain text so it's good for change management. It also features a nice GUI editor. You may need some time to learn to write project files but once you get it you can write versatile project files and, by writing scripts to manipulate them, implement complex project scheduling schema.

If you want to generate Gantt from TaskJuggler tasks without human interference, you can use this tool: http://blog.gustavobarbieri.com.br/2007/04/19/generating-gantt-graphs-from-taskjuggler-projects-without-using-its-ui/

My Home Storage Reached 2 TB

Today I bought and installed a 750GB hard drive to my backup server. This made my total storage at home reached 2.11 TB. The usable storage is more than 500 GB (others are used for backup).

OpenStreetMap is an Interesting Project

www.openstreetmap.org is an really interesting project, I've been contributing to it for some time, mainly covers Beijing urban and rural area.

My HD High Load/unload Cycle Problem Fix

I've just applied http://en.opensuse.org/Disk_Power_Management to fix the high load/unload cycle problem of my IBM ThinkPad T43p laptop. My Load_Cycle_Count grows at around 30 per hour, which is not very high, but theoratically limit my HD's life to around 4 years.

PS. though I dislike SuSE due to their pact with Microsoft, they are doing really well in some fields. The article mentioned above is well written, neat and clean, and the script is in good quality. I was told that openSUSE has already included patches to fix this problem, while Ubuntu's users are still struggling to either fix it by themselves or talking in the forums, trying to understand what to do.

IBM/Lenove ThinkPad Battery Maintenance in Linux

I felt the deteriorate of the lifetime of the battery on my IBM ThinkPad T43p. This article has a good discussion on best practices. I also installed the tm_smapi driver and set charge threshold for my laptop.

Save battery, no waste. :)

China Earthquake Relief, Sichuan, 2008

The 2008 Sichuan earthquake occurred at 14:28:04.1 CST (06:28:04.1 UTC) on 12 May 2008, with its epicenter in Wenchuan County, Sichuan province of the People's Republic of China. It had a magnitude of 7.8 Ms according to the State Seismological Bureau of China and 7.9 Mw according to the United States Geological Survey.

With more than 60,000 recently confirmed dead so far, it was the deadliest and strongest earthquake to hit China since the 1976 Tangshan earthquake, which killed approximately 250,000 people.

Please consider donation to China Red Cross! Thank you!

Qt Problem of Google Earth 4.3 Beta for Linux

On some systems (as Ubuntu Hardy 8.04) the Google Earth 4.3 Beta doesn't work very well, the fonts are ugly and tiny. You may try this workaround:
http://groups.google.fr/group/earth-linux/browse_thread/thread/e10a5337b0b47ba1#

On other systems (as CentOS 5.1) it may work very well out-of-box.

HOWTO: vsftpd's Virtual User Use Local Privileges

If you are using vsftpd with virtual users, as described in official example, and are suffering problems like this:

226 Transfer done (but failed to open directory).

(you may see above text in client or server log), you may consider using "virtual_use_local_privs".

Per study of cpdohert, if you want vsftpd's virtual user follow local privileges, you have to use:

virtual_use_local_privs=YES

FTP Ascii Mode in KDE (kioslave)

Write a ~/.kde/share/config/kio_ftprc, with

[hostname]
textmode=true

Or use [] instead of [hostname] to make it used everywhere.

Warning: this will likely mess up any transfers of binary files.

In courtesy of David Faure.

Steps to Upgrade MoinMoin Wiki from 1.5.* to 1.6.*

I've been using the MoinMoin Wiki from Debian Testing for some time and this morning I found it was upgraded from 1.5.8 to 1.6.2, and unsurprisingly, my Wiki installation stopped working, very obviously some data migration may be necessary.

After wasted some time, I found a good article: Thomas Waldmann's AnotherWayToMigrateFrom1.5To1.6. I couldn't find a good official migration guide like this, which told you that you must run cache cleaning and data migration.

Hope this is useful to other MoinMoin users.

Use pdftk to Update PDF File's Meta Data

A good tool, pdftk, can be used to manipulate PDF files, changing meta-data, page-juggling, etc., you name it! And it's GPLed.

Easy, peasy, japanesey.

Error of Running VMware Server 1.0.5 on Debian 4.0 Etch

I installed VMware Server 1.0.5 on Debian 4.0 Etch (i686) today and couldn't start vmware-console due to this error:

(vmware:20921): libgnomevfs-WARNING **: Cannot load module `/usr/lib/gnome-vfs-2.0/modules/libfile.so' (/usr/lib/vmware/lib/libgcc_s.so.1/libgcc_s.so.1: version `GCC_4.2.0' not found (required by /usr/lib/libstdc++.so.6))

To fix this problem, start VMware by running:

VMWARE_USE_SHIPPED_GTK=yes vmware

For more information on this please click the title, which leads to a post on VMware community website.

RPM Spec: conditional build based on host distro

When writing spec of RPM, often you need to write some special instructions that are needed only on some distros or versions, but not all. You can use this:

# Do something for FC4 and beyond.
%if 0%{?fedora} >= 4
    # ...
%endif

Tweak GNOME Auto Mount Options

Use gconf-editor to edit the key:

/system/storage/default_options/vfat/mount_options

The most useful one for me to add is "iocharset=utf8".

Ignoring .svn directories with grep

Have lived with Subversion for many years and finally I found this (link to outside my blog) simple little trick. :)

Wikipedia Unblocked (temp) in China

Now I can access Wikipedia in China without using a proxy, not all contents, only a large part. This is a big step forward.

As well as blogspot.com, wow!

May His Holiness let this last.

Choose a Linux-friendly Bank in China

At last I thought I should spend some time on finding a Linux-friendly bank. By Linux-friendly I meant that the e-Bank services should be able to achieved from a Linux machine running native browsers such as Firefox. I've been using China Merchant Bank for nearly 10 years and although generally good, their eBank services are designed around IE and ActiveX. What's more annoying is that their bank software could not be run within a virtual machine, alleged for the sake of better security, which shut all doors to run it on Linux.

I used eBank to manage loads of things such as utility payment, donations, credit card payment and investment management. So it's really a waste of time to reboot to MS Windows^TM to do them.

So finally I chose Shanghai Pudong Development Bank. It's eBank services are great with Firefox! Till now I'm happy with it. I've also applied for the credit card and hope to move all my credit card subscriptions to it gradually.

Now the only gray area for me is that my employer doesn't support to pay into SPDB. I'll keep complaining to my beloved HR manager. :)

N.B. HSBC is doing very well with Linux too. Who (I guess) has been running IBM's AIX and System z for a long time and is generally browser-independent.

A Talk with Andrew Morton

On Feb 20th, 2008, Tina and I attended the Eating the Linux Kernel party:
"Thanks to the Linux Foundation and China OSS Promotion Union we're
organizing a diner with all the great speakers of the Linux Developer
Symposium. This is going to be the one opportunity to share food and
thoughts with people like Andrew Morton, Jonathan Corbet, Dave Neary,
Jeff Waugh and many many others. In total probably 20 specialists from
the kernel development, Linux embedded or Gnome development
exclusively for Beijing LUG members..."

Fortunately I had a chance to have a talk to Andrew Morton who seated
besides me:

(This is not a verbal transcript of the talk, nor has Andrew Morton
endorsed it and he has no responsibility for whatever listed here.
This is just a fast record of a gossip and please don't take it
seriously.)

YL: Hi Andrew, you look tired, are you working busy on 2.6.25?

AM: Yes, it takes me lot of time.

YL: When will you leave Beijing?

AM: Friday. Take a tour tomorrow (Thursday) in Beijing.

YL: Can I ask you some questions about the kernel? Why Linux doesn't
maintain a stable ABI. Currently each time the kernel updates, all
3rd party modules will have to be recompiled. And this is really
annoying for many users.

AM: What 3rd party modules?

YL: For example, the one of Madwifi, and fglrx, the proprietary driver
for ATI cards.

AM: Are they GPL?

YL: Oh, no, part of it is a non-GPL blob. So I see the reason why the
kernel doesn't support them well is that they are not free software.
So can I say that the rumor is true that you intentionally break the
ABI to make using 3rd party modules difficult?

AM: No, we don't break them _intentionally_. We just don't care about
it. If we were to support a stable ABI, it will make the development
and maintenance of the kernel much more difficult. We won't support
non-GPL code by compromising free software.

YL: But if you made the installation of some proprietary drivers
easier, more users would come to use Linux. And if you had more
users, you have more developers. And this will promote the development
of the kernel.

AM: Yeah, it's true. But we just can't support proprietary software by
ways of compromising free software.

YL: Do you have a plan, or a long-term goal of the kernel development?

AM: No, we have no plan. And this is just the point of kernel
development (and maybe other free software project). The users decide
where the kernel goes. If you need a function, just as he (a man sit
besides AM) just said, if you need to support Chinese display in
Linux, just implement it and send me the patch, and if it's good
enough, we'll include it in the kernel.

YL: If the kernel itself has no concrete plan for what functions
should be supported and what not, and accepts everything good enough
from the users, won't the kernel became bloated quickly?

AM: No it won't. Everything is configurable. If you don't need it,
disable it.

YL: But the kernel is already very large, the binary is 20Mb, and the
source codes are somewhat more than 300Mb.

AM: I think it's not a problem. The core part of the kernel takes only
1Mb memory at run-time. And I see 300Mb is not a problem for today's
hard drive.

YL: The kernel is so large, and has too many things in it. I see
another problem with this approach. Say if you found a bug in a driver
for an ancient piece of hardware and you updated the kernel. Then all
users have to update just to pull in something they'll never used.

AM: Oh, I've to say sorry for that. You can say it's a problem.

YL: Can we just split this large kernel into several loosely coupled
projects? Say, one project for a kernel core, one project for file
systems, one project for the network layer and some other projects for
drivers.

AM: That maybe to difficult for maintenance.

YL: So do you thing micro-kernel would solve this problem, or are you
a fan of monolithic kernel?

AM: Monolithic kernel is much more efficient and easy to develop.

YL: Another question, does the kernel team have a plan to add
automatic test to cover some key functions?

AM: No, auto test doesn't work as we tried before. I've been
following the LTP project for more than 2 years, and only 2 bugs were
found by those automatic tests. Most of the kernel bugs are only
reproducible on some special pieces of hardware and you just have no
way to use automatic test to cover them.

For the core part, things like scheduler, file system, etc., is used
by everyone so they are thorough tested by everyone, and for the other
parts, like various drivers for special hardware, only those have that
hardware can test it. So auto test has few use for this.

YL: Do you think the kernel can be better maintained or developed by a
team?

AM: No. It won't get better if you put a lot people in a building.
The communication would take too much time. I see it's efficient
enough to communication by using mailing list. I get bored soon
during a long meeting so I just leave and ask them to send mails and
code snips to me and we can have better understanding on technical
problems.

YL: Do you think attending symposiums, lectures and parties like this
is a waste of time? Do you think you can better use the time for
hacking?

AM: Oh, I don't like this before I go, it's not good to travel for
long time on plane. But when the event was finished I thought "oh,
it's not bad." These are things for promotion. For example, before I
went to Japan some years ago, there's few patch from Japanese
developers. And after some lectures there, the patches from Japan now
skyrocketed to be around 20% among all patches. So it has a snowball
effect and that's why I attended these events. I hope I can get more
patch from Chinese developers.

YL: Do you have a laptop?

AM: Yes, it's a ThinkPad.

YL: Oh, I'm using a ThinkPad too. But I have to use proprietary blobs
to support the wireless as well as ATI video card. Though I don't use
3D acceleration much, I do need the 2D acceleration for daily work.
The open source driver is slow on 2D performance.

AM: Yeah, ThinkPad has poor 2D performance but it's bearable. I just
use the default open source driver. For wireless my laptop is running
ipw3945 driver and that's the only binary blob on my laptop.

YL: So does your suspend and hibernation work well?

AM: No, they don't. (shrug)

YL: What's your main work time and for how long you sleep a day?

AM: Huh! I work most of the time, and sleep from around 3am to, say,
11am.

YL: I remember you wrote an article said that the 2.6 kernel is slowly
getting buggier.

AM: Yes. People should spend more time on fixing bugs.

YL: What's your personal plan for the following years?

AM: I may quit in about 1 or 2 years and then I won't spend much time
on maintenance. I'll continue to work on the kernel, perhaps focusing
on bug fixing.

YL: So how about the mm tree?

AM: No need to worry, some other people will set their own branches to
take the position of it.

YL: Have Google asked you to do something special for them?

AM: No, they just pay me to do whatever I want to do.

(Don't) Use "Subpixel Smoothing" for Fonts in GNOME

I proposed that "Subpixel smoothing" was slow in my testing. But that result was machine-dependent. Some drivers were doing better than others. Personally I think Intel's video cards are better than that from nVidia (ATI's seems not bad either). I have to admit for some configurations subpixel rendering products better result. So it's totally up to you to decide whether to use it or not (I found myself enabled it on all my machines for now, even on a slow machine with nVidia card).

Old post follows:

It's extremely slow, while the gain in font appearance is little. This setting has contributed a lot to my bad impression about GNOME's sluggish display. Now I'm happy with "Best shapes."

In gnome-terminal, Subpixel smoothing is 3 times as slow as "Best shapes" rendering, on my laptop.