My words on free/open source software

Tuesday, October 20, 2009

Understanding how fontconfig scan conf files

I've been fixing fonts issues in Moblin for some time, and I've to admit that GNOME fontconfig is really hard to understand.

The rules in /etc/fonts/conf.d/ are scanned in alphanumeric order. After carefully read the fontconfig user's manual, I thought that if they were processed in alphanumeric order, the later conf should overwrite the earlier conf. WRONG! But rules scanned first take precedence over rules later. Let's see how they are applied to the Pattern.

For example: if you run `fc-match sans`, the Pattern here is family="sans".

Let's say we've a 01-rulefile:
<alias>
<family>sans</family>
<prefer><family>DejaVu<family></prefer>
</alias>



After processing it the Pattern becomes:

family="DejaVu", "sans"


Then a 02-rulefile:
<alias>
<family>sans</family>
<prefer><family>Droid<family></prefer>
</alias>



After it the Pattern becomes:

family="DejaVu", "Droid", "sans"


Noted that the "Droid" from 02-rulefile is prepended to "sans", not
the whole Pattern. So 01-rulefile takes precedence over 02-rulefile.

Tuesday, October 13, 2009

Zero Config! Huawei EC1260 / China Telecom CDMA2000 EVDO 3G USB Dongle on Debian Sid

I bought a Huawei EC1260 USB modem to access the CDMA2000 EVDO mobile broadband provided by China Telecom. To my big surprise, the USB modem works out-of-box on my Debian Sid. Plug the USB modem and click the CDMA item in my NetworkManager and Bingo! I'm online! This is really zero config!

Thank you, great developers of NetworkManager and Debian for making this great progress.

Sunday, March 15, 2009

Beijing Tax Declaration Website Supports Firefox, Great!

I'm glad to finish my tax declaration online by using Firefox running on Linux. Every function of Beijing Taxation Bureau's website works well with Firefox.

Though there are spaces for them to improve, such as adding HTTPS support.

Compiling GIMP 2.6.5 on CentOS 5

GIMP 2.6.5 is the fantastic tool for photographers that live in the free world. But sadly it has no official RPM built for CentOS/RHEL users. I have spent 4 hours and compiled GIMP 2.6.5 on CentOS 5 for Tina by using the official tarball.

I planned to use the GIMP SRPM from Fedora Rawhide (which was 2.6.5) but after Fedora 10 the Rawhide had moved to a new format of RPM and I have no way to install nor unpack those new SRPMs to get the code (neither on CentOS 5 or Debian Unstable). Therefore I had to fall back to use official tarball and install it to /opt. By this I lost the benefit of package management. So I have no RPM here to share with you but only a rough record of the steps:

  1. download, build and install babl 0.0.22

  2. download, build and install gegl 0.0.22

  3. install intltool RPM from Fedora 10

  4. download gettext SRPM (gettext-0.17-8.fc10.src.rpm) from Fedora 10 and rpmbuild it (with Java disabled). the GCJ shipped with CentOS 5 couldn't compile the Java code within gettext so they had to be disabled by the patch following.

  5. build GIMP 2.6.5


The patch to disable Java of gettext:

--- /usr/src/redhat/SPECS/gettext.spec 2008-08-29 09:07:58.000000000 +0800
+++ gettext-for-el5.spec 2009-03-15 11:06:11.000000000 +0800
@@ -1,5 +1,5 @@
# libintl.jar requires gcj >= 4.3 to build
-%define buildjar 1
+%define buildjar 0

Summary: GNU libraries and utilities for producing multi-lingual messages
Name: gettext
@@ -86,12 +86,12 @@
export JAR=fastjar
%endif
%configure --without-included-gettext --enable-nls --disable-static \
- --enable-shared --with-pic-=yes --disable-csharp --enable-java \
+ --enable-shared --with-pic-=yes --disable-csharp --disable-java \
--disable-rpatha
make %{?_smp_mflags} GCJFLAGS="-findirect-dispatch"

# For generating gettext-tools/src/gnu.gettext.*
-make -C gettext-tools/src gnu.gettext.DumpResource
+#make -C gettext-tools/src gnu.gettext.DumpResource


%install
@@ -215,7 +215,7 @@
%{_infodir}/gettext*
%{_mandir}/man1/*
%{_libdir}/%{name}
-%exclude %{_libdir}/%{name}/gnu.gettext.*
+#%exclude %{_libdir}/%{name}/gnu.gettext.*
%dir %{_datadir}/%{name}
%{_datadir}/%{name}/archive.tar.gz
%{_datadir}/%{name}/styles
@@ -244,8 +244,8 @@
%{_libdir}/preloadable_libintl.so
%{_mandir}/man3/*
%{_datadir}/%{name}/javaversion.class
-%doc gettext-runtime/intl-java/javadoc*
-%{_libdir}/%{name}/gnu.gettext.*
+#%doc gettext-runtime/intl-java/javadoc*
+#%{_libdir}/%{name}/gnu.gettext.*


%files libs

Wednesday, February 04, 2009

Another GPL violation case from top China IT firm? Hanwang this time.

Hanwang's eBook Reader products (汉王电纸书产品系列, http://www.hw99.com/product/productkind-13.htm) are very popular in China market. They features an ARM processor and are running Linux with cramfs, which can be easily discovered from the update firmware the company has released. Also they are shipping BusyBox within them.

By no means I can find the source code to their Linux kernel / BusyBox. So perhaps this is yet another GPL violation case from top China IT firm. Anyone bother a law suit?

Saturday, January 03, 2009

Safely remove an USB hard drive in Linux

I bought a new USB hard drive, Western Digital My Book® Essential Edition™ 1TB, and use it with Linux. Here I'm discussing about how to safely remove (then disconnect) it from a Linux system.

The problem: according to its user manual, you should disconnect the drive safely when it is not active, but in a Linux system how to do this is not very straightforward. You may notice that after you unmount it (whether through command-line or a desktop environment), the drive is still spinning and it's LED on. If you read it's user manual carefully you'll find that the manufacturer never said it's safe to disconnect it from your system in such condition.

The quick solution to this problem:

  1. send SYNC, then STOP command to the device, this can be done easily in Linux by unbinding the device

  2. suspend the USB port by echoing a "suspend" to the "/sys/bus/usb/devices/$DEVICE/power/level", where $DEVICE corresponding to the device of your USB device.

To put it simple, I wrote a script to do this for you. Download and try it. For developers, you can get it from the github repo.

If you saw an error message like this when running it:
bash: /sys/bus/usb/devices/5-8/power/level: No such file or directory

that means your kernel doesn't support USB suspend. You can just disconnect your device now, it's already safer enough to unplug it.

To those who is interested, discussion of technical detail following.

From the hardware aspect, the drive should only be removed when the following requirements are met:

  1. there's no pending I/O request and software cache flushed

  2. it's hardware cache flushed

  3. it's driver spins down (means "not spinning")

  4. (optional) the USB port is put into "suspend" mode


The user's manual read (page 6, section "Turning Off/Disconnecting the Device"): "(for Windows systems): Right-click the Safely Remove Hardware icon in your system tray and select Safely Remove Hardware. You may hear the drive power down before the Power LED turns off. The drive is now shut down properly, and you may disconnect the drive safely. (for Macintosh systems) Drag the My Book icon to the Trash icon for proper dismount. You may hear the drive power down as the Power LED flashes. When the Power LED is steady, you may press the Power button once or disconnect the drive’s power cord to turn off the drive safely." (there's a mistake, this device doesn't have a Power button) If you read carefully you can tell that Windows and Macintosh don't do the same things when told to remove an USB device. As the bold text as shown, in a Windows system it will be turned off while in a Macintosh system you hear the drive power down but the Power LED would finally become steady rather than off. Therefore we know that Windows does all 4 steps and Macintosh do the first 3 steps only when told to remove a USB device. (BTW, it's funny that you have to use your ear before disconnecting a device.)

(This is only the case for Windows XP. For Windows Vista seems only the first 3 steps are done thus the drive remained active after you removing it from the system.)


Some may argue that they always disconnect their drives directly after unmounting (step 1 only) and never experience any damage, but as an operating system developer and a serious user, I always follow the manufacturer's manual or specification whenever it make sense and possible because I know electronic devices are fragile, rigid and obtuse if you don't use them as the way they are designed. For a well-designed and robust piece of hardware, theoretically you can disconnect the device safely when the first two steps are done. The firmware on the device would notice the USB cable was unplugged and shut down the disk drive properly. If not, the drive would be stopped sharply as if you pulled it's power when it's running and this damages the hardware gradually. As an outsider of the manufacturer, we hardly know whether it's well-designed or not so the only safe principle is to follow it's manual, means we should not disconnect the device until at least all three steps are done.

At the Linux kernel part. To finish all the 4 steps, you have to unmount the device, unbind the device from the driver, then tell the USB core driver to put that device into suspend mode, as shown in the solution above. And you have to be running a kernel with CONFIG_USB_SUSPEND enabled.

CONFIG_USB_SUSPEND is not enabled by default in a vanilla kernel. I'm pushing it in this discussion.

Properly removing a device from a running Linux should be done by either the HAL or desktop manager. If you can, please push the developers you know working on any DM that missing this feature to implement this function properly.

Try my script, and write me if you found a problem. Welcome patches.

Saturday, December 27, 2008

Using ext4 and 2.6.28 kernel on my new laptop

Hoolia! Now on my new laptop (Lenovo ThinkPad T61). I'm using Debian 5.0 (Lenny) with self-built shinning new 2.6.28 kernel and ext4. So far so good.

Next step is to install the latest Intel Graphics drive to benefit from GEM.

Friday, November 07, 2008

Don't use APT::Default-Release in Ubuntu unless you know what you are doing

Setting APT::Default-Release in Ubuntu blocks all future security fixes and updates.

This is related to all versions before Hardy (include). I haven't tested this on Intrepid so I'm not sure about those versions after Hardy.

According to apt_preferences manpage, the target release can be set on the apt-get command line or in the APT configuration file /etc/apt/apt.conf, and "APT::Default-Release "stable";" is given out as an example. This is a very common and popular practice used in Debian community to set the default release and using apt-pin, but doing this in Ubuntu leads to serious security impact with no obvious warning.

After setting APT::Default-Release to "hardy", which is the "Suite" name for main hardy source, no security fixes nor updates would be installed unless their priorities are also set explicitly in apt_preferences. This is because that in Ubuntu's world, security fixes are from "hardy-security" source and other updates are from "hardy-updates" source, which bear different "Suite" from the main source. Setting APT::Default-Release rises the priority of packages from main source to 990, but doesn't cover packages from hardy-security and hardy-updates, so the latter are ignored since their packages now has lower priority (priority 500 only) than those old ones in main source (990).

I set APT::Default-Release to "hardy" on Sep this year until I found this problem today. Removed that setting and I'm surprised to found that I can install 46 security fixes and updates accumulated. Which is pretty sad to me that got known I haven't got security fixes for more than 2 months.

This is a radical deviation from the Debian practice. In Debian all security fixes and updates bear the same "Suite" (etch or lenny) so setting APT::Default-Release to "etch" covers all security fixes and updates.

I think it's unlikely that Ubuntu changes the organization of it's source, so at least a fix to this problem is patching the apt_preferences manpage, alerting people not to use APT::Default-Release like they have used this in Debian and the reason and the following impacts.

I've opened a bug about this: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/295448

Friday, October 31, 2008

Ripping VCD in Linux

I've got some old VCD discs so I've spent some time on studying how to rip them for backing up.

There are two methods available:
  1. To make a perfect ripping, use cdrdao read-cd:

    cdrdao read-cd --device ATA:1,1,0 --driver generic-mmc-raw --read-raw image.toc

    the --device is optional. By using this you got a TOC/BIN file pair that contains all the tracks from VCD which can be used to produce exact 1:1 copies. You can also generate a CUE/BIN pair by passing a filename with .cue extension. Although it is said that mplayer and totem can play the BIN file if proper decoding plugins are installed, I have never been able to do that, which is a big problem for me.

    Credit goes to manmath: http://ubuntuforums.org/archive/index.php/t-217997.html

  2. For easy playing back, you can extract MPG files by using GNU VCDImager, which is included in many popular distros. Using the vcdxrip program:

    vcdxrip -C=/dev/cdrom

    By this you got all videos in usable MPG files.

Though you may be able to copy DAT files from VCD to your hard drive directly in Windows, this is not the case in Linux systems, you'll get I/O errors. The reason is that the file system of VCD disc is not valid ISO-9660 and never meant to support direct copying video data as plain files. Windows must be doing some conversion behind the hood.

mplayer and ASS subtitles in UTF-16 with BOM

Tina got a piece of .mkv video with ASS subtitles, which was in UTF-16 and began with BOM (0xFF 0xFE), perhaps generated by using some programs from Windows kingdom. mplayer can't load ASS that begins with BOM and produces the following error message:


Matroska file format detected.
VIDEO: [avc1] 864x480 24bpp 23.976 fps 0.0 kbps ( 0.0 kbyte/s)
SUB: Could not determine file format
Cannot load subtitles: file.ass


The remedy:

  1. Convert the ASS file to UTF-8 (or whatever encoding you are comfortable with). I used iconv: iconv -f utf16 -t utf8 < file.ass > result.ass

  2. Play it by using the "-utf8" switch of mplayer: mplayer -ass -utf8 file.mkv. If the ASS contains special characters (such as CJK chars), use a good font by "-font /usr/share/font/cjk.ttf"

Saturday, September 20, 2008

Fix nVidia's Proprietary Driver's Poor 2D Performance on Linux

Call me a fool. I bought an XFX GeForce FX 5200 PCI for my workstation (which is actually a Dell SC440 tower server who has no AGP slot). And nVidia's poor proprietary driver is notorious slow of 2D on Linux. The CPU usage of the Xorg process easily goes up to around 70% on my Xeon Core 2 @ 1.86GHz even when scrolling the text in gnome-terminal, not to say reading web pages in Firefox, which slows down everything else. And I've been sticking to a years old pristine GNOME (2.14 shipped with Debian 4.0 Etch) with nothing eye-candy here.

Finally I've found a workaround: http://www.nvnews.net/vbulletin/showpost.php?p=1704073&postcount=1: Update (2008-07-18) worked for me. In courtesy of xsolarwindx.

The improvement: now with this little fix the time used to build an Linux kernel on my workstation is only half as before just because the little baby likes writing its progress onto the terminal and old nVidia hates this so it made the displaying of text even more time-consuming than compiling the multi-million-line kernel itself!

I've been running nVidia's driver 173.14.12 x86_64 on a stock Debian 4.0 Etchnhalf kernel (2.6.24-etchnhalf.1-amd64).

I'll never buy an nVidia unless they released full GPL drivers.

Sunday, September 14, 2008

Install ATI's fglrx Catalyst 8.8 Driver (v 8.522) on Linux 2.6.27

The Linux kernel 2.6.27 has changed an API (smp_call_function()) that breaks compilation of ATI's Catalyst 8.8 fglrx driver. You get error like this:


/usr/src/fglrx-8.522/firegl_public.c:3080:50: error: macro "smp_call_function" passed 4 arguments, but takes just 3
/usr/src/fglrx-8.522/firegl_public.c: In function ‘__ke_flush_cache’:
/usr/src/fglrx-8.522/firegl_public.c:3080: error: ‘smp_call_function’ undeclared (first use in this function)
/usr/src/fglrx-8.522/firegl_public.c:3080: error: (Each undeclared identifier is reported only once
/usr/src/fglrx-8.522/firegl_public.c:3080: error: for each function it appears in.)
...


Loïc Minier has made a patch for Catalyst 8.7, and I've adapted it to Catalyst 8.8:

From 42a6390f599294c60a4960b0a6cd4e399b1a81ee Mon Sep 17 00:00:00 2001
From: Yan Li
Date: Sun, 14 Sep 2008 17:45:48 +0800
Subject: [PATCH] patch for building on 2.6.27

ref: http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43
---
firegl_public.c | 15 +++++++++++----
1 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/firegl_public.c b/firegl_public.c
index bdc94b2..d85a567 100644
--- a/firegl_public.c
+++ b/firegl_public.c
@@ -202,6 +202,13 @@
#define preempt_enable()
#endif

+/* Since 2.6.27 smp_call_function doesn't have a nonatomic/retry argument */
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,27)
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, wait)
+#else
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, retry, wait)
+#endif
+
// ============================================================
/* globals */

@@ -3077,7 +3084,7 @@ int ATI_API_CALL __ke_flush_cache(void)
{
#ifdef __SMP__
/* write back invalidate all other CPUs (exported by kernel) */
- if (smp_call_function(deferred_flush, NULL, 1, 0) != 0)
+ if (SMP_CALL_FUNCTION(deferred_flush, NULL, 1, 0) != 0)
panic("timed out waiting for the other CPUs!\n");

/* invalidate this CPU */
@@ -4796,7 +4803,7 @@ struct _agp_memory* ATI_API_CALL __ke_agp_allocate_memory_phys_list(
void ATI_API_CALL KCL_CallFuncOnOtherCpus(firegl_void_routine_t func_to_call)
{
#ifdef CONFIG_SMP
- smp_call_function( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
+ SMP_CALL_FUNCTION( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
#endif
}

@@ -4910,7 +4917,7 @@ static int ATI_API_CALL KCL_enable_pat(unsigned int save_orig_pat)
}

#ifdef CONFIG_SMP
- if (smp_call_function(KCL_setup_pat, NULL, 0, 1) != 0)
+ if (SMP_CALL_FUNCTION(KCL_setup_pat, NULL, 0, 1) != 0)
return 0;
#endif
KCL_setup_pat(NULL);
@@ -4929,7 +4936,7 @@ static void ATI_API_CALL KCL_disable_pat(void)
}

#ifdef CONFIG_SMP
- if (smp_call_function(KCL_restore_pat, NULL, 0, 1) != 0)
+ if (SMP_CALL_FUNCTION(KCL_restore_pat, NULL, 0, 1) != 0)
return;
#endif
KCL_restore_pat(NULL);
--
1.5.4.3


I suggest use the --buildpkg function of the ATI driver to build packages for your system. Those packages use dkms to build the kernel module needed. (For Debian/Ubuntu flavor Linux, I suggest use the way as described in https://help.ubuntu.com/community/BinaryDriverHowto/ATI.) Apply the above patch after installed the fglrx-kernel-source.*.deb to /var/lib/dkms/fglrx/8.522/source, then run "sudo dkms build -m fglrx -v 8.522" to build the kernel module.

Reference:
http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43

Thursday, August 07, 2008

Generating Gantt from Bugzilla Tasks by Using TaskJuggler

We've been using Bugzilla in various projects to track tasks' status and it turned out to be an excellent tool for collaboration and project management. Gantt is useful in various ways for project status checking and planning. Today I've spent some time to set up automatically generation of Gantt from Bugzilla tasks by using a combination of various tools.


  • TaskJuggler: excellent free software on task scheduling and report generation, with beautiful GUI and powerful command-line interface.

  • Bugzilla:TaskJugglerIntegration: export Bugzilla tasks to TaskJuggler project file.



The script bztotj.py from Bugzilla:TaskJugglerIntegration is easy to understand, and by changing the SQL in it you can export tasks from Bugzilla by arbitrary filters.

TaskJuggler's task is written in plain text so it's good for change management. It also features a nice GUI editor. You may need some time to learn to write project files but once you get it you can write versatile project files and, by writing scripts to manipulate them, implement complex project scheduling schema.

If you want to generate Gantt from TaskJuggler tasks without human interference, you can use this tool: http://blog.gustavobarbieri.com.br/2007/04/19/generating-gantt-graphs-from-taskjuggler-projects-without-using-its-ui/

Tuesday, July 22, 2008

My Home Storage Reached 2 TB

Today I bought and installed a 750GB hard drive to my backup server. This made my total storage at home reached 2.11 TB. The usable storage is more than 500 GB (others are used for backup).

Thursday, July 03, 2008

OpenStreetMap is an Interesting Project

www.openstreetmap.org is an really interesting project, I've been contributing to it for some time, mainly covers Beijing urban and rural area.

Sunday, June 29, 2008

My HD High Load/unload Cycle Problem Fix

I've just applied http://en.opensuse.org/Disk_Power_Management to fix the high load/unload cycle problem of my IBM ThinkPad T43p laptop. My Load_Cycle_Count grows at around 30 per hour, which is not very high, but theoratically limit my HD's life to around 4 years.

PS. though I dislike SuSE due to their pact with Microsoft, they are doing really well in some fields. The article mentioned above is well written, neat and clean, and the script is in good quality. I was told that openSUSE has already included patches to fix this problem, while Ubuntu's users are still struggling to either fix it by themselves or talking in the forums, trying to understand what to do.

Monday, May 26, 2008

IBM/Lenove ThinkPad Battery Maintenance in Linux

I felt the deteriorate of the lifetime of the battery on my IBM ThinkPad T43p. This article has a good discussion on best practices. I also installed the tm_smapi driver and set charge threshold for my laptop.

Save battery, no waste. :)

Tuesday, May 13, 2008

China Earthquake Relief, Sichuan, 2008

The 2008 Sichuan earthquake occurred at 14:28:04.1 CST (06:28:04.1 UTC) on 12 May 2008, with its epicenter in Wenchuan County, Sichuan province of the People's Republic of China. It had a magnitude of 7.8 Ms according to the State Seismological Bureau of China and 7.9 Mw according to the United States Geological Survey.

With more than 60,000 recently confirmed dead so far, it was the deadliest and strongest earthquake to hit China since the 1976 Tangshan earthquake, which killed approximately 250,000 people.

Please consider donation to China Red Cross! Thank you!

Sunday, May 04, 2008

Qt Problem of Google Earth 4.3 Beta for Linux

On some systems (as Ubuntu Hardy 8.04) the Google Earth 4.3 Beta doesn't work very well, the fonts are ugly and tiny. You may try this workaround:
http://groups.google.fr/group/earth-linux/browse_thread/thread/e10a5337b0b47ba1#

On other systems (as CentOS 5.1) it may work very well out-of-box.

About Me

My photo
Santa Cruz, California, United States