Dell Precision 15 7510 Running CentOS 7.5

I set up my new laptop/mobile workstation. It's a certified refurbished Dell Precision 15 7510 from eBay. I'm pretty satisfied with the hardware as it looked brand new to me, although FedEx's shipping caused me some confusion and trouble. I installed CentOS 7.2 and am happy with the results.

Hardware

I purchased this latest Skylate model because I checked dell.com and was sure that it has pretty good Ubuntu support. The following is a summary of this laptop's hardware:

• 6th Generation Intel Core i7-6820HQ (Quad Core 2.70GHz, 3.60GHz Turbo, 8MB 45W, w/Intel HD Graphics 530)
• Graphics: 4GB Nvidia Quadro M2000M
• 15.6" UltraSharp UHD IGZO (3840x2160) Wide View Anti-Glare LED-backlit
• HD1: 512GB M.2 Samsung PCIe NVMe Class 40 Solid State Drive
• HD2: SK hynix SC210 2.5 7MM 512GB
• Intel Dual-Band Wireless-AC 8260 Wi-Fi with Bluetooth 4.1 Wireless Card (2x2)
• Webcam: 0x1bcf Sunplus Innovation Technology Inc.

Storage Performance: RAID-0 slower than single disk?

The new laptop has two drives:

1. 512 GB M.2 PCIe NVMe Class 40 Solid State Drive
2. 512 GB 2.5 inch High Performance SATA Class 30 Solid State Drive

I'd like my CentOS to run fast so it should be on the NVMe drive. I was curious about whether I can use RAID0 to make a super fast partition so I set up Linux RAID using two partitions of the same size on these two drives and compared its performance with a regular XFS partition on the NVMe drive. The result was a surprise.

First the mkfs.xfs ran very slowly on the RAID0 partition. It took about 30 minutes to finish while when running on a non-RAID partition it needed only a few seconds.

I then run fio using the ssd-test.fio workload. I changed the run time to 300 seconds to get more accurate numbers. The results are:

The RAID-0 chunk size was 128 kB. File system was XFS with 4096 sector size. You can see that, without encryption, RAID-0 is actually slower than NVMe in every workload, and LUKS encryption has a pretty big impact on I/O throughput.

Setting It Up

1. Updated BIOS to the latest version. This can be done in Windows or using a FreeDOS USB if your Windows is gone.
2. Disabled SecureBoot in BIOS. It is possible to install CentOS with SecureBoot but I don't want to deal with it right now.
3. Disabled Intel Rapid Storage Technology and switched to AHCI mode for SATA. I tried accessing Intel RST disk using a Fedora 25 Beta LiveCD, it still can't be recognized by the 4.8.0 kernel.
4. Disabled built-in Intel graphics and using nVIDIA only, because there's no easy way to do live switching between graphics card with X yet.
5. Installed all Windows 10 Pro updates and cleaned up unnecessary bits to reduce the Windows footprint
1. Uninstall unneeded software: Intel management stuff, trialware if there are any.
2. Turn off unneeded Windows feature (can be found on the Uninstall control panel module).
3. Disable hibernation: powercfg /hibernate off (as administrator) to free up around 32 GB (your RAM size).
4. Reduce page file size if needed.
5. Install WinDirStat and see if there are other useless files.
6. Disk cleanup -> System cleanup -> More options -> Cleanup old restore points.
7. Run a full system disk cleanup.
6. Created a Windows recovery USB (16 GB is necessary) in case I need to recover it later.
7. Luckily the pre-shipped Windows 10 Pro is on the SATA drive so I have the whole NVMe drive for CentOS. I used gparted and shrunk the Windows partition so I can save more things on the SATA drive using CentOS. There are around 3 or 4 other tiny partitions on the SATA drive for UEFI booting and I just left them alone.
8. Ran memcheck to make sure the RAM is in good shape (it also tested that my machine isn't vulnerable to the Row hammer attack).
9. Boot up a CentOS 7 Live USB.
10. Created partitions on /dev/nvme0n1 using fdisk because I don't know how Anaconda aligns partitions.
    • nvme0n1p1: EFI partition (150 MB), nvme0n1p2: /boot (2 GB), nvme0n1p3: / (70 GB), nvme0n1p4: swap (16 GB), nvme0n1p5: /home (389 GB)
    • I was overcautious and made sure all partitions are 4 MiB aligned using fdisk (parted also has auto-alignment checking but I couldn't figure out what align size parted was using).
    • Created LUKS partitions and made XFS file systems on top of them. I don't need LVM and wasn't sure if I could disable LVM in Anaconda so I just created the LUKS partition and file systems myself. I used "mkfs.xfs -s size=4096" to create XFS with 4096 sector size.
    • I don't know if this is correct but NVMe reports its physical sector size as 512 (/sys/block/nvme0n1/queue/physical_block_size) so mkfs.xfs creates file systems with 512 sector size. I guess this is actually harmless but I'm just a little paranoid about using 4096 sector size since all my other SSDs are reporting 4096 sector size. I created the /home partition myself and told Anaconda not to format it. But Anaconda insisted that the / partition must be formatted by itself and didn't provide an easy way to pass parameters to mkfs and it. What I did is to rename /usr/sbin/mkfs.xfs to /usr/sbin/mkfs.xfs.orig and create a script in place to always pass -s size=4096 to mkfs.xfs. After installation I checked using xfs_info and all newly created XFS partitions are using 4096 sector size.
11. After the installation finished, the system wouldn't boot. efibootmgr correctly created a new entry "CentOS Linux" that points to /EFI/somedrive/centos/shim.efi. However, it pointed to the WRONG drive. In my case, it pointed to the SATA drive that had a Windows partition instead of the NVMe drive that hosted the Linux UEFI partition. This is very likely a bug in Anaconda, which just uses whichever UEFI partition it detected first and assumed the system has only one UEFI partition. Finding the cause wasted me some time but once you found the reason fixing is easy: just boot into BIOS and create a new UEFI boot entry that points to the right shim.efi on the right drive. Dell's BIOS is pretty powerful and can do all sorts of UEFI/legacy booting mode mingled together.
12. Installed the kernel-ml package since CentOS 7.2 doesn't support Skylake yet. CentOS 7.3 and later has Skylake support out-of-the-box.
13. HiDPI (my laptop has a 4K internal display) works great after installing the mainline kernel.
14. Added rd.luks.options=discard to /etc/default/grub and discard to /etc/crypttab to enable LUKS discard support. This enables the LUKS device mapper layer to pass along the discard/trim command from the file system above and can help to keep the flash drives run at optimal performance. The downside is this can make the LUKS encryption less secure by leaking locations of the unused blocks (which is not a big deal for me).
15. I didn't mount XFS with discard, because that slows down daily operation and the XFS FAQ warns against that. Instead I just enabled weekly fstrim timer in systemd by running: systemctl enable fstrim.timer
16. Installed iwl7260-firmware-25.30.13.0-68.fc25.noarch from Fedora 25 to provide the firmware for the Intel 8260 wireless card.
17. Installed nVIDIA Proprietary driver 370.28 (latest version).
18. Connected my external low DPI monitor. By default everything looked huge on it because of the 2.0 scaling factor GNOME defaults to on the internal monitor. I used the nvidia-settings tool and enabled panning and changed scaling on the external monitor. The external monitor's resolution is 1920x1200, so I enabled panning 3840x2400 and set ViewPortIn to 3840x2400. After that the external screen became blank because of a bug so I also needed to run this command to enable "ForceFullCompositionPipeline=On":

    nvidia-settings --assign CurrentMetaMode="DPY-5: nvidia-auto-select @3840x2160 +0+0 {ViewPortIn=3840x2160, ViewPortOut=3840x2160+0+0}, DPY-2: nvidia-auto-select @3840x2400 +3840+0 {ViewPortIn=3840x2400, ViewPortOut=1920x1200+0+0, ForceFullCompositionPipeline=On}"

19. Used PowerTOP to check that the CPU can actually enter deep sleeping mode (C10). If not your CPU might be dying faster.

              Package   |             Core    |            CPU 0    CPU 4
    POLL        0.0%    | POLL        0.0%    | POLL        0.0%    0.0 ms  0.0%    0.0 ms
    C1E-SKL     0.8%    | C1E-SKL     1.3%    | C1E-SKL     1.1%    0.4 ms  1.6%    3.2 ms
    C3-SKL      0.4%    | C3-SKL   1.1%    | C3-SKL 2.1%    0.4 ms  0.1%    0.4 ms
    C6-SKL      7.9%    | C6-SKL     13.3%    | C6-SKL     25.0%    0.7 ms  1.7%    1.1 ms
    C7s-SKL     0.0%    | C7s-SKL     0.0%    | C7s-SKL     0.0%    0.0 ms  0.0%    0.0 ms
    C8-SKL     29.0%    | C8-SKL     31.0%    | C8-SKL     43.4%    1.6 ms 18.6%    3.3 ms
    C9-SKL      0.0%    | C9-SKL   0.0%    | C9-SKL 0.0%    0.0 ms  0.0%    0.0 ms
    C10-SKL    51.0%    | C10-SKL    44.0%    | C10-SKL    17.0%    5.1 ms 71.0%   16.9 ms
    

20. Updated on Nov. 24th, 2016: I discovered that the headphone jack had no sound. This is fixed by adding a file /etc/modprobe.d/alsa-base.conf with the following content:

    options snd-pcsp index=-2
    alias snd-card-0 snd-hda-intel
    alias sound-slot-0 snd-hda-intel
    options snd-hda-intel model=laptop
    options snd-hda-intel position_fix=1 enable=yes
    

    I heard that you will need to completely shutdown and restart the laptop to make this work. I got the instruction from here.
21. Brightness of internal display is not remembered across reboot. Per Soon's comment, you can set a fixed value on boot by adding:

    echo 100 > /sys/class/backlight/acpi_video0/brightness
    chmod 666 /sys/class/backlight/acpi_video0/brightness
    

    to /etc/rc.d/rc.local, and then chmod +x /etc/rc.d/rc.local.
22. Updated on Apr. 25th, 2017: Setting the screen backlight brightness to max when working in direct sun. CentOS 7.3's kernel defaults to using ACPI backlight control. It works great but cannot set the screen brightness to the maximum possible level, i.e., it's max brightness level is a little darker than the max level the hardware can reach. For now, in order to reach the max brightness level you can do a reboot and set the brightness level to max in BIOS (and disable any change to the brightness level you added such as the one above). After the system boots up, if you change the brightness level you wouldn't be able to come back to max again. The other workaround is to add the acpi_backlight=vendor kernel option, which switches to using the dell_backlight driver. The dell_backlight driver, however, is buggy; it would fix the brightness at the max level and you wouldn't be able to change it.

End Result

Everything else works well, including suspend-to-RAM, external display, HDMI, DisplayPort, webcam, sound, Wi-Fi, SD Card Reader, and USB-C.

Some nitpicks

• Brightness of internal display is not remembered across reboot.
• Sometimes after suspend to RAM, the CPU frequency scaling doesn't work well and all CPUs are stuck at around 400 to 500 MHz, like:

  analyzing CPU 0:
    driver: intel_pstate
    CPUs which run at the same hardware frequency: 0
    CPUs which need to have their frequency coordinated by software: 0
    maximum transition latency: 0.97 ms.
    hardware limits: 800 MHz - 3.60 GHz
    available cpufreq governors: performance, powersave
    current policy: frequency should be within 800 MHz and 3.60 GHz.
                    The governor "powersave" may decide which speed to use
                    within this range.
    current CPU frequency is 468 MHz.
    boost state support:
      Supported: yes
      Active: yes
  

  It doesn't happen most of the times, and I haven't figured out why. If I feel the machine is sluggish I just run sudo cpupower frequency-set --governor performance (with it binded to a hotkey). There are some other people having the same issue.
• I wish the battery could last longer. I don't have time to do any manual tuning yet. Now when on battery I can get like 4 hours regular use or 1 hour heavy use (running my CPU heavy machine learning code). This is on par with an early review of this laptop running Windows. I am pretty sure it can do better if I do some power usage tuning in PowerTOP.
• Live switching between nVIDIA and Intel graphics card not working but I'm ok with using nVIDIA only since the Intel Skylake driver is buggy anyway. The downside is probably more power consumption.

---

Some random thought. This new laptop's nVIDIA graphics feels much faster than the Intel graphics in my old Late 2013 Retina MacBook Pro. I'm just talking 2D performance here. The old Intel graphics in the rMBP always stutters when driving one external monitor and crashes when I add two external monitors. It drives two external monitors fine when running macOS so this is likely a software issue. I can't help but think that Intel doesn't allocate enough resources for their Linux drivers. There are other problems with Intel's driver:

• No easy way to install different versions on RHEL/CentOS. The official installer doesn't support RHEL/CentOS yet. The nVIDIA driver, while being closed source, always support most Linux distros on the market and I can install any version I choose.
• Buggy and incomplete 2D acceleration support.
• 3D performance much slower than Windows driver.
• The Xorg running on my old rMBP had an about 20% chance crashing when plugging in an external monitor. I don't know if I should blame the Xorg server or the Intel driver but with the same Xorg using nVIDIA driver in the new laptop, it has never crashed once yet.

Even though I hail Intel's engagement with the open source community and their open source drivers, I feel that the quality of their drivers is not on par with nVIDIA's proprietary drivers.

Updates

• Oct. 2018: Upgraded to CentOS 7.5
• Nov. 2017: Tested USB-C using a Google Pixel 2 XL
• Dec. 2016: Upgraded to CentOS 7.3

Using rsync to migrate data between Linux and Apple Mac OS X

I got a new MacBook Air running Mac OS X (10.7.1) "Lion", and I managed to migrate my archive data (around 1.3TB) from an old Linux laptop running Debian Testing "Wheezy." The process was not smooth and I hit several blocks. Here I document several points that you need to pay attention to when migrating data across operating systems:

Zeroth

Use the latest rsync (3.0.8 from MacPorts was used here). The old version of rsync is buggy on Mac. The version of rsync on Linux doesn't matter much because rsync on Linux has been quite stable since long ago.

First

Disable "Ignore ownership on this volume" (you can use Finder or from command line). If not rsync won't be able to set file permission correctly. I think if you don't instruct rsync to preserve permission and mode then you might not need to do this but I'm not very sure.

Second

Disable antivirus software like McAfee because they interfere with rsync's time preservation.

Third

Use rsync's "--iconv=UTF8-MAC,UTF-8" option (only available after rsync 3.0), because Mac's HFS+ decompose UTF-8 before storing it so the filename would be different from that of Linux if no conversion is done right.

If you saw "iconv_open("UTF-8", "UTF8") failed", try swapping the parameters of iconv.

Fourth

This is not directly related to rsync nor Mac, but a general rule for handling archive data: use another piece of software to do the verification. Your rsync might be buggy, the OS might be buggy, the disk or USB cable might be kaput, you might have used the wrong options or settings. Therefore you should always use another piece of software other than rsync to verify the data before and after the migration. I used AIDE to do file checksum and time verification.

openSUSE Build Service (OBS): Break the Link Between a Branched Package and Its Source

We are using openSUSE Build Service (OBS) for the development of MeeGo. I have an old package, which was branched from a Trunk package long ago. But every time the source pakcage in Trunk is updated my package would be broken because the changes between them are so large that it's become meaningless to keep the link between them any more. So I think I should just remove the link between them.

The simplest way may be just removing my package and start a new package from scratch, however, this action not only removes the package history but will also reset the cumulative release version number, which is unacceptable since that would make it impossible for client machines to follow updates to this package.

I have carried out a long search but failed to find a hint to this issue, and after some trail-and-error I think I have found a way:

1. Check out the project's unexpanded files by using "osc co -u PRJ/PKG"


2. Remove the "_link" file by running "osc rm _link"


3. Copy in the latest package files since the link is now broken


4. Check-in the changes

I've done a test and the steps above worked. However, it seems that by those steps the release version number will also be reset.

Update: darix from freenode/#opensuse-buildservice said 'osc in git has "osc detachbranch" exactly for that.'

HOWTO: Run Plants vs Zombies™ in Linux under Wine (includes how to Register!)

I have to admit that I found PopCap's Plants vs Zombies™ (referred to as PvZ below) interesting. It runs on Windows, Mac and iPhone but not on any Linux yet.

You can download the trial version from the official site above, and it runs amazingly well on a recent Wine, without any special configuration. Well done, Winehq!

Now comes the problem, if you decide to buy it after 60 minutes play, the registration process is a bit hard with Wine. The reason is that PvZ requires an Internet Explorer or Firefox to finish the registration process. Obviously it has to gather some local hardware/system information and upload it to their activation server so that they can know how many machines you are running PvZ on, therefore you won't to able to give away free copies of the game to all your friends. I think this whole thing is OK, though instead of writing codes in PvZ to let it communicate with their server, instead, sadly, they choose to use ActiveX running in IE for this.

Some early posts reported that the registration process can be finished if you install an IE into the same Wine directory. Running IE on Wine is not impossible but hard, especially when IE4Linux project is no longer maintained.

The HOWTO

Now I have found that you can also finished the registration process using not IE but Firefox instead. Thanks for their being in the same free software camp, Firefox runs very well on Wine. Just grab a Windows copy and install it into the same Wine directory.

Then run Firefox in your Wine, go to PopCap's Registration Page, follow the instruction there to download and install the Firefox plugin. After that, close your Firefox (for some unknown reasons, the Firefox Wine process may still linger in the background after its window is closed, run "ps -ef | grep -i firefox" and kills the linger process if need).

Now starts PvZ by running its in your winefile. Because the PvZ and Firefox need to be started in a same Wine instance so they collect the identical hardware/system information. Register it following the normal process. It may fail at the first attempt to start a web browser, keep clicking the "Retry" button until it shows it's waiting for the registration process to finish (it's waiting for web browser plugin to return registration key). Start a Firefox instance from that same winefile program. Go to PopCap's Registration Page, now if your plugin is installed successfully, it should show a web page asking for your POP Order Number, give it and the registration process should finish successfully and the PvZ program should proceed to the game.

It's not rocket science, just nuisance.

PvZ runs very well in Wine during my 2 days play. I can say it's perfectly running on Linux for now.

Known Problems

1. Sound effects may not work on some machines with some old Wine versions (try upgrade to latest Wine.)


2. The full-screen mode will be always "full-screen," which means that if you have a non-4:3 screen, it will be slightly torted.


3. The between-battle preview screen may be a bit slow sometimes, but as soon as the battle begins the speed returns to normal (not sure if it shows similar issue when running on a real Windows.)
   

MeeGo is in Google Summer of Code 2010

MeeGo has joined Google Summer of Code 2010. If you are a student, have some free time this summer and have interests in developing operating system or applications for new small form factor devices, please consider join us!

MeeGo is the combined Moblin and Maemo and is sponsored by Linux Foundation, Nokia and Intel. To make it simple, MeeGo is the new operating system, inspired by the rising interests and challenge in smart phone and mobile internet devices, for future Intel and Nokia ARM platforms (or more if other handset makers join).

To distinguish it from other smart phone platforms such as iPhone and Android, MeeGo is proudly bearing the badge "open." It is not controlled by a single authority, but under the non-profit consortium Linux Foundation and welcome all parties and individuals who interests in making the best mobile platform in the world.

Another thing that distinguishes MeeGo from other mobile platforms is that MeeGo runs full Linux desktop stack. It is running unmodified Linux kernel, X server, compiled by unmodified GCC, includes full GTK / GNOME and Qt libraries. So all your expertise on desktop Linux continues to thrive here.

But MeeGo is not yet-another-Linux-distro. Smart minds may already spot the great challenge here. How could we build a competing platform by using just commodity GNU/Linux OS codes? The answer from us is that we are focusing on usability plus aggressive optimization. We believe the codes from those great upstream free software community is the best in the world and we can build the solid state-of-art platform based on them. What we have to do is try our best to integrate them together and focus on design a great user experience.

So if you are interested, please take a look at the MeeGo project ideas. Of course we are more than happy if you can come up with you own great ideas. And try to get yourself a $5,000 stipend.

(all words above are my personal opinion and reflect no idea from MeeGo committee, Linux Foundation, Nokia or Intel)

Another GPL violation case from top China IT firm? Hanwang this time.

Hanwang's eBook Reader products (汉王电纸书产品系列, http://www.hw99.com/product/productkind-13.htm) are very popular in China market. They features an ARM processor and are running Linux with cramfs, which can be easily discovered from the update firmware the company has released. Also they are shipping BusyBox within them.

By no means I can find the source code to their Linux kernel / BusyBox. So perhaps this is yet another GPL violation case from top China IT firm. Anyone bother a law suit?

Safely remove an USB hard drive in Linux

I bought a new USB hard drive, Western Digital My Book® Essential Edition™ 1TB, and use it with Linux. Here I'm discussing about how to safely remove (then disconnect) it from a Linux system.

The problem: according to its user manual, you should disconnect the drive safely when it is not active, but in a Linux system how to do this is not very straightforward. You may notice that after you unmount it (whether through command-line or a desktop environment), the drive is still spinning and it's LED on. If you read it's user manual carefully you'll find that the manufacturer never said it's safe to disconnect it from your system in such condition.

The quick solution to this problem:

1. send SYNC, then STOP command to the device, this can be done easily in Linux by unbinding the device


2. suspend the USB port by echoing a "suspend" to the "/sys/bus/usb/devices/$DEVICE/power/level", where $DEVICE corresponding to the device of your USB device.

To put it simple, I wrote a script to do this for you. Download and try it. For developers, you can get it from the github repo.

If you saw an error message like this when running it:

bash: /sys/bus/usb/devices/5-8/power/level: No such file or directory

that means your kernel doesn't support USB suspend. You can just disconnect your device now, it's already safer enough to unplug it.

To those who is interested, discussion of technical detail following.

From the hardware aspect, the drive should only be removed when the following requirements are met:

1. there's no pending I/O request and software cache flushed


2. it's hardware cache flushed


3. it's driver spins down (means "not spinning")


4. (optional) the USB port is put into "suspend" mode

The user's manual read (page 6, section "Turning Off/Disconnecting the Device"): "(for Windows systems): Right-click the Safely Remove Hardware icon in your system tray and select Safely Remove Hardware. You may hear the drive power down before the Power LED turns off. The drive is now shut down properly, and you may disconnect the drive safely. (for Macintosh systems) Drag the My Book icon to the Trash icon for proper dismount. You may hear the drive power down as the Power LED flashes. When the Power LED is steady, you may press the Power button once or disconnect the drive’s power cord to turn off the drive safely." (there's a mistake, this device doesn't have a Power button) If you read carefully you can tell that Windows and Macintosh don't do the same things when told to remove an USB device. As the bold text as shown, in a Windows system it will be turned off while in a Macintosh system you hear the drive power down but the Power LED would finally become steady rather than off. Therefore we know that Windows does all 4 steps and Macintosh do the first 3 steps only when told to remove a USB device. (BTW, it's funny that you have to use your ear before disconnecting a device.)

(This is only the case for Windows XP. For Windows Vista seems only the first 3 steps are done thus the drive remained active after you removing it from the system.)


Some may argue that they always disconnect their drives directly after unmounting (step 1 only) and never experience any damage, but as an operating system developer and a serious user, I always follow the manufacturer's manual or specification whenever it make sense and possible because I know electronic devices are fragile, rigid and obtuse if you don't use them as the way they are designed. For a well-designed and robust piece of hardware, theoretically you can disconnect the device safely when the first two steps are done. The firmware on the device would notice the USB cable was unplugged and shut down the disk drive properly. If not, the drive would be stopped sharply as if you pulled it's power when it's running and this damages the hardware gradually. As an outsider of the manufacturer, we hardly know whether it's well-designed or not so the only safe principle is to follow it's manual, means we should not disconnect the device until at least all three steps are done.

At the Linux kernel part. To finish all the 4 steps, you have to unmount the device, unbind the device from the driver, then tell the USB core driver to put that device into suspend mode, as shown in the solution above. And you have to be running a kernel with CONFIG_USB_SUSPEND enabled.

CONFIG_USB_SUSPEND is not enabled by default in a vanilla kernel. I'm pushing it in this discussion.

Properly removing a device from a running Linux should be done by either the HAL or desktop manager. If you can, please push the developers you know working on any DM that missing this feature to implement this function properly.

Try my script, and write me if you found a problem. Welcome patches.

Don't use APT::Default-Release in Ubuntu unless you know what you are doing

Setting APT::Default-Release in Ubuntu blocks all future security fixes and updates.

This is related to all versions before Hardy (include). I haven't tested this on Intrepid so I'm not sure about those versions after Hardy.

According to apt_preferences manpage, the target release can be set on the apt-get command line or in the APT configuration file /etc/apt/apt.conf, and "APT::Default-Release "stable";" is given out as an example. This is a very common and popular practice used in Debian community to set the default release and using apt-pin, but doing this in Ubuntu leads to serious security impact with no obvious warning.

After setting APT::Default-Release to "hardy", which is the "Suite" name for main hardy source, no security fixes nor updates would be installed unless their priorities are also set explicitly in apt_preferences. This is because that in Ubuntu's world, security fixes are from "hardy-security" source and other updates are from "hardy-updates" source, which bear different "Suite" from the main source. Setting APT::Default-Release rises the priority of packages from main source to 990, but doesn't cover packages from hardy-security and hardy-updates, so the latter are ignored since their packages now has lower priority (priority 500 only) than those old ones in main source (990).

I set APT::Default-Release to "hardy" on Sep this year until I found this problem today. Removed that setting and I'm surprised to found that I can install 46 security fixes and updates accumulated. Which is pretty sad to me that got known I haven't got security fixes for more than 2 months.

This is a radical deviation from the Debian practice. In Debian all security fixes and updates bear the same "Suite" (etch or lenny) so setting APT::Default-Release to "etch" covers all security fixes and updates.

I think it's unlikely that Ubuntu changes the organization of it's source, so at least a fix to this problem is patching the apt_preferences manpage, alerting people not to use APT::Default-Release like they have used this in Debian and the reason and the following impacts.

I've opened a bug about this: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/295448

Ripping VCD in Linux

I've got some old VCD discs so I've spent some time on studying how to rip them for backing up.

There are two methods available:

1. To make a perfect ripping, use cdrdao read-cd:
   
   cdrdao read-cd --device ATA:1,1,0 --driver generic-mmc-raw --read-raw image.toc
   
   the --device is optional. By using this you got a TOC/BIN file pair that contains all the tracks from VCD which can be used to produce exact 1:1 copies. You can also generate a CUE/BIN pair by passing a filename with .cue extension. Although it is said that mplayer and totem can play the BIN file if proper decoding plugins are installed, I have never been able to do that, which is a big problem for me.
   
   Credit goes to manmath: http://ubuntuforums.org/archive/index.php/t-217997.html


2. For easy playing back, you can extract MPG files by using GNU VCDImager, which is included in many popular distros. Using the vcdxrip program:
   
   vcdxrip -C=/dev/cdrom
   
   By this you got all videos in usable MPG files.

Though you may be able to copy DAT files from VCD to your hard drive directly in Windows, this is not the case in Linux systems, you'll get I/O errors. The reason is that the file system of VCD disc is not valid ISO-9660 and never meant to support direct copying video data as plain files. Windows must be doing some conversion behind the hood.

mplayer and ASS subtitles in UTF-16 with BOM

Tina got a piece of .mkv video with ASS subtitles, which was in UTF-16 and began with BOM (0xFF 0xFE), perhaps generated by using some programs from Windows kingdom. mplayer can't load ASS that begins with BOM and produces the following error message:

Matroska file format detected.
VIDEO:  [avc1]  864x480  24bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
SUB: Could not determine file format
Cannot load subtitles: file.ass

The remedy:

1. Convert the ASS file to UTF-8 (or whatever encoding you are comfortable with). I used iconv: iconv -f utf16 -t utf8 < file.ass > result.ass


2. Play it by using the "-utf8" switch of mplayer: mplayer -ass -utf8 file.mkv. If the ASS contains special characters (such as CJK chars), use a good font by "-font /usr/share/font/cjk.ttf"

Fix nVidia's Proprietary Driver's Poor 2D Performance on Linux

Call me a fool. I bought an XFX GeForce FX 5200 PCI for my workstation (which is actually a Dell SC440 tower server who has no AGP slot). And nVidia's poor proprietary driver is notorious slow of 2D on Linux. The CPU usage of the Xorg process easily goes up to around 70% on my Xeon Core 2 @ 1.86GHz even when scrolling the text in gnome-terminal, not to say reading web pages in Firefox, which slows down everything else. And I've been sticking to a years old pristine GNOME (2.14 shipped with Debian 4.0 Etch) with nothing eye-candy here.

Finally I've found a workaround: http://www.nvnews.net/vbulletin/showpost.php?p=1704073&postcount=1: Update (2008-07-18) worked for me. In courtesy of xsolarwindx.

The improvement: now with this little fix the time used to build an Linux kernel on my workstation is only half as before just because the little baby likes writing its progress onto the terminal and old nVidia hates this so it made the displaying of text even more time-consuming than compiling the multi-million-line kernel itself!

I've been running nVidia's driver 173.14.12 x86_64 on a stock Debian 4.0 Etchnhalf kernel (2.6.24-etchnhalf.1-amd64).

I'll never buy an nVidia unless they released full GPL drivers.

Install ATI's fglrx Catalyst 8.8 Driver (v 8.522) on Linux 2.6.27

The Linux kernel 2.6.27 has changed an API (smp_call_function()) that breaks compilation of ATI's Catalyst 8.8 fglrx driver. You get error like this:

/usr/src/fglrx-8.522/firegl_public.c:3080:50: error: macro "smp_call_function" passed 4 arguments, but takes just 3
/usr/src/fglrx-8.522/firegl_public.c: In function ‘__ke_flush_cache’:
/usr/src/fglrx-8.522/firegl_public.c:3080: error: ‘smp_call_function’ undeclared (first use in this function)
/usr/src/fglrx-8.522/firegl_public.c:3080: error: (Each undeclared identifier is reported only once
/usr/src/fglrx-8.522/firegl_public.c:3080: error: for each function it appears in.)
...

Loïc Minier has made a patch for Catalyst 8.7, and I've adapted it to Catalyst 8.8:

From 42a6390f599294c60a4960b0a6cd4e399b1a81ee Mon Sep 17 00:00:00 2001
From: Yan Li 
Date: Sun, 14 Sep 2008 17:45:48 +0800
Subject: [PATCH] patch for building on 2.6.27

ref: http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43
---
 firegl_public.c |   15 +++++++++++----
 1 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/firegl_public.c b/firegl_public.c
index bdc94b2..d85a567 100644
--- a/firegl_public.c
+++ b/firegl_public.c
@@ -202,6 +202,13 @@
 #define preempt_enable()
 #endif
 
+/* Since 2.6.27 smp_call_function doesn't have a nonatomic/retry argument */
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,27)
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, wait)
+#else
+#define SMP_CALL_FUNCTION(func, info, retry, wait) smp_call_function(func, info, retry, wait)
+#endif
+
 // ============================================================
 /* globals */
 
@@ -3077,7 +3084,7 @@ int ATI_API_CALL __ke_flush_cache(void)
 {
 #ifdef __SMP__
     /* write back invalidate all other CPUs (exported by kernel) */
- if (smp_call_function(deferred_flush, NULL, 1, 0) != 0)
+ if (SMP_CALL_FUNCTION(deferred_flush, NULL, 1, 0) != 0)
   panic("timed out waiting for the other CPUs!\n");
 
     /* invalidate this CPU */
@@ -4796,7 +4803,7 @@ struct _agp_memory* ATI_API_CALL __ke_agp_allocate_memory_phys_list(
 void ATI_API_CALL KCL_CallFuncOnOtherCpus(firegl_void_routine_t func_to_call) 
 {
 #ifdef CONFIG_SMP
- smp_call_function( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
+ SMP_CALL_FUNCTION( firegl_smp_func_parameter_wrap, (void*)func_to_call, 0, 1 );
 #endif
 }
 
@@ -4910,7 +4917,7 @@ static int ATI_API_CALL KCL_enable_pat(unsigned int save_orig_pat)
    }
 
 #ifdef CONFIG_SMP
-   if (smp_call_function(KCL_setup_pat, NULL, 0, 1) != 0)
+   if (SMP_CALL_FUNCTION(KCL_setup_pat, NULL, 0, 1) != 0)
        return 0;
 #endif
    KCL_setup_pat(NULL);
@@ -4929,7 +4936,7 @@ static void ATI_API_CALL KCL_disable_pat(void)
     }
 
 #ifdef CONFIG_SMP
-    if (smp_call_function(KCL_restore_pat, NULL, 0, 1) != 0)
+    if (SMP_CALL_FUNCTION(KCL_restore_pat, NULL, 0, 1) != 0)
        return;
 #endif
     KCL_restore_pat(NULL);
-- 
1.5.4.3

I suggest use the --buildpkg function of the ATI driver to build packages for your system. Those packages use dkms to build the kernel module needed. (For Debian/Ubuntu flavor Linux, I suggest use the way as described in https://help.ubuntu.com/community/BinaryDriverHowto/ATI.) Apply the above patch after installed the fglrx-kernel-source.*.deb to /var/lib/dkms/fglrx/8.522/source, then run "sudo dkms build -m fglrx -v 8.522" to build the kernel module.

Reference:
http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/6086b1cb7674fc43

My HD High Load/unload Cycle Problem Fix

I've just applied http://en.opensuse.org/Disk_Power_Management to fix the high load/unload cycle problem of my IBM ThinkPad T43p laptop. My Load_Cycle_Count grows at around 30 per hour, which is not very high, but theoratically limit my HD's life to around 4 years.

PS. though I dislike SuSE due to their pact with Microsoft, they are doing really well in some fields. The article mentioned above is well written, neat and clean, and the script is in good quality. I was told that openSUSE has already included patches to fix this problem, while Ubuntu's users are still struggling to either fix it by themselves or talking in the forums, trying to understand what to do.

IBM/Lenove ThinkPad Battery Maintenance in Linux

I felt the deteriorate of the lifetime of the battery on my IBM ThinkPad T43p. This article has a good discussion on best practices. I also installed the tm_smapi driver and set charge threshold for my laptop.

Save battery, no waste. :)

Use pdftk to Update PDF File's Meta Data

A good tool, pdftk, can be used to manipulate PDF files, changing meta-data, page-juggling, etc., you name it! And it's GPLed.

Easy, peasy, japanesey.

Error of Running VMware Server 1.0.5 on Debian 4.0 Etch

I installed VMware Server 1.0.5 on Debian 4.0 Etch (i686) today and couldn't start vmware-console due to this error:

(vmware:20921): libgnomevfs-WARNING **: Cannot load module `/usr/lib/gnome-vfs-2.0/modules/libfile.so' (/usr/lib/vmware/lib/libgcc_s.so.1/libgcc_s.so.1: version `GCC_4.2.0' not found (required by /usr/lib/libstdc++.so.6))

To fix this problem, start VMware by running:

VMWARE_USE_SHIPPED_GTK=yes vmware

For more information on this please click the title, which leads to a post on VMware community website.

RPM Spec: conditional build based on host distro

When writing spec of RPM, often you need to write some special instructions that are needed only on some distros or versions, but not all. You can use this:

# Do something for FC4 and beyond.
%if 0%{?fedora} >= 4
    # ...
%endif

Tweak GNOME Auto Mount Options

Use gconf-editor to edit the key:

/system/storage/default_options/vfat/mount_options

The most useful one for me to add is "iocharset=utf8".

Ignoring .svn directories with grep

Have lived with Subversion for many years and finally I found this (link to outside my blog) simple little trick. :)

Choose a Linux-friendly Bank in China

At last I thought I should spend some time on finding a Linux-friendly bank. By Linux-friendly I meant that the e-Bank services should be able to achieved from a Linux machine running native browsers such as Firefox. I've been using China Merchant Bank for nearly 10 years and although generally good, their eBank services are designed around IE and ActiveX. What's more annoying is that their bank software could not be run within a virtual machine, alleged for the sake of better security, which shut all doors to run it on Linux.

I used eBank to manage loads of things such as utility payment, donations, credit card payment and investment management. So it's really a waste of time to reboot to MS Windows^TM to do them.

So finally I chose Shanghai Pudong Development Bank. It's eBank services are great with Firefox! Till now I'm happy with it. I've also applied for the credit card and hope to move all my credit card subscriptions to it gradually.

Now the only gray area for me is that my employer doesn't support to pay into SPDB. I'll keep complaining to my beloved HR manager. :)

N.B. HSBC is doing very well with Linux too. Who (I guess) has been running IBM's AIX and System z for a long time and is generally browser-independent.