i-am.ws |
Alpha for PNG with NetPBM
As a co-developer of pnmtopng / pngtopnm it's already a long time my plan to update those two NetPBM programs for the PAM format, allowing for proper alpha channel support. On sourceforge you can find a pngtopam program — not sure how full featured it is — but there is currently no pamtopng, which is IMHO the more important of the two.
Two weeks ago I was myself in desperate need for a PAM to PNG converter, and I didn't have the time to do some proper development, therefore decided to do a quick & dirty one. Luckily I wrote 10+ years ago two PngMinus tools, which was targeting the same type of converters, but without the need for the NetPBM libraries.
Long story short, I wrote a "pam2png" utility, which is not as feature rich as a pamtopng program would be, but it does the job. Its main purpose is to convert 4-channel "RGB_ALPHA" pam images to png, but it does the conversion of 2-channel "GRAY_ALPHA" as well. However, the latter wasn't extensively tested.
To be honest, I stopped testing and debugging when the program was good enough for the images I needed to convert to PNG. And I will not spend too much more time on it, because the real goal is a proper pamtopng implementation. But if you find bugs, send me the fixes and I will incorporate them. Don't waste your time on adding functionality, because this is just a temporary job.
Finally, if you don't know how to create those PAM images with R+G+B+A channels, check out the pamstack command. This tool allows you to combine a regular RGB ppm image with an alpha channel image packaged in a pgm file. The README file in the tar-ball shows how to do it.
@ 2011 Dec 18 12:33 AM, entry [ alpha_for_png_with_netpbm ] was posted in [ linux ]
CentOS 6 boot from USB
Installing CentOS 6 on a laptop without CD/DVD seemed to be easiest by using a Live-USB stick. However, on the net there were warnings that version 6 behaves different from CentOS 5. And indeed, it took me too a couple of attempts before I was successful. I downloaded the LiveCD ISO for CD, not DVD, because with 700 MB, it would fit on the 1 GB USB key I had lying around.
After a little research, I opted for this livecd-iso-to-disk script (or get it from GIT), which promises to do it all automagically. My first problem was that my CentOS 5 system didn't have 'udevadm', which I solved by changing in the script "udevadm info" into "udevinfo" and swapping "udevadm settle" for "udevsettle". After that, the USB key would build, until the stick had to be made bootable. Long story short, the isolinux / extlinux in CentOS 5 is too old. I could have downloaded a more current tarball and built it, but I was too scared that that would break something else.
My solution was to move the whole project to another system running a more current OS, in this case Fedora 15. After installing extlinux with a "yum install syslinux-extlinux", not only my udevadm issues were solved, but it also had a more recent isolinux, version 4. One tip: if you copy everything to the USB key and then at the end something breaks, like my extlinux, you can use the option "--skipcopy" to save quite a bit of time. Finally, I used an ext3 filesystem on my stick and it is a good idea to start each new attempt with a "mkfs.ext3 /dev/sd@#".
After booting from the USB stick, the next steps were of course an "Install to Hard Drive", then a "yum update", followed by using the GUI to install a whole slew of RPM packages.
@ 2011 Oct 15 06:03 PM, entry [ centos_6_boot_from_usb ] was posted in [ linux ]
Slimp3 server with Voyage
With two SLIMP3 players in the bottom drawer (if I remember correctly, each around sixty bucks on eBay), it got time to make some better use of them. Now I've always had a SlimpServer / SqueezeBox running on the same box that's doing my MythTV, but I needed a setup independant from that. A small server, running Linux and Slimp, making no noise and preferably being cheap. That was the goal.
I started my research for Single Board Computers (SBC) again, checking out the latest status on Fit-PC and SheevaPlug. But then I found this little gem: the "alix3d3" from PC Engines. A Swiss company that makes SBC's in a form factor smaller than micro ITX. The board I got is 10x16 cm, powered by an AMD Geode processor and includes VGA, network port, 2x USB, RS232. And if you add a mini-PCI you can even do wireless.
For my Slimp3 server, I added a 512 MB CompactFlash to boot from and a 16 GB USB stick for the music library. I could order all of that for less than 200 bucks including a pretty cool aluminum enclosure. The total size is smaller than an external 3.5" harddisk. Now the thing I'm still most amaized about is that that little Kingston thumbnail on the right is storing 300 CDs. I need quite a couple of bookshelves to store the physical CDs these MP3's came from.
Now the fun part was of course the software install. Did a bit of research and with this reduced hardware, a full blown Fedora or Ubuntu is not the way to go ..... but who needs X-Windows on a media server. The answer is a distro called Voyage. Based on Debian, but targetted at this type of SBC controllers. By itself it fits in roughly 100 MB, but when you include the squeezebox software and then the various dependant packages, it just fits in the 1/2 GB I had at hand.
Here follows my little HOWTO on installing it. The first step is to install Voyage on your CompactFlash, while it is still hooked up to your dekstop:
# fdisk -l /dev/sda Disk /dev/sda: 512 MB, 512483328 bytes 16 heads, 63 sectors/track, 993 cylinders Units = cylinders of 1008 * 512 = 516096 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 993 500440+ 83 Linux # mkfs.ext2 /dev/sda1 # tune2fs -c 0 /dev/sda1 # bunzip2 -k ./Voyage-0.7.5.tar.bz2 # tar xvf ./Voyage-0.7.5.tar # cd ./Voyage-0.7.5 # more ./README # more ./README.live-cd # # ./usr/local/sbin/voyage.update # What would you like to do? [Create new Voyage Linux disk] What would you like to do? [Select Target Profile] Please select Voyage profile: [ALIX] What would you like to do? [Select Target Disk] Which device accesses the target disk [/dev/sda]? Which partition should I use on /dev/sda for the Voyage system [1]? Where can I mount the target disk [/mnt/cf]? What would you like to do? [Select Target Bootstrap Loader] Which loader do you want (grub or lilo) [grub]? Which partition is used for bootstrap [1]? What would you like to do? [Configure Target Console] Select terminal type: [Console] What would you like to do? [Partition and Create Filesystem] What shall I do with your Flash Media? [Use Flash Media as-is] What would you like to do? [Copy Distribution to Target] OK to continue (y/n)? y What would you like to do? [Exit]
Next step is to move the CF card to your ALIX computer and boot that one. Then you've to install the SlimpServer / SqueezeBox packages. One of the problems is that the name of these packages has changed a couple of times, while Logitech took over Slimp. Based on this WiKi page, here is what finally worded for me:
# cat /etc/apt/sources.list deb http://debian.slimdevices.com stable main deb http://ftp.hk.debian.org/debian/ squeeze main contrib non-free deb http://security.debian.org/ squeeze/updates main contrib non-free # apt-get remove --purge slimserver # apt-get remove --purge squeezecenter # apt-get remove --purge squeezeboxserver # apt-get update # apt-get install squeezeboxserver
With that, your Slimp3 should work, but only when it is in "remountrw" mode. That's because SqueezeBox writes a cache to /var/lib, which is on Voyage a 'read-only' protected filesystem. To run your whole system, including the squeezebox software, in "remountro" mode, the trick is to modify your /etc/init.d/squeezeboxserver so that it copies the contents of /var/lib/squeezeboxserver to a new directory you create as /tmp/lib/squeezeboxserver. Next you've to change couple of environment variables in the squeezeboxserver setup to point to that directory instead of using /var/lib.
Enjoy the music!! The audio quality will be noticably better than what your typical computer can produce.
@ 2011 Oct 09 12:08 AM, entry [ slimp3_server_with_voyage ] was posted in [ linux ]
PnmBlend Disappearing Act
Recently I was working on an automated tool to create images of 19" racks with their content. I got a nice picture of the new Cisco R42610 rack, but with the doors off, it still showed the hinges. No big deal, but it made the picture a little ugly.
I wanted to let those hinges disappear and had to be able to make it a repeatable thing. Therefore I didn't want to do it with PhotoShop, but preferred to script it using NetPbm. Now, NetPbm didn't have the right tool for the job, so it was time to write my own. Wasn't the first time, ten years ago I wrote together with Alex the pnmtopng/pngtopnm converters and more recently I contributed "pnmmercator", a cartography application.
The result of my current effort is "pnmblend", a little utility that will take the top and bottom row of an image (or the left and right column) and replace everything in between with a fading (or gradient) from one to the other.
On the left you see the rack with the hinges still there. The next picture zooms in and the third shows the piece we've cut out to work on with "pnmblend". After that you see the output of my new utility and the last image is the result of putting it back in the original image using "pnmpaste".
This code isn't yet ready to be included in the official NetPbm package. For that I've to convert it from a 'pnm' to a 'pam' utility. Secondly, the current code isn't based on the NetPbm libraries, but is doing it all in itself. Which could in certain cases even be an advantage.
@ 2011 Jan 29 09:14 PM, entry [ pnmblend_disappearing_act ] was posted in [ linux ]
Apache mod_substitute
When building my new Apache Roller system, I had quite a few challenges with the i-am.ws domain, served from a web-server in the DMZ and Roller on a Tomcat server in the backend. The mod_proxy Apache module takes care of most of it, but hardcoded URLs in the html code need to be translated as well from the internal IP to the external domain name.
A month went by and I found the solution in stumbling on the "mod_substitute" and "mod_sed" modules. With those, the proxy can fix on the fly the hard-coded URLs generated by Roller. In this scenario, substitute "http://192.168.1.23:8080/roller/" for "http://www.i-am.ws/" and you're all set. The only thing to be aware of is that you must be running a pretty recent version of Apache (2.2.7 for "mod_substitute" and even 2.3 for "mod_sed").
Now that sounds like a done deal, but it took me couple of weekends to figure out a nasty snag with "mod_substitute" and the like. I thought it was caused by mod_proxy and mod_substitute clashing with each other. So I thought I was clever :-) and moved mod_substitute to the back-end. I installed another Apache on a different port (8642) that would do a local proxy to Tomcat (on 8080). It didn't fix the problem, but by doing I discovered that when I retrieved the web-page with "wget" (one of my favorite hacking tools) instead of Firefox, string substitution worked fine. Now my suspision moved to issues with HTTP 1.0 vs. 1.1. Again, that wasn't the case.
Getting desperate, I decided to write my own proxy server. Grabbing some old bits and pieces of code still lying around :-), that wasn't too tough. At first, things went fine and then the same thing happened again. But with my own code – not running in the background – I had a much better handle on debugging. Checking the HTTP headers of the request, I suddenly noticed that Firefox sends to the server an "Accept-Encoding: gzip, deflate" http header record (BTW, same with IE). And yes, the server replied with all html code nicely compressed. Which suddenly explained why all this time mod-substitute couldn't do anything with it. It also explained why wget, not sending that header, was working fine.
For now, I'm sticking with my home-grown proxy, but the proper solution is I guess to use mod_rewrite to get rid of those compression HTTP headers. And then mod_substitute can do what it's supposed to do.
@ 2010 Apr 25 11:33 AM, entry [ apache_mod_substitute ] was posted in [ linux ]
Alsa Microphone
While trying to install Skype on RedHat Linux, I discovered that although I thought that sound was running fine on my CentOS 5.3 box, in reality it was only doing half the job. Playing music was fine, but capturing sound from a microphone had never properly worked. Both "/usr/bin/system-config-soundcard" and "gnome-sound-properties" didn't show anything obvious (Autodetect for the three playback options and ALSA for the sound capture), but clicking the last test button didn't result in hearing the sound of the microphone on my headset.
Another way to test this is with:
# arecord -d 5 -f cd -t wav foobar.wav # aplay foobar.wav
and in my case, I didn't hear anything while doing the playback.
It took lots of googling, but here is what you need to do. First of all, you can have the problem that your system has audio ports both on the front and the back of the system. For speakers or headsets you just plug it in anywhere and it works. Typically the front plug will disable what's plugged into the back. But for mics those two ports are not the same.
# /usr/bin/amixer ... Simple mixer control 'Mic Select',0 Capabilities: enum Items: 'Mic1' 'Mic2' Item0: 'Mic1' ...
On my system Mic1 was the input on the back of my PC, so I had to switch to Mic2 to use my headset that is plugged into the front. To change, use "/usr/bin/alsamixer", which opens a clumsy GUI, then click 'right' to get to "Mic Select" and use 'up'/'down' to change the Mic. Press 'Esc' to quit.
But that's not enough. Let's go back to our 'amixer' output.
# /usr/bin/amixer ... Simple mixer control 'Mic',0 Capabilities: pvolume pvolume-joined pswitch pswitch-joined cswitch cswitch-exclusive Capture exclusive group: 0 Playback channels: Mono Capture channels: Front Left - Front Right Limits: Playback 0 - 31 Mono: Playback 22 [71%] [-1.50dB] [on] Front Left: Capture [on] Front Right: Capture [on] Simple mixer control 'Mic Boost (+20dB)',0 Capabilities: pswitch pswitch-joined Playback channels: Mono Mono: Playback [on] ...
When I was running this the first time, it showed "Front Left/Right: Capture [off]". That's no good!! But how to change, because my alsamixer didn't show this as an option that can be changed. The following command will do the job:
# amixer sset Mic Capture cap
After that, maybe you should go back to alsamixer and switch the "Mic Boost" option on. You do that by moving right with the cursor and then press the 'M' key. Another option further right is "Mono Out", which can be 'Mic' or 'Mix'. I'm not 100% sure what it should be, I settled for 'Mix'.
Finally, while you're doing all this, you should keep your 'Volume Control' window open. For example, playing test sounds with the soundcard config tool will mute the microphone. And in other cases the level gets automagically :-) set back to zero. You better watch out.
This was for me just the prologue for installing Skype. The best way to do that successfully I found on this Hackery blog. Here is the summary.
# the skype binaries are 32-bit, so if you're running a 64-bit system, you # need to make sure you have various 32-bit libraries installed in parallel yum install glib2.i386 qt4.i386 zlib.i386 alsa-lib.i386 libX11.i386 \ libXv.i386 libXScrnSaver.i386 # installing to /opt cd /tmp wget http://www.skype.com/go/getskype-linux-beta-static cd /opt tar jxvf /tmp/skype_static-2.1.0.47.tar.bz2 ln -s skype_static-2.1.0.47 skype # setup some symlinks (the first is required for sounds to work) ln -s /opt/skype /usr/share/skype ln -s /opt/skype/skype /usr/bin/skype
Other good tips can be found on this CentOS HowTo webpage.
@ 2010 Apr 07 02:13 PM, entry [ alsa_microphone ] was posted in [ linux ]
Yum Local Install
Yesterday I was trying to configure a music player on an old laptop running Fedora 8. It required some additional packages for mp3, but when I tried to do a "yum install gstreamer-plugins-ugly" it appeared that this old stuff didn't exist anymore in any of the on-line repositories.
Luckily, pbone.net came to the rescue. I downloaded the RPM package, then tried to install, but there were a whole slew of dependencies that needed to be resolved first. Damn .... normally it means that one-by-one you've to search the net to figure out which RPM includes that specific shared library and then the result is often even more unresolved dependencies.
Don't know why, but I checked the man-page for yum and found that there is a "localinstall" command. What it does is that you use yum to install an already downloaded rpm package, but it will then use the on-line repositories to resolve the dependencies. Kind of "best of both worlds" approach.
# yum localinstall gstreamer-plugins-ugly-0.10.8-1.fc8.i386.rpm
Of course you have to hope that not too many of the other packages you need has the same issue of having disappeared from the repositories. But luck was on my side yesterday. Couple of minutes later I had my Muine Music Player running and the MP3 music was flowing out of the speakers.
@ 2010 Mar 21 11:04 AM, entry [ yum_local_install ] was posted in [ linux ]
Bizarre ftp Behaviour
After spending couple of hours editing a webpage, I lost all that effort when copying the file to my webserver. And of course before the automatic backup had been made. :-(
As always, there is a lesson to be learned.
Try this yourself:
$ ftp webserver ftp> prompt Interactive mode off. ftp> mput * local: img_2.gif remote: img_2.gif 227 Entering Passive Mode (192,168,32,72,126,33) 150 Opening BINARY mode data connection for img_2.gif. 226 Transfer complete. 12980 bytes sent in 4.2e-05 seconds (3e+05 Kbytes/s) local: img_3.jpg remote: img_3.jpg 227 Entering Passive Mode (192,168,32,72,126,33) 150 Opening BINARY mode data connection for img_3.jpg. 226 Transfer complete. 28488 bytes sent in 0.017 seconds (1.6e+03 Kbytes/s) local: webpage.html remote: webpage.html 227 Entering Passive Mode (192,168,32,72,164,239) 150 Opening BINARY mode data connection for webpage.html. 226 Transfer complete. 12498 bytes sent in 4.5e-05 seconds (2.7e+05 Kbytes/s) ftp> ls * 227 Entering Passive Mode (192,168,32,72,219,226) 150 Opening ASCII mode data connection for /bin/ls. total 268 -rw-r--r-- 1 wwwillem wwwillem 12980 May 20 09:08 img_2.gif -rw-r--r-- 1 wwwillem wwwillem 28488 May 20 09:08 img_3.jpg -rw-r--r-- 1 wwwillem wwwillem 12498 May 20 09:08 webpage.html 226 Transfer complete. ftp> ls -l 227 Entering Passive Mode (192,168,32,72,207,63) 150 Opening ASCII mode data connection for /bin/ls. total 268 -rw-r--r-- 1 wwwillem wwwillem 12980 May 20 09:08 img_2.gif -rw-r--r-- 1 wwwillem wwwillem 28488 May 20 09:08 img_3.jpg -rw-r--r-- 1 wwwillem wwwillem 12498 May 20 09:08 webpage.html 226 Transfer complete.
So far all has gone well. Notice the file-size of webpage.html, 12 kB.
Now comes the problem:
ftp> ls -l *html 227 Entering Passive Mode (192,168,32,72,142,146) 150 Opening ASCII mode data connection for /bin/ls. 226 Transfer complete. ftp> quit 221-Thank you for using the FTP service on webserver. 221 Goodbye. $ ls -l webpage.html total 62 -rw-r--r-- 1 willem willem 209 May 20 09:07 webpage.html
Notice how the file has shrunk from 12 kB to only couple of hundred bytes? Let's have a look at the (new) content.
$ cat webpage.html total 123 -rw-r--r-- 1 willem willem 12980 May 20 09:01 img_2.gif -rw-r--r-- 1 willem willem 28488 May 20 09:01 img_3.jpg -rw-r--r-- 1 willem willem 209 May 20 09:07 webpage.html
What happened here? The 'ls *' ftp command was ok, same for 'ls -l'. However when we did 'ls -l *html', the result of the 'ls -l' command was written to the local html file, in this case webpage.html. That's definitely not what I would have expected to happen. Very weird behaviour!!
Finally, the client was ftp on CentOS 5 and the server was an ancient RedHat 6.2 server. And no, 6.2 is not the successor of EL5. :-)
@ 2009 May 20 08:42 AM, entry [ bizarre_ftp_behaviour ] was posted in [ linux ]
Apache Virtual Host
If you're like me and over the years you've collected a couple of domain names because at the time they were "way too cool", or when you're serving out of your basement the vacation pictures of a couple of friends, then Apache's "Virtual Host" is what you're looking for. What this web server option allows you to do is to have a single web server handle the traffic for multiple domains.
For this example let's assume you got (long time ago, when still available :) the domain name "www.myself.com". And now you claimed for your buddy the domain "www.myfriend.com". And let's also assume that you either have a static IP (probably not) or that your ISP is changing their DHCP leases very infrequently (probably yes). So you've managed to configure (with your registrar) your domain name to point to your current IP address '12.34.56.70'. Now go to the registrars web site for "myfriend.com" and (in the DNS section) let it point similarly to '12.34.56.70'.
In the following I'm configuring an Apache server on a Cobalt Qube. However, this works exactly the same on any other Apache server (on Linux). To start, create with the Cobalt's web interface a new user called 'friend'. If that worked as it should, you will be able to browse to your buddy's website by pointing your browser at http://12.34.56.70/~friend/. Probably you will just see the boring standard template page. To help with debugging create a different webpage by editing "/home/users/friend/web/index.html".
Now we're going to edit '/etc/httpd/conf/httpd.conf' (better save a copy first!!). At the bottom there is a VirtualHost section, but (at least with a Cobalt) it's all commented out. Leave that as it is, but add the following seven lines:
# VirtualHost: Allows the daemon to respond to requests for more than one # server address, if your server machine is configured to accept IP packets # for multiple addresses. This can be accomplished with the ifconfig # alias flag, or through kernel patches like VIF. <VirtualHost www.myfriend.com> ServerName myfriend.com ServerAlias www.myfriend.com ServerAdmin my.friend@his.email.com DocumentRoot /home/users/friend/web TransferLog logs/myfriend-access_log ErrorLog logs/myfriend-error_log </VirtualHost>
The only important piece here is the 'DocumentRoot' parameter. You can point it to any place in your directory tree, but to make it the same as '.../~friend' does make sense. On some other platforms this will be '/home/friend/public_html/'. If you omit the two lines for logfiles, the logs for your friend will go to the same files as the rest of the web-server.
Now restart the web-server with "/etc/rc.d/init.d/httpd restart" (on some other platforms this is "/etc/init.d/apache restart" or similar variants) or simply restart your computer. That done, point your browser to http://www.myfriend.com/ and you should see the page that you edited as '/home/users/friend/web/index.html'.
OK, you're ready to become a mini ISP. But be aware, if you've a standard residential fast Internet line, your ISP probably doesn't allow you to have a web-server in your basement!! On the other hand, they probably don't mind, as long as you keep it small-and-personal. Just a couple of pictures showing off the new house or baby, and probably nobody cares.
@ 2006 Jun 03 09:11 PM, entry [ apache_virtual_host ] was posted in [ linux ]
Shell Hell
Unix shell programming appears to be an art, not a science. That's probably also why I'm not so much of a Perl fan. Take note, I do like both scripting and art a lot, but when it comes to Computer Science (mind that last word :) I think computer languages should be very unambiguous and easy to read. If that makes a compiler's life a little harder or means a bit more typing for the programmer, so be it....
But this story is not about bashing Perl, it's about unexpected problems with simple shell scripting. I was installing W3C's libwww package to get going with some XML-RPC development. It was the typical process: "./configure" followed by "gmake", nothing fancy. The configure step went fine, but while gmaking, it hicked up severely. I traced it back to some "test" commands in the libtool script, when the condition of 'test' had an empty variable.
Something like:
#!/bin/sh
something=""
if test $something = "yes"
then
echo "something is yes"
fi
Or when the something="" statement had been absent at all. In both cases, the result is an error at the "if test" line.
The cause of all this trouble is that for undefined variables there is a difference between
if test $something = "yes"; then
and
if test "$something" = "yes"; then
Mind the additional quotes. The libwww package is missing those a couple of times, which appears to be fine when the variable has a value, but not when it is undefined.
Back to art vs science. That things like this are illegal syntax I'm OK with. But it should be made clear up front. Preferably as a compile error, although I realize that for scripting languages that is not too applicable. I had hoped for an error like "line 13: illegal syntax" or something similar. But that's not what I got:
./test.sh: test: argument expected
So it took me 10 mins and many more echo statements to find out which of all those test commands in the 5000+ lines of code script was the culprit. After finding it, of course the problem repeated itself, quite a couple of times. It took a while to get this script fixed, but don't worry, I did ... :-)
Later, when redoing the install on RedHat, I discovered RH8 is having the same issues with the test command, but it states:
./test.sh: line 13: test: =: unary operator expected
The message is even more cryptic, but it has the big advantage of giving you a line number. It's the small things that matter :-). Which would have saved me many echo statements and a couple of hours.
With all these problems out of the way, I could dig into XMLRPC-C to build an interface between a Windows client and a Solaris backend. So far, I find XML-RPC to be an elegant protocol, which results in quick implementations and simple solutions.
@ 2006 Apr 16 08:58 PM, entry [ shell_hell ] was posted in [ linux ]