sakuramboo.com

Let’s face it, upgrading a distribution sucks. In my entire career in Linux, I have had to upgrade my distribution of choice at least once (usually because I’m changing distributions and upgrades are done every 6 months). Out of each and every distribution upgrade I have done, every single one broke my system. Not always to the point that I lost all my data, that only happened maybe once or twice. But, stability and speed usually would take a nose dive every time. There is a simple and easy solution to this that I want to go over.

Installation.

That’s right, do not upgrade, just install the latest version. This will give you a fresh install every 6 months and if your system is set up just right, will seem more like an upgrade because none of your data or settings will change. First, I need to go over partitioning.

If you are running your operating system on one sole partition, you need to be shot. There are so many benefits to partitioning your hard drive that it only makes sense to. Let me go over some of the benefits.

1. Speed. There is a slight increase in seek times if there is a smaller partition that needs to be scanned for a file.

2. Security. Easily specify which partitions should have noexec permissions or read-only file systems mounted, cutting down on a lot of maliciousness.

3. Organization. Knowing where everything is and should be makes life a whole lot easier.

And now the list of down sides to partitioning.

1. Takes slightly more time to set up.

Okay, now that is out of the way, next up is what to do with your brand new hard drive. Think of it in this manner.

[Operating system]|[User space]

Now, those two should be jailed off from each other. This will make sure that all user data will stay in one location, untouched by any system operations (unless specified to). Now, within the [Operating system] space, there are more splits needed for things like SWAP space, logs and tmp files. Here is my current partition scheme on my main desktop.

[~]$ df -ht ext4
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda6              47G  7.2G   38G  17% /
/dev/sda5             4.7G  138M  4.4G   4% /tmp
/dev/sda1             958M   58M  851M   7% /boot
/dev/sda8             861G  360G  457G  45% /home

Now, the thing you have to see is, I have 4 partitions on my hard drive. Really, for my main desktop, that is all I need. Now, of course, if this was a production server, well, it wouldn’t be a production server. But, I will go over a little bit on this.

/home is my home directory. This is where all my user data and user installed games and applications go. By “user installed” I am talking about games and programs which had a binary already in the tarball or zip and did not require installation into /usr to run. This partition will remain untouched by our installation process so we will never have to deal with it.

/boot is always supposed to be the first partition and placed in the beginning, I don’t feel like I should have to explain why.

/tmp is, of course, the temp directory. It has it’s own partition because of security reasons (remember I mentioned that before?). This partition will get mounted with noexec at boot time. This will prevent someone from running arbitrary code from there.

And the last partition is /. Nothing else really needs to be said about that. (I could have put /var/log on its own partition, but, like I said, this is a desktop, not a server.) The only other partition that is not displayed in the output of df is the SWAP partition. And that is because, really, you shouldn’t have direct access to that.

Now, out of those 4 partitions, there is only one that will ever get touched when doing an upgrade, that is the root(/) partition. All of your settings and configurations are stored in ~/.config (for the most part). So, even if you uninstall an application, the users configuration file will still be there, just in case if they decide to reinstall it later.

Now, when it comes time to upgrade your distribution, you just need to download the ISO, burn it to CD, boot off it and when it gets to the part of the installation where it asks to partition your hard drive, you select to do it manually and all you have to do is tell it to format /dev/sda6, set its mount point to root(/) and tell it to mount the other partitions without touching them. And now, you have the latest version of your distribution and all of your data is still in tack. But, there are some other things that will require you truly get your system back to the way it was. And for this, it requires a lot of leg work early on, but makes it so much more simpler later on. You need to create a bash script to handle downloading and installing all third party repositories and installing all other applications that were installed after the initial installation. I will show you my script to give you an example of what I had to do.

# Install packages from Ubuntu
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install -y a7xpg a7xpg-data abuse abuse-lib abuse-sfx audacity audacity-data eclipse eclipse-jdt eclipse-pde eclipse-platform eclipse-platform-data eclipse-plugin-cvs eclipse-rcp gimp gimp-data gimp-data-extras gunroar gunroar-data hwinfo kobodeluxe kobodeluxe-data moc mplayer mu-cade mu-cade-data nautilus-open-terminal noiz2sa noiz2sa-data parsec47 parsec47-data pavucontrol pidgin pidgin-data pidgin-libnotify rrootage rrootage-data sound-juicer titanion titanion-data torus-trooper torus-trooper-data ubuntu-restricted-extras val-and-rick val-and-rick-data xchat xchat-common virtualbox-ose virtualbox-guest-additions thunderbird chromium-browser build-essential dpkg-dev checkinstall xz-utils libsdl1.2-dev cmake compizconfig-settings-manager php5-cli
if [ $? != 0 ]; then
echo "The Install failed!"
else
echo "The install finished. Moving on to step two."
fi
# Add Wine PPA and install
sudo add-apt-repository ppa:ubuntu-wine/ppa
if [ $? != 0 ]; then
echo "Could not add the Wine ppa!"
else
echo "Added the Wine ppa."
fi
sudo apt-get update
sudo apt-get install -y wine
if [ $? != 0 ]; then
echo "Installation of Wine failed!"
else
echo "Installed Wine."
fi # Add PlayDeb and install games
sudo echo "deb http://archive.getdeb.net/ubuntu lucid-getdeb games" | sudo tee /etc/apt/sources.list.d/getdeb-games.list
if [ $? != 0 ]; then
echo "Failed installing the playdeb repo!"
else
echo "Added GetDeb Repository"
fi
wget -q -O- http://archive.getdeb.net/getdeb-archive.key | sudo apt-key add -
if [ $? != 0 ]; then
echo "Failed to install the PGP key for Playdeb!"
else
echo "Installed the PGP key for Playdeb."
fi
sudo apt-get update
sudo apt-get install -y astromenace astromenace-data vavoom autodownloader nazghul nazghul-data soulfu soulfu-data violetland violetland-data warzone2100 warzone2100-data urbanterror urbanterror-data teeworlds teeworlds-data
if [ $? != 0 ]; then
echo "Failed to install the games!"
else
echo "Installed the games."
fi

Yes, I did put checks at almost every step of the script and there is one part that I will have to manually edit every time, but once I change the location of the getdeb repository, all I have to do is run that script and everything I installed will be reinstalled on the new version. The only problem is, writing this script took a really long time all because I had to generate the list of applications AFTER having them all installed already. Here is an easy method if this is the first time you installed Ubuntu. After your fresh install, run the following command…

This will place a list of all packages installed. Then, when you are ready for the upgrade, run the same command to a different file, then use diff to make a list of all names that are in the second list but not in the first to get a complete list of all programs you installed after the initial installation.

This will produce an output something like this.

The “7a8,10” can be omitted as well as the greater-than symbols. But, there you have it. Another method to handling distribution upgrades. Like I said, yes, this is a slight pain in the ass to set up, but once you are done, it makes life all that much better.

Recently, there was an article that came out that stated that Linux has a higher ratio of spam computers running Linux than Windows. Even though the ratio is higher, to which the ratio is always supposed to be looked at as appose to the total number, there is a reason for this that many people are not bothering to look at.

If we were to look at the actual number of desktops verse servers, the ratio of Linux based desktops would be close to non-existent like the OSX counter part. Security on the desktop in Windows is a joke. Not that it can’t be done. It very well can be done right and made secure enough to never get any piece of malware ever. But, the problem with this is, almost every Windows user knows close to nothing of computer security. They, for the most part, run as administrator and will run just about any unsigned binary just because they want some free piece of software to copy DVDs or to illegally download games and other pieces of commercial-ware. Malware writers know that and they take advantage of this. Now, can this be done in Linux? Sure. But Linux does not have the market share in the desktop market for them to actually gain anything substantial.

One of the articles I read said that one of the reasons why the ratio is higher is because of the fact that many ISPs run Linux mail servers and that will act like a proxy when sending out spam from someones infected Windows desktop. To a point, this makes a bit of sense when explained, but this is the wrong reason.

The real reason why the Linux ratio is higher is because of the server market. Spammers require two things to be considered successful, high bandwidth and high uptime. That is the definition of a Linux server. This is further multiplied because a lot of Linux administrators think that because they are running Linux that they are secured by default. This is one of the biggest reasons why Linux servers are highly attacked and become infected with spam servers.

The original article posted by MessageLabs also hinted that the reason for the higher Linux ratio is because of the ISP mail redirect. So, let us look at this logically.

1. Windows makes up the largest number of spammers.
2. Linux has a higher ratio as seen in the mail headers by the received field in the mail headers.
3. This means that many of the computers behind the Linux email redirects could, in fact, be Windows based.
4. This means that the Windows ratio is actually much higher and the Linux ratio is much lower.
5. Many of the Linux numbers are actually Linux-based servers and not desktops.
6. Email traffic was analyzed but the original sender was not.
7. They used desktop market share only.
8. Server traffic was included in the article but was not included in the market share.
9. This article is flawed.

So, in a nut shell, MessageLabs are posting articles that are bogus.

Ubuntu is owned by a company and not by the community. This is the statement that a lot of Ubuntu lovers do not seem to understand. The underlined plan of Canonical is to make money, not to make a free operating system. This means that even if there is something that the community as a whole does not like doing, if it means the company will make money, guess what? The community is out of luck and will have to deal with it.

There have been a few changes in Ubuntu that the community seemed to get all up in arms about recently, that to me, makes perfect sense. One of the changes is the placement of the window buttons, moving from the right side to the left side. All that was commented about was that there is a plan for this. Yet, the community seemed to get all bent out of shape about this. Who knows what the ultimate goal of this is? Maybe there is some new button or feature that is to be placed on the upper right corner of the windows. Maybe there is supposed to be a new status button that is supposed to go there. Maybe there is supposed to be a new window tabbing feature that is going there. Maybe that tabbing feature was originally a Gnome extension that defaulted to the right side and to incorporate it, it would be easier to just move the window buttons than to recode the extension. Who knows. How about instead of bitching and complaining, just sit back and see what will happen.

Another problem that people seem to be having is how Ubuntu will support the 7digital to provide an online music store for fast and easy music purchases and downloads. The problem is not with the store, it is with the music only being made available in mp3 format. I’m sorry to burst peoples bubbles here, but Canonical does not really have a say in the matter. If you have a problem with it, take it up with 7digital. But, there was a comment made Canonical will be trying to find a way to offer better quality compression and possibly file formats at a later date.

Personally, I still say, buy the CD and rip it yourself. Odds are, if 7digital were to start offering alternate file formats, they will most likely just convert all their mp3s into ogg or flac and say “Here you go.” Which, for anyone who deals with audio, knows that is not a good way to do it. In fact, converting file formats will degrade the quality of the file over time.

The thing is, Ubuntu back in the day is not the same Ubuntu of today. When Ubuntu first started out, it was looked at as a fully community based distribution. This worked for the longest time. But then, something happened, Canonical realized that if they are dumping all of their money into it, they might as well make some money in return from it. And this means that Ubuntu is now owned by Canonical, as apposed to just financially supporting them. And with ownership means a removal of a community voice.

That is just the way it is.

I have a problem with Ubuntu Studio. There, I said it. I think that Ubuntu Studio is a wonderful distribution, packed with all the programs needed for my music composing hobby. But, there is a problem. Ubuntu Studio follows the same exact release schedule as Ubuntu, meaning, when there is a new version of Ubuntu, there is a new version of Ubuntu Studio. This means that if you want the latest software updates and latest programs, you must do a dist-upgrade every 6 months. Why is that? With an audio workstation, the last thing I want to worry about is wondering if my upgrades will take, that all my software upgrades flawlessly and that all of my data and projects will continue to work.

A workstation should not be subject to upgrades every 6 months. Each upgrade will force the user to run a risk of something not working properly. To give you an example, when 9.04 shipped, there was a bug in the real-time kernel that made the Ubuntu Studio maintainers not include the rt kernel. They recommended using 8.10 if you required the rt kernel. If you were using 8.10, you could do the upgrade to 9.04, but just keep on using the older rt kernel. Then, you would just have to wait until 9.10 comes out to get the updated rt kernel or compile it yourself. And this is just a problem with the kernel, let alone any number of the various other bugs introduced from doing a distribution upgrade.

If Ubuntu Studio wants to be thought of as a leading operating system for media creation, they should follow only the LTS releases of Ubuntu. This will make sure that the core of Ubuntu Studio will be secure and stable. The packages should then be offered in a rolling distribution fashion, meaning, instead of back porting only security and major bug fixes, just release the newer version. I honestly do not understand the difference of grabbing the latest source tar ball and packaging that or patching the older version and recompile that. Both will produce the same output, only difference is by patching, you can maintain the same version number.

If anything, I would suggest that Ubuntu Studio should treat their operating system maintenance in the same fashion as Suse by providing service packs that will update the core of the operating system, meanwhile the packages are constantly updated to stay current. When a new LTS comes out, treat that as a whole new version of the operating system.

I bring this up because I have Ubuntu Studio 9.04 installed on a second hard drive that was upgraded from 8.10. Now that 9.10 is released, the only reason why I would want to do the upgrade is to get the latest kernel, which I would have to recompile the ALSA modules anyway after installation. Why should I go through all of this work? I want to write music, not be an administrator of my workstation.

This was a problem I faced when I upgraded to the latest Ubuntu. It was not Ubuntu’s fault, but rather, it was my own fault. The problem came from when I tried to load my playlists into my audio player. Let me explain the problem and how it happened.

Originally, I had my music files located on a separate partition that was mounted in /home/sakuramboo/Media and from there I would access the Music directory. When I got my new 1TB hard drive, I copied all my music from /home/sakuramboo/Media/Music to /home/sakuramboo/Music and because of that, I would need to update the few playlists that I have. The good news is, there are not that many playlists that I had. If I wanted to, I could just use mocp to create the playlists just fine, but I felt like getting a little creative, plus, I wasn’t using the terminal in a while, so I figured I should give it a go. The command I use was a rather simple one liner.

cat playlist.m3u | sed 's/\/Media//g' > new_list.m3u

That is all there is to it. A brief idea on what’s going on. First, we cat the playlist file which just displays the contents of the file. With the pipe (|) we are passing that output to the next program which is sed. What we are doing in sed is, we are going to switch “/Media” with “” (nothing). The ‘g’ at the end just tells it to do it at every instance instead of just the first match. Then, the output of that goes to a new file called new_list.m3u.

Enjoy.

Now that Ubuntu 9.10 is finally released and everyone has finished praising/pandering it, I felt that I should talk about what happened with me, and boy, what a story this is.

It all started when my update manager notified me that a new version of Ubuntu is available. So, naturally, I would wait a few weeks before doing the upgrade, just to make sure that I would have a pretty good connection to the servers. This time, I decided to jump right into it. The download was the longest part of the whole ordeal, which lasted about 12 hours. But, once the download was finished, the update was pretty painless. A few minutes later and a reboot, I was starting up in 9.10. Or, was I?

I have multiple hard drives in my desktop. I had one with Ubuntu 9.04, another one with UbuntuStudio 9.04 and the third was a media storage hard drive shared between the two. This is where the mix up came from. I was using the second hard drive’s master boot record to boot into the first hard drives OS. When the update to 9.10 finished, I swapped the boot order so that I was using the first hard drive exclusively, but Grub did not update correctly. I was booting into an older kernel with a newer OS on the wrong hard drive. Needless to say, nothing was working. So, after I fixed that problem, I was booting right into 9.10 without a hitch. But, there is more.

There is a new hard drive SMART reader that monitors the life of your drives. Once 9.10 booted, I was notified that my primary hard drive has many bad sectors and that I should replace it soon. The current count was around 32 bad sectors. I can live with 32 bad sectors. But, then the slowdowns started happening. Not in the OS or in Gnome or anything, but in games. Frame rates dropped by more than half. Then there was the sound issue. Music would play just fine, but when I would play a game, the sound became very distorted and filled with static. So, I figured, I’ll just buy a new hard drive, do a fresh install and be done with it.

Now, I need to point that out that I screwed up the initial install of Ubuntu by not partitioning the hard drive properly. If I would have partitioned the home directory, I could have just done a fresh install on the current hard drive and still keep all of my configurations. But, like an idiot, I figured I would just click right through the install. Now, I’m paying the price.

So, I decided on ordering up a brand new 1TB Western Digital hard drive. The price was really great, about 80 bucks. But, when it finally arrived, I slapped it into my desktop, booted into the live CD version of Ubuntu and noticed the SMART manager telling me of a bad hard drive. Right away, I just pushed it off, thinking it was telling me of the current bad one with 32 bad sectors. But, I decided to look at it anyway. The brand new 1TB WD hard drive had a reported 1265 bad sectors and I was told that it will fail any minute. I thought it was maybe a bug in the reporting, so I installed Ubuntu anyway. That was easy, but when it booted up, it worked for about 10 minutes, then dmesg got flooded with I/O errors on /dev/sda. It got flooded to the point that nothing was working. I would go to launch something and got error messages popping up. So, needless to say, I had to send the drive back.

Got the hard drive replaced and installed 9.10 yet again. This time, everything worked and I was just amazed at how well it worked. I tried playing a game, frame rate was top notch. Music worked perfectly. I added all the old software from before, video played fine, web browsing, flash, all worked. But, then I noticed the sound problem that I mentioned before was not fixed. So, back to Google I went. After a while of searching, I found that some other people were experiencing the same exact problem. However, no one was using the same soundcard as I was. I have a Creative SoundBlaster Live Audigy Platinum soundcard. The soundcard that everyone was reporting problems with was either the Intel HDA or the Nvidia HDA chipsets. There were many solutions that I came across, all of which I did not want to do because they involved removing Pulseaudio. Even though I am not a fan of Pulseaudio, I do think that it is best to keep my system as true to what they recommend as possible, for stability reasons. I finally came across a solution that required upgrading ALSA. It turns out that Ubuntu 9.10 shipped with a buggy version of ALSA and no one noticed until AFTER it shipped. Good thing is, I found someone on Ubuntuforums.org that decided to write a shell script to handle the process of downloading, compiling and installing the latest ALSA drive and its dependencies. After running the script and waiting for everything to run its course, I rebooted the machine and now Betsy is running at top notch.

There was also one minor bug that I came across that will be a slight pain in the ass for me to fix. I was copying all of my music from my media hard drive on to my new 1TB hard drive. I figured, I have the space, might as well use it and get rid of the extra hard drive from my system. The copying finally finished and now I’m listening to my music, I go to one of my Duke Ellington albums and then noticed that there was a song that played twice. I checked the list and there were no duplicate entries. It turned out that during the copy, the same song was copied twice but was renamed. So, now I have to find the song and copy over the right one. But, the problem is, I don’t know how many other times that happened. Now I have the arduous task of checking all of my music.

All in all, I must say that Ubuntu is growing into quite the operating system and that everyone really should watch it. I’m not saying that you need to use it, but just watch out. I have a feeling that you will be seeing Ubuntu loaded computers in the stores soon.

Why must an OS upgrade take over 10 hours? That is the question I must ask here. I know, I know, it has to do with the fact that everyone and their mothers who are running Ubuntu all decide to upgrade at the same time (the moment it is released). That, their network gets over saturated and it just can not handle the massive number of requests at one time. So, instead of dropping connections, they try to throttle down everyone. I understand all of this. But, the real question here is, why must I have to upgrade every single package, especially when the vast majority of the larger packages only have a version name change and nothing more?

I truly do not understand why, let’s say, Nexuiz needs to be redownloaded if the version is exactly the same, just to make it compatible with the latest version of Ubuntu. This has been a problem in Linux since I can remember, and I got started in Linux back in ‘02 with Slackware 10 and Fedora Core 4. The way I see it, the problem rests with the package maintainers and how the OS packages are handled.

The way I see things, there should be the underlined OS. This would consist of the kernel, libraries and BASH. We can throw in there some other goodies like X server, but it would be better handled another way. Now, everything else are to be considered third party packages. With third party packages, they only change when there is a version upgrade. If there is no version upgrade, there is no need to redownload/reinstall the package.

There is also another problem that I see happening with the way package maintenance is handled. Let us say that there is a specific package whose maintainer decided to quit and no one picked up the now opened position. What happens to that package? It just gets removed from the supported packages list. But, if that package was never going to see an upgrade (let us say it was a final version of a game that the developer decided to hang up the towel on, but still built packages), why must the package maintainer now have to rebuild the package every time the OS gets an upgrade? It is just extra work which should be spent somewhere else.

Now, before I get some flack for this, yes, I do know that there are third party packages that operate in this manner. Kudos to them. It makes handling upgrades all that much easier. Also, I understand the concept of backports. But, if packaging is handled the way I described it, there wouldn’t be a need to backport anything. Again, just extra work for the maintainers.

There are many things that make up a good Linux system administrator. There is no way that I can go over every aspect of that in this little blog. Hell, that’s why there are whole books and series dedicated to teaching good ethics for system administration. What I do want to point out, however, is one very vital part that not that many admins realize and that there seriously needs to be a lot of work put into making it easier for us to do our job.

Package management is very vital to a stable system. It makes installing/updating/removing programs and services easy. But there are inherent problems with using pre-made packages. The biggest problem is, you are now tied to when your repository updates the package. If there is a security advisory that comes out with a patch, it might take days, weeks, even months for the package maintainer for your distribution to actually create the package and send it upstream to the repository. This is no good. This is also the reason why a quality admin will build their own packages.

Now, there are two main ways to install something in Linux. You can install the package or you can compile from source. If you are building your own packages, you are doing both. This is not an explanation of each package manager, but rather, the tools available to help build your packages.

The first is with Debian based distributions. The utility is call checkinstall. With this utility, you specify the folder which has the source code, specify the dependencies, any build flags. For RedHat based distributions, there is rpmbuild. This one is a little different in that you need to make spec files for each package you wish to make. This spec file contains all the same information that checkinstall asks for, but you need to manually make the spec file. Yes, it is a slight more pain in the ass, but once you have the spec file, you can just use that file every time you need to compile that program. It is the same as if you are using templates. For Slackware packages, there is makepkg. This is on the same lines as checkinstall but the difference is, you need to run ./configure before makepkg. This ensures that all the dependencies are met before compiling.

There is one other program that is not needed for building packages, but is another reason to. Patch will patch a piece of source code with anything that available. Without going too in depth with how software development is done, patch will apply new code to existing code automatically. Once patching is done, rebuild the package and update the program.

Now, start reading those man pages everyone.

So, Google wants to release their own distribution of Linux. Google Chrome OS seems to be the new hot topic right now. The thing that I seem to never see, though, is Chrome OS’s affect on Linux in general. A lot of blogs and news sites seem to be comparing Chrome OS and Windows 7 and what this means for Microsoft. I will put this to rest right now. This will only impact Microsoft by about 1-2% market share, end of story. However, this WILL affect Linux in general.

There have been various Linux distributions that have tried to go after Microsoft’s market share. The problem with them all is that they were affectively destroyed because even though they had a very solid operating system, third party companies did not want to deal with them. Canonical started to help out with Ubuntu, working on convincing more companies they should develop drivers for their hardware, supporting ISO standards and other things. They even got some hardware vendors to come out with laptops and netbooks with various Linux distributions. But, within a few months, it got harder and harder to find a netbook with Linux installed. There is a reason for this, Microsoft has put so much effort into making sure that those companies make sure that Linux doesn’t take off. Some of these tactics are things like supplying your local computer stores with ONLY Windows loaded computers, having them to make their website difficult to navigate to their Linux offered computers and the reason why things like this was harmful was because companies like Canonical and Redhat are not going to invest financially into going up against Microsoft.

Now that Google is going to attempt to tackle this, they are going to fail in comparison to Microsoft’s market share. But what WILL happen is, Google will be doing everything they can to make sure that third party hardware companies will offer products that work with Linux. We are going to start to see more wireless devices, more printers, more video cards all being supported natively in Linux now. Google has the financial backing to get more companies invested into Linux. With this, we are going to see the Linux kernel working on more and more platforms and hardware out of the box.

Even though one of the biggest myths right now about Linux is its lack of hardware support, after being debunked numerous times, people still think its true. With Google delivering their own distribution of Linux, we are going to see even less hardware incompatibilities and even more people finally realizing that Linux “just works.”