Again, apologies for the much delayed posting.  Finals together with the sluggishness of my backup computer made me unable to write this until now.

For those who don’t remember, we are in the middle of a series on what interesting developments and capabilities are ongoing within the Free Desktop to improve its fitness for both personal and enterprise usage.

Infrastructural Enhancements

This category of purely under-the-hood work is not immediately obvious and thus is often under-appreciated, but yields many useful improvements to the Free Desktop.

One notable entry here is automatic code parallelization in GCC (the main compiler on Linux) through Graphite, which should lead to significant performance improvements for all programs on computers with multiple processing cores.  Of course, the scope of what can be done is limited due to this sort of conversion requiring a higher-level view of what a block of code is meant to do, but nevertheless this is very much a welcome feature that will keep our beloved compiler in line with the current state of compiler technology.

Filesystems have always been an area of interest for the Free desktop; from the plain-spoken and default ext3 to the fast ReiserFS and SGI XFS, Linux especially has always had a variety of powerful filesystems available for any task. The march of progress has thankfully not left this critical job by the wayside.  ext4, the successor to ext3 featuring much improved performance through more flexible filesystem clustering through extents, and new features such as defragmentation support and checksumming of filesystem journals for increased reliability became stable in Linux 2.6.28.[22][23] Fedora 11 is going to include it included by default, which should come as either a pleasant or a terrifying prospect depending on one’s view on stability.[24] I’d say that I’m psyched, but I’m using it right now on my Arch Linux installation so such a statement would be misleading.

However, by the ext4 project leader’s (Theodore Ts’o) own admission, ext4 is not revolutionary enough.[25] The next major filesystem on the horizon is btrfs, designed to overcome many of the limitations of ext3 and ext4 in filesystem size and administration.[26] Among the interesting features it will bring will be transparent checksumming and compression for files, defragmentation and filesystem checking while the filesystem is mounted, and more sophisticated data striping.  If you’re keen to try it out, it should be included with Fedora 11, but be warned that the filesystem format is not yet stabilized or optimized and thus your data may be prematurely ended.[27]

Security

The infamous security framework soap opera is still going on, of course, with the usual politics and petty bickering within the Linux kernel development community.  Although the free desktop takes much well-deserved pride in its security, the Unix-like user-group-other security model (Discretionary Access Control, or DAC) traditionally used for managing permissions has proven to be both insecure and distinctly un-Unix-like.  Not only was it crafted before any concept of wide networking had really been established, but it also far too permissive and course-grained in its management.  Hence the need for a new security system that builds on top of the traditional model; there are many competing modules alive in the ecosystem, but it remains to be seen which (if any; after all, FLOSS is all about choice) will prevail.  An important thing to be aware of within this realm is the LSM, or the Linux Security Modules; a common Mandatory Access Control (MAC) security framework API within the kernel.

POSIX File Capabilities, while not strictly a security framework, are a minimalistic approach to security available since Linux 2.6.24.  Basically, they use the extended attribute system of modern filesystems such as ext3, ext4, btrfs, etc. to allow a program to do tasks that would traditionally require root permissions.  Far short of being a clone of the classic SETUID bit that forces a program to be run as root (an over-simplification), it instead limits what a program can do to the minimum that is needed.  The implications of this are invasive and far-reaching; for instance, using file capabilities it is no longer necessary to be root in order to burn an optical disk.  LWN as always has a good read on the topic.

SELinux (Security Enhanced Linux) is another security framework, only instead of merely being an extra set of permission bits it is a full-blown MAC solution.  Developed by the National Security Agency in 2000, SELinux was the first major security framework available for Linux and is now used most notably by Red Hat and Fedora.  Although it is generally considered quite secure because it uses an inode-based system instead of filenames (avoiding issues of different restrictions referring to the same file), it is also notorious for being a nightmare to configure.[28]

AppArmor is the main competitor to SELinux, developed originally by Immunix but later released and maintained by Novell after they perchased the former.[29] It is currently used by most notably by OpenSUSE and Ubuntu, presumably due to its ease of configuration compared to SELinux.  However, it is considered less secure due to its reliance on filenames instead of filesystem objects, and thus there has been a great deal of controversy both petty and significant to the point that Linus Torvalds himself has had to step into the fray despite his self-proclaimed disinterest in security frameworks.[28] Like SELinux, it is a MAC security framework using LSM, but unlike it, AppArmor has failed to be merged into the mainline kernel tree due to both technical troubles and political infighting.[29]

SMACK (Simplified Mandatory Access Control Kernel) is a much more recent entry into the security game available since Linux 2.6.25 that promises drastically simpler configuration at the cost of less power and capabilities compared to SELinux.  In addition to eased configuration, it apparently also uses a hybrid approach to the inode/path debate.[30]

Lastly, TOMOYO is the latest entry to this… active… environment of security frameworks.  It is not yet available in a stable kernel release, but should be available in Linux 2.6.30.  Much like AppArmor, it uses pathnames as its basis for handling files, but among other things is designed to be more robust by working around weaknesses in LSM allowing more intellegent handling of file objects.

Recently the database at PHPBB.com was hacked, exposing the passwords of all 20,000 users on the popular site. Of course, PHPBB.com is mainly used as a trouble-shooting forum for the software itself. This basically means that most people registered there will probably be fairly tech-literate. However, you wouldn’t think so based on the red-alert list of passwords. The number one password was “123456″, followed by “password”, then “12345678″.  “1234″, and a word which would be best left unsaid to preserve the integrity of this site.

So, why such poor passwords? Well, people probably don’t feel that their account will ever be hacked (and in this case it was the whole forum that was), or at least worth hacking. However, you should always play it safe and go with a good password. Though for many people these key points will be common sense, I feel they are worth some ink:

1. Compose your password of upper and lowercase letters, numbers, and special characters (!@#$%^&*<>|{}).
2. If you are good at remembering, choose random strings (ek:yqEO*>#6hWb) or, if you (like myself) could never remember that, then go with l33t speak; replace letters with similar looking special characters and numbers. Also use a pattern to alternate lower and uppercase letters. Also make sure that the phrase you base it off is not common, like password (so don’t go with p@5sW()P,d).
3. Avoid using patterns on the keyboard, like !@#$%^&*(), 134567890, qwertyui, asdfghj, zxcvbn, qazwsxedc, and the list goes on.
4. Always make sure to use different passwords for different sites, just in case one of the logins is discovered.
5. If you have a problem remembering passwords, don’t store a list electronically. Instead, keep all logins on physical paper and locked in a safe. Certainly don’t use the key-under-the-mat trick, or, in this case, store your password on the back of your computer or under your keyboard.
6. Don’t tell anyone else your passwords. Obviously, there are people you can probably trust (like your best friend since junior high school), but still be careful.
7. Be extra careful with sites containing sensitive information, like bank accounts. To a lesser extent, sites like Facebook and Myspace should also be closely guarded.
8. Finally, change your password every so often.

Really, to sum things up, use common sense and you should be safe.

I haven’t been too excited about web apps. Sure, I use Gmail, Google Docs, Google Reader, and various other online tools, but I’m rather cautious about their use.

So, I log into my super secret mail URL, as I use Google Apps for Your Domain to check my email. I see that I’ve been sent an email from a friend containing a Word document.

To view the file, I click the View as HTML link that Gmail displays next to the attachment. In a new tab, the HTML rendition of the Word document is supposed to appear, but it doesn’t.

Not discouraged, I click Download Original Attachment, not knowing what is to come.

Click on the image to view it full-size

Yes, Google says that my account is now locked. Although I have planned for such a lockdown, Google denying my access to the account shocked me. Fortunately, the account was unlocked in a few minutes.

I repeated the test three times (not wanting to test it further due to Google possibly getting suspicious), and my account was locked down each time I tested it. I conclude that it is a bug within Gmail that set off the alarms, causing my account to be temporarily locked up.

The lesson? Depending on web applications to keep data secure or accessible is dangerous.

The sad part is that my story isn’t unique; many people have faced similar problems with a variety of web applications.

Remember, friends don’t let friends use web applications unsafely.

The web application in permanent beta is the latest fashion in today’s Internet world. Some folks believe that after several years of testing, that web applications ought to shed the beta tag and call themselves stable.

I disagree.

Open source desktop software is significantly more flexible than a closed source web application in terms of giving what users want. Desktop applications have extensibility through plugins, extensions, themes, and so forth. Web applications currently have only a weak extensibility through Greasemonkey.

Facebook Apps are closer to true extensibility, but Facebook remains in control over Facebook Apps, which results in rumors that Facebook is going to close down third party apps. Developers creating extensions of desktop applications usually do not have to worry about their extensions being wiped off the face of the Earth.

Also regarding Facebook, some users are not happy with Facebook’s transition to a new user interface. There is even a petition for the old user interface to return. I suggest you compare Facebook to WordPress. Nobody is going to force a blogger to upgrade to the latest version of WordPress, but there is little one can do if Facebook decides to switch to a different (and worse, in the mind of the user) UI.

Does my blog post ring a bell? Oh yeah, this blog post sounds roughly similar to Richard Stallman’s opinions on cloud computing. Stallman is dead right. One should only truly trust open source software on hardware within the ownership of the user.

So where does the whole “keep Web 2.0 apps in permanent beta” idea come into play? My point is that Web 2.0 apps ought to keep themselves in permanent beta as a reminder to users that no Web 2.0 app outside of the user’s control is as safe as an application within the user’s control.

Use Web 2.0 apps responsibly™.

Edit: The people on Reddit informed me that some of my questions go against the idea that premature optimization is a bad practice.

Introduction

“Listen. Easy now,” said the old man gently. “I know, I know. You’re afraid of making mistakes. Don’t be. Mistakes can be profited by. Man, when I was younger I shoved my ignorance in people’s faces. They beat me with sticks. By the time I was forty my blunt instruments had been honed to a fine cutting point for me. If you hide your ignorance, no one will hit you and you’ll never learn. [...]”

–Fahrenheit 451

Newbie programmers, when set to complete task, will often complete the task in the worst way possible. Of course, most programmers graduate beyond newbies. This post will detail some questions that newbies can ask in order to build better programs.

The below questions are things that all newbie programmers should be asking themselves so they can produce good programs. The questions can also be used as a very simple litmus test to see whether a programmer is a completely lost newbie or not. The Five Essential Phone-Screen Questions has a set of questions that help find good programmers.

<insert image here>

Photo credit: Flickr user anomalous4

When coming up with ideas for solutions

1) What patterns or algorithms can help me solve the problem?

Computers work well with numbers, and numbers are full of patterns and sequences. A simple example of a sequence is the Fibonacci sequence, where every number is the sum of the previous two numbers (1, 1, 2, 3, 5, 8, 13…).

A simple and useful algorithm is the Sieve of Eratosthenes, which is used to create lists of prime numbers.

2) Has somebody else already solved this problem or a similar problem?

Google called; they said they have the answer to your problem.

For those that can handle looking at the possibly messy code of others, looking online for open source code that can partly, mostly, or completely fill a need is a good idea.

A somewhat recent example is that I was looking for an open source Digg clone in PHP to make slight modifications to, and managed to find Pligg. Unfortunately, Pligg didn’t manage to fit my needs, but I managed to find PHPDug, which I like better.

3) What are potential issues I could run into when solving this problem?

It is important to think ahead about what issues could be faced when solving a problem.

For an important website, the power of the hardware, the power of the software, and the precautions taken for data loss and downtime, and more should be considered.

When looking at possible solutions

The first group of questions are things that newbie, or n00b, programmers should ask themselves when looking at a source code or a way to solve a problem.

1) Is this method fast as it can be?

Writing fast programs is a very important thing. Too many people are not patient enough for slow programs, and writing programs that can easily be made faster is not going to make users happy.

2) Is this method secure?

I sometimes wonder if it would be more beneficial to beginning developers Baller shouted “Security!” instead of “Developers!”.

Some common, but dangerous security mistakes that n00bs make are:

1) not escaping user input

2) not using file permissions properly

Chmodding everything is 777 is bad. Very bad.

3) using improper encryption to store things

3) Is there a simpler way of solving this problem?

If a person has two solutions to solve a problem, that person should always choose the simpler solution. The more complexity there is in a system, the more difficult it is to use and maintain that system.

4) Is this method scalable?

Creating software that works perfectly, with the exception it not being scalable at all, is setting up the programmer(s) for some major headaches down the road. A classic example is Twitter’s severe instability a few months ago.

Final notes

All programmers ought to subscribe to Coding Horror, a blog with far more useful content than you’ll ever find in this waste of disk space.

Also remember to ask questions on mailing lists, forums,  and so forth, but only after several hours of experimenting with Google keywords.

A couple days ago, I saw a couple FriendFeed messages linking to FriendFeed accounts that spam. To me, this was quite alarming. I love FriendFeed partly due to it’s excellent community, and the idea of another good community being destroyed by spammers wasn’t one that I wanted to think about.

Of course, not thinking about the problem is a pretty bad way of solving it. I just read Mike Fruchter’s post titled “Spam invades Friendfeed“, and it brought to my attention an idea that Robert Scoble had about controlling spam on FriendFeed. Scoble’s idea is to put a FriendFeed account into a jail if there are a certain number more blocks than subscriptions.

Of course, Robert Scoble’s idea, although interesting, can be improved upon. What if spammers create an enormous amount of accounts to all subscribe to each other in an attempt to balance the increasing number of the spammer accounts that were blocked?

The first thing to reduce this as a possibility is to ensure that blocks increase faster than subscriptions. Assuming that FriendFeed doesn’t already do this, the number of accounts created per IP per hour should be limited to a number that won’t be too much of a hassle for computers sharing an IP, but should slow down individual bots. Of course, various techniques could be used by spammers to get around the IP limit, but at least it would make the job more difficult for spammers and might encourage at least a few to go other places where spamming is easier.

Second, instead of seeing if there are a significant number more blocks than subscriptions, I would use a separate flag specifically reserved for spammers. On the user interface, this might look like, “Report this user as a spammer,” or some other phrase. The reason for a separate, spammer-only flag is that the normal blocking feature is currently used to generally hide all activity of certain users that others do not want to see. Highly well-known, but very disliked FriendFeed users might accidentally trigger the anti-spam mechanism because of a high number of blocks relative to subscriptions. For easy access as well as keeping the user interface clean, I would add a “Report spam” button to the box that users on FriendFeed see whenever they mouseover a link to a FriendFeed profile. I would change it from what it currently is,

and add the link as shown below.

I would suggest marking users as spammers if a significant number of comments are identical or contain URLs to the same websites, but that is easily defeated by posting slightly different variations of spam messages and using different URL shortening services to mask the URL.

Now, if you’re reading this, you are probably thinking that my ways to improve FriendFeed’s spam handling and reporting capabilities aren’t that good, you’re probably right. The problem with spam is that it’s so difficult to automatically handle. CAPTCHAs can be broken, the spam of one spammer come from multiple places (such as accounts or IPs), and a variety of other techniques can be defeated by spammers.

Since the most accurate way to make sure that spam comments are not displayed is moderation, I’ll briefly explain how you can delete FriendFeed comments on your entries.

So, how would you reduce the spam on FriendFeed?