Code Vault

My NDA Policy ~ I have decided to starting signing them! : Texas Startup Blog discusses Non-Disclosure Agreements and how the author has decided to react to it.

I think it is brilliant.

Also of interest are the other linked articles such as To NDA or Not to NDA? That is the question.

My stance on NDAs is that i only sign specific ones that actually relate to confidential information. And i only do that to make newer clients feel more at ease. Frankly, i have turned down some projects before because of this.

Honestly, if the strength of your business idea relies on the assumption that people won’t find out about it. Then I don’t think you have much of an idea. Certainly not one to base a business on.

I often brought in Gary Hoover, founder of Hoovers, BookStop, and TravelFest as well as author of Hoover’s Vision as a guest speaker. He says he talks to so many businesses and students that when someone asks him to sign an NDA he proceeds to ask them ‘Why?’ and immediately wonders about the viaObility of the concept.

This quote from the linked article shows that i am not the only one who thinks this way. And this quote comes from a serial entrepreneur!

One designer i was working with, after telling him that the project is funded by an angel investor, expressed interest in meeting the investor to tell him about all his “great ideas.” He pitched his greatest idea to me, and it was neither original nor did it have any revenue model(or even a future one). I do agree with the “1% inspiration and 99% perspiration” saying, and it is often the case that people who choose not to do the 99% part are the ones with the most “great ideas.”

Now, i didn’t even talk about the legal aspects of an NDA. But even with those issues aside, NDAs are no protection whatsoever. They are just plain stupid(using Linus’s term). I won’t go as far as saying “ideas are worthless,” but i will say that an idea without a proper implementation plan is.

Recently, a new client decided against using an NDA after i expressed that i am uncomfortable with generic NDAs. Certainly, a smart decision. I am very fortunate to have such clients.

I finally got around to releasing the BombMaker Gnome applet that i blogged about. I’ll share with you a little secret, I’ve actually been using it for the past 6 months or so. I even learned to package debian packages just so that i can release it.(manually installing gnome applets is a bit too much)

There were a couple of snags along the way, mainly in the packaging part and specifically in using autotools for that. I didn’t use autotools because i think they’re way too complex for something that should be relatively simple. Sadly, the complexity is necessary for autotools because it has to work on some many systems. Whether that complexity is still needed today, especially that the whole world has converged on a few key systems, is a question that is asked often.

So, I’ll blog about it sometime. But for now, you can visit the project page on the main site here. And as always, if anybody would like to work with me on this project, I’m just an email away.

Strange title, isn’t? I don’t actually make bombs although during the last war, a lot of people probably thought i did.

In my KDE days, i used to make a lot of use of an applet called KTeaTime(http://docs.kde.org/stable/en/kdetoys/kteatime/introduction.html#whats-kteatime). Its a nice little applet that tells you when your tea is ready after you select what type of tea you want. So if you select extra black tea for example, it will remind you that your tea is done after 10 mins for example. I used this when i wanted to make some hot drink for my coding sessions. So i would setup the drink and i will let it cool and use that to remind me when it would be done. Or when i would run a long compilation process that takes several minutes to finish, i just put it on another workspace and set this to remind me after a few minutes.

Bomb Maker is a gnome applet that tries to clone this with a twist. It is a reminder applet in which you choose a certain bomb type and it goes off after its time period elapses. You can also choose to manufacture a custom bomb type which has a custom time period till explosion and includes a custom message. Did you ever tell yourself that you will attend to a certain task in 10 mins and then forgot? Well, try this….

Some features include:
-Multiple notifications
-Multiple Time Periods
-Custom Message for Custom Bomb Type
-Custom Time for Custom Bomb Type
-Unlimited persistence of notifications until you dismiss them

The code is not yet ready for release, this is just to wet your appetite. As you can see from the screenshot, some paths still reference my machine and so it is not yet ready for distribution. But i will soon release it for debian based systems as a deb package.

Update: I released the applet, and you can get it here.

Most Hadoop tutorials use the wordcount application as a demo application. And while this might be a good demo application, it is not particularly helpful. So i wanted to think of an idea for a more useful application to use on a cluster.

My first thought was trying to implement the famous Sieve of Eratosthenes. But it turned out that this might not be a good first application because i think there needs to be a shared state between map tasks. So its better left for another time.

So instead, i opted for the methods mentioned in Primality Test. And went with the first naive approach as an example. So i tested numbers by checking if any number perceeding them divides them.(again,no optimization)

This might seem like a simple application to do, since you can basically change a couple of things in the wordcount demo application. But you see, the first thing that i noticed while reading up on Hadoop is that most apps focus on the same file-based approaches. Which makes sense when you consider the project’s roots in indexing. And that is a limitation if you consider the possible projects that can and do make use of distributed computing such as medical research, cryptography work, and so on…

Well, while Hadoop does have built-in classes to deal easily with file based input, you can easily make it deal with any other kind of input you want. So while most apps don’t actually go deep enough to tell you how to do this, I will. Although i am still scratching the surface, i hope this would be useful to somebody out there.

So, if you just finished reading about the wordcount application, you’d think that you can the prime test by writing all the numbers that you want to check into a file and then just use that file as input to your MapReduce app. While this would work, why do the extra step of writing all the numbers to a file and then reading from that file again. We’re already have to generate the numbers, so just hand out those numbers to the Map tasks.

From this point on, I assume that you’ve already read the tutorial that comes with Hadoop, so I won’t go into details on anything basic.

In a nutshell, to make the mapping tasks acccept generated input, you have to define a new type of input format. You do that by develping a class that implements InputFormat. With that, you also have to implement a RecordReader and an InputSplit.

In the end, you will have the following files:

• PrimeTest.java: Contains the MapReduce procedures.
• PrimeTestInputFormat.java: Defines the input format and provides the input splits for the map tasks and also defines the record reader.
• PrimeInputSplit.java: Defines a split in the input data for each map task.
• PrimeRecordReader.java: Responsible for returning <Key, Value> records from bye-based input splits.

This application will use a record <K,V> of <long, long> type with both the key and value being the prime number to test. And the intermediate record <K’, V’> from the map tasks will be of types <boolean, <Long….>> with the key being a true/false primality result. The result record <K”, V”> from the reduce tasks will be of type <boolean, Long> and is again the number keyed by primality result. Not really the best of use for records, but for out specific example, that’s all we need.

Starting with the main file, In PrimeTest.java, we define the map/reduce procedures.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79



import java.io.IOException;
import java.util.*;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*;
import org.apache.hadoop.util.*;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

public class PrimeTest
{
    private static final Log LOG = LogFactory.getLog("PrimeTest");

    public static class Map extends MapReduceBase implements Mapper<LongWritable, LongWritable, BooleanWritable, LongWritable>
    {
        private final static BooleanWritable truePrime = new BooleanWritable(true);
        private final static BooleanWritable falsePrime = new BooleanWritable(false);

        public void map(LongWritable key, LongWritable value, OutputCollector<BooleanWritable, LongWritable> output, Reporter reporter) throws IOException
        {
            long suspectPrimeValue = value.get();
            boolean isComposite = false;

            for(long i = 2; i < suspectPrimeValue - 1 ; i++)
            {
                if(suspectPrimeValue % i == 0)
                {
                    output.collect(falsePrime, value);
                    isComposite = true;
                    break;
                }
            }

            if(!isComposite)
            {
                output.collect(truePrime, value);
            }
            System.out.println(suspectPrimeValue);
        }
    }

    public static class Reduce extends MapReduceBase implements Reducer<BooleanWritable, LongWritable, BooleanWritable, LongWritable>
    {
        public void reduce(BooleanWritable key, Iterator<LongWritable> values, OutputCollector<BooleanWritable, LongWritable> output, Reporter reporter) throws IOException
        {
            while(values.hasNext())
            {
                LongWritable vCheck = values.next();
                output.collect(key, vCheck);
            }

        }

    }

    public static void main(String[] args) throws Exception
    {
        JobConf conf = new JobConf(PrimeTest.class);
        conf.setJobName("primetest");

        conf.setOutputKeyClass(BooleanWritable.class);
        conf.setOutputValueClass(LongWritable.class);

        conf.setMapperClass(Map.class);
        conf.setCombinerClass(Reduce.class);
        conf.setReducerClass(Reduce.class);

        conf.setInputFormat(PrimeTestInputFormat.class);
        conf.setOutputFormat(TextOutputFormat.class);

        conf.setOutputPath(new Path(args[0]));

        JobClient.runJob(conf);
    }
}

The map procedure is a direct implementation of the prime test algorithm. And so is quite straightforward. So is our reduce procedure; just placing the values in the output collector.

You’ll notice that we define our own input format and that we do not need to define an input path since our input is generated.

Now we need to implement the input format class. And it would look like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50



import java.util.ArrayList;

import org.apache.hadoop.mapred.InputFormat;
import org.apache.hadoop.io.BooleanWritable;
import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapred.RecordReader;

public class PrimeTestInputFormat implements InputFormat<LongWritable, LongWritable>
{
    public RecordReader<LongWritable, LongWritable> getRecordReader(InputSplit split, JobConf job, Reporter reporter)
    {
        return new PrimeRecordReader((PrimeInputSplit)split);
    }

    public InputSplit[] getSplits(JobConf job, int numSplits)
    {
        long startingNumber = 2; //from 
        long endingNumber = 265; // to, exclusive

        long numbersInSplit = (long)Math.floor((endingNumber - startingNumber)/numSplits);
        long startingNumberInSplit = startingNumber;
        long endingNumberInSplit = startingNumberInSplit + numbersInSplit;
        long remainderInLastSplit = (endingNumber - startingNumber) - numSplits*numbersInSplit;

        ArrayList<PrimeInputSplit> splits = new ArrayList<PrimeInputSplit>(numSplits);

        for(int i = 0; i < numSplits - 1; i++)
        {
            splits.add(new PrimeInputSplit(startingNumberInSplit, endingNumberInSplit));
            startingNumberInSplit = endingNumberInSplit;
            endingNumberInSplit = startingNumberInSplit + numbersInSplit;
        }

        //add last split, with remainder if any
        splits.add(new PrimeInputSplit(startingNumberInSplit, endingNumberInSplit + remainderInLastSplit));

        return splits.toArray(new PrimeInputSplit[splits.size()]);
    }

    public void validateInput(JobConf conf)
    {
        //valid input since we are generating it
    }

}

The only interesting part in this class is the getSplits() method which splits the input data to feed the map tasks. One thing to note here is that the “from “and “to” numbers are hardcoded while it would have been better to place them as properties and retrieve them via the JobConf. Also, the input generation is a naive one and has many assumptions.

Looking at the PrimeTestInputFormat class, you’ll also notice that we’re referncing a PrimeInputSplit class and a PrimeRecordReader. So we also need to implement those.

As mentioned earlier, the PrimeInputSplit class represents a byte view to a particular data split. In our case, this class will hold the starting and ending numbers for each split. So it could like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53



import java.io.IOException;
import java.io.DataInput;
import java.io.DataOutput;

import org.apache.hadoop.mapred.InputSplit;

public class PrimeInputSplit implements InputSplit
{
    private long m_StartNum;
    private long m_EndNum;

    PrimeInputSplit(){}

    public PrimeInputSplit(long p_Start, long p_End)
    {
        this.m_StartNum = p_Start;
        this.m_EndNum = p_End;
    }

    public long getLength()
    {
        return (this.m_EndNum - this.m_StartNum) * 8;
    }

    public String[] getLocations() throws IOException
    {
        return new String[]{};
    }

    public void readFields(DataInput in) throws IOException
    {
        this.m_StartNum = in.readLong();
        this.m_EndNum = in.readLong();
    }

    public void write(DataOutput out) throws IOException
    {
        out.writeLong(this.m_StartNum);
        out.writeLong(this.m_EndNum);
    }

    public long getStartNum()
    {
        return this.m_StartNum;
    }

    public long getEndNum()
    {
        return this.m_EndNum;
    }

}

Next, we implement the record reader which translates the byte view input split into a record for the map task.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

53
54
55
56
57
58
59
60
61



import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.mapred.RecordReader;

public class PrimeRecordReader implements RecordReader<LongWritable, LongWritable>
{
    private long m_End;
    private long m_Index;
    private long m_Start;

    public PrimeRecordReader(PrimeInputSplit split)
    {
        this.m_End = split.getEndNum();
        this.m_Index = split.getStartNum(); //index at starting number of split
        this.m_Start = split.getStartNum();
    }

    public LongWritable createKey()
    {
        return new LongWritable();
    }

    public LongWritable createValue()
    {
        return new LongWritable();
    }

    public void close(){}

    public float getProgress()
    {
        if(this.m_Index  == this.m_End)
        {
            return 0.0f;
        }
        else
        {
            return Math.min(1.0f, (this.m_Index - this.m_Start) / (float)(this.m_End - this.m_Start));
        }
    }

    public long getPos()
    {
        return this.m_End - this.m_Index;
    }

    public boolean next(LongWritable key, LongWritable value)
    {
        if(this.m_Index < this.m_End)
        {
            key.set(this.m_Index);
            value.set(this.m_Index);
            this.m_Index++;

            return true;
        }
        else
        {
            return false;
        }
    }
}

And that’s about it. Just jar up the application and deploy it on Hadoop.

There are a couple of interesting things to try here. First up would be to implement some optimizations to the prime testing function. Another thing would be to implement a better way to generate input splits to accommodate sparse data for example. Another interesting thing that I’d like to try sometime is do a multi-threaded application with the same functionality and then compare its performance to a small non-uniform cluster. Ofcourse the overhead of Hadoop on a single node cluster would mean that the local app would be faster so i am interested in observing how much this overhead affects performance so spreading it out over a couple of machines should balance things out. At least in theory…so we’ll see.

The source files are available here.(you can also directly deploy this jar)

Comments, feedback are welcome.

My last twitter was “Its after midnight and i just finished my first MapReduce application. Next up, blogging about it.”

Well now its time to blog about it. And share some code…won’t that be fun?

So I’ve always wanted to do some distributed programming or parallel programming(technically not the same, but aim at solving similar problems), but the barrier of entry was always too high and the learning curve a bit steep. At least for a hobby. I remember getting a demo account on my university’s cluster when i was still a student after attending a workshop but i never had the time to try it out and it silently expired.

Well, I’ve been reading about hadoop for quite some time and i finally decided to dedicate some time to learn how to use it. I was pleasantly surprised. I gotta say, that’s one well written application. Doug Cutting is one exceptional programmer; Not that he’s waiting for my recognition. The project wouldn’t be where it is without the Hadoop community, so kudos to all.

The whole experience only took a couple of hours. That included reading the documentation, setting up hadoop as a single node cluster and writing my first application.

Hadoop can certainly make use of some more tutorials. The documentation is very good, but at first view might look a little overwhelming. Let’s hope that more people will use it and share their experience.

Next up, my first app, the problems i faced and my comments.

As a follow-up on I am getting Ruby-fied, Ruby’s not ready - glyphobet • ???????? • ???????? does a much better job at going over Ruby’s faults. Ruby on Rails is not discussed except to say that it and Pylons are very similar, but i tend to agree that a lot of the different web frameworks out there are quite similar because they all include what is needed of a modern framework.

The author has had more experience in Ruby(and Ruby on Rails) than i did obviously, so i won’t try to go into the things he did. I also can’t validate all his concerns, but i can validate most and agree with most. So i can’t go as deep as he could in the discussion because as expected, the RoR project has been stalled for the past month or so. But let me review my previous assumptions and determine what was and wasn’t true. Note that i will refrain from commenting on the things that i can not yet validate to keep things objective.

Ruby seems like en easy to learn language. A lot of the language syntax and Ruby-way-doing-things(yes, that’s one word) is meant as a productivity boost.

That is definitely true. But the many quirks in there make it really difficult to learn the language in any deep manner. I mean to_s ? Come on. Really? to_str i might understand(which does another thing all together) but to_s? And that’s just saying one thing that i find highly annoying, take a look at the linked article and be amazed.

Ruby is not a new language, it is as old as Python and Java.

Definitely true. Although, Python and Java being old has worked to their advantage. Both languages have evolved in a generally useful direction. Ruby on the other hand seems to be this toy language that suddenly got immensly popular and is now struggling to become a modern language.

Python and Ruby have similar principles. The most interesting perhaps is that code should be easy to read and some would say beautiful to look at.(should be true of any language, but depends on the programmer i guess)

Hmmmm…. I am not sure about this one. Ruby is very lax in its syntax and that makes it terrible to read and definitly not beautiful. In fact, this whole flexibility has turned me off the language completely.

It was meant to solve a general problem and is not meant as an end-all-be-all type of solution.

That’s true. If you turn off the noise generated by the fan boys then you would find some wise people telling you not to use Ruby for things it wasn’t designed for.

I come from a c background so i had trouble getting used to languages that don’t follow the C syntax(and its older parents). Earlier on in my career, i decided that i shouldn’t let something like different syntax stop me from learning what might be a truly great technology. I followed this approach with Python and was quite successful and will do the same with Ruby.

I did follow that approach on leanring Ruby and well, i think that with proper coding standards, Ruby code can be quite elegant. My only problem is fan boys turning turning Ruby into php.

I think that a lot of the features in Ruby 1.9 should really have been in earlier versions. I don’t know why is that the case though, maybe there’s a good reason.

Also true. In fact, my suspicions were quite correct here in that the language doesn’t have a very clear path. But again, this seems to be a subjective issue.

I don’t appreciate the Ruby community believing that the Ruby language is the best thing since sliced bread. It is possible that it is an excellent language but definitely not the answer to everything. Passion is always good, but zealotry is not. I believe that good programmers should choose the best tool for the job. I wouldn’t advocate Python for every kind of problem out there.

Well, there’s nothing i can do about that expcept advise my clients(and anybody else who asks) about the proper technology.

A lot of people also complain about some of the conventions used in RoR and how dreadful they are. That is somewhat scary, but then again, all frameworks have some of those. It would be possible to create a framework that doesn’t, but then it would be too complex to learn. A compromise is always needed. I don’t yet know enough to comment on this, I’ll report back.

Well, this wasn’t very accurate. I wasn’t intimidated by the convention in RoR, and actually think they’re quite reasonable. The conventions and quirks in Ruby however were a PITA. In an RoR project however, you would be more worried about developing within RoR rather than worrying about Ruby’s idiosyncricities.

People always rave on about the great toolset that RoR has. In particular, capistrano and rake. I think those are definitely good tools but there are several different alternatives for different techs as well.

I haven’t worked with capistrano yet, but Rake is cool. I like the way migrations work with some minor comments(should be fixed in RoR 2.1 i think) and i also like the way unit and integration tests are considered to be part of the project development as opposed to “extra development”. I hate the way JS using prototype is dealt with in RoR and prefer Django’s “use whatever you like” atitude and for good reason. Scaffolding is nice at first look, but nobody actually sticks with it except for very small applications. The book i was reading didn’t describe static scaffolding, that seems to be a nice approach to it. And so on….

I went with Django earlier last year because of the excellent way the framework developed. It has very good documentation and a healthy development environment. At the time, i couldn’t see that in RoR.

Things have definitely changed with RoR. I still prefer Django immensly because of the general atitude of its developers, but…..

RoR and consequently Ruby got its fame largely from the efforts of one company. 37Signals may be great, but i wouldn’t trust the development of a general purpose framework to one small company.

True at first, but RoR is its own beast now. There are still some existing problems, but then again, there always is.

Django seems like field-produced framework. Initially, it constituted the guts of a CMS and was later extracted and made in a general framework. I find that it particularly adept at solving problems that you would encounter when writing a data driven web application which a CMS is an extreme example of.

This is definitly also true of RoR. So the comment is valid, but it doesn’t apply correclty to RoR. RoR was extracted from a 37Signals application(backpack was it?) and so is also a field-produced framework.

So in summary, i still wouldn’t use either Ruby or RoR if i had the choice. But depending on its couterpart in comparsion, they might just win.

Tomorrow is the first of April…i had actually forgotten about that if it weren’t for the news interviewing a sample of people about lying.

The strange thing is that most people not only admitted to lying constantly, but also remarked that it is a necessity in this world. And also boldly claimed that “you have to lie to survive”. The most interesting perhaps is the person who challenged the interviewer to get another person who claims not to lie so that he can call him/her a lier to his/her face.

This is interesting because just the other day, someone was explaining to me that i don’t know anything about life and that i am an “idiot” because i believe that we should not lie and that we should try to be civil with each other. The story in question was later on revoked because i was able to do what i needed in the way that i thought to be correct and thus the “idiot” part is no longer valid.(they didn’t actually say “idiot” to my face, it takes big balls to do that but it was something along those lines)

Well, to the people who agree with the sample, you should consider listening to Steve Jobs amazing Stanford speech(youtube)…mainly the part that implies to live your life the way you see fit and don’t get stuck in dogma. For the people in the sample i say, What kind of world are you leaving behind for your children?(and for the previous generation, what the hell were you thinking?)

Too idealist? Perhaps…But there really is a lot of shit in this world, i would really prefer to not be yet another turd in the pile.

Does anybody actually ever read those EULAs? There has been some news lately of how companies are updating them in a haphazard way. So the question really is, does anyone actually read like 10 pages of legalese just to install a piece of software?(i don’t say use, because you have to agree to the EULA before using it)

One of the projects that i am working on is about to do a switch from a php platform to a Ruby on Rails custom platform. But wait, judging from my previous posts, am i not a Python guy? Don’t i love Django and heavily recommend it?

I sure am, and I sure do. And that will not change any time soon.

The decision for choosing Ruby on Rails was not a technical one. The other members of the team(2-3 people) are big RoR fans(in fact, i call them fan boys). And so i thought that it wouldn’t be wise to suggest anything else. Was this decision good or bad? Time will tell. And so will this blog, so stay tuned.

Anyway, so since having done many languages and many platforms over the years, i consider that i have reached some sort of language(platform)-agonistic maturity. I also consider that having several different technologies in your arsenal will definitely allow for more flexibility and power when attacking problems.

So, considering this as an experiment, i thought it would be useful to document my progress on this blog and by doing so also discuss the weaknesses and strength of Ruby on Rails. Ok, so here it goes.

The only exposure that i have of Ruby on Rails and even just Ruby as a language is what i read over the web and some slight interest when i was about to choose between Python and Ruby as my main language earlier last year. So i’ll try to list what i already believe to be true about the platform and as i go on with my education, i’ll look back and see what was true and what wasn’t.

- Ruby seems like en easy to learn language. A lot of the language syntax and Ruby-way-doing-things(yes, that’s one word) is meant as a productivity boost.

- Ruby is not a new language, it is as old as Python and Java.

- Python and Ruby have similar principles. The most interesting perhaps is that code should be easy to read and some would say beautiful to look at.(should be
true of any language, but depends on the programmer i guess)

- It was meant to solve a general problem and is not meant as an end-all-be-all type of solution.

- I come from a c background so i had trouble getting used to languages that don’t follow the C syntax(and its older parents). Earlier on in my career, i decided that i shouldn’t let something like different syntax stop me from learning what might be a truly great technology. I followed this approach with Python and was quite successful and will do the same with Ruby.

- I think that a lot of the features in Ruby 1.9 should really have been in earlier versions. I don’t know why is that the case though, maybe there’s a good reason.

- I don’t appreciate the Ruby community believing that the Ruby language is the best thing since sliced bread. It is possible that it is an excellent language but definitely not the answer to everything. Passion is always good, but zealotry is not. I believe that good programmers should choose the best tool for the job. I wouldn’t advocate Python for every kind of problem out there.

- Some of the fan boys mentioned above keep on talking about how great the Ruby and RoR community is. Might be, we’ll see. I find that the Python community is amazing. I also find that the free software community is incredible and and and….

- Some time ago, there was a big debate about the original Mongrel author and the RoR community. I read that with interest, though in all fairness, i find that what he said might apply to a lot of other communities as well.

- Scalability with RoR has always been an issue. I think this was made famous by twitter. I did read DHH’s response though, and i find that what he suggests are actually scalability advice that you can can apply everywhere. And so, i don’t believe that there’s any inherent problem with RoR itself.

- A lot of people also complain about some of the conventions used in RoR and how dreadful they are. That is somewhat scary, but then again, all frameworks have some of those. It would be possible to create a framework that doesn’t, but then it would be too complex to learn. A compromise is always needed. I don’t yet know enough to comment on this, I’ll report back.

- People always rave on about the great toolset that RoR has. In particular, capistrano and rake. I think those are definitely good tools but there are several different alternatives for different techs as well.

- I went with Django earlier last year because of the excellent way the framework developed. It has very good documentation and a healthy development environment. At the time, i couldn’t see that in RoR.

- RoR and consequently Ruby got its fame largely from the efforts of one company. 37Signals may be great, but i wouldn’t trust the development of a general purpose framework to one small company.

- Django seems like field-produced framework. Initially, it constituted the guts of a CMS and was later extracted and made in a general framework. I find that it particularly adept at solving problems that you would encounter when writing a data driven web application which a CMS is an extreme example of.

This is getting to be a long post, and i could probably go on. But i’ll spare the readers. One final comment though. One developer recently commented that “Ruby is glorious”. That kind of nonsense makes very angry because it tells me that this particular developer has not seen anything but Ruby and is inexperienced enough to make you afraid of working with him. You can substitute the word Ruby in that statement with any other language and the comment would still be valid. I try to stay away from discussions with such people because its a complete waste of my time.

I still get a lot of searches and hits on my “Mytube” page as well as some inquires by mail…which tells me two things. One is that people still do not fully understand how to fully use flex and the other is that RIAs are pretty popular these days.

To help people who come searching for this, let me shine some light on the matter. I’ll do so in the form of bullet points to speed up the process.

• flex is really cool. It will allow you a lot of flexibility(no pun) to write rich clients that run in the browser in a cross platform manner.
• flex is a client side technology. Remember that. So the entire thing runs on the user’s browser.
• To make your flex app serve dynamic data or generally have a dynamic nature, you need to use some server side technology and communicate with the client.
• The server side technology can be as simple as a single php script or as complex as a full blow Django based website.(or some other framework)
• The server side technology used will be responsible for dealing with things like database queries and user authentication and so on…
• For communication between the flex client and the server side tech, you can use several methods depending on your needs and the complexity required of the app.
• The communication methods can be simple one way xml documents from the server to the client or using something more complex like the different amf frameworks for interop(or flash remoting or something like the flex data services…but the latter used to be a commercial offering and i prefer the approaches that i describe here).
• The MyTube tutorial uses the simpler one way xml documents where the TurboGears web app sends the needed xml to the flex UI. It takes care of turning the dataset in the database to xml.
• In general, i recommend using one of the amf frameworks for your particular server technology(so for php use amfphp, for python use pyamf, for Django you can use DjangoAMF….)
• The amf frameworks will allow you much better interop between clients and servers because they allow both to speak the same language, mainly the Action Message Format was which open source’d by Adobe a few weeks ago.
• The amf frameworks will allow you to send native flash data to the server side tech without needing to worry about some common data format(not to mention that it would allow for more complex data handling)

So there you go…i hope this information will help. I’ll try to publish some more tutorials and some more information when i can. And as usual, for anything more shoot me an email.

Please note that although i wrote that tutorial using TurboGears, i no longer use TurboGears nor do i recommend it to my clients. I didn’t like where the project was heading and the way it was being developed and so i switched to Django which is absolutely amazing and has a great community behind it. I have been using it for about a year now and i fully recommend it. Ofcourse to be fair, you should make up your own mind about this, but i already have.