Thanks to Github + Jekyll + JekyllNow, I finally have my own blog up and running in really no time (Okay 10 minutes to be precise).
For more instructions head over to the Jekyll Now repository on GitHub or visit Homepage. You can also check the nicely detailed post by him about Building a Blog UsingJekyll Now
Though if you want to explore this into more details and learn you might not prefer this approach. To setup a blog similar to one that JekyllNow helps to build, you will need to explore GithubPages and Jekyll.