One year closer to 2012

So happy new year everyone!!!

I got a feeling this will be a interesting year. For everyone.

A little misadventure with django and database

** this could be long**

It is almost the end of my project now, along the way, found a misadventure with quite number of stuff. Which included Django ORM. What is not to love for a ORM, it make a nicer abstraction to wring program that need to talk to db. Making us, to worry less about sql. Using django ORM with south, it make refactoring database a lot easier.

Well in my current job. The concept of using ORM is pretty new in the company(from what I see). So the projects they done is done like pre-ORM days. Writing SQL, doing select etc. For a person that is used to ORM, I am thinking like "Really?". So the way they do things is obviously very different, and this with Django's ORM, not a pretty sight.

For example, when integrating with other project. The expectation is, from a temp table that stores information from the other DB then write a INSERT statement, and push it to the target table. I was like again thinking "Really? They really did that?", especially the target table is huge.

And of course, in Django, I pretty used to have foreign key to reference to another models(table), which I have a few. And because I use django south, so that I can refactor the database from time to time. The order of the column is weird. Not to mention that the idea of a ORM is to make it look like a object, in my case python object, so I used to use boolean type, which from other table, uses char. So I imagine create a INSERT statement with a few SELECT, and map the variable properly. Pretty scary to me. I ends up writing a python script that uses the ORM to do it.

For the temp table, that contains info from the program i need to integrate with. It tend to be created by hand, because it make the existing script to export the table to sane. So have to create a unmanaged table. So django don't create the same table. Which I found, that for that particular case, I can't create a django fixtures for that models without losing data. Which I still investigating. In the end of the day, I just do a sql dump, just in case.

Also, as we build the application, the team is expecting a totally different behavior for Django queryset API. When in SQL, when you can't just delete a table that have another table reference to it. Django queryset don't seems to care, and will delete it anyway. So that is causing issue in a way that, we have to add checking before the user can delete anything in the application.

Talk about deletion, django ORM added constraints on column that refer to another table as they create the table. So manually delete and insert via SQL is a pain. But it is always faster to do that!!! Because the queryset API can be slow. And sometime they have quite a number of magic that scares me sometime and sometime bites.(or maybe I just need to understand it more).

After all these misadventure. I just realized, you can't just drop a database guy can get them to use ORM. because the workflow is totally different. The technology is different. Everybody need agree that approach, or properly planned it. Actually I am not sure. But what is sure, mix of ORM and non-ORM approach can be a mess.

Load Testing with Python Multi Mechanize : The setup

Few week back, I was demonstrating a script to open a webpage with python using mechanize. Extend upon that, I use this along with multi mechanize. A Python based tools to do load testing on a webpage. What it does differently with mechanize is, this actually run a set of test script in parallel. And you can use any library to access the page, it can be mechanize, or urllib or httplib.

To setup, you will need python 2.6 above. Because it uses multiprocessing module that is available for python 2.6 and above. You will also need matplotlib which also require scipy.

Then you just download multi mechanize and extract the file to any accessible directory. Go to the project folder there will be a default folder, that is a example project. A project will need 2 folders and a config file, create a new project folder, within it create results directory, and a test_scripts directory, and copy config.cfg from default folder.

Next round we going to write a script

Startup in 3 days aka #startupweekendkl2010

I got a chance to join Startup Weekend KL 2010 last weekend, thanks to Daniel Cerventus. The idea of Startup Weekend is to launch a startup within 54 hours. It is a place where aspiring entrepreneurs share and grow their idea, and hopefully launch a startup by the end of the program.

We start by sharing our idea on Friday and form a team on the same day. On Saturday we begin to work on our idea, with whatever we have, some work on presentation, some doing business plan, some codes. We also have visits from Mentors to guide us our way to the business . This continue until the end of Sunday where we will pitch to Mentors which already launch their own startups before.

I join a team called, findmiddle.com. We produce a webapp that recommend place for meetup based on a center point from 2 location. Our team is awesome in a way that we are one of the very few team that manage to produce a product in the end of the day, we also have a balance team, where we have techie and business guy.

What I learn on the day on startups.

• Get a balance team, we got a combination of tech and business, builder and speaker.
• Keep it small, so that we can focus, so that we can build it fast
• Web api is free and useful, use it. It would be impossible to build findmiddle without google map, 4sq api and bit.ly api.
• Cloud based collaboration tools is free and useful. We use gdoc, dropbox, and google analytics.
• Open Source Software is easily available, and is full of libraries. We use a php, jquery and php mailer.
• Tech doesn't matter, who can use the tools fast, rules. I prefer python, but we got 2 php guy in the team, so findmiddle.com is on php. 
• Web hosting is cheap, few hundreds Ringgits. Nuff said
• Human attention span is short, make a pitch short and interesting. 
• Talk is cheap. Implementing findmiddle is not without bumps, but in the end it worth it. 
• Murphy's Law is everywhere.

In the end, I would like to thank, all the organizer of startup weekend in kl. I have learn a lot, and it is a interesting experience. Sorry for the wordy story, I forget to bring my camera.

Learn more about startup weekend here
http://startupweekend.org/

You can look at our slide here
https://docs.google.com/present/view?id=0ARiCz9gqQzLNZGM1d3RxYnFfNmZwODVxd2Ny&hl=en

Photo is here, thanks to vivian
http://picasaweb.google.com/GEWMY2010/StartupWeekendKL02

We have a facebook group in
http://on.fb.me/startupweekendkl

Our product is in
findmiddle.com

Meetup with Graham Dumpleton

Hi 

Everyone is invited for a meetup with Graham Dumpleton, he is the developer of modwsgi, a implementation of WSGI standard for as a apache module . WSGI is a interface for python web application in accordance with PEP333 

Details

Date: 16th October 2010

Time: 2.00 PM 

Location: Hackerspacekl Damansara Uptown

Anyone that is interested, join us.  

fun with python mechanize

In the Perl world, there is WWW-mechanize module which is used as a web browser in perl to use to get load a webpage. In python there is a mechanize module that do precisely that.

To install just:
pip install mechanize

Lets assume that we have a webapp at address http://mywebapp

here

The code above shows how to open a page, fill in a form field, and submit it, and get the next page. It will print whole page in html, from the result of a query .

This is useful for automate access to a webpage, in my case, to test a webapp in the office. I also use this to do load test with the python multi mechanize. Pretty nifty as it is actually a browser itself.

Setting up a python environment on windows

One advantage of working in current company each got a laptop. But like most IT company in Malaysia, it uses windows. In the job I ssh to a linux dev server for work, for personal project in python to be done locally, and setting up python on windows is a pain.

The common way to install python on windows is

1. download python from python.org
2. install
3. setup the path

And repeat the process for setuptools, and virtualenv, it is a pain because not just have to setup the path, and have to go to a page to install it.

Another way is to use active python. Which is free, and their community license is pretty acceptable. And the best of all, it bundle with easy_install, pip, virtualenv and the path is done for us properly. You can download it here

SFDKL The Key take away

While we didn't do a lot during Software Freedom Day, we do have a interesting informal discussion on how things works in Malaysia. When Ashik and Rajiv is asking how the open source community is like in Malaysia. To make it short, it is real sad situation.

Apparently in Kerala, schools is required to teach open source, along with others. There is a active community, with some of the community involve in Open Source Related business, too do their part. For example a computer distributor that with request from user willing to ubuntu on the computer with reduced price. Will charge for proprietary OS. And they actually have a install fest.

For Malaysia, the case is different, school don't teach. And people can't see the viability of Open Source Software related business here and the govt just paying lip service to Open Source. Among other problem.

Short take away, we have a long way to go. But from the discussion, it seems that, the best way to promote open source is to start a business with it, and make money, and make it a case study.

BTW: Thanks for hackerspacekl for the space

SFDKL The end

Software Freedom Day was held in PJ, Malaysia Last week at Hackerspacekl

The event itself draws a small crowd, I wouldn't call it a party, but it is a start(of a new thing I hope).

The guys that comes much later

While things don't goes as planned, and the LAN party didn't happen, video screening starts much later. And we have a Coffee session and Lunch Session, so we ends up with a very informal discussion. Especially with 2 very interesting gentlemen from India working in Malaysia, which also involved a lot in Open Source Community in their home town.

Rajiv and Ashik from India

The session starts with the screening of Code Rush, after lunch

Code Rush, video Screening

Then introduction of the unix-g33kz session on hackerspacekl

Then we ends with another round of coffee session.

Software Freedom Day, how to go

To go to hackerspacekl, the precise address is

21B, Jalan SS21/1A,

Damansara Utama,

47400 Petaling Jaya,

Selangor, Malaysia

It is at uptown damansara near TM point, you can see the entrance at the shop under a big Antique Shop Sign.


The number to call if you need direction is 017 626 6011

#sfdkl status on game survey

Alright guys/gals, Software Freedom Day is tomorrow!!!

A little update

The result of the game survey, suggest that we should have a multiplayer round of hedgewars
https://spreadsheets.google.com/ccc?key=0AmN4VC_DvFzBdHZzZkdwR0psSy0tMTItNGRsMDg2YlE&hl=en#gid=0

Want to know what is it about go to this side,
http://www.hedgewars.org/

It works major platform, and quite well on eeepc(which I own)

#sfdkl status redux

This is schedule for the Day, the planned talk don't goes as plan, it is replaced by Lightning Talk and video screening. I still hoping to have a gathering of foss geeks in KL.... 

A little explanation on what is Lightning Talk. Lightning talk is a 5 minute presentation volunteered by attendee on topic related to the event. So I hope to hear on foss related story that you have, or free culture, or open hardware stuff, from all of you guys during SFDKL. Though I doubt that 30 minute is enough for a lightning talk

• 10:00 AM : Welcome + Lightning Talk
• 11:00 AM : Screening of Revolution OS / Patent Absurdity/Code Rush
• 12:30 PM : Lunch and mingle around aka networking
• 13:30 PM : Unix G33kz
• 15:30 PM : Frag Fest

I hope to see all of you guys there!!!!

When Python talk to Arduino

Yesterday I show a code where when we send a 't' via serial connection, the LED will blink while the arduino board, send back a series of string via serial, the code below is what I used to run on arduino, to test with the python code.
link


In the arduino development environment, there is a serial monitor that do the the testing.

Since it is just a standard serial interface, so one can just do it anyway they want. Here I show how to do it via python. First what we need, is the pyserial library, you can install it using pip or easy_install or grab it from pyserial site

To install, just use to following command. if you need more detail, read the docs, because I use python 2.7 it just works for me,

pip pyserial

or

easy_install pyserial

To use it, with the arduino code from the top of the page just, do the following
link


The line s=serial.Serial(2) points to com3 on windows, on linux it should be s=serial.Serial('/dev/ttyUSB0'), or other port that shown by the arduino serial monitor. Pyserial actually have a few options on the Serial object, but I found that for arduino, the default is adequate.

And this is how to use arduino to interface to your computer using python. enjiy

Confession of a arduino newbie

So one of the side effect of hanging around with hackerspacekl too much is, one begin to develop feeling toward hacking hardware... Which in one of my post shows their project creating their own arduino severino.

I got myself freeduino thanks to Kakeman, our resident hardware guru in hackerspacekl. It is not the latest, but it is good enough. After following the tutorial, I begin to feel to blink a LED, which built into the board
link.



Then I play around with the serial library,
link


In the end of the day it is fun, much before I continue, I need to find time to get a breadboard and more components, like LED,transistors, resistors etc, to make more stuff. Think i am going to play with this for a long time to come

Software Freedom Day KL status

It is a while since I actually started Software Freedom Day in KL. Since I am like swarmed with job. So finally get a few thing done
A bit of a status report for Software Freedom Day KL 2010

What we have:

• 17 attendee registered
• A space, in hackerspacekl
• A switch for the LAN party
• A site at https://sites.google.com/site/sfdkl2010/
• And aircond(ok...hackerspacekl have it for sometime)

What we don't have,

• Speakers, for the sessions
• Helpers to help, not that the event need many people

So what next, if we have speaker, we just continue as planned, else, we might reschedule the whole thing it depends on how.

Doing ajax with django models serialization: jquery part

We have finished(somewhat) with our json source, now we can do ajax. With the help of jquery, we can use the data source to create something interactive. One of the key ingredient is the jquery ajax function, there is a few such as, $.post, $.get, $.getJSON etc. Here I use $.getJSON, because

Lets start with load data to a table from a json source last post. 

Loading data into table alone is not that exciting, and you can do that using django templates, but this shows a example of loading data using json

You can get the example project here
http://github.com/sweemeng/Sweemeng-django-example/tree/master/djangoajax/

Software Freedom Day @ KL

Another year, another Software Freedom Day

I have registered a team for sfdkl, and the page is at https://sites.google.com/site/sfdkl2010/

It will be held on 19 Sept 2010, a Sunday, at hackerspacekl

And with the help of hackerspacekl, i borrowed the space for software freedom day. Which is cool, because there is a few place to eat there.

What we plan to do

1. Have a gathering of foss geeks on the day itself, 
2. Someone will give a talk on the day itself
3. Frag fest aka Lan party of foss game, in the end of the day, this is the only event that is confirmed
4. t will also be held with a unix-g33kz session which  is part of hackerspacekl own project, while this is payed project, the rest of the event is not

What we need now, speaker, well attendee. And last but not least, participant for our Frag Fest(we even got a switch ready),
you can register on the google site page, add your talk there as well.
https://sites.google.com/site/sfdkl2010/

Doing ajax with django models serialization: json part

Finally a technical post after a long while, here goes

On my new job, the site requires a crazy level of interactivity, probably for the wrong reason, but lets not go there now.

Along the way, we decide to use a mix of javascript, and html, with a help of json. Thats is where, django serialization comes in. What it does is, it serialize a django queryset into xml or json. If i need to use that to generate json for django queryset, because simplejson cannot do it, not directly.

To output a queryset to json is pretty easy.


from django.core import serializers
data = serializers.serialize('json', queryset)


where queryset your djangoqueryset


in a sample database, in a called addressbook





to play around with the django shell, assume that data already there


>>> from django.core import serializers
>>> from addressbook.models import Contact
>>> serializers.serialize('json', Contact.objects.all())
[{"pk": 1, "model": "addressbook.contacts", "fields": {"phone": "012345678", "post_code": "68100", "name": "sweemeng", "address": "somewhere in selayang"}}]


To make it useful for my purpose to use in a javascript, i need to put this into a views, point your urls.py to the views, and run the server, and point your browser to it, you should able to see a json output





But here is the catch, one, this is strictly only for serializing django queryset, you can still use simplejson for other type of data. two it seems that django serializer don't serialize a single model instance.


Next post, using it with a jquery

more information on django serialization

http://docs.djangoproject.com/en/dev/topics/serialization/

A day in unix-g33k @ #hackerspacekl

One of the big project of the year in hackerspacekl is unix-g33k. the goal of unix-g33k is to expose geek/geekette and non-geek alike on unix, particularly on freebsd. The workshop done in Suresh. And organize with the help of hackerspacekl particularly Sniifit and Brian Ritchie,.

The workshop covers basic OS concept for the first day, with Suresh prepared a VM running freebsd that allow us to access it via SSH for playing around. Today session itself is enlightening, while it is also serve as a reminder of the OS course i took in uni, with some extra stuff that Suresh give that involved that stuff involve the real world.

The attendee is from various background, most from IT, some runs their own business. And doing a session on a floor is a real interesting experience, kudos for the organizer for preparing more plug points for the participant, and the session is not formal, and that is always a good thing.

After the session just hang around in a coffee shop nearby. And discuss all geeky stuff.

What to look forward to for the next unix-g33k session? It involve more practical hands on freebsd. More deep stuff.

BTW the photo is here, thanks to CLchow,


**ps, no pics again, no camera sorry.

Open Source Conference by osdc.my

MOSC 2010 is a open source conf organized by osdc.my. It will be held in  29th June to 1st June 2010) Like last year it will be held in Berjaya Time Square. Which is cool, because it is very accessible via public transport.

This year conference will have participation from a few key open source player, Red Hat and Novell is there, also a representative from the Linux Foundation will be on the conference as well. The community is represented by ubuntu.my, joomla, OWASP, wordpress, and a few others. 

This year talk have more variety than last year, it covers talk from Cloud Computing, to Security, to Uses in academics. There will be a few mini events within mosc 2010, such as Milking The Cloud competition by Microsoft, and Gecko Moving Forward by Novell. Of course there will be many community events from joomla, and wordpress among others. And not to mention show case a foss project in malaysia. 

This year conference will be a interesting conference to look forward to. And this year, I will wear a difference hat in the conference. 

Btw Check the link for more information

Malaysia Open Source Conference

New job, now beginning(?)

So I have change my job, which is interesting in many levels.

For one it is one of the very few python development shop in malaysia, which uses a open source stack. It is the first time I involve in a enterprise development in coding side. And probably need to get used to a structured environment unlike the web startup i involved not too long ago.

First thought is, I probably need to know more of django, because they uses stuff that I didn't use before, the web startup i involve switch to a nosql store because of flexibility. The company project using django south, and uses part of django that I didn't use before. But than django can be big....

So I think that would be a interesting experience........

get the correct datetime value for python datetime module

UPDATE: Turn out that i did it in django shell, which, set the time to chicago, which make the time wrong in my system, the information here is right, except on the datetime.now() error. in normal python shell, or apps, datetime.now() should show the correct datetime. but on django, we need to setup the  timezone

So I have to use datetime in this project. the more I use it I realized that it can bite, unless used properly.

Let start with a simple example, start with

from datetime import datetime

from the doc, datetime.now() show today date and time.

the time i tested the line is 12:47 on 12 may 2010.

>>> datetime.now()

datetime.datetime(2010, 5, 11, 23, 46, 50, 383091)

Notice that the day and the date, is wrong. you should try it to see it yourself. Not wrong but unexpected

Run

>>> datetime.utcnow()

datetime.datetime(2010, 5, 12, 4, 56, 13, 315120)

better, but in utc time. That is in 12:56 pm on 12 may 2010

The way to solve it is to implement tzinfo class, which I didn't do, or we can use pytz

Which does it for us anyway. Run the following in the shell

import pytz

mytz = pytz.timezone('Asia/Kuala_Lumpur')

datetime.now(mytz)

after setting up the timezone with the right tzinfo. datetime.now() should show the correct datetime.

Note to self, python datetime can bite us,

arduino's from #hackerspacekl open hardward ftw

So today I go to hackerspacekl which is a awesome place. One of their project now involving arduino. And they making their own. For the rest, I allow the pictures to do the talking. Check the links too

off the shelve components, from pasar rd

prepare acids, for etching, 

schematic from arduino websites

now you etch it, before that you need a laser printer

after that based on schematics, solder it

now you get a arduino

Components RM55, Solder with own hand free. Having a hackable hardware, priceless

android, foss and geeks @ #barcampkl 2010

BarcampKL 2010 just ended yesterday. It is a pretty awesome with many geeky talk, me as a geek, like it. 

Yesterdays BarcampKL feature a few first in BarcampKL, 

1. we have the first podcast of This Week In Asia, and it is pretty awesome, it is something of a talk show with audience, now transmitted to cyberspace thanks to p1 wimax(aka the sponsor of BarcampKL)
2. First time in barcampkl, we voted for talk. Where everywhere else, it is there already. And interestingly it serve as a check and balance of the talk. So we ends up with pretty interesting talk
3. For the first time, I've seen Jedi vs Ninja.... What really happens is, KageSenshi was in the room where somebody brought a Light Saber, and kagesenshi teaching some sword fighting technique. and the other guys, practice their light saber skills. lolz

What is notable for barcampKL,

1. Android hacks, member of Code Android malaysia, showing a android controlled robot, using a g1 and arduino. With python. No worry, the robot is benign, it still not a rival of the Governator. 
2. Time management by kaeru, i surprisingly learn alot
3. Ditesh enlighten us with javascript
4. Project Nimbus the Geeks On A Train in Singapore, they have manage to convence the government on singapore and some company to release data and using Open Data Protocol, they releases data to be used. 
5. Nginx talk is awesome too. 
6. Cookies and Cupcakes, that is real tasty!!!!

There is plenty to take away from barcampkl. Things to learn, things to do. So this year, it is awesome. 

p.s Code Android Malaysia, thanks for make me doubt my decision of getting a n900(luckily didn't buy yet)

Accessing world bank api using python

World bank have recently release their data and you can access it through
http://data.worldbank.org/

It also includes API access through it. Which is a web api. Either way it is pretty easy to access from my 5 minute toying around. My experiment attempting to find population of malaysia,

import urllibimport urllib2

# here i use json, but the api also support xml

import json

# for url parameter, just to make it look niceparam = urllib.urlencode({'api_key':'your api key','format':'json'})

# get population of malaysia

url = 'http://open.worldbank.org/countries/my/indicators/SP.POP.TOTL'

data = urllib2.urlopen(url+'?'+ param)result = json.load(data)

# btw result[0] is the data on the pages etc, data start at index 1

print result[1]

This is the result, after i clean up my codes and clarify it

BarcampKL is back 2010

Barcampkl is back to town, and this time it will be held in Segi Summit USJ on 24-25 April 2010. 

Barcamp for a uninitiated is a ad-hoc conference, where people share topic of their interest. and due to the origins of barcamp, it tend to be geeky in nature. 

This year barcamp will feature topic to be voted by the participant, and we have dedicated track for foss.my and social media club KL aka smckl

BarcampKL is free to join, come join us on that day, 

for information

http://barcamp.org/

http://barcamp.my/blog/

http://barcamp.my/blog/sessions

http://twitter.com/#search?q=%23barcampkl

twitter hashtag #barcampkl

and register here

http://barcampkl.eventbrite.com/?ref=ecal

using the subprocess module

I was trying to automate some task involving shell programs, which actually a handful to type. As a lazy programmer, AUTOMATE WE MUST. Just turn out it involve with virtualenv, or rather virtualenv bootstrap script(that is another story altogether). So I do it in python. 

One of the thing added in python 2.6 is the subprocess module. To start use it


import subprocess


to use it, after the import, run this

subprocess.call(['programname','parameter1','parameter2'......])

for example to call some unix command

subprocess.call(['ls','-l'])

subprocess.call(['ps','-aux'])

or in my case, i use it to run setup.py in bootstrap script for virtualenv 

subprocess.call(['python','setup.py','install'])

subprocess.call(['easy_install','-U','packagename'])

Just a little writeup of subprocess, probably gonna use this a lot

A date parser from python dateutil library

Sometime you need to extract date form a string, and that is actually pretty hard to do even on python. On the other hand, there is dateutils, a python library which you can install using easy_install. The library is pretty nifty, and have many component, but now i concentrate on the date parser

Start by 
from dateutil.parser import *


lets have fun, after the import, we should have the parser imported


parse("monday")


You should get the date of next monday as python datetime.datetime object.


parse('14th of april')
parse('14th of april 3pm')


you should get a datetime.datetime object of the date, 


For python programmer, check it out, it is pretty useful. 


http://labix.org/python-dateutil

Adventure with Virtualenv

In the python development world, people are using virtualenv, to isolate their python development environment from their system. Why that is cool,

• You don't conflict with the global python library. Because all new library will be in the isolated environment, on linux many software uses python
• Then you have a fine grain control of what library is needed what is not, for each environment. 
• Because all of the cool pythonista use that...ok that is not quite true, but many python based project strongly recommend using virtualenv, such as turbogears. 
• and some tools are available for this, such as pip. 
• It make it more easier to move project between directory

Lets get started,
here I assume that python setuptools is installed, i assume each project is one environment
to install, just run below as administrator.

easy_install -U virtualenv

 To start a project

virtualenv projectname

This will setup an environment, with their own setuptools.
Let start using it

 source projectname/bin/activate

and it should have the whole python environment there, isolated from the system. including python, their own site-packages

To stop using it

deactivate

To totally isolate from the system, start a new project by

virtualenv --no-site-packages projectname

 It will not use library from the system.

Some tools already begin to support virtualenv. Such as pip, a replacement for easy_install, with uninstall support. To use pip with virtualenv

pip -E projectname packagename

So, virtualenv is now the requirement of most project now. Time to learn it...

What I really working on

I am currently with a local startup, which don't want to be named. Our goal is to increase local usage of local content, to build a content company that conform to the local culture, and not letting the foreigner corrupt our beloved local culture!!!

We can't really tell what we really try to do, we can however show our technology stack, which we all really proud of :) and hopefully certain information minister would be proud too.

OS

We previously use Ubuntu, because african culture is OK, until they plan to put in a music store. That shows that Ubuntu really foreign company that try to corrupt our culture, we cannot allow that.

After much consideration, we going to move to Linux From Scratch, so that we can really check what we want to add and what not to add. We must not allow the westerner have a chance.

Which hopefully we don't need to rewrite an OS, because we really can't, because the lecturer in universiti never taught us that.

Web development

We originally plan to use python and django, now we know, python is really named after Monty Python, and Django, is named after Django Reinhardt, a jazz guitarist.

Two western media that will corrupt our mind, we cannot allow a joke on Spam to ruin us, you know Spam is a serious problem, how can you make it into a joke?? Django Reinhardt is a jazz guitarist, malaysian must not listen to that, no we must watch wayang kulit and all malaysian thing.

To minimize that, we will use haskell, precisely with happstack. because it does not show too much of the western culture.

Conclusion
It is hope that by starting from the technical level, we can be a content company that is suitable for the local culture.

changed: droping the minister name

Happy Document Freedom Day

Have you ever
1) Cannot read a file after you switch a software,
2) as if you can switch software
3) Someone send you a file that you cannot read,
4) Not without a proprietary software
5) assuming that the software is still available,


This would not happen if everyone stick with Open Standard and Open Documents.
Why?

• If the software not available for a variety of reason, people can build a reader/writer program to read your file
• A vendor will not take advantage of you, aka over pricing software etc
• A file(pictures,email etc) belongs to you, don't let people stop you from reading it.
  

So tell your friend, your co-worker, your boss, family, everyone about the importance of Open Standard, and Open Document.

http://www.documentfreedom.org/

Post TEDxKL

First thing first, i forget to bring my camera along, EPIC FAIL....
so bear with me

For those that follows TED, TED is a conference with the tagline, Idea Worth Sharing. TEDx is a TED, but independently organized. KL already have 2 TEDx, this year it is held in Microsoft Auditorium in KLCC, which have a fantastic view of KL(sorry no pics)

This year the speaker comes from a variety of background, includes mountaineer, ceo, designer, teacher, etc.

One can't help but immersed in the stories and photos of mountains climbed by Surech Kuppusamy.
Touched by Yvonne Foong on her story with Neurofibromatosis
Amazed at things Peter Nicols on helping the homeless + poor people in KL.
Entertained by Afzal Abdul Rahim's eye opening talk on Malaysian Broadband.
Enlightened by the a few piece of history of Graphic Design in malaysia by Ezrena Marwan
Knowing how the mind works with Urmilah Dass
Look into the world via photos of Aloha Livina.
and finally a reminder of thing that malaysian should do by Pete Teo.

All the talk is enlightening, sure open my eyes to the world. It is certainly a event worth going. I am looking forward to the next one.

p.s To the organizer of tedxkl, nice work for everyone. Looking forward to the next one

From tedxkl

We got a good crowd down here, we have the usuals from barcamp malaysia. And interestingly a few guy from python.my

Then we have techies and other demography.

Things will be interesting here tedxkl

p.s sorry for no pictures, probably will link to someone's.

btw the tag is tedxkl, the hashtag for twitter is #tedxkl

http://twitter.com/#search?q=%23tedxkl

Why some project demo feels like a first date

Based, on the feeling i have now, because will meeting with a client, with a prototype,

1) First impression is important
2) You need to touch up before show up
3) You need to make sure everything works as planned
4) Elaborate preparation is needed
5) You don't have to feel good, but you need to look good(the program anyway)
6) Thing will never works well.
7) you might ends up getting a long term commitment
8) or getting ditched within first hour..

FAQ for Chinese/Lunar New Year 2010

FAQ on CNY

 ****Will definitely change during the get together*****

0. About this FAQs
1. On relationships

2. On career

==================================================
0. About this FAQs

0.1. What is this FAQs about?

This is to answer, the common question asked by relatives during the New Year Get Together

0.2. Why?

Because, it is usually the same question, and I usually give the same answer
(truth, it is mostly for the lulz, I hope)

0.3. Disclaimer,

It is meant to be funny, nothing serious. Which most probably fail........

==================================================

1. On relationship

1.0. What is the usual question on relationship?

- When is your turn to give ang pao?
- Got girlfriend or not?

1.1 When is your turn to give ang pao?

-  Not Now, Bad economy....

1.2. Got girlfriend or not?

- Got someone you can introduce to me?

1.2.1. Why still not have girlfriend?

- Because of here and here 
- Because there is a bigger change of winning toto(so I concentrate on that instead)

================================================
2. On Career

2.0 What is the usual question

- What are you doing now

2.1 What are you doing?

- Programmer, writing codes
- print "programmer"
- #("ongettogether").html("programmer");
- Let see how I can explain it in lay person

2.2 How much do you make?
- Not enough to give out an ang pao :-(
- Which is why the angpao is really important!!!


Now I hope to see FAQ from everyone else that celebrate the lunar new year

django, solr and a few interesting stuff Part 2 : Haystack....

****continuation from the last story******

As the Adventure Continue

One of the nice thing about django, it have all sort of application. Haystack is such a application. Not just haystack support solr, it also support a few other backends. This include whoosh, and xapian.

The fun part about this, it is quite pleasant to work with. So we redo all of out previous code to work with this.

Let The Game Begin

Again the tutorial is helpful, it pretty much describe what i am have done anyway. And the documentation covers a lot!! So I will skip the setup, and index.

After following the tutorial to setup.And modify the search_indexes.py for each models that we need to find. And generate the solr index, and copy to the solr/conf.

We basically have a django application ready to do search. To test, is a matter of going to shell.

from haystack.query import  SearchQuerySet

q = SearchQuerySet().all()

of course, you can also, do

q = SearchQuerySet().filter(fieldname='value')
for i in q:
      print i.object.value

It is similar to django queryset api. Except it is for search. You can do a lot more, like faceted search, chain the filter, etc.

Are we There Yet. 

Not quite, there yet. Because haystack also have a search page, which you can enable, also covered in the tutorial. You can inherit from the search view, and forms to fit your need.

An example is for a form that I need(it is generic anyway)




So you don''t even need to create your own form, and views.
Just import the views, and change the parameter



haystack is pretty comprehensive, have many stuff done for us, from views, forms that is functional. Doing indexing, generating index for solr, etc. I think this is a nice project to use in django for providing search in your web app

Django, solr and a few interesting stuff Part 1 : Solr power!!!!

****Long story alert, no code, but tech related, jump to lesson learned in the end*****

The Beginning

One of the thing I am working on now involving solr, django and a few thing. Along the way we discovered a haystack. 

So the story started when we found a project, that generate cash. After talking to the client, and looking at their system. We decided to try, and we decide build it using django, and solr, among a few component. Though we did not confirm that we will offer search for the first place...

To keep it short. Thus we begin to build it. 

To Serve Just java -jar start.jar

We started with solr, because it is easy to install anyway. Just grab solr from http://lucene.apache.org/solr/. 

What really cool about solr, within the solr folder, is example folder. That is a fully functional solr project. 

just cd into example folder. and run java -jar start.jar. Follow the tutorial to use the example folder

Since we are a lazy bunch, so we just copy the example folder into our project folder. And modify schema.xml in example/solr/conf/. Since schema.xml in the folder is usable, we just modify the fields. 

Roadblock

Since this project original database have quite a lot of table. So FIRST STEP is denormalize the data. Flatten the whole thing. 

And we decided to use solrpy for our script to load the data in database into solr. Which look ok until we have a situation of 1 to many relation in the database. In solrpyr, each repetitive field is put into a list and added to solr. 

This is not the worst issue. When we pull the data from solr. The field is not in order. Thus, it doesn't maintain the structure.

Thus we learn, JUST store the ID or primary key in SOLR, so that it can be refered to the database, just use the KEY to pull data from database. 

Then after many trial of error, to detect empty field etc. 

We give up, we discovered haystack!!!

Lesson Learned in Using Solr

1) Denormalized all fields to be stored in Solr.

2) Sometime, solr output does not reflect to the structure of the database(ok, we are very new in solr)

3) So if it is from a database, indexed everything  and store the key(probably a bad idea, because we are not to hit the db a lot, i don't know really)

4) Don't reinvent the wheel. Turn out that haystack have solved our problem and more.

5) Partially number 4, the client just dump us a file outputed in database, we tried to be hero, but turn out better to store in db first....because it is a mess processing the data..

listing open file with lsof

So the story goes that I need to process (quite) a number of files..
Which I didn't write  a output to say what is being process..

lsof save that day. lsof can monitor file being open, which includes sockets.
For my case, the usage is easy

lsof +r -p  

The -p is for opening process, +r is to repeat until no file is opened by the process.

enjoy