Why some project demo feels like a first date

Based, on the feeling i have now, because will meeting with a client, with a prototype,

1) First impression is important
2) You need to touch up before show up
3) You need to make sure everything works as planned
4) Elaborate preparation is needed
5) You don't have to feel good, but you need to look good(the program anyway)
6) Thing will never works well.
7) you might ends up getting a long term commitment
8) or getting ditched within first hour..

FAQ for Chinese/Lunar New Year 2010

FAQ on CNY

 ****Will definitely change during the get together*****

0. About this FAQs
1. On relationships

2. On career

==================================================
0. About this FAQs

0.1. What is this FAQs about?

This is to answer, the common question asked by relatives during the New Year Get Together

0.2. Why?

Because, it is usually the same question, and I usually give the same answer
(truth, it is mostly for the lulz, I hope)

0.3. Disclaimer,

It is meant to be funny, nothing serious. Which most probably fail........

==================================================

1. On relationship

1.0. What is the usual question on relationship?

- When is your turn to give ang pao?
- Got girlfriend or not?

1.1 When is your turn to give ang pao?

-  Not Now, Bad economy....

1.2. Got girlfriend or not?

- Got someone you can introduce to me?

1.2.1. Why still not have girlfriend?

- Because of here and here 
- Because there is a bigger change of winning toto(so I concentrate on that instead)

================================================
2. On Career

2.0 What is the usual question

- What are you doing now

2.1 What are you doing?

- Programmer, writing codes
- print "programmer"
- #("ongettogether").html("programmer");
- Let see how I can explain it in lay person

2.2 How much do you make?
- Not enough to give out an ang pao :-(
- Which is why the angpao is really important!!!


Now I hope to see FAQ from everyone else that celebrate the lunar new year

django, solr and a few interesting stuff Part 2 : Haystack....

****continuation from the last story******

As the Adventure Continue

One of the nice thing about django, it have all sort of application. Haystack is such a application. Not just haystack support solr, it also support a few other backends. This include whoosh, and xapian.

The fun part about this, it is quite pleasant to work with. So we redo all of out previous code to work with this.

Let The Game Begin

Again the tutorial is helpful, it pretty much describe what i am have done anyway. And the documentation covers a lot!! So I will skip the setup, and index.

After following the tutorial to setup.And modify the search_indexes.py for each models that we need to find. And generate the solr index, and copy to the solr/conf.

We basically have a django application ready to do search. To test, is a matter of going to shell.

from haystack.query import  SearchQuerySet

q = SearchQuerySet().all()

of course, you can also, do

q = SearchQuerySet().filter(fieldname='value')
for i in q:
      print i.object.value

It is similar to django queryset api. Except it is for search. You can do a lot more, like faceted search, chain the filter, etc.

Are we There Yet. 

Not quite, there yet. Because haystack also have a search page, which you can enable, also covered in the tutorial. You can inherit from the search view, and forms to fit your need.

An example is for a form that I need(it is generic anyway)




So you don''t even need to create your own form, and views.
Just import the views, and change the parameter



haystack is pretty comprehensive, have many stuff done for us, from views, forms that is functional. Doing indexing, generating index for solr, etc. I think this is a nice project to use in django for providing search in your web app

Django, solr and a few interesting stuff Part 1 : Solr power!!!!

****Long story alert, no code, but tech related, jump to lesson learned in the end*****

The Beginning

One of the thing I am working on now involving solr, django and a few thing. Along the way we discovered a haystack. 

So the story started when we found a project, that generate cash. After talking to the client, and looking at their system. We decided to try, and we decide build it using django, and solr, among a few component. Though we did not confirm that we will offer search for the first place...

To keep it short. Thus we begin to build it. 

To Serve Just java -jar start.jar

We started with solr, because it is easy to install anyway. Just grab solr from http://lucene.apache.org/solr/. 

What really cool about solr, within the solr folder, is example folder. That is a fully functional solr project. 

just cd into example folder. and run java -jar start.jar. Follow the tutorial to use the example folder

Since we are a lazy bunch, so we just copy the example folder into our project folder. And modify schema.xml in example/solr/conf/. Since schema.xml in the folder is usable, we just modify the fields. 

Roadblock

Since this project original database have quite a lot of table. So FIRST STEP is denormalize the data. Flatten the whole thing. 

And we decided to use solrpy for our script to load the data in database into solr. Which look ok until we have a situation of 1 to many relation in the database. In solrpyr, each repetitive field is put into a list and added to solr. 

This is not the worst issue. When we pull the data from solr. The field is not in order. Thus, it doesn't maintain the structure.

Thus we learn, JUST store the ID or primary key in SOLR, so that it can be refered to the database, just use the KEY to pull data from database. 

Then after many trial of error, to detect empty field etc. 

We give up, we discovered haystack!!!

Lesson Learned in Using Solr

1) Denormalized all fields to be stored in Solr.

2) Sometime, solr output does not reflect to the structure of the database(ok, we are very new in solr)

3) So if it is from a database, indexed everything  and store the key(probably a bad idea, because we are not to hit the db a lot, i don't know really)

4) Don't reinvent the wheel. Turn out that haystack have solved our problem and more.

5) Partially number 4, the client just dump us a file outputed in database, we tried to be hero, but turn out better to store in db first....because it is a mess processing the data..

listing open file with lsof

So the story goes that I need to process (quite) a number of files..
Which I didn't write  a output to say what is being process..

lsof save that day. lsof can monitor file being open, which includes sockets.
For my case, the usage is easy

lsof +r -p  

The -p is for opening process, +r is to repeat until no file is opened by the process.

enjoy