Copy and Paste on GNU Screen

So recently I finally found out how to do copy and paste on gnu screen, previously I just do a drag on a mouse.

So copy and paste on gnu screen,

- ctrl-a [ 

- move to the first character of the string to copy, press enter

- move to the last character of the string to copy, press enter

- now to paste, press ctrl-a ]

p.s kina ashamed that it take me so long to figure out =.=

Converting PDF to Text with pdftohtml

Previously I have tried to extract pdf information by converting PDF to text, as described here.

Problem is,  a big wall of text is very hard to process.
Here come pdftohtml it is part of the poppler package on linux. But gnuwin do not have it for windows. Which is one reason I use pdftotext.

pdftohtml convert pdf to html. simple usage is

pdftohtml yourpdffile.pdf

You will get your html file. But it is a bit plain as they just extract text from it. It there is image inside pdf, or you pdf is pretty complicated, like Malaysian Hansard. You can use the -c

pdftohtml -c yourpdffile.pdf

Here is the catch, it will generate 1 html per page in the pdf, with images. But the layout is maintained. For document like Malaysian Hansard, it would be hundreds of page.

Then there is way to produce xml

pdftohtml -xml yourpdffile.pdf

You will get an xml file which the position information.

p.s I'm using this for http://opendataday.org/ Whether there will be result today. 

 

 

Converting PDF to Text

So I have recently involved with a project to extract data from PDF. Which is actually evil, but that is not important now.

On linux there is a set of utilities comes with xpdf program. It should be part of the default package installation, if not, you just apt-get or yum it.

On windows you can go to the gunwin32 page, I just download the zip just so i would not have to remove it with a uninstaller.
http://gnuwin32.sourceforge.net/packages/xpdf.htm

I don't really need the layout information, on it. so I just use pdftotext.

On windows

program_location/pdftotext.exe -layout pdf_file.pdf

On linux, just

pdftotext -layout pdf_file.pdf

The -layout would maintain the layout of the text as from the pdf. Otherwise, the positioning for certain text will be inconsistent.

Cheers

Accessing Server from Android

Recently I help maintain some server, sometime I tend to move around. So I decide to make my phone to be useful.

Android actually have a couple of app that is useful to remotely access a machine. Some of them is free.
For connecting to SSH, I found that connectbot works extremely well. It only does ssh and telnet, and thats about it. It is pretty straight forward to use. For accessing windows server, I use 2x client. Which again another another straight forward RDP client. Both connectbox and 2x client is free, and that is awesome.

The only issue on using android phone to access a server remotely is. I have a desire hd. While the screen is pretty large for a phone, typing command via ssh or, navigate around a windows server via RDP can be still a pain. It is still smaller that most desktop screen. And I don't have a full size keyboard on the phone. Which is another pain especially I access linux server most of the time.

So it can be a pain to use at time. But for quick fix or checking on server. This work pretty well.

I attached some links for the app below

2X Client RDP/Remote Desktop for Android on AppBrain

ConnectBot for Android on AppBrain

Many Ways To Grep File Content

So not too long ago I have posted on twitter

This spin to a few other way to do grep.

A few have suggested on IRC and facebook, the i parameter is to make keyword not case sensitive.

grep -iR keyword directory 

Another suggestion on IRC. 

grep -iR --exclude=file-to-ignore keyword directory

Another tweet i have receive is,

Then the last one I discovered on google is ack-grep

ack-grep keyword directory

and again, -i make case insensitive search.

ack-grep -i keyword directory

ack-grep output is nicer, and automatically ignore binary. It is slightly different than grep. But both get the job done., to me anyway

Diff'ing and Patching

So I got like a few folders with different version source code, erm text files which I am comparing in my work. Either way, thanks to unix tools like diff and patch, things is some what easier. Though it is better to use version control, which I stupid enough not to

to compare files with diff, it is just a matter of

diff original new

then my team is creating the file on windows, which have \r\n, and me on linux, which \n. So to avoid this on diff

diff -w original new

this should able to ignore difference in white space character such as \r\n, which may not be a wise choice for python, because of the indentation sensitivity, which I trust my team sane enough not to mix tab and space(yes, for non-python progammer you can all laugh)

Since I have a directory, so i would run

diff -rw original-folder new-folder

pipe it to less just for making it readable.

If i'm lucky, it is just adding stuff to the text file, i would generate a patch, for that file by

diff -u original new > original.patch

and applying it with

patch original < original.patch 

in the folder. which i should really use -p1 option in the patch command, but there is a few folder have the same name, so i just put it in that folder and run it without -p1

of course sometime thing is not as easy. so it is nice to use to see all the difference with highlighting. which here i use vimdiff. which i use

vimdiff original new

Then I realize that doing all the diff and patch can be a pain in the back. So, a better use the code with version control, what ever reason, or how rush it is.

Fun with Linux: Taque

So i got a remember the milk account. What's cool is, it is a nice web based, todo list. What really sucks is, there is no desktop client on linux. Until now.

Tasque is a todo list manager, it have a local file backend, and most importantly, it can use remember the milk, as backend.

It is pretty easy to use, just enter task, and it is done. One interesting thing is, you can enter "do something today" it will automatically, fill in the date for today.

What really sucks is, it don't have notification. But it is useful enough, to use.

To install on ubuntu,
sudo apt-get install tasque

But i can't seems to get it work, with fedora. Enlighten me please

Is loving screen

So I got a job, which allows me to develop on linux...(w00t!!!).
Along the way, I have found screen....

screen is a full screen windows manager that, multiplexes several process. Usually active shells. As quoted from the man page.

The thing is, it is useful that, I can run several shells. Without actually have to open several instance of gnome terminal. Which is useful, because I use a lot of vim nowadays. And really have to open several windows just to have several terminal tend to be distracting.

screen is not hard to use, 
Ctrl-a + c        : to open new windows
Ctrl-a + n        : switch to next windows
Ctrl-a + p        : switch to prev windows
Ctrl-a + Ctrl-a: toggle between 2 windows
Ctrl-a + ?        : is the help(if you don't remember the rest, just remember this one)

BTW I found that this is useful for my eeepc as well. Consider that, I found that console apps tend to save a lot of screen space...And too much windows open, then to be hard to navigate.

Learned how to use rsync

So I need to sync a folder between 2 computer my laptop, and PC.
So one way is to use scp, but I don't quite want to replace the folder. Just files I created. I just want to make sure the folder is in sync.

So I play with rsync. Which track changes of the files, and transfer the changed file only. From what I understand from wikipedia anyway.

And what so cool.The command is similar to scp, or cp. So the command for rsync is:

rsync source dest #for files
rsync src@hostname:source-file dest #from remote host to local file
rsync src dest@hostname:dest-file #from local file to remote host
rsync -r src dest #this is to transfer folder

So just change source with files, hostname with hostname or ip.

So another tools I learned as a newbie......

Linux Emergency: entering Single User Mode in fedora

So I installed ati driver on Fedora, BAD IDEA. Because, once you finish booting, you cannot even call virtual console like alt-f2.

Solution is to access Single User Mode. Then run the uninstall script.

Actually single usermode is not limited to this, it is meant to be used for some of the worst case emergency.

To go to single user mode
Once rebooted,
1) Go to grub menu
2) Select the option to boot into linux
3) Press a
4) it should shows something like this

kernel /vmlinuz-2.6.29.4-167 ro root=LABEL=/ acpi=on rhgb quiet

Just add "1" behind it

kernel /vmlinuz-2.6.29.4-167 ro root=LABEL=/ acpi=on rhgb quiet 1

5) Press enter

You should be able to access the shell as root without login. Now you can do anything to fix your problem.

This is dangerous,because you can access everything, yes everything. Enjoy!!!

lesson of the day: proprietary driver cannot be trusted......

So I finally installed fedora 11, upgrade using preupgrade. (YAY!!!)

So first thing first, DESKTOP EFFECT, then I check whether it is supported,
For open source driver, it support 2d acceleration, which is sad
http://cgit.freedesktop.org/xorg/driver/xf86-video-radeonhd/plain/README

So I got grab driver from ATI......Bad idea.
http://support.amd.com/us/gpudownload/linux/Pages/radeon_linux.aspx?type=2.4.1&product=2.4.1.3.5&lang=English

Installing is easy, but once I finish install, reboot, it render the system unusable.

Thanks to ditesh, on twitter, Problem is solved. That would be another blog post.

I still hopeful that open source 3d driver will be here. For the graphic card that I own anyway.

Upgrading fedora aka lesson of the day

Yesterday, I am downloading the whole image to upgrade my fedora installation.

I just learned that, I can just use Preupgrade

Just run preupgrade, as root

Enabling Telnet(ssh) to a DNS-323

I actually own an NAS, a DNS-323. Which happens to runs linux. It just the system is locked down.


I don't really do much except to stash my stuff on it. Until now. Turns out that there is a community around this device, and they have done impressive stuff on it.

Most of it revolve around fun_plug.

To use it is. Essentially, Download fun_plug and fun_plug.tgz from here  .
And copy to Volume_1

`

And reboot

In the end of the day, you will able to telnet to the NAS, as root without password.

So you should run in telnet, pwconv,

pwconv
passwd
usermod -s /ffp/bin/sh root

pwck
grpck

login

if it is working, run the command in telnet to store password.

store-passwd.sh

        
You should really enable ssh, in telnet.

cd /ffp/start
sh sshd.sh start

And you should at least enable login for telnet, go to /ffp/start/telnetd.sh, find and remove '-l /ffp/bin/sh'.Use vi to do that..

Now we have a linux machine finally look like a linux machine, use shell, and ssh. That's not all.

The community have a few package to be used in funplug, you can investigate here, at middle of the page. They have ported backupPC, lighthttp, python etc.

So we have a full scale linux machine, and due to GPL compliance, dlinks have the tool chain plus kernel open to us http://wiki.dns323.info/howto:crosscompile. So we can cross compile software to this device too. And actually install debian, but with risk of bricking.

Have fun!!!!

linux community promotional effort

What I noticed is there is a number of promotional effort by the Linux community in malaysia. Usually by the same group. In the malay language

Here is the links for their slide show in
http://www.jomlinux.com/v1/index.php/20090208211/Artikel/Berita/Slideshow-dari-e-kedai-Jom-Linux

They also have a video on how Ubuntu linux is used in the Parliment Office in Ampang.
http://www.jomlinux.com/v1/index.php/20090203209/Artikel/Berita/Dokumentari-Mini-Ubuntu-Di-Parlimen-Ampang

My Adventure with Photorec

The whole story started when my cousin ask me to restore data on her broken hard disk, I don't think it is broken btw, more like corrupted partition. Either way, I still need to recover her data.

That's where photorec comes in. Photorec is a tools to recover data from harddisk, it is a program comes with testdisk. Don't be fooled by the name, it can recover documents, video, mp3 etc, not just photos.

First you need to find the partition, you can just type in terminal

fdisk -l

Then in terminal

photorec /dev/

replace device name with sdb, hdb etc.

Then just follow instructions.

And there you have it, it will create a folder on where you create it, with all the data. It is quite easy to use.

To install on ubuntu

system -> adminstration -> synaptic package manager

 Find testdisk

On fedora

system -> administration -> add/remove software

search testdisk.

BTW there is a windows version. But i test this on ubuntu.

P.S This actually is a bit anticlimatic, I actually expect to create a disk image, and use sleuthkit. photorec is much more easier. That atchek, hehe.

 

Sharing Keyboard and Mouse using synergy

Some time it is nice to actually able to share keyboard and mouse between my laptop and desktop. 

 

The good news is, synergy actually allows such functionality, and it give the ability to share the clipboard too, meaning you can drag and drop between 2 computer. 

To start using synergy

On ubuntu go to the menu

system -> administration -> synaptic to install it. Or you can just

sudo apt-get install synergy

On fedora go to menu

system -> administration -> add/remove software and search synaptic

Or you can just
yum install synergy

To start using synergy
create a text file called synergy.cond

section: screens
        comp1:
        comp2:
end
section: links
        comp1:
                right = comp2
        comp2:
                left = comp1

end

change comp1 into the name main computer or the server, and comp2 into the secondary computer

On your main machine, open a terminal and type

synergys -f synergy.conf

On the second machine

synergyc -f 


Now you can share keyboard and mouse, with another machine.
BTW there is a windows version too. 

http://synergy2.sourceforge.net/

this article originally in mandarin, on myclug.my, written by me

http://www.myclug.my/wordpress/?p=74 

A useful tools: Powertop

One problem with my eeepc now, is that I didn't get the battery life it claim, on windows and linux on the eee pc. That is 8 hours. Currently it is only 4.5 hours.

One way to figure out what is using the power is, using powertop. It is a open source tools to use to display the process that is using the power, and give recommendation. Which is cool, because, it actually very convenient. But of course, you really need to do it manually, because the solution is missing after a reboot.

For ubuntu user, just

sudo apt-get install powertop

And powertop need to be run by root.

Fun With Linux: bootchart

It is another day where I have a little adventure with my new toy, my eeepc 1000h. What I don't satisfied is the boot time, I got around 40 second on ubuntu on ubuntu using the adam's kernel.

So before to optimize, we need to investigate.

Bootchart is a way to show the boot process, in the end of the boot, you will get a chart, that shows the process.

To install on ubuntu

sudo apt-get install bootchart

The chart generated from the boot process, is stored in /var/log/bootchart. It is a png image.

So the result that I get, from my setup

It is one thing to know, now, how to optimise it?

Adventure with eeePc 1000h, using ubuntu 8.10 intrepid.

Got myself a laptop finally, it is a eeepc 1000h, comes with windows xp, the fun part is, I got it slightly cheaper than most, and 2 gig of ram, which added extra, also 2 year warranty, for RM1845. At lowyat.

It is a machine, that comes with windows xp, which I format it. And I install ubuntu on it.

Step 1:
Download ubuntu, then copy to a pendrive, using unetbootin. The instruction is on the page.

Step 2:
Plug in the pendrive, press  ESC key, when you turn on the computer, then select the pendrive, you should be able to boot into ubuntu.

Step 3:
Setup ubuntu, that is straight forward.

Step 4:
Now install the custom kernel. Using array.org kernel. After this, the wireless should be working. And other hardware too, to be honest, didn't test the camera.

Step 5:
Install the acpi script. Just to make it work nicer.

Step 6:
Now one more thing didn't work well, the sound is not loud. Download the alsa upgrade script

Step 7:
Install netbook remix.

As you see, I didn't use anything that is special, just use instruction from the web. And it works well.

Adventure with fedora: day 1

So I installed fedora. Not quite a new user in fedora. Just not use it as much as Ubuntu. Below is my fedora desktop, modified.

First thing I notice is that. It didn't come with open office. Which is interesting. While I don't quite rely on it. I pretty much do all of my work in office. They tend to be crazy when it comes to data security. Well, abiwords is there.

In a way, fedora sounds like linux few years back. When codec have to installed, third party repository have to be added. Which I could understand why. It just that ubuntu mitigate it by having a way to install it more easily.  And it applies to flash as well.


Packagekit is nice package manager. But a bit more simplified. Probably I just get used to synaptic. One difference is in ubuntu/debian, there is meta package to install group software. But on fedora it is group install.

But like linux nowadays, everything works well, in term of hardware. Except ATI is still a pain in the back. Linux is not hard anymore, from a guy who use linux for few years.


There is more to share on my observation on fedora. So there will be more to come