Full text search – bachelor’s thesis

Long after the bachelor’s exam I am uploading my bachelor’s thesis project, as I promised some time ago.
It is a desktop text search engine, written entirely in C++ and the Qt libraries, with all the code related to scanning, storing and searching the information written from scratch and only using the basic C++ and Qt data structures.

You may download the torrent (http://thepiratebay.org/torrent/4538524) and can also find it here, on a thing called sendspace (first upload service that Google revealed, I don’t know if it is working).
I will upload also the paper when I’ll find it, altough that is not a big thing either.

The archive contains:
-Windows executable + Qt libraries
-Linux executable (debug build).
-All the source code.

The scanning speed is CPU bound, and it is incovenient to fix. It toped at around 1MB/s on the old system and 1.5MB/s on the new one. The space overhead tends to around 13% of the scanned contents. The search is fast and can use simple AND, OR and exact phrase combinations. I even used it for a few weeks until going back to Recoll.
The source code is GPL v2+, as Qt requires. There is no license notice on each source file but nobody will die from that. Try the tree viewer as I consider it very nice for the small quantity of time I put into it.

The blatant limitations and defects:
-the scanning speed;
-the inconsistent and unfinished threading + crashes.
-the bad and unfinished user interface;
-the overall usefulness is by far not that of a real program anyway.

It comes so late as I did not have a Windows machine or a cross compiler until now, and I finally installed a Windows copy in VirtualBox. That’s enough for such a small project.
Note:
Remember to add a screenshot tomorrow .

Using the microphone in Debian – I hope Windows XP works on my laptop

Update: Using the ncurses program “alsamixer” I was able to get decent settings and make the mic. work right. Unlike KMix and Alsamixergui, changes made with alsamixer were applied. It is also keyboard friendly (it’s stupid console in fact, couldn’t be otherwise). I’m still thinking about other OS, this was too big a hassle

VoIP is finally reaching us, and offers prices that are unbelievably good. Buy credit from a VoIP provider (TerraSIP in my case), start a VoIP program (WengoPhone in this Debian case), do some settings to the program and call home. I have mention that quality is also surprising, I guess that in more than half of the case it is better than phone to phone. Being abroad this means very important savings.

And I did this, until Debian sabotaged me. To make the microphone work correctly on this laptop, a fair bit of experimentation is necessary, I did that and everything worked. After some time, with no conscious ALSA related changes on my system, this doesn’t work anymore, and the sound is no longer clear. It now misses all the higher frequency sounds and transforms them in squeaks.

The bad thing is that I don’t know where the problem is coming from, and more, I have no idea how to solve it.

Here’s a picture showing Alsamixergui with the best settings I could get. The program has tons of controls and none of them is explained. Googling didn’t give me too many answers. Some of the controls cannot be used (although KMix uses them, but to no effect), and also the range of possible combinations, without using a manual, is frightening.

So counting the sacrifices done for running Debian:
-no 3d acceleration -> No playing CS with friends. No other interesting gaming. I never got too far with Blender to feel the need for real 3d, but I will.
-problems with opening fancy shit office documents, not using proprietary math software from the university, others like this -> A few bad marks and some time wasted.
-calling at home from abroad with prices as much as 15 times higher than the VoIP ones. Well, this is the hot point, the costs are directly measurable as money. I guess now I have enough reasons to leave the Debian away.

So, where to?
I guess Windows XP/Server 2003. (The Vista saga is also written on this blog, and it is also not happy.) After about 3 years of running Linux on an underpowered desktop (but at least with good sound ), I have to give up goodies like KDE+Kwin, a decent console, nice package management from Debian, easy development with Qt and easy plugging in of external libraries, the Xorg middle click paste and easy switching of users and desktops. The good thing is that Amarok, Kate, Konqueror, KOrganizer and others make their way to Windows, I will be able to use the decent Visual Studio debugger and some others.

There are alternatives, but with shortcomings:
KUbuntu, I’am afraid they will try on me stuff like PulseAudio, the kernel TCP/IP bug (they didn’t release this one after all), and other half working thinks.
SuSE looks attractive, but it seems they said goodbye to KDE 3 and I guess there are not as many programs as with Debian.
Mandriva. My all time favorite, although the lack of proprietary packages (and also free ones) makes it hard to use.
And also, there’s no guarantee the sound problems will go away, while with Windows I have high hopes they will.

Oh, and later random thoughts:
-the man/woman/guy/dude (I didn’t dig to see who he/she is) from linuxhatersblog said he would stop. I still think he knew a lot more about Linux than I do.
-I still hate AMD/ATI for tricking me into buying a laptop with a radeonhd “supported” shit video card. I hope they will get their shit together though, as competition is always good.
-I will ask some help from some ALSA guys before leaving Debian. I would do some coding myself, but it’s a bit beyond my power.
-I think this message has a record number of “shit” words, I think 5 if we include this one. There is no f-word though. I hope there’s none on this whole blog.

Running this recovery DVD will delete all data on your disk

Not long ago, I had to buy my first laptop, and after spending lots of time thinking about the right decision, I made the wrong one.

Finally, I went to the shop and made the decision based on:
-(screen size + memory + hardware performance + warranty) / price;
-looking capable to run Linux;
-having AMD/ATI graphics, as these guys sponsor free video driver development.

The though point:
1.The recovery DVD from Asus.
Later after buying the laptop I erased the vfat recovery partion and made the C: smaller. This was a mistake, as now Windows is not booting anymore (I only found out today). I can imagine some reasons, but from the point of view of the stupid user, this is a Microsoft screw-up. If the Windows partition is bootable by itself, why does it need to find information about anything else?. This was a strong point of other Windows versions: just select the partition, and bang, it works. As somewhat experienced luser, I am able to repair a Linux boot, which used to be somewhat hard, but I have no idea how to repair the Windows one.

With the laptop I also received a recovery DVD. I ran it thinking that in 2 minutes I will have Windows repaired with no other complications. Here comes the big blow: “All the data on your partition(s) will be deleted. Asus shall not be held liable…”. This text was written on the _second_ paragraph of some long enough message. My mouse left button was already pressing the “Next” button, waiting for the release that would go to the next step. Only some masochistist curiosity for reading long messages made me skim through the text that predicted havoc.
Thank you Asus for such a lovely experience.

The rest:
1. Beginning, partitioning and Vista from Asus.
I knew that the laptop comes only with 32 bit Windows Vista, I tried not to pay for the OS, but I did not manage to do that, I’m too lazy to ask for a OS refund, I might try a refund for the entire laptop though, and you’ll se why.
The first surprise was partitioning, 20 GiB vfat partition for recovery, around 120 GiB C: with Vista. This only left me around 90 GiB for other stuff. What was worse, is that the Debian installer was not able to resize the oversize C:, so I had to accept the situation and go with small Linux partition for my data. This will come and punish me in part 3.
The other thing is that the OS was configured such that the Antivirus + Security Manager (Symantec crap I think) was always asking me to register, had an unkillable window and kept asked things I found difficult to answer. There where also other programs that started with the computer and made the disk seek like mad and keep the computer slow for minutes after startup. Also, I think was not entirely Asus’s fault, as the OS was also doing huge unknown jobs on its own.
The good thing is that the OS came with disk burning software preinstalled.

2. Linux support.
2.1)Video. “This comes with modern discrete ATI graphics card, Radeonhd will sure do.” False. Radeonhd does not work on my laptop at the moment. Radeonhd is the only driver that does not work.
Fglrx makes 2D feel slow, takes ages to start Xorg, and hard crashes the computer at each logout. Only the open source ATI (2D only) driver works fine.
The Debian installer left me with the vesa driver, that only gave me small resolutions, anything better was on my own. It was good though, as X worked from the beginning, unlike other installs. So I had to do xorg.conf configuration by myself until I got the right results.
At the moment Nvidia is a dream for me. I don’t know what they do, but their shit works.

2.2)Wi-Fi. “Linux is the Chuch Norris of Wi-Fi.” False. I had to do good googling to find out that I need to install (from the repository, fortunately) some firmware. Also worth bo be said, NetworkManager seems to take longer than Vista to connect to wireless networks and it does always simply work. There are two graphical interfaces to NM: KNetworkManager that is not always working correctly; nm-applet, which works correctly, but looks like shit and does give me enough control. NM is strange anyway.

2.3)Sound. “Everyone does sound.” False. a)The internal speakers do not mute when inserting headsets. b)I had to trial and error flip the switches in the ALSA control panel until I got working microphone. c)Not all programs are able to access the microphone as they should. Some get the full volume, others only inaudible volumes, no matter what I do. Skype seems to be the only that will get good quality sound, for reasond I don’t know.

2.4)Suspend to ram. “That’s a done deal, long ago.” False. Does not work, I can’t say why, but at wake-up it used to reboot, after some update it goes to a hard crash.

2.5)Others.
Included camera. Not very important, but that is not working with most programs either.
Blutetooth. I was not able to get that working. I would have a use for it, but not important now., and I didn’t even know the laptop was capable when I bought it, so this is only and unreachable bonus.

Kate on Windows

The KDE developers do a great job, as always, and they are moving the KDE 4 programs to Windows.

I use Kate for all my text editing (I don’t do office, I hope I never will) and I am quite dependent on its shortcuts and facilities. Coming back on Windows at work, I have to choose something to suit my needs. Notepad++ is a surprinsignly good editor and this is what I currently use, only its silly name kept me from trying it. What Notepat++ is still missing is a better autocomplete, some smarter code folding, some code formating options and a few more configuration options. I also tried Jedit and a few others. I don’t count in the commercial editors, which may be even smarter.

So here is Kate on Windows:

Unfortunately it will take some time until a stable version comes out, the current one doesn’t really work, it only managed to open one file from more than 10. People who want to give a hand in testing KDE, this is your chance.

What is really nice is the way the KDE installer works. Coming from the Unix world, where people put effort into having nice packaging, KDE offers a Cygwin-like installer that gives an interface for choosing which programs to install, where to install and so on, and from how the interface looked, it am almost sure it could also do the updates.

I shall come with back with some details in installing and configuring Cygwin, Launchy, Katmouse and others, in order to make these pesky Windows computers be a bit more tolerable.

For those who will say about XEmacs/Emacs and Vi, I did try XEmacs a few times, but:
1.It is totaly different from conventional editors that I would have to spend quite some time to get used to it
2.Emacsen are somewhat ugly and non standard on the desktop, with the current ones not looking like the rest of the desktop, and the previous ones even having some really nasty menus and scrollbars.

Hello?! Display producers, you can continue evolution now!

For quite a while the pixels per inch value (PPI, very similar to DPI) of monitors has stagnated. In fact, if I remember correctly it has somewhat gone down (when I first bought my first display I could find a 19″ 1600*1200 lcd, now I can’t, I might not remember correctly though).

So here we are, today we have the same 96 PPI we had maybe 10 years ago (someone please confirm), though IBM made a 204 PPI monitor in 2001. Some things improved, but colors and PPI are mostly the same.

The first source of information on the problem is on Anti-Grain Geometry page about fonts: http://antigrain.com/research/font_rasterization/. On a discussion on sayit.ro someone has pointed out that the Windows desktop would be very hard to use with larger PPI because of the way it scales its contents (and so producers would have nothing to gain from providing monitors that few would buy).

The question was if Windows Vista is capable of producing nice results. Not (wanting to) having such a system I had to google, and it seems Vista does the job: http://www.istartedsomething.com/20061211/vista-dpi-scaling/.
Notice the details of the icons in the taskbar at the 480 DPI. They are lovely.

I had too see how my desktop would do, and the results are mixed:
Default, around 96 DPI:

200 DPI:

480 DPI:

(I have only a KDE 3.5.5 for the moment, so please someone give links to some screenies with Gnome and KDE 4.)
As you can see the text is ok, menus, standard buttons and the window decoration work well, the rest not really.

There are some things to note though, it appears the Vista Aero thingy is does things a bit different from what KDE programs do. Vista uses its compositing engine and keeps the correct proportions, like vector graphics, while the KDE programs have to resize themselves to have the right height and width as they can. Of course, I would appreciate some details about Vista here too, and also some information if non Microsoft programs perform as well as those frome the picture.

So this is it, Vista is able to do it, the open source programs will catch any gap in no time, please provide us with nice displays. IBM has been doing this since 2001: http://en.wikipedia.org/wiki/T221

So, anyone who has details please share them.
In xorg.conf you have to write the dimensions of the screen in milimeters (that is: resolution / DPI * 25,4), and it works at least for the ATI driver. I don’t know how to change DPI settings without logging out. Here is how my monitor sections looks now:
Section “Monitor”
Identifier “17” LCD”
Option “DPMS”
VertRefresh 50-75
HorizSync 38-83
#96 DPI
# DisplaySize 338 270
#120 DPI
# DisplaySize 270 216
#200 DPI
# DisplaySize 162 130
#480 DPI
# DisplaySize 67 54
EndSection

This article continues with part II: http://alecs1.wordpress.com/2008/04/05/hello-display-producers-you-can-continue-evolution-now-part-ii/

People doing their best at making something bad

Last message was about Vista and KDE 4 and how to work hard and efficient to get the finest of crap.
The crazy s***it preview that comes with WordPress fits perfectly in the category. All blog people, please disable the snap preview, here you can see how:
http://lorelle.wordpress.com/2007/09/05/how-to-turn-off-snap-preview-on-wordpress-blogs/

I will rebuild KDE 4 tomorrow and see if they solved that menu , so that they ca leave this honouring category of wasted efforts.
It’s Friday and KDE is still not functional. We will meet again.

Windows Vista – first encounter, now I know where the bad ideas from KDE 4 come

Today I had to give a Linux programming test using putty from Windows Vista. I have never touched such a machine ( a Vista one) until today because I had a guess that it is total crap, and it didn’t dissapoint me, it is total shit (this is my point of view, it would take a lot of effort to convince me I’m wrong).
As a lwn.net user said, “do people actually use that voluntarily?”
So:
1. I couldn’t get a terminal, I did not find a way to run a cmd/other_console program.
2. The start menu is a tiny box where you have to scroll if the programs do not fit! If the menus have submenus, this becomes a crime. I take this as an insult to human intelect, a good try to make your brain explode, it like I am a burglar and the program must be hidden from me. Put it simple: I have never seen so much intellectual effort put into creating crap.
The same is true for the KDE 4 start menu I have tested 2 weeks ago, only much worse because of implementation bugs. While the crap from Microsoft is refined, the shit from KDE is buggy. Yet, KDE 4 is not released yet, and there will be choice of course.
Note: I am a KDE fan, this is only a critique to their menu.
3. Internet Explorer 7 (this is the first time I see that too ) has some tooltips that appear over the text you write!! How is hiding what I type going to help me? Insane.
4. There is no advanced usability feature:
-the focus still follows the keyboard, so you have to click in order to scroll (hint: use katmouse, it worked for Windows 2003).
-there is not alt + left mouse button to move a Window, alt + right mouse button to resize or other nice helpers.
Ok, I spent enough time on the Vista, I hope to never see it again.