Posts

Showing posts from 2007

Update to clipboard-modifier

Image
Of my tools that I've written, the one I use most often is clipboard-modifier. I have it on my panel so that it's only one click away.
I had longtime desire to make this tool more flexible, before I would have to always modify the code in order to have it do something different. Now each plugin can add wxPython widgets on the dialog, thus they can control their own state and parameters.
Perhaps the most useful is the ability to send the clipbard (via stdin) to any program and then update the clipboard from the program's output. A good example is sorting the clipboard. Another is filtering it through grep, say.

The program still needs more polish, but it's already very useful, for me.

Presentation: Using Python in Lieu of Bash

I just gave a presentation at Google today. Part of my promised results this quarter was to give at least one tech talk, so on the last day, that's what I did!
Normally I would point you to the slides, but some parts were Google confidential, so I can't. Basically, my talk covered how you can use Python instead of Bash for scripting purposes and showed some concrete examples with the subprocess module.
My talk suffered from Murphy's Law for demos, and failed in some places. It was because I had file called subprocess.py which was being included instead of the standard subprocess module (oops).
Once again I used my mm2s5 to create my presentation, and it worked like a charm. I had planned to use Open Office.orgs's Presentation, but ended up running out of time and using S5 instead. What I think I need is an mm2odp program to make an .odp file from my mindmap.

CAS and Python

I've been somewhat interested in CAS (Computer Algebra System) ever since I used Mathematica for a year while doing TA work for Lorne Nelson. Now, of course, there are some open source CAS programs you can use.

Sage
Sage looks quite interesting:
Use SAGE for studying a huge range of mathematics, including algebra, calculus, elementary to very advanced number theory, cryptography, numerical computation, commutative algebra, group theory, combinatorics, graph theory, and exact linear algebra.
[...]
SAGE makes it easy for you to use most mathematics software together. SAGE includes interfaces to Magma, Maple, Mathematica, MATLAB, and MuPAD, and the free programs Axiom, GAP, GP/PARI, Macaulay2, Maxima, Octave, and Singular.
[...] With the SAGE notebook you can create embedded graphics, beautifully typeset mathematical expressions, add and delete input, and start up and interrupt multiple calculations. Sage uses Python and allows you to access other CAS systems if they are installed. I th…

Pvote, Secure Voting with Python

Ka-Ping Yee wrote is PhD dissertation (University of Waterloo, Ontario) on Pvote. A secure voting software written in 460 lines of Python Code.
Ping's blog post and part of the abstract is below.
I examine the question of how to design election-related software, with particular attention to the threat of insider attacks, and propose the goal of simplifying the software in electronic voting machines. I apply a technique called prerendering to reduce the security-critical, voting-specific software by a factor of 10 to 100 while supporting similar or better usability and accessibility, compared to today's voting machines. Smaller and simpler software generally contributes to easier verification and higher confidence.

I demonstrate and validate the prerendering approach by presenting Pvote, a vote-entry program that allows a high degree of freedom in the design of the user interface and supports synchronized audio and video, touchscreen input, and input devices for people with disab…

Fun with PyEphem

I just tried finding out when and if the International Space Station would pass overhead. First I installed pyephem for linux:
$ sudo easy_install pyephemThen started python and typed the following:Python 2.5.1 (r251:54863, Oct 5 2007, 13:36:32)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ephem
>>> home=ephem.Observer()
>>> home.long = -43.9
>>> home.lat = 19.9
>>> home.elevation = 1000
>>> home.date = "2007/12/09 20:00"Getting the information about the ISS was a bit harder. Had to go to this page at celestrak and get the first three lines then enter it in ephem.
>>> iss = ephem.readtle("ISS (ZARYA)",
... "1 25544U 98067A 07343.37458451 .00020246 00000-0 12456-3 0 4837",
... "2 25544 51.6404 280.6485 0000524 313.0633 141.2500 15.7720376451…

My First Google Feature

I launched my first real user facing feature at 100% today (took a week to get to 100%).
Here's the link to the feature on the Orkut Blog (and in Portuguese).
Basically, we can show the user that they received new scraps in orkut right away as an animated pop up, similar to how you might get chat messages in gmail.

Toxoplasmosis

From the blood tests it seems that I do have toxoplasmosis and have had it for a while, but the parasite has only attacked now. I've taken about 10 days of drugs which has it's own side effects (like insomnia). At night I get this nice purple color shift when I look at lights. Another neat thing is when I turn off the lights at night I see some activity still going on in the affected area (a little light show for 5 minutes). I'd say that the lesion has diminishing, although slowly. I can now see the tops of letters, but not quite enough to read comfortably with the right eye. My peripheral vision is unaffected but there's a big blind spot to the right and down of where by gaze would focus, and it makes things look sort of blurry when looking through both eyes.
I think my brain is starting to get used to looking mostly through the left eye, and I can work pretty much normally.
I'll be taking my drugs until the day or two after Christmas. I'm also off: sof…

Trouble Seeing out of One Eye

About a week ago I noticed that right eye had a part of the vision 'missing'. I setup an appointment with my eye doctor the next day and it got worse. So bad that I can't read with my right eye anymore. I've since gone to a specialist and have started taking some drugs. There's a chance that it's Toxoplasmosis, which isn't great, but at least it's curable. There are no other effects besides the loss of vision (it's where I look, the peripheral vision is still ok).

Odd Portuguese

Image
There are some words in Portuguese that are odd to my English ears.
puxe (pronounced "push"), means pull confuses the hell out of me when it's written on the door. pretende in means intend in English.entendermeans to understand.sex means Fri or sexta-feira. Every Friday it seems the elevator wants to have sex as it shows the date at the top. no means in or at. So the other day I saw "1/2 preço no happy-hour" and it made me sad that there's no happy hour.
meia means half, socks, and six. Apparently, the six comes from "half" a dozen.
de 30 em 30 dias means every 30 days but to my mathematical mind 30/30 = 1, it should be every day.legal means legal and cool. But I would think that only lawyers think legal is cool.direito means right, but direto means straight ahead. Very confusing when someones giving instructions while driving.jóia means jewelry and great. Actually, this one does kinda make sense. If it meant BJ it would make even more sense.

You know you're old when....

I was at a Google party and we were talking about our first computers etc. and got a big laugh when I admitted that I remembered the day we got the new DOS with subdirectories (i.e. mkdir).
That would have been MS-DOS 2.0 in 1983, I was 16 years old then.

Content Aware Image Sizing

Image
Content aware image sizing is a method of resizing an image horizontally or vertically by removing pixels. In the picture above we're resizing horizontally and the cyan line is the pixels that will be removed. What's even neater is you can give hints of what parts to preserve or which parts to preferencially remove.
Watch the video it explains it all.
Here's the research paper.

Nintendo Wii outsells XBox 360

Image
Looks like the Wii has finally outsold the XBox 360 in worldwide sales, despite the one year lead by Microsoft.
Although my favorite site doesn't yet say that yet.

Welcome to the Uncanny Valley

Image
This model follows your cursor with her eyes and is very well done.
So well done it's creepy.

Calendar says 41, web says 25

Image
I like the really high life expectancy too.
Not sure if I can trust this program though, it's not too sure on how to spell "Expectancy".

Eyebrows serve to show where we are looking

Why do we have a line of hair sitting caterpillar-like above each of our eyes? Forget such mundanities as dust protection or ornamentation, a new study suggests our eyebrows serve to control how easily other people can tell where we are looking.

Dreams of the Ultimate Calculator

I have this recurring fantasy of programming the perfect calculator. I've found two or three very good ones:
Frink, is a Java based calculator that I've become very fond of. It nearly has it all:Pros
Runs anywhere, including cell phones and other small devices that have Java.Unit conversion including currencies, and timezones.Unlimited precision, interval arithmetic, complex numbers.Really fun tutorial.
ConsClosed source.Java ;-)Qalculate!ProsOpen Source
Auto complete as you type, pretty cool.
Pretty prints results, ex. sqrt(2) => √(2) or 2*3 => 2·3.
Notion of exact values and showing fractional values.Weird numbers formats like Roman, SexagesimalCan create your own functions that has dialog boxes.
RPN, plotting, history
ConsToo many menus buttons.Not fun, a little dry.Missing Brazilian Real for currency!No timezones.wxMaximaProsOpen sourceComplete computer algebra system.PlottingConsNo unit conversionsNo timezone conversionsIf I were to make a calculator what features would I…

Python 3000

Python 3000 will be the first Python version which is free to break backward compatibility, a chance to get rid of old problems and cruft that have accumulated over the years.
I like most of the changes and agree with their reasons but I'm sad about one thing. Before you could create a program like this:
print "Hello world!"
For beginners, this is great, no extra semicolons, parentheses etc. Now you will have to write:
print("Hello world!")
It's not the end of the world, but it just got a little more difficult for beginners. On the other hand it will introduce them to functions a little earlier.
Another difference I'm not so sure about is removing the % operator. I like that formatting operator:
print "Your are %d years old" % age
now becomes:
print("You are {0:d} years old".format(age))
sigh.

Simple fix to Gnome Terminal

I often double click on a filename or folder and then paste it with the middle button (the scroll wheel), in Linux. What annoyed me was that it doesn't get the ~ character so if I'm cutting an pasting my current path as shown on the command line
scott@home:~/src/folder/name$Double click, middle click gives me /src/folder/name and not ~/src/folder/name. How annoying, I have to cursor all the way to the beginning and enter the tilde character.
Even worse if there's an error message like:
Error in ~/src/folder/name.cc:123Double click, middle click gives me now /src/folder/name.cc:123. Now I have to edit both sides.

The fix is simple, go to your Gnome Terminal|Profiles... and edit your default profile. The first pane called "General" and has a field called "Select-by-word characters". Add the tilde (~) and remove the colon (:) and you're done.
There, I just saved you 2 thousand keystrokes a year.

It's Hard to Keep Up

Our language is changing faster and faster, it's hard to keep up. Even yesterday I used the word w00t!.
Thankfully, our technology helps keep us up-to-date.
Now I know about LOLCats, L33 speak, And all the Portuguese internet expressions too.
If fact there's a whole list of words I've never heard before to describe these things: backcronyms (and anacronyms, apronyms) , snowclones, image macros, and other forms of patois.
kthxbye
shortening of "k thx bye". The K is short for OK, which is short for oll korrect, which is a facetious alteration of All Correct. thx is short for thanx which is a facetious alteration of thanks which is short for thank you. Bye is short for goodbye, which is an alteration of alteration of God be with you. "kthxbye" is the pinnacle of English's advancement, shortening "All correct, Thank you, God be with you." into seven lowercase letters. Humanity is doomed. Obviously, it is used to end a conversation *fast* that you d…

I'm Not Playing Tennis the Way I Used To

I played two sets over the weekend and noticed where I used to grunt like Nadal when hitting the ball, now I only grunt when I pick up the ball.

Why would I ever want to listen to 'static' music again?

I've been playing Guitar Hero ( at work of course ;-) ), and Elite Beat Agents on the DS and I'm thinking to myself, "Why would I want to listen to music that I can't interact with ever again?".

American's Don't Read?

This page summarizes some startling fact about Americans and books. Most of these facts come from the Jenkins group, and to me, don't make a lot of sense:
33% of high school graduates never read another book after high school
42% of college graduates never read another book after college.So, on the surface, it looks like college students read less high school graduates. I think you want to count only the high school graduates that never went to college and try and get people at nearly the same age group. An 50 year old is more likely to have read a book since high school than a 20 year old.

Also, according to Jenkins, 80% of the US population want to write a book, but 80% of the population has not bought or read a book in the past year. 70% have not bought a book in the past 5 years. That's even more bizarre, everyone wants to publish but are too lazy to read? Are they going to write a book and never proof read it?
I think I'd be more alarmed if the facts showed some coh…

File Rename Anyone?

I was in a meeting, editing a file and realized that the filename had a typo. To fix this I have only two choices, either do a "Save As..." which is probably the best solution, but then I would have to remember to delete the original file at some later point. Or I can quit the program, rename the file at the command line, load up the program and load up the new filename, ugh.
Why doesn't all software have File|Rename... menu option? Seems like such a common need, and it's lack encourages the use of bad filenames, it's just too much work to fix.
After programs like eclipse added the ability to rename variables and methods, the names I end up using are greatly improved. I don't need to add comments for most of my code because the names are self documenting.
Please, spend the extra hour to allow renaming in your app.

I'm now a Vi user

Image
I've used my fair share of text editors including Emacs for 2 years and Eclipse several years.
My favorite open source program has always been Scite, and I still have it on my launch panel. But now I'm a vi man.
I tried to go back to Emacs, but it takes so long to get used to and there are too many funny keystrokes. Emacs users tend to get an Emacs Pinky from using the Ctrl key so much, which was not a problem I was looking forward to. Emacs also seems to have too many choices, too many cool downloads to get, each of which slows down the startup time.
I liked Scite because the keyboard was setup similar to Visual Studviio, which I had used for years. It is also a solid program that is very customizable. Also, being open source is great, I even contributed some code to the project.
But after reading this article I decided to give vim another try. What got me interested again was the suggestion to always be in 'normal' mode and to be in insert mode only for short bursts…

Python is great for Science

Image
SciPy is very useful for doing matrix like operations.
And Python out of the box you can do thing like:
In [1]: 2**3000
Out[2]: 123023192216111717693155881327675251464071389573683371576
611802916005880061467294877536006783859345958242964925405180490
851288418089823682358508248206534833123495935035584501741302332
011136066692262472823975688041643447831569367501341309075720869
037679329665881066294182449348845172650530371291600534674790862
370267348091935393681310573662040235274477690384047788365110032
240930198348836380293054048248790976348409825394072868513204440
886373475427121259247177864394948668851172105156197043278074745
482377680846418069710308386181218434856552274019579668262220551
184551208055201031005025580158934964592800113374547422071501368
341390754277906375983387610135423518424509667004216072062941158
150237124800843044718484209861032058041799220666224732872212208
851364368390767036020916265367064113093699700217050067550137472
3998766005827579300723253474890612250135171889174899079911291…

And now, for something completely different

Image
I've launched another Python* open source program called mm2notes.
I can sure pick those product names can't I: I've launched mm2s5, xml2ddl, wxOptParse maybe I should work for HP so I can name their products like the "HP xw9300". In my defense, at least my names are easy to google.
Anyway, the idea is to take notes during a meeting using the excellent FreeMind program, and my software will convert those notes into a nicely formatted "meeting minutes". The cool part is that even if you fill the nodes of the mind-map under different locations at different times, the program will unroll these nodes based on the time the node was created.
I guess a picture will help out. Look at the FreeMind document which is a little hard to understand and the meeting notes, which is organized by time.

MyZones launched

Image
I've posted another open source Python program called MyZones.

I basically had the desire to know what time it is in another timezone and feel confident that the answer is true. By using pytz, I feel much more confidant that I will always have the correct time for the various timezones. The program is simple and simply works, but I have some additional things I'd like to add.
Right now you can change the time in any timezone shown and the clock will stop updating and all the other zones will be synchronized correctly. But changing the time is a little clumzy, maybe a dropdown of hours in 30 minute increments might be nicer.
The program is a little plain, could use some sprucing up and wxPython has analog clocks and "digital" displays that I can use for that purpose.
I think I've caught an OpenSource bug, I keep writing these little programs.

First time upgrading Ubuntu successfully

The last two time I upgrade Ubuntu I had a lot of problems. After rebooting, my X windows was usually messed up and I had additional problems with the last upgrade.
However, this last upgrade was relatively painless, but it did fail the upgrade and then leave the Update Manager without any hint what to do. Of course, now I'm pretty experienced and know all the apt commands by heart. I ran "sudo apt-get -f install" to find the problem then "sudo apt-get remove python-setuptools" which seemed to be the problem package. Then I ran the Update Manager again and it went on it's way.
For some reason, after installing everything the Update Manager said it was finished and never asked me if I wanted to reboot so I was a little extra hesitant the first time I rebooted. I crossed my fingers and rebooted and a new Ubuntu came out without any errors.
I'd say the nicest thing I've seen so far with the new Feisty Fawn is running "python" and seeing vers…

Another reason I like working at Google

Another great post from Peter Norvig makes me proud to be a googler. In this article (which he wrote most of the code on a plane ride) he shows in 20 lines how to write a spell corrector similar to the "Did you mean: ....?" spelling corrector in the Google search page. Just like his Sudoku solver earlier, his code is both clear and concise (without being overly cryptic). By the end of the article you feel like you would have been able to write the code yourself. One thing I like about his articles is I often learn something new about Python like the: collections.defaultdict(lambda:1)
snippet in this article. Also, he goes into some of the theory, like the Bayesian theory in this case. His links were useful too, like the Spelling Error Corpus, and Google's trillion word N-gram corpus. Peter is the director of research so you might expect him to be smarter than the average bear, but there are an amazing number of very bright lights here at Google who I have the honor t…

dos2unix

The other day I was going through some of my files and noticed that some files had the DOS style CR/LF lines, ugh. I couldn't remember what the standard tool in linux is to convert this, turns out it's dos2unix which is linked to fromdos command.
The package is called, forgettably, tofromdos, i.e. to install it you need to call:
sudo apt-get install tofromdosFor such a simple utility it doesn't provide very much, there's no man page and only -h works and not the more common --help, but it does work as advertised. I was surprised that it isn't included in the standard Linux install.

Released Clipboard Modifier

I released another open source program called Clipboard Modifier. I guess I'm amortizing all the effort it takes to open source a program my doing them back-to-back, when all the details are still fresh in my mind.
This little utility uses wxPython and modifies your clipboard using plugins that you provide.
I needed this at work to copy and paste a spreadsheet into a Wiki. Wiki's aren't very good at making tables and this little program leverages the capabilities of a spreadsheet to insert columns rows, etc.. Basically, I create the spreadsheet in OpenOffice and then using Clipboard Modifier copy and paste a wiki style table into the Wiki.
To make the program a little more complete, I've also added some other plugins. For example, there is one that takes some text and converts it into something that Java or Python would like (in double quotes, etc.).
I have more ideas of this type. Looks like I'm going for quantity, instead of quality.

Created open source utility 'mm2s5'

Image
I just finished posting mm2s5 to code.google.com this weekend. It's a small Python utility to convert a FreeMind mind-map (.mm) file into an S5 presentation. It was a little hack that I put together to make presentations quickly and somebody else found it useful, so I decided to put it on the net.
You can see an example presentation here and a picture of what that same presentation looks like in FreeMind here.

Received my OLPC

Image
I asked for and got an OLPC to test and work on. My first impressions are very favorable. It's very small and cute and something that kids are immediately drawn to, because it's not adult sized. My 6 year old son has been playing with it on and off and I just saw him today explaining how to play on the laptop. I even overheard him saying that the laptop connects to other laptops which he must have overheard me mentioning to someone else.
The keyboard is very small, but it's perfectly acceptable for two finger typing. My only complaint is that the space bar and enter keys have dead areas in it, but that ought to be fixed pretty soon.
The biggest complaint from my son is the trackpad. He was almost brought to tears trying to use it. It was frustrating for him because he would try to move right and it would move left. Even build 303 seems to have that problem on occasion. I never had problems with it, but I'm more used to using a trackpad. It looks like he was star…

I'm getting an OLPC

I've asked for and looks like will receive an BTest-1 OLPC laptop for development and testing. My son is working through letters and numbers so I'm looking on adapting a program for that. It should be very interesting seeing how he uses the laptop as my tester. Perhaps I can convince some of my colleagues at Google Brazil to contribute some time as well, it's a very exciting project.

OLPC and Security

The One Laptop Per Child (OLPC) project has an interesting take on building security for the laptop. Here is my list of why the OLPC is especially vulnerable to security issues.
100's of millions of these near identical laptops will be produced, making it a target for crackers.The OLPC group want the child to be able to hack their own laptops, get right down to the code.
Some of the children are going to be very young, entering a password or even replying to dialog prompts may be asking too much.These devices automatically connect to each other and automatically connect to the Internet.So what will they be doing to improve security? They have a system called Bitfrost, which comes from Norse mythology, where Bitfröst is the bridge which keeps mortals who live in Midgard from venturing into Asgrad, the realm of the gods.
Nearly all programs running on the laptop will be running in a fortified chroot, sort of like BSD jail.Programs will store their data on a new object oriented filesys…

Sony's Culture and the PS3

I think I've been vindicated when I said that the Sony PS3 would be a failure. The main issue with the console was the insistence of putting a BluRay player on the device. I think it shows how there's an organizational/cultural problem at Sony. They insist on putting an expensive add-on to the game simply to satisfy some people in the media division of the company. The BluRay player does nothing to improve the gaming experience and in fact it a little slower than a fast DVD player. The rationalization was that some people would by the PS3 just for the BluRay player. And maybe, they might buy a game too. What kind of game do you sell to someone who doesn't really want to play games?

In a related note you can buy the PS3 with Linux pre-installed. Although that's really neat, it means there's one more reason to buy a PS3 and not play games!

Every current PS3 sold is sold at a loss. The only way Sony can recoup that money lost is by the licensing money from PS3 g…

Google Notebook, great for blogging

I just discovered Google Notebook that I knew existed but had never tried it, I heard it referred to as a post-it note type tool, but it's more useful than that (I'm not a big Post-it™ fan). If you install the extension, you can quickly select and gather snippets from a web page and throw it in your notebook. It automatically keeps track of the originating link and the date you clipped it. The notebook supports rich text and images, in that sense it's quite complete.
The only thing missing is the ability to take an entire notebook and make a blog entry for it. Maybe I could make it my 20% project.

The Real Cost of the OLPC

Image
Professor José Antonio Meira da Rocha has done some number crunching and found that the OLPC XO laptop will cost about R$0.59 per child per hour (about 27 cents US). This includes the cost of the laptop, electricity, training, OLPC servers, and internet connection. In comparison having Computer labs where 5 or more students share one computer would cost R$9.90 per hour.
His numbers show that the total cost of ownership (TCO) OLPC will be R$94.07 per year (vs. a computer lab's TCO of $R586.00). The problem with computer labs is that the child doesn't feel any ownership of the shared computer, he or she can't just pick it up any time and start using it. Also kids don't like sitting in chairs for hours at a time. On top of that the OLPC has a better display than a typical computer and so is better for reading.
Having just spent R$300.00 on books, pencils and paper for my son for school this year, $94.07 looks like a steal.