Diddlbiker is now a permanent permanent resident!

The last Tuesday of september was a historical day in Diddlbiker's life: I-751 (Removal Of Conditions) was approved! No more INS (at least not for a long, long, time...). For the un-initiated, which I guess is about 99.9% of humanity:
After marrying my American wife I applied for a "Permanent Resident Status", usually referred to as 'Green Card' (it's pink, by the way). There are many ways to get a GC, and pretty much all of them are slow, unsure (capped) and take a tremendous amount of red tape.
Getting a GC through marriage, however, is relative straightforward (notice the use of the word 'relative') and has a 99% succes rate (that is, if you're married).
Now, you can call the INS many things, but they're not stupid. Bureaucratic, slow, anal, a lot of things, but not stupid. So, they want to know if you're really married, and not just faking it. This is where The Interview comes in. During the interview the INS officer will go over the proof you submit for having a shared address and life. Mind you, this is a bit of a catch-22 situation. Without the right papers (green card or visa), you pretty much don't exist in American society. You can't open a bank account on an expired tourist visa, you can't get a driver's license, get a credit card, etc, etc. So, proving that you live together is kind of sketchy regarding the address thing. Luckily I did get some services on my name (rent, phone, utilities) and my wife had the rest, so that worked out for the first interview.
So, three years ago we had the first interview, 21 months into our marriage. Why is that important? Because as long as you're not married for 2 years (24 months), the green card will be 'conditional' - you have to come back for a second interview, two years later.

Now, here's where things get weird. The second interview is to confirm that you didn't have a 'convenience marriage'. However, if you get divorced between interview #1 and #2, everything is pretty much just fine. The alien can file a waiver and skip the second interview. Any letters from the ex-spouse stating that the marriage was fake are routinely ignored sinces ex-husbands and wifes tend to send nasty stuff to the INS all the time, just to make life of their ex hell. So, the INS ignores that.
So, basically, divorce and you don't have to prove that your marriage was real. But, if you're still together, you do have to prove that you're still married. Go figure... Anyway, we passed that interview, and for being a PR, the INS is done with me. I'll receive my unconditional green card soon (valid for 10 years, renewing is a no-hassle operation) and I can finally move my 4-inch immigration binder from home into our storage space.

And if you think I got the name wrong: yes, nowadays the Customs and Immigration Service (CIS) is doing all this. But all the letterheads still read "Immigration and Naturalisation Service" - as said before, it's an organisation that moves very, very slow...

The bull is off the premises...

That was an email I received yesterday at work. Apparently, a bull had entered the office campus last week while I was on vacation. Although the bull was considered to be harmless and friendly, people were advised to go jogging or walking somewhere else during lunch!

...talking about bull stories!

Farber's gone ... there IS justice after all?

I was reading through the newspapers that arrived during vacation this morning, and learned something great: Farber has resigned!
For those who do not live in Bergen County, who is Farber and why is this such a big deal?

Farber is the state's D.A. (District Attorney, Officier van Justitie). Her boyfriend got pulled over for not wearing his seatbelt (pretty stupid during a Click it or ticket campaign), and while the police officers where checking his license (revoked) and his car (expired registration), he called the missus.
So, while he was handed out citations, the state D.A. arrives, in a state vehicle with a trooper as driver (Farber doesn't drive herself since she had 12 standing traffic violations). This is where things get interesting. The normal procedure for an offence like this is that -apart from getting a big pile of summonses - is that your car gets impounded. Not in this case, the car was escorted to the local D.M.V. (Department of Motor Vehicles), where mr. Goore got his license reinstated and a new registration for his car.
Yes, for normal people this would take at least a full day of very skillful fighting bureaucracy, but mr. Goore got his stuff handed over as soon as he arrived there. Of course, his girlfriend mrs. State D.A. calling to the D.M.V. had nothing to do with it.

So far, just some real stupid stuff that -for any normal human being with some moral fibre- would mean instant resignation. This is where ms. Farber is making all kinds of interesting remarks:

• I just did what everybody would do for their partner (except that we wouldn't since we wouldn't have such a powerful position)
• I didn't say anything, and it was not my intention to influence the police officers (who tried to void the tickets they wrote out after she arrived at the scene). Suuure...
• I wasn't there to help mr. Goore out, I just wanted to retrieve some sensitive papers I left in his car (now that's showing how good of a D.A. you are - leaving sensitive state paperwork in the back of a van)
• I just misjudged the situation a bit (not a bit, besides that, it's your job to judge situations, nice to know that you're not very good at it...)

So this week, ms. Farber resigned from office, stating that the state is putting its 'ethics bar too high'. Her partner is fired from his state job as well (no reason given...) - together they've lost a total of $220,000 in income. And for what? A $600 traffic ticket.

An unexpected vacation encounter

The last thing I expected to see at my vacation at the shore (my hair cutter explained to me that in New Jersey you refer to 'the shore', in NYC you refer to 'the beach'. I told him that as a Dutch-born, I should refer to that sandy strip as 'the beach' but he replied "You're in Jersey now") was a real live shark.
Granted, most people - including me - will think of 'shark' as an 800 pound killing machine as seen in Jaws, and not the shrimp-like creature (this picture, courtesy of Cabo Yachts, is a good example of the monster that I saw) that I saw, but still.
This was a live shark, with tail, fins, gills, eyes, teeth, the whole thing.
A f##ing shark!
In New Jersey!
The family sitting next to us at the beach came with it out of the water. Probably lost and stunned by the (relative) cold water temperature. I heard that it was probably a Sand Shark (hence the picture of one), but I can't be sure about that - except that it really was a shark.

Some marvelous freeware

Diddlbiker works in a non-IT environment, and that means that the goodies-bag is as good as empty. Don't get me wrong, my employer does not skimp on what they do provide - Windows XP, latest version of Microsoft Office, decent virus scanner, etc.

It's just that for any of those handy-dandy extra tools that, as a geek, you have uses for, but unfortunately, none of the suits do. It is tempting to install local copies of whatever you've bought at home, but most EULA's don't provide in that.

So, I set forth on the task of finding out what is actually free. Oh, did I mention that it has to be good as well? It is surprising what you'll find once you start looking...

Editor:
Notepad++ - this text editor is good, and I mean really good. It has even replaced my payware Textpad editor at home - since Notepad++ is sooo much better! Some features: multi-document support with tabs, column selection, syntax highlighting, executing scripts/preview in browser, code folding, find & replace with regular expressions, and still a 'lightweight' look & feel.

Zip:
Izarc. Windows XP has standard zip functionality, but I think that it is a little bit too transparent. The de facto standard is of course WinZip, but that has it's limitations - especially with spanning (that might have changed over the years but they lost me when spanning still needed to be done through 3.5" disks and everybody else in the industry had a better solution). For years I used Powerarchiver, and I still think it is a very, very good product. But Izarc is almost as good, and completely free!

Picture Viewer:
Picasa. ACDSee is great, but not for free, and the latest versions started to turn into bloatware. Irfanview is really good, but doesn't allow you to view the contents of an entire folder at once. Picasa is, if you're willing to accept that it is from evil Google, a very nice product to use.

PDF Creation:
Go2PDF. Creating PDF's used to involve expensive software (Adobe Distiller), or complicated setups with GhostScript and all kinds of reroutings. Go2Pdf is simple to use, and offers a full feature set.

If you know of any other great freeware, please let me know!

LaTex Schmatex?

Diddlbiker doesn't even know what triggered it but he decided recently to get some hands-on experience with LaTeX. Now, to correct the title, I actually [b]do[/b] like it, [i]but...[/i]
It is not as wonderful and perfect as everybody says it is. Clearly, its biggest appeal to most Linux Kiddiez is that it is not 'Micro$oft'. As we all know, each and every Microsoft product sucks beyond belief. Actually, it is all these immature kids that want to differentiate themselves from the big crowd that suck, they usually don't know what they are talking about.

So, what about LaTeX. First of all, I absolutely [b]hate[/b] software with funny names. As if "Latex" isn't bad enough, an apparently even "LaTeX" couldn't do the job for sufficient weirdness (see logo - this is how the software is referred to in the manual on a continuous basis. The horror!). It is like software as Lotus Symphony! or Act!, or that wonderful non-IBM compatible 80186 powered monster that Philips introduced in the 80's, the :Yes! (an exlcamation mark is sooo common, let's throw in a colon for good measure).

Before I continue to rant; Latex is really, really good. I like it. However, saying that 'Word is the worst word-processor ever invented' and 'no match for Latex capabilities' is like making claims that 'Outlook is much better at email than Powerpoint' - you're comparing two different pieces of software. Latex is purely typesetting software. It is language, not a word processor. The result is the same (a printed document), but the software - and the expectations - are different. So, in defence of word, some of the 'Latex myths' about how horrible word is - and what in my view the reality is.

Latex' output is much better than in word.
Could be, but that's probably because you're looking at something that LaTeX excels at (mathematical formulas for instance). I've been looking for the example that I found that was 'honest' proof - two identical documents with just plain text. "Look at how much better LaTeX handles whitespace, especially at the second sentence". The text in case had an indentation at the LaTeX document, forcing the words closer together. Of course did the Word document look bad...

Latex can handle large documents, Word can't
Technically speaking, LaTex doesn't allow editing of any document - you will have to edit your files with a seperate text editor. Stuck with an old-fashioned Notepad in Windows 98? Too bad, only 1 Mbyte...

Microsoft Office is such a memory hog.
Excuse me? I installed LaTeX (the ProText package) on my machine, which took a whopping 800 megabyte. With Microsoft Office, you get Word, Excel, Powerpoint, Outlook, Access and a whole bunch of tools - and there's still room left

Word is so slow, LaTex is muuuuch faster.
True, if you're interested in a 500 page dissertation laced with formulas every other paragraph. If on the other hand, the scenario "Boss: can make something to show sales blah blah blah geographical bla bla costs blah?" is your daily routine, Word can usually help you out in less than 30 minutes (and that includes making the two bar charts in Excel and the map in Mappoint)

WISYWIG is horrible!
I saved the best for last. WISYWIG may not always be the best solution, but it is pretty much convenient. Switch to normal mode, show hidden characters on and there's not a lot left to surprise you in Word. And of course, make documents the way you're supposed to - using templates and styles. But no, reading the manual is only allowed for LaTeX, with Word we expect you to do everything correctly at once. And at the same time you see all those LaTeX editors that do have WISYWIG ability. Oh yeah, soooo horrible...

I could go on for a long time. Again, LaTeX is excellent and fun to play with for Geeks. It is definetely better than Word when it comes to large, complex documents. But fight Word on it's hometurf (30 page documents with lots of integrated charts, pictures, and excel spreadsheets embedded in it) and it won't come out that pretty...

Places I visited

I stumbled accross www.world66.com today, and one of the things you can do there is make maps of places that you visited.

This is me in the USA:


create your own personalized map of the USA
or check out ourCalifornia travel guide

And this is me in the whole world:



create your own visited country map
or check our Venice travel guide

Rather disappointing :-)

Reminder: reading from disk is expensive

Diddlbiker is working on a little project at work. What the project consists of is not important, but in one step of the process I have to find transportation links between two locations.
There is all kind of theory written over this, but in this case, the problem was very simple: I have rail links in table 'A', and truck links in table 'B'. What is the best routing option from Houston to New Jersey? Straight over Newark or is railing to Philadelpia so much cheaper that the extra trucking cost doesn't matter.
So, for each route there are about thirty rail links to choose from, and there is only one truck link that connects the rail link to the final destination. So what's the big deal? I have to do it for 200,000 entries, and that takes some time.
The initial approach was a brute-force pure-SQL attempt. Well, that didn't go over that well.
The second approach was a VBA approach. Each link would start with a recordset of all rail links originating from it's starting point, for each rail link I can than look up (another recordset) the truck link, and when I all have them, find the cheapest one. That worked, but it took an amazing amount of time - after 45 minutes of processing, only about 1% of the entries was done!
From here I had three options:

1. Intelligent approach: go and study and find an optimal way of doing it. I don't think there is a lot to be gained here - I'm only investigation 30 options per link, after all


2. Reduced data: instead of 30 options per link, I could reduce the rail set by 30% by predeterming what the 10 links are that are closest to the final destination. I wouldn't want to go below 10 - 5 or 6 links might be the closest location, but different stations, and you really want to see if routing over a different location works. It would reduce processing time by 70% - but processing would still take around 20 to 30 hours.


3. Caching: a lot of time is spent reading the truck rates, and there aren't that many of them (about 110,000 to choose from). Writing a simple cache wasn't that hard, and the result was amazing - total processing time went down to 45 minutes.

One of the nice things about VBA is that you get a pretty good Dictionary for free (it's in the Scripting Runtime Library). A small hashing function was written in minutes.

Total time invested: about one hour
Total time saved: about 80

Yay me!

Brilliant!

In really, really rare occasions Diddlbiker feels like he accomplished something brilliant. Today was one of those days. And the best thing is: my artwork is actually something small, elegant and comprehensible. What happened, was that my clients would need a table in a certain format, but the model it supported would demand a different format:

Client Wishes Model Demands 2006 0% 2006 1.0000 2007 5% 2007 1.0500 2008 4% 2008 1.0920 2009 3% 2009 1.1248 2010 2% 2010 1.1473 2011 1% 2011 1.1587

Clearly, my users want to specify a percentage increase for each year. I on the other hand, want a factor that compounds the increases year after year. The solution I came up with was non-equi self join with a kick: SELECT tblYear.Year, Exp(Sum(Log([tblRate].[Rate] + 1.0))) AS Factor FROM tblIncrease AS tblYear INNER JOIN tblIncrease AS tblRate ON tblRate.Year <= tblYear.Year GROUP BY tblYear.Year; I achieve two things in the query. First of all, by using a JOIN ... ON ... <= ... I'm able to pick up all years up to and including the 'current' year. Second, SQL doesn't have a PRODUCT aggregate function. By turning the factors into logarithms I can multiply them by adding them together - and SUM is something that you can do in SQL! Once they're all summed, I can reverse the logarithm by using an EXP (exponent) function. The query basically turns the first table (input-friendly) into the second table (model-friendly) without any scripts, temp tables or other garbage. Brilliant!

Speaking of brilliant: I saw a link on Joel on Software today about Elastic Tabstops - I wished every editor was that smart!

Nature in Bergen County

Diddlbiker's father-in-law got himself a bird feeder. That doesn't mean that you will get birds, though. Those dinosaur-descendants are very, very careful and seem to have a sixth sense when it comes to photography: they will simply not show up.

The good news is that those dang squirells have no problem with biped mammals at all, so while I was sitting outside in the blistering heat, there was at least something that I could take pictures of. An FIY for my European friends: squirells are not looked upon kindly in this country. They are as much enjoyed as pigeons in large European cities. So, in analogy of winged rats one could call them rats with fluffy tails.
So, why the picture of the squirell? This would be a great moment to quote Ansel Adams, or Cartier Bresson, about photographing the ordinary, but the truth is: they are adorable creatures to photograph!

So what about the birds? Eventually they got used to me and a couple of birds did show up. Thanks to Birds of the Mid-Atlantic I am able to find out what bird this is: a Mourning Dove. Just like in Dutch, American bird names are weird. I mean, you might think "tjiftjaf" and "lepelaar" are weird names, but what about "Grackle" or "Tufted Titmouse"?

I got even luckier after a few more minutes when another rare bird showed up, in this case a House Sparrow. I even managed to get them into a single frame! Without being sarcastic, there are a lot of interesting birds to be seen - I just never carry my camera with me when I see them (which is rare). So far I have seen:

• Pelican (in Charleston)
• Turkey Vultures (lots)
• Eagles (not bald)
• Cardinals (without bats)

And of course a whole list of ducks, warblers, sparrows, geese and other common birds.

And one day I will get a picture of that special bird. It will just take a while - I'm not a bird watcher after all.

...and team Holland is next

Couldn't watch the entire game today, Diddlbiker had to take his son to a birthday party. When I got back, Holland was trailing 1-0 from Portugal and an incredible bunch of red and yellow cards was handed out.
Portugal limped into the next round, decapitated and with most of their players vulnerable with yellow cards - they will not survive the encounter with England.

Team USA goes home

The USA team lost today in a crucial match at the world cup football (I refuse to call it soccer). Not unexpected, when in a poul together with Italy and the Czech republic, although the final ranking of the pool (Italy - Ghana - Czech - USA) was not completely expected.
The inexperience of the American press with the football and the role the USA team plays in it was fascinating to see. A lot of value was given to FIFA rankings. Rankings that work well in sports like tennis, golf and chess, since the players play a lot against each others, so their ratings are calibrated all the time. National teams, on the other hand, only play a handfull of matches each year, and that makes the ratings far from reliable.
So, going into 'Poule E' as the #5 team of the world looked like there would be a solid chance. To be honest, the team is good enough to consider it a serious chance for ending high in the poule. But some realism should be applied as well - both Czech and Italian teams are fearfull opponents, and African teams have proven over the years to be tough opponents as well.
So, the loss against Czech republic came as a total shock. Losing 3-0? Sadly, that was the only match that the Czech played really good, and the USA played really bad. Hence the result. So far, nothing shocking. Anybody can lose from the Czechs, there's nothing to be ashamed about.

What surprised me was the 'analysis' before the second game. Ghana absolutely kicks ass against the Czechs, and the reaction of the commentators: 'this is good news for team USA. They should get hope out of this game'. Basically, now 'all' that is needed is a draw against Italy, winning from Ghana and Italy beating the Czechs, putting the USA in second place. Let me repeat this: your previous opponent, who humiliated you, gets the crap kicked out of them by the team you will have to beat in the last round. And that is good news?

To be honest, I really hoped that the USA team would have advanced to the next round. Hopefully football will survive a blow like this - a lot of the enthusiasm for 'soccer' had to do with the good performance of the USA team so far. This moment had to come at one point; you can't progress forever, and hoping that team USA would be a serious contender for the world cup in the next ten years would be hopelessly optimistic. Any MLS game (and I've watched too many) will confirm that. Still, an 'honorfull' exit against Brazil would have been better than an early exit after two losses and a draw. But the agony of defeat makes the next succes taste better - something that team USA hopefully will learn now as well.

A trip to the terminal

Most terminals that Diddlbiker is visiting these days are airport terminals, but sometimes I get lucky. Last week Diddlbiker's department had its bi-annual meeting.
This year, the meeting took place at a container terminal, and included a tour as well! Now here is a place where I'd love to walk around for the entire day, taking pictures at will. Unfortunately, that will not happen for several reasons:

• Liability: chances that Diddlbiker gets run over by a truck and that the terminal has to pay a huge amount of money to Diddlinabiker and kid is fairly large
• Security: the usual vague reasons that are always given - anything that is worth photographing seems to be a potential terrorist target nowadays (remember, taking pictures of the GWB will get you arrested!)
• Competition - competitors of terminal X would love to know how many cranes there are on terminal Y, etc (after all, it's not like you can see those cranes from 5 miles distance...)

So, locked up in a van that rarely stopped, and of course sitting at the left (wrong) side of the van, I still managed to shoot some nice pictures. Worth a visit!

Back from Charleston

One day later than expected. My flight home on friday night was cancelled. I spent the night in the Embassy Suites hotel in Charleston, near the airport - I can recommend it to anyone!
The flight home brought me right over the house that Diddlbker lives in. Unfortunately, I couldn't see it - it was under the plane, not next to it...

Visiting Charleston

Diddlbiker is staying a couple of days in Charleston, for work. Since bringing your camera is always a good idea, I'm able to show some pictures as well.
The hotel is located on the bay in South Carolina, with a view on magnificent new bay bridge, and the USS Yorktown (an old aircraft carrier).
A few things to note about the hotel:

• There is no gym (kind of surprising for a resort)
• When going out for a walk in the nearby marshes, watch out for alligators!

Shooting pictures of the bridge is harder then I thought. But I won't be leaving without them!

How on earth do we fly in this country

Diddlbiker is writing today from his hotel room in Charleston, SC. The hotel room has free internet, which is nice, by the way. Anyway, my boss decided that we'd leave earlier - I didn't understand way, because the weather wasn't that severe. Just some rain.
Long story short: our 15:15 flight left at 17:45 - and that just for some rain?! No wonder airlines operate at a loss if a few drops of rain already ruin their schedules.

Interesting stuff to read: somebody decided to eat monkey food (as in dog food, but then for monkeys) for a week - after all, humans are primates. Hilarious!

Transcending into the next level

Diddlbiker got a little bit closer to Python-Zen today. Discovering that you have a folder with over two hundred files named '05????.txt' and '06????.txt' isn't really fun if they should be named 'm05????.txt' and 'm06?????.txt'. I surprised myself with conseriding this the easiest solution:

import os
myfolder = 'C:\\blablabla'
for myfile in os. listdir(myfolder):
__oldname = os.path.join(myfolder, myfile)
__newname = os.path.join(myfolder, 'm' + myfile)
__os.rename(oldname, newname)

(underscores used to indicate spaces; blogspot removes them no matter what I do, even nbsp's)
Time to write script: about 40 seconds - and done!

One day, I'll be a real Pythonista.

There are no dumb questions...

But some of them... Diddlbiker was doing his regular workout during lunch: 30 minutes on the stationary bike ('Cascades', 30 min, level 15). The program involves two cascades of increasing and decreasing resistance. I like the program; most of the time you spend spinning with low resistance, but you also spend a couple of minutes at each of the two peaks at high resistance.
Anyway, as I was just past the second 'top', a coworker who is working out on the elliptical machine next to me asks me is it possible to get to a good workout on that bike? Well, let me see... My heartrate is racing at 172, sweat is literaly flowing down my face in rivers, and my shirt if soaking wet from top to bottom. But I stayed friendly, and instead of being sarcastic and answering 'no, it's like a walk in the park', I explained that it all depends on the resistance level and the pedal speed. The weird thing is, she took spinning classes in the past, so she should know better - maybe because it is a recumbent bike?

They canceled my radio show!

Diddlbikers morning commute is a 40 minute trip over Rt 80 and Rt 287. Listening to a decent radio station makes the trip feel shorter. Living in the NYC are means that there is a good selection of radio stations available. Adding a few reasonable constraints (decent music, English language, traffic information) cuts the list pretty short. The one I stuck with was KTU. I really liked the morning show with DJ's Goumba Johnny and Balthazar.
So, surprisingly, they weren't there on thursday morning. Just another DJ, presenting his records as if he always did. Co-hosts Speedy and Cindy where acting like it was the most normal thing in the world as well. Mmmh. Both hosts sick at the same time?
Friday, same story. Now I start to do some researching, but apart from learning that Goumba Johnny (real name John Sialiano) did six months of jail time early 2000 for tax evasion, at first I learn nothing. Then I find out that they fired the hosts of a succesful morning show to replace them with Whoopi Goldberg. Hopefully they'll move to another station. As for KTU - I don't think their morning show will survive this. Grr!

JPEG Myths

Today was a rainy day, so Diddlbiker decided to try something out. What I wanted to do was show how image quality deteriorates through subsequent saves as a JPEG file. After all, JPEG is a lossy format, so every time you save, you lose some image quality, right?
Wrong. It seems that the truth is slightly more complicated. Save a picture with exactly the same image quality over and over again, and the image data will be saved in exactly the same way. Here is an illustration:

This is the original picture.
This is the picture saved at a 25% quality level. Yes, it's pretty bad.
This is the previous picture opened, and saved again at a 25% quality level. You'd expect an even worse picture, but it looks exactly the same!
The process is repeated two more times (four times in total), but no difference with the first picture...

Amazingly, image quality doesn't degrade over consecutive saves. There is the loss of image quality due to the JPEG quality used, but that's about it. I'm figuring that once the JPEG compression has taken place, the end result will, when compressed again, yield the same file. Some caveats however:

• This only works if the picture stays the same (except for the parts that are changed, of course). Resize the picture, crop it, add border around, mirror, etc, and you'll get another round of quality degradation. Any part that is changed will suffer as well of course.
• There is of course the initial quality loss - and there's not a lot of difference in file size between 95% quality and 75% quality (more about that in another blog)
• JPEG supports rotation (at 90 degrees) as well. If you are using a good image editor, all that wil happen when you rotate the picture is that a rotation flag will be changed - the picture itself will still stay unchanged.
  

Lesson learned: never believe 'general knowledge' if you can check the results for yourself - happy experimenting!