Disqus - Friends of crispy

Re: RockStarProgrammer - Greetings

dustin — Tue, 15 Aug 2006 04:07:00 -0000

That's "conceited bastard" to you. :)

Re: RockStarProgrammer - More Than You Want to Know About Java Serialization

dustin — Thu, 17 Aug 2006 15:49:00 -0000

Hey, thanks. I wasn't even aware that they added XMLDecoder and XMLEncoder. Most of the effort that led to this post was in regards to space efficiency of serialization.

Re: RockStarProgrammer - Day One

dustin — Tue, 29 Aug 2006 22:44:00 -0000

Yeah, I get the feeling that people are generally pretty smart here. At my Last Company (I really hate saying that), it definitely started out that way, but they slowly lost interest in hiring good people. Not that they were opposed to it, but hiring people had a higher priority than hiring good people.

It's mostly the weird tools that are making things more of a burden to an open and productive development team than anything else I'd consider being in the way at this point. I don't want to walk in and try to chane everything right away, though.

I've got a machine that's unsupported by IT. I'll be quiet with it or a while, and then maybe see if I can get people using something like a real wiki or similar communication tool.

Re: RockStarProgrammer - Space Efficient Encrypted Backups

dustin — Thu, 31 Aug 2006 14:32:00 -0000

That is correct. My data is a stream, and the largest things going into it are append-only. I have occasionally done larger changes that have caused huge stream changes, but those are more rare.

I've done similar work with other types of things as well. For example, mail folders. Those are also append-mostly (assuming order is enforced). Deltas are small.

This mechanism is working so well for me, I thought it'd be a good idea to write about it in case it's a fit for anyone else.

Re: RockStarProgrammer - Outsourcing IT?

dustin — Tue, 19 Sep 2006 12:22:00 -0000

Yeah, I guess that's about it. It's similar to the feeling I got when I stopped running NetBSD on my desktop and started running OS X. It seemed wrong in a way, but left me doing more. I no longer have my pimped out twm config that lets me point to focus and just press “x” on the background to get an xterm to pop up instantly, but I can write much better software in Cocoa than I ever did in...well, any X toolkit I tried.

I still like doing portable software development when there's not a GUI component, but some projects aren't going to be interesting forever.

BTW, do you get to keep your inferior human organs?

Re: RockStarProgrammer - If I Could Remove Two Keywords from Java...

dustin — Wed, 28 Feb 2007 03:17:00 -0000

Jonathan,

The syntax isn't as important to me as the semantics. Of course, none of this matters as nobody is going to change java to suit my desires. These are just a couple of things I've seen used in ugly ways that were bothering me.

You make a good point about single entry, single exit checking being something that could be implemented by the compiler or another code checking device that could be plugged into the build system can validate that nobody is doing such a thing in a particular team.

Re: RockStarProgrammer - Fast and Easy Token Classification in C

dustin — Tue, 15 May 2007 01:01:00 -0000

It's possible...I don't have K&R handy.

A for loop over the structs would yield similar performance to the original, so I don't know that I'd worry about that one too much. One could sort the array, but it'd still be more comparisons than what I'm doing now.

The hash function could certainly work, and someone else had suggested that. I suppose it is an obvious way to do it and would still apply to case insensitive searches.

I think the main reason I didn't use the hash function was because I sort of approached this problem completely backwards.

I wasn't trying to actually create a fast routine for matching strings. I was looking at a piece of someone else's C code that was doing this sort of matching and wondering if it was really worth it. So I started wondering how I'd do something like that. Then I wondered if it mattered anyway.

Somewhere along the way, I'd completely forgot what the original point was and just went to bed. I figured someone might be able to use the results, though.

Re: RockStarProgrammer - Java Exception Antipatterns

dustin — Mon, 11 Jun 2007 13:37:00 -0000

This is what I meant when I said I understand the theoretical value in checked in exceptions. In theory, you're correct and proper use of checked exceptions would raise awareness and people would deal with problems sooner rather than later.

In practice, you're also correct in that the JDK itself has a lot of bad uses of checked exceptions that make it more difficult to do things right.

It's rarely a specific caller that can do anything useful with an exceptional condition. It's generally something that's useful way up top. If I'm ten or so frames down a stack trying to initialize something and a network resource is unavailable and I can't do my part, it seems that the only right thing to do is to throw an exception...which is what is going to happen anyway.

And this is the basic issue. People catch exceptions when they absolutely shouldn't because the compiler tells them they have to deal with a particular problem.

Personally, I have seen way more bugs from people catching exceptions than not catching them. Keep in mind an exception has to have occurred either way. Whatever actually caused the exception is where the problem with the code is. Your best bet in most parts of most applications is to leave that information alone and let your top level error handler sort it out as a general failure.

Re: RockStarProgrammer - Empty Catch Blocks are Always Wrong

dustin — Sun, 17 Jun 2007 14:35:00 -0000

@nope

Well, first, that's not an empty catch block. That's a catch block with a comment describing why you *always* assume that any NumberFormatException means don't run your batch job.

However, your code still has bugs that it's hiding from you. For example:

What if the code providing profileId as a string is providing a valid number with leading and/or trailing whitespace? It's not hard to think of ways this might happen with user-supplied input.

What if you burn through enough IDs that suddenly your IDs are greater than Integer.MAX_VALUE? I don't know where your IDs come from, but many places like to use DB sequences which are actually much larger than ints, but still represent them in an int type.

In either of these cases, your batch job is *mysteriously* not run while providing no information as to why.

If you honestly consider that an integer that cannot be parsed *always* means you've run into a case where you don't want to run your batch job, what's wrong with at least logging the decision at debug or trace? Personally, if my system is receiving invalid input, I like to at least be able to know what it looks like.

Re: RockStarProgrammer - Please Wake Me From My Hibernation

dustin — Sun, 24 Jun 2007 14:52:00 -0000

@Jay

Hey, thanks for the useful info. I actually agree with much of what you said.

Regarding IDs, I generally ID my objects in my own projects at creation using an incrementing integer mechanism that's DB backed. It works somewhat like a sequence, but nodes cache chunks of them. It's absolutely no performance difference, but you end up with IDs that are a little easier to deal with.

For unit testing DB stuff, I've written a JDBC driver that makes it really easy to write DB tests. There's no DB involved, and I can simulate any sort of behavior for any sort of request.

The upgrading is interesting. We're deploying an .ear, but it sounds like it might be similar. I've run into conflicts with different versions of the same class in different classloaders, so I try to avoid that.

I agree that hibernate probably saves time relative to some other ways of doing things. The mechanisms I've developed in my own projects are actually really efficient for both the developers and the machines. Hibernate meets some edge cases that mine doesn't, of course. However, I've been blocked by some of the things hibernate is supposed to do that it fails to do correctly.

I also agree that not everybody on the whole team should have to deal with it, but we shifted from working on layers to implementing vertical features. In doing so, we each have to make a small data model change to support our respective tasks and things break.

I do not agree that I need to spend 5 or 6 hours reading a book before trying stuff out. I did decent work in struts, gwt, guice and several other frameworks without reading a book. Same with erlang, ocaml, etc... It almost seems that if it requires reading a book, it's too complicated and non-obvious and needs to be simplified some.

Re: RockStarProgrammer - The Fastest Month Parser in the West

dustin — Thu, 28 Jun 2007 16:55:00 -0000

I've looked at the listing and suppose I should qualify that I don't think that my binary has the fastest possible matching.

However, it seems to me that any additional optimization is really just working around a compiler bug that is likely to be fixed at some point.

There's no reason for a compiler to produce sub-optimal code for a switch. However, I don't *really* know what's optimal on a given chip. For example, my ancient SGI compiler generates the following at its highest optimization level:

blt $2, 1249209392, $34
beq $2, 1249209903, $40
beq $2, 1298231855, $37
beq $2, 1298233647, $39
beq $2, 1315927599, $45
beq $2, 1331917871, $44
beq $2, 1399156783, $43
b $47
$34:
beq $2, 1097888303, $38
beq $2, 1098213167, $42
beq $2, 1147495215, $46
beq $2, 1181049391, $36
beq $2, 1247899183, $35
beq $2, 1249209391, $41
b $47

It decided to break the switch into two parts and do six beq ops on each side. It might be that the cost of doing more blt ops is greater than the worst-case on one side of the branch, so splitting it further would make things worse.

Of course, this is more balanced than the GCC listing, but it'd be good to consider icc's case. I started doing that, but I'm at work and have spent more time than I intended to spend analyzing this problem. Here's a poor-man's assessment, though:

% repeat 5 ./parsemonth-gcc
3.911u 0.024s 0:04.02 97.7% 0+0k 0+0io 0pf+0w
3.899u 0.021s 0:03.97 98.4% 0+0k 0+0io 0pf+0w
3.903u 0.022s 0:03.98 98.4% 0+0k 0+0io 0pf+0w
3.897u 0.025s 0:03.99 97.9% 0+0k 0+0io 0pf+0w
3.905u 0.021s 0:03.99 98.2% 0+0k 0+0io 0pf+0w
% repeat 5 ./parsemonth-icc
3.392u 0.018s 0:03.54 96.0% 0+0k 0+2io 0pf+0w
3.391u 0.015s 0:03.44 98.8% 0+0k 0+0io 0pf+0w
3.393u 0.016s 0:03.44 98.8% 0+0k 0+0io 0pf+0w
3.391u 0.016s 0:03.44 98.8% 0+0k 0+0io 0pf+0w
3.393u 0.017s 0:03.44 98.8% 0+0k 0+0io 0pf+0w

Note that the above is http://hg.west.spy.net/hg/s... modified to run 1,000,000 iterations on my old macbook.

Re: RockStarProgrammer - The Fastest Month Parser in the West

dustin — Fri, 06 Jul 2007 23:32:00 -0000

@Kelvin

Well, that'd be a bug in whatever produced the logs in this case. However, it's not a hard problem to deal with. I just have to lowercase them in my switch and add a tolower() from ctype.h for each one. It doesn't increase the algorithmic complexity.

In the current code, that'd be considered an invalid line (which I'm not too worried about).

Re: RockStarProgrammer - How to Look Like an Idiot

dustin — Fri, 21 Sep 2007 19:15:00 -0000

I do think yours is a fine argument, although it really hasn't affected me much in practice.

Your argument is welcome here. This is my web server and I brought it up.

The *primary* point I had, though, was that a ``python is ugly and I'll never use it because it makes you indent properly and Guido is an idiot'' argument doesn't belong on an erlang list.

Specifically, there aren't a lot of people looking at this page hoping to learn something about erlang or python or any other specific topic, so if you wanted to argue that ocaml is so ugly and unmaintainable that you'll never bother trying to use it, you at least wouldn't be causing a large annoyance here. I'd still disagree with my most recent consulting check in hand, though. :)

Re: RockStarProgrammer - Wanted: Git Cheat Sheet for Collaboration

dustin — Wed, 06 Feb 2008 13:10:00 -0000

Codeless,

``git show | head'' is pretty close to ``hg id''

yada,

``hg bundle'' produces what is effectively a diff between what's available in a given repository and another repository. It's a compressed binary format that contains all of the changesets in such a way that can be easily transported as a single file and applied to a repo (even if there's overlap).

So if you clone someone's tree, you can make some changes, do some commits, ``hg bundle /some/file'' and email the file. The other side will ``hg unbundle /some/file'' and have your changes.

From what I can tell, ``git archive'' is the same as ``hg archive'' -- it just produces an export of the full tree in a tar or zip file.

Re: The top 5 Digg Tools to Make you a Better Digger

dustin — Fri, 15 Feb 2008 13:46:57 -0000

All this talk of RSS, and no mention of diggwatch:

http://bleu.west.spy.net/di...

Re: RockStarProgrammer - The Differences Between Mercurial and Git

dustin — Mon, 07 Apr 2008 13:46:00 -0000

@confused - thanks, corrected the typo

Re: RockStarProgrammer - The Differences Between Mercurial and Git

dustin — Mon, 07 Apr 2008 15:32:00 -0000

@robewald

That's true, the reflog is great. I was referring more to the published history. There are tools (mq again) that can do this same sort of thing in mercurial, but it's not common practice.

@Miran

That's true. I *almost* mentioned it, but it was too wordy as it is.

A branch is any point where a changeset has more than one children. You can have unnamed branches the same way as you have named branches, but it's a bit harder to keep up with.

You're right about fewer commands, though. Darcs 2.0 is out today, and should be considered the goal to approach in user interface.

Re: Some thoughts on git

dustin — Thu, 03 Jul 2008 04:21:31 -0000

I don't see the problem with the source code. I had the opposite experience, really.

I wanted to add a feature to git. I found it fairly easy to figure out where to put it and how to write it. I did make a couple of mistakes in my initial version (which were pointed out to me early). Making git do what I wanted it to do was easy. The hard part was getting it accepted.

Junio has got to be one of the busiest software volunteers on the planet, but he meticulously reviewed my code for both style and function. I had to send it through several times and wrote *way* more unit tests than code. In the end, it made my confidence *really* high.

Re: Twitter Pseudotrack

dustin — Fri, 11 Jul 2008 17:23:44 -0000

Hey, it seems that a lot of people had the same idea around the same time. It's unfortunate it has to happen, but the reaction is somewhat revealing (i.e. users like it a lot, but twitter, summize, et. al are at best neutral to the idea).

twitterspy in particular is ``slower'' in the sense that it polls for a given track no more than once every ten minutes (per configuration). I configured it to be as nice to summize as possible. For example, I've got a peak request rate against summize, so if online users and tracks grow great enough, it'll slow down even more as starts deferring updates. Tracks are only updated for online users, and are shared, so it's possible that there are times of the day when I'm not hitting summize at all.

I could make it faster, but I get the impression that they already don't like me enough. :/

Anyway, glad we have more people doing something about things that are annoying. I kept dealing with the bait from twitter about how they're going to bring back XMPP RSN until I realized twitter was getting nearly useless for me.

Re: RockStarProgrammer - What's Up? — An XMPP-based Web Monitor

dustin — Fri, 19 Sep 2008 16:41:53 -0000

Interestingly, there was very little response to this -- a service I figured everyone would like. Everybody does seem to like twitterspy. Oh well, I still like whatsup.

Re: RockStarProgrammer - Automating Git Bisection for Rails Apps

dustin — Sun, 28 Sep 2008 12:33:47 -0000

Hey, great idea. That seems a lot less messy than my stash script. Thanks.

Basically, that'd be something like this (assuming working on master with a bug that didn't exist about 50 changesets back):

git checkout -b findbug
[write and commit tests]
git rebase -i HEAD~51
[move test changeset to the top]

Then you should be able to do the easy case as described above.

Re: RockStarProgrammer - What's Up? — An XMPP-based Web Monitor

dustin — Sun, 28 Sep 2008 21:54:46 -0000

Neat, glad someone found it useful. :) It's been indispensable to me.

I did use DM here just to learn it. The one thing DM does *really* badly that I'm too lazy to fix is http://wm.lighthouseapp.com... (that is, you can't cache objects).

Re: Memcache, keeping data in the handiest place: memory

dustin — Sun, 05 Oct 2008 01:04:07 -0000

Just for clarity (since this has come up on the list), every memcached operation is atomic, but the cache is not transactional. The CAS operation exists in the text protocol for performing an atomic read/update operation, and is extended to every operation in the binary protocol.

Re: RockStarProgrammer - What Matters in an Asynchronous Job Queue

dustin — Sun, 05 Oct 2008 15:46:58 -0000

Thanks for the comment.

I agree that queue durability is desirable, it's just a really low priority for me. I've just seen at least two queue systems built on top of memcached recently and it's getting a bit tiring. In one of the recent ones, the author didn't want to use starling because their company had no ruby expertise, gearman because they had no perl expertise, and beanstalkd because it didn't have queue durability -- so they wrote a new one.

I crash job workers all the time. We have various god config to kill the workers for various reasons so I get restart IMs at various points in the day.

The main point of this post was to challenge people who keep writing new job queue services to consider more than just this one aspect everyone seems to be focused on. Or maybe it's to convince them that if an existing one doesn't do what they want, that they should work on enhancing it slightly.

If people want starling, they should just use it.

You're right, though, if you want to guarantee every job, you've got a huge task ahead of you and hopefully very simple jobs.

Re: RockStarProgrammer - Using Git Bundle When Your Central Repo Fails

dustin — Mon, 06 Oct 2008 14:29:05 -0000

Thanks a lot for the pull tip. Admittedly, I was in a bit of a hurry to actually get my code deployed, so I just did the first thing that worked. :)

pull definitely looks more natural.