You have less worries
Yes, the sky is blue

C++ for C programmers, part 1 of 2

3 May 2010, by Ben    6 comments

As Stroustrup said, there’s no such programming language as C/C++. C and C++ are two different beasts.

I’m not going to argue here about which is better. For small or embedded projects, the complexities of C++ can get in the way. For larger projects, some of the features of C++ are very useful.

I’m simply going to list the features C++ adds to C, from an ex-C programmer’s point of view (yes, I was one of those people who believed that C++ was just “C with classes”). The idea is to follow the links to further information if you want to know more about any of them.

This is part 1 of 2, the non-OO features:

Please send your feedback, and let me know if I’ve missed any non-OO features. Next week I’ll publish the second part describing the object-oriented features C++ has added.

Oyster.com looking for Pythonistas

26 April 2010, by Ben    add a comment

An Oyster HotelOyster Hotel Reviews is the place to go for high-quality hotel photos and unbiased reviews.

But what might interest you is that they’re currently looking for experienced web developers, especially developers with strong Python skills. I’m slightly partial, as I’ve started working with them recently, but I can already say they’re great folks to work with, and the company is a happening web startup with a very positive trajectory.

Oh, and 37signals really likes Oyster.

Anyway, if you’re a Python web developer, you can apply via their Joel on Software job listing. The low-down for the position from Oyster is:

We are aggressively expanding and are looking for rock star software engineers to join our team.

The hired individual will work on projects including massive scalability and storage problems, information retrieval algorithms, internal tools and end user features. You should have a neurotic attention to detail and be willing to get your hands dirty building every aspect of a consumer website that will attract millions of customers. In this role you will primarily be doing development in Python and will be touching code at every level–user interface, core server infrastructure and the database.

We launched recently and have been written about in the New York Times, Wall Street Journal, LA Times, Economist, USA Today and many others.

Requirements

  • BS or MS degree or higher in Computer Science
  • 3+ years of professional development experience in Python is a must
  • You are a star; You should be able to code 3x faster than the other software developers you know
  • Experience in database programming
  • Experience with web development
  • Interest in working with other disciplines including customers (end users of the software you write) and product management
  • Must be a team player and open to change
  • Must have unrestricted work authorization to work in the United States

What We Offer

Joining our team full-time brings genuinely unique benefits:

  • 30″ monitors with the best computers money can buy
  • Amazing server infrastructure with which you can do very unique things (details in the interview)
  • We believe that one excellent worker is far better than two adequate performers; as such, we pay well and reward star performers as stars
  • Medical benefits
  • Dental benefits
  • 401(k)
  • Meaningful equity/stock options in a startup with a very bright future
  • Friendly coworkers who are the best in their respective fields
  • A seasoned and successful entrepreneurial leadership team that has already achieved great success and knows how to focus and get stuff done

To apply, submit your resume to jobs@oyster.com with “Python Software Engineer” as the subject.

Why I’m not moving to Linux just yet

24 April 2010, by Ben    10 comments

Linux has come a long way from when I first tried it about 10 years ago. Just not quite far enough, at least in certain respects. Call me a heretic, but I’m sticking with Windows for now (on my PC anyway — servers are another matter).

Preface: I’m not RMS, so I don’t shudder at the thought of proprietary software. And I’m quite willing to pay a reasonable price for good software.

But here’s why I’m not switching to Linux just yet:

Comparison of fonts in Linux and Windows XP

But I don’t want to be all moans and groans. Like I said, it’s come a long way, especially in the last few years. And I think the Ubuntu people have done a great job of packaging a decent, friendly OS that’s easy enough to use for The Average User. Here’s some really positive stuff about Ubuntu:

And to end on a software engineering (read: hackish) note, I’m going to make a crazy suggestion about improving fonts: I wonder if you could make Windows-rendered bitmap versions of (say) Verdana or Trebuchet MS and then use those in Linux?

DecentURL now requires an account

15 January 2010, by Ben    2 comments

So I finally gave up: DecentURL — my URL nicifier inspired by a reddit comment — now requires you to have an account to create URLs. It was just used too much for phishing, spam, and porn.

I tried various measures to reduce spam: deleting bad URLs by hand, bot detection, IP blacklisting, etc. None of them were very effective — most of the spammers were probably human, and they always seemed to find ways around my (admittedly simple) protection schemes.

Plus, DecentURL has always been just a neat little side project. But I just don’t have the incentive or time to keep deleting spam. Not to mention actioning emails from PayPal asking me to delete nasty phishing URLs.

I also realise that URL redirection services have their problems: they add another link in the chain, slowing things down and meaning a higher probability for an outage.

Most URL redirectors also obscure the destination. I like to think DecentURL does a bit better here, because it keeps (at least part of) the original domain in the final URL. So http://xxxyucksite.com/blahblah will turn into something like http://xxxyucksite.decenturl.com/z instead of http://tinyurl.com/z.

And I’d like to think DecentURL is still useful for turning URLs like

http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=& q=hagley+park,+christchurch&sll=-43.537552,172.617488& sspn=0.012802,0.01826&ie=UTF8&hq=&hnear=Hagley+Park&z=15&iwloc=A

into

http://maps.google.decenturl.com/hagley-park

So I’ve kept DecentURL running. All existing URLs will continue to redirect fine, but you’ll need a DecentURL username/password to make new ones. Just contact me if you’d like one.

MRO: Map Rows to Objects with web.py

6 January 2010, by Ben    2 comments

MRO is not an ORM. It’s not even the reverse of an ORM. Very simply, MRO maps rows to objects. It’s a thin layer on top of web.py’s equally thin database wrapper.

Why? Well, for a minimalist framework, I do like web.py’s close-to-the-SQL approach. But as soon as you have more than a couple of database operations, you find you’ve got code that repeats itself repeats itself. In the case of Gifty, I was repeating column names.

But I didn’t want a fancy ORM that gave me a new domain-specific language to worry about, or that supported every left, right, inner and outer join under the sun. I’ve found that for simple web apps, all I want is a bit of object with my row: a class that has attributes for columns, and lets you select and save rows.

I ended up with something that looks quite similar to Django’s database layer, albeit much simplified. (In hindsight, I may well have used Django for this project if I was doing it again.)

So here’s what MRO looks like:

# define a User object and its columns (SQL table name is "users")
class User(Table):
    _table = 'users'
    id = Serial(primary_key=True)
    username = String(secondary_key=True)
    hash = String()
    time = Timestamp(not_null=True, default='now()')

# create the users table with its columns and indexes
User.create()

# insert a new user into the database (defaults used for id and time)
bob = User(username='bob', hash='1234')
bob.save()

# fetch an existing user and update its hash column
bob = User('bob')
bob.hash = '4321'
bob.save()

# fetch an existing user (this time by primary key) and delete it
bob = User(42)
bob.delete()

# fetch an existing user (or None if no user called 'bill')
bill = User.get('bill')
if not bill:
    print 'Old Bill seems not to exist'

# get list of Users whose usernames start with 'ab' (also shows interpolation)
abusers = User.select(where='username LIKE $u', vars={'u': 'ab%'})

So, if you use web.py for a small web app, but you want a touch of class (ahem) with your database operations, go ahead and use MRO. Be aware that it’s an in-house tool (for instance, it only supports PostgreSQL at the moment).

Get MRO’s source code: mro.py

Go Forth and WikiReadit

4 December 2009, by Ben    5 comments

WikiReaderThe WikiReader is a little $99 gizmo that lets you read Wikipedia. Yep, that’s all it does. No mobile phone, no movie player, no Webkit-enabled browser.

There’s something about a product that does one simple thing well. But what really sets the WikiReader apart is that it lasts a year on 2 AAA batteries with no charging. How? The low-power LCD screen, and the tiny microprocessor.

But what’s even cooler, at least for someone who learned to program by dabbling in Forth, is that the device has a built-in Forth interpreter for testing the hardware and running small programs.

I was pleasantly surprised – I know that Forth is good for embedded work on tiny micros, but since the main WikiReader app is written in C, I was curious why they chose Forth for testing and apps. So I asked Christopher Hall, one of the main firmware developers. His reply was very informative, and he’s kindly allowed me to copy it here:

I have written testing programs in several languages, but compiled programs always have the problem of the edit, cross compile, load, and try to debug. Sometimes the platform can run BSD or Linux, and then you can have the full suite of tools on the platform. This is okay if the person doing the initial testing can write programs, but often the test is how to toggle a particular I/O line on/off and see the effect on the rest of the circuit. Then having some kind of scripting on the platform seems the best way to achieve this.
For the initial testing, just start the interpreter REPL and you can start the initial tests. Initially I looked at TCL and Python which I have used before, but they would take far too long to port since they need a lot of Posix system calls which do not exist for this platform.
I also considered Hedgehog, Pico Lisp or perhaps some simple Scheme interpreter but the syntax would probably be too difficult for the hardware engineers to use. Forth is pretty simple syntax and RPN was probably not too difficult for them to learn. Also it was easy to build the Forth interpreter, incrementally adding features until it is now an almost ANSI standard Forth.
Since I added all the device registers the hardware engineers can use commands like the following (I used the same register names as the datasheet):
P0_P0D p?    \ display value of port
1 P0_P0D p!  \ set port to 1
While waiting for the main application development I could build tests for items like the LCD and CTP with just a serial connection on the device itself – using cut/paste from Emacs to picocom to upload Forth words. This is much quicker than cross-compiling and swapping SD cards.
The Forth is rather slow in compiling, the dictionary search is quite slow for example, and the indirect threading adds run-time overhead so in its present form it is probably not fast enough for the main reader application, but for quick applications to try things out I find it very convenient.
Also, the first version was hand translated from a version of EForth for Linux before I migrated it to the ANSI standard. (I kept copies in samo-lib/forth/EForthOriginals subdirectory.)

Very neat. If Lisp is the secret weapon for developing web apps, maybe Forth is it for embedded apps. Both are extensible at the language level and both have real macros, but Lisp is high level and Forth is low level.

Well, you know what to buy me for Christmas:

feeling-nice? if  WikiReader buy  then

catdoc ported to Windows

15 September 2009, by Ben    3 comments

Recently I had to automatically extract text from a bunch of Word documents under Windows. I liked the looks of catdoc, but didn’t see a native Win32 port around. The source code looked so very close to compiling under MinGW, so I made the few minor changes necessary and got it working (catdoc, catppt, and xls2csv). Native Win32 executables, support for long filenames, etc.

Basically all I did was:

Nothing special, and it’s not perfect. But here is a zip of the compiled binaries and (GPL-licensed) source code, just for you:

catdoc-0.94.2-win32.zip

Code generation with X-Macros in C

21 August 2009, by Ben    6 comments

C and C++ are relatively non-dynamic languages, and one thing this means is that not repeating yourself (aka DRY) is often harder than in a language like Python.

For instance, when you’ve got a config file, a config structure, config defaults, and a config printer, you want all those things to come from a single spec. One good way around this problem is code generation — for example, using an XML spec with Python and Cheetah templates to generate C code.

But for simple C projects this can be overkill. And it turns out the age-old C preprocessor contains a few goodies that help with DRY programming. As the Wikipedia article says, one little-known usage pattern of the C preprocessor is known as “X-Macros”.

So what are X-Macros?

An X-Macro is a standard preprocessor macro (or just a header file) that contains a list of calls to a sub-macro. For example, here’s the config.def file for the INI-parsing code we’ll be looking at (uses my simple INI parser library):

/* CFG(section, name, default) */
CFG(protocol, version, "0")
CFG(user, name, "Fatty Lumpkin")
CFG(user, email, "fatty@lumpkin.com")
#undef CFG

That’s an X-Macro that defines a config file with a protocol version and user name and email fields. If we weren’t following DRY, our main code would specify the field names in the struct definition, repeat them for setting the default values, and repeat them again for loading and printing the structure.

To do this in X-Macro style, we just #include "config.def" repeatedly, but #define CFG to what we need each time we include it. Sticking with show-me-the-code, here’s a program that loads, stores, and prints our config:

#include <stdio.h>
#include <string.h>
#include "../ini.h"

/* define the config struct type */
typedef struct {
    #define CFG(s, n, default) char *s##_##n;
    #include "config.def"
} config;

/* create one and fill in its default values */
config Config = {
    #define CFG(s, n, default) default,
    #include "config.def"
};

/* process a line of the INI file, storing valid values into config struct */
int handler(void *user, const char *section, const char *name,
            const char *value)
{
    config *cfg = (config *)user;

    if (0) ;
    #define CFG(s, n, default) else if (stricmp(section, #s)==0 && \
        stricmp(name, #n)==0) cfg->s##_##n = strdup(value);
    #include "config.def"

    return 1;
}

/* print all the variables in the config, one per line */
void dump_config(config *cfg)
{
    #define CFG(s, n, default) printf("%s_%s = %s\n", #s, #n, cfg->s##_##n);
    #include "config.def"
}

int main(int argc, char* argv[])
{
    if (ini_parse("test.ini", handler, &Config) < 0)
        printf("Can't load 'test.ini', using defaults\n");
    dump_config(&Config);
    return 0;
}

Note that config.def is included 4 times, so you’d have to repeat yourself 3 times with no X-Macros. I admit it’s not beautiful artwork. But it’s not too ugly either — and it gets the job done with nothing but C’s built-in code generator.

Site Doublers: Website optimization

7 August 2009, by Ben    add a comment

Recently we’ve been running some Google AdWords and doing some SEO (Search Engine Optimization), and I must say it helps to know what you don’t know.

John Hyde of Site Doublers has been a great help on this score. He’s a consultant that helps you as a business optimize traffic to your website, via search engines and advertisements, and helps you convert visitors to sales once people are going to your website.

John’s been very professional to work with: he knows what he’s on about, he asks the right questions, and he does his homework. All of which to say, if you run a website, talk to John on +64 3 942 3799 or visit his website:

SiteDoublers logo

P.S. And no, John didn’t pay me to write this. :-)

fabricate: The better build tool

28 July 2009, by Ben    add a comment

We’ve been using Bill McCloskey’s memoize to build projects for a while now, and it works nicely, but only on Linux.

So enter fabricate. It was developed by us guys at Brush Technology for in-house use, but we thought it was cool enough to release into the wild.

From the project page:

fabricate is a build tool that finds dependencies automatically for any language. It’s small and just works. No hidden stuff behind your back. It was inspired by Bill McCloskey’s make replacement, memoize, but fabricate works on Windows as well as Linux.

Read more articles …