Emacs tip #1

As I become more familiar with emacs, I'll post random short snips that aren't exactly easy to find on the internets.

I ran across these instructions for installing emacs with modern fonts on Ubuntu. Visually, this is a radical improvement over the previous state of affairs. It's like Emacs joined the 21st century :).

Sean

Our new Emacs overlords?

As programmers, we should fret over our tools. Well, I don't know about everyone else, but I fret over my tools. This leads me to spend time learning tools that would cause normal people, even normal programmers, to shy away.

I'm a programmer of many languages, so I find IDE's, while helpful sometimes, are not something I can universally rely upon. For instance, I've yet to see a C development environment that is more productive than VIM and GCC on Linux, and this is not for want of looking. When I use more esoteric languages, it's often that the closest thing to editor support I'm going to get is some syntax highlighting in VIM and Emacs.

When I was a teenager being first exposed to the world of Linux, for some reason (I really don't remember why), I learned VIM first. I got pretty good at it. I could use VIM to produce C code like nobodies business. I find VIM the optimal editor for Perl coding. I have several hundred VIM shortcuts memorized. In QWERTY. In Dvorak, I know something like 20 VIM commands.

This is a problem. I've spent the last year hobbled in my VIM usage. I feel like a complete novice. I'm not even sure exactly what key to press to yank without looking at the keyboard. Advanced editing is way out of the question. It's as if I've been using the big blade on a Swiss army knife to cut cheese, leaving the other blades untouched and unused.

This is obviously not optimal. While I still beat out most other editors (except the all-mighty Eclipse/Java pair), I know my efficiency has been hurt. I know my code is suffering. I don't even try to do LaTeX in VIM anymore. So it's time for me to re-evaluate my tools.

I tried learning VIM for Dvorak. I really did. I spent a few hours running command practice. I printed a cheat sheet optimized for my Dvorak troubles. I've stuck with it for a year, hoping that I'd learn the keys by repetition the same way I did the first time. No such luck. Dvorak + VIM is an illogical beast of several hundred irrational keys to memorize. There's no core logic to the keyboard layout that justifies the common choices, and none of the keys make sense. To make matters worse, I still haven't gotten the bottom row of Dvorak imprinted into muscle-memory so I miskey unless I'm typing a word. This simply isn't working.

Which leads me to Emacs. I decided years ago that "some day" I would learn Emacs. Of course, picking up Emacs as a skilled VIM user is not really an easy task. It's about as hard as a skilled QWERTY typist switching to Dvorak. In the beginning, you spend absurd amounts of time looking up basic commands, killing your efficiency. I'm fairly certain, if I hadn't made the switch to Dvorak, I'd never get to the Emacs switch. It's a big investment, and I'm putting my time in to become proficient. I've been using Emacs extensively for three days now and I'm starting to get the basics by route. It's definitely feeling more Dvorak-friendly than VIM ever did (though that could be because I had experience with Vim in QWERTY). And overall, I'm pretty happy with the switch.

Which brings me to the point of this post. Everyone with a keyboard made in the last 3-5 years has noticed (or not) a little button on the right side that's allegedly a "menu" button. Allegedly. On occasion I've accidentally pressed it and weird things have happened, usually context menus. It's always been a mystery to me how this button made it onto keyboards, as it's clearly the most useless button ever. That is until I started learning Emacs, when I realized it's a shortcut for M-x. I think this is clear proof that there is a vast Emacs using keyboard conspiracy out there, and we should probably start protesting the Emacs-illuminate.

PS: If you read this, and you are still using QWERTY, you should hop back up to the first paragraph. The only keys that I'm not happy about in Dvorak are {}, but even that is pretty easy for me to deal with (not that I voluntarily use a curly-bracket languages any more). In all other ways Dvorak is vastly superior, and I've never heard a good argument not to use it other than "I don't feel like learning it."

Sean

jdodgl G xfgk f;glu h.soave

Kjg; rs;k g; kjd pa;k rs;k G ,gpp d.do mavd gl h.soave Flysokflakdptw G ja.d apodaht ijaludh mt vdt marrgluw ;s G hsfnk kjd .a;k mavsogkt sy kjd glkdoldk ,gpp nd anpd ks odah gke Gy tsf malaud ks ygufod gk sfkw a,d;smde

Alt,atw h.soav ,a; a,d;smdw alh G odappt pgvdh gkq; dlupg;j ktrglu da;de Flysokflakdpt kjd .a;k mavsogkt sy f;do glkdoyaid; aod mahd yso qwerty alh kjf; G am usglu naiv ,.dokte Mt jalh; apodajht jakd mde

Goodbye dvorak.

Web frameworks, aka low hanging fruit.

I was reading Shannon Behrens's post about how everyone writes a web framework.

I think he hit the nail on the head. Web programming feels like a cesspool of knockoffs and "me-too" developers. Of course, brilliant people throw their hat in every now and then, but the overwhelming noise drowns it out.

I think there are a few types of framework creators. Those that have written a few applications of the same form and evaluate how to make similar jobs easier by abstracting part of the coding away. I've written a framework like this once - for interfacing a hospital registration system. These sort of frameworks are generally very useful, and have very limited use cases. On the other hand there's the J2EE style framework. A bunch of people with some minor experience in the project domain get together and attempt to predict every situation their framework could ever be used in. I've had the unfortunate experience of using a framework like this once, and I will never do it again unless I'm literally starving and it's the only job I can get.

The third example of what gets called a "framework" is really a domain specific language. Ruby on rails, while probably the leading example of this, is not really a good demonstration of this power. A good domain specific language will allow non-programmers to believe they are scripting the system, while hiding the abstractions so far that they never leak. Visual Basic 6 is a good example of a well implemented domain specific language, and I haven't really seen a comparable example in the web framework market.

Anyway, I'm starting to get off topic. I wanted to concur, the reason we see so many web frameworks is that it's such low hanging fruit. I had to code a web server as a 2 week homework assignment once, most web-frameworks I've seen really aren't even as complex as a web server.

Back when I was an undergraduate it was all the rage for professors to say dumb things like "Your projects must have a web component." Apparently, they thought this was somehow harder than writing a high quality GUI app (using unexplainable logic). Usually we'd hear motivations like, "It's a good exercise, you will learn sockets." Sockets! Sockets!?! You heard me right, they thought we'd end up using sockets to write our web based programs. And you know what the worst part is? We did.

Here's how the discussion on such a project normally went. Hey guys, we need to throw together a web component, what language is everyone familiar with? Java Java Java C++. Ok well, C++ is out, so Java it is. Alight, does anyone know any Java tools for building web pages? "Uh, I sort of know JSP." Ok, then lets discuss the requirements. 45 minutes later we finally re-invented a login system (much to my annoyance... everyone loves rambling about web authentication). Then we'd move on to the database layer we'd spend an hour debating and my push for ORM's would always win (which I would then be responsible for creating). Then the core application structure would come up. Because of the requirements we'd make a 3-tiered architecture and me and the other smart kid on the team would pretend it was completely designed after a while and adopt responsibility.

Then came coding time, this is where the story gets interesting. I'd go to make the ORM. Mind you, while I've never been entrenched in the Java "frameworks" field, I'm aware things like "A library that converts my database into usable stuff" should exist. Of course, due to Java's dead-code static nature, I now realize it can't exist (making new types on the fly? hah). I'd spend 30-40 minutes looking at the current "trendy" database technologies, before realizing it'd take me longer to figure out the "framework" than it would take me to just write hand-rolled code for our 15 domain objects, further my code would make more sense to the rest of the team, as it matched our domain modelling discussions.

Then we'd move on to the "framework-y" bit. The glue code that handled web requests and sent requests to the database server. Every time we ended up using raw sockets. Not that we didn't look to see if someone else had written a better "web" framework, we always did. And after an hour of looking we came to the conclusion none of them could possibly be productive, and we'd just hand-roll something ourselves. In one project we used a session id based session manager, completely hand rolled. It took me about 5 hours to put everything together including the hand-rolled multithreaded server. All of this I of course considered a phenomenal waste, but there were really no better options. On the other project we used a python script to generate C# classes from a description of our over the wire protocol. This was a much better solution, and I was generally pretty happy with how that project turned out.

So after all that, I'm qualified to write a web framework, right? Obviously not. Yet, if I had a bigger ego and no self control, I'd get tapping away right now to release my new and improved web framework to the world. Naturally, it wouldn't solve any of the hard problems (how to separate presentation logic from HTML/Javascript crap code being the hardest), but it would undoubtedly be a "web framework."

As an afterthought I just noticed why all web frameworks aren't. They usually try to solve a few problems at once, in the same language. This is obviously not going to raise the level of thinking - Java describes Java, Java doesn't describe a ORM. An ORM can be built using Java, but to say Java does anything but allow it to happen is overstating Java's readability.

On the other hand, after our very successful over the wire code generation on our second project, I'm now thinking perhaps a framework should be about code generation. I cited two really annoying parts to program, both where I had to map a list of nouns to Java. The ORM transforms databases into objects that represent the domain model. I am fairly sure a very strong domain specific language that generated Java/C#/... code would be a perfect match here. And of course the wire stuff was already solved by us using a code generator / DSL.

PS: On one project we used Java beans for authentication, which worked semi-acceptably... it was only 150 loc to say "this app needs authentication". On the other app I think he hand-rolled authentication, though I wasn't involved.

Sean

Hocus-pocus, we have recursion!

I was recently reading Reg Braithwaite's post "But Y whould I want to do a thing like this?" when from some sense of foreshadowing I decided on first few paragraphs to answer the question "How do you curry a y-combinator?"

Naturally, I chose scheme instead of ruby (I find scheme is the easiest language for dealing with complex lambda thinking, for obvious reasons).

My very first stab at the problem looked like this.

(lambda (y)
  (lambda (x) 
    (if (eq? x 0) 5 
        (y (- x 1)))))

This is obviously wrong, though it seemed right enough that I typed it out. One might gain some insight from the way I initially thought about this problem.

"The inner function won't be visible to the global environment, so there would be no way for someone to reference it while calling. To solve this I can reverse the order of the parameters and pass the function to the outer lambda. This should make things work."

Now, naturally, if you're familiar with scheme this function obviously won't work after thinking it through. Say we called it define-five (you'll note this toy example always returns 5 for positive integers), we might call like ((define-five define-five) 1) which sets y to be define-five. Now when the recursion is attempted it will call (define-five (- 1 1)). Of course, this will return a function... because the return value of define-five is a lambda expression!

Obviously, I must have been wrong about my parameter ordering, so I'd try currying it the other way, giving it a quick attempt I started with.

(lambda (x)
  (lambda (y) 
    (if (eq? x 0) 5 
        (y (- x 1)))))

Tracing through this, we find out it won't work at all. The return value of define-five is now a y-combinator with a fixed x value. I decided this might be a little harder problem than I anticipated at this point and sat down for a few minutes to think it through. After some thought, I decided to investigate using let. A few false starts later I realized let isn't needed, but it does simplify the code a lot. Here is the shortest version of the curried y-combinator factorial function I've written.

(define fact-generator
  (lambda (y)
    (lambda (cur)
      (if (eq? cur 0) 1
          (* cur ((y y) (- cur 1)))))))

This is the first working example on this page. You'll note that in the process of making it work I switched it to a realistically testable example (the standard factorial). I will use the let-ified version below for explanation because it is much easier to read.

(define fact-generator
  (lambda (y)
    (lambda (cur) 
      (if (eq? cur 0) 1
          (let ((rec-fun (y y)))
            (* cur (rec-fun (- cur 1))))))))

This is pretty exciting. Before I go too much further, lets show what it looks like to call fact-generator.

> (fact-generator fact-generator)
#<procedure>
> ((fact-generator fact-generator) 5)
120

To walk through this function is fairly informative. We note the y combinator is passed to the outer lambda, which we deal with later. The inner lambda starts out like all recursive factorial functions, if cur is zero return one. Then we have the clever bit that took some effort to arrive at. Since we are passed a function that generates recursive functions when passed itself, we go ahead and call it on itself and it returns a recursive function! Note that since y is fact-generator, rec-fun is being set to (fact-generator fact-generator) which of course is the inner lambda with an appropriately bound y! Of course we finish by multiplying cur with the result of the recursive call one smaller.

I then went back to reading the linked post, in which he starts talking about y-combinator generators. At first I thought we were talking about the same thing (mind you, my ruby is shaky so I had trouble reading some of his code), but over time I grew more concerned that I had solved the same problem as him. I went back over to DrScheme to see if I could code a function that takes a y-combinator-generator and generates a "recursive" function. For some reason, after reading his ruby I didn't see the result right away... I'd already written that function.

(define gen-recurse
  (lambda (fun-generator)
    (fun-generator fun-generator)))

> ((gen-recurse fact-generator) 5)
120

Of course putting it all together in glorious scheme goodness gives us the following

(define rec-fact
  (let ((rec-generator 
    (lambda (y)
      (lambda (cur) 
        (if (eq? cur 0) 1
            (let ((rec-fun (y y)))
              (* cur (rec-fun (- cur 1)))))))))
    (rec-generator rec-generator)))

> (rec-fact 5)
120

And we have transparently created recursion from nothing, such that people calling rec-fact have no idea it's even defined recursively. Score one for lambda.

Sean

Interesting stuff

I just ran across this post about a language called Io. Despite it's unfortunate name, it looks to have a lot of interesting ideas brought to fruition.

I must play with it more when I have time.

Review: An introduction to python.

In an effort to actually write some content for this blog, I am going to start posting notes to particularly good books that I've read recently.


An introduction to python is the best book I've ever read on the subject of Python programming. It's a very short book - for a very short language. The biggest advantages to this book over the others intro books is the example code and explanations are pythonic, also this is one of the least expensive programming texts around.

Mind you, this book won't teach you to program. It won't teach you to deal with Twisted, or web.py, or any other library. This book will teach you the core Python syntax, semantics, and way of thinking. It assumes, quite strongly, that you have experience in programming. It assumes you're going to continue reading about framework x.y.z you want to use in python. However, it does what it sets out to do and it does a beautiful job.

If you set out to read this book, I suspect it will take you less than a day to finish (I finished in about 3 hours). So I'll provide a brief overview of some of the cool parts and why you should read this, but I'd recommend just buying it (it's cheap!) and reading it for yourself.

Chapter 1 and 2 are everything you need to know to get started using a python interpreter. Sum total of 11 pages, and not a single wasted screenshot of how to install it on platform x, y, or z. Readers would be well advised to note that ipython is a much better shell than the default implementation, but obviously a book on the language can refrain from saying things like that.

Chapter 3 is the standard "whirlwind tour" of the language, starting with a calculator and ending with while loops. There is little value to this chapter excluding syntax issues (if you come from another language).

Chapter 4 runs through the rest of the control flow syntax, a typical example quality is, after defining a working Fibonacci number calculator, they proceed to explain the basics of functional programming. If you already know how to program, this makes sense, if you only know Java this chapter might get a bit hairy.

<<fb
<function fib at 10042ed0>
<< f = fib
<< f(100)
1 1 2 3 5 8 13 21 34 55 89

With that, the core language you need to "get things done" is well defined by the time chapter 5 arrives on page 38. This manages to beat even Core Java 2 in succinctness.

Chapter 5 is the standard boring text on "we support these data structures out of the box." The primary value in reading this chapter is to memorize the pythonic syntax and read the examples for general ideas on how python code should look (and these examples are great).

Chapter 6 covers the module system. This is the most exhaustive reference on any module system in any introductory text ever. Not only does it explain the syntax for the module system, it explains several advanced features such as packages that were previously fuzzy to me.

Chapter 7 and 8 define basic IO and Exceptions. Neither of these were particularly interesting, and while they cover it well enough there isn't much else to say.

Chapter 9 covers objects is where the book really shines. This is probably the hardest to read chapter of the book, as it's a bit closer to a standard textbook than an introductory tutorial as has been going on. I think it's a good idea for everyone to read all the text -- no matter how much OOP you think they know. Unless you use one of a small set of unpopular languages, you don't think of objects the same way Python does. If you see this book in a store, "skim" this chapter thoroughly, it's well worth reading.

Chapter 10 and 11 cover the standard library in 20 words or less. I didn't find it exceptionally valuable, except that it came right after chapters 6 and 9, cementing those ideas in my head. The material in this text is more of a introduction to modules in the Standard Library... to help you find more documentation about them.

The rest of the book is mostly appendixes. While short, this book managed to help me understand what the word "pythonic" means. The prose is very approachable, and the examples are well justified. Universally the code in this book is of high quality, and this book jump starts a developer to be a Python developer. Also, it's really cheap, and well worth the money.

Moving on, "The Python Language Reference Manual" fits this book like chapter 2. I'll be reviewing it sometime shortly.