TAG http

A Tour of Go

Last week we had to write what a small specialized HTTP client that connects to a server and issues GET requests to fetch data. Simple enough.

The only issue is that the data provider required us to use maximum one connection at any time, using HTTP pipelining to issue all the GET requests in the same connection and then block until they respond with new data. In practice this is not very complex, but PHP (our go-to hammer) is not really shining in this area.

All the HTTP abstractions around curl/http streams like Buzz or Guzzle do not seem to support pipelining (if I missed that, let me know), and doing it by hand is not particularly fun. The http extension does support that through some undocumented feature though, but I didn’t have that extension at hand, so I skipped that. In any case, PHP is not really the best tool for doing this sort of long running script, although it has gotten much better in recent years. So we looked for something else, and long story short, we decided to try Go. It is a new-ish language that is aiming to replace C/C++/Java for systems programming, mixing and matching features from everywhere while focusing on language terseness.

So what about Go?

Well it is interesting. I just opened an editor and within a few hours I had a first prototype running, a few more hours of polishing and tweaking later I had a finished product that has now been running for a week. It handled a few tens of thousands of entries, and especially did not crash once ;) The standard library is quite complete for what I wanted to do here, and there is already quite a large ecosystem of extra libs available for more specific tasks.

The documentation. It is not awful but it is a young language, and it could benefit from having more examples and cross-references. On the other hand I started coding without reading any of the basics, so I had to learn a few things the hard way. One big plus though is that the documentation links to the sources, so you can very easily check out what something does behind the scenes.

Static typing is a big change coming from PHP which abstracts away a lot of this. However thanks to their light approach to typing, you don’t feel it too much. Basically using := for assignment you can define a variable and have its type automatically inferred by the value you assign it. That gives you typing without the verbosity. Then of course you get compile-time errors based on all this which is great for debugging. Overall this saved me from doing a bunch of mistakes, and did not cause so much pain.

The recover function is also quite a nice fit for the use case at hand. It allows you to recover from any fatal error, and then keep running some code. Using this I made sure the program never dies, because if anything bad happens it will error out and restart doing its job instantly. It reacts much faster than monit or similar watchdog programs would typically do since it is triggered by the application failure.

Finally I want to quickly touch on goroutines. It allows you to do parallel processing very very easily. All you have to do is prefix a function call with the go keyword, and it turns into a background task of sorts. You can then use channels to communicate and block between multiple goroutines. It is a great model and should allow people to write parallelized programs without the pain usually associated.

To sum up, Go is quite fun. I would not use it to build a large web app though. I don’t think the tools are there yet and we have a great ecosystem in PHP for that. But for simple utils I would definitely recommend it to anyone that feels like trying something new.

June 22, 2012 by Jordi Boggiano in Development // Tags: , , 3 Comments