Deploy Go with Capistrano

I’ve been using Capistrano for a while now, even for non-Rails projects, and I honestly think it’s too tightly coupled with Rails. But, anyway, I’ve always managed to deploy my projects, be it through really ugly hacks or simply sticking to the “Capistrano-way”.

In this post I’d like to cover how I successfully managed to deploy a Go/Gin web-service sticking to Capistrano as much as possible. This approach is quite general purpose but it all started trying to stick with the following 2 requirements:

  • Check out the code locally, rather than on the target machine
  • Build the code locally, for the target machine, and then only push the binary

As you may or may not know, Capistrano handles version-controlled code rather well but assumes you want to download your source on the target machine. In my particular case I didn’t really fancy the idea of setting up a Go environment on my target machine. I didn’t want that, mostly for laziness. But setting up a whole Golang environment on the target machine would also have implied more security vulnerability for my machine.

The code

Last time I checked, this feature wasn’t really documented on the Capistrano docs but turns out you can override the default Git Strategy that by default handles your code checkout on the target machine.
To be honest, the strategy itself doesn’t really enforce the checkout on the target machine, but Capistrano wraps every method of the strategy in an on block.
In order to bypass this behaviour I simply copied the Git strategy from the Capistrano sources and wrapped back each method in a run_locally block.

Copy the script in where you keep your deployment code. Be sure to load it. A good place can be your Capfile. After loading it, make sure to set the appropriate strategy in your recipe:

 

As you can see from the snippet, the repository will be cloned into repo_path, so be sure to specify a valid path. Now that we have instructed Capistrano on how to checkout the code on our local machine we have to instruct it on how to build and then push our Go project. The following snippet exposes a couple of Rake tasks to perform such things. This script assumes that you’re a deploying linking the log and pid directories. The pid is going to hold the pid file of your binary after executing it. This is gonna be useful across deployments to be able to automatically shut your process down and restart with the new version.

This rake task handles 3 useful options that you can specify using the set method provided by Capistrano:

  • go_build_env
  • go_run_env
  • go_run_args

go_build_env has to be used to specify the target machine operating system in order for Go to be able to cross-compile your code. In my case, for example, I want to deploy to a linux x64 architecture:

You can check for more environments here.

Use go_run_env to specify environment variables to be set upon starting your executable. Use go_run_args to provide your executable with the startup arguments it needs. As you can see, this rake file already hooks into the right Capistrano tasks to makes sure it builds and uploads after fetching the right repository revision.

Wrapping up

If you’ve correctly included the previous snippets, configuring your deployment should be straightforward. We have managed to deploy our Go code without having to setup another Go environment on the target machine. I think this is great since the cross-compilation produces a static executable that has 0 dependencies to be run on our target machine.

As I said, already, I think the guys over at Capistrano could do a little more to make the tool really general purpose and be more helpful for such integrations. But overall, this integration hasn’t really required too much code after all.

I hope you enjoyed this quick overview on how to deploy your Go code with Capistrano.

GitHug

Hey ya all,
I’ve spent the last weekend coding with Ruby again for the RailsRumble contest working on a crazy idea we came up a couple of days before the contest actually started.

It’s GitHug. We’ve always enjoyed GitHub but we’ve never been able to discover new interesting repositories through the site itself. So we created this simple service that applies a really basic (quite dumb) machine learning algorithm to try to learn from your activity on GitHub in order to suggest you repositories that may be of interest for you.

I’d really like you to give it a try, especially if you use GitHub on a daily basis.

I promise there’ll be a version of this service unrelated to the contest in order to deliver better performances and results. But this won’t happen any time this week 🙂

Cheers!

Gogoa – Cocoa bindings for Go

As usual, some time has passed since my last entry here.

Anyway, I just wanted to share my 2 cents about what has been a nice discovery for me in the last 2 hours.

I don’t actually know why but I was wondering how easy it is to invoke Cocoa from Go. Well [SPOILER] looks like it’s super easy!

The first comfortable piece I encountered, actually, was Go 1.3 changelog where it states:

Finally, the go command now supports packages that import Objective-C files (suffixed .m) through cgo.

 

Since I’m now with Go 1.4.2 I thought: “This should be even easier now!”

Reading through the cgo reference looks like interoperability is getting stronger with Go, despite this comment on SO.

Hahaha… Go’s biggest weakpoint… interoperability. –  Matt JoinerJun 12 ’11 at 13:54

Gimme the code

I know, too much words and no code so far.

Turns out that showing a Cocoa window with Go may be as easy as the following:

If you think this is too clean to be true you are kind of right. I actually started this project on GitHub which performs all the ugly bits underneath. I’d really love you to help me get this a little further.

How does it work?

There’s this go tool called cgo which does most of the magic by itself when you build a hybrid project like this.

The reference is rather comprehensive but I’d like to highlight a few issues I encountered while wrapping the code up.

1. Keep import “C” after the C statements

Let’s take my application.go as an example:

As you can see import "C" is preceded by three directives as per the cgo standard. Make sure the import "C" directive is right after the actual C directives otherwise your code won’t build. This is because C is a pseudo package generated also using the code you provide through those preceding directives.

2. Always specify the compiler directives

Every file exposing the ObjectiveC bindings should specify the following flags at least.

Otherwise you would end up in a long long list of compilation errors. At least this is what happened to me.

3. Always void*

If you end up working with pointers to ObjectiveC classes always convert them into void* before returning them back to Go. If you don’t, passing them back to C/ObjectiveC might become painful or you may end up with errors like the following:

struct size calculation error off=8 bytesize=0

At least this is what occurred to me. You may help me see the light with this.

4. Keep the non-Go code in separate files

Although cgo allows for the whole non-Go code to go in the comments, please, don’t do that. You’ll end up in unreadable code. Check out my repository for a possible solution.

Now what?

I don’t know actually. It really took me 1 hour to get this code compiling and running just for the sake of seeing a Cocoa Window. Didn’t expect much more.
Also, I’ve been doing some work with bindings in the past (Qt – Android – JNI, Cocoa – Qt – C++) and it always gets painful when the main framework is not run in the main thread. I don’t know if this will ever the case with Go. I’m not even sure how far this can be pushed especially when it’s about goroutines and defer‘ed stuff. But, despite not so pleasant past experiences I’d like experimenting this with Go and Cocoa as well.

A question for you

How would you write tests for those bindings?

Please, contribute

If you are any interested in this project, please, shout out loud. That would really help me experiment a bit more. Even if you just want to complain about something terribly wrong I did, please, do!

Cheers.

I’d rather Go complain about Rails

[SPOILER ALERT]

Actually this is not a complaint. I don’t root for any particular technology. It’s just I found Go outperforming Rails in this particular use case.

[END OF SPOILER]

I’ve had this post in the queue for a while but never managed to finish it. Part of the reason for this is due to what I’m going to tell you with the sequel of this post.

I’m writing this post to share my experience but also request for comments. I might really have misunderstood something in the process so I want to hear back from you.

I had this little notification service that mostly did this:

  1. accept an HTTP connection
  2. setup a thread listening on a channel over Redis
  3. setup a thread that pings the client to know for premature disconnects
  4. join the 2 threads with the current one and wait for a notification from the Redis channel

My Rails controller was pretty much like the following:

This controller was simply keeping the incoming connection open as long as new notification records were ready to be dispatched to the client. The begin, rescue, ensure dance was there to handle the case of premature client disconnects. Prematurely closing clients were keeping the controller in use until the timeout was reached. But I needed to save resources and be able to release them as soon as the client disconnected. I found it pretty hard in Rails to hook the client disconnection and it looks like it’s pretty much implementation-dependent and varies from server to server.

Anyway, this worked pretty nicely. As long as there were no more than ~4 connected clients. :/

More than ~4 connected clients made my nginx+passenger freeze. No way I could figure it out. I didn’t manage to fix that. Also, I wasn’t sure how many MySQL connection Rails was using. Connections to the database were being open and never closed until server restart.

So, given I had pretty nice results with similar services in the past with Go, I decided to spend the night and port the notifications controller to a tiny Go-based (Martini-helped) web service.

Here’s the result (actually a lighter version of it).

I feel this version of the service more consistent. Also, the language provides me with the means to discover when the client disconnects. And even the Redis library this time helps me understand when un-subscription occurs.

This, of course, will be compiled and turned into a native server that only handles the notification system. As a result, this outperforms the Rails-based version of the service. I found it pretty stable also. This service has then been proxied through Nginx and has never crashed since then (~5 months). I don’t serve much clients actually, ~30-40 simultaneous connections, but this amount wasn’t sustainable with Rails. Also, the MySQL connections were open and closed as expected and the number of simultaneous connections was reasonable.

I’d like your feedback regarding this experience. I know Go would always outperform Ruby in cases like this but I’m interested in understanding what was wrong with my Rails implementation. Also, can you confirm detecting client disconnect is pretty hard with Rails (4.2) ?

Cheers everybody.

Go Gocountries!


I’ve had this post in the queue for a while.

Ironically, I ended up writing a tiny little wrapper around an awesome service that has to do with countries information fetching. I say ‘ironically’, because I’m about to change the country I live, actually. But that’s another story.
Going back to the tiny wrapper, it started just as a way to keep up with the Go language, which I like a lot but have few chances to play with, actually. Anyway, the tiny wrapper is around an awesome service run by fayder, RestCountries. There’s plenty of wrappers already, among which you can find the Ruby gem, restcountry, by a dear friend of mine, Daviducolo.

Guess which? 😛

BTW, there’s a glitch in the README.md of this Go library. Can you spot it and fix it? I’d be happy to merge your pull request. 🙂

Cheers.

I’m moving [Part 1]

Hey,
As usual it’s been some time since my last post. I swear I’ve been through busy days. Anyway, I’m posting just to keep up with you and let you know I’m moving the blog (actually I’ve moved it) to this owned site. Please, join me here!

BTW1, this is just part 1 as I’m actually moving in my real life. But that’s another story and a specific blog post will be dedicated to that 🙂

Cheers!

A native iOS app with a Qt third-party library

I’m wrapping up here pieces of information that I used to setup a working native iOS project taking advantage of a library I wrote mainly using Qt. The library I wrote has nothing to do with GUI, it just helps me dealing with connectivity, REST API interaction, file system access, compression… Also, the library was born for the Desktop but the porting effort to mobile is feasible due to the lack of GUI interaction.
In particular, the following has been tested using Qt 5.3.

Despite the huge integration level now Qt has reached over the year even on the iOS platform, I still prefer having the UI developed using the native SDK. I’m a rather experienced Cocoa developer, not that experienced with Cocoa Touch, but I managed to have everything I needed working. Since the information I found is rather sparse regarding this topic, I thought it could be nice to have everything here in a blog post.

Project info

In the following post I’m assuming you want to link a native iOS Xcode project to a third party library written using C++ and Qt. I’m also assuming you are using statically linked Qt for this project (which is the default for the iOS solution).

So what we have:

  • A native Xcode project for iOS: NativeHelloWorld.xcodeproj
  • A static lib Qt project: FancyStaticLib.pro

What we are going to have

  • NativeHelloWorld.xcodeproj
  • FancyStaticLib.xcodeproj as a subproject to NativeHelloWorld
  • Qt for iOS properly linked to make NativeHelloWorld run and use FancyStaticLib

Let’s get things started

So, first of all let’s instruct Xcode about where to find Qt on our system.
Go to Xcode -> Preferences -> Locations (Tab) -> Source Trees (Sub Tab).
Add the path to your Qt for iOS packages and name it as you wish. I just chose QTLIB_IOS.
Screen Shot 2014-12-18 at 18.25.05

I like setting my paths this way in order to keep my project as “exportable” as possible. This way, other devs can join my project pretty easily.

Now, if you haven’t already, you should create an Xcode project for your static library you want to link into your native iOS project.
In order to do so you have to run something like this:

/path/to/your/Qt/ios/bin/qmake -spec macx-xcode -r /path/to/FancyStaticLib.pro CONFIG+=staticlib CONFIG+=whatever_you_need

This will output FancyStaticLib.xcodeproj file for your static library. You can drag it to your our NativeHelloWorld.xcodeproj inside Xcode and add its product (the static lib) as a link dependency to your project.
NOTE: You will have to re-generate the FancyStaticLib.xcodeproj each time you change your static library .pro file

Link to Qt

Now that we have the project feeling more like a completely native Xcode one we have to set a few things up in order to easily keep developing directly from Xcode through our NativeHelloWorld.xcodeproj

First of all, look for the Headers Search path section in the Build Settings section of your Xcode project:
Screen Shot 2014-12-18 at 18.54.07

We want to make it easy for Xcode to find Qt headers, and also our static lib headers.
Now the variable we previously defined through the Source Trees section in the Xcode preferences comes in handy.
Let’s add the following to the Headers Search Path section:

  • $(QTLIB_IOS)/include
  • /path/to/FancyStaticLib/headers

Now, the actual linker flags.
You will probably need to start your project inside the emulator. When doing so bare in mind that the simulator has a different architecture from your iOS device. Simulator runs an i386 architecture and we want to link our project both to the static Qt lib files compiled for such architecture and for arm. This way we will be able to run our project both in the simulator and on the native device.

Scroll down to the Other Linker Flags section: you should at least have a Debug section. Under Debug, as a child item, you should have Any iOS Simulator SDK. If you don’t, click the little “+” icon on the side of the Debug item and add Any iOS Simulator SDK as child item.Screen Shot 2014-12-18 at 19.06.30

Our project dependencies are satisfied by the following modules:

  • QtCore
  • QtGui
  • QtNetwork
  • QtScript

The Debug section will host the option for running our app on a native device with debug symbols:

-L$(QTLIB_IOS)/lib -lQt5Core_debug -lQt5Gui_debug -lQt5Network_debug -lQt5Script_debug
Also, don’t forget to include proper platform support with:
-lQt5PlatformSupport_debug -L$(QTLIB_IOS)/plugins/ -lqios_debug
You’ll also need -lz -lqtharfbuzzng_debug most probably.
Also, if you are taking advantage of the bearer plugin to handle connectivity, add the following:
-L$(QTLIB_IOS)/plugins/bearer -lqgenericbearer_debug

Now the Any iOS Simulator SDK section:
Simply replace what you typed in the previous section changing “_debug” with “_iphonesimulator_debug” and you are good to go.

The last touch

Your Qt lib will most probably need an instance of QGuiApplication. This usually requires you to replace the default main coming with your project template with a custom one that actually calls QGuiApplication::exec(). Luckily, Qt has made things relatively easy and you won’t need a custom main body. Looks like the Qt guys are cool enough to inject their Qt Event Dispatcher inside the main CocoaTouch run loop making it easy to spawn QTimers and queued methods invocations from Objective-C(++).
Just make sure you initialize a QGuiApplication instance (but you won’t need to call .exec()).

We are going to add the following piece of code inside your application delegate - (BOOL)application:(UIApplication *)application willFinishLaunchingWithOptions:(NSDictionary *)launchOptions after having renamed the application delegate file from .m to .mm. Renaming to .mm enables Objective-C++ which helps us mix C++ and Objective-C in the same source.

Conclusions

This is pretty much what is needed to mix a native iOS project with a Qt library. If you encounter linker issues you should be able to easily address them by inspecting the .a symbols you can find inside your $(QTLIB_IOS)/{lib,plugins} directories.
Otherwise please post here your issues so that we can try address them together.

Cheers.

Workaround Windows Tray Area Item Preference

Introduction

I’m not an experienced Windows developer.
I had to make it clear 🙂

Today I had the chance to implement kind of a nasty hack on Windows.
I had to make my application tray icon always visibile, at least by default. I swear I honor user preference then. I know this is one of those don’tswhen working with Windows API since it is clearly stated in the docs developers have no control over the notification area. But sometimes you feel it deep in your heart that your application user experience would benefit a lot from making your tiny tray icon visible by default. This was my case, and as I can see on the internet, this is the case of a lot of apps out there.

I just wanted to write this down as an excercise to help me get back on sharing what I code (not always feasible, though).

There’s plenty of information out there, it’s just you won’t find a single pice of it and you’ll have to digg a lot before being able to workaround this limitation over the tray area. At least this was my experience as a non-experienced Windows developer.

Now that my reasons have been stated clear we can go ahead and see some code.

Let’s get started

So, there’s this incredible resource by Geoff Chappell which you should check if you want to know more about some undocumented/private APIs on Windows. It looks he has done a huge amount of work around the notification area documenting well enough how to workaround the default limitations exposed by the documented API.

As he states here explorer.exe exposes an ITrayNotify implementation through COM. This interface can be used to know user preferences and status regarding the notification area items. And you of course can also use it to modify such preferences.

So what we are going to do now is requesting the implementation through the canonical CoCreateInstance passing in the required CLSID information. Such information is retrievable from the docs by Geoff. But you can also look for ITrayNotify through regedit.exe in order to find the needed CLSID.

Bringing a few pieces together, here is a quick recap of the declarations you’ll need to make the CoCreateInstance call succeed.

This is enough for requesting the instance through CoCreateInstance. Unfortunately, as I discovered testing my own code, this won’t work on Windows 8 where apparently this private API has changed. You know, this is the drawback of using private APIs :).
Anyway, I spent the day looking for the solution and fortunately I found the appropriate interface also for Windows 8. You can find the same information by running OllyDbg against explorer.exe on Windows 8.

Getting the instance to the appropriate ITrayNotify interface, though, is not enough. We are going to use another private interface, called INotificationCB, which will help us get the current information regarding our notification item.

So let’s write down our little helper class that will take care of modifying the preferences for our notification item.

Now let’s see the actual implementation for our helper.

I see what you did there

So, TinyTrayHelper basically does 4 things here:

  1. Creates a NOTIFYITEM instance based on a NOTIFICATIONDATA instance
  2. Chooses the appropriate ITrayNotify instance based on the current OS
  3. Registers itself as an instace of INotificationCB to receive the relevant information inside the Notify method
  4. Finally calls SetPreference to change the preference regarding the notification area item

What now?

What you need now is just to create an instance of our TinyTrayHelper and pass in your NOTIFICATIONDATA instance reference. Then call ensureTrayIconVisible to change the notification area preference regarding your item.

Please note that I adapted a more complex code to build this example so I didn’t test this code specifically. Use at your own risk.

I hope this will be useful to you. Please, let me know if I made tremendous mistakes here, I’ll try to fix!

Cheers.

I used to blog

I’m a bit shocked about how much I used to blog when I was a KDE contributor.
I lost myself in my previous blog this morning, it’s damn full of words. I was rather confident about what I was doing, that’s pretty cool. Anyway lots of events in my life brought me to stop contributing to KDE (I’m really sad about that, I apologize) and to stop blogging also.
Not that I haven’t had anything to blog about, but I’ve never had time (or will) to start a new blog from scratch. I didn’t feel like keeping the old blog. I feel it somewhat complete and it represents a specific part of my life I’m pretty happy about. So I just want to keep it as it is now.

Now, enough for the nostalgia. I’m still not really sure what I’m going to blog about here. It’s been busy years and I learnt a lot about software engineering. I love software engineering and I hope to share my experiences and have feedback from you.

OK, this is enough as first post (really?), so let’s stick with it.