Dev Thoughts

Just a quick post announcing that I’ve finally managed to launch my new web application: Applicatr. While the name might sound a bit unusual, and the look isn’t all there yet, it’s pretty functional already (we’ve been using it to find the right graphical designer amongst other people).

For those of you that haven’t clicked the link to check it out, the gist is that its a web application that helps you and your team (or company) collaborate more efficiently during the hiring process so that you end up hiring the right person for the job and for the team.

So if you’re someone that will be involved in hiring anytime soon, give it a go, and feel free to send any feedback.

For the past few months I’ve been working on a startup, something very useful that fills in a need that businesses have.

Right now I’m deep into adding the finishing touches on it so that it can be released for consumption. This does not mean that it’s finished by any stretch of the imagination – I have a todo list that’s as long as a piece of string – but it does mean that we’ve crossed enough t’s and dotted a sufficient number of i’s so that other people can use the website.

If all goes well the site proper should be up and live in a few days at the start of February, and I’ll do another post then.

Stay tuned!

I have been using the Google search engine for many years now, and have found it to produce good search results, that is until recently. What’s been annoying me now is that I do frequent searches to find information related to fields that change frequently, such as Ruby and Rails, and the results that I get are the oldest ones which are no longer valid or are innacurate.

Generally when I search for a particular problem I am having, the first results that come up have been from a blog that is 2-3 years old, and about a dozen versions out of date. Though it’s not only Google that has this problem, but a lot of the other major search engines. It’s that they give more precedence to how old a particular site is rather than if it’s relevant anymore. This also disproves the adage that there’s no more innovation to be had in search.

For now though what would be really useful would be the ability to sort the search results by date. From newest to oldest, oldest to newest, so that the information for the newest versions would come out on top rather than be buried in the search results. I’d love to see this as an advanced search option, or even one of those Google search box commands.

Recently I’ve developed a site using the simple StaticMatic website generator. StaticMatic is a nice and simple static website generator that uses HAML and SASS to describe the site content while also providing an easy way of using templates much like in Rails. However one thing that it does not do is provide an easy way of doing deployment to a server.

Looking over the deployment options I decided to use Capistrano, mainly as a learning experience before trying to use it to deploy proper Ruby on Rails applications. Though I found out that it’s not that difficult to set up. So here goes my story:

Installation

To deploy StaticMatic via Capistrano you need to use the railsless-deploy package which cuts out the Rails specific cruft from Capistrano.

You might need to add http://gems.github.com (I know they don’t do Ruby gems anymore, but it still works for me) and http://gemcutter.org to your gem sources list. To do this use:

gem sources -a <SOURCE>

Then installing the required packages is simple:

gem install capistrano
gem install railsless-deploy

Setup

First you’ll need to Capify the site, so in the StaticMatic site root directory execute:

capify .

Which will create the Capfile and config/deploy.rb files. These now need to be edited to work with railsless-deploy and StaticMatic.

So the Capfile with the appropriate modifications should look something like:

load 'deploy' if respond_to?(:namespace) # cap2 differentiator

require 'rubygems'
require 'railsless-deploy' # Removes railsisms from Capistrano
load 'config/deploy'

Then the config/deploy.rb needs to be worked on. This is where you set up the configuration for how to deploy the site. There are many settings that need to be set, such as the application, the repository, and the roles available (I just set the roles to the same thing). In addition I use git for source control of this site, so some of my settings are:

set :deploy_to, "/var/www/#{application}"
set :deploy_via, :export
set :scm, :git
set :shared_children, []

Using the export deployment method means that there is no .git directory hanging around on the server for people to muck around with. Setting the shared_children variable to an empty array means that no shared directories are present on the site. While this isn’t essential, I use it so that no superfluous directories are created.

In addition to these settings, a couple of small task definitions need to be included in the config/deploy.rb file, namely:

namespace :deploy do
  task :update do
    transaction do
      update_code
      build_code
      symlink
    end
  end

  task :build_code, :except => { :no_release => true } do
    run "staticmatic build #{latest_release}"
  end
end

These tasks are necessary to build the site on the server, with the first task redefining the deployment task so that the code is built at the appropriate stage, and the second task doing the actual building of the site.

Deploying

Once the site has been configured appropriately a few Capistrano commands need to be executed to set up the environment on the server. Firstly the directory structure needs to be set up, so run:

cap deploy:setup

Then we check that everything is configured appropriately by running:

cap deploy:check

And finally we can deploy the first version of the site using:

cap deploy

Assumptions

Now of course this post isn’t a complete how-to of deploying a StaticMatic site onto a server. Things that I have omitted are setting up a deployment user on the remote site, generating a SSH key, configuring Capistrano with that user information, etc. However there are various other guides out there with this information so it shouldn’t be too hard to find.

I’ve added a page that contains a list of books that I’ve found interesting from both a personal and professional standpoint. More importantly there are links to both Amazon.com (USA) and Amazon.co.uk (UK) in case anyone wants to buy them, of course I get a tiny cut if people do this.

I’ve been doing some Ruby on Rails development recently and one thing that’s bugged me a bit is how to deal with test data while you’re developing the application. Not data that is used just for testing, but data that lives in the development database that you use and fiddle around with while developing the site itself. I’ve found that while developing features of the site I need a fairly large and varied set of data to use, and this data takes a while to enter into the site each time I modify the database.

So instead of repeatedly entering this data I wrote some simple Rake tasks that can automate exporting and importing the data from the database. I can’t take all the credit as the idea was inspired/based on/stolen from the Advanced Rails Recipes book.

Exporting works by writing each table to its own CSV file in the db/dataset directory (or subdirectory) in the rails project. The first line of the file is the name of the columns in the database.

  desc "Exports the database contents into CSV files."
  task :export => :environment do
    require "FasterCSV"
    require "FileUtils"

    path = File.join(DATASET_PATH, (ENV["DATASET"] || "."))

    FileUtils.mkdir_p(path)

    ActiveRecord::Base.establish_connection
    database = ActiveRecord::Base.connection

    (database.tables - SKIP_TABLES).each do |table_name|
      FasterCSV.open(File.join(path, "#{table_name}.csv"), "w") do |csv|
        # Write column names
        csv << database.columns(table_name).map(&:name)
        # Write rows
        database.select_rows("SELECT * FROM %s" % table_name).each { |row| csv << row }
      end
    end
  end

Importing works much the same way by reading all of the CSV files in the db/dataset directory (or subdirectory) and attempting to load the data into the database. However it does check that the table of that name exists first and skips it otherwise.

  desc "Imports data from CSV files into the database."
  task :import => :environment do
    require "FasterCSV"

    path = File.join(DATASET_PATH, (ENV["DATASET"] || "."))

    ActiveRecord::Base.establish_connection
    database = ActiveRecord::Base.connection
    tables = (database.tables - SKIP_TABLES)

    database.disable_referential_integrity do
      Dir.chdir(path) do
        Dir.glob("*.csv") do |file_name|
          table_name = file_name[/(.+)\.csv/, 1]
          next unless tables.include?(table_name)

          rows = FasterCSV.read(file_name)

          # Construct a list of column headings from the first row
          heading_row = rows.shift.map { |heading| database.quote_column_name(heading) }.join(",")

          rows.each do |row|
            # Need to escape the data correctly so that inserting is successful
            row_data = row.map { |item| database.quote(item) }.join(",")
            database.execute("INSERT INTO %s(%s) VALUES (%s);" % [table_name, heading_row, row_data])
          end
        end
      end
    end
  end

These are faily simple tasks and therefore I’m releasing the code to the public domain, grab the file and use it freely.

To make it work just add it to your lib/tasks directory in your Rails project. The tasks can then be invoked with:

rake dataset:export
rake dataset:import

You can also specify a specific subdirectory by passing the DATASET option to rake, like:

rake dataset:export DATASET="dev"

Of course I know there are many things missing from this script, such as being able to specify the rails environment to use, but I’ve kept it simple so adding these features is left as an exercise to the reader.

I’ve been programming in C++/CLI at work recently, and I thought I’d share a snippet/trick or two that I’ve learnt.

When working with the .Net framework in C++ you can come across places where it’s easier to use the Win32 functions to do a particular task rather than the .Net ones (or there’s no direct .Net version of it available). In those cases you may need to get Win32 handles from some .Net classes, or extract primitive types from a handle to an Object instance.

To get the window handle (HWND) of a .Net Form:

HWND form_handle = reinterpret_cast<HWND>(form.Handle.ToPointer());

This also works for other handle types used in Windows, such as HDC.

Effectively this is just unboxing the variable from the .Net type. However the Handle property returns a type of IntPtr and therefore it needs the ToPointer() call to work properly.

Another trick to proper unboxing is when trying to convert from an Object^ to a primitive type, say UINT, or LONG. I discovered that using a safe_cast directly to the primitive type did not work, instead it just raised an exception. So to correctly unbox primitive variables you need to unbox them to a primitive, non-typedef’d type, something like:

int a = 10;
Object^ a_obj = gcnew Object(a); // Automatic boxing
UINT new_a = safe_cast<int>(a_obj); // Unboxes

Also don’t forget to wrap this unboxing code in a try – catch block as it might still throw exceptions.

It’s funny how things have come full circle in the Windows game development world – when it comes to processing player input of course. It used to be back in the day (I think around 2000 or so) that a game engine would use DirectInput to manage user input, but that has now been deprecated in favour of using plain old native Win32 messages.

(Though you still need to use DirectInput for Joysticks that require it, and XInput for newer devices – such as XBox360 gamepads. There’s an article on how to use both side by side.)

I came to this realisation when working on my own engine recently, as I was planning on coding up a nice primarily DirectInput based system, and only using the native windows functions to handle the mouse input when windowed. However after reading an article or two and looking at some of my old projects I decided against it and just used simple windows message handling to get the job done.

Some of the reasons to no longer used DirectInput are:

• That it isn’t really all that direct at all, it just spawns another thread that listens for the input related windows messages and processes them for you.
• It also doesn’t play nice with the user’s input related settings in the control panel, such as the key repeat rate, and mouse speed and acceleration settings.
• And it doesn’t handle international keyboards as well as the alternative.

Additionally it turns out that not only is using the native windows messages easier to code up, but the code is also shorter and it makes the other architecture cleaner and more elegant to boot.

Technical Details

To use the windows messages as input all you need to do is to process all of the relevant messages:

• WM_KEYDOWN, WM_KEYUP – For the regular keystrokes
• WM_SYSKEYDOWN, WM_SYSKEYUP – For the ALT key, can be processed by the same functions as the regular keystrokes.
• WM_MOUSEMOVE – For the mouse movement
• WM_MOUSEWHEEL – For the mouse wheel (Only on Windows XP and above)
• WM_*BUTTONDOWN, WM_*BUTTONUP – (Where * is L, R, M, X) For mouse buttons
• WM_CAPTURECHANGED – Needed to properly handle some aspects of the mouse in windowed mode.

(There is also another way of getting far more precise mouse values using the more generic input messages, but I did not need this for the project so I did not use it. However developers of first person shooters or other twitch games should investigate this method.)

Then pack the data into your own custom structure so that you don’t have to have the windows message handling values all through your code, and then dispatch the data structure to your game state controller for handling.

Implementation Notes

Now there are a few things to watch out for when creating this system, mainly dealing with the processing of mouse based input, and especially when operating in windowed mode.

Mouse Position

You should cache the mouse position as it is received from the WM_MOUSEMOVE message and only dispatch the new mouse position once per frame in an update function or similar. The reason for this is that the mouse update rate is far higher than the frame rate and movements in the mouse can generate many such messages which flood the system. This is especially bad when expensive computations are performed on the movement of the mouse (such as camera orientation/position) as they will suck up all of the CPU time doing the processing.

When switching the mouse from reporting absolute to relative coordinates there are several things that need to happen. The mouse needs to be captured, the cursor hidden, the current coordinates stored for later, and then set to the centre coordinates of the window. I found that sometimes the mouse cursor show count is greater than one, so I repeatedly call the ShowCursor(FALSE) function to make sure it is hidden. Otherwise the user can still see the mouse as it is wildly moving about the centre of the window. Then at the end when switching back into absolute coordinates don’t forget to restore the previous position of the mouse.

Mouse Wheel

For every mouse that I have encountered the value reported is a multiple of 120 units, with 120 (or WHEEL_DELTA in the code) units representing a single “click” or movement of the mouse wheel. As the Microsoft documentation explains this is so that in future we might have mice that scroll at a finer granularity. Therefore I made sure in the processing that I first accumulated the incoming value and then only reported the movement if it moved more than 120 units.

Additionally I discovered that (at least) in debug mode when I scrolled the mouse wheel I would sometimes get multiples of 120 showing up as the incoming delta. Where as I had not seen this behaviour before at work or otherwise. Therefore it is possible to return more than one “click” or significant movement of the scroll wheel in a single message.

Mouse Buttons

When the user presses down on a mouse button within your client area the mouse should be captured by your application. This is so that you will get a mouse button up message when the button is released while the mouse is no longer above your client window. (That’s also why the WM_CAPTURECHANGED message is listed above.)

Keyboard

One tip for dealing with the keyboard is to only report the first instance of a key down message. You can detect this by checking the status flags on the incoming message (Bit 30 of the wparam parameters I believe, but check the documentation). This blocks the key repeat feature of the messages from going to the rest of your application so you’ll only get one key down event. Hence enable it for the general game but disable it when the user is typing into a text input field, as you’ll want to allow the default system key repeat settings to take effect.

That’s it for now. Hopefully this has helped someone that was in a similar situation to me before.

This is an update on the game development project I am working on as a hobby. Initially the goal was to make an engine and a small game an enter it into the TSumea AI programming challenge. However I wasn’t able to complete the entry in time as I really bit off far more than I could chew.

It came down to the fact that developing an engine, a game, and an interesting AI simulation is far more for one person to do in three weeks of hard coding. There are some questions to consider as to why I didn’t have an entry ready.

• Maybe I could have used an existing open source engine – but none of them gelled with me nicely.
• Maybe I could have started developing the game and engine back in September (when the challenge started) – but there were work deadlines to make.

Regardless, the final result is that I didn’t get the engine or game completed. Not only that, but from all the people that suggested interest in the challenge there was only one final entry. How disappointing.

But still I have developed something that looks like an engine.

In three weeks “working” about 12 to 16 hours a day of programming/research I have managed to create a nice engine, with (what I hope is) a good architecture underneath it. I even spent a weekend working out mathematical and physics formulae (for collision detection and response) on my Tablet PC, and I can honestly say that I haven’t done that much mathematics since University.

(I have to note that it’s quite nice to scribble out mathematics on a tablet. As you can use normal handwriting with the correct mathematical symbols and all on a grid ruled background. Also you can move stuff around and erase with ease if you make a mistake.)

For the time spent on the project I have had a blast. It reminded me of the good old Uni days staying up till the early hours in the morning coding, and yes I did that this time as well. I learnt to use some interesting bits of the Boost library, experimented with better ways of writing game architecture code, and re-learnt a whole bunch of mathematics and phyics.

However the engine and game is not complete, and since I stopped working on it on the 15th I haven’t done much to it since. (I’ve been playing games instead!) But I’ll start working on it again now and see if I can create a decent game out of it. Also I have a couple of articles that are in various stages or preparation and will post them when they are done.

Have a happy new year too!

Now that I’ve actually been using Visual Studio for a lot of development work at home I’ve encountered some annoyances. One of these has been that sometimes when switching between windows using the Ctrl+Tab/Ctrl+Shift+Tab shortcuts it gets stuck and refuses to take any more keyboard input until I click on it with the mouse.

A quick glance at the problem reveals that it’s a focus issue with the window that pops up listing the open documents, where if you switch quickly enough it doesn’t get time to properly return focus to the main window.

Luckily there’s a handy solution out there that fixes the problem, you simply swap what the Ctrl+Tab and Ctrl+F6 keyboard shortcuts are bound to (but don’t forget the Shift variants).