Projects

I have a collection of some of my interesting personal projects. Starting with the most recent.

Whiteboard Capture Projects

The overall goal was to make it easy to capture a whiteboard drawings. I feel like drawing something digitally hampers the flow of ideas, you either use a tool that slows you down, or you easily get distracted and start polishing what's on the screen. For most people, using pen and paper or working with a whiteboard is the easiest way to brainstorm and be creative.

I had these two projects close to production quality before my children were born and I stopped working on them. Recently, Microsoft released Office Lens to solve this problem.

Cleanup

The first tool cleans up a whiteboard image. The original version followed a research paper from Microsoft, but I have since rewritten every step using different techniques. The new tool is equally fast and can better deal with glare. This tool can output a bitmap, or vectorize the outline of each mark on the whiteboard.

Recognition

The second tool extracts the original marker strokes from the whiteboard image. This is best illustrated with an example:

It is relatively straightforward to get the outline of a mark on the whiteboard. For the letter "I" in the Jurassic Park logo, that would be finding the outline of the white rectangle.

When someone draws an "I", though, they actually draw a vertical line, not a rectangle. Hover or touch the Jurassic Park logo - the red lines are the marker strokes. This problem gets tricky fast - "I" is easy, but the "K" has a lot of intersections, and the outlines of the different letters touch. Finding these marker strokes is what this tool does.

After estimating the original marker coordinates, the points have a lot in common with touch gestures. Tools written to work for touch devices can now work with pictures! The proof of concept works, though it needs some more polishing. I successfully connected the marker coordinates output to an old project I worked on, GrafPad, which converts touch coordinates to shapes. Using this, I converted pictures of a whiteboard with a block diagram into a PowerPoint with the same block diagram.

GrafPad

GrafPad is a drawing tool designed for touch devices. The idea is to act like a digital pen and paper. The input is free form - either touch or mouse inputs, and the tool recognizes objects and automatically cleans them up. So lines that are near horizontal or vertical are made perfectly horizontal or vertical, and polygons are recognized and normalized wherever possible (opposite sides set to equal size, angles converted to 90 degrees, etc).

I wrote the backend, including designing and building all the cleanup algorithms. You can see it in action below.

RapydScript I

This project is actually the brainchild of Alex Tsepkov, but I helped build the original code and define the syntax so I thought it worth mentioning.

I call RapydScript a Python to JavaScript transpiler, as opposed to a compiler, since the language behaves like JavaScript. RapydScript has a Pythonic syntax but is JavaScript under the hood - similar to how CoffeeScript brings a Ruby-like syntax to JavaScript. It also has some syntatic sugar to make some tricker aspects easier, like inheritance.

I helped design a lot of the syntax where things weren't so clear, and created the first implementation of several key features like tuple packing/unpacking and exception handling. I also built one of the largest initial proof of concepts, Cyborg Challenge, which is a RapydScript clone of Chips Challenge that you can play online.