The droid-VNC-server projects consists in two parts: the daemon and GUI.

GUI handles general user input, and connects to the daemon using local IPC. You can compile it using Eclipse or Ant (the latter not tested though).

If you wish to contribute to the project, just fork it and commit some interesting features...

-------------- Compile C daemon -------------------
NOTE: to compile the C daemon you must have a working AOSP source tree, and copy it to external folder.
Do something like:

$AOSP_SRC=/path/to/aosp
$VNC_DAEMON=/path_to_vnc/droidvncdaemon

cp -fr $VNC_DAEMON/droidvncdaemon $AOSP_SRC
cd $AOSP_SRC
. build/envsetup.sh
# lunch
mmm external/droidvncdaemon

Thats it. You will have androidvncserver_froyo (for 2.2 down) and androidvncserver_ginger (for 2.3+) in $AOSP_SRC/out/target/product/generic/system/bin/

I think this process is quite difficult, anyone knows how to build external packages outside AOSP tree?