Fixing gluUnProject in Android Froyo

Lately, I’ve been working on a new Android app which will make use of 3D graphics. The Android SDK provides its own implementation of OpenGL ES which differs between Android releases. Since my current phone runs Android 2.2, I’m stuck with that release for the time being, and it looks like the version of the gluUnProject method that comes with it is thoroughly broken: while writing some basic code to convert screen coordinates of a touch event back into 3D coordinates, I noticed that the values were generally pointing in the right direction, but oddly scaled and thus not usable for this purpose.

I ended up writing my own unProject method which uses the Android SDK and is based on the MESA C implementation:

private boolean unProject(float winx, float winy, float winz,
                          float[] modelMatrix, int moffset,
                          float[] projMatrix, int poffset,
                          int[] viewport, int voffset,
                          float[] obj, int ooffset) {
  float[] finalMatrix = new float[16];
  float[] in = new float[4];
  float[] out = new float[4];
	 
  Matrix.multiplyMM(finalMatrix, 0, projMatrix, poffset, 
    modelMatrix, moffset);
  if (!Matrix.invertM(finalMatrix, 0, finalMatrix, 0))
    return false;
	    	 
  in[0] = winx;
  in[1] = winy;
  in[2] = winz;
  in[3] = 1.0f;
	 
  // Map x and y from window coordinates
  in[0] = (in[0] - viewport[voffset]) / viewport[voffset + 2];
  in[1] = (in[1] - viewport[voffset + 1]) / viewport[voffset + 3];
	 
  // Map to range -1 to 1
  in[0] = in[0] * 2 - 1;
  in[1] = in[1] * 2 - 1;
  in[2] = in[2] * 2 - 1;
	 
  Matrix.multiplyMV(out, 0, finalMatrix, 0, in, 0);
  if (out[3] == 0.0f) 
    return false;
	    
  out[0] /= out[3];
  out[1] /= out[3];
  out[2] /= out[3];
  obj[ooffset] = out[0];
  obj[ooffset + 1] = out[1];
  obj[ooffset + 2] = out[2];

  return true;
}

In order to get the 3D coordinates of a point x/y on the screen, you only have to call the new method once for each intersection with the near and far clipping planes:

// near clipping plane intersection
float[] objNear = new float[4];
unProject(screenX, screenY, 0.1f, 
	viewMatrix, 0, projectionMatrix, 0, view, 0, objNear, 0);

// far clipping plane intersection
float[] objFar = new float[4];
unProject(screenX, screenY, 1.0f, 
	viewMatrix, 0, projectionMatrix, 0, view, 0, objFar, 0);

You can then calculate the difference vector (objFar – objNear) and check what location in 3D space corresponds to the screen event.