Someone Wrong on the Internet

I spent the weekend writing a very simple application that uses OpenGL ES on the iPhone to display a square with a picture (texture) on it. You can then use the traditional iPhone one-finger drag to move the picture around, and pinch in and out to zoom in and out on it.

Simple in concept, anyway, but execution, especially for someone new to iPhone development and very rusty at OpenGL development, was very difficult. I may have spent the better part of Saturday tracking down a bug due to a one letter typo in my code.

The major difficulty is that since I was developing the application strictly in OpenGL (as a base for a game I am working on), I couldn't take advantage of the iPhone's built-in libraries to handle anything. I had to develop the code for dragging (easy) and pinch zoom (less easy).

The way pinch zoom is implemented on the iPhone, if your pinch is on the right side of the picture, you zoom in on the right side of the picture. If your pinch is at the top side of the picture, you zoom in on the top of the picture. You can try this on the iPhone yourself, or make the jump to see screenshots and the code I wrote to implement this feature.

The above animation shows what happens when you pinch and zoom on an image. The white circles indicate where the users fingers are, and the red crosshair was added post-production in order to help you see where the midpoint of the pinch is. Note that the part of the image in the midpoint of the pinch, at the center of the crosshairs, does not move. The image zooms in (and out) around that point.

I Googled around a bit trying to find the algorithm that Apple uses for pinch and zoom, but I found nothing. I didn't search too hard, though, because I kind of wanted to come up with the code myself. Here is that code:

- (void)touchesBegan:(NSSet*)touches withEvent:(UIEvent*)event
{
	NSSet *allTouches = [event touchesForView:self];
	NSUInteger touchCount = [allTouches count];
	
	NSLog(@"touchesBegan count: %d", touchCount);
	
	// drag
	if(touchCount == 1)
	{
		UITouch *touch = [allTouches anyObject];
		
		if(touch == nil)
			return;
		
		// store where the user first touched
		_touchStart = [touch locationInView:self];
	}
	// zoom
	else if(touchCount == 2)
	{
		NSArray *touchArray = [allTouches allObjects];
		CGPoint first = [[touchArray objectAtIndex:0] locationInView:self];
		CGPoint second = [[touchArray objectAtIndex:1] locationInView:self];
		
		// store midpoint between touches in _touchStart
		_touchStart.x = (first.x + second.x) / 2;
		_touchStart.y = (first.y + second.y) / 2;
		// and store the distance between the touches
		float dx = first.x - second.x;
		float dy = first.y - second.y;		
		_lastTouchDistance = sqrt(dx*dx + dy*dy);
	}
	
	_lastTouchCount = touchCount;
}

The touchesBegan function is called whenever the user puts a finger down on the screen. When this happens, we check to see if the user has two fingers on the screen. If he does, we assume he is beginning a pinch. We store how far apart the touches are in _lastTouchDistance, and the midpoint of the pinch in _touchStart. _touchStart will be the point around which all of our zooming in and out takes place.

You can also see the code that handles the one finger case of dragging the image around. I will not explain that, because it's simple.

- (void)touchesMoved:(NSSet*)touches withEvent:(UIEvent*)event
{
	NSSet *allTouches = [event touchesForView:self];
	NSUInteger touchCount = [allTouches count];
	
	NSLog(@"touchesMoved count: %d _lastTouchCount: %d", touchCount, _lastTouchCount);
	
	if(touchCount != _lastTouchCount)
	{
		//we should probably reinitialize state instead of just bailing
		return;
	}
	
	// drag
	if(touchCount == 1)
	{
		UITouch *touch = [allTouches anyObject];
		
		if(touch==nil)
			return;
		
		CGPoint touchEnd = [touch locationInView:self];
		CGFloat dx, dy;
		// convert how far the user moved their finger from display coordinates into world coordinates
		dx = (_touchStart.x - touchEnd.x) / (320 / _viewWidth); 
		dy = (touchEnd.y - _touchStart.y) / (480 / _viewHeight); 
		
		if(dx == 0 && dy == 0)
			return;
		
		//NSLog(@"drag %f, %f",dx,dy);
		
		_viewX += dx;
		_viewY += dy;
		
		_touchStart = touchEnd;
	}
	// zoom
	else if(touchCount == 2)
	{
		NSArray *touchArray = [allTouches allObjects];
		CGPoint first = [[touchArray objectAtIndex:0] locationInView:self];
		CGPoint second = [[touchArray objectAtIndex:1] locationInView:self];
		
		
		float dx = first.x - second.x;
		float dy = first.y - second.y;
		// how far apart the touches are
		float touchDistance = sqrt(dx*dx + dy*dy);
		// the ratio of how far apart the touches are now to how far apart they were to start with
		float ratio = touchDistance / _lastTouchDistance; // greater than one for pinching out to zoom in
		// new width and height of the view port into the game world
		float newWidth = _viewWidth / ratio;
		float newHeight = _viewHeight / ratio;
		// how much the width and height changed in world coordinates
		float dWidth = _viewWidth - newWidth;
		float dHeight = _viewHeight - newHeight;
		// the position of the midpoint of the touch as a percentage of the total distance
		float rx = _touchStart.x / 320;
		float ry = 1 - (_touchStart.y / 480); // touch y and game y are going different directions
		// apply changes
		_viewX += dWidth * rx;
		_viewY += dHeight * ry;
		_viewWidth = newWidth;
		_viewHeight = newHeight;
		
		// reset previous distance since we've applied changes
		_lastTouchDistance = touchDistance;
	}
	
}

touchesMoved is where the real magic takes place. This is the function that is called whenever the user moves her fingers on the screen. Here we take the ratio of how far apart the user's fingers are now divided by how far apart they were to begin with and use that as our zoom factor. If the user is pinching out, the ratio is greater than one, causing the view width to get smaller, which causes the "zoom in" effect.

In order to ensure that the point at the midpoint of the pinch does not appear to move, we must also adjust _viewX and _viewY which define the bottom-left corner of the viewport. In order to do this, we determine how far across the screen the midpoint of the pinch is, expressed as a percentage from 0 to 1 (from left to right or top to bottom). Note that we have to invert the y coordinate from the pinch, because the origin of the screen in normal iPhone coordinates is at the top left corner, with y increasing as you go down, while in OpenGL it is (by default, you can change this if you want) at the bottom right corner, with y increasing as you go up. We multiply that percentage by the change of width and add that to the x coordinate of the viewport (this will be negative for zooming out) and likewise for the y coordinate. This adjusts the position of the viewport so as to keep the image fixed at the center of the pinch.

After we apply the zoom, we save the current distance of the pinch back to _lastTouchDistance, because the user is probably not done zooming, even though we have already saved changes to the viewport.

And this is how I set the viewport when I render the view:

    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrthof(_viewX, _viewX + _viewWidth, _viewY, _viewY + _viewHeight, -1.0f, 1.0f);

The arguments to glOrthof are left, right, bottom, top, near, far.

Please note that this is pre-alpha, proof-of-concept code. It has no checks to make sure that the user doesn't zoom in or out too far, and is at present quite prone to crashing. Furthermore it is not optimized at all. The basic concept is sound, however.

UPDATE: I have fixed the code. The problem was that the touches parameter passed to the touchesBegan: and touchesMoved: event handlers does not contain all the touches, only the touches that began or moved. Because of the way the simulator works, the touches both occurred and moved simultaneously and the code worked. However in the real world it is very difficult for the user to hit the screen with both fingers at the same time, and therefore the application will get two touchesBegan: calls, one for each finger. Similarly, if the user only moves one finger in their pinch, there would only be one touch in the parameter for touchesMoved: which would crash the code as it was previously posted. The code now works on a real iPhone.