Just for the benefit of anyone else finding this thread, here's the AppleScript that incorporates my suggestions. To recap:

Break the cycle by splitting one cell into two: create a new cell (the "output" cell) into which you copy the formula from the original cell. Remove the formula from the original cell (the "input" cell), so it becomes an ordinary data entry cell. Nothing else needs to change. In particular, all formulas that referred to the original cell still do, and no formulas refer to the new cell. Convergence means the values in the "input" and "output" cells are (approximately) equal.

Customize the following AppleScript and run it. The customizations all appear near the top of the script, and correspond roughly to entering values into Excel's preferences to limit the number of iterations and set a convergence factor. I've added the ability to set a time limit rather than (or in addition to) an iteration limit, something that Excel doesn't provide but that I think is actually more meaningful.

In the example, the limits are encoded into the AppleScript, but they could equally well be pulled from the spreadsheet itself.
Code:
-- maximum number of iterations. 0 means unlimited
set iteration_limit to 100
-- relative precision required. 0.0 means must be exact
set precision to 1.0E-9
set deadline to (current date) + 5 * minutes


tell application "Numbers"
	set inputCell to a reference to cell 2 of row 1 of table "Guesses" of sheet 1 of document 1
	set outputCell to a reference to cell 2 of row 2 of table "Guesses" of sheet 1 of document 1
	
	-- ***** Only the part of the script above this line needs to be customized ***** --
	

	set oldvalue to value of outputCell
	repeat
		-- always do at least one iteration
		set value of inputCell to oldvalue
		set newvalue to value of outputCell
		
		-- close enough?
		if my closeenough(oldvalue, newvalue, precision) then exit repeat
		set oldvalue to newvalue
		
		-- too many iterations?
		-- test for =0 instead of <0, so setting limit to 0 will mean unlimited
		set iteration_limit to iteration_limit - 1
		if iteration_limit = 0 then exit repeat
		
		-- out of time?
		if (current date) comes after deadline then exit repeat
	end repeat
end tell

on abs(x)
	if x < 0 then return -x
	return x
end abs

on max(a, b)
	if a > b then return a
	return b
end max

on closeenough(a, b, prec)
	if a = b then return true -- exact is always close enough, even if prec = 0.0
	return abs(a - b) < prec * (max(abs(a), abs(b)))
end closeenough