ASTest
0.6.0

----------------------------------------------------------------------
SUMMARY

A testing framework for AppleScript.

----------------------------------------------------------------------
DESCRIPTION

Provides a convenient foundation for developing test code for your scripts. See the Unit Testing chapter of the online book Dive Into Python  for a good introduction to unit testing.

xUnit frameworks are best known, e.g. JUnit for Java. ASTest's design is influenced by xUnit family, but altered to accommodate the more basic AppleScript language.

-------

Differences between ASTest and a typical xUnit framework:

- Whereas an xUnit framework such as PyUnit uses classes for fixtures and methods within that class for test cases, ASTest uses script objects to encapsulate both fixture code and test case code - necessary due to AS's lack of support for callbacks.

- No callback mechanism means that assertError also works differently: calls to target handler must be "sandboxed" in a separate event handler, callHandler, defined by user in TestCase or test case's Fixture.

- AS lacks introspection, so auto-registering of tests is handled through an alternate mechanism at compile-time. (No manual test-registering mechanism is needed - or provided.) This alternate mechanism does mean that user-defined Fixture and TestCase objects are limited to using parent objects (Fixture, TestCase) defined by ASTest and cannot be (usefully) subclassed, though this shouldn't be a significant problem for users.

- Result reporting is slightly different (this is only an issue for developers seeking to extend ASTest functionality).

-------

How tests are constructed:

- Test Suite: In ASTest, a test suite is represented by a script file that takes the ASTest library as its parent. A test suite contains one or more test cases.

- Test Case: Contains the code for testing one particular aspect of behaviour of a script. A test case consists of  a fixture and the code that performs the actual test.

- Fixture: Contains any set-up and clean-up code to be executed before and after the test code itself, along with any sample data or other material needed by the test code. For convenience, a single fixture may be shared by any number of test cases.

-------

Every TestSuite script must begin with these four lines:

	property _Loader : run application "LoaderServer"
	property parent : _Loader's makeLoader()'s loadLib("ASTest")
	property testCases : missing value
	property suiteName : "UNTITLED"

The first two properties bind the ASTest framework to the script at compile-time. The third property is used by ASTest to store compiled test cases. (These lines must always appear exactly as shown.) The fourth property contains the name of your test suite; change its value to suit.

-------

To define a fixture:

	script FIXTURE_NAME
		property parent : makeFixture(me)
		
		-- fixture code goes here...
		
		-- test case objects go here...
		
	end script

The parent property must appear first in the script object. Any fixture code, eg setUp and cleanUp handlers, must appear next. Test case object(s) should be last.

-------

To define a test case:

		script TEST_CASE_NAME
			property parent : makeTestCase(me)
			
			on run
				-- test code goes here...
				
			end run
		end script

(Note that run handler can be implicitly or explicitly declared.)

-------

-- TO FINISH!!!

----------------------------------------------------------------------
COMMANDS

Main Suite : General commands

	countTests() -- get number of test cases
		Result : integer
	
	listTests() -- get names of test cases
		Result : list of strings
	
	targetScript() -- get the script being tested
		Result : script
	
	runTests(targetScript) -- run all test cases
		targetScript : script -- the script to test
		Result : anything -- value returned by the installed result listener's getResult method (e.g. DefaultListener returns text)



Accessor Suite : Commands used to retrieve test decorator and result listener collections

	addTestDecorator(testDecorator) -- install a user-defined test decorator for use
		testDecorator : script -- a test decorator object (see also OBJECTS > Test Decorators Suite > Decorator)

	setResultListener(resultListener) -- install a user-defined test result listener
		resultListener : script -- a result listener object (see also OBJECTS > Test Result Suite > Listener)



Constructor Suite : Commands used to construct test cases, custom test decorators and result listeners

	makeFixture(userFixture)
		userFixture : script -- user-defined fixture object
		Result : script -- a Fixture base object

	makeTestCase(userTestCase)
		userTestCase : script -- user-defined test case object
		Result : script -- a Test base object

	makeDecorator()
		Result : script -- a Decorator base object
	
	makeListener()
		Result : script -- a Listener base object
	
	makeDefaultListener()
		Result : script -- a DefaultListener object (see OBJECTS > Test Result Suite > DefaultListener)

----------------------------------------------------------------------
OBJECTS

Test Case Suite : Objects used to construct test cases

	TestSuite : (inherits from ASTest) A user-defined script (file) containing one or more test cases
		Properties:
			testCases : missing value -- used by ASTest to store compiled TestCase objects. Do not access this yourself.
			suiteName : string [r/o] -- name of this test suite



	Fixture : Provides foundation to a user-defined test fixture
		Properties:
			checkList : list of strings -- a list of names of any test decorators to be applied to test [1]
			
		Events:
			setUp() -- perform any pre-test preparation; e.g. establishing an internet connection before testing an internet-related handler [2]
		
			cleanUp() -- perform any post-test cleanup; e.g. breaking the internet connection after testing an internet-related handler [2]
		
			callHandler(userParams) -- called by assertError; must call the handler being tested [3]
				userParams : list -- list of parameters to be used by target handler
	
	
	
	Fixture : (inherits from Fixture) A user-defined test fixture; may override some/all/none of Fixture's properties and/or event handlers



	TestCase : (inherits from Fixture) Provides foundation to a user-defined test case
		Methods:
			targetScript() -- get the script being tested
				Result : script
		
			failTest(msg) -- force this test to fail
				msg : string -- the message to report
			
			assertTrue(actualResult, msg) -- fail this test if actualResult is not true
				actualResult : anything -- the value to test
				msg : string -- the message to report if assert fails
		
			assertFalse(actualResult, msg) -- fail this test if actualResult is not false
				actualResult : anything -- the value to test
				msg : string -- the message to report if assert fails
			
			assertEqual(actualResult, expectedResult, msg) -- fail this test if actualResult is not expectedResult
				actualResult : anything -- the value to test
				expectedResult : anything -- the value expected
				msg : string -- the message to report if assert fails
			
			assertNotEqual(actualResult, unexpectedResult, msg) -- fail this test if actualResult is expectedResult
				actualResult : anything -- the value to test
				unexpectedResult : anything -- the value not expected
				msg : string -- the message to report if assert fails
			
			assertError(userParams, expectedErrorNum, msg) -- fail this test if error number expectedErrorNum is not thrown
				userParams : list -- list of parameters to pass to callHandler event handler [4]
				expectedErrorNum : integer -- the error number which should thrown by the target handler
				
		Events:
			run -- run the test code for this test case; this must be overridden in TestCase
		


	TestCase : (inherits from TestCase)  A user-defined test fixture



Test Decorators Suite : Objects used to extend test case functionality

	Decorator : Provides foundation to a user-defined test decorator (See AT_Decorators component for more info)
		Properties:
			name : string -- decorator's name
		Events:
			callTestCase(testCase) -- wraps events sent to test case's setUp, run and cleanUp handlers [5]
				testCase : script -- TestCase object
			
			callRun(testCase) -- wraps event sent to test case's run handler [6]
				testCase : script -- TestCase object



Test Result Suite : Objects used to record and report test results

		TestCaseResult : Contains results for a single test case
			Properties:
				fixtureName : string [r/o]
				testCaseName : string [r/o]
				failed : boolean [r/o]
				errored : boolean [r/o]
				errorLocation : missing value or string [r/o] -- point at which test failed, where applicable
				userFailed : boolean [r/o] -- true if user's test code called failTest
				assertGaveWrongResult : boolean [r/o] -- true if target handler gave unexpected result
				assertFailedToRaiseError : boolean [r/o] -- true if target handler failed to raise an expected error
				expected : missing value [r/o] -- expected result or error number, where applicable
				actual : missing value [r/o] -- actual result or unexpected error number, where applicable
				paramsAreAvailable : boolean [r/o] -- true if parameters that caused failed test are available for inspection (only available with assertError)
				params : missing value or anything [r/o] -- parameters that caused failed test (only available with assertError)
				msg : string [r/o] -- user-defined message passed by assert call
	
	

	Listener : Provides foundation to a user-defined result listener
		Events:
			startSuite(testSuite) -- event sent before running all tests in test suite [7]
				testSuite : script -- the test suite script

			stopSuite(testSuite) -- event sent after running all tests in test suite [7]
				testSuite : script -- the test suite script

			startTestCase(testCaseResult) -- event sent before calling a test case's setUp, run and cleanUp handlers [7]
				testCaseResult : script -- the TestCaseResult object used by this test case
		
			stopTestCase(testCaseResult) -- event sent after calling a test case's setUp, run and cleanUp handlers [7]
				testCaseResult : script -- the TestCaseResult object used by this test case
			
		Methods:
			getResult() -- [7]
				Result : anything



	DefaultListener : (inherits from Listener) Records test results and reports them as formatted text

--

[1] Built-in checking options:
	"considering/ignoring" -- see if string comparison operations are affected by different considering/ignoring options
	"TIDs" -- see if external changes to AppleScript's text item delimiters property affect target handler's results
	"TIDs+" -- see if external changes to AppleScript's text item delimiters property affect target handler's results AND check that target handler preserves previous TIDs
	Additional user-defined test decorators may be supplied via addTestDecorator command

[2] Overriding one or both of these handlers is optional.

[3] This must be overridden if test case contains any assertError calls.

[4] Automatically calls user-defined callHandler handler (see OBJECTS > Fixture), passing it the userParams list.

[5] Overriding this handler is optional. If done, handler must include a 'continue callTestCase(testCase)' statement.

[6] Overriding this handler is optional. If done, handler must include a 'continue callRun(testCase)' statement.

[7] Overriding any of these handlers is optional.

----------------------------------------------------------------------
NOTES

- Property/handler names beginning "AT_", "_AT_", "__AT_" are reserved for ASTest's private use.

----------------------------------------------------------------------
TO DO

- How can users bind additional libraries to their script if the inheritance structure doesn't (legitimately) allow them to be referenced? This is a problem, esp. given the removal of convenience functions such as presentData from test case objects.

- Ability to run selected tests? If so, how best to implement? (Probably easiest to add a runSelectedTests command that takes a list of test names, and skip tests not named in that list. Or maybe a list of each fixture, treating test cases that share a single fixture as a group. Or maybe both?)

- Rework TestCaseResult and result listeners into TestResult class?

    startTest(test) -- Called when the given test is about to be run
	stopTest(test) -- Called when the given test has been run
	addError(test, err) -- Called when an error has occurred. 'err' is a tuple of values as returned by sys.exc_info().
	addFailure(test, err) -- Called when an error has occurred. 'err' is a tuple of values as returned by sys.exc_info().
	addSuccess(test) -- Called when a test has completed successfully
	wasSuccessful() -- Tells whether or not this result was a success
	stop() -- Indicates that the tests should be aborted
	
	Note: one reason it wasn't done this way is that AS doesn't have a nice way of generating tracebacks, formatting parameters, etc. Also not much point in passing test object, as there's not much can be done with that (e.g. can't inspect it for name; that info has to be provided separately).
	
	Current approach is clumsy but works, and since it only really affects developers writing their own result recording tools (e.g. graphical test runners) this is low priority.

- Reduce size of test scripts by binding proxy objects at compile-time and only loading the real objects at runtime? (Note: it's a lot simpler if users just save scripts in uncompiled form as saved file size then ceases to be a problem.) Low priority.

- Check that events are sent to correct object, e.g. setUp/cleanUp messages ought to go directly to Fixture, not to TestCase. e.g. IIRC, callHandler message is sent to TestCase, despite what documentation says.

- Finish documentation. Check for possible confusion due to listing handlers as "Events" under base object (e.g. Fixture base), rather than object that receives those events (e.g. Fixture).

----------------------------------------------------------------------
DEPENDENCIES

OSAXen
	- Standard Additions

Libraries
	- ConvertToString
	- Loader
	- Timer

----------------------------------------------------------------------
HISTORY

2003-10-01 -- 0.6.0; Loader support; heavy restructuring; code now a bit simpler, much cleaner and easier to follow; no more event handler 'init-run-flush' sequence, so more flexible; added/renamed/removed many commands/objects (see COMMANDS and OBJECTS); now has fixed class "ASTest"

2003-03-08 -- 0.5.0; fixed default considering/ignoring check (now considers diacriticals); all check decorators now include the 'expansion' option; added presentData function to TestBase class

2002/10/11 -- 0.4.1; double quick patch release... Cripes! Caught nasty file-bloating, internal table-overflowing problem. Context changes, object copies... all cunning booby-traps laid to make AS blow its lid in outright indignation when given the slightest excuse. Oh, the fragility!

2002/10/09 -- 0.4; lots of internal restructuring: totally changed the way test decorators are implemented, and added TestSuite class, although the user doesn't access any of this stuff directly - or even need to know it exists. (Each fixture forms one test suite containing all tests for that fixture, while the fixture suites are themselves consolidated as a single suite, stored in the testCases record. This cleans up the internal structure a bit, and will hopefully make it easier to amalgamate tests from different UnitTest files, once ASTest itself implements a suitable facade.) Documentation for implementers adding their own decorators/listeners or wishing to do clever things with test stuites should follow... eventually.

2002/10/04 -- 0.3.1; ASTest now has class "ASTestFramework and compiled UnitTests now have class "ASUnitTest", allowing them to be distinguished from each other and from non ASTest-related scripts; added trimNumber(num, toPlaces), compareReals(real1, real2)

2002/10/02 -- 0.3; heavy redesign

2002/09/25 -- 0.2; first release made for feedback purposes

----------------------------------------------------------------------
AUTHORS

- HAS

----------------------------------------------------------------------
COPYRIGHT

Copyright (c) 2003 HAS

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.