Strategic preparation for achievement testing

Applets

Ben Wilbrink

For the text on the SPA model see the project.

applets     advanced applets         other applets

    1         1a    Generator        — 1m. 3-valued — 1o. 1oa advanced
    2         2a    Mastery Envelope
    3         3a    Predictor
    4         4a    Ruling
    5         5a    Learning
    6         6a    Expectation     6b. special 6.1a.  advanced
    7         7a    Last Test
    8         8a    Strategy
    9         9a True Utility

applet 1

• If the applet bar chart above does not show up - it should look like the (scaled down) picture at the left because this applet will plot only after clicking the ‘Go’ button - the cause might be a faulty program. In that case the older applet (see below) might still function appropriately. If the applet itself is OK, then check the preferences of your browser as far as Java is concerned (JAVA 1.4.2), try a newer version of your browser, or download the new browser Firefox by Mozilla. Firefox will offer you the opportunity to download the appropriate Java software from the Sun site. Under Windows Firefox probably will run applets smoothly. On the Mac OS X platform the applet will run smoothly under Safari and Firefox, while Internet Explorer does not quite correctly plot the bar chart but is acceptable. Currently (April 2005) the JVM used is 1.4.2.
  
  

menu items

• Clicking the Go button will start a new simulation and/or analysis using the values specified in the menu.

  exception: using option 1 to leave the analysis out, and next another option, the analysis will take affect ony at the second ‘Go.’

  Because the applet is in a scrollable window, scrolling will cause the applet to execute again. Remember this whenever you declare a larger number of runs or observations. Eventually I might make the applets available in stand-alone format. Such as spa_applet1.sa.htm

• Be aware that all parameters, as well as the option, must hae a specific value. The applet will not run on either a blanc, or anything else that is not a discrete or real number.

• The simulation simulates test scores for the specified number of observations or cases, assuming mastery having the value specified; the resulting distribution of test scores will be plotted as a solid green bar diagram. For larger numbers of observations the simulation will will be closer to the binomial distribution plotted in blue. If you want the analysis only, turn the simulation offf by putting the number of observations = 0.

• The analysis evaluates a probability distribution function called the binomial distribution; the distribution plotted is this mathematical distribution. The binomial distribution is the theoretical distribution of test scores given a particular mastery, for the number of observations growing very large. If you want the simulation only, put option = 1 to turn the analysis off.

• Mastery may be chosen between 0 and 1; mastery as a concept is defined on the set of test items that every concrete test is supposed to have been sampled from: it is the proportion of items the candidate would answer correctly if given the opportunity to try alle items in the domain.

• The number of test items is the length of the test; this number is not limited.

• The number of observations or runs is the number of runs for the simulation, or, equivalently, the number of cases or persons that is represented in the simulated distribution.

• The reference point is a particular test score that is of interest to the user of the applet. It might be the cutoff score in pass-fail testing, the minimum score that is still considered to be a level of sufficient mastery, etcetera. It is marked by a solid vertical in the plot. Its value may be set at 0.

• The reported probability is the proportion of scores equal to or higher than the reference point.
• The option option will make it possible to run special analyses without cluttering the menu of the applet; available options will be mentioned in the applet window if option = 0.

• The horizontal scale is that of test scores obtained, the lowest score being 0.

• The vertical scale as a default will be automatically chosen and depends on the highest value of the analytical distribution, this point is tabmarked. The vertical values are frequencies in the case of simulations, and probabilities in the case of analyses. Their values may be plotted using option 203 and 204, respectively. Of course frequencies and probabilities will be comparable. In fact, option 2 will plot the vertical differences separately.

• The statistics reported are the mean and standard deviation for every distribution. Because the theoretical value of the binomial statistics assumes the number of observations to be infinite, there will be small differences between the actual statistics and the statistics obtained by evaluating their formulas. Simulation and analytical statistics will differ from each other, of course; the differences wil be smaller the larger the number of observations is chosen.

• NOT specifying a value for any of the parameters makes it impossible for the applet to run: at least specify a default value 0 or 0.0 (zero). Becareful to use decimal dots, not periods. Anything that is not a number, will obstruct the applet.
• Illegal specifications will be changed to default values that will enable to applet to run. That one of the values has been changed, will be reported in the plot, including thevariable number thatthe program itself uses. The numbers here are: 40: runs, 1: items, 13: mastery and 0: option. If other numbers are reported to have been changed, the fault is in the program itself: contact me, mentioning the number.
• Simulating a binomial process will result in a distribution that approaches the binomial distribution, the larger the number of runs. Of course it is possible to test for the fit between simulation and the binomial distribution by evaluating the chi-square statistic. But of what use would this be? There is no question about whether the binomial simulation will approximate the binomial distribution. It will do so. Different simulations of the same magnitude will result in different values in chi-square. So what? There is nothing here that turns about having available the chi-squarestatistic, so it is not made available. It is a big nuisance to program because of low cell frequencies that must be handled in one way or another. Another way to argue the irrelevance of testing for fit is that the statistical process generating the simulated scores is known perfectly well: it has been programmed. Testing for ‘fit’ would be totally out of place here. Of course, the program might be at fault; because simulation and analysis are strictly independent in the program, except for their input and output procedures, it is highly unlikely that a mistake in the simulation program would not be revealed by plotting simulation results against the theoretical results, for a number of runs that is large enough.

If you come across an applet that is not functioning properly, please mail me. It is not possible always to check all applets for unintended consequences of changes in classes. As this is a project in progress, such changes are made on a routine basis.

Applets are known to work correctly under:
Internet Explorer under Windows XP
Firefox 1.0.7 under Windows XP
Safari 1.2 under MacOSX 10.3.9
FireFox 1.5.0.4 under MacOSX 10.3.9

It might be the case that the applets do not open properly in browsers under Windows, or in browsers other than Safari under MacOS X: the applet field remains gray or blank.
In module chapters original applets have been replaced with screenshots; therefore applet problems should not hinder readers of the SPA project. Readers not able to use the applets in their browser, and yet willing to do so, may contact me, if preferences of the browser pertaining to Java do not seem to be the problem.

Information about Java, and applets in particular:
http://java.sun.com
http://java.sun.com/applets/

MacOS X: There is a problem with Java versions 1.4 for browsers other than Safari. See http://javaplugin.sourceforge.net/Readme.html; http://developer.apple.com/documentation/Java/Conceptual/Java131Development/deploying/chapter_3_section_5.html; simile.mit.edu/repository/ misc/java_embedding_plugin/readme.rtf
MacOS X: Opera, version 8.5, produces 'java.lang.UnsupportedClassVersionError: Spa_BinomialApplet (Unsupported major.minor version 48.0)
MacOS X: Internet Explorer 5.2 for Mac, [preferences: enable Java on; cookies: never ask; web content: enable plug-ins on] produces 'java.lang.UnsupportedClassVersionError: Spa_BinomialApplet (Unsupported major.minor version 48.0)

Windows: Java Applet plug-in makes it possible for your computer (including Windows® XP, Me, NT, 2000, 98, or 95) to run applets in your browser. http://www.mcdonalds.com/search/help/plug_play/sunmicro.html

Mail your opinion, suggestions, critique, experience on/with the SPA

  http://www.benwilbrink.nl/projecten/spa_applet1_bon.htm