Shuffle Testing

This document should help clarify how to test our shuffling algorithm with the help of some handy python scripts.

Tools You Will Use

The following python scripts should help you create some measurable testing scenarios.

utils/generatePhraseFile.py
utils/dupetests.py

Pack Generation (generatePhraseFile.py)

Currently there are three packs that are already in the PhraseCraze build.

pack1.json (500 cards)  
pack2.json (500 cards)  
pack3.json (250 cards)

The difficulty of these packs is evenly distributed (0,1,2) -- generatePhraseFile.py does this. It might be worth parameterizing this as part of pack generation (i.e. all of pack1 will be difficulty X, etc.)

To generate a pack:

You can use the below command to create some test packs.

python utils/generatePhraseFile.py <# phrases to generate> <pack Id> <starting ID> <name of outfile>

This will create a pack with phraseIds of ${startingID} to ${startingID + # of phrases}. So an example of how I generated my three packs above is:

python utils/generatePhraseFile.py 500 1 0 res/raw/pack1.json  
python utils/generatePhraseFile.py 500 2 500 res/raw/pack2.json  
python utils/generatePhraseFile.py 250 3 1000 res/raw/pack3.json

An excerpt of the packs this will generate follows

Pack1
{"phrase": "WORD 1 - PACK 1", "_id": 1, "difficulty": 0}
{"phrase": "WORD 2 - PACK 1", "_id": 2, "difficulty": 1}
{"phrase": "WORD 3 - PACK 1", "_id": 3, "difficulty": 2}
...
Pack2
{"phrase": "WORD 1 - PACK 2", "_id": 501, "difficulty": 0}
{"phrase": "WORD 2 - PACK 2", "_id": 502, "difficulty": 1}
{"phrase": "WORD 3 - PACK 2", "_id": 503, "difficulty": 2}
...
Pack3
{"phrase": "WORD 1 - PACK 3", "_id": 1001, "difficulty": 0}
{"phrase": "WORD 2 - PACK 3", "_id": 1002, "difficulty": 1}
{"phrase": "WORD 3 - PACK 3", "_id": 1003, "difficulty": 2}
....

If you add more to these three

You will need Deck.java to installPackFromResource() your new pack.

Add a Pack object to the Deck class

  mStarterPack = new Pack(1, "starterPack", "freepacks/starterPack.json", "icons/starter_icon.png",
    "Description of pack1", 125, PackPurchaseConsts.PACKTYPE_FREE, 0, true);
  ...

Add your pack to this code block at line 500.

/**
 * Install all the packs that come with the app into the database.
 * Since the pack
 */
public synchronized void installStarterPacks(Pack starterPack) {
  Log.d(TAG, "installStarterPacks()");
  installPackFromResource(starterPack, R.raw.starter);
}

Then it will insert your pack during FIRST app installation.

Shuffle Testing (dupetests.py)

You can use this tool to help test the shuffle algorithm.

What it does

This script takes a streaming input from adb logcat and parses the output for "Dealing". When that is seen, it parses the line to find which word (or phrase) was dealt. It will continue until you hit Control C, unplug the phone, or reach a number of lines specified as a parameter.

To run it

First you should clear the log file (I tried scripting that but it didn't work and it's only a minor inconvenience to do).

adb logcat -c

Next you simply get the emulator or phone running and use the following command

adb logcat | python utils/dupetests.py <# of phrases to test>

This will pipe all of the logcat output into the script and will stop when ${# of phrases to test} is reached.

Example Output

Feel free to modify the script as needed, but right now it outputs the following data.

TOTAL WORDS: 397
PACK PERCENTAGES: {'1': 156, '3': 81, '2': 160}
PackID 1 had 156 cards shown which was 39.29% of total
PackID 3 had 81 cards shown which was 20.4% of total
PackID 2 had 160 cards shown which was 40.3% of total
DISTRIBUTION OF WORD_COUNTS: {1: 397}
Number of words seen 1 time(s): 397

Tips

Modify Turn.java to increase the number of words seen.

 private final OnClickListener mSkipListener = new OnClickListener() {
    public void onClick(View v) {
    int i=0;
    while (i<500) {
      Turn.this.doSkip();
      ++i;
    }
  }

Turn off shuffling to eliminate some false duplicates (the shuffling allows for words to come up early by BACK_CACHE_SIZE # of phrases). In the Deck class comment out...

// 3. Now shuffle
Collections.shuffle(mBackCache);

Test Scenarios

We need to ensure the following scenarios work

• Packs are represented proportionally to their size against the entire set of playable cards
  i.e. If a pack is 25% of the playable cards, 25% of the cards you see should come from that deck
• Phrases do not repeat until the entire pack has been traversed. And then they should not repeat until the entire pack is traversed again.
• Difficulty changes in Preferences correctly change logic of shuffling.
  i.e. If you are playing with Kid's pack with only Easy on, and you play with a normal pack, then Kids pack now proportionally is much more of the total playable cards. So if you have 500 Kids words, and 500 normal pack words, then now that you are playing on easy, Kids pack should be much more than 50% of the words seen.
• Remainders are allocated in a way that does not cause errors. It will however, slightly affect the proportions, as the remainder is randomly assigned to a pack. We need to make sure this doesn't cause duplicates somehow though.
• Modifications to Deck.THROW_BACK_PERCENTAGE in no way affect the Pack proportions. I still like this mechanic but it does complicate things.
• Modifications to other Deck constants do not affect results in adverse ways. For example, if we double the FRONT_CACHE_MAXSIZE.

Known Issues

I need to get the deck to update onPause() so you don't see duplicates when starting up a game (since it doesn't write to the db that the word has been seen).