DXR will be turned off on Tuesday, December 29th. It will redirect to Searchfox.
See the announcement on Discourse.

DXR is a code search and navigation tool aimed at making sense of large projects. It supports full-text and regex searches as well as structural queries.

Mercurial (2028a9d7ce91)

VCS Links

Line Code
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235
# Test Pilot Experiment Repo and Website

## url

https://testpilot.mozillalabs.com/  OR  http://testpilot1.vm.labs.scl3.mozilla.com (as of 7/2012).

## about `index.json` files

hints:

*   the 'Dependencies' base classes are needed by all studies in Test Pilot 1.
    This allows us to update code more responsively.  Leave these sections in :)
*   `python test_valid.py some.json` can diagnose a lot of common problems with these files.
*   The format is poorly documented.  Sorry!  We are working on that for TP2.


## adding or removing testcases / experiments / surveys to the manifest ##

(you will need access to MPT:  https://intranet.mozilla.org/IT_MPT-RemoteAccess)

`testcases/index.json` and friends are where the listing of all running experiments live.

On your local machine.

1.  `hg clone http://hg.mozilla.org/labs/testpilotweb/`
2.  edit `testcases/index.json` or whatever is used by your pref.
    (for instance `index-tb.json` for Thunderbird)

    The format of these is beyond scope of this document, but is
    part of the TestPilot code.  For purposes here, adding or
    taking them out is enough to start and stop studies.

3.  `cd testcases/ && python test_valid.py index.json`
    This checks for valid json, makes sure the experiement jars
    are consistent, etc.  (`base_classes` throws warnings, ignore them)

4.  Make sure you are current with central.

    `hg ci -m "my edit"; hg pull; hg up; hg push;`

5.  install python with fabric... suggestion:

    `pip install -r requirements.txt`

6.  make sure you are on MV or MPT vpns.
7.  `fab push -u user.name`, confirming as appropriate.  See `fabfile.py` to see
    what is actually happening.

    (Hint:  `yes | fab push -u username` if you *really know* you got it right)

8.  VERIFY your changes!  `curl https://testpilot.mozillalabs.com/testcases/index.json | less`


## making simple surveys (new, possibly unstable)

1.  `cd testcases`
2.  then:

    fab simplesurvey:"http://some/external/url","My Survey","Awesome survey about whatever\nin two lines";
    \# writing to: my_survey/my_survey.js
    \# edit: my_survey/my_survey.js
    \# then:
    fab package:my_survey


Read `fabfile.py` for more details.

## Debugging TP studies / setting up a local server. ##

### short version = new hotness

1.  use `testpilothelper` (http://github.com/gregglind/testpilothelper.git)
2.  rm -rf /tmp/moz1 && cfx run -b `alias aurora | cut -d= -f2 | tr -d "'" ` --addon ~/hgs/testpilot/testpilot-1.2.2.xpi  -p /tmp/moz1

### long version  - for historical interest only

WARNING: gory, and not safe for children and sysadmins

(editorial note:  this is absolutely not the *simplest* way to do this,
just one that works!  Suggestions and simplifications are totally totally welcome!)

(note:  see the "local study code" subsection below for a summarized view)

0. `hg up` to have  the 'ssl not required' patch, so the remote loader won't care.
   (this should be in tip versions of testpilot, after 1.2.1 final)
1.  install aurora or your favorite browser of choice.
2.  setup alias to aurora (optional!), like:

    `alias aurora='/Applications/Aurora.app/Contents/MacOS/firefox'`

3.  create a new profile (say, 'aurora-debug'):

    `aurora -p`

4.  install the addon (live, so changes track):

    `glind ~/hgs/testpilot [hg:v121] $ python manage.py install -p aurora-debug`

    NOTE:  you might have to hand-delete the addon from profile first to make sure it is gone.

5.  url: `about:config`, then change these settings:

    * `browser.dom.window.dump.enabled` -> `true`
    * `extensions.testpilot.indexFileName` -> `index-my.json`
    * `extensions.testpilot.indexBaseURL` -> `http://localhost:8000/testcases/
    * `extensions.testpilot.popup.delayAfterStartup` -> `1000`
    * `extesnsions.testpilot.ssldownloadrequired` -> `false`

6.  make sure you have `fabric` installed!  (`pip install fabric`, or virtualenv, etc.)

7.  shutdown, and clean any lingering experiments out from the `testpilotweb`:

    ` ~/hgs/testpilotweb/testcases $ fab clear:aurora-debug`

8.  set up a local server.  This will start on 8000

    `~/hgs/testpilotweb $ python -m SimpleHTTPServer 8000 &`

9.  Start `aurora` from the cli, and watch for output.  You should see hits on the server process!

10. url:  `resource://testpilot/content/debug.html` and `about:config` to enjoy the fun!

11. tailing the ErrorLog helps (see code below).  `dump` liberally.

12. if you make changes to a study, you need to repackage (see later topic), so the `.jar` gets updated!


### local study code ###

    #!/bin/bash

    shopt -s expand_aliases
    alias aurora='/Applications/Aurora.app/Contents/MacOS/firefox'
    PROFILE=aurora-debug
    TPWEB=~/hgs/testpilotweb
    TP=~/hgs/testpilot
    TPURL=http://localhost:8000/testcases/
    TPPROFILEDIR=$(fab -f "$TPWEB/fabfile.py" profile:"$PROFILE" | head -n1)

    ## setup - get a 'live' tp install.  (usual add-on developer tricks)
    # destroy all old tp installs
    rm -rf "$TPPROFILEDIR"/extensions/*testpilot*
    # install a pointer the 'live' TP code
    python "$TP"/manage.py install -p "$PROFILE"

    # verify it works... this should be the same at "$TP"
    if [ $(cat "$TPPROFILEDIR"/extensions/testpilot@labs.mozilla.com) = \
        "$TP"/extension ]; then echo "live install verified"; fi

    ## now go through and do all your settings
    (echo "# user prefs for debugging a local testpilot install" &&
    echo 'user_pref("browser.dom.window.dump.enabled",true);' &&
    echo 'user_pref("extensions.testpilot.indexFileName","index-my.json");'  &&
    echo 'user_pref("extensions.testpilot.indexBaseURL","'$TPURL'");'  &&
    echo 'user_pref("extensions.testpilot.popup.delayAfterStartup",1000);'  &&
    echo 'user_pref("extensions.testpilot.ssldownloadrequired",false);' ) > "$TPPROFILEDIR"/user.js

    ## python server startup... or choose some other mechanism.
    curl -s "$TPURL"/index-my.json >/dev/null || \
        (cd "$TPWEB"
        python -m SimpleHTTPServer &
        cd -)

    # the very first time, you have to 'enable' tp, from the addons window.

    # (re)starting the aurora for actually running it.
    # assuming you set the server and port to 8000 etc.
    # (annoyingly becuase of the &&, this won't save to bash history! (solution?))
    curl -s "$TPURL"/index-my.json >/dev/null && \
    fab -f "$TPWEB/fabfile.py" clear:"$PROFILE" && \
    aurora -new-instance -P "$PROFILE" \
      "about:config" "resource://testpilot/content/debug.html" "about:addons" &

## packaging a study ##

To update a test ready for pushing out:

### (simple version:) ###

1.  `fab package:heatmap,"Desktop Heatmap 2012"`
2.  put the created json in your `index.json`
3.  `python test_valid.py index.json`

### Long-form extended party mix ###

1.  jar all the files into one jar, e.g.

    `jar cvf week_life.jar weeklife.js`

2.   Generate an sha256 sum

     `shasum -a 256 week_life.jar`

3.   In the relevant index*.json file, add/update the section similar to:

          {"default":
            {"name": "A Week in the Life of a Mail Client",
             "jarfile": "thunderbird/week_life/week_life.jar",
             "studyfile": "week_life.js",
             "hash": "baa51717cdf964934528837ae1be9a4ddef8a666fa54c9cdeb83147bc807c920"
            }
          }

4.   Run the `test_valid.py` script to check for validity

        # python test_valid.py index-tb-dev.json
        INFO:root:JSON is good
        INFO:root:studyfile good dummy_study.js
        INFO:root:studyfile good week_life.js

5.  Check for errors in the above. The warnings for root:study missing attributes and not finding the base file are fine. The rest of the studies should say "good".
6.  Check the index*.json file is loaded correctly in Test Pilot. Check that your test runs fine, etc!


## Thunderbird Debug ##

from Mark Banner (mbanner@):

Open Preferences > advanced > (config editor):

    mailnews.start_page.override_url -> chrome://testpilot/content/debug.html

Or:

    user_pref("mailnews.start_page.override_url","chrome://testpilot/content/debug.html");

## To get data from studies
1. cd into your tp-dataviz-prototype directory
1. scp gimmedata.py to research-gw.research.stage.metrics.scl3.mozilla.com
2. ssh into machine (you will need MPT VPN) and go to directory where you put gimmedata.py
3. follow instructions on gimmedata.py