@@ -3,10 +3,12 @@ title: "Introductory exercise to ASReview LAB"
33author : " The ASReview Academy Team"
44---
55
6- ## Introduction to the software ASReview LAB
6+ ** This exercise was made using ASReview LAB version 1.6. If you have a different version you can observe differences between your setup and the exercise.**
7+
8+ ## Introduction to the software ASReview LAB v1.6
79
810The ** goal** of this exercise is to get familiar with AI-aided screening
9- by making use of ASReview LAB v1.x .
11+ by making use of ASReview LAB v1.6 .
1012
1113You will learn how to install and set up the software, upload data,
1214screen records, and export and interpret the results. The exercise will
@@ -29,92 +31,50 @@ Before you start, you might want to read a bit more on:
2931- The paper that was published in [ Nature Machine
3032 Intelligence] ( https://www.nature.com/articles/s42256-020-00287-7 ) {target="_ blank"}
3133
32- ## The software
33-
34- ### Step 1: Installing Python and ASReview LAB
35-
36- First, you need to install [ Python] ( https://www.python.org/downloads/ ) {target="_ blank"}.
37-
38- Once you have Python installed, you can go through the easy 3-step
39- [ guide to installing (or upgrading)
40- ASReview] ( https://asreview.nl/download/ ) {target="_ blank"} on the ASReview-website.
41-
42- More detailed installation information, troubleshooting, and
43- installation on a server or via a Docker are available on
44- [ ReadTheDocs] ( https://asreview.readthedocs.io/en/latest/installation.html#install-asreview ) {target="_ blank"}.
45-
46- * Have you installed the latest version of ASReview? You can proceed to
47- step 2!*
48-
49- ### Step 2: Starting ASReview LAB
50-
51- To open ASReview LAB in your browser, you need to start it in the
52- command prompt (i.e. CMD.exe or Terminal). You can open your command
53- prompt by typing ‘cmd’ (for Windows) or ‘terminal’ (for Apple) in your
54- computer’s search bar (select ‘Run as administrator’ if you have this
55- option).
56-
57- ![ ] ( images/ASReviewLAB/CMD1.png ) {width=95%}
58-
59- The command prompt will open, in which you can type the following
60- command and press enter:
34+ ## Prerequisites
6135
62- ``` bash
63- asreview lab
64- ```
36+ ### Installing ASReview LAB
6537
66- ![ ] ( images/ASReviewLAB/CMD2.png ) {width=95%}
38+ Before you start the tutorial ASReview LAB needs to be installed; see for instructions the [ ASReview website ] ( https://asreview.nl/download/ ) {target=" _ blank"}.
6739
68- It takes a few seconds for ELAS - your Electronic Learning Assistant -
69- to start the software. It will appear in your (default) web browser.
40+ * Have you installed ASReview LAB? You can proceed to the exercise!*
7041
71- But why do you need to start it up by running code in your command
72- prompt? This ensures that ASReview LAB is run locally. More
73- specifically, <u >your data is and stays your own</u >. Small price to pay
74- for complete privacy, right?! Read more about the key principles in the
75- [ Zen of Elas] ( https://asreview.nl/blog/the-zen-of-elas/ ) {target="_ blank"}!
42+ ## Exercise
7643
77- Note that you have to keep your command-line interpreter running while
78- using ASReview LAB, even though the interface is in your browser!
79-
80- You can also run the software via a server, but you need to take care of
81- [ hosting the server
82- yourself] ( https://asreview.readthedocs.io/en/latest/installation.html#server-installation ) {target="_ blank"}
83- (or ask your IT-department).
44+ ### Step 1: Starting ASReview LAB
8445
46+ Open ASReview LAB in your browser. Note that if you do this via Command Prompt (Windows) or Terminal (MacOS) you have to keep your command-line interpreter running while using ASReview LAB, even though the interface is in your browser!
8547
8648* Have you opened ASReview LAB in your browser? If so, you can proceed to
87- step 3 !*
49+ step 2 !*
8850
89- ### Step 3 : Creating a project
51+ ### Step 2 : Creating a project
9052
91- Now that you have installed and opened ASReview LAB, you can create a
92- new project. Below you will find a step-by-step guide. Note that the
93- screenshots shown below are made in [ dark
94- mode] ( https://asreview.readthedocs.io/en/latest/screening.html#display ) {target="_ blank"}.
53+ Now that you have opened ASReview LAB, you can create a
54+ new project. Below you will find a step-by-step guide.
9555
96561 . * New project;*
9757
98- Hover your mouse over the ‘create’ button with the plus sign in the
58+ Hover your mouse over the ‘` create ` ’ button with the plus sign in the
9959bottom right corner.
10060
101- ![ ] ( images/ASReviewLAB/Interface2 .png ) {width=95% }
61+ ![ ] ( images/ASReviewLAB/step_2a_V1_6 .png ) {width=85% fig-align="center" }
10262
103632 . * Project name;*
10464
105- Select Validation Mode, fill out a project name and press ‘NEXT’. Note
65+ Select Validation Mode, fill out a project name and press ‘` NEXT ` ’. Note
10666that you can fill out your name and a description as well.
10767
108- ![ ] ( https://raw.githubusercontent.com/asreview/asreview/main/docs/images/setup_project_modes. png) {width=95% }
68+ ![ ] ( images/ASReviewLAB/step_2b_V1_6. png) {width=85% fig-align="center" }
10969
110- For this exercise we are screening in the so-called ‘Validation Mode’
70+ For this exercise we are screening in the so-called ‘` Validation Mode ` ’
11171of ASReview. By screening in the [ Validation
11272Mode] ( https://asreview.readthedocs.io/en/latest/project_create.html#project-modes ) {target="_ blank"},
11373we are going to make use of a [ benchmark
11474dataset] ( https://asreview.readthedocs.io/en/latest/data_labeled.html#fully-labeled-data ) {target="_ blank"}.
11575This means that all records in the dataset have already been labeled as
11676relevant or irrelevant. This is indicated to the user through a banner
117- above each article. Note that in ‘Oracle Mode’ - when screening your own
77+ above each article. Note that in ‘` Oracle Mode ` ’ - when screening your own
11878dataset - the relevant papers will not be marked; you, the oracle, have
11979to make the decisions.
12080
@@ -123,33 +83,33 @@ More detailed information about setting up a project can be found on
12383
12484
12585* Have you started creating a new project? If so, you can proceed to step
126- 4 !*
86+ 3 !*
12787
12888## Project setup
12989
130- ### Step 4 : The dataset
90+ ### Step 3 : The dataset
13191
13292Now that you have created your ASReview project (woohoo!), you need to set it
13393up. Without data, we have nothing to screen. So, you need to tell ELAS
13494which dataset you want to screen for relevant articles.
13595
136- Click on the ‘ADD’ button next to ‘Add dataset’. Now a menu appears
96+ Click on the ‘` ADD ` ’ button next to ‘` Add dataset ` ’. Now a menu appears
13797where you can choose how to load the dataset. You can add your dataset
13898by selecting a file or providing an URL. For this exercise, we will use
13999a benchmark dataset.
140100
141- Go to the ‘Benchmark datasets’ button, open the first dataset (i.e. the
101+ Go to the ‘` Benchmark datasets ` ’ button, open the first dataset (i.e. the
142102Van de Schoot (2017) dataset about PTSD trajectories) and click on
143- ‘SELECT ’. After you select the dataset, click on ‘SAVE’.
103+ ‘` ADD ` ’. After you select the dataset, click on ‘` SAVE ` ’.
144104
145- ![ ] ( images/ASReviewLAB/Interface5 .png ) {width=95% }
105+ ![ ] ( images/ASReviewLAB/step_3_V1_6 .png ) {width=85% fig-align="center" }
146106
147107
148108
149109* Have you successfully selected/uploaded the dataset? If so, you can
150- proceed to step 5 !*
110+ proceed to step 4 !*
151111
152- ### Step 5 : Prior knowledge
112+ ### Step 4 : Prior knowledge
153113
154114Before you can start screening the records, you need to tell ELAS what
155115kind of records you <u >are</u > and what kind of records you <u >are
@@ -165,16 +125,15 @@ irrelevant record). However, because you are using the Validation Mode
165125of ASReview, the relevant records are known; the original authors have
166126already read ALL records.
167127
168- To select the prior knowledge you first need to click on the ‘ADD’
169- button next to ‘Add prior knowledge’; see also the documentation
128+ To select the prior knowledge you first need to click on the ‘` ADD ` ’
129+ button next to ‘` Add prior knowledge ` ’; see also the documentation
170130about the selection of [ prior knowledge] ( https://asreview.readthedocs.io/en/latest/project_create.html#select-prior-knowledge ) {target="_ blank"}.
171131Now you will see a menu about selecting prior knowledge.
172132
173133The following five papers are known to be relevant:
174134
175- - Latent trajectories of trauma symptoms and resilience: the 3-year
176- longitudinal prospective USPER study of Danish veterans deployed in
177- Afghanistan (DOI: [ 10.4088/JCP.13m08914] ( https://doi.org/10.4088/jcp.13m08914 ) {target="_ blank"})
135+ - Latent Trajectories of Trauma Symptoms and Resilience
136+ (DOI: [ 10.4088/JCP.13m08914] ( https://doi.org/10.4088/jcp.13m08914 ) {target="_ blank"})
178137- A Latent Growth Mixture Modeling Approach to PTSD Symptoms in Rape
179138 Victims (DOI: [ 10.1177/1534765610395627] ( https://doi.org/10.1177/1534765610395627 ) {target="_ blank"})
180139- Peace and War: Trajectories of Posttraumatic Stress Disorder
@@ -185,22 +144,22 @@ The following five papers are known to be relevant:
185144- Trajectories of trauma symptoms and resilience in deployed US
186145 military service members: Prospective cohort study (DOI: [ 10.1192/bjp.bp.111.096552] ( https://doi.org/10.1192/bjp.bp.111.096552 ) {target="_ blank"})
187146
188- To add the relevant records, you click on ‘Search’, copy and paste the titles
147+ To add the relevant records, you click on ‘` Search ` ’, copy and paste the titles
189148of these relevant records one by one in the search bar and add them as
190149relevant.
191150
192- ![ ] ( https://raw.githubusercontent.com/asreview/asreview/main/docs/images/setup_prior_knowledge_random_validate. png) {width=95% }
151+ ![ ] ( images/ASReviewLAB/step_4_V1_6. png) {width=85% fig-align="center" }
193152
194153
195154After adding all five relevant records, you can add some irrelevant ones
196- by clicking the ‘Random’ button (use the arrow in the upper left corner
197- to be able to select this button) and by changing ‘relevant’ to
198- ‘irrelevant’. Select five irrelevant records and click on ‘CLOSE’.
155+ by clicking the ‘` Random ` ’ button (use the arrow in the upper left corner
156+ to be able to select this button) and by changing ‘` relevant ` ’ to
157+ ‘` irrelevant ` ’. Select five irrelevant records and click on ‘` CLOSE ` ’.
199158
200159* Have you selected five relevant and five irrelevant records? If so, you
201- can proceed to step 6 !*
160+ can proceed to step 5 !*
202161
203- ### Step 6 : Active learning model
162+ ### Step 5 : Active learning model
204163
205164The last step to complete the setup is to select the active learning
206165model you want to use. The default settings (i.e. Naïve Bayes, Max and
@@ -210,8 +169,8 @@ read which options are
210169or add your own model via a
211170[ template] ( https://github.com/asreview/template-extension-new-model ) {target="_ blank"}.
212171
213- You can click on ‘NEXT’. A menu with the defaults will appear. Since we
214- are using the defaults, you can click on ‘NEXT’ again. In the last step
172+ You can click on ‘` NEXT ` ’. A menu with the defaults will appear. Since we
173+ are using the defaults, you can click on ‘` NEXT ` ’ again. In the last step
215174of the setup, ASReview LAB runs the feature extractor, trains a model,
216175and ranks the records in your dataset. Depending on the model and the
217176size of your dataset, this can take a couple of minutes (meanwhile, you
@@ -220,13 +179,15 @@ Learning?](https://asreview.nl/blog/active-learning-explained/){target="_blank"}
220179
221180After the project is successfully initialized, you can start reviewing.
222181
223- ![ ] ( images/ASReviewLAB/Interface9.png ) {width=95%}
182+ ![ ] ( images/ASReviewLAB/step_5a_V1_6.png ) {width=85% fig-align="center"}
183+
184+ * Have you finished the setup? If so, you can proceed to step 6!*
224185
225- * Have you finished the setup? If so, you can proceed to step 7! *
186+ ![ ] ( images/ASReviewLAB/step_5b_V1_6.png ) {width=85% fig-align="center"}
226187
227188## Screening phase
228189
229- ### Step 7 : Screening the records
190+ ### Step 6 : Screening the records
230191
231192Everything is set up and ready to screen, well done!
232193
@@ -254,7 +215,7 @@ statistics and interpret the corresponding charts, check out the
254215[ documentation] ( https://asreview.readthedocs.io/en/latest/progress.html#analytics ) {target="_ blank"}
255216on the Analytics page.
256217
257- The Van de Schoot (2017) dataset contained 43 relevant records in this
218+ The Van de Schoot (2017) dataset contained 38 relevant records in this
258219particular example. Did you get to label all of them as relevant before
259220you reached your Stopping Rule? If you did, great!
260221
@@ -278,14 +239,14 @@ Rule. Read more about Stopping Rules and how to decide on a good
278239strategy for your data on the [ discussion
279240platform] ( https://github.com/asreview/asreview/discussions ) {target="_ blank"}.
280241
281- ### Step 8 : Extracting and inspecting the data
242+ ### Step 7 : Extracting and inspecting the data
282243
283244Now that you found all or most relevant records, you can export your
284245data using [ these
285246instructions] ( https://asreview.readthedocs.io/en/latest/progress.html#export-results ) {target="_ blank"}.
286247If you choose to inspect your data in Excel, download the data in
287- ‘Excel’ format. If you prefer to inspect your data in R, download the
288- ‘CSV (UTF-8)’ format and open it in R.
248+ ‘` Excel ` ’ format. If you prefer to inspect your data in R, download the
249+ ‘` CSV (UTF-8) ` ’ format and open it in R.
289250
290251You can find all the data that was originally imported to ASReview in
291252the exported data file, in a new order and with two new columns added at
@@ -302,7 +263,7 @@ the original ordering, do the included articles come from?
302263
303264For the last exercise, it is important to change the order back to the order
304265provided by ASReview. Lastly, check out the first few records with no number
305- in the ` included ` column. Are those articles labeled as ‘relevant’ in the
266+ in the ` included ` column. Are those articles labeled as ‘` relevant ` ’ in the
306267original dataset? (Whether or not a record was pre-labeled as relevant is
307268shown in the column ` label_included ` in the original dataset.)
308269
@@ -342,8 +303,8 @@ Some suggestions:
342303 code] ( https://github.com/asreview/asreview/tree/master/asreview ) {target="_ blank"} on
343304 Github… +1 for open-science!)
344305
306+ ![ ] ( images/ASReviewLAB/game_V1_6.png ) {width=85% fig-align="center"}
345307
346- ![ ] ( https://raw.githubusercontent.com/asreview/asreview/main/docs/images/game.png ) {width=95%}
347308
348309[ ^ 1 ] : Wang Z, Nayfeh T, Tetzlaff J, O’Blenis P, Murad MH (2020) Error
349310 rates of human reviewers during abstract screening in systematic
0 commit comments