NRT (Naplan Reporting Tool) is a command-line utility that processes results data from the NAPLAN online testing platform.
NRT takes one or more results-reporting-dataset (often called RRD) xml data files generated by the online platform, and processes them to produce a variety of csv reports that are easier for most users to work with.
There are some 60+ reports available from NRT, and the scope of reports that are run can be configured by the user, either by the use of command-line flags passed to the nrt application, or by editing the report configuration files that can be found in the /config folder of the distribution.
NRT is intended as a direct replacement for the existing nias2 naprrql application.
NRT aims to process results data faster than naprrql, and depending on configuration and input volume typically performs 6x to 50x faster.
Download a pre-built binary distribution for your platform (Windows, Linux, Mac), from the releases area of this repository:
https://github.com/nsip/dev-nrt/releases/latest
Unpack the downloaded zip file, and you should see a folder structure like this:
/NRT-Mac-v0_0_1
nrt(.exe)
├── config
│ └── internal
└── testdata
in the folder is
- the nrt executable (nrt.exe if on windows),
- a subfolder called /config that contains the configuration files for each report produced by the tool
- a subfolder called /testdata that has two sample zipfiles of results data
- n2sif.xml.zip; is the same sample file as accompanies nias2 naprrql
- rrd.xml.zip; a much larger results file (100 schools, 5k students)
nrt expects results data files that it will process to be placed into an /in folder.
You can either create the folder yourself, or do the following:
Open a terminal window at the folder containing your nrt distribution.
Run nrt for the first time as
> ./nrt
for Linux/Mac or
> nrt.exe
on Windows
this will produce the following message:
--- Ingesting results data:
2021/02/08 17:06:22 No results data *.zip *.xml.zip or *.xml files found in input folder in
but the folder layout of your installation will now look like this:
/NRT-Mac-v0_0_1
nrt(.exe)
├── config
│ └── internal
├── in
├── kv
└── testdata
the /in folder has been created as has a /kv folder the app will use internally.
Copy either of the xml.zip files from the testdata area to the /in folder in order to have them processed by nrt.
(nrt will process multiple files so you can copy both if you wish to, it just means processing will take longer).
nrt does read zipped xml files, or you can unzip them (if you do so remember to delete the zipped version or nrt will process both). However, nrt is not able to open password-protected zip files.
now if you invoke nrt again, it will ingest and process the data file/s.
typical output as follows:
| => ./nrt
--- Ingesting results data:
Processing XML File: (in/n2sif.xml.zip)
0s [===================================================================>] 100%
5190 data-objects parsed
2021/02/08 17:11:56 Results data ingest took 1.038s
--- Data stats:
NAPStudentResponseSet: 1061
NAPTestScoreSummary: 154
NAPTestItem: 2310
NAPCodeFrame: 22
SchoolInfo: 10
StudentPersonal: 250
NAPTestlet: 174
NAPTest: 22
NAPEventStudentLink: 1187
--- Running Reports (-QA reports):
Simple reports: 0s [====================================================================] 100%
Codeframe reports: 0s [====================================================================] 100%
Event-based reports: 1s [====================================================================] 100%
Student-based reports: 1s [====================================================================] 100%
2021/02/08 17:11:59 Report processing took 2.642s
______________________
First nrt ingests the data, and presents a set of summary statistics, then it proceeds to run the core report set, which are the most commonly used.
The indicator '(-QA reports)' shows that qa reports are not included in this set, this changes to (+QA reports) if qa reports have been requested by the user.
All generated reports are output to the /out folder which nrt creates. The folder structure after a full run of data ingest, and all reports would look like this:
| =>
/NRT-Mac-v0_0_1
nrt(.exe)
├── config
│ └── internal
├── in
├── kv
├── out
│ ├── item_printing
│ ├── qa
│ ├── system_reports
│ └── writing_extract
└── testdata
A description of the various command-line options to configure the reporting output of nrt is available at any time by invoking ./nrt --help:
| => ./nrt --help
Usage of ./nrt:
-allReports
runs every report including qa reports
-coreReports
run core naplan reports (on by default) - (alias for n2 -reports flag)
-forceIngest
always ingest data from files in the input folder (default true)
-ingest
halt processing after data has been ingested
-inputFolder string
folder containing results data files for processing (default "./in/")
-itemLevel
include item-level reports - very detailed (alias for n2 -itemprint flag)
-itemprint
include item-level reports - very detailed (alias for nrt -itemLevel flag)
-qa
include QA reports in results processing
-report
run core naplan reports - (alias for nrt -coreReports flag)
-showProgress
show progress bars in console for report processing (default true)
-skipIngest
don't ingest data again, move directly to processing of reports
-writingExtract
(alias) run writing extract reports; generates writing inputs for marking systems
-writingextract
run writing extract reports; generates writing inputs for marking systems
The -itemLevel/-itemprint flags (aliases of each other, for compatibility with naprrql command-line) are of particular note.
These flags invoke the most performance-heavy reports in the collection, they are output to the out/item_printing folder.
These reports are at the highest level of granularity, producing very large numbers of rows (event-based) and columns (student-based). Event-based generates a row for every item seen by every student in every testlet in every test. Student-based creates a row with columns per student containing personal information, event information, every and every test, testlet and item seen by the student, typically around 1400 columns for each student.
These reports were created through user demand, but if not explicitly needed, performance is far better without them.
Command-line flags can be used together in whatever combination helps most.
The -writingExtract/-writingextract flags if used on their own will result in only the writing extract reports being run. These are typically needed before the full rrd dataset is produced as they export writing results for ingest into external marking systems.