Test identification

Name of test Renaissance Star Reading
Version 3rd Edition
Previous version(s) 1st and 2nd Edition
Subjects Literacy
Summary Online assessment of reading growth, showing mastery of skills aligned to the National Curriculum.

Assessment screening

Subscales Single test covering word reading, vocabulary and comprehension
Authors Dr Damian W Betebenner
Publisher Renaissance Star Assessments
Test source http://www.renlearn.co.uk/star-reading/
Guidelines available? Yes
Norm-referenced scores. Yes
Age range 6-8 years
Key Stage(s) applicable to KS1
UK standardisation sample Yes
Publication date 2019
Re-norming date n/a

Eligibility

Validity measures available? Yes
Reliability measures available? Yes
Reason for exclusion from shortlist shortlisted

Evaluation and Appraisal

Additional information about what the test measures reading comprehension
Are additional versions available? This is the 3rd version of Star Reading. It is the first to be designed as a standards based test to align to UK national curriculum
Can subtests be administered in isolation? n/a
Administration group size small group, whole class
Administration duration 15-30 minutes
Description of materials needed to administer test computer or tablet
Any special testing conditions? no

Response format

Response mode Electronic
What device is required computer or tablet
Queston format. multiple choice
Progress through questions adaptive

Assessor requirements

Is any prior knowledge/training/profession accreditation required for administration? no
Is administration scripted? Yes

Scoring

Description of materials needed to score test automated scoring
Types and range of available scores Scaled scores (Rasch ability scale, 0-1400). Criterion referenced scores for subskills. Standardised score.Percentile rank. Reading age equivalent score. Estimated oral reading fluency (how many words can they read in a minute). Student growth percentile (provides a measure of how a student changed from one star testing to the next relative to other students with similar starting scores). Lexile measures (from BR400L to 1825L). Lexile score is used to equate books to a child's reading ability, e.g. in Accelerated Reader.
Score transformation for standard score Age standardised
Age bands used for norming 1 month
Scoring procedures computer scoring with direct entry by test taker/computer scoring with manual entry of responses from paper form/simple manual scoring key – clerical skills required
Automatised norming computerised

Construct Validity

Does it adequately measure literacy, mathematics or science?
Does it reflect the multidimensionality of the subject? Generic literacy
Construct validity comments (and reference for source) The technical manual indicates strong face validity (content validity) - constructs are based on UK national curriculum, TIMSS data, US curricular documents etc. Item Response Theory was used during development. Items that do not correlate well are excluded. Structural validity is indexed through exploratory and confirmatory factor analyses, which indicate excellent fit with single scale for years 7-13.Concurrent validity estimates with the Suffolk reading scale ranged from 0.74 to 0.89 with a median of 0.85 for the total scores, from 0.76 to 0.86 with a median of 0.81 for the standardised scores and from 0.77 to 0.89 with a median of 0.85 for the reading age scores. In the same study, the authors reported overall correlations of 0.83 across Years between Star Reading and teacher-created reading assessments.Discriminant validity is confirmed as correlations with Star Maths are generally lower than correlations with other reading measures (e.g. Suffolk Reading Scale), but they are still moderately high, ranging from 0.58 to 0.68

Criterion Validity

Does test performance adequately correlate with later, current or past performance?
Summarise available comparisons Star reading predicts which children will reach expected levels on their key stage 2 reading test with 79% accuracy in a sample of 12,212 pupils. Sensitivity was 0.84, specificity was 0.70. PRediction accuracy for KS2 SPAG test was 82%. Several US studies looking at predicting readiness for college show similar levels of accuracy.

Reliability

Is test performance reliable?
Summarise available comparisons The technical manual reports multiple measures of reliability with very large UK samples. Internal (Generic) reliability calculated by using conditional standard error of measurement statistics. Estimates ranged from 0.94 for year 1 same to 0.97 for year 13. Split half reliability estimates showed a range between 0.93 for year 5 children and 0.97 for years 12 and 13. Correlations between two different forms of the test (note questions are different every time so this reflect equivalence reliability more than test-retest reliability) taken between 74 and 125 days apart. Year 1: 0.87; year 2: 0.85; year 3 0.84; year 4 0.86; year 5 0.86; year 6 0.86; year 7 0.85; year 8 0.86; year 9 0.86; year 10 0.87; year 11 0.87; year 12 0.90; year 13 0.89. total 0.90.IRT testing with children in the US stratified to be close to a nationally representative sample. Children took 44 item tests. 2100 items were tested. Items with item correlation with overall score <.30 were discarded. So were items too easy or too difficult, or not showing a Rasch curve. Also items where a distracter showed a positive discrimination.

Is the norm-derived population appropriate and free from bias?

Is population appropriate and free from bias? No
If any biases are noted in sampling, these will be indicated here. oversampling from disadvantaged schools and from England rather than Scotland or Wales.

Sources

Sources Renaissance. (2019). Star Assessments for Reading: Technical Manual. London, UK: Renaissance Learning, Inc.Sewell, J., Sainsbury, M., Pyle, K., Keogh, N., & Styles, B. (2007). Renaissance Learning Equating Study. Report.