Established author-level metrics — the h-index, the Journal Impact Factor and journal quartile rankings — are precisely the instruments the San Francisco Declaration on Research Assessment (DORA) cautions against using to judge individuals. They are field-blind, so scholars in low-citation disciplines such as education appear systematically weaker; and they reward citation volume in high-prestige journals while remaining blind to what the Scholarship of Teaching and Learning (SoTL) actually produces: open educational practice, partnership, and a sustained, public, peer-reviewed body of teaching scholarship. This paper describes the SoTL Index: a free, open, in-development instrument that places a scholar (or institution) on a 1–7 developmental spectrum derived from five transparent, field-normalised dimensions — reach, openness, SoTL focus, collaboration, and sustained contribution — computed entirely from open OpenAlex bibliographic data in the user's browser. We set out its conceptual grounding (Boyer, Glassick, DORA), its computation, a worked example, and its limitations. The Index is offered explicitly as a tool for reflection and conversation, not for ranking individuals.
Research assessment is dominated by citation-based author metrics, chief among them the h-index (Hirsch, 2005). These metrics carry three well-documented problems for the Scholarship of Teaching and Learning. First, they are field-blind. Citation densities differ by orders of magnitude across disciplines; a citation rate that is exceptional in education is unremarkable in molecular biology. Comparing raw counts across fields — as the h-index implicitly does — systematically disadvantages teaching-focused and social-science scholars. Second, they privilege journal prestige. The Journal Impact Factor and SCImago/Scopus quartiles measure the journal, not the work, yet are routinely read onto individuals. Third, they are blind to the outputs SoTL most values — open educational resources, students-as-partners work, sustained programmatic contribution, and dissemination into practice rather than into highly cited journals.
The San Francisco Declaration on Research Assessment (DORA, 2012), the Leiden Manifesto (Hicks et al., 2015) and the Hong Kong Principles (Moher et al., 2020) together call for assessment that is field-normalised, transparent, holistic, and focused on the merits of the work rather than the venue. The SoTL Index is a deliberate, modest attempt to operationalise those principles for teaching scholarship.
The Index draws on three traditions. Boyer's (1990) Scholarship Reconsidered establishes the scholarship of teaching as a legitimate form of scholarship alongside discovery, integration and application. Glassick, Huber and Maeroff's (1997) standards — clear goals, adequate preparation, appropriate methods, significant results, effective presentation, reflective critique — frame what quality teaching scholarship looks like. DORA and the responsible-metrics movement supply the assessment principles: field-normalise; value openness; never reduce a person to a single hidden number; assess the body of work, not the venue. The Index translates these into five visible dimensions and a developmental band rather than a precise score.
For a given scholar (identified by ORCID) or institution, the Index computes five dimension scores, each on a 0–100 scale, then forms a weighted composite and maps it to a 1–7 developmental spectrum (Emerging · Building · Developing · Establishing · Embedding · Leading · Transforming). A spectrum is used deliberately in place of a precise score: it communicates a developmental position without inviting the fine-grained ranking of individuals that DORA warns against. Every dimension is always displayed, and the user may re-weight any dimension live, so the composite can never operate as a single opaque figure.
All inputs are drawn from OpenAlex (Priem, Piwowar & Orr, 2022), a fully open index of scholarly works released under CC0, queried directly from the browser via its public API. No proprietary database (Scopus, Web of Science, Clarivate) is used, and no data is stored or transmitted beyond the OpenAlex request needed to fetch the public record. This keeps the instrument free, reproducible and independent of paywalled metric providers.
To make Reach comparable across disciplines, each journal is assigned a primary field (the modal field of its OpenAlex topics). A field factor is derived from a reference distribution of journal two-year mean citedness, sampled from ~7,000 active journals (filtered to type:journal, ≥100 works, non-zero recent citedness) and bucketed by primary field to obtain a median citedness p50field per field and a global median p50global:
A scholar working in a low-citation field (e.g. Social Sciences, p50 ≈ 0.19) receives an upward adjustment (factor ≈ 1.8), while a high-citation field (e.g. Medicine, p50 ≈ 0.51) is adjusted downward (factor ≈ 0.66). This is the same normalising logic SCImago applies within subject categories, but computed transparently from open data.
With n total outputs, c̄ mean citations per output, oa open-access outputs, j journal outputs, s SoTL-venue outputs, k distinct co-authors, and publication years spanning span with active distinct active years:
The reference constants (8 field-adjusted citations per output; 25 distinct collaborators; 6 active years for "maturity") are deliberately exposed in the tool and are intended to be calibrated with the sector rather than fixed by fiat.
Books, chapters, conference papers and repository deposits are counted toward openness, collaboration and sustained contribution but are excluded from the journal-based SoTL-focus share — and the tool states plainly that many of the most valuable SoTL outputs (books, open resources, datasets) carry no journal quartile at all.
For the author's own public record (ORCID 0000-0002-1428-2921; 36 outputs, 2013–2026), the Index returns Band 6 of 7 — "Leading", decomposed as: Reach 100 (5.6 citations/output × field factor 1.81 = 10.1 adjusted), Openness 58 (21/36 open access), SoTL focus 76 (13 of 17 journal outputs in teaching-and-learning venues), Collaboration 100 (60 distinct co-authors), and Sustained 93 (12 active years across a 14-year span). The decomposition is the point: the band is legible, but the five components — and the ability to re-weight them — keep the judgement open.
The SoTL Index runs free and login-free at ntlsn.com/sotl-index.html, entirely in the browser. The accompanying Responsible Research Assessment hub (ntlsn.com/responsible-assessment.html) curates the wider DORA toolkit. This methodology is released under CC BY 4.0; reuse and adaptation are encouraged with attribution.
Boyer, E. L. (1990). Scholarship Reconsidered: Priorities of the Professoriate. Carnegie Foundation for the Advancement of Teaching.
DORA. (2012). San Francisco Declaration on Research Assessment. https://sfdora.org/read/
Glassick, C. E., Huber, M. T., & Maeroff, G. I. (1997). Scholarship Assessed: Evaluation of the Professoriate. Jossey-Bass.
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. PNAS, 102(46), 16569–16572. https://doi.org/10.1073/pnas.0507655102
Hutchings, P., & Shulman, L. S. (1999). The scholarship of teaching: New elaborations, new developments. Change, 31(5), 10–15.
Moher, D., Bouter, L., Kleinert, S., et al. (2020). The Hong Kong Principles for assessing researchers. PLoS Biology, 18(7), e3000737. https://doi.org/10.1371/journal.pbio.3000737
Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. arXiv:2205.01833.