> For the complete documentation index, see [llms.txt](https://fairsharing.gitbook.io/fairsharing/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://fairsharing.gitbook.io/fairsharing/fair-assistance/fair-support-and-definitions/fair-benchmark-institutional-repository-datasets.md).

# FAIR Benchmark - Institutional Repository Datasets

## **Introduction**

FAIRsharing is collaborating with several organisations, including UKRN (via the ORP, see [project page](https://www.ukrn.org/fair-institutional-repository-metrics-firm/)) and the University of Oxford (see our [news item](https://oerc.ox.ac.uk/news/oxford-leads-in-shaping-fair-evaluation-and-guidance-for-institutional-repositories)), to develop a FAIR benchmark for datasets held in institutional repositories. The benchmark has been created using the [OSTrails Assess-IF framework](https://docs.ostrails.eu/en/latest/). It uses a mixture of metrics specific to the needs of institutional repositories as well as reusing existing generic metrics, enabling communities to build upon established FAIR definitions wherever possible. This approach promotes consistency, comparability, and transparency, with all benchmark components, including metrics, tests, definitions, and justifications, being openly registered, discoverable, and reusable. Using this framework allows communities to explicitly define what FAIR means in their own context while remaining aligned with a broader ecosystem of shared FAIR assessment components.

The [Institutional Repository Datasets](https://fairsharing.org/7598) Benchmark provides a structured framework for assessing and improving the FAIRness of metadata describing research datasets deposited in institutional repositories. It operationalises the FAIR Principles in a practical and transparent manner, supporting alignment with community-endorsed standards and good research data management practices.

The benchmark is intended for two primary audiences. Institutional repository teams can implement and run the associated assessments as part of repository workflows, while researchers depositing datasets can use the resulting feedback to understand the FAIRness of their records and identify opportunities for improvement. By supporting both repository managers and depositors, the benchmark promotes FAIR awareness and enables targeted FAIRification actions.

Designed for use with publicly accessible repository records, the benchmark supports consistent FAIR assessment across institutional repositories and research disciplines. It distinguishes between FAIR-enabling properties implemented by the repository itself and FAIR characteristics that depend on the deposited record, while clearly identifying where existing metrics have been reused, adapted, or newly developed to address the specific needs of institutional repositories.

## Weighting and Compliance Measurement

This benchmark evaluates FAIRness through a collection of FAIR metrics aligned to the FAIR Principles. Each metric is implemented by one or more tests that return a **pass**, **fail** or **indeterminate** result. All metric results are retained and reported as part of the assessment record to ensure full transparency and traceability of the assessment process.

Each metric contributes to the principle-level assessment for the FAIR principle to which it belongs. A passing metric contributes the maximum available score for that metric, while a failing metric contributes no score. Metric scores are first combined to produce a weighted score for each FAIR principle. The relative contribution of individual metrics within a principle reflects the community priorities defined for this benchmark. The numerical scores and resulting compliance bands (see below) are not intended to function as a grade, ranking, or overall measure of dataset quality. T**he goal of the benchmark is to identify gaps in compliance by the repository and/or the submitter such that tailored guidance can be created.**

**Principle-level alignment** is calculated from the combined weighted results of all metrics associated with that principle and are expressed as a percentage of the maximum possible score for that principle. These principle-level scores are then combined to produce category scores for the four FAIR dimensions (Findable, Accessible, Interoperable and Reusable). Unlike earlier versions of the methodology, the principles within each FAIR dimension are not treated equally. Instead, each principle contributes according to a community-defined weighting that reflects its relative importance within the context of institutional repositories. This allows communities to express priorities while maintaining a transparent and reproducible scoring methodology.

An **overall FAIR** result is calculated as the average of the four normalised F, A, I and R category scores. This ensures that each FAIR dimension contributes equally to the overall assessment, while allowing different communities to assign different relative importance to the principles that comprise each FAIR dimension.

### Compliance Bands

For reporting and visualisation purposes, normalised scores are mapped onto four compliance bands: 25%, 50%, 75% and 100%. A score reaches a given band when the corresponding normalised score equals or exceeds that threshold. The 100% band is reserved for complete compliance and is only awarded when all metrics contributing to the relevant principle or FAIR category have passed.

These bands provide a simple and consistent representation of FAIR maturity across all principles and FAIR dimensions while preserving the underlying metric-level assessment results.

### Repository-Managed and Submitter-Managed Metrics

During development of the benchmark, metrics were classified into two categories:

* Repository-managed metrics, representing characteristics primarily determined by repository infrastructure, policies, or technical implementation.
* Submitter-managed metrics, representing characteristics that can be directly influenced by the individual creating or editing a metadata record.

This distinction was introduced to support future development of more targeted FAIR assessment and improvement workflows. For example, repository-managed and submitter-managed scores could be reported separately, enabling users to distinguish between issues that require repository-level intervention and those that can be addressed through metadata curation.

The current version of the benchmark records this classification but does not use it during score calculation, aggregation, or visualisation. The repository-managed versus submitter-managed classification is therefore retained as benchmark metadata for future use. A current listing of which tests are considered repository-managed and submitter-managed metrics is available in the "banding model" tab of the benchmark's [scoring algorithm](https://docs.google.com/spreadsheets/d/1Lbo0KzlN-RPRJ5POqHKh8YE2GXSv0bnaKZzwkaCOP9M/edit?usp=sharing).