Metadata Standards and Format Interoperability

  • Bioinformatics research generates large volumes of biological data from genome sequencing, transcriptomics, proteomics, metabolomics, structural biology, microscopy, clinical studies, and other experimental approaches. However, biological data are useful only when researchers can understand what they represent, how they were generated, which samples they describe, and how they relate to other datasets. Metadata provide this essential context. Metadata standards and format interoperability enable biological information to be described consistently, exchanged between different software systems, integrated across databases, and reused in independent research projects. They are fundamental to data sharing, reproducibility, collaborative research, and large-scale computational biology.
  • Metadata are structured information that describe, explain, or provide context for a dataset. In bioinformatics, metadata may describe the biological sample, organism, tissue, cell type, experimental conditions, sequencing platform, laboratory protocol, reference genome, data-processing software, analytical parameters, and data-generation date. For example, a gene expression matrix contains numerical measurements, but without metadata researchers may not know which samples were analyzed, which organism was studied, whether the values represent raw counts or normalized expression, or which experimental conditions were applied. Metadata make biological measurements interpretable and provide the information required to compare and combine datasets.
  • Metadata can describe several different aspects of a biological study. Descriptive metadata identify and summarize a dataset through its title, description, authors, organism, keywords, and research purpose. Structural metadata describe how information is organized, including file structures, column definitions, relationships between records, and data types. Technical metadata describe the instruments, sequencing platforms, software, protocols, and computational parameters used to generate or process data. Administrative metadata include information about data ownership, access permissions, licensing, funding, and submission details. Provenance metadata document the origin of data and the transformations performed during analysis. Biological metadata describe the experimental and biological context, such as tissue type, developmental stage, disease condition, genotype, and treatment.
  • In a typical RNA sequencing study, metadata may include the sample identifier, organism, tissue, experimental group, treatment, biological replicate, sequencing platform, library preparation method, read length, reference genome assembly, and software used for analysis. A sample metadata table might contain the following fields:

Sample_ID Organism Tissue Condition Replicate Platform

S01 Homo sapiens Liver Control 1 Illumina

S02 Homo sapiens Liver Control 2 Illumina

S03 Homo sapiens Liver Treatment 1 Illumina

S04 Homo sapiens Liver Treatment 2 Illumina

  • This table provides basic information about the samples, but a real study may require additional details, including age, sex where relevant, sample collection method, storage conditions, extraction protocol, library preparation, sequencing instrument model, and data-processing parameters. Metadata should be sufficiently detailed to support interpretation and reuse without unnecessarily collecting sensitive personal information.
  • Metadata standards are agreed frameworks that define how information should be described, organized, named, and exchanged. They may specify required fields, permitted values, data types, controlled vocabularies, identifiers, relationships, and validation rules. Standards help prevent different research groups from describing the same biological concept in incompatible ways. For example, one laboratory might describe a sample as “liver tissue,” another as “hepatic tissue,” and a third as “liver.” Although these expressions may refer to the same biological material, inconsistent terminology makes automated searching and integration difficult. Standardized terminology and ontology identifiers help resolve such differences.
  • A metadata schema defines the structure of a metadata record. It specifies the fields that may or must be included, their data types, and their relationships. A schema might require each biological sample to have a unique identifier, organism name, tissue description, and experimental condition. It may also define which fields are optional and which values are permitted. Metadata schemas can be implemented using JSON Schema, XML Schema Definition (XSD), relational database structures, or other validation mechanisms. A well-defined schema helps ensure that different applications interpret metadata consistently.
  • Controlled vocabularies and biological ontologies are important components of metadata standardization. A controlled vocabulary provides an agreed set of terms for describing a particular domain. An ontology goes further by defining concepts and relationships between them. For example, the Gene Ontology (GO) describes molecular functions, biological processes, and cellular components. The Cell Ontology provides standardized descriptions of cell types, while the Uberon ontology represents anatomical structures across species. The Human Phenotype Ontology (HPO) describes human phenotypic abnormalities, and the Disease Ontology provides structured disease terminology. These resources allow researchers to annotate biological information using consistent concepts rather than relying exclusively on free-text descriptions.
  • Ontology terms often have persistent identifiers that can be used across databases and software systems. For example, a tissue annotation can be associated with an ontology accession rather than being represented only by a text label. This improves consistency even when the preferred display name changes or alternative synonyms are used. Ontology-based annotation also enables more advanced queries, such as retrieving samples associated with a particular tissue and its related anatomical concepts. However, selecting an ontology term requires biological judgment. An incorrect ontology mapping can introduce misleading information even when the identifier is technically valid.
  • Several international initiatives and community standards support metadata consistency in bioinformatics. The International Nucleotide Sequence Database Collaboration (INSDC), which includes GenBank, the European Nucleotide Archive (ENA), and the DNA Data Bank of Japan (DDBJ), provides shared standards and processes for exchanging nucleotide sequence records and associated metadata. The INSDC framework supports consistent sequence submission and cross-database exchange, helping researchers access nucleotide records through multiple international resources.
  • NCBI BioProject and BioSample provide important structures for organizing biological research data. BioProject describes a larger research project, while BioSample describes the biological material from which data were generated. Sequencing experiments and associated data can be connected to these records through resources such as the Sequence Read Archive (SRA). These relationships help connect raw sequencing data, sample descriptions, and the wider research project. Similar submission and metadata structures are available through the European Nucleotide Archive and other international repositories.
  • The Minimum Information about a Microarray Experiment (MIAME) standard was developed to improve the description and reporting of microarray experiments. It specifies the information needed to interpret and evaluate microarray data, including experimental design, samples, hybridization procedures, array design, data processing, and raw and processed data. Although microarray technology is less dominant than it once was, MIAME remains an important example of how community-defined minimum information requirements can improve the quality and reproducibility of experimental data.
  • For high-throughput sequencing, the Minimum Information about a High-Throughput Nucleotide Sequencing Experiment (MINSEQE) provides guidance on the information needed to describe sequencing experiments. It addresses experimental design, samples, sequencing procedures, data processing, and the availability of raw and processed data. MINSEQE helps researchers provide sufficient information for independent interpretation and evaluation of sequencing results. Related community recommendations and repository-specific submission requirements may also apply depending on the type of experiment.
  • The Minimum Information about any (x) Sequence (MIxS) framework, developed through the Genomic Standards Consortium, provides checklists for describing genome, metagenome, and related sequence datasets. MIxS includes general information and extensions for different environments, organisms, and sequencing applications. It is particularly important in microbial genomics, environmental sequencing, metagenomics, and biodiversity research, where sample origin and environmental conditions can strongly influence biological interpretation. MIxS checklists help standardize information such as geographic location, environmental material, collection conditions, and sequencing methods.
  • The Investigation-Study-Assay (ISA) framework provides a structured approach for describing experimental investigations and their associated studies and assays. An investigation represents the overall research project, a study describes a specific experimental component, and an assay represents a particular measurement or analytical procedure. ISA supports the organization of experimental metadata across different biological domains, including genomics, transcriptomics, proteomics, and metabolomics. The ISA model is particularly useful when a project contains multiple experiments, technologies, samples, and data-processing stages. It helps preserve relationships between experimental design, biological materials, analytical methods, and resulting datasets.
  • Proteomics research also uses specialized metadata standards. The Proteomics Standards Initiative (PSI), associated with the Human Proteome Organization, develops community standards for proteomics data and metadata. Standards such as mzML describe mass spectrometry data, mzIdentML represents peptide and protein identification results, and mzTab provides tabular summaries of proteomics and metabolomics results. Controlled vocabularies and metadata conventions help describe instruments, experimental parameters, modifications, identification evidence, and quantitative measurements. These standards improve interoperability between mass spectrometry instruments, analytical software, and public repositories such as PRIDE.
  • Imaging and spatial biology also require specialized metadata standards. The Open Microscopy Environment (OME) develops data models and formats for microscopy metadata and image data. OME-TIFF and OME-Zarr support the organization and exchange of multidimensional imaging information, including image dimensions, channels, spatial coordinates, and acquisition metadata. These standards are increasingly relevant to spatial transcriptomics and imaging-based omics, where molecular measurements must be interpreted alongside tissue images and spatial organization.
  • Biodiversity and ecological genomics use additional metadata standards. Darwin Core is a widely used vocabulary for sharing information about biological specimens, species occurrences, collections, and biodiversity observations. It supports consistent descriptions of taxonomic names, collection events, locations, institutions, and specimen records. It is useful when connecting genomic information with biodiversity collections, environmental observations, and species-distribution research. Darwin Core is not a general replacement for sequencing-specific metadata standards, but it provides important terminology for describing organisms and their occurrence records.
  • Metadata interoperability depends on more than choosing a standard. Different databases may use different field names, identifiers, data types, and terminology for similar biological concepts. One database may use organism, another may use scientific_name, and another may use a taxonomic identifier. Similarly, a sample may be described using a laboratory-specific code in one system and a repository accession in another. Interoperability requires mappings between these representations, together with clear rules about which information is equivalent and which differences must be preserved.
  • Data interoperability refers to the ability of different systems, tools, and organizations to exchange information and use it meaningfully. Syntactic interoperability means that systems can read and process the same data representation, such as JSON, XML, TSV, or a defined binary format. Structural interoperability means that they understand the organization of fields, records, and relationships. Semantic interoperability means that they interpret the meaning of the information consistently. For example, two systems may successfully exchange a field called expression_value, but meaningful comparison also requires knowing whether the value represents raw counts, transcripts per million (TPM), fragments per kilobase per million (FPKM), or another measurement.
  • Interoperability also requires consistent identifiers. Biological databases assign accession numbers and persistent identifiers to genes, proteins, sequences, samples, studies, structures, and other records. These identifiers allow records to be referenced across different resources. For example, a gene record may contain an Ensembl gene identifier, an NCBI Gene identifier, and links to UniProt protein accessions. Cross-references connect these resources, allowing researchers to retrieve complementary information without relying exclusively on text-based name matching. Identifier mapping must be performed carefully because identifiers may be retired, merged, split, or associated with different biological entities across database releases.
  • Persistent identifiers are especially important for long-term data sharing. Examples include DOI identifiers for publications and datasets, accession numbers for sequence and structural records, and stable identifiers used by biological databases and ontologies. A persistent identifier provides a reliable reference to a resource, although the underlying record may be updated over time. Researchers should record the relevant database release, accession version where available, and retrieval date when these details affect reproducibility.
  • Application programming interfaces (APIs) provide another important mechanism for metadata interoperability. APIs allow software applications to retrieve and exchange structured information from biological databases without manually downloading individual records. REST APIs commonly use JSON or XML responses, while other services may use additional protocols and formats. For example, NCBI E-utilities provide programmatic access to Entrez resources, and Ensembl and UniProt provide services for retrieving gene, sequence, and protein information. APIs can support automated metadata retrieval, identifier mapping, and integration of information from multiple databases. However, applications must account for differences in field names, response structures, authentication requirements, rate limits, and versioning policies.
  • The following JSON example illustrates a simple metadata record for a sequencing sample. It uses descriptive field names and demonstrates how controlled terms and identifiers can be included alongside ordinary values. It is an illustrative structure rather than a formal implementation of a particular repository standard.

{

  “sample_id”: “SAMPLE_001”,

  “organism”: {

    “scientific_name”: “Homo sapiens”,

    “taxonomy_id”: “9606”

  },

  “sample_type”: {

    “label”: “liver tissue”,

    “ontology_id”: “ONTOLOGY_ACCESSION”

  },

  “experiment”: {

    “technology”: “RNA sequencing”,

    “platform”: “Illumina”,

    “condition”: “control”,

    “replicate”: 1

  },

  “data”: {

    “raw_file”: “sample_R1.fastq.gz”,

    “format”: “FASTQ”,

    “reference_assembly”: “GRCh38”

  }

}

  • Metadata can be validated automatically using schemas. JSON Schema, for example, can define required properties, data types, and permitted values for JSON records. XML Schema can provide similar validation for XML documents. Repository submission systems may also apply their own validation rules and controlled vocabularies. These checks help identify missing fields, invalid data types, and inconsistent metadata before records are deposited or used in an analysis pipeline.
  • The following simplified JSON Schema illustrates how a sample metadata record could be validated. It is an example created for this article and is not an official repository submission schema.

{

  “$schema”: “https://json-schema.org/draft/2020-12/schema”,

  “type”: “object”,

  “required”: [

    “sample_id”,

    “organism”,

    “experiment”

  ],

  “properties”: {

    “sample_id”: {

      “type”: “string”,

      “minLength”: 1

    },

    “organism”: {

      “type”: “object”,

      “required”: [“scientific_name”, “taxonomy_id”],

      “properties”: {

        “scientific_name”: {

          “type”: “string”

        },

        “taxonomy_id”: {

          “type”: “string”

        }

      }

    },

    “experiment”: {

      “type”: “object”,

      “required”: [“technology”],

      “properties”: {

        “technology”: {

          “type”: “string”

        },

        “platform”: {

          “type”: “string”

        },

        “replicate”: {

          “type”: “integer”,

          “minimum”: 1

        }

      }

    }

  }

}

  • Python libraries can be used to validate metadata records against such schemas. For example, the jsonschema package can check whether a JSON object follows a specified JSON Schema.

import json

from jsonschema import validate

with open(“sample_metadata.json”, “r”, encoding=”utf-8″) as file:

    metadata = json.load(file)

with open(“metadata_schema.json”, “r”, encoding=”utf-8″) as file:

    schema = json.load(file)

validate(instance=metadata, schema=schema)

print(“Metadata structure is valid.”)

  • This process checks structural compliance with the supplied schema. It does not establish that the scientific information is correct. For example, a record may contain a valid taxonomy identifier that does not correspond to the actual organism in the sample. Biological validation should therefore include checks against authoritative databases, controlled vocabularies, and experimental records.
  • Format interoperability is also important when moving data between analytical environments. A study may generate raw sequencing files in FASTQ format, alignment records in BAM or CRAM, variant results in VCF, gene annotations in GFF3 or GTF, and sample descriptions in CSV or JSON. Different tools may require different representations, so conversion and metadata mapping are often necessary. However, converting a file does not automatically preserve all metadata. A CSV export may retain sample names and measurements but lose hierarchical relationships, ontology identifiers, or processing history. Similarly, conversion from a specialized omics container to a simple matrix may discard information about layers, cell annotations, spatial coordinates, or analysis results.
  • Successful interoperability therefore requires explicit mapping between data models. A mapping identifies how fields in one system correspond to fields in another. It should document differences in names, data types, units, coordinate systems, identifier conventions, and permitted values. For example, a field called sample_name in one repository may correspond to sample_id in another, but the two fields may not necessarily have identical meanings. Mapping should be based on documented definitions rather than superficial similarity between field names.
  • Metadata integration can also involve data harmonization. Harmonization is the process of making information from different sources more comparable by standardizing terminology, units, identifiers, and representations. In a multi-study gene expression project, researchers may need to harmonize tissue labels, organism identifiers, disease classifications, sequencing platforms, and experimental conditions. However, harmonization should not erase meaningful biological differences. For example, two samples described broadly as “brain tissue” may originate from different anatomical regions and should not automatically be treated as equivalent.
  • Units and measurement conventions are particularly important. A numerical value without a unit or measurement definition can be difficult to interpret. Gene expression values may be reported as raw counts, TPM, FPKM, or normalized values. Protein concentrations may be expressed in different mass or molar units, and genomic positions may follow different coordinate conventions. Metadata standards should describe these measurement properties explicitly. When converting units or coordinate systems, the transformation should be documented and validated.
  • Metadata interoperability is closely related to the FAIR principles: Findability, Accessibility, Interoperability, and Reusability. These principles provide guidance for making research data easier to discover, access, integrate, and reuse. Findability requires descriptive metadata and persistent identifiers. Accessibility requires clear information about how data can be retrieved and under what conditions. Interoperability requires the use of shared formats, vocabularies, and standards. Reusability requires sufficient documentation, provenance, licensing information, and information about how the data were generated. FAIR does not require that every dataset be publicly available without restrictions; sensitive data may require controlled access while still being described using FAIR-oriented metadata.
  • Data provenance is another essential component of metadata interoperability. Provenance records describe the origin of a dataset and the steps through which it has been transformed. In a sequencing project, provenance may include the original sample, extraction protocol, sequencing instrument, reference genome, software versions, command-line parameters, and quality-control procedures. Workflow systems can record these details automatically, while metadata repositories and structured analysis reports can preserve them for later review. Provenance helps researchers distinguish original measurements from processed results and reproduce the steps that generated a particular dataset.
  • Reproducibility also depends on version management. Biological databases and ontologies are updated regularly, and annotations may change as scientific knowledge develops. A gene may receive a revised annotation, a taxonomy record may be updated, or an ontology term may be replaced or deprecated. If a study uses metadata from a particular database release, that release should be recorded whenever possible. Software versions, schema versions, and retrieval dates should also be documented. Without version information, an analysis may be difficult to reproduce because the same query can return different results at a later date.
  • Metadata quality remains a major challenge in bioinformatics. Common problems include missing values, inconsistent terminology, duplicate sample identifiers, ambiguous descriptions, incorrect organism names, incomplete experimental information, and inconsistent reference genome versions. Metadata may also contain errors introduced during manual entry, spreadsheet editing, file conversion, or database submission. These problems can affect downstream analysis and make datasets difficult to compare. Quality-control procedures should include required-field checks, controlled-vocabulary validation, identifier verification, duplicate detection, consistency checks, and review by researchers familiar with the experimental system.
  • Automated metadata validation should be combined with human review. Automated tools can identify missing fields, incorrect formats, invalid identifiers, and values outside permitted ranges. However, they may not recognize whether a tissue description, experimental condition, or biological interpretation is scientifically appropriate. Researchers should therefore review metadata for biological meaning, not only technical correctness. Clear documentation and communication between experimental scientists, bioinformaticians, data managers, and repository administrators are important for maintaining high-quality metadata.
  • Metadata standards and format interoperability are essential for integrating biological information across databases, technologies, and research institutions. Standards such as MIAME, MINSEQE, MIxS, ISA, PSI, OME, and Darwin Core address different scientific domains and provide guidance for describing experimental data. Controlled vocabularies, ontologies, persistent identifiers, schemas, APIs, and provenance records further improve consistency and reuse. By applying appropriate standards and documenting transformations, researchers can make biological datasets more understandable, interoperable, reproducible, and valuable for future scientific discovery.
Author:

Leave a Reply

Your email address will not be published. Required fields are marked *