physionet b2ai-voice 1.1 d4d

Datasheet for Dataset - Human Readable Format

🔍

Collection Process

How was the data acquired?

bridge2ai-voice
Bridge2AI-Voice
Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information
Bridge2AI-Voice is a comprehensive, ethically-sourced collection of data derived from voice recordings linked to clinical information, intended to enable artificial intelligence research into voice as a biomarker of health. Version 1.1 provides 12,523 recordings for 306 participants collected across five sites in North America, focusing on cohorts with conditions known to manifest in the voice (voice disorders, neurological disorders, mood disorders, and respiratory disorders). This release contains low-risk derived data (e.g., spectrograms, MFCCs, static acoustic/phonetic/prosodic features) and associated phenotype data; raw audio is not included and is available only via controlled access to protect participant privacy.
English
https://docs.b2ai-voice.org
https://physionet.org
  • voice
  • speech
  • biomarker
  • health
  • Bridge2AI
  • spectrograms
  • MFCC
  • acoustic features
  • phenotypes
  • clinical data
  • Alistair Johnson
  • Jean-Christophe Bélisle-Pipon
  • David Dorr
  • Satrajit Ghosh
  • Philip Payne
  • Maria Powell
  • Anais Rameau
  • Vardit Ravitsky
  • Alexandros Sigaras
  • Olivier Elemento
  • Yael Bensoussan
  1. ID
    bridge2ai-voice-v1.1
    Name
    Bridge2AI-Voice v1.1
    Title
    Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information (v1.1)
    Description
    Version 1.1 of Bridge2AI-Voice includes derived voice data (spectrograms, MFCCs, static acoustic/phonetic/prosodic features) and linked phenotype data for an adult cohort. It excludes raw audio waveforms. Files listed for this version include: spectrograms.parquet, mfcc.parquet, static_features.tsv (+ JSON data dictionary), phenotype.tsv (+ JSON data dictionary). Access is restricted to registered users under a data use agreement. Raw audio may be requested via controlled access.
    Language
    English
    Page
    https://doi.org/10.13026/249v-w155
    DOI
    doi:10.13026/249v-w155
    Issued
    2025-01-17
    Version
    1.1
    License
    Bridge2AI Voice Registered Access License
    Keywords
    • voice
    • speech
    • biomarker
    • health
    • Bridge2AI
    • spectrograms
    • MFCC
    • acoustic features
    • phenotypes
    • clinical data
    Is Tabular
    mixed (Parquet, TSV, JSON)
    Purposes
    • Name
      Purpose
      Response
      Create an ethically-sourced flagship dataset to enable AI research on voice as a biomarker of health and support critical clinical insights.
    Tasks
    • Name
      Task
      Response
      Voice-based biomarker discovery, disease classification/screening, and analysis of acoustic/phonetic/prosodic features linked to health.
    Addressing Gaps
    • Name
      AddressingGap
      Response
      Provide a large, multi-institutional, diverse voice dataset linked to health information with standardized collection protocols and explicit ethical oversight, addressing limitations of prior small and demographically limited datasets.
    Funders
    GrantorGrant NameGrant Number
    National Institutes of Health (NIH)Bridge2AI: Voice as a Biomarker of Health - Building an ethically sourced, bioacoustic database to understand disease like never before3OT2OD032720-01S1
    National Institute of Biomedical Imaging and Bioengineering (NIBIB), NIHPhysioNet infrastructure supportR01EB030362
    Instances
    CountsData TypeInstance TypeLabelNameRepresentationSampling Strategies
    12523Derived features from audio (spectrograms: 513xN; MFCC: 60xN; static acoustic/phonetic/prosodic feat...Multiple instance types: recordings (sessions/tasks) with derived data and per-participant phenotype...No explicit supervised label provided; clinical/phenotype variables availableVoice-derived data instancesDerived voice data per recording session (e.g., spectrograms, MFCCs, static features){'name': 'Cohort sampling strategy', 'strategies': ['Targeted enrollment of patients in five predetermined clinical cohorts (respiratory, voice, neurological, mood, pediatric; adult cohort available in v1.1)'], 'is_sample': ['yes (targeted clinical cohorts)'], 'is_random': [False], 'source_data': ['Patients presenting at specialty clinics across five sites in North America'], 'is_representative': ['No (targeted cohorts; not representative of general population)'], 'why_not_representative': ['Enrollment focused on diseases with recognized vocal manifestations and unmet needs, to fuel disease-relevant voice AI research']}
    306Tabular phenotype data (phenotype.tsv with data dictionary phenotype.json)Participants with demographics, questionnaires, and clinical phenotype dataParticipantsStudy participants (adult cohort in v1.1)
    External Resources
    1. Name
      Raw audio access
      External Resources
      • Original raw audio available only via controlled access request (contact: DACO@b2ai-voice.org)
      Future Guarantees
      • Controlled access to protect participant privacy
      Archival
      • Derived datasets (spectrograms, MFCCs, features, phenotypes) are archived with DOI at PhysioNet
    Confidential Elements
    • Name
      Confidentiality
      Description
      • No raw audio recordings released in v1.1; dataset contains only derived features and de-identified phenotype data to reduce risk
    Content Warnings
    • Name
      Content warnings
      Warnings
      • None noted
    Subpopulations
    • Name
      Adult cohort
      Identification
      • Adult participants (v1.1 includes adult cohort only)
      Distribution
      • 306 participants across five North American sites; disease-focused cohorts (respiratory, voice, neurological, mood; pediatric planned)
    Sensitive Elements
    • Name
      Sensitive elements
      Description
      • Clinical/health-related phenotype data; identifiers removed per HIPAA Safe Harbor and additional de-identification
    Acquisition Methods
    1. Name
      Instance acquisition
      Description
      • Standardized, protocol-driven data collection using a custom tablet application; headset used when possible; demographic and clinical questionnaires; targeted tasks including sustained vowel phonation
      Was Directly Observed
      yes (audio tasks recorded)
      Was Reported By Subjects
      yes (questionnaires and targeted confounders)
      Was Inferred Derived
      yes (features, spectrograms, MFCCs, prosodic/phonetic metrics, transcriptions)
      Was Validated Verified
      Standardized multi-site protocol; preprocessing with defined parameters
    Collection Mechanisms
    • Name
      Collection mechanisms
      Description
      • Custom application on a tablet; headset microphone when possible; REDCap-based data export and conversion using an open-source library
    Data Collectors
    • Name
      Data collectors
      Description
      • Project investigators at specialty clinics across five North American sites screened and enrolled patients, obtained consent, and conducted standardized data collection sessions
    Ethical Reviews
    • Name
      IRB approval
      Description
      • Data collection and sharing approved by the University of South Florida Institutional Review Board
    Preprocessing Strategies
    • Name
      Audio preprocessing and feature extraction
      Description
      • Raw audio converted to mono and resampled to 16 kHz with a Butterworth anti-aliasing filter
      • Spectrograms via STFT (25 ms window, 10 ms hop, 512-point FFT; 513 x N)
      • 60 MFCCs derived from spectrograms (60 x N)
      • Acoustic features via openSMILE capturing temporal dynamics and acoustic characteristics
      • Phonetic and prosodic features via Parselmouth and Praat (e.g., F0, formants, voice quality)
      • Transcriptions generated using OpenAI Whisper Large (used for derivations; free-speech transcripts removed)
      Used Software
      IDNameURL
      openSMILEopenSMILEhttps://audeering.github.io/opensmile/
      praatPraathttps://www.fon.hum.uva.nl/praat/
      parselmouthParselmouthhttps://parselmouth.readthedocs.io/
      torchaudioTorchaudiohttps://pytorch.org/audio
      whisper-largeOpenAI Whisper Largehttps://github.com/openai/whisper
    Cleaning Strategies
    • Name
      De-identification and content removal
      Description
      • HIPAA Safe Harbor identifiers removed (e.g., names, finer-than-year dates, contact numbers, IPs, MRNs, device IDs, URLs, images, etc.); state/province removed; country retained
      • Transcripts of free speech audio removed
      • Audio waveforms omitted from release; only derived data (spectrograms and other features) provided in v1.1
    Raw Sources
    • Name
      Raw audio data
      Description
      • Original audio waveforms exist but are not distributed in v1.1; may be requested via controlled access by contacting DACO@b2ai-voice.org
    Future Use Impacts
    • Name
      Considerations for future use
      Description
      • Use of derived features only (no raw audio) reduces re-identification risk but may limit some modeling approaches; cohort-targeted sampling may impact generalizability; users should consider potential bias and fairness implications
    Discouraged Uses
    • Name
      Discouraged uses
      Description
      • Uses that attempt to re-identify individuals or reconstruct content beyond the scope of the derived features; any uses outside the Data Use Agreement and Registered Access License
    Distribution Formats
    • Name
      Formats
      Description
      • Parquet (spectrograms.parquet, mfcc.parquet)
      • TSV (phenotype.tsv, static_features.tsv)
      • JSON (phenotype.json, static_features.json)
    Distribution Dates
    • Name
      Initial distribution (v1.1)
      Description
      • 2025-01-17
    License And Use Terms
    Name
    License and terms
    Description
    • Bridge2AI Voice Registered Access License
    • Bridge2AI Voice Registered Access Agreement (Data Use Agreement)
    Ip Restrictions
    Name
    IP restrictions
    Description
    • Not specified; derived datasets distributed under Registered Access License and DUA on PhysioNet
    Regulatory Restrictions
    Name
    Export controls and regulatory restrictions
    Description
    • Not specified
    Maintainers
    • Name
      Maintainers
      Description
      • MIT Laboratory for Computational Physiology (PhysioNet)
      • For controlled access to raw audio: DACO@b2ai-voice.org (Bridge2AI Voice Data Access)
    Updates
    Name
    Versioning and updates
    Description
    • v1.0: initial release (derived data; 2024, cited in Health Data Nexus)
    • v1.1: added MFCCs (2025-01-17)
    • v2.0.0: released 2025-04-16 (latest superseded)
    • v2.0.1: released 2025-08-18 (latest as of listing)
    Version Access
    Name
    Access to prior versions
    Description
    • Files for v1.1 are no longer available on PhysioNet; the DOI remains for citation. Users are directed to the latest version (v2.0.1).
    Extension Mechanism
    Name
    Extension and reproducibility
    Description
    • Preprocessing/merging code released as the open-source b2aiprep library to facilitate reproducibility and extension of data processing pipelines
    Is Deidentified
    Name
    De-identification status
    Description
    • Yes. HIPAA Safe Harbor identifiers removed; state/province removed; free-speech transcripts removed; only derived features released in v1.1
    Third Party Sharing
    Name
    Distribution to third parties
    Description
    Yes, to registered users under the Registered Access License and Data Use Agreement via PhysioNet
    Was Derived From
    Original raw voice recordings collected under a standardized protocol; v1.1 consists of derived representations and features (no raw audio)
Generated on 2025-11-09 10:17:35 using Bridge2AI Data Sheets Schema