# physionet b2ai-voice 1.1 d4d

Datasheet for Dataset - Human Readable Format

🔍

## Collection Process

How was the data acquired?

ID *
bridge2ai-voice
Name
Bridge2AI-Voice
Title
Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information
Description
Bridge2AI-Voice is a comprehensive, ethically-sourced collection of data derived from voice recordings linked to clinical information, intended to enable artificial intelligence research into voice as a biomarker of health. Version 1.1 provides 12,523 recordings for 306 participants collected across five sites in North America, focusing on cohorts with conditions known to manifest in the voice (voice disorders, neurological disorders, mood disorders, and respiratory disorders). This release contains low-risk derived data (e.g., spectrograms, MFCCs, static acoustic/phonetic/prosodic features) and associated phenotype data; raw audio is not included and is available only via controlled access to protect participant privacy.
Language
English
Page
https://docs.b2ai-voice.org
Publisher
https://physionet.org
Keywords

- voice
- speech
- biomarker
- health
- Bridge2AI
- spectrograms
- MFCC
- acoustic features
- phenotypes
- clinical data

Created By

- Alistair Johnson
- Jean-Christophe Bélisle-Pipon
- David Dorr
- Satrajit Ghosh
- Philip Payne
- Maria Powell
- Anais Rameau
- Vardit Ravitsky
- Alexandros Sigaras
- Olivier Elemento
- Yael Bensoussan

Resources

1. ID
     bridge2ai-voice-v1.1
   Name
     Bridge2AI-Voice v1.1
   Title
     Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information (v1.1)
   Description
     Version 1.1 of Bridge2AI-Voice includes derived voice data (spectrograms, MFCCs, static acoustic/phonetic/prosodic features) and linked phenotype data for an adult cohort. It excludes raw audio waveforms. Files listed for this version include: spectrograms.parquet, mfcc.parquet, static_features.tsv (+ JSON data dictionary), phenotype.tsv (+ JSON data dictionary). Access is restricted to registered users under a data use agreement. Raw audio may be requested via controlled access.
   Language
     English
   Page
     https://doi.org/10.13026/249v-w155
   DOI
     doi:10.13026/249v-w155
   Issued
     2025-01-17
   Version
     1.1
   License
     Bridge2AI Voice Registered Access License
   Keywords
     - voice
     - speech
     - biomarker
     - health
     - Bridge2AI
     - spectrograms
     - MFCC
     - acoustic features
     - phenotypes
     - clinical data
   Is Tabular
     mixed (Parquet, TSV, JSON)
   Purposes
     - Name
         Purpose
       Response
         Create an ethically-sourced flagship dataset to enable AI research on voice as a biomarker of health and support critical clinical insights.
   Tasks
     - Name
         Task
       Response
         Voice-based biomarker discovery, disease classification/screening, and analysis of acoustic/phonetic/prosodic features linked to health.
   Addressing Gaps
     - Name
         AddressingGap
       Response
         Provide a large, multi-institutional, diverse voice dataset linked to health information with standardized collection protocols and explicit ethical oversight, addressing limitations of prior small and demographically limited datasets.
   Funders

     Grantor | Grant Name | Grant Number
     --- | --- | ---
     National Institutes of Health (NIH) | Bridge2AI: Voice as a Biomarker of Health - Building an ethically sourced, bioacoustic database to understand disease like never before | 3OT2OD032720-01S1
     National Institute of Biomedical Imaging and Bioengineering (NIBIB), NIH | PhysioNet infrastructure support | R01EB030362

   Instances

     Counts | Data Type | Instance Type | Label | Name | Representation | Sampling Strategies
     --- | --- | --- | --- | --- | --- | ---
     12523 | Derived features from audio (spectrograms: 513xN; MFCC: 60xN; static acoustic/phonetic/prosodic feat... | Multiple instance types: recordings (sessions/tasks) with derived data and per-participant phenotype... | No explicit supervised label provided; clinical/phenotype variables available | Voice-derived data instances | Derived voice data per recording session (e.g., spectrograms, MFCCs, static features) | {'name': 'Cohort sampling strategy', 'strategies': ['Targeted enrollment of patients in five predetermined clinical cohorts (respiratory, voice, neurological, mood, pediatric; adult cohort available in v1.1)'], 'is_sample': ['yes (targeted clinical cohorts)'], 'is_random': [False], 'source_data': ['Patients presenting at specialty clinics across five sites in North America'], 'is_representative': ['No (targeted cohorts; not representative of general population)'], 'why_not_representative': ['Enrollment focused on diseases with recognized vocal manifestations and unmet needs, to fuel disease-relevant voice AI research']}
     306 | Tabular phenotype data (phenotype.tsv with data dictionary phenotype.json) | Participants with demographics, questionnaires, and clinical phenotype data |  | Participants | Study participants (adult cohort in v1.1) |

   External Resources
     1. Name
          Raw audio access
        External Resources
          - Original raw audio available only via controlled access request (contact: DACO@b2ai-voice.org)
        Future Guarantees
          - Controlled access to protect participant privacy
        Archival
          - Derived datasets (spectrograms, MFCCs, features, phenotypes) are archived with DOI at PhysioNet
   Confidential Elements
     - Name
         Confidentiality
       Description
         - No raw audio recordings released in v1.1; dataset contains only derived features and de-identified phenotype data to reduce risk
   Content Warnings
     - Name
         Content warnings
       Warnings
         - None noted
   Subpopulations
     - Name
         Adult cohort
       Identification
         - Adult participants (v1.1 includes adult cohort only)
       Distribution
         - 306 participants across five North American sites; disease-focused cohorts (respiratory, voice, neurological, mood; pediatric planned)
   Sensitive Elements
     - Name
         Sensitive elements
       Description
         - Clinical/health-related phenotype data; identifiers removed per HIPAA Safe Harbor and additional de-identification
   Acquisition Methods
     1. Name
          Instance acquisition
        Description
          - Standardized, protocol-driven data collection using a custom tablet application; headset used when possible; demographic and clinical questionnaires; targeted tasks including sustained vowel phonation
        Was Directly Observed
          yes (audio tasks recorded)
        Was Reported By Subjects
          yes (questionnaires and targeted confounders)
        Was Inferred Derived
          yes (features, spectrograms, MFCCs, prosodic/phonetic metrics, transcriptions)
        Was Validated Verified
          Standardized multi-site protocol; preprocessing with defined parameters
   Collection Mechanisms
     - Name
         Collection mechanisms
       Description
         - Custom application on a tablet; headset microphone when possible; REDCap-based data export and conversion using an open-source library
   Data Collectors
     - Name
         Data collectors
       Description
         - Project investigators at specialty clinics across five North American sites screened and enrolled patients, obtained consent, and conducted standardized data collection sessions
   Ethical Reviews
     - Name
         IRB approval
       Description
         - Data collection and sharing approved by the University of South Florida Institutional Review Board
   Preprocessing Strategies
     - Name
         Audio preprocessing and feature extraction
       Description
         - Raw audio converted to mono and resampled to 16 kHz with a Butterworth anti-aliasing filter
         - Spectrograms via STFT (25 ms window, 10 ms hop, 512-point FFT; 513 x N)
         - 60 MFCCs derived from spectrograms (60 x N)
         - Acoustic features via openSMILE capturing temporal dynamics and acoustic characteristics
         - Phonetic and prosodic features via Parselmouth and Praat (e.g., F0, formants, voice quality)
         - Transcriptions generated using OpenAI Whisper Large (used for derivations; free-speech transcripts removed)
       Used Software

         ID | Name | URL
         --- | --- | ---
         openSMILE | openSMILE | https://audeering.github.io/opensmile/
         praat | Praat | https://www.fon.hum.uva.nl/praat/
         parselmouth | Parselmouth | https://parselmouth.readthedocs.io/
         torchaudio | Torchaudio | https://pytorch.org/audio
         whisper-large | OpenAI Whisper Large | https://github.com/openai/whisper

   Cleaning Strategies
     - Name
         De-identification and content removal
       Description
         - HIPAA Safe Harbor identifiers removed (e.g., names, finer-than-year dates, contact numbers, IPs, MRNs, device IDs, URLs, images, etc.); state/province removed; country retained
         - Transcripts of free speech audio removed
         - Audio waveforms omitted from release; only derived data (spectrograms and other features) provided in v1.1
   Raw Sources
     - Name
         Raw audio data
       Description
         - Original audio waveforms exist but are not distributed in v1.1; may be requested via controlled access by contacting DACO@b2ai-voice.org
   Future Use Impacts
     - Name
         Considerations for future use
       Description
         - Use of derived features only (no raw audio) reduces re-identification risk but may limit some modeling approaches; cohort-targeted sampling may impact generalizability; users should consider potential bias and fairness implications
   Discouraged Uses
     - Name
         Discouraged uses
       Description
         - Uses that attempt to re-identify individuals or reconstruct content beyond the scope of the derived features; any uses outside the Data Use Agreement and Registered Access License
   Distribution Formats
     - Name
         Formats
       Description
         - Parquet (spectrograms.parquet, mfcc.parquet)
         - TSV (phenotype.tsv, static_features.tsv)
         - JSON (phenotype.json, static_features.json)
   Distribution Dates
     - Name
         Initial distribution (v1.1)
       Description
         - 2025-01-17
   License And Use Terms
     Name
       License and terms
     Description
       - Bridge2AI Voice Registered Access License
       - Bridge2AI Voice Registered Access Agreement (Data Use Agreement)
   Ip Restrictions
     Name
       IP restrictions
     Description
       - Not specified; derived datasets distributed under Registered Access License and DUA on PhysioNet
   Regulatory Restrictions
     Name
       Export controls and regulatory restrictions
     Description
       - Not specified
   Maintainers
     - Name
         Maintainers
       Description
         - MIT Laboratory for Computational Physiology (PhysioNet)
         - For controlled access to raw audio: DACO@b2ai-voice.org (Bridge2AI Voice Data Access)
   Updates
     Name
       Versioning and updates
     Description
       - v1.0: initial release (derived data; 2024, cited in Health Data Nexus)
       - v1.1: added MFCCs (2025-01-17)
       - v2.0.0: released 2025-04-16 (latest superseded)
       - v2.0.1: released 2025-08-18 (latest as of listing)
   Version Access
     Name
       Access to prior versions
     Description
       - Files for v1.1 are no longer available on PhysioNet; the DOI remains for citation. Users are directed to the latest version (v2.0.1).
   Extension Mechanism
     Name
       Extension and reproducibility
     Description
       - Preprocessing/merging code released as the open-source b2aiprep library to facilitate reproducibility and extension of data processing pipelines
   Is Deidentified
     Name
       De-identification status
     Description
       - Yes. HIPAA Safe Harbor identifiers removed; state/province removed; free-speech transcripts removed; only derived features released in v1.1
   Third Party Sharing
     Name
       Distribution to third parties
     Description
       Yes, to registered users under the Registered Access License and Data Use Agreement via PhysioNet
   Was Derived From
     Original raw voice recordings collected under a standardized protocol; v1.1 consists of derived representations and features (no raw audio)

Generated on 2025-11-09 10:17:35 using Bridge2AI Data Sheets Schema