# VOICE Dataset Documentation

Datasheet for Dataset - Human Readable Format

🎯

## Motivation

Why was the dataset created?

Purposes

Description | ID
--- | ---
Integrate the use of voice as a biomarker of health in clinical care by generating a substantial multi-institutional, ethically sourced, and diverse voice database linked to multimodal health biomarkers to fuel voice AI research and build predictive models to assist in screening, diagnosis, and treatment of a broad range of diseases. | voice:purpose:1
Create an ethically sourced flagship dataset of 10,000 voices linked to health information to enable future research in artificial intelligence and support critical insights into the use of voice as a biomarker of health, addressing the pressing need for large, high quality, multi-institutional and diverse voice databases linked to other health biomarkers. | voice:purpose:2
Establish standards, best practices, and guidelines for voice data collection and analysis to advance the field of acoustic biomarkers by developing new standards that are AI/ML friendly and enable voice to emerge as a biomarker of health. | voice:purpose:3
Address ethical, legal, and social challenges surrounding voice AI including risks of voice re-identification, vulnerabilities like voice AI hacking, concerns around voice data sharing and privacy, and the influence of gender and racial diversity on voice AI. | voice:purpose:4

Funders

Description | ID
--- | ---
National Institutes of Health (NIH) Common Fund Bridge2AI Program. Grant number: 3OT2OD032720-01S3. Opportunity Number: OTA-21-008. Project dates: September 1, 2022 to November 30, 2026. Total funding in 2025: $4,660,942 (Direct Costs: $4,072,321, Indirect Costs: $588,621). Administering Institute: NIH Office of the Director. Study Section: Data Coordination, Mapping, and Modeling (DCMM). | voice:funder:1
National Institute of Biomedical Imaging and Bioengineering (NIBIB). Supports PhysioNet managed by MIT Laboratory for Computational Physiology under NIH grant number R01EB030362, which serves as the primary distribution platform for the Bridge2AI-Voice dataset. | voice:funder:2

📊

## Composition

What do the instances represent?

Instances

1. ID
     voice:instance:1
   Description
     Adult participants presenting at specialty clinics across multiple sites in North America. Participants selected based on membership to five predetermined disease cohort groups: Voice Disorders, Neurological and Neurodegenerative Disorders, Mood and Psychiatric Disorders, Respiratory Disorders, and Pediatric Voice and Speech Disorders. Version 3.0 contains approximately 833 adult participants with ~61,937 voice-derived recordings. Pediatric dataset v1.0 adds 300 participants. Enrollment anticipated to reach 10,000 participants by 2027.
   Instance Type
     Human participants with clinical diagnoses recruited from specialty clinics
   Counts
     833
   Label
     True
   Label Description
     Diagnostic labels assigned by clinical assessment at each site. Clinicians provided diagnoses based on clinical interview and appropriate work-up including laryngoscopy, stroboscopy, MRI, CT, whole genome sequencing, EHR records, and medication prescriptions. Labels include diagnostic categories: vocal pathologies, neurological disorders, psychiatric conditions, respiratory disorders, and pediatric voice/speech disorders. Per Bridge2AI protocols and ICD-10 codes. Single labeler per participant (site clinician).

Subpopulations

Description | ID | Name
--- | --- | ---
Participants with laryngeal disorders including laryngeal cancer (T1-T4, biopsy proven), laryngitis (acute, chronic, bacterial, fungal, autoimmune), pre-cancerous lesions (keratosis, leukoplakia), benign vocal cord lesions (nodules, polyps, cysts, Reinke's edema, recurrent laryngeal papilloma), muscle tension dysphonia, spasmodic dysphonia and laryngeal tremor, unilateral vocal fold paralysis, and glottic insufficiency/ presbyphonia. Validated by laryngoscopy images and stroboscopy videos. | voice:subpop:1 | Voice Disorders cohort
Participants aged 44-85 with clinical diagnoses including mild cognitive impairment, Alzheimer's disease, other dementias (frontotemporal, Lewy body, vascular, mixed, alcohol-induced), ALS (sporadic, familial, spinal/limb-onset, bulbar-onset), and Parkinson's disease (idiopathic PD, multiple system atrophy, progressive supranuclear palsy, corticobasal degeneration). Validated by CT Brain, MRI Brain, serum biomarkers, and whole genome sequencing. | voice:subpop:2 | Neurological and Neurodegenerative Disorders cohort
Participants with clinical diagnoses including depression/major depressive disorder, bipolar I/II disorder, anxiety disorder, schizophrenia, ADHD, PTSD, OCD, borderline personality disorder, and other psychiatric disorders. Validated by electronic health summary, clinical diagnosis from psychiatrist, and medication history. | voice:subpop:3 | Mood and Psychiatric Disorders cohort
Participants with respiratory conditions including airway stenosis (bilateral vocal fold paralysis, supraglottic, glottic, posterior glottic, subglottic, tracheal stenosis, multi-level upper airway stenosis) and chronic cough (bothersome cough >8 weeks). Validated by spirometry, flow volume loops, and CT scan of neck/chest. | voice:subpop:4 | Respiratory Disorders cohort
Pediatric participants aged 2-18 with voice and speech disorders recruited exclusively from Hospital for Sick Children (SickKids). Grouped by age: 2-4, 4-6, 6-10, 10+ years. Data collected using reproschema-ui with Bridge2AI-Voice pediatric protocol. Includes 36 pediatric-specific acoustic tasks and specialized questionnaires (C-VHI-10, PVOS, PVRQOL, PHQ-A). Pediatric dataset v1.0 available separately from adult dataset. | voice:subpop:5 | Pediatric cohort
Healthy volunteer control participants who complete common questionnaires and voice tasks (Voice Handicap Index-10, PHQ-9, GAD-7, Winograd) to provide normative comparisons. | voice:subpop:6 | Control participants

Distribution Formats

Description | ID | Name
--- | --- | ---
Time-varying features (spectrograms, mel spectrograms, MFCCs, pitch contour, SPARC features, PPGs) stored in Parquet format compatible with Python datasets library. Each element contains participant_id, session_id, task_name, and feature arrays. Excluded for open-response tasks in public release. | voice:format:1 | Parquet format for spectrograms and derived features
Phenotype data in tab-delimited format (confounders.tsv, demographics.tsv, diagnosis/ *.tsv, enrollment/*.tsv, questionnaire/*.tsv, task/*.tsv) with JSON data dictionaries per file. Static acoustic features (OpenSMILE, Praat, Parselmouth, torchaudio) in static_features.tsv with static_features.json data dictionary. One row per participant or recording as appropriate. BIDS v1.9.0 compliant structure. | voice:format:2 | TSV/JSON for phenotype and static features
Original raw audio waveforms in WAV format following BIDS structure: sub-{participant_id}/ses-{session_id}/audio/sub{id}_ses{id}_task-{task_name}.wav with companion JSON metadata. Available through controlled access only. Contact DACO@b2ai-voice.org to request access. | voice:format:3 | WAV audio format (controlled access only)

Distribution Dates

- ID
    voice:distdate:1
  Description
    Dataset first published and made available late November 2024 through Health Data Nexus. PhysioNet releases: v1.0 January 17, 2025 (306 participants, 12,523 recordings); v1.1 January 17, 2025 (added MFCC features); v2.0.0 April 16, 2025; v2.0.1 August 18, 2025; v3.0.0 released 2025 (833 adult participants, ~61,937 recordings). Semi-annual releases planned. Pediatric v1.0 available separately.

Distributions

Description | Format | ID | Media Type | Path
--- | --- | --- | --- | ---
Featurized dataset (registered access) distributed through PhysioNet. Contains AI-ready derived features: OpenSMILE eGeMaps, Parselmouth/Praat speech features, torchaudio features, spectrograms, mel spectrograms, MFCCs, SPARC features, PPGs. Phenotype data in BIDS v1.9.0 compliant TSV/JSON format. HIPAA Safe Harbor de-identified. Parquet files used for time-varying features (spectrograms, MFCCs, pitch contour). | TSV | voice:dist:1 | text/tab-separated-values | https://physionet.org/content/b2ai-voice/
JSON data dictionaries and metadata distributed alongside TSV phenotype files. Each TSV file has a companion JSON data dictionary describing field names, types, and allowed values. Follows BIDS v1.9.0 specification. | JSON | voice:dist:2 | application/json | https://physionet.org/content/b2ai-voice/
Raw audio dataset (controlled access only) distributed through PhysioNet and Health Data Nexus. Contains original raw audio waveforms compressed in GZ archives following BIDS v1.9.0 structure. Requires DACO approval and institutional DTUA. Contact DACO@b2ai-voice.org to request access. | GZ | voice:dist:3 | application/gzip | mailto:DACO@b2ai-voice.org
Pediatric dataset v1.0 (registered access) distributed through PhysioNet as separate download. Contains data from 300 pediatric participants with 36 pediatric-specific acoustic tasks and specialized questionnaires. BIDS v1.9.0 compliant TSV/JSON format. | TSV | voice:dist:4 | text/tab-separated-values | https://physionet.org/content/b2ai-voice/

🔍

## Collection Process

How was the data acquired?

ID
https://doi.org/10.13026/37yb-1t42
Name
Bridge2AI-Voice
Title
Bridge2AI-Voice - An ethically-sourced, diverse voice dataset linked to health information
Description
The Bridge2AI-Voice project seeks to create an ethically sourced flagship dataset to enable future research in artificial intelligence and support critical insights into the use of voice as a biomarker of health. The human voice contains complex acoustic markers which have been linked to important health conditions including dementia, mood disorders, and cancer. When viewed as a biomarker, voice is a promising characteristic to measure as it is simple to collect, cost-effective, and has broad clinical utility. This comprehensive collection provides voice recordings with corresponding clinical information from participants selected based on known conditions which manifest within the voice waveform including voice disorders, neurological disorders, mood disorders, and respiratory disorders. The dataset is designed to fuel voice AI research, establish data standards, and promote ethical and trustworthy AI/ML development for voice biomarkers of health. Data collection occurs through a multi-institutional collaborative effort using standardized protocols, custom smartphone applications, and rigorous ethical oversight. Version 3.0 provides approximately 61,937 voice-derived recordings from 833 adult participants collected across multiple sites in North America, with derived features such as spectrograms, MFCCs, acoustic features, and clinical phenotype data. The pediatric dataset v1.0 is also available with data from 300 participants. Raw audio data is available through controlled access to protect participant privacy.
Page
https://docs.b2ai-voice.org
Language
en
Tabular Data
False
Keywords

- voice biomarker
- acoustic biomarker
- Bridge2AI
- voice AI
- voice disorders
- neurological disorders
- neurodegenerative disorders
- mood disorders
- psychiatric disorders
- respiratory disorders
- pediatric voice disorders
- speech disorders
- Parkinson's disease
- Alzheimer's disease
- depression
- schizophrenia
- bipolar disorder
- stroke
- ALS
- autism spectrum disorder
- speech delay
- laryngeal cancer
- vocal fold paralysis
- muscle tension dysphonia
- laryngeal dystonia
- COPD
- chronic cough
- airway stenosis
- obstructive sleep apnea
- spectrogram
- MFCC
- mel-frequency cepstral coefficients
- OpenSMILE
- Praat
- Parselmouth
- torchaudio
- federated learning
- ethical AI
- multimodal health data
- electronic health records
- EHR
- radiomics
- genomics
- FAIR principles
- CARE principles
- PhysioNet
- Health Data Nexus
- BIDS
- Brain Imaging Data Structure
- b2aiprep

Publisher
Bridge2AI-Voice Consortium, University of South Florida
Research Gaps Addressed

Description | ID
--- | ---
Address the lack of large, high quality, multi-institutional and diverse voice databases linked to multimodal health biomarkers (demographics, imaging, genomics, risk factors) necessary to fuel voice AI research and answer tangible clinical questions. | voice:gap:1
Overcome limitations in existing voice and psychiatric disorder research that has relied on small datasets with limited demographic diversity reporting, lack of standardized data collection protocols, and possible confounders limiting external validity. | voice:gap:2
Fill the gap in pediatric voice and speech analysis research, which is sparser partly due to ethical concerns and challenges in data acquisition for this cohort, particularly for autism spectrum disorder and speech delay detection. | voice:gap:3
Establish missing standards for voice data collection, acoustic analysis, and ethical frameworks for consenting to voice data collection, sharing, and utilization in the context of voice AI technology development and clinical adoption. | voice:gap:4
Develop software and cloud infrastructure for automated voice data collection through a smartphone application (Bridge2AI-Voice App) that allows non-invasive, user-friendly, high quality voice data collection while minimizing human manipulation and implementing federated learning technology to minimize data sharing while preserving patient privacy. | voice:gap:5

Creators

Role | Name | ORCID | Affiliation
--- | --- | --- | ---
Contributor |  | voice:creator:1 | -

Known Biases

Description | ID
--- | ---
Clinic-based recruitment introduces selection bias toward treatment-seeking populations. Participants are recruited from high-volume specialty clinics, which may not represent the full spectrum of disease severity or the general population with these conditions. Underrepresentation of individuals with less trust in the medical system or less proximity to collection sites. | voice:bias:1
Current releases contain only English-speaking participants, which may bias acoustic features and models trained on this data toward English-language phonological patterns. Spanish protocols are under development for future releases. | voice:bias:2
Limited geographic diversity: data collected at five North American sites only, which may not capture regional variation in voice characteristics, dialects, or environmental acoustic conditions. | voice:bias:3
Imbalanced distribution across disease categories in public releases; the dataset does not contain equal representation across all five disease cohort groups. This may affect the performance and fairness of models trained on the dataset. | voice:bias:4

Known Limitations

Description | ID
--- | ---
Version 3.0 (833 adult participants) represents an interim dataset from an ongoing collection targeting 10,000 participants by 2027. Statistical power for some analyses, particularly for less prevalent conditions, may be limited. | voice:limitation:1
Raw audio waveforms are excluded from public release and available only through controlled access. This restricts certain types of analyses requiring the original audio signal and adds access overhead for researchers. | voice:limitation:2
Remote data collection not included in initial releases; all data collected in clinical settings by research assistants, which may not generalize to naturalistic or home settings. | voice:limitation:3
Multimodal data (imaging, genomics, full EHR data) not included in current public releases; only derived clinical features and diagnoses are provided. | voice:limitation:4
No analysis of potential re-identification impact has been formally conducted on the de-identified dataset beyond HIPAA Safe Harbor standards. | voice:limitation:5

Confidential Elements

Description | ID
--- | ---
Raw audio waveforms are considered biometric identifiers under HIPAA and are restricted to controlled access only. Researchers must apply through the Data Access Compliance Office (DACO) and sign an institutional Data Use and Transfer Agreement (DTUA). | voice:confidential:1
Open-response audio features (spectrograms, MFCCs, mel spectrograms, transcriptions, EMAs, and PPGs from open-response prompts) are removed from the public registered access dataset to protect participant privacy. | voice:confidential:2
The dataset is covered under a Certificate of Confidentiality, which provides legal protection against compulsory demands such as court orders and subpoenas for identifying information about research participants. | voice:confidential:3

Sensitive Elements

Role | Name | ORCID | Affiliation
--- | --- | --- | ---
Contributor |  | voice:sensitive:1 | -
Contributor |  | voice:sensitive:2 | -
Contributor |  | voice:sensitive:3 | -
Contributor |  | voice:sensitive:4 | -
Contributor |  | voice:sensitive:5 | -

Collection Mechanisms

Description | ID
--- | ---
Voice data collected in clinic using custom Bridge2AI-Voice App on iPad (9th or 10th generation) or iPad Air (5th generation) with Avid AE-36 microphone and Apple dongle connector. App collects breathing sounds and voice, speech, and linguistic tasks along with health information through surveys and validated questionnaires. Research assistant present during collection. Future remote data collection planned but not included in current releases. | voice:collection:1
Pediatric data collected using reproschema-ui with Bridge2AI-Voice pediatric protocol. Same iPad hardware as adult protocol with age-appropriate tasks grouped by age range (2-4, 4-6, 6-10, 10+ years). Questions read to participants when needed. | voice:collection:2
Clinical data including EHR information, imaging (laryngoscopy, stroboscopy, MRI, CT), and genomic data extracted from sites independently and uploaded through REDCap database. No external multimodal data released in current dataset versions. | voice:collection:3

Acquisition Methods

Description | ID
--- | ---
22 acoustic tasks recorded through Bridge2AI-Voice App for adult participants including: Non-Voice (Respiration, Cough, Breath Sounds, Voluntary Cough); Voice/Non-Speech (Prolonged Vowel /e/, Maximum Phonation Time, Glides, Loudness /Hey/, Diadochokinesis /pa/ta/ka/); Speech (Rainbow Passage, Caterpillar Passage, Cape-V Sentences, Free Speech, Picture Description, Story Recall, Animal Fluency, Open Response Questions, Word-Color Stroop, Productive Vocabulary, Random Item Generation, Cinderella Story). Tasks vary by disease cohort (Part A/Voice/Resp/Mood/Neuro). | voice:acquisition:1
36 pediatric acoustic tasks collected via reproschema-ui including: Speech tasks (ABC's, Ready for School, Favorite Show, Favorite Food, Outside of School, Months, Counting, Naming Animals, Naming Food, Identifying Pictures, Picture Description, Caterpillar Passage, Repeat Words, Role Naming, Repeat Sentences); Voice/Non-Speech tasks (Long Sounds, Noisy Sounds, Silly Sounds /PUH TUH KUH/). | voice:acquisition:2
Self-reported demographic data and medical history questionnaires administered through app. Validated questionnaires integrated for each disease cohort including: VHI-10, PHQ-9, GAD-7, PANAS, Custom Affect Scale, PTSD Adult, ADHD Adult, DSM-5 Adult, Dyspnea Index, Leicester Cough Questionnaire, Winograd Questionnaire, MOCA, C-VHI-10 (peds), PVOS (peds), PVRQOL (peds), PHQ-A (peds). | voice:acquisition:3
Electronic health record (EHR) access for consenting participants permitting investigators to access medical information through EHR platforms to perform gold standard validation of diagnoses and symptoms. Linkage to multimodal health biomarkers including laryngoscopy imaging, radiomics, genomics, respiratory function tests. | voice:acquisition:4

Collection Timeframes

- ID
    voice:timeframe:1
  Description
    Data collection ongoing from September 2022 through November 2026 (project end date). Initial dataset releases began in late 2024. Version 1.0 released January 17, 2025 (306 participants). Version 3.0 released 2025 (833 adult participants). Semi-annual releases planned. Enrollment anticipated to reach 10,000 participants by 2027.

Data Collectors

- ID
    voice:datacollector:1
  Description
    Research teams at each of the five North American collection sites, including medical graduate and undergraduate students coordinating with site clinicians and doctors. Clinicians and doctors listed under IRB as co-investigators and added to consortium. Participants compensated via electronic gift cards: $40 for sessions under 90 minutes, $80 for sessions over 90 minutes, maximum 3 sessions and $120 total compensation.

Sampling Strategies

1. ID
     voice:sampling:1
   Description
     Non-probability purposive sampling from specialty clinics (high volume expert clinics, outpatient clinics seeing >50 patients per month from same disease category). Patients presenting at clinics screened for eligibility per inclusion/exclusion criteria outlined in protocol Table 1. Not representative of general population due to limited geographic locations and clinic-based recruitment. Current v3.0 dataset is a sample of an ongoing collection targeting 10,000 participants by 2027. English-speaking adult participants (18-120 years); Spanish protocols under development for future releases.
   Sample
     True
   Random Sampling
     False
   Representative Sample
     False

Missing Data Documentation

- ID
    voice:missingdata:1
  Description
    Missingness tables generated and included with dataset as part of the audit protocol. Audio quality control metrics applied including silence amount, duration, and speech-to-passage accuracy checks. Processing using b2aiprep and SenseLab toolkits. Some participants may have incomplete questionnaire responses or missing sessions.

Raw Data Sources

Description | ID | Source Description
--- | --- | ---
Raw audio waveforms in WAV format collected via Bridge2AI-Voice App on iPad devices with Avid AE-36 microphone. Stored in BIDS v1.9.0 compliant format per participant session and acoustic task. Available through controlled access only. | voice:rawsource:1 | Raw audio WAV files recorded directly by participants in clinical settings using the Bridge2AI-Voice App on iPad with Avid AE-36 microphone. Organized in BIDS v1.9.0 directory structure: sub-{participant_id}/ses-{session_id}/audio/ with companion JSON metadata sidecar files. Available through DACO-controlled access only.
REDCap database exports containing EHR-linked clinical data, demographic information, disease-specific validated questionnaire responses, and diagnostic information. Pediatric data first extracted from reproschema-ui to REDCap format. | voice:rawsource:2 | REDCap (Research Electronic Data Capture) database exports and reproschema-ui exports containing clinical phenotype data: demographic information, validated questionnaire responses (VHI-10, PHQ-9, GAD-7, etc.), diagnostic information, and EHR-linked clinical assessments. Pediatric data first extracted from reproschema-ui then converted to REDCap format before BIDS conversion.

Raw Sources

Description | ID
--- | ---
Raw audio WAV files recorded through the Bridge2AI-Voice App, organized in BIDS v1.9.0 compliant directory structure. Available through controlled access via DACO approval. | voice:rawsrc:1
REDCap database exports and reproschema-ui exports containing clinical phenotype data, questionnaire responses, and participant demographics. | voice:rawsrc:2

Preprocessing Strategies

Description | ID
--- | ---
Raw audio preprocessing using b2aiprep library: conversion to monaural audio, resampling to 16 kHz with Butterworth anti-aliasing filter. Standardization ensures consistent format across all recordings for downstream feature extraction. | voice:preproc:1
Spectrogram extraction using short-time Fast Fourier Transform (FFT): 25ms window size, 10ms hop length, 512-point FFT. Output spectrograms have 513xN dimensions where N is proportional to audio length. Stored in Parquet format. | voice:preproc:2
Mel-frequency cepstral coefficients (MFCC) extraction: 60 MFCCs extracted from spectrograms capturing perceptually-relevant spectral envelope characteristics. Output dimension 60xN. Stored in Parquet format. | voice:preproc:3
OpenSMILE eGeMaps acoustic feature extraction capturing temporal dynamics and acoustic characteristics. One row per unique recording in static_features.tsv. | voice:preproc:4
Parselmouth and Praat phonetic and prosodic feature computation providing fundamental frequency (F0), formants, and voice quality measures. Documented in static_features.json data dictionary. | voice:preproc:5
Torchaudio-based feature extraction including pitch contour, spectrograms, mel spectrograms, MFCCs. SPARC-based features including electromagnetic articulography (EMA) estimates, loudness, periodicity, and pitch measures. Phonetic posteriorgrams (PPGs). | voice:preproc:6
Transcription generation using OpenAI Whisper model applied to audio recordings. Raw audio transcripts reviewed and any recordings containing potentially identifying information or external voices removed. All transcriptions, EMAs, and PPGs from open-response prompts removed from public feature-only dataset. | voice:preproc:7
REDCap and reproschema-ui data exported and converted to BIDS v1.9.0 format using b2aiprep open-source library. Phenotype data organized in tab-delimited files with JSON data dictionaries. Pediatric data first extracted from reproschema-ui to REDCap format then converted to BIDS. | voice:preproc:8

Cleaning Strategies

Description | ID
--- | ---
HIPAA Safe Harbor de-identification for public release: removed direct identifiers (names, civic addresses, social security numbers), indirect identifiers creating significant re-identification risk (select geographic/demographic identifiers, household composition, cultural identity), and sensitive information (household income, mental health status, traumatic life experiences). Geographic data limited; state/province removed, country retained. | voice:cleaning:1
Audio privacy protection for public release: all raw audio waveforms excluded from public dataset. All spectrograms, MFCCs, mel spectrograms, transcriptions, EMAs, and PPGs from open-response prompts removed from feature-only dataset. Free speech transcripts removed. Raw audio available only through controlled access with DACO approval. | voice:cleaning:2
Sensitive field removal based on REDCap data dictionary: all fields encoded as sensitive (column "Identifier?" in REDCap data dictionary CSV) removed from dataset. | voice:cleaning:3
Audit protocol applied: missingness tables generated and included with dataset; distribution and outlier checks; categorical responses checked against schema; audio quality control metrics including silence amount, duration, and speech-to-passage accuracy checks. Processing using b2aiprep and SenseLab toolkits. | voice:cleaning:4

Labeling Strategies

- ID
    voice:labeling:1
  Description
    Diagnostic labels assigned by site clinicians based on clinical assessment and gold standard validation methods per Bridge2AI-Voice protocol Table 1 and ICD-10 codes. For voice disorders: laryngoscopy and stroboscopy. For neurological disorders: CT Brain, MRI Brain, genome sequencing, serum biomarkers. For mood/psychiatric disorders: EHR records, psychiatrist diagnosis, medication history. For respiratory disorders: spirometry, flow volume loops, CT scan. Single labeler (site clinician) per participant.

Machine Annotation Tools

Description | ID
--- | ---
OpenAI Whisper model used for automated transcription of audio recordings. Transcripts reviewed for quality; recordings with potentially identifying information or external voices removed. Transcriptions from open-response prompts removed from public release. | voice:machanno:1
b2aiprep open-source library used for automated preprocessing of raw audio, REDCap data conversion, and BIDS formatting. SenseLab toolkit used for audio quality control and feature extraction. | voice:machanno:2

Maintainers

Description | ID
--- | ---
Bridge2AI-Voice Consortium (University of South Florida, lead institution) supported by NIH Bridge2AI program. Contact: [email protected]. Responsible for dataset curation, standards development, ethics oversight, versioning, and updates. Data Access Compliance Office (DACO) manages controlled access applications. Contact: DACO@b2ai-voice.org. | voice:maintainer:1
PhysioNet (MIT Laboratory for Computational Physiology, supported by NIBIB NIH grant R01EB030362) serves as primary distribution platform for registered access dataset. Provides technical infrastructure and access management. | voice:maintainer:2
Health Data Nexus (Temerty Centre for Artificial Intelligence Research and Education in Medicine, T-CAIREM, University of Toronto) maintains alternative platform providing cloud compute alongside dataset. Earlier dataset versions available here. Contact: [email protected]. | voice:maintainer:3

Retention Limit

ID
  voice:retention:1
Name
  Data retention and disposition policy
Description
  Data Transfer and Use Agreement (DTUA) specifies two-year agreement term from start date. Upon termination or expiration, data shall be destroyed per provider instructions with written certification required within 30 days. Recipient may retain one copy to comply with records retention requirements under law, regulation, institutional policy, and for research integrity and verification. Ongoing restrictions apply to retained copies. Provider may unilaterally amend agreement if federal sponsor requires revision. Health Data Nexus retains dataset as long as useful for research purposes, possibly indefinitely. Version-specific DOIs maintained for all historical versions.

Extension Mechanism

ID
  voice:extension:1
Name
  Dataset extension and contribution mechanisms
Description
  Derivative datasets can be published on Health Data Nexus referencing original source under same access conditions. Open-source repositories (b2aiprep, SenseLab) have discussion forums, issue pages, and pull request mechanisms for contributing improvements to preprocessing code. REDCap data dictionary available at https://github.com/eipm/bridge2ai-redcap with MIT license. Bridge2AI-Voice documentation at https://github.com/eipm/bridge2ai-docs. Future augmentations to voice collection protocol coordinated through consortium.

Ethical Reviews

- ID
    voice:ethicalreview:1
  Description
    IRB review and approval obtained from University of South Florida Single IRB with subsite IRB approvals through Single IRB process. Study reviewed and approved for human subjects research. Bioethics guidance integrated throughout study design and conduct by consortium bioethicists (Belisle-Pipon, Ravitsky). Ethics module develops new guidelines for consenting to voice data collection, voice data sharing, and utilization in context of voice AI technology. Project addresses ethical issues from voice data generation through clinical adoption and downstream health decisions.

At-Risk Populations

ID
  voice:atrisk:1
Name
  Pediatric participants protection
Description
  Pediatric participants (aged 2-18) require additional protections. Pediatric data collected exclusively at Hospital for Sick Children (SickKids) with age-appropriate protocols. Distinct IRB considerations for pediatric cohort. Pediatric questionnaires and acoustic tasks specifically designed for different age groups (2-4, 4-6, 6-10, 10+ years). Minimum age for adult dataset is 18 years. Pediatric dataset released separately with additional privacy precautions.

Intellectual Property Restrictions

ID
  voice:ip:1
Name
  Intellectual property restrictions
Description
  Recipient shall not develop or use any intellectual property protections, database rights, or related rights in any element of the dataset or in conclusions derived from it that would prevent or block access to any element of the dataset or those conclusions. Fort Lauderdale Agreement principles apply. Recipients must respect Open Science principles. Recipient shall not disclose, release, sell, rent, or lease data to third parties without prior written consent of Provider.

Regulatory Restrictions

ID
  voice:regulatory:1
Name
  Regulatory restrictions and compliance requirements
Description
  Dataset covered under Certificate of Confidentiality per 45 CFR 46 (Common Rule), which must be asserted against compulsory legal demands such as court orders and subpoenas. HIPAA Safe Harbor de-identification standards applied. Data covered as Personally Identifiable Information per OMB Memorandum M-07-16. No export control restrictions apply. Authorized Researchers must comply with all applicable laws and regulations. DTUA specifies two-year term with data destruction requirements.

Deidentification

ID
  voice:deident:1
Name
  HIPAA Safe Harbor de-identification
Description
  HIPAA Safe Harbor de-identification applied. All direct identifiers removed (names, civic addresses, social security numbers). Indirect identifiers removed where creating significant re-identification risk (geographic/demographic identifiers, household composition, cultural identity). Non-identifying sensitive information removed (household income, mental health status, traumatic life experiences). All raw voice data removed from public release. Sensitive fields per REDCap data dictionary removed. Direct identifiers removed: Yes. HIPAA de-identification rules applied: Yes. Dates rebased: Yes. Geographic information removed/generalized: Yes. Narrative text fields removed: Yes. K-anonymization: No.

External Resources

Description | External Resources | ID | Name
--- | --- | --- | ---
Primary registered access distribution platform for adult and pediatric datasets | https://physionet.org/content/b2ai-voice/ | voice:resource:1 | PhysioNet Dataset Landing Page
Comprehensive project documentation, collection methods, governance, healthsheet | https://docs.b2ai-voice.org | voice:resource:2 | Bridge2AI-Voice Project Documentation
Source code for docs and dashboard at docs.b2ai-voice.org (MIT license) | https://github.com/eipm/bridge2ai-docs | voice:resource:3 | Bridge2AI-Voice GitHub Documentation Repository
Open source library (Apache-2.0 license) for preprocessing raw audio waveforms into parquet files and merging source data into phenotype files | https://github.com/sensein/b2aiprep | voice:resource:4 | b2aiprep Software Library
REDCap data dictionary and metadata (MIT license) | https://github.com/eipm/bridge2ai-redcap | voice:resource:5 | Bridge2AI REDCap Data Dictionary
Federal grant information for project 3OT2OD032720-01S3 | https://reporter.nih.gov/project-details/11376382 | voice:resource:6 | NIH RePORTER Project Details
Alternative platform providing cloud compute alongside earlier dataset versions | https://healthdatanexus.ai/content/b2ai-voice/1.0/ | voice:resource:7 | Health Data Nexus
Bensoussan Y. et al. (2024). eipm/bridge2ai-redcap. Zenodo archive. | https://doi.org/10.5281/zenodo.13834653 | voice:resource:8 | Zenodo Archive (REDCap data dictionary)
Bensoussan et al. "Developing Multi-Disorder Voice Protocols: A team science approach involving clinical expertise, bioethics, standards, and DEI." Proc. Interspeech 2024. | https://doi.org/10.21437/Interspeech.2024-1926 | voice:resource:9 | Interspeech 2024 Protocol Publication
Contact for controlled access to raw audio data and institutional DTUA | mailto:DACO@b2ai-voice.org | voice:resource:10 | Data Access Compliance Office (DACO)
Parent NIH Common Fund program supporting AI-ready biomedical datasets | https://bridge2ai.org | voice:resource:11 | Bridge2AI Program
Training opportunities for using the dataset | https://www.b2aivoicescholars.org/ | voice:resource:12 | Bridge2AI Voice Scholars Training

🚀

## Uses

What (other) tasks could the dataset be used for?

Tasks

Description | ID
--- | ---
Enable development of AI/ML predictive models for screening, diagnosis, and treatment of voice disorders including laryngeal cancers, vocal fold paralysis, muscle tension dysphonia, laryngeal dystonia, benign laryngeal lesions, and pre-cancerous lesions. | voice:task:1
Support machine learning models for neurological and neurodegenerative disorders including Alzheimer's disease, Parkinson's disease, mild cognitive impairment, other dementias, and ALS, detecting voice and speech changes. | voice:task:2
Develop AI algorithms for mood and psychiatric disorder detection including depression, schizophrenia, bipolar disorder, anxiety disorders, ADHD, PTSD, OCD, and borderline personality disorder. | voice:task:3
Create machine learning models for respiratory disorder screening and therapeutic monitoring using respiratory sounds, cough sounds, and voice, applicable to conditions such as chronic cough, COPD, airway stenosis, and other respiratory conditions. | voice:task:4
Build AI models for pediatric voice and speech disorder detection using 36 pediatric-specific acoustic tasks and specialized questionnaires, addressing the relative scarcity of pediatric voice data for participants aged 2-18. | voice:task:5
Promote application of AI/ML for voice research through workforce development, curriculum creation on voice biomarkers of health for FAIR and CARE AI models, and fostering collaborations especially with researchers from underserved communities. | voice:task:6
Enable AI model pretraining, fine-tuning, benchmarking, and validation using the standardized BIDS-compliant dataset structure with features extracted via b2aiprep, SenseLab, OpenSMILE, Parselmouth, Praat, and torchaudio toolkits. | voice:task:7

Intended Uses

Description | ID
--- | ---
Primary intended use: development and validation of AI/ML models for voice as a biomarker of health, supporting screening, diagnosis, and treatment of voice disorders, neurological disorders, mood disorders, respiratory disorders, and pediatric speech disorders through model pretraining, fine-tuning, benchmarking, and validation. | voice:use:1
Research into acoustic biomarkers and development of standards for voice data collection and analysis. Establishing best practices for AI/ML-friendly voice datasets and contributing to the field's maturation as a clinical diagnostic modality. | voice:use:2
Training and education in voice AI research through workforce development initiatives, curriculum creation on FAIR and CARE voice AI model development, and fostering collaborations between medical voice researchers, acoustic engineers, and AI/ML specialists, especially from underserved communities. | voice:use:3
Multimodal health research combining voice data with EHR information, radiomics, genomics, imaging, and other health biomarkers to understand complex disease relationships and improve diagnostic accuracy. | voice:use:4

Existing Uses

- ID
    voice:existinguse:1
  Description
    A restricted version of the dataset containing raw audio has been used in the Bridge2AI Summer School and hackathon for education and research training purposes.

Discouraged Uses

Description | ID
--- | ---
Non-clinical applications such as hiring decisions, insurance premium adjustments, or any form of surveillance that could lead to discrimination or harm based on health conditions or voice characteristics. These applications could negatively impact individuals. | voice:discouraged:1
Any attempt to re-identify research participants or use data in ways that could foreseeably cause harm or stigmatization to research participants, their families, communities, or specific populations. Dataset covered under Certificate of Confidentiality. | voice:discouraged:2
Development or use of intellectual property protections, database rights, or related rights in ways that would prevent or block access to any element of the dataset or conclusions derived from it. Must respect Fort Lauderdale Agreement and Open Science principles. | voice:discouraged:3

Prohibited Uses

Role | Name | ORCID | Affiliation
--- | --- | --- | ---
Contributor |  | voice:prohibited:1 | -

Future Use Impacts

Description | ID
--- | ---
Potential risks from future uses include voice re-identification despite de-identification efforts, discriminatory application of voice biomarker models in clinical or non-clinical settings, and reinforcement of health disparities if models are trained on non-representative data. The Certificate of Confidentiality and access controls mitigate some of these risks. | voice:futureimpact:1
Positive anticipated impacts include acceleration of clinical voice AI adoption for screening and diagnosis of conditions currently lacking cost-effective diagnostic tools, improved health equity through diverse data collection, and establishment of FAIR and CARE-compliant data standards for the voice AI field. | voice:futureimpact:2

License and Use Terms

ID
  voice:license:1
Name
  Bridge2AI Voice Registered Access License
Description
  Public access dataset distributed through PhysioNet under Bridge2AI Voice Registered Access License. Only registered users who sign the specified Data Use Agreement (Bridge2AI Voice Registered Access Agreement) can access files. Data covered under Certificate of Confidentiality which must be asserted against compulsory legal demands. Raw audio available through controlled access only via Data Access Compliance Office (DACO) requiring distinct application and DTUA signed by institutional official. Recipient must adhere to PhysioNet requirements managed by MIT Laboratory for Computational Physiology. Recipient encouraged to publish results in open-access journals. No export controls apply. Commercial and non-commercial research use permitted for Authorized Researchers.

📤

## Distribution

How will the dataset be distributed?

DOI
10.13026/37yb-1t42
License
Bridge2AI Voice Registered Access License
Download URL
https://physionet.org/content/b2ai-voice/
Version Access

ID
  voice:versionaccess:1
Name
  Version access policy
Description
  All dataset versions available through PhysioNet at https://physionet.org/content/b2ai-voice/ with version-specific DOIs. DOI for latest version: https://doi.org/10.13026/37yb-1t42. Earlier versions also available on Health Data Nexus at https://healthdatanexus.ai/content/b2ai-voice/1.0/. By default, older versions continue to be supported, hosted, and made available. Dataset publishers reserve right to remove access to older versions. Each version has unique DOI.

🔄

## Maintenance

How will the dataset be maintained?

Version
3.0.0
Updates

ID
  voice:updates:1
Name
  Versioned releases with ongoing data collection
Description
  Dataset updated with versioned static releases semi-annually as data collection progresses. Users notified through news items on platforms and standard communication channels. v1.0 released January 17, 2025 (306 participants, 12,523 recordings); v1.1 added MFCC features; v2.0.0 April 16, 2025; v2.0.1 August 18, 2025; v3.0.0 released 2025 (833 adults, ~61,937 recordings). Pediatric v1.0 released separately. Data collection ongoing through November 30, 2026. Target: 10,000 participants by 2027. Future releases will expand Spanish language protocols and add additional multimodal data (imaging, genomics). Version-specific DOIs maintained. Older versions continue to be supported and hosted.

👥

## Human Subjects

Does the dataset relate to people?

Human Subject Research

ID
  voice:hsr:1
Name
  Bridge2AI-Voice Human Subjects Research
Description
  Observational study (cross-sectional design) involving direct collection of voice recordings, questionnaire responses, and EHR linkage from human participants. Prospective informed consent obtained from all participants. IRB approved by University of South Florida Single IRB with subsite IRBs through Single IRB process. Not a drug or medical device study. No data monitoring committee appointed. Study ID: OT2OD032720. Involves human subjects: Yes. IRB approval: University of South Florida Institutional Review Board (Single IRB). Special populations: Pediatric participants (aged 2-18 years) at Hospital for Sick Children. Regulatory compliance: HIPAA Safe Harbor, 45 CFR 46 (Common Rule), Certificate of Confidentiality, OMB Memorandum M-07-16.
Involves Human Subjects
  True

Informed Consent

Role | Name | ORCID | Affiliation
--- | --- | --- | ---
Contributor |  | voice:consent:1 | -

Generated on 2026-04-21 15:34:11 using Bridge2AI Data Sheets Schema