The dataset has been audited internally for missingness and consistency by the data release team. A missingness table is included with the dataset. Certain aspects of the data (e.g., transcription) were generated using off-the-shelf models that have not been audited for correctness.

Static

Yes, the current version of the dataset comes with predefined tasks and labeling. The tasks are primarily designed for training machine-learning models for disease detection and classification using voice data. Labels include diagnostic categories such as vocal pathologies, neurological disorders, psychiatric conditions, and respiratory disorders. However, there are no predefined recommended data splits for training, validation, or testing. Researchers are encouraged to create their own data splits based on their specific requirements. More details regarding task definitions and labeling can be found in the dataset.

The Bridge2AI Voice dataset was created to address a gap in the availability of large-scale, diverse, and well-documented voice data for use in clinical machine-learning applications. Previous studies on machine learning-based voice diagnosis produced promising results, but their sample sizes were too small, or they lacked the key metadata needed for training robust, clinically useful models. The dataset aims to bridge this gap by providing an ethically sourced, large, and diverse dataset to develop, benchmark, or validate clinically applicable AI/ML models. The goal is to facilitate the use of voice as a non-invasive, cost-effective biomarker for the screening, diagnosis, and monitoring of a wide range of health conditions.

The Bridge2AI Voice dataset is primarily intended for research applications, specifically in the development of AI and machine-learning models for healthcare. It aims to support clinical research in disease screening, diagnosis, and monitoring through voice biomarkers. The dataset can be used for AI model pretraining, fine-tuning, benchmarking, or validation.

Yes, there are restrictions on the use of this dataset, as detailed in the Registered Data Access Agreement. These restrictions reflect the B2AI-Voice Consortium’s commitment to advancing ethical and trustworthy research practices that respect and protect the rights and interests of research participants. Accordingly, the dataset is intended solely for commercial and non-commercial research purposes by Authorized Researchers. Specifically, the dataset is not to be used a) to attempt to re-identify research participants, nor any actions that could reasonably lead to re-identification; and b) for any purpose that could foreseeably cause harm or stigmatization to research participants, their families, communities, or specific populations. Lastly, intellectual property protections, database rights, or related rights may not be used in a manner that restricts or limits access to any part of the dataset or to any conclusions derived from it. This restriction ensures that future use of the dataset remains unrestricted, in alignment with the Open Science principles upheld by the B2AI-Voice Consortium. Specifically, the dataset should not be used for non-research applications, such as hiring decisions, insurance premium adjustments, or any form of surveillance that could lead to discrimination or harm. These limitations are intended to prevent unethical or biased outcomes that could negatively impact individuals based on their health conditions or voice characteristics.

Voice as a Biomarker of Health is being co-led by Dr. Yaël Bensoussan, MD, MSc, from USF Health Morsani College of Medicine and Olivier Elemento, PhD, from Weill Cornell Medicine, who are co-principal investigators for the project, which is funded by the NIH Common Fund within the Bridge2AI Program. The project also includes lead investigators from 10 other universities in North America; Alexandros Sigaras, MSc and Anaïs Rameau, MD, MPhil (Weill Cornell Medicine), Maria Powell, CCC-SLP, PhD (Vanderbilt University Medical Center), Ruth Bahr, CCC-SLP, PhD (USF Health Morsani College of Medicine), Jennifer Sui, MD (Hospital for Sick Children), Philip Payne, PhD (Washington University in St. Louis), David Dorr, MD (Oregon Health & Science University), Jean-Christophe Bélisle-Pipon, PhD (Simon Fraser University), Vardit Ravitsky, PhD (The Hastings Center), Satrajit Ghosh, PhD (Massachusetts Institute of Technology), Frank Rudzizc, PhD (University of Toronto), Jordan Lerner-Ellis, PhD (Sinai Health) and Don Bolser, PhD (University of Florida). There are over 50 other investigators, clinicians, scholars, and trainees who have contributed to the development of this dataset. Please see full list of collaborators here: The Bridge2AI-Voice Consortium (2024)

The NIH Common Fund
3TF-OT2ActfOD032720Projectf01S1

The curators and generators of the Bridge2AI Voice dataset come from a diverse range of backgrounds and areas of expertise, reflecting the interdisciplinary nature of the project. The team includes:

1. Clinicians and Healthcare Professionals: Practicing doctors and healthcare workers involved in the direct collection of clinical data and providing practical insights into the medical relevance of the dataset.
2. Biomedical Researchers: Experts in clinical medicine, neurology, and psychiatry, contributing deep knowledge of the medical conditions being studied.
3. Machine Learning and AI Specialists: Researchers and engineers with expertise in machine learning, artificial intelligence, and data science, focusing on developing models and algorithms for analyzing voice data.
4. Data Scientists and Statisticians: Professionals skilled in data curation, preprocessing, and statistical analysis, ensuring the dataset is robust and suitable for machine learning applications.
5. Social Scientists and Ethicists: Experts in ethics, sociology, and human subjects research, ensuring the dataset is ethically sourced and meets standards for privacy and consent.
6. Engineers and Technologists: Individuals with experience in software development, systems engineering, and data infrastructure, contributing to the technical aspects of data collection, storage, and dissemination.

Each instance represents a person.

There are currently around 833 instances.

833

It is a sample of a larger, ongoing collection. The data is not representative because it was collected at a limited number of geographic locations. We hope to make it more representative by shifting to remote collection and designing our recruiting approach in a way that controls for more variables.

Audio recordings, questionnaire responses.

Raw audio and questionnaire response data, as well as extracted audio features.

Yes, some questions are optional. There may be data collection irregularities that caused some information to be missing from individual instances. Each individual answered a common set of questions and then responded to additional questions relevant to their primary diagnostic category.

No, they are unrelated.

Yes, different sites have different collection configurations. The collection protocol changed over the course of the study.

a. Are there guarantees that they will exist, and remain constant, over time?
NA
b. Are there official archival versions of the complete dataset (i.e., including the external resources as they existed at the time the dataset was created)?
NA
c. Are there any restrictions (e.g., licenses, fees) associated with any of the external resources that might apply to a future user? Please provide descriptions of all external resources and any restrictions associated with them, as well as links or other access points, as appropriate.
It is self-contained.

No

This dataset includes the transcription of free speech tasks. While the inclusion of information of that type is unlikely, it cannot be completely avoided, as research participants are responsible for their choice of language.

This dataset has been de-identified through removal of all audio data and certain sensitive fields identified by a team of ethicists.

Yes:
racial or ethnic origins: The dataset includes race information.
sexual orientations: The dataset includes sexual orientation information.
financial or health data: The dataset includes socioeconomic and health information.

Data is collected on iPads (9th or 10th generation), iPad Air (5th generation) using an Avid AE-36 microphone and an Apple dongle to connect it to the iPad.

a. Number and diversity of devices included in the dataset.
Distinct iPad devices were used at each site.
b. Data recording specificities, e.g., the view for a chest x-ray image.
The data were recorded with a head-mounted headset with a microphone that could be at slightly different distances. The clinical diagnosis, depending on the disorder, was performed by one clinician or based on an EHR record or prescription.
c. Number and diversity of recording sites included in the dataset.
There are five recording sites included in the dataset.
d. Distribution shifts over time.
Changes in diagnostic criteria or practices could be a source of distribution shift.

Noise artifacts, variations in diagnostic practices, inaccurate questionnaire responses, underreporting.

Noise artifacts, variations in diagnostic practices, inaccurate questionnaire responses, underreporting.
a. Interactions between demographic or historically marginalized groups and data recordings, e.g., were women patients recorded in one site, and men in another?
Groups that have less trust in the medical system, AI, or are less proximal to the collection sites would have been less likely to be recruited.
b. Interactions between the labels and data recordings, e.g. were healthy patients recorded on one device and diseased patients on another?
Participants were screened for different disorders based on site, so they also had their data collected with different devices.

Age, Gender, Sex, Ethnicity, Socioeconomic status

Yes, the data were extracted from the raw audio to limit re-identification and only the extracted features are being released with the dataset.

No

Yes, it is saved and is not accessible publicly.

No

a. If yes:

• What are the labels provided?
• Who performed the labeling? For example, was the labeling done by a clinician, ML researcher, university or hospital?

Diagnostic labels are the result of a clinical assessment of the participant. At each site, a local clinician provided the diagnosis based on a clinical interview and appropriate work-up. For the psychiatric disorders cohort, this assessment was determined by using the participants EHR record or using an active prescription, which was done outside of data collection by an appropriately licensed clinician.
b. What labeling strategy was used?
Gold standard label available in the data (diagnosed by a clinician).
c. Human-labeled data:

• How many labelers were considered?
  Single labeler per data
• What is the demographic of the labelers? (countries of residence, of origin, number of years of experience, age, gender, race, ethnicity, etc.)
  Typically, the clinician at site of data collection, or external to (for prior diagnostic assessment)
• What guidelines did they follow?
  Per Bridge2AI Protocols and ICD-10 codes.
• How many labelers provide a label per instance?
  1

It varies widely.

Yes. https://github.com/sensein/b2aiprep, https://github.com/sensein/senselab

The process for any new labels should be described alongside any release of a model or publication. This process should include exact variables used for this determination.

Yes

The data was directly observable and reported by subjects. Clinical diagnoses were verified by clinicians reviewing audio and/or imaging data, looking at electronic health records, or medication prescriptions.

The data were collected using an iPad app.

Research teams, which may include medical, graduate, or undergraduate students, coordinated with clinicians/doctors to identify appropriate participants. These clinicians and doctors were listed under IRB as co-investigators, and were added to the consortium so that their names are included on consortium-level publications that emerge from the research. Hospital staff were not involved in scheduling but assisted in the logistics of coordinating data collection.

Participants were compensated for their time through electronic gift cards. Participants currently receive $40 for a data collection session that takes less than 90 minutes, and $80 for a session that takes over 90 minutes, for no more than a total of 3 sessions and maximum compensation of $120.

The data was collected over a period of 12 months.

Yes

Directly

Yes, participants went through an IRB-approved consent process.

Yes, all participants have been duly informed and agreed to the collection and the use of their data via prospective informed consent.

Consenting participants are informed that they may withdraw from the study at any point. If a participant chooses to withdraw during or before the voice data collection, their data will not be included in the database. Participants are informed that research data (including voice recordings) cannot be removed from the database once the voice data collection process is completed.

USA and Canada

No

English language was used for communication with study participants.
The only language option for v2.0.0 is English. Spanish versions of the protocol are under development.

The protocol asks about disabilities. Collection accessibility was facilitated through the normal means of the collection sites, including reading questions to participants when needed.

A restricted version of the dataset containing raw audio has been used in the Bridge2AI Summer School and hackathon.

Bensoussan, Yael, et al. “Developing Multi-Disorder Voice Protocols: A team science approach involving clinical expertise, bioethics, standards, and DEI.” Proc. Interspeech 2024. 2024. https://www.isca-archive.org/interspeech_2024/bensoussan24_interspeech.html

No

Yes, this dataset has skews based on disorder category, site, and other demographic factors. Users should consider the multivariate distribution when assessing utility for different questions.

Yes, there are certain applications that are discouraged from using this dataset. Specifically, the dataset should not be used for non-clinical applications such as hiring decisions, insurance premium adjustments, or any form of surveillance that could lead to discrimination or harm. These discouraged uses are intended to prevent unethical or biased outcomes that could negatively impact individuals based on their health conditions or voice characteristics. The dataset is intended strictly for research that prioritize patient safety, privacy, and ethical use.

The dataset will be distributed broadly to individuals outside of the entity who created the dataset.

The dataset will be distributed through a data publishing platform accessible at https://healthdatanexus.ai/

This platform provides publicly accessible metadata regarding the dataset with a DOI for persistent resolution. The dataset itself requires registered access.

The data was published and made available at the end of November, 2024.

Users of the dataset must agree to terms laid out in the registered access agreement. The terms relating to intellectual property are repeated here for informational purposes only:

INTELLECTUAL PROPERTY RIGHTS. You understand and acknowledge that the Data may be protected by copyright and other rights, including other intellectual property rights. Duplication, as reasonably required to carry out Your Research Project with the Data, is nonetheless permitted. Sale of all or part of the Data on any media is not permitted. You recognize that nothing in this Agreement shall operate to transfer to You any intellectual property rights in or relating to the Data. You agree not to make intellectual property claims on the Data. You agree not to use intellectual property protection in ways that would prevent or block access to, or use of, any element of these Data, or conclusions drawn directly from the Data. You can elect to perform further research that would add intellectual and resource capital to the Data and decide to obtain intellectual property rights on these downstream discoveries. You agree to implement licensing policies that will not obstruct further research. You agree to respect the Fort Lauderdale Agreement.

There are no fees associated with these restrictions.

No IP-based restrictions have been imposed by third parties.

No export controls apply to the dataset.

The dataset is supported by the NIH via the Bridge2AI project.
The dataset is hosted by the Health Data Nexus, a data publishing platform maintained by the Temerty Center for Artificial Intelligence Research and Education in Medicine (T-CAIREM) based at the University of Toronto. The Health Data Nexus maintains the technical infrastructure hosting dataset and provides continued access to interested researchers.

The platform team may be contacted through: [email protected]
The curator of the data may be contacted through: [email protected]

There is no erratum. A changelog for each dataset version is published online with the dataset metadata.

Yes, further versions of the dataset will be released on a semi-annual (twice a year) basis. These updates will be distributed as new versions of the dataset on the Health Data Nexus platform. Users will be notified through news items on the platform as well as through standard communication channels.

Once data is contributed, the data will be retained as long as it is useful for research purposes, possibly indefinitely.

By default, older versions of the dataset will continue to be supported, hosted, and made available to researchers. Each version of the dataset has a unique DOI. The dataset publishers reserve the right to remove access to older versions.

For dataset extensions and augmentations, it is possible for others to publish a derivative dataset on the Health Data Nexus which references the original source. These derivative datasets may be made available under the same conditions as the source data.

For augmentations to the code used to produce the data, the open-source repositories have discussion forums and issue pages which allow for public discussion of data preprocessing. The repository also has a mechanism (“pull requests”) for contributing improvements to the data preprocessing code.