Rubric10-Semantic Evaluation Report

40.0/50.0
80.0% Overall Score

Element Scores

Element None: None
4/5 (0%)
Persistent Identifier (DOI, RRID, or URI) 1/1
Fields:
Rationale: DOI present: https://doi.org/10.13026/37yb-1t42. ID field present with DOI format.
Dataset Title and Description Completeness 1/1
Fields:
Rationale: Title: 'Bridge2AI-Voice - An ethically-sourced, diverse voice dataset linked to health information'. Description: 313 words covering purpose, scope, conditions, methods, release details.
Keywords or Tags for Searchability 1/1
Fields:
Rationale: 47 keywords covering: voice biomarkers, disease categories (neurological, psychiatric, respiratory, pediatric), acoustic features (MFCC, spectrograms), tools (OpenSMILE, Praat), standards (FAIR, CARE), platforms (PhysioNet).
Landing Page and Resources (page, hierarchical resources) 1/1
Fields:
Rationale: Landing page: https://docs.b2ai-voice.org. Multiple external resources including PhysioNet, GitHub, NIH RePORTER, Zenodo, Health Data Nexus.
Hierarchical Structure (parent datasets, relationships) 0/1
Fields:
Rationale: No parent_datasets or related_datasets fields present. Dataset appears to be standalone without hierarchical linkages.
Element None: None
4/5 (0%)
Access Policy and IP Restrictions Defined 1/1
Fields:
Rationale: license_and_use_terms present with comprehensive Bridge2AI Voice Registered Access License including: registered access required, DUA signature mandatory, two-year term, no third-party sharing, Certificate of Confidentiality protections.
Regulatory Compliance and Confidentiality Classification 0/1
Fields:
Rationale: No regulatory_restrictions, confidentiality_level, hipaa_compliant, other_compliance, or governance_committee_contact fields present.
Download URL or Platform Link Available 1/1
Fields:
Rationale: distribution_formats contain access_urls pointing to https://physionet.org/content/b2ai-voice/ for public datasets. Controlled access via DACO@b2ai-voice.org email.
Distribution Formats and File Types Specified 1/1
Fields:
Rationale: 5 distribution formats: Parquet (spectrograms, MFCCs), TSV (phenotype, static features), Raw audio (controlled access). Detailed descriptions of format structure and contents.
Related Datasets and External Resources Linked 1/1
Fields:
Rationale: 11 external resources: PhysioNet landing page, project documentation, GitHub repository, NIH RePORTER, Health Data Nexus, Zenodo archive, publication DOI, DACO contact, Bridge2AI Program, b2aiprep library.
Element None: None
3/5 (0%)
License Terms Allow Reuse 1/1
Fields:
Rationale: License allows research use with registered access and DUA. Public disclosure of results encouraged in open-access journals. Recognition of data source required in publications.
Data Formats Are Standardized (encoding, format) 1/1
Fields:
Rationale: Standard formats: Parquet (for array data), TSV (for tabular data), JSON (for data dictionaries). Common in data science ecosystems.
Schema or Ontology Conformance Stated 0/1
Fields:
Rationale: No conforms_to or conforms_to_schema fields present. No explicit schema conformance declarations.
Variable Metadata with Identifiers Defined 0/1
Fields:
Rationale: No variables field present. Data dictionaries mentioned (phenotype.json, static_features.json) but not represented as structured variable metadata.
Use Guidance Provided (intended, prohibited uses) 1/1
Fields:
Rationale: 5 intended_uses covering AI/ML development, acoustic biomarker research, education, multimodal research, ethical AI modeling. 4 discouraged_uses covering clinical decision-making without validation, re-identification attempts, consent violations, surveillance technologies.
Element None: None
5/5 (0%)
IRB or Ethics Review and Data Protection Impact 1/1
Fields:
Rationale: human_subject_research present with University of South Florida IRB approval. irb_approval and ethics_review_board fields populated. No data_protection_impacts field.
Deidentification Method Described 1/1
Fields:
Rationale: Cleaning strategy cleaning-001 describes HIPAA Safe Harbor de-identification: 18 identifier categories removed, geographic data limited to country, date precision limited to year.
Privacy Protections and Re-identification Risk Assessment 1/1
Fields:
Rationale: 4 sensitive_elements entries describing: voice as biometric identifier (raw audio controlled access), EHR data protections, demographic de-identification, Certificate of Confidentiality protections. No explicit reidentification_risk field.
Informed Consent Obtained from Participants 1/1
Fields:
Rationale: human_subject_research describes written informed consent for data collection initiative and data sharing. Consent process includes authorization for voice data collection, EHR access, and research data sharing.
Vulnerable Populations and Compensation Documented 1/1
Fields:
Rationale: Pediatric cohort (subpop-005) identified as requiring additional ethical considerations and privacy precautions. Pediatric data not yet released (v1.1) pending additional safeguards. No participant_compensation field.
Element None: None
4/5 (0%)
Cohort or Subpopulations Characteristics Described 1/1
Fields:
Rationale: 5 subpopulations: Voice Disorders, Neurological/Neurodegenerative, Mood/Psychiatric, Respiratory, Pediatric. Each with detailed condition descriptions.
Number of Instances or Samples Reported 1/1
Fields:
Rationale: instance-001 reports 306 participants with 12,523 recordings. Multi-institutional enrollment across five sites. v1.0 release statistics provided.
Variable-Level Metadata, Tabular Flag, and Data Splits 0/1
Fields:
Rationale: No variables, is_tabular, is_data_split, or is_subpopulation fields present. Data dictionaries mentioned (phenotype.json, static_features.json) but not as structured metadata.
Data Topics or Conditions Represented 1/1
Fields:
Rationale: Comprehensive condition coverage documented in keywords and subpopulations: voice disorders (laryngeal cancer, vocal fold paralysis), neurological (Parkinson's, Alzheimer's, ALS, stroke), psychiatric (depression, schizophrenia, bipolar), respiratory (COPD, pneumonia), pediatric (autism, speech delay).
Data Quality, Anomalies, and Missing Data Documented 1/1
Fields:
Rationale: sampling_strategies describes non-random, non-representative targeted recruitment. No explicit anomalies or missing_data_documentation fields, but quality considerations implied through standardized protocols.
Element None: None
4/5 (0%)
Dataset Version Number Provided 0/1
Fields:
Rationale: No version field present at dataset level. Version information exists in updates description (v1.0, v1.1, v2.0.0, v2.0.1) but not as top-level version field.
Version Access Methods Documented 1/1
Fields:
Rationale: updates description specifies 'Latest version available at https://doi.org/10.13026/37yb-1t42. DOI for latest version vs version-specific DOIs.' PhysioNet provides version-specific access.
Change Descriptions and Errata Provided 1/1
Fields:
Rationale: updates.update_details lists version-specific changes: v1.0 (initial release, 306 participants), v1.1 (added MFCC features), v2.0.0 (expanded cohort), v2.0.1 (latest). No errata field.
Update Schedule or Frequency Indicated 1/1
Fields:
Rationale: updates.frequency: 'Periodic versioned releases during data collection period (2022-2026)'. Data collection ongoing through November 30, 2026. Future releases planned.
Provenance, Source Derivation, and Raw Data Sources 1/1
Fields:
Rationale: D4D file header includes generation metadata: 'Generation Method: Claude Code Agent Deterministic', source file (VOICE_preprocessed.txt, 89K, 9 source files), schema source. Collection mechanisms describe data flow from REDCap to b2aiprep conversion.
Element None: None
5/5 (0%)
Motivation or Purpose for Dataset Creation 1/1
Fields:
Rationale: 3 purposes: integrate voice as health biomarker in clinical care, create ethically sourced flagship dataset for AI research, establish standards/best practices for voice data collection.
Primary Research Objectives or Tasks 1/1
Fields:
Rationale: 6 tasks covering: voice disorder AI/ML models, neurological disorder detection, mood/psychiatric disorder algorithms, respiratory screening, pediatric disorder models, workforce development/education.
Funding Sources and Mechanisms Listed 1/1
Fields:
Rationale: 2 funders: NIH Office of the Director (primary grant), NIBIB (PhysioNet support). Detailed funding information provided.
Grant IDs or Award Numbers Present 1/1
Fields:
Rationale: NIH grant: 3OT2OD032720-01S3. NIBIB grant: R01EB030362. Opportunity Number: OTA-21-008. Total funding amounts and cost breakdown provided.
Creators and Acknowledgements Documented 1/1
Fields:
Rationale: 13 creators including Contact PI (Yael Bensoussan), 11 Co-PIs with roles (bioethics lead, data management lead, etc.), and Bridge2AI-Voice Consortium. Affiliations mentioned for PI.
Element None: None
5/5 (0%)
Collection Mechanisms and Settings Described 1/1
Fields:
Rationale: 3 collection_mechanisms: custom smartphone app with tablet/headset, multi-institutional 5-site enrollment (2022-2026), standardized protocol with demographics/questionnaires/voice recordings. IRB-approved protocols.
Data Acquisition Methods Listed 1/1
Fields:
Rationale: 3 acquisition_methods: voice recording tasks (sustained phonation, respiratory sounds, cough, free speech), self-reported questionnaires (demographics, disease-specific, confounders), EHR access for gold standard validation.
Preprocessing, Cleaning, Labeling, and Annotation Quality 1/1
Fields:
Rationale: 7 preprocessing_strategies: audio conversion (monaural, 16kHz, Butterworth filter), spectrogram extraction (FFT parameters), MFCC extraction (60 coefficients), OpenSMILE features, Praat/Parselmouth features, Whisper transcription, REDCap export via b2aiprep. 3 cleaning_strategies: HIPAA Safe Harbor de-identification, privacy protections (audio omission, transcript removal), multi-site standardization.
Software and Tools Documented 1/1
Fields:
Rationale: Software tools mentioned: OpenSMILE (acoustic features), Parselmouth/Praat (phonetic/prosodic features), OpenAI Whisper Large (transcription), b2aiprep (REDCap conversion), REDCap (data management). No structured software_and_tools field.
External Standards, Resources, and Imputation Protocols 1/1
Fields:
Rationale: 11 external_resources including code repositories (GitHub, b2aiprep), data platforms (PhysioNet, Health Data Nexus), documentation, publications. Standards implied (FAIR, CARE principles in keywords). No conforms_to or imputation_protocols fields.
Element None: None
2/5 (0%)
Known Limitations Documented 0/1
Fields:
Rationale: No known_limitations field present. Limitations implied in sampling description (non-random, non-representative) and subpopulation notes (pediatric data not released).
Biases Categorized Using Standard Taxonomy (RAI-aligned) 0/1
Fields:
Rationale: No known_biases field present. Sampling bias implied (non-random, non-representative, specialty clinic recruitment) but not categorized using BiasTypeEnum.
Data Anomalies and Quality Issues Noted 0/1
Fields:
Rationale: No anomalies field present. Quality considerations addressed through standardized protocols and multi-site harmonization but no explicit anomaly documentation.
Sensitive Content and Warnings Provided 1/1
Fields:
Rationale: 4 sensitive_elements: voice as biometric identifier (controlled access for raw audio), EHR data sensitivity, demographic de-identification, Certificate of Confidentiality protections. No content_warnings field.
Ethical Review and Social Impact Analysis 1/1
Fields:
Rationale: IRB approval documented. Bioethics integration mentioned in human_subject_research and use-005 (ethical AI modeling). discouraged_uses addresses surveillance and discrimination risks. No formal future_use_impacts field.
Element None: None
4/5 (0%)
Dataset Published on a Recognized Platform 1/1
Fields:
Rationale: PhysioNet distribution explicitly documented. maintainer-002 identifies PhysioNet/MIT Laboratory for Computational Physiology as distribution platform. No publisher field.
Citation and DOI for Cross-referencing 1/1
Fields:
Rationale: DOI: 10.13026/37yb-1t42. Publication DOI referenced: 10.21437/Interspeech.2024-1926. License requires 'recognition of data source required in publications.' No formatted citation field.
Community Standards or Schema Conformance 1/1
Fields:
Rationale: Standards implied through keywords: FAIR principles, CARE principles. Parquet and TSV formats are community standards. Bridge2AI program standards compliance implied. No conforms_to field.
Outreach Materials and Documentation Links 1/1
Fields:
Rationale: Comprehensive external resources: project documentation (docs.b2ai-voice.org), GitHub repository, Zenodo archive, NIH RePORTER, publication, b2aiprep library, Bridge2AI program site.
Related Datasets with Typed Relationships 0/1
Fields:
Rationale: No related_datasets field present. Dataset could link to other Bridge2AI datasets (AI-READI, CM4AI, CHORUS) or voice datasets.

Semantic Analysis

Recommendations

Recommendations

HIGH PRIORITY: Add known_limitations documenting: non-representative sampling (specialty clinic recruitment), adult-only data release (pediatric pending), institutional population constraints, multi-session variability, condition coverage gaps
HIGH PRIORITY: Add known_biases with BiasTypeEnum categorization: selection_bias (specialty clinic recruitment, targeted disease cohorts), demographic_bias (institutional populations, geographic concentration), measurement_bias (device variability across sites)
HIGH PRIORITY: Add regulatory compliance fields: hipaa_compliant (compliant), confidentiality_level (controlled_access_required), governance_committee_contact (DACO), other_compliance (Certificate of Confidentiality)
MEDIUM PRIORITY: Add hierarchical dataset relationships: related_datasets linking to AI-READI, CM4AI, CHORUS (Bridge2AI consortium datasets); parent_datasets linking to Bridge2AI program collection
MEDIUM PRIORITY: Add schema conformance: conforms_to (D4D schema, FAIR principles, CARE principles, Bridge2AI standards), conforms_to_schema (phenotype.json, static_features.json data dictionaries)
MEDIUM PRIORITY: Add structured variable metadata: variables array with key phenotype and feature variables, is_tabular flag, reference to external data dictionaries
MEDIUM PRIORITY: Add top-level version field with current version (v2.0.1)
MEDIUM PRIORITY: Add publisher field identifying PhysioNet as publishing platform for machine-readable attribution
MEDIUM PRIORITY: Add citation field with formatted citation string (DataCite/BibTeX) and citation instructions
LOW PRIORITY: Migrate ethics information to structured fields: informed_consent, participant_privacy, vulnerable_populations (pediatric cohort), participant_compensation, data_protection_impacts, reidentification_risk
LOW PRIORITY: Add software_and_tools array with versions and citations: OpenSMILE, Praat/Parselmouth, OpenAI Whisper Large, b2aiprep, REDCap
LOW PRIORITY: Add provenance metadata: was_derived_from (source documentation), raw_data_sources (original data collection), future_use_impacts (social impact analysis per CROISSANT RAI)
Generated on 2026-01-13 16:57:40 using Bridge2AI Data Sheets Schema